Skip to content

AI analytics#1339

Open
aadesh18 wants to merge 98 commits into
devfrom
ai-analytics
Open

AI analytics#1339
aadesh18 wants to merge 98 commits into
devfrom
ai-analytics

Conversation

@aadesh18
Copy link
Copy Markdown
Collaborator

@aadesh18 aadesh18 commented Apr 15, 2026

This PR adds an AI analytics dashboard to the internal tool and centralises AI query observability. Every call through the /ai/query and /integrations/ai-proxy endpoints is now logged to SpacetimeDB with token counts, cost, latency, and full conversation replay. The internal tool gains a new "Unified AI Endpoint Analytics" tab backed by a new useAiQueryLogs SpacetimeDB subscription, plus reviewer enrollment and unmark-reviewed flows.

How to review this PR:

  1. Go through the changes in apps/backend/src/app/api/latest/ai/query/[mode]/route.ts and apps/backend/src/app/api/latest/integrations/ai-proxy/[[...path]]/route.ts. These are the entry points to the logging
  2. Go through the apps/backend/src/app/api/latest/internal folder. It contains the routes that interact spacetime db
  3. Other changes in backend:
    1. Under lib/ai - most of these files are helper files that abstract a lot of logic from the ai-proxy and ai-query endpoints, especially the logging logic
    2. .env - added STACK_SPACETIMEDB_SERVICE_TOKEN
  4. Look through the internal tool

Summary by CodeRabbit

  • New Features

    • AI Query Usage dashboard: filtering, sorting, charts, metrics, details view
    • Reviewer enrollment for SpacetimeDB and “Unmark reviewed” action
  • Improvements

    • Separate Usage and Calls tabs with session-persistent selection
    • Call log: pagination, sortable “Human Reviewed” column, optimistic mark/unmark UX
    • Public published Q&A projection for safe read-only consumption
  • Infrastructure

    • Centralized AI proxy, logging, and query telemetry; HTTP-based reducer calls and safer DB access controls

mantrakp04 and others added 30 commits March 23, 2026 10:48
- Added new internal API endpoint for documentation tools, allowing actions such as listing available docs, searching, and fetching specific documentation by ID.
- Updated environment configuration to support optional internal secret for enhanced security.
- Refactored existing search functionality to utilize the new docs tools API instead of the previous MCP server.
- Improved error handling and response parsing for documentation-related requests.
- Expanded documentation to clarify the relationship between the new tools and existing API functionalities.

This update streamlines the documentation access process and enhances the overall developer experience.
- Introduced error capturing for failed HTTP requests in the docs tools API, improving debugging capabilities.
- Updated the API response for unsupported methods to include an 'Allow' header, clarifying the expected request type.

These changes enhance the robustness of the documentation tools integration and improve developer experience.
- Updated the key name in the capabilities section of the API documentation to follow a consistent naming convention, improving clarity and maintainability.
The .gitmodules was updated in d22593d to point at
apps/backend/src/private/implementation, but the gitlink entry (mode
160000) was never added to the tree. This caused
`git clone --recurse-submodules` to silently skip the private submodule.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on for docs tools

- Added `STACK_DOCS_INTERNAL_BASE_URL` to backend `.env` and `.env.development` files for AI tool bundle configuration.
- Removed references to `STACK_INTERNAL_DOCS_TOOLS_SECRET` from backend and docs environment files and validation logic from the docs tools API route.
- Introduced a new `.env` file for the docs app with essential configuration variables.
Copy link
Copy Markdown
Collaborator Author

@aadesh18 aadesh18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made changes based on comments

Comment thread apps/backend/src/app/api/latest/ai/query/[mode]/route.ts
Comment thread apps/backend/src/app/api/latest/internal/mcp-review/delete/route.ts
Comment thread apps/backend/src/app/api/latest/internal/mcp-review/update-qa-entry/route.ts Outdated
Comment thread apps/backend/src/lib/ai/loggers/ai-query-logger.ts Outdated
Comment thread apps/backend/src/lib/ai/prompts.ts
Comment thread apps/dashboard/src/components/vibe-coding/chat-adapters.ts Outdated
Comment thread apps/dashboard/src/components/vibe-coding/chat-adapters.ts
Comment thread apps/e2e/tests/spacetimedb/private-tables.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants