Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ mcp-data-platform/
│ ├── pkcestore/ # In-flight PKCE state for outbound OAuth (oauth-start → callback)
│ ├── platform/ # Core orchestration: facade, config, options, lifecycle (fieldcrypt/, instructions/, personastore/ = seams shared with pkg/admin; other facade-internal seams live under internal/platform/)
│ ├── portal/ # Asset portal data layer (assetindex/, collectionindex/, datahubapi/, knowledgepage/, threads/, ...)
│ ├── prompt/ # Prompt management for the MCP data platform
│ ├── prompt/ # Prompt management: versioned store contract, review gate (versionhttp/ = version-history/approval/usage REST for the admin + portal surfaces)
│ ├── query/ # Query execution provider abstraction (trino/ = Trino adapter)
│ ├── ratelimit/ # Shared per-IP token-bucket limiter + trusted-proxy client-IP resolver (portal viewer, OAuth endpoints)
│ ├── registry/ # Toolkit registration and management
Expand Down
2 changes: 1 addition & 1 deletion docs/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2173,7 +2173,7 @@ Structured feedback from reviewers (including non-agent subject-matter experts a
The single home for the Memory to Insight to Knowledge lifecycle (formerly the separate Knowledge Pages, Knowledge & Memory, and admin Knowledge & Memory routes, which now redirect here). A header teaches the model: everything learned is a Memory; a memory others would benefit from becomes an Insight (a proposal awaiting review); whoever holds the `apply_knowledge` capability promotes good insights into Knowledge (business/domain facts become knowledge pages, technical/entity facts go to the DataHub catalog). Three tabs, with review/promote affordances gated on the `apply_knowledge` tool (a capability, not an admin role). Knowledge (default): unified search across every accessible source grouped by source with a coverage summary (the same federation as the `search` tool, over `GET /api/v1/portal/search`); with an empty query, browse of canonical knowledge pages (create/edit/remove for `apply_knowledge` holders); and for `apply_knowledge` holders the changesets (the record of insights promoted into knowledge, with rollback) since a changeset is created only at apply time and belongs with the promoted knowledge, not the unpromoted insights. Insights: the review pipeline only (insights are the one memory type that crosses between users) - your captured insights with status and relevance search, plus for `apply_knowledge` holders the full review queue (approve/reject); a pending-review count is badged on the sidebar Knowledge item and the Insights tab. Memory: personal, scoped to your own records (the cross-user unit is the insight), classified by lifecycle class (`sink_class`: Preference, Event, Business knowledge, Operational rule, Schema/entity). The Knowledge tab also has Catalog and Context Docs sub-tabs (#719/#720), first-class routes at `/knowledge/catalog` and `/knowledge/context-docs`, that surface the DataHub catalog and context documents in the portal. Catalog: pick a connection, browse/search datasets, open one to see description, tags, owners, glossary terms, domain, and columns, and edit each facet inline when the persona grants `datahub_update` and the connection is writable (no dataset create/delete since datasets originate in source systems). Context Docs: browse/search and full create/edit/delete of markdown context documents through a markdown editor, gated on `datahub_create`/`datahub_update`/`datahub_delete`; a document attaches only to Dataset/GlossaryTerm/GlossaryNode/Container. Both are backed by the portal DataHub REST API at `/api/v1/portal/datahub/{connection}/...` (`GET .../connections` lists connections with a writable flag): reads require DataHub access on the persona; a write requires the matching MCP tool grant AND a write-enabled connection (`read_only: false`), both enforced server-side and recorded in the audit log. Tag and glossary-term edits use batched add/remove sets (the clobber-safe write path, #721/#729).

## Prompts
Personal, available, and shared prompt templates. Three tabs: Personal (create/edit/delete own prompts), Available (global/persona prompts), and Shared (prompts other users shared with you). Search ranks approved prompts by relevance to a phrase (semantic vector similarity when an embedding provider is configured, keyword fallback otherwise) across the caller's full visibility, best-first; browse mode keeps sortable columns. The same ranking backs the MCP tool `manage_prompt list query=...`. Approved prompts are embedded off the request path by the shared index-jobs framework (source_kind `prompts`); editing a prompt's title, description, body, or tags clears its vector so it re-embeds against the new text. Visibility (own personal, global, matching-persona; all approved for admins) is applied before ranking, so a prompt you cannot read is never returned. Sortable columns, expandable rows with full content and copy button. Scope badges: Personal, Global, Persona, System. Lifecycle status badge (draft, approved, deprecated, superseded) and comma-separated tags on create/edit. Request Promotion on a personal prompt asks an admin to promote it to a chosen persona or to global; the prompt stays personal with a "Promotion requested" badge until an admin approves or rejects it. Share sends a personal prompt directly to another user by email (owner-initiated, no approval): the recipient gets a real runnable prompt, not a markdown snapshot, and it appears on the Prompts page's Shared tab. Personal prompt names are unique per owner; when served over MCP, names carry a scope prefix computed at serve time: `personal-<name>`, `<persona>-<name>` (one per persona), `global-<name>`, or `shared-<name>` for prompts shared with the caller, keeping the surface collision-free by construction; every descriptor carries a `title` from display_name. Users never need machine names: agents resolve any handle (stored name, display name, `mcp:prompt:<id>`, or free text) to a ready-to-run prompt with the `manage_prompt` `use` command, which returns rendered content, argument specs, and provenance, or ranked candidates when ambiguous.
Personal, available, and shared prompt templates. Three tabs: Personal (create/edit/delete own prompts), Available (global/persona prompts), and Shared (prompts other users shared with you). Search ranks approved prompts by relevance to a phrase (semantic vector similarity when an embedding provider is configured, keyword fallback otherwise) across the caller's full visibility, best-first; browse mode keeps sortable columns. The same ranking backs the MCP tool `manage_prompt list query=...`. Approved prompts are embedded off the request path by the shared index-jobs framework (source_kind `prompts`); editing a prompt's title, description, body, or tags clears its vector so it re-embeds against the new text. Visibility (own personal, global, matching-persona; all approved for admins) is applied before ranking, so a prompt you cannot read is never returned. Sortable columns, expandable rows with full content and copy button. Scope badges: Personal, Global, Persona, System. Lifecycle status badge (draft, approved, deprecated, superseded) and comma-separated tags on create/edit. Request Promotion on a personal prompt asks an admin to promote it to a chosen persona or to global; the prompt stays personal with a "Promotion requested" badge until an admin approves or rejects it. Share sends a personal prompt directly to another user by email (owner-initiated, no approval): the recipient gets a real runnable prompt, not a markdown snapshot, and it appears on the Prompts page's Shared tab. Personal prompt names are unique per owner; when served over MCP, names carry a scope prefix computed at serve time: `personal-<name>`, `<persona>-<name>` (one per persona), `global-<name>`, or `shared-<name>` for prompts shared with the caller, keeping the surface collision-free by construction; every descriptor carries a `title` from display_name. Users never need machine names: agents resolve any handle (stored name, display name, `mcp:prompt:<id>`, or free text) to a ready-to-run prompt with the `manage_prompt` `use` command, which returns rendered content, argument specs, and provenance (including version, approver, and approval time), or ranked candidates when ambiguous. Every database prompt is versioned (#1009): each mutation of content, display name, description, arguments, or tags snapshots an immutable `prompt_versions` row with its author, and approval stamps bind to the specific version approved (approving v5 never alters v4's recorded approval). Editing the content or arguments of an approved global or persona prompt does not change what is served: the edit lands as a pending draft version (manage_prompt update returns status pending_approval; a gated content edit cannot be combined with scope/status changes in one call) and the approved snapshot keeps serving until an admin approves the draft via `POST /api/v1/admin/prompts/{id}/versions/{version}/approve` (reject with `.../reject`; full history with author and content via `GET /api/v1/admin/prompts/{id}/versions`, owner-readable per prompt via `GET /api/v1/portal/prompts/{id}/versions`). Metadata-only edits (tags, category, description, display name) apply directly; personal prompts version silently. Served prompts carry provenance: `prompts/get` stamps `prompt_version` / `prompt_approved_by` / `prompt_approved_at` / `prompt_reference` into `_meta`. Usage stats are aggregated from `prompt_serve` audit events (emitted on every database-prompt `prompts/get` and resolved `use`, within the audit retention window): `manage_prompt get` reports `run_count` and `last_run_at`, and `GET /api/v1/admin/prompts/usage` / `GET /api/v1/portal/prompts/usage` return the per-prompt rollup (portal scoped to the caller's visible prompts) for library curation and dead-prompt detection.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ mcp-data-platform is the orchestration layer for the txn2 MCP ecosystem. DataHub
- [The Data Stack](https://mcp-data-platform.txn2.com/concepts/components/): Why DataHub, Trino, and S3, what each component brings, and how cross-enrichment makes them greater than the sum of their parts
- [Operating Modes](https://mcp-data-platform.txn2.com/server/operating-modes/): Two deployment modes, standalone (no database) and database-backed. Feature availability by mode, example configurations, decision guide
- [Deployment](https://mcp-data-platform.txn2.com/server/deployment/): Docker Compose and Kubernetes/Helm deployment guides, including how connected agents pick up tool-contract changes across an upgrade
- [Tools](https://mcp-data-platform.txn2.com/server/tools/): Complete tool list for every toolkit (DataHub, Trino, S3, knowledge, memory, portal, gateways), plus platform_find_tools semantic tool discovery, manage_prompt with the use resolve-and-run verb (any handle: name, display name, mcp:prompt:<id>, free text), and the uniform structured error envelope
- [Tools](https://mcp-data-platform.txn2.com/server/tools/): Complete tool list for every toolkit (DataHub, Trino, S3, knowledge, memory, portal, gateways), plus platform_find_tools semantic tool discovery, manage_prompt with the use resolve-and-run verb (any handle: name, display name, mcp:prompt:<id>, free text), prompt versioning with approval provenance (content edits to approved shared prompts pend as draft versions until admin approval; served prompts carry version and approver; run counts from prompt-serve audit events), and the uniform structured error envelope
- [Multi-Provider](https://mcp-data-platform.txn2.com/server/multi-provider/): Connect multiple instances of each service
- [Audit Logging](https://mcp-data-platform.txn2.com/server/audit/): PostgreSQL-backed audit logging for tool calls: schema and field reference, parameter sanitization with configurable redact_keys and log_parameters opt-out, async vs sync delivery semantics and the audit_events_dropped_total metric, caller-class separation, monthly partition rotation, and retention
- [Observability (Metrics)](https://mcp-data-platform.txn2.com/server/observability/): OpenTelemetry Prometheus metrics covering tool calls, gateway HTTP calls, and toolkit/provider internals, plus optional OTLP distributed tracing and an authenticated PromQL proxy for the portal
Expand Down
2 changes: 2 additions & 0 deletions docs/server/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,8 @@ All registered prompts (platform + toolkit) are included in the `platform_info`

**Prompt titles and resolution.** Database prompts are served on the native MCP prompts surface under per-viewer scope-prefixed names (`global-<name>`, `<persona>-<name>`, `personal-<name>`, `shared-<name>`), which keeps the surface collision-free by construction; every descriptor carries a `title` from `display_name` so clients show the human name regardless. Users never need to know any machine name: agents resolve a prompt from whatever handle the user says (stored name, display name, `mcp:prompt:<id>`, or free text) with the `manage_prompt` `use` command.

**Versioning and approval provenance.** Every database prompt is versioned: each mutation of its content, display name, description, arguments, or tags snapshots an immutable version row with the author, and approval stamps bind to the specific version that was approved. Editing the content or arguments of an approved global or persona prompt does not change what is served: the edit is saved as a pending draft version, and the approved snapshot keeps serving until an admin approves the draft (admin API `POST /api/v1/admin/prompts/{id}/versions/{version}/approve`, or reject with `.../reject`; history via `GET .../versions`). Metadata-only edits (tags, category, description, display name) apply directly. Personal prompts version silently without review. Served prompts carry their provenance: `prompts/get` responses stamp `prompt_version`, `prompt_approved_by`, `prompt_approved_at`, and `prompt_reference` into `_meta`, and `manage_prompt use` reports the same in its provenance block, so an agent can state "running Daily Sales Report v4, approved by jane@example.com" before executing. Run counts and last-run timestamps are aggregated from prompt-serve audit events (within the audit retention window) and exposed on `manage_prompt get` and the `GET /api/v1/admin/prompts/usage` and `GET /api/v1/portal/prompts/usage` endpoints.

### Streamable HTTP Configuration

The HTTP transport serves both legacy SSE (`/sse`, `/message`) and Streamable HTTP (`/`) endpoints. Streamable HTTP session behavior is configured under `server.streamable`:
Expand Down
4 changes: 3 additions & 1 deletion docs/server/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -1015,10 +1015,12 @@ This is discovery, not routing: the agent still chooses which returned tool to c
- an `mcp:prompt:<id>` reference (the same reference `search` results carry),
- or free text, ranked against the library.

A single confident match returns `status: "resolved"` with the prompt content (argument values passed in `args` are substituted), its argument specs, any required arguments still missing, and provenance (scope, status, approver, owner, reference) so the agent can confirm what it is about to run. An ambiguous handle returns `status: "ambiguous"` with a short ranked candidate list to disambiguate, never an error and never a silent first-match. Operator, workflow, and toolkit prompts resolve through `use` as well (they remain read-only to the management commands); the auto-generated `platform-overview` prompt is served only on the native prompts surface.
A single confident match returns `status: "resolved"` with the prompt content (argument values passed in `args` are substituted), its argument specs, any required arguments still missing, and provenance (scope, status, version, approver, owner, reference) so the agent can confirm what it is about to run: "running Daily Sales Report v4, approved by jane@example.com". An ambiguous handle returns `status: "ambiguous"` with a short ranked candidate list to disambiguate, never an error and never a silent first-match. Operator, workflow, and toolkit prompts resolve through `use` as well (they remain read-only to the management commands); the auto-generated `platform-overview` prompt is served only on the native prompts surface.

Non-admins manage only their own personal prompts; admins manage every scope. On the native MCP prompts surface, database prompts are presented under per-viewer scope-prefixed names (see [Configuration: Prompts](configuration.md#prompts)); `use` insulates users from those presented names entirely.

**Versioning and review.** Every mutation of a prompt's content, display name, description, arguments, or tags snapshots an immutable version with its author; approval stamps bind to the specific version approved. Editing the content or arguments of an **approved global or persona** prompt does not apply immediately: `update` returns `status: "pending_approval"` with the draft's `pending_version`, and every caller keeps being served the approved snapshot until an admin approves the draft (in the admin portal, or `POST /api/v1/admin/prompts/{id}/versions/{version}/approve`). A gated content edit cannot be combined with scope/status/other non-versioned changes in one call; submit them separately. Personal prompts and never-approved drafts version silently. `get` also reports `run_count` and `last_run_at`, aggregated from prompt-serve audit events (each `prompts/get` and resolved `use` counts as a serve).

---

## Next Steps
Expand Down
Loading
Loading