diff --git a/AGENTS.md b/AGENTS.md index d293a539a..d09ee4378 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -130,6 +130,10 @@ Co-Authored-By: (agent model name) - `specs/scheduler.md` (scheduled Junior task contract) - `specs/plugin-heartbeat.md` (plugin heartbeat and tool hook contract) - `specs/plugin-dispatch.md` (durable plugin agent dispatch contract) +- `specs/plugin-prompt-hooks.md` (plugin prompt contribution, turn observation, and session append state contract) +- `specs/plugin-database.md` (plugin packaged SQL migrations and ctx.db contract) +- `specs/plugin-cli.md` (future plugin-contributed host CLI command contract) +- `specs/memory-plugin/index.md` (long-term memory plugin storage, recall, passive learning, tools, visibility, and lifecycle contract) - `specs/harness-agent.md` (agent loop and output contract) - `specs/agent-session-resumability.md` (multi-slice agent-run resumability and timeout recovery contract) - `specs/agent-execution.md` (agent execution rubric and completion gates) diff --git a/packages/docs/src/content/docs/reference/api/README.md b/packages/docs/src/content/docs/reference/api/README.md index ac4ff4e3d..24498a4b4 100644 --- a/packages/docs/src/content/docs/reference/api/README.md +++ b/packages/docs/src/content/docs/reference/api/README.md @@ -7,8 +7,6 @@ title: "@sentry/junior" ## Interfaces -- [AgentPluginConversations](/reference/api/interfaces/agentpluginconversations/) -- [AgentPluginConversationSummary](/reference/api/interfaces/agentpluginconversationsummary/) - [ConversationFeed](/reference/api/interfaces/conversationfeed/) - [ConversationReport](/reference/api/interfaces/conversationreport/) - [ConversationRunReport](/reference/api/interfaces/conversationrunreport/) @@ -23,6 +21,8 @@ title: "@sentry/junior" - [JuniorPluginSetOptions](/reference/api/interfaces/juniorpluginsetoptions/) - [JuniorReporting](/reference/api/interfaces/juniorreporting/) - [JuniorVercelConfigOptions](/reference/api/interfaces/juniorvercelconfigoptions/) +- [PluginConversations](/reference/api/interfaces/pluginconversations/) +- [PluginConversationSummary](/reference/api/interfaces/pluginconversationsummary/) - [PluginOperationalReport](/reference/api/interfaces/pluginoperationalreport/) - [PluginOperationalReportFeed](/reference/api/interfaces/pluginoperationalreportfeed/) - [PluginPackageContentItemReport](/reference/api/interfaces/pluginpackagecontentitemreport/) @@ -36,10 +36,10 @@ title: "@sentry/junior" ## Type Aliases -- [AgentPluginConversationStatus](/reference/api/type-aliases/agentpluginconversationstatus/) - [ConversationReportStatus](/reference/api/type-aliases/conversationreportstatus/) - [ConversationSurface](/reference/api/type-aliases/conversationsurface/) - [JuniorPluginInput](/reference/api/type-aliases/juniorplugininput/) +- [PluginConversationStatus](/reference/api/type-aliases/pluginconversationstatus/) - [TranscriptPartType](/reference/api/type-aliases/transcriptparttype/) - [TranscriptRole](/reference/api/type-aliases/transcriptrole/) diff --git a/packages/docs/src/content/docs/reference/api/functions/createApp.md b/packages/docs/src/content/docs/reference/api/functions/createApp.md index e48785c61..8d9b6eff0 100644 --- a/packages/docs/src/content/docs/reference/api/functions/createApp.md +++ b/packages/docs/src/content/docs/reference/api/functions/createApp.md @@ -7,7 +7,7 @@ title: "createApp" > **createApp**(`options?`): `Promise`\<`Hono`\<`BlankEnv`, `BlankSchema`, `"/"`\>\> -Defined in: [junior/src/app.ts:332](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L332) +Defined in: [junior/src/app.ts:327](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L327) Create a Hono app with all Junior routes. diff --git a/packages/docs/src/content/docs/reference/api/functions/defineJuniorPlugins.md b/packages/docs/src/content/docs/reference/api/functions/defineJuniorPlugins.md index 97ab51efa..f1dcbf10d 100644 --- a/packages/docs/src/content/docs/reference/api/functions/defineJuniorPlugins.md +++ b/packages/docs/src/content/docs/reference/api/functions/defineJuniorPlugins.md @@ -7,7 +7,7 @@ title: "defineJuniorPlugins" > **defineJuniorPlugins**(`inputs`, `options?`): [`JuniorPluginSet`](/reference/api/interfaces/juniorpluginset/) -Defined in: [junior/src/plugins.ts:102](https://github.com/getsentry/junior/blob/main/packages/junior/src/plugins.ts#L102) +Defined in: [junior/src/plugins.ts:100](https://github.com/getsentry/junior/blob/main/packages/junior/src/plugins.ts#L100) Define package-name plugins and JS plugin definitions for one app. diff --git a/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversationSummary.md b/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversationSummary.md deleted file mode 100644 index 5a156a23a..000000000 --- a/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversationSummary.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -editUrl: false -next: false -prev: false -title: "AgentPluginConversationSummary" ---- - -Defined in: [junior-plugin-api/src/index.ts:377](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L377) - -## Properties - -### channelName? - -> `optional` **channelName?**: `string` - -Defined in: [junior-plugin-api/src/index.ts:378](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L378) - ---- - -### conversationId - -> **conversationId**: `string` - -Defined in: [junior-plugin-api/src/index.ts:379](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L379) - ---- - -### displayTitle - -> **displayTitle**: `string` - -Defined in: [junior-plugin-api/src/index.ts:380](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L380) - ---- - -### lastActivityAt - -> **lastActivityAt**: `string` - -Defined in: [junior-plugin-api/src/index.ts:381](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L381) - ---- - -### lastUpdatedAt - -> **lastUpdatedAt**: `string` - -Defined in: [junior-plugin-api/src/index.ts:382](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L382) - ---- - -### source? - -> `optional` **source?**: `"slack"` \| `"plugin"` \| `"local"` \| `"api"` \| `"internal"` \| `"scheduler"` - -Defined in: [junior-plugin-api/src/index.ts:383](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L383) - ---- - -### status - -> **status**: [`AgentPluginConversationStatus`](/reference/api/type-aliases/agentpluginconversationstatus/) - -Defined in: [junior-plugin-api/src/index.ts:384](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L384) diff --git a/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversations.md b/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversations.md deleted file mode 100644 index f4c563cb3..000000000 --- a/packages/docs/src/content/docs/reference/api/interfaces/AgentPluginConversations.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -editUrl: false -next: false -prev: false -title: "AgentPluginConversations" ---- - -Defined in: [junior-plugin-api/src/index.ts:387](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L387) - -## Methods - -### listRecent() - -> **listRecent**(`options?`): `Promise`\<[`AgentPluginConversationSummary`](/reference/api/interfaces/agentpluginconversationsummary/)[]\> - -Defined in: [junior-plugin-api/src/index.ts:388](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L388) - -#### Parameters - -##### options? - -###### limit? - -`number` - -#### Returns - -`Promise`\<[`AgentPluginConversationSummary`](/reference/api/interfaces/agentpluginconversationsummary/)[]\> diff --git a/packages/docs/src/content/docs/reference/api/interfaces/JuniorAppOptions.md b/packages/docs/src/content/docs/reference/api/interfaces/JuniorAppOptions.md index 9f3845f25..2ee362b3a 100644 --- a/packages/docs/src/content/docs/reference/api/interfaces/JuniorAppOptions.md +++ b/packages/docs/src/content/docs/reference/api/interfaces/JuniorAppOptions.md @@ -5,7 +5,7 @@ prev: false title: "JuniorAppOptions" --- -Defined in: [junior/src/app.ts:61](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L61) +Defined in: [junior/src/app.ts:60](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L60) ## Properties @@ -13,7 +13,7 @@ Defined in: [junior/src/app.ts:61](https://github.com/getsentry/junior/blob/main > `optional` **configDefaults?**: `Record`\<`string`, `unknown`\> -Defined in: [junior/src/app.ts:70](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L70) +Defined in: [junior/src/app.ts:69](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L69) Install-wide provider defaults (`provider.key` format). Channel overrides take precedence. @@ -23,7 +23,7 @@ Install-wide provider defaults (`provider.key` format). Channel overrides take p > `optional` **conversationWork?**: `VercelConversationWorkCallbackOptions` -Defined in: [junior/src/app.ts:72](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L72) +Defined in: [junior/src/app.ts:71](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L71) Queue consumer wiring for the durable conversation worker. @@ -33,7 +33,7 @@ Queue consumer wiring for the durable conversation worker. > `optional` **plugins?**: [`JuniorPluginSet`](/reference/api/interfaces/juniorpluginset/) -Defined in: [junior/src/app.ts:74](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L74) +Defined in: [junior/src/app.ts:73](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L73) Direct plugin set override. Usually omitted when `juniorNitro()` uses a plugin module. @@ -43,7 +43,7 @@ Direct plugin set override. Usually omitted when `juniorNitro()` uses a plugin m > `optional` **sandbox?**: `object` -Defined in: [junior/src/app.ts:76](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L76) +Defined in: [junior/src/app.ts:75](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L75) Sandbox execution options. @@ -61,7 +61,7 @@ Entries may be exact domains or leading wildcard domains such as > `optional` **slack?**: `object` -Defined in: [junior/src/app.ts:63](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L63) +Defined in: [junior/src/app.ts:62](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L62) Slack-specific overrides applied after env parsing. @@ -83,4 +83,4 @@ Slack emoji shown while Junior is processing. Defaults to `eyes`. > `optional` **waitUntil?**: `WaitUntilFn` -Defined in: [junior/src/app.ts:84](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L84) +Defined in: [junior/src/app.ts:83](https://github.com/getsentry/junior/blob/main/packages/junior/src/app.ts#L83) diff --git a/packages/docs/src/content/docs/reference/api/interfaces/JuniorPluginSet.md b/packages/docs/src/content/docs/reference/api/interfaces/JuniorPluginSet.md index 340a3cf74..3c7afb74e 100644 --- a/packages/docs/src/content/docs/reference/api/interfaces/JuniorPluginSet.md +++ b/packages/docs/src/content/docs/reference/api/interfaces/JuniorPluginSet.md @@ -33,7 +33,7 @@ Manifest-only plugin packages included by package name. ### registrations -> **registrations**: `JuniorPluginRegistration`[] +> **registrations**: `PluginRegistration`[] Defined in: [junior/src/plugins.ts:22](https://github.com/getsentry/junior/blob/main/packages/junior/src/plugins.ts#L22) diff --git a/packages/docs/src/content/docs/reference/api/interfaces/JuniorReporting.md b/packages/docs/src/content/docs/reference/api/interfaces/JuniorReporting.md index f4aabda9d..0999c99c0 100644 --- a/packages/docs/src/content/docs/reference/api/interfaces/JuniorReporting.md +++ b/packages/docs/src/content/docs/reference/api/interfaces/JuniorReporting.md @@ -133,7 +133,7 @@ Read discovered skill names for reporting consumers. ### listRecentConversations()? -> `optional` **listRecentConversations**(`options?`): `Promise`\<[`AgentPluginConversationSummary`](/reference/api/interfaces/agentpluginconversationsummary/)[]\> +> `optional` **listRecentConversations**(`options?`): `Promise`\<[`PluginConversationSummary`](/reference/api/interfaces/pluginconversationsummary/)[]\> Defined in: [junior/src/reporting.ts:104](https://github.com/getsentry/junior/blob/main/packages/junior/src/reporting.ts#L104) @@ -149,4 +149,4 @@ Read recent conversation summaries without transcript payloads. #### Returns -`Promise`\<[`AgentPluginConversationSummary`](/reference/api/interfaces/agentpluginconversationsummary/)[]\> +`Promise`\<[`PluginConversationSummary`](/reference/api/interfaces/pluginconversationsummary/)[]\> diff --git a/packages/docs/src/content/docs/reference/api/interfaces/PluginConversationSummary.md b/packages/docs/src/content/docs/reference/api/interfaces/PluginConversationSummary.md new file mode 100644 index 000000000..423c8d9cf --- /dev/null +++ b/packages/docs/src/content/docs/reference/api/interfaces/PluginConversationSummary.md @@ -0,0 +1,64 @@ +--- +editUrl: false +next: false +prev: false +title: "PluginConversationSummary" +--- + +Defined in: junior-plugin-api/src/operations.ts:12 + +## Properties + +### channelName? + +> `optional` **channelName?**: `string` + +Defined in: junior-plugin-api/src/operations.ts:13 + +--- + +### conversationId + +> **conversationId**: `string` + +Defined in: junior-plugin-api/src/operations.ts:14 + +--- + +### displayTitle + +> **displayTitle**: `string` + +Defined in: junior-plugin-api/src/operations.ts:15 + +--- + +### lastActivityAt + +> **lastActivityAt**: `string` + +Defined in: junior-plugin-api/src/operations.ts:16 + +--- + +### lastUpdatedAt + +> **lastUpdatedAt**: `string` + +Defined in: junior-plugin-api/src/operations.ts:17 + +--- + +### source? + +> `optional` **source?**: `"slack"` \| `"plugin"` \| `"local"` \| `"api"` \| `"internal"` \| `"scheduler"` + +Defined in: junior-plugin-api/src/operations.ts:18 + +--- + +### status + +> **status**: [`PluginConversationStatus`](/reference/api/type-aliases/pluginconversationstatus/) + +Defined in: junior-plugin-api/src/operations.ts:19 diff --git a/packages/docs/src/content/docs/reference/api/interfaces/PluginConversations.md b/packages/docs/src/content/docs/reference/api/interfaces/PluginConversations.md new file mode 100644 index 000000000..bd839c647 --- /dev/null +++ b/packages/docs/src/content/docs/reference/api/interfaces/PluginConversations.md @@ -0,0 +1,28 @@ +--- +editUrl: false +next: false +prev: false +title: "PluginConversations" +--- + +Defined in: junior-plugin-api/src/operations.ts:22 + +## Methods + +### listRecent() + +> **listRecent**(`options?`): `Promise`\<[`PluginConversationSummary`](/reference/api/interfaces/pluginconversationsummary/)[]\> + +Defined in: junior-plugin-api/src/operations.ts:23 + +#### Parameters + +##### options? + +###### limit? + +`number` + +#### Returns + +`Promise`\<[`PluginConversationSummary`](/reference/api/interfaces/pluginconversationsummary/)[]\> diff --git a/packages/docs/src/content/docs/reference/api/interfaces/PluginOperationalReport.md b/packages/docs/src/content/docs/reference/api/interfaces/PluginOperationalReport.md index 1862dadb8..56228838b 100644 --- a/packages/docs/src/content/docs/reference/api/interfaces/PluginOperationalReport.md +++ b/packages/docs/src/content/docs/reference/api/interfaces/PluginOperationalReport.md @@ -5,7 +5,7 @@ prev: false title: "PluginOperationalReport" --- -Defined in: [junior-plugin-api/src/index.ts:439](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L439) +Defined in: junior-plugin-api/src/operations.ts:74 ## Extends @@ -17,7 +17,7 @@ Defined in: [junior-plugin-api/src/index.ts:439](https://github.com/getsentry/ju > `optional` **generatedAt?**: `string` -Defined in: [junior-plugin-api/src/index.ts:433](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L433) +Defined in: junior-plugin-api/src/operations.ts:68 #### Inherited from @@ -29,7 +29,7 @@ Defined in: [junior-plugin-api/src/index.ts:433](https://github.com/getsentry/ju > `optional` **metrics?**: `PluginOperationalMetric`[] -Defined in: [junior-plugin-api/src/index.ts:434](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L434) +Defined in: junior-plugin-api/src/operations.ts:69 #### Inherited from @@ -41,7 +41,7 @@ Defined in: [junior-plugin-api/src/index.ts:434](https://github.com/getsentry/ju > **pluginName**: `string` -Defined in: [junior-plugin-api/src/index.ts:440](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L440) +Defined in: junior-plugin-api/src/operations.ts:75 --- @@ -49,7 +49,7 @@ Defined in: [junior-plugin-api/src/index.ts:440](https://github.com/getsentry/ju > `optional` **recordSets?**: `PluginOperationalRecordSet`[] -Defined in: [junior-plugin-api/src/index.ts:435](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L435) +Defined in: junior-plugin-api/src/operations.ts:70 #### Inherited from @@ -61,7 +61,7 @@ Defined in: [junior-plugin-api/src/index.ts:435](https://github.com/getsentry/ju > `optional` **title?**: `string` -Defined in: [junior-plugin-api/src/index.ts:436](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L436) +Defined in: junior-plugin-api/src/operations.ts:71 #### Inherited from diff --git a/packages/docs/src/content/docs/reference/api/type-aliases/AgentPluginConversationStatus.md b/packages/docs/src/content/docs/reference/api/type-aliases/AgentPluginConversationStatus.md deleted file mode 100644 index 317e66656..000000000 --- a/packages/docs/src/content/docs/reference/api/type-aliases/AgentPluginConversationStatus.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -editUrl: false -next: false -prev: false -title: "AgentPluginConversationStatus" ---- - -> **AgentPluginConversationStatus** = `"active"` \| `"completed"` \| `"failed"` \| `"hung"` \| `"superseded"` - -Defined in: [junior-plugin-api/src/index.ts:370](https://github.com/getsentry/junior/blob/main/packages/junior-plugin-api/src/index.ts#L370) diff --git a/packages/docs/src/content/docs/reference/api/type-aliases/JuniorPluginInput.md b/packages/docs/src/content/docs/reference/api/type-aliases/JuniorPluginInput.md index a196e754a..000781b37 100644 --- a/packages/docs/src/content/docs/reference/api/type-aliases/JuniorPluginInput.md +++ b/packages/docs/src/content/docs/reference/api/type-aliases/JuniorPluginInput.md @@ -5,6 +5,6 @@ prev: false title: "JuniorPluginInput" --- -> **JuniorPluginInput** = `JuniorPluginRegistration` \| `string` +> **JuniorPluginInput** = `PluginRegistration` \| `string` Defined in: [junior/src/plugins.ts:8](https://github.com/getsentry/junior/blob/main/packages/junior/src/plugins.ts#L8) diff --git a/packages/docs/src/content/docs/reference/api/type-aliases/PluginConversationStatus.md b/packages/docs/src/content/docs/reference/api/type-aliases/PluginConversationStatus.md new file mode 100644 index 000000000..22cad7842 --- /dev/null +++ b/packages/docs/src/content/docs/reference/api/type-aliases/PluginConversationStatus.md @@ -0,0 +1,10 @@ +--- +editUrl: false +next: false +prev: false +title: "PluginConversationStatus" +--- + +> **PluginConversationStatus** = `"active"` \| `"completed"` \| `"failed"` \| `"hung"` \| `"superseded"` + +Defined in: junior-plugin-api/src/operations.ts:5 diff --git a/packages/junior-dashboard/src/index.ts b/packages/junior-dashboard/src/index.ts index de9f71175..cc87acc3c 100644 --- a/packages/junior-dashboard/src/index.ts +++ b/packages/junior-dashboard/src/index.ts @@ -1,7 +1,7 @@ import { - type AgentPluginRoute, + type PluginRoute, defineJuniorPlugin, - type JuniorPluginRegistration, + type PluginRegistration, } from "@sentry/junior-plugin-api"; import { buildDashboardConversationURL, normalizeDashboardPath } from "./url"; import { createDashboardApp, type JuniorDashboardOptions } from "./app"; @@ -36,9 +36,7 @@ function dashboardRoutePaths(options: JuniorDashboardPluginOptions): string[] { ]; } -function dashboardRoutes( - options: JuniorDashboardPluginOptions, -): AgentPluginRoute[] { +function dashboardRoutes(options: JuniorDashboardPluginOptions): PluginRoute[] { let app: ReturnType | undefined; const fetch = (request: Request) => { app ??= createDashboardApp(options); @@ -54,9 +52,8 @@ function dashboardRoutes( /** Register dashboard routes and Slack footer links through plugin hooks. */ export function juniorDashboardPlugin( options: JuniorDashboardPluginOptions = {}, -): JuniorPluginRegistration { +): PluginRegistration { return defineJuniorPlugin({ - name: "dashboard", manifest: { name: "dashboard", displayName: "Dashboard", diff --git a/packages/junior-dashboard/tests/plugin.test.ts b/packages/junior-dashboard/tests/plugin.test.ts index 4a8f647eb..fa698d79f 100644 --- a/packages/junior-dashboard/tests/plugin.test.ts +++ b/packages/junior-dashboard/tests/plugin.test.ts @@ -33,7 +33,6 @@ describe("juniorDashboardPlugin", () => { it("registers an inline dashboard manifest", () => { const plugin = juniorDashboardPlugin(); - expect(plugin.name).toBe("dashboard"); expect(plugin.manifest).toMatchObject({ name: "dashboard", description: "Junior dashboard routes and Slack footer links", diff --git a/packages/junior-evals/evals/behavior-harness.ts b/packages/junior-evals/evals/behavior-harness.ts index 4d8263396..95fd7b527 100644 --- a/packages/junior-evals/evals/behavior-harness.ts +++ b/packages/junior-evals/evals/behavior-harness.ts @@ -3,8 +3,9 @@ import { spawn, type ChildProcess } from "node:child_process"; import { generateKeyPairSync } from "node:crypto"; import { createServer, type Server } from "node:http"; import { fileURLToPath } from "node:url"; +import { vi } from "vitest"; import type { Message } from "chat"; -import type { Destination } from "@sentry/junior-plugin-api"; +import type { Destination, PluginDb } from "@sentry/junior-plugin-api"; import { interceptTestHttp, resetTestGitHubHttpFixtures, @@ -30,12 +31,20 @@ import { deleteMcpStoredOAuthCredentials, getLatestMcpAuthSessionForUserProvider, } from "@/chat/mcp/auth-store"; -import { getAgentPlugins, setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { getPlugins, setPlugins } from "@/chat/plugins/agent-hooks"; +import { + createPluginDbForExecutor, + migratePluginSchemas, + readPluginMigrations, +} from "@/chat/plugins/db"; +import * as pluginDbModule from "@/chat/plugins/db"; import { getPluginOAuthConfig, setPluginCatalogConfig, } from "@/chat/plugins/registry"; import { generateAssistantReply } from "@/chat/respond"; +import type { JuniorDatabase } from "@/chat/sql/db"; +import { juniorSqlSchema } from "@/chat/sql/schema"; import { schedulerPlugin } from "@sentry/junior-scheduler"; import { getStateAdapter } from "@/chat/state/adapter"; import { resetSkillDiscoveryCache } from "@/chat/skills"; @@ -65,6 +74,10 @@ import { readCapturedSlackApiCalls, type CapturedSlackApiCall, } from "@junior-tests/msw/captured-slack-api-calls"; +import { + createLocalPgliteFixture, + type LocalPgliteFixture, +} from "@sentry/junior-test-fixtures/pglite"; import { createSlackDestination } from "@/chat/destination"; import { ALL as sandboxEgressProxyALL } from "@/handlers/sandbox-egress-proxy"; import { createMockImageGenerateDeps } from "./fixtures/image-generate"; @@ -390,7 +403,12 @@ function toEvalToolInvocation(input: { const EVAL_PACKAGE_ROOT = path.resolve( fileURLToPath(new URL("..", import.meta.url)), ); +const SCHEDULER_MIGRATIONS_DIR = path.resolve( + EVAL_PACKAGE_ROOT, + "../junior-scheduler/migrations", +); type HarnessStateAdapter = ReturnType; +type EvalSchedulerSqlFixture = LocalPgliteFixture; const THREAD_STATE_TTL_MS = 30 * 24 * 60 * 60 * 1000; const EVAL_SLACK_TEAM_ID = "TEVAL"; @@ -433,6 +451,25 @@ function createEvalDestination(thread: TestThread): Destination { return destination; } +async function createEvalSchedulerSqlFixture(): Promise<{ + db: PluginDb; + fixture: EvalSchedulerSqlFixture; +}> { + const fixture = + await createLocalPgliteFixture(juniorSqlSchema); + await migratePluginSchemas( + fixture, + readPluginMigrations({ + dir: SCHEDULER_MIGRATIONS_DIR, + pluginName: "scheduler", + }), + ); + return { + db: createPluginDbForExecutor(fixture), + fixture, + }; +} + // --------------------------------------------------------------------------- // Environment snapshot helper // --------------------------------------------------------------------------- @@ -1694,13 +1731,29 @@ export async function runEvalScenario( ): Promise { const logRecords = options.logRecords ?? []; const env = await setupHarnessEnvironment(scenario); - let previousAgentPlugins: ReturnType | undefined; + let previousPlugins: ReturnType | undefined; + let schedulerSqlFixture: EvalSchedulerSqlFixture | undefined; + let pluginDbSpy: { mockRestore(): void } | undefined; try { - const currentAgentPlugins = getAgentPlugins(); - previousAgentPlugins = setAgentPlugins([ + const getPluginDbForRegistration = + pluginDbModule.getPluginDbForRegistration; + const schedulerSql = await createEvalSchedulerSqlFixture(); + schedulerSqlFixture = schedulerSql.fixture; + pluginDbSpy = vi + .spyOn(pluginDbModule, "getPluginDbForRegistration") + .mockImplementation((plugin) => + plugin.manifest.name === "scheduler" + ? schedulerSql.db + : getPluginDbForRegistration(plugin), + ); + + const currentPlugins = getPlugins(); + previousPlugins = setPlugins([ schedulerPlugin(), - ...currentAgentPlugins.filter((plugin) => plugin.name !== "scheduler"), + ...currentPlugins.filter( + (plugin) => plugin.manifest.name !== "scheduler", + ), ]); const slackAdapter = new FakeSlackAdapter(); @@ -1771,9 +1824,11 @@ export async function runEvalScenario( observations, ); } finally { - if (previousAgentPlugins) { - setAgentPlugins(previousAgentPlugins); + if (previousPlugins) { + setPlugins(previousPlugins); } + pluginDbSpy?.mockRestore(); + await schedulerSqlFixture?.close(); await teardownHarnessEnvironment(scenario, env); } } diff --git a/packages/junior-evals/package.json b/packages/junior-evals/package.json index 3f0a18f95..9bf104849 100644 --- a/packages/junior-evals/package.json +++ b/packages/junior-evals/package.json @@ -15,6 +15,7 @@ "@sentry/junior-plugin-api": "workspace:*", "@sentry/junior-scheduler": "workspace:*", "@sentry/junior-sentry": "workspace:*", + "@sentry/junior-test-fixtures": "workspace:*", "@sentry/junior-testing": "workspace:*", "chat": "4.29.0", "tinyrainbow": "^3.1.0", diff --git a/packages/junior-github/index.d.ts b/packages/junior-github/index.d.ts index c6ba9ff5c..936d82081 100644 --- a/packages/junior-github/index.d.ts +++ b/packages/junior-github/index.d.ts @@ -1,4 +1,4 @@ -import type { JuniorPluginRegistration } from "@sentry/junior-plugin-api"; +import type { PluginRegistration } from "@sentry/junior-plugin-api"; export type GitHubAppPermissionLevel = "read" | "write" | "admin"; @@ -46,6 +46,4 @@ export interface GitHubPluginOptions { } /** Register GitHub manifest content and runtime hooks. */ -export function githubPlugin( - options?: GitHubPluginOptions, -): JuniorPluginRegistration; +export function githubPlugin(options?: GitHubPluginOptions): PluginRegistration; diff --git a/packages/junior-plugin-api/package.json b/packages/junior-plugin-api/package.json index 4d2140847..2078719cf 100644 --- a/packages/junior-plugin-api/package.json +++ b/packages/junior-plugin-api/package.json @@ -22,7 +22,8 @@ "src" ], "dependencies": { - "zod": "^4.4.3" + "drizzle-orm": "catalog:", + "zod": "catalog:" }, "scripts": { "build": "tsup && tsc -p tsconfig.build.json --emitDeclarationOnly", diff --git a/packages/junior-plugin-api/src/context.ts b/packages/junior-plugin-api/src/context.ts new file mode 100644 index 000000000..305ea2d00 --- /dev/null +++ b/packages/junior-plugin-api/src/context.ts @@ -0,0 +1,72 @@ +import { z } from "zod"; +import { + destinationSchema, + localRequesterSchema, + requesterSchema, + slackRequesterSchema, + sourceSchema, +} from "./schemas"; +import type { PluginDb } from "./database"; + +export type Requester = z.output; +export type SlackRequester = z.output; +export type LocalRequester = z.output; +export type Source = z.output; +export type SlackSource = Extract; +export type LocalSource = Extract; + +export type Destination = z.output; + +export type SlackDestination = Extract; + +export type LocalDestination = Extract; + +export interface PluginMetadata { + name: string; +} + +export interface PluginLogger { + error(message: string, metadata?: Record): void; + info(message: string, metadata?: Record): void; + warn(message: string, metadata?: Record): void; +} + +export interface PluginContext { + /** Shared database connection for plugins that declare database access. */ + db?: PluginDb; + log: PluginLogger; + plugin: PluginMetadata; +} + +interface BaseInvocationContext { + /** + * Opaque Junior conversation/session identity for this invocation. + * Interactive Slack turns use `slack:{channelId}:{threadTs}`. + */ + conversationId?: string; +} + +export interface SlackInvocationContext extends BaseInvocationContext { + /** Runtime-owned default outbound destination for this invocation, if any. */ + destination?: SlackDestination; + requester?: SlackRequester; + /** Runtime-owned source where the invocation came from. */ + source: SlackSource; +} + +export interface LocalInvocationContext extends BaseInvocationContext { + /** Runtime-owned default outbound destination for this invocation, if any. */ + destination?: LocalDestination; + requester?: LocalRequester; + /** Runtime-owned source where the invocation came from. */ + source: LocalSource; +} + +export type InvocationContext = LocalInvocationContext | SlackInvocationContext; + +/** Narrow a runtime destination to the Slack-specific address shape. */ +export function isSlackDestination( + destination: Destination | undefined, +): destination is SlackDestination { + return destination?.platform === "slack"; +} diff --git a/packages/junior-plugin-api/src/credentials.ts b/packages/junior-plugin-api/src/credentials.ts new file mode 100644 index 000000000..0fd7bfc00 --- /dev/null +++ b/packages/junior-plugin-api/src/credentials.ts @@ -0,0 +1,200 @@ +import { z } from "zod"; +import type { PluginContext } from "./context"; +import { nonBlankStringSchema, pluginCredentialSubjectSchema } from "./schemas"; + +const pluginProviderNameSchema = z.string().regex(/^[a-z][a-z0-9-]*$/); +const pluginGrantNameSchema = z.string().regex(/^[a-z][a-z0-9.-]*$/); +const pluginGrantAccessSchema = z.union([ + z.literal("read"), + z.literal("write"), +]); + +/** Runtime schema for provider authorization a plugin may request. */ +export const pluginAuthorizationSchema = z + .object({ + provider: pluginProviderNameSchema, + scope: nonBlankStringSchema.optional(), + type: z.literal("oauth"), + }) + .strict(); + +/** Runtime schema for a provider account attached to stored OAuth tokens. */ +export const pluginProviderAccountSchema = z + .object({ + id: nonBlankStringSchema, + label: nonBlankStringSchema.optional(), + url: nonBlankStringSchema.optional(), + }) + .strict(); + +/** Runtime schema for a plugin-defined outbound credential grant. */ +export const pluginGrantSchema = z + .object({ + access: pluginGrantAccessSchema, + name: pluginGrantNameSchema, + reason: nonBlankStringSchema.optional(), + requirements: z.array(nonBlankStringSchema).min(1).optional(), + }) + .strict(); + +/** Runtime schema for plugin-issued header mutations. */ +export const pluginCredentialHeaderTransformSchema = z + .object({ + domain: z.string().min(1), + headers: z + .record(z.string(), z.string()) + .refine((headers) => Object.keys(headers).length > 0), + }) + .strict(); + +/** Runtime schema for a short-lived plugin-issued credential lease. */ +export const pluginCredentialLeaseSchema = z + .object({ + account: pluginProviderAccountSchema.optional(), + authorization: pluginAuthorizationSchema.optional(), + expiresAt: z.string().refine((value) => Number.isFinite(Date.parse(value))), + headerTransforms: z.array(pluginCredentialHeaderTransformSchema).min(1), + }) + .strict(); + +/** Runtime schema for the result returned by a plugin credential hook. */ +export const pluginCredentialResultSchema = z.discriminatedUnion("type", [ + z + .object({ + lease: pluginCredentialLeaseSchema, + type: z.literal("lease"), + }) + .strict(), + z + .object({ + authorization: pluginAuthorizationSchema.optional(), + message: nonBlankStringSchema, + type: z.literal("needed"), + }) + .strict(), + z + .object({ + message: nonBlankStringSchema, + type: z.literal("unavailable"), + }) + .strict(), +]); + +export type PluginCredentialSubject = z.output< + typeof pluginCredentialSubjectSchema +>; + +export type PluginGrantAccess = z.output; + +/** Provider authorization Junior can start when a plugin-owned grant is missing. */ +export type PluginAuthorization = z.output; + +/** Interrupt sandbox egress so Junior can start provider authorization. */ +export class EgressAuthRequired extends Error { + authorization?: PluginAuthorization; + + constructor( + message: string, + options?: { + authorization?: PluginAuthorization; + cause?: unknown; + }, + ) { + super(message, { cause: options?.cause }); + this.name = "EgressAuthRequired"; + this.authorization = options?.authorization; + } +} + +/** Provider account identity resolved by a plugin OAuth hook. */ +export type PluginProviderAccount = z.output< + typeof pluginProviderAccountSchema +>; + +/** Plugin-defined grant required before Junior can forward one outbound request. */ +export type PluginGrant = z.output; + +/** Request details available while selecting the grant for sandbox egress. */ +export interface PluginEgressRequest { + /** Capped request body text when the host exposes it for provider-specific grant classification. */ + bodyText?: string; + method: string; + url: string; +} + +export interface EgressHookContext extends PluginContext { + request: PluginEgressRequest; +} + +export interface PluginEgressResponse { + /** Snapshot of upstream response headers; mutations do not affect pass-through. */ + headers: Headers; + readText(maxBytes: number): Promise; + status: number; +} + +export interface EgressResponseHookContext extends PluginContext { + grant: PluginGrant; + permissionDenied(message: string): void; + request: Omit; + response: PluginEgressResponse; +} + +/** Header mutations a plugin-issued credential lease may apply to owned domains. */ +export type PluginCredentialHeaderTransform = z.output< + typeof pluginCredentialHeaderTransformSchema +>; + +/** Short-lived credential headers issued by a plugin for a selected grant. */ +export type PluginCredentialLease = z.output< + typeof pluginCredentialLeaseSchema +>; + +export type PluginCredentialResult = z.output< + typeof pluginCredentialResultSchema +>; + +export type PluginCredentialActor = + | { + type: "system"; + id: string; + } + | { + type: "user"; + userId: string; + }; + +export interface PluginResolvedCredentialUser { + type: "user"; + userId: string; +} + +export interface PluginStoredTokens { + account?: PluginProviderAccount; + accessToken: string; + expiresAt?: number; + refreshToken: string; + scope?: string; +} + +export interface PluginUserTokenSlot { + get(): Promise; + set(tokens: PluginStoredTokens): Promise; + userId: string; +} + +export interface PluginTokenStore { + credentialSubject?: PluginUserTokenSlot; + currentUser?: PluginUserTokenSlot; +} + +export interface ResolveOAuthAccountHookContext extends PluginContext { + tokens: PluginStoredTokens; +} + +export interface IssueCredentialHookContext extends PluginContext { + actor: PluginCredentialActor; + credentialSubject?: PluginResolvedCredentialUser; + grant: PluginGrant; + tokens: PluginTokenStore; +} diff --git a/packages/junior-plugin-api/src/database.ts b/packages/junior-plugin-api/src/database.ts new file mode 100644 index 000000000..6b93ba328 --- /dev/null +++ b/packages/junior-plugin-api/src/database.ts @@ -0,0 +1,22 @@ +import type { PgDatabase } from "drizzle-orm/pg-core"; +import type { PgQueryResultHKT } from "drizzle-orm/pg-core/session"; + +export type PluginDrizzleDatabase = PgDatabase< + PgQueryResultHKT, + Record +>; + +export interface PluginDb { + delete: PluginDrizzleDatabase["delete"]; + execute(statement: string, params?: readonly unknown[]): Promise; + insert: PluginDrizzleDatabase["insert"]; + query( + statement: string, + params?: readonly unknown[], + ): Promise; + select: PluginDrizzleDatabase["select"]; + transaction(callback: (tx: PluginDb) => Promise): Promise; + update: PluginDrizzleDatabase["update"]; +} + +export type PluginDatabaseConfig = Record; diff --git a/packages/junior-plugin-api/src/dispatch.ts b/packages/junior-plugin-api/src/dispatch.ts new file mode 100644 index 000000000..20d9749a6 --- /dev/null +++ b/packages/junior-plugin-api/src/dispatch.ts @@ -0,0 +1,22 @@ +import { z } from "zod"; +import { dispatchOptionsSchema } from "./schemas"; + +export type DispatchOptions = z.output; + +export interface DispatchResult { + id: string; + status: "created" | "already_exists"; +} + +export interface Dispatch { + errorMessage?: string; + id: string; + resultMessageTs?: string; + status: + | "pending" + | "running" + | "awaiting_resume" + | "completed" + | "failed" + | "blocked"; +} diff --git a/packages/junior-plugin-api/src/hooks.ts b/packages/junior-plugin-api/src/hooks.ts new file mode 100644 index 000000000..6a58e78b2 --- /dev/null +++ b/packages/junior-plugin-api/src/hooks.ts @@ -0,0 +1,67 @@ +import type { + EgressHookContext, + EgressResponseHookContext, + IssueCredentialHookContext, + PluginCredentialResult, + PluginGrant, + PluginProviderAccount, + ResolveOAuthAccountHookContext, +} from "./credentials"; +import type { + HeartbeatHookContext, + HeartbeatResult, + OperationalReportHookContext, + PluginOperationalReportContent, + PluginRoute, + RouteRegistrationHookContext, + SlackConversationLink, + SlackConversationLinkHookContext, + StorageMigrationContext, + StorageMigrationResult, +} from "./operations"; +import type { + BeforeToolExecuteHookContext, + PluginToolDefinition, + SandboxPrepareHookContext, + ToolRegistrationHookContext, +} from "./tools"; + +export interface PluginHooks { + beforeToolExecute?(ctx: BeforeToolExecuteHookContext): Promise | void; + grantForEgress?( + ctx: EgressHookContext, + ): Promise | PluginGrant | undefined; + heartbeat?( + ctx: HeartbeatHookContext, + ): Promise | HeartbeatResult | void; + issueCredential?( + ctx: IssueCredentialHookContext, + ): Promise | PluginCredentialResult; + onEgressResponse?(ctx: EgressResponseHookContext): Promise | void; + operationalReport?( + ctx: OperationalReportHookContext, + ): + | Promise + | PluginOperationalReportContent + | undefined; + resolveOAuthAccount?( + ctx: ResolveOAuthAccountHookContext, + ): + | Promise + | PluginProviderAccount + | undefined; + routes?(ctx: RouteRegistrationHookContext): PluginRoute[]; + sandboxPrepare?(ctx: SandboxPrepareHookContext): Promise | void; + slackConversationLink?( + ctx: SlackConversationLinkHookContext, + ): SlackConversationLink | undefined; + tools?( + ctx: ToolRegistrationHookContext, + ): Record; + migrateStorage?( + ctx: StorageMigrationContext, + ): + | Promise + | StorageMigrationResult + | undefined; +} diff --git a/packages/junior-plugin-api/src/index.ts b/packages/junior-plugin-api/src/index.ts index a100bfe97..29ee97c2e 100644 --- a/packages/junior-plugin-api/src/index.ts +++ b/packages/junior-plugin-api/src/index.ts @@ -1,875 +1,11 @@ -import { z } from "zod"; - -const slackTeamIdSchema = z.string().regex(/^T[A-Z0-9]+$/); -const slackConversationIdSchema = z.string().regex(/^(C|G|D)[A-Z0-9]+$/); -const localConversationIdSchema = z - .string() - .regex(/^local:[a-z0-9_-]+:[a-z0-9][a-z0-9_-]*$/); -const exactActorUserIdSchema = z - .string() - .min(1) - .refine( - (value) => value === value.trim() && value.toLowerCase() !== "unknown", - ); -const nonBlankStringSchema = z - .string() - .refine((value) => value.trim().length > 0); - -/** Runtime-owned Slack address for routing future work or side effects. */ -export const slackDestinationSchema = z - .object({ - platform: z.literal("slack"), - teamId: slackTeamIdSchema, - channelId: slackConversationIdSchema, - }) - .strict(); - -/** Runtime-owned local CLI conversation address. */ -export const localDestinationSchema = z - .object({ - platform: z.literal("local"), - conversationId: localConversationIdSchema, - }) - .strict(); - -/** Runtime-owned provider-neutral address for routing future work or side effects. */ -export const destinationSchema = z.discriminatedUnion("platform", [ - slackDestinationSchema, - localDestinationSchema, -]); - -/** Runtime-owned Slack coordinates for the inbound invocation. */ -export const slackSourceSchema = z - .object({ - platform: z.literal("slack"), - teamId: slackTeamIdSchema, - channelId: slackConversationIdSchema, - messageTs: nonBlankStringSchema.optional(), - threadTs: nonBlankStringSchema.optional(), - }) - .strict(); - -/** Runtime-owned local CLI coordinates for the inbound invocation. */ -export const localSourceSchema = localDestinationSchema; - -/** Runtime-owned provider-neutral coordinates for the inbound invocation. */ -export const sourceSchema = z.discriminatedUnion("platform", [ - slackSourceSchema, - localSourceSchema, -]); - -/** Stable user credential subject shape accepted from plugins. */ -export const agentPluginCredentialSubjectSchema = z - .object({ - type: z.literal("user"), - userId: exactActorUserIdSchema, - allowedWhen: z.literal("private-direct-conversation"), - }) - .strict(); - -/** Shared exact actor profile fields for platform-scoped requesters. */ -const requesterProfileSchema = { - email: nonBlankStringSchema.optional(), - fullName: nonBlankStringSchema.optional(), - userId: exactActorUserIdSchema, - userName: nonBlankStringSchema.optional(), -}; - -export const slackRequesterSchema = z - .object({ - ...requesterProfileSchema, - platform: z.literal("slack"), - teamId: slackTeamIdSchema, - }) - .strict(); - -export const localRequesterSchema = z - .object({ - ...requesterProfileSchema, - platform: z.literal("local"), - }) - .strict(); - -/** Runtime-provided requester identity visible to plugin hooks. */ -export const requesterSchema = z.discriminatedUnion("platform", [ - slackRequesterSchema, - localRequesterSchema, -]); - -const dispatchMetadataSchema = z - .record(z.string(), z.string()) - .superRefine((metadata, ctx) => { - const entries = Object.entries(metadata); - if (entries.length > 20) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: "Dispatch metadata has too many keys", - }); - return; - } - for (const [key, value] of entries) { - if (!key.trim()) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: "Dispatch metadata values must be strings", - path: [key], - }); - continue; - } - if (key.length > 128) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: "Dispatch metadata key exceeds the maximum length", - path: [key], - }); - } - if (value.length > 512) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: "Dispatch metadata value exceeds the maximum length", - path: [key], - }); - } - } - }); - -/** Plugin dispatch request accepted by Junior core. */ -export const dispatchOptionsSchema = z - .object({ - idempotencyKey: nonBlankStringSchema.pipe(z.string().max(512)), - credentialSubject: agentPluginCredentialSubjectSchema.optional(), - destination: slackDestinationSchema, - input: nonBlankStringSchema.pipe(z.string().max(32_000)), - metadata: dispatchMetadataSchema.optional(), - }) - .strict(); - -export type Requester = z.output; -export type SlackRequester = z.output; -export type LocalRequester = z.output; -export type Source = z.output; -export type SlackSource = Extract; -export type LocalSource = Extract; - -export interface AgentPluginMetadata { - name: string; -} - -export interface AgentPluginEnv { - get(key: string): string | undefined; - set(key: string, value: string): void; -} - -export interface AgentPluginDecision { - deny(message: string): void; - replaceInput(input: Record): void; -} - -export interface AgentPluginLogger { - error(message: string, metadata?: Record): void; - info(message: string, metadata?: Record): void; - warn(message: string, metadata?: Record): void; -} - -/** Thrown when a plugin tool rejects invalid model or user input. */ -export class AgentPluginToolInputError extends Error { - constructor(message: string, options?: { cause?: unknown }) { - super(message, options); - this.name = "AgentPluginToolInputError"; - } -} - -export interface AgentPluginContext { - log: AgentPluginLogger; - plugin: AgentPluginMetadata; -} - -interface BaseInvocationContext { - /** - * Opaque Junior conversation/session identity for this invocation. - * Interactive Slack turns use `slack:{channelId}:{threadTs}`. - */ - conversationId?: string; -} - -export interface SlackInvocationContext extends BaseInvocationContext { - /** Runtime-owned default outbound destination for this invocation, if any. */ - destination?: SlackDestination; - requester?: SlackRequester; - /** Runtime-owned source where the invocation came from. */ - source: SlackSource; -} - -export interface LocalInvocationContext extends BaseInvocationContext { - /** Runtime-owned default outbound destination for this invocation, if any. */ - destination?: LocalDestination; - requester?: LocalRequester; - /** Runtime-owned source where the invocation came from. */ - source: LocalSource; -} - -export type InvocationContext = LocalInvocationContext | SlackInvocationContext; - -export interface AgentPluginSandbox { - juniorRoot: string; - root: string; - readFile(path: string): Promise; - run(input: { - args?: string[]; - cmd: string; - cwd?: string; - env?: Record; - sudo?: boolean; - }): Promise<{ - exitCode: number; - stderr: string; - stdout: string; - }>; - writeFile(input: { - content: string | Uint8Array; - mode?: number; - path: string; - }): Promise; -} - -export interface SandboxPrepareHookContext extends AgentPluginContext { - requester?: Requester; - sandbox: AgentPluginSandbox; -} - -export interface BeforeToolExecuteHookContext extends AgentPluginContext { - decision: AgentPluginDecision; - env: AgentPluginEnv; - requester?: Requester; - tool: { - input: Record; - name: string; - }; -} - -export type AgentPluginToolExecute = { - bivarianceHack( - input: TInput, - options: { experimental_context?: unknown }, - ): Promise | unknown; -}["bivarianceHack"]; - -export interface AgentPluginToolDefinition { - annotations?: unknown; - description: string; - executionMode?: unknown; - inputSchema: unknown; - prepareArguments?: (args: unknown) => unknown; - /** - * @deprecated Put tool-selection and usage guidance directly in `description` - * and parameter descriptions. Retained for compatibility; may be removed in a - * future major version. - */ - promptGuidelines?: string[]; - /** - * @deprecated Put tool-selection and usage guidance directly in `description` - * and parameter descriptions. Retained for compatibility; may be removed in a - * future major version. - */ - promptSnippet?: string; - execute?: AgentPluginToolExecute; -} - -export interface SlackToolRegistrationHookContext { - /** - * Capabilities of the source Slack conversation exposed to this plugin. - * Recomputed from `source.channelId`, not from `destination`. - */ - channelCapabilities: { - canAddReactions: boolean; - canCreateCanvas: boolean; - canPostToChannel: boolean; - }; - credentialSubject?: AgentPluginCredentialSubject; -} - -interface BaseToolRegistrationHookContext extends AgentPluginContext { - /** - * Opaque Junior conversation/session identity for this turn. - * Interactive Slack turns use `slack:{channelId}:{threadTs}`. - * Scheduled/API turns use an internal id such as `agent-dispatch:{id}`. - * Do not parse as Slack unless the value starts with `slack:`. - */ - conversationId?: string; - state: AgentPluginState; - userText?: string; -} - -interface SlackToolRegistrationContext - extends BaseToolRegistrationHookContext, SlackInvocationContext { - slack: SlackToolRegistrationHookContext; -} - -interface LocalToolRegistrationContext - extends BaseToolRegistrationHookContext, LocalInvocationContext { - slack?: never; -} - -export type ToolRegistrationHookContext = - | LocalToolRegistrationContext - | SlackToolRegistrationContext; - -export type AgentPluginCredentialSubject = z.output< - typeof agentPluginCredentialSubjectSchema ->; - -export type Destination = z.output; - -export type SlackDestination = Extract; - -export type LocalDestination = Extract; - -/** Narrow a runtime destination to the Slack-specific address shape. */ -export function isSlackDestination( - destination: Destination | undefined, -): destination is SlackDestination { - return destination?.platform === "slack"; -} - -export type DispatchOptions = z.output; - -export interface DispatchResult { - id: string; - status: "created" | "already_exists"; -} - -export interface Dispatch { - errorMessage?: string; - id: string; - resultMessageTs?: string; - status: - | "pending" - | "running" - | "awaiting_resume" - | "completed" - | "failed" - | "blocked"; -} - -export interface AgentPluginState { - delete(key: string): Promise; - get(key: string): Promise; - set(key: string, value: unknown, ttlMs?: number): Promise; - setIfNotExists(key: string, value: unknown, ttlMs?: number): Promise; - withLock( - key: string, - ttlMs: number, - callback: () => Promise, - ): Promise; -} - -export interface AgentPluginReadState { - get(key: string): Promise; -} - -export type AgentPluginConversationStatus = - | "active" - | "completed" - | "failed" - | "hung" - | "superseded"; - -export interface AgentPluginConversationSummary { - channelName?: string; - conversationId: string; - displayTitle: string; - lastActivityAt: string; - lastUpdatedAt: string; - source?: "api" | "internal" | "local" | "plugin" | "scheduler" | "slack"; - status: AgentPluginConversationStatus; -} - -export interface AgentPluginConversations { - listRecent(options?: { - limit?: number; - }): Promise; -} - -export interface HeartbeatHookContext extends AgentPluginContext { - agent: { - dispatch(options: DispatchOptions): Promise; - get(id: string): Promise; - }; - nowMs: number; - state: AgentPluginState; -} - -export interface HeartbeatResult { - dispatchCount?: number; -} - -export type PluginOperationalTone = "danger" | "good" | "neutral" | "warning"; - -export interface PluginOperationalMetric { - label: string; - tone?: PluginOperationalTone; - value: string; -} - -export interface PluginOperationalField { - key: string; - label: string; -} - -export interface PluginOperationalRecord { - id: string; - tone?: PluginOperationalTone; - values: Record; -} - -export interface PluginOperationalRecordSet { - fields?: PluginOperationalField[]; - emptyText?: string; - records?: PluginOperationalRecord[]; - title: string; -} - -export interface PluginOperationalReportContent { - generatedAt?: string; - metrics?: PluginOperationalMetric[]; - recordSets?: PluginOperationalRecordSet[]; - title?: string; -} - -export interface PluginOperationalReport extends PluginOperationalReportContent { - pluginName: string; -} - -export interface OperationalReportHookContext extends AgentPluginContext { - conversations: AgentPluginConversations; - nowMs: number; - state: AgentPluginReadState; -} - -export type AgentPluginRouteMethod = - | "GET" - | "POST" - | "PUT" - | "PATCH" - | "DELETE" - | "HEAD" - | "OPTIONS" - | "ALL"; - -export type AgentPluginRouteHandler = { - bivarianceHack(request: Request): Promise | Response; -}["bivarianceHack"]; - -export interface AgentPluginRoute { - handler: AgentPluginRouteHandler; - method?: AgentPluginRouteMethod | AgentPluginRouteMethod[]; - path: string; -} - -export interface RouteRegistrationHookContext extends AgentPluginContext {} - -export interface SlackConversationLink { - url: string; -} - -export interface SlackConversationLinkHookContext extends AgentPluginContext { - conversationId: string; -} - -const agentPluginProviderNameSchema = z.string().regex(/^[a-z][a-z0-9-]*$/); -const agentPluginGrantNameSchema = z.string().regex(/^[a-z][a-z0-9.-]*$/); -const agentPluginGrantAccessSchema = z.union([ - z.literal("read"), - z.literal("write"), -]); - -/** Runtime schema for provider authorization a plugin may request. */ -export const agentPluginAuthorizationSchema = z - .object({ - provider: agentPluginProviderNameSchema, - scope: nonBlankStringSchema.optional(), - type: z.literal("oauth"), - }) - .strict(); - -/** Runtime schema for a provider account attached to stored OAuth tokens. */ -export const agentPluginProviderAccountSchema = z - .object({ - id: nonBlankStringSchema, - label: nonBlankStringSchema.optional(), - url: nonBlankStringSchema.optional(), - }) - .strict(); - -/** Runtime schema for a plugin-defined outbound credential grant. */ -export const agentPluginGrantSchema = z - .object({ - access: agentPluginGrantAccessSchema, - name: agentPluginGrantNameSchema, - reason: nonBlankStringSchema.optional(), - requirements: z.array(nonBlankStringSchema).min(1).optional(), - }) - .strict(); - -/** Runtime schema for plugin-issued header mutations. */ -export const agentPluginCredentialHeaderTransformSchema = z - .object({ - domain: z.string().min(1), - headers: z - .record(z.string(), z.string()) - .refine((headers) => Object.keys(headers).length > 0), - }) - .strict(); - -/** Runtime schema for a short-lived plugin-issued credential lease. */ -export const agentPluginCredentialLeaseSchema = z - .object({ - account: agentPluginProviderAccountSchema.optional(), - authorization: agentPluginAuthorizationSchema.optional(), - expiresAt: z.string().refine((value) => Number.isFinite(Date.parse(value))), - headerTransforms: z - .array(agentPluginCredentialHeaderTransformSchema) - .min(1), - }) - .strict(); - -/** Runtime schema for the result returned by a plugin credential hook. */ -export const agentPluginCredentialResultSchema = z.discriminatedUnion("type", [ - z - .object({ - lease: agentPluginCredentialLeaseSchema, - type: z.literal("lease"), - }) - .strict(), - z - .object({ - authorization: agentPluginAuthorizationSchema.optional(), - message: nonBlankStringSchema, - type: z.literal("needed"), - }) - .strict(), - z - .object({ - message: nonBlankStringSchema, - type: z.literal("unavailable"), - }) - .strict(), -]); - -export type AgentPluginGrantAccess = z.output< - typeof agentPluginGrantAccessSchema ->; - -/** Provider authorization Junior can start when a plugin-owned grant is missing. */ -export type AgentPluginAuthorization = z.output< - typeof agentPluginAuthorizationSchema ->; - -/** Interrupt sandbox egress so Junior can start provider authorization. */ -export class EgressAuthRequired extends Error { - authorization?: AgentPluginAuthorization; - - constructor( - message: string, - options?: { - authorization?: AgentPluginAuthorization; - cause?: unknown; - }, - ) { - super(message, { cause: options?.cause }); - this.name = "EgressAuthRequired"; - this.authorization = options?.authorization; - } -} - -/** Provider account identity resolved by a plugin OAuth hook. */ -export type AgentPluginProviderAccount = z.output< - typeof agentPluginProviderAccountSchema ->; - -/** Plugin-defined grant required before Junior can forward one outbound request. */ -export type AgentPluginGrant = z.output; - -/** Request details available while selecting the grant for sandbox egress. */ -export interface AgentPluginEgressRequest { - /** Capped request body text when the host exposes it for provider-specific grant classification. */ - bodyText?: string; - method: string; - url: string; -} - -export interface EgressHookContext extends AgentPluginContext { - request: AgentPluginEgressRequest; -} - -export interface AgentPluginEgressResponse { - /** Snapshot of upstream response headers; mutations do not affect pass-through. */ - headers: Headers; - readText(maxBytes: number): Promise; - status: number; -} - -export interface EgressResponseHookContext extends AgentPluginContext { - grant: AgentPluginGrant; - permissionDenied(message: string): void; - request: Omit; - response: AgentPluginEgressResponse; -} - -/** Header mutations a plugin-issued credential lease may apply to owned domains. */ -export type AgentPluginCredentialHeaderTransform = z.output< - typeof agentPluginCredentialHeaderTransformSchema ->; - -/** Short-lived credential headers issued by a plugin for a selected grant. */ -export type AgentPluginCredentialLease = z.output< - typeof agentPluginCredentialLeaseSchema ->; - -export type AgentPluginCredentialResult = z.output< - typeof agentPluginCredentialResultSchema ->; - -export type AgentPluginCredentialActor = - | { - type: "system"; - id: string; - } - | { - type: "user"; - userId: string; - }; - -export interface AgentPluginResolvedCredentialUser { - type: "user"; - userId: string; -} - -export interface AgentPluginStoredTokens { - account?: AgentPluginProviderAccount; - accessToken: string; - expiresAt?: number; - refreshToken: string; - scope?: string; -} - -export interface AgentPluginUserTokenSlot { - get(): Promise; - set(tokens: AgentPluginStoredTokens): Promise; - userId: string; -} - -export interface AgentPluginTokenStore { - credentialSubject?: AgentPluginUserTokenSlot; - currentUser?: AgentPluginUserTokenSlot; -} - -export interface ResolveOAuthAccountHookContext extends AgentPluginContext { - tokens: AgentPluginStoredTokens; -} - -export interface IssueCredentialHookContext extends AgentPluginContext { - actor: AgentPluginCredentialActor; - credentialSubject?: AgentPluginResolvedCredentialUser; - grant: AgentPluginGrant; - tokens: AgentPluginTokenStore; -} - -export interface AgentPluginHooks { - sandboxPrepare?(ctx: SandboxPrepareHookContext): Promise | void; - beforeToolExecute?(ctx: BeforeToolExecuteHookContext): Promise | void; - grantForEgress?( - ctx: EgressHookContext, - ): Promise | AgentPluginGrant | undefined; - issueCredential?( - ctx: IssueCredentialHookContext, - ): Promise | AgentPluginCredentialResult; - onEgressResponse?(ctx: EgressResponseHookContext): Promise | void; - resolveOAuthAccount?( - ctx: ResolveOAuthAccountHookContext, - ): - | Promise - | AgentPluginProviderAccount - | undefined; - routes?(ctx: RouteRegistrationHookContext): AgentPluginRoute[]; - tools?( - ctx: ToolRegistrationHookContext, - ): Record; - heartbeat?( - ctx: HeartbeatHookContext, - ): Promise | HeartbeatResult | void; - operationalReport?( - ctx: OperationalReportHookContext, - ): - | Promise - | PluginOperationalReportContent - | undefined; - slackConversationLink?( - ctx: SlackConversationLinkHookContext, - ): SlackConversationLink | undefined; -} - -export interface JuniorPluginOAuthConfig { - authorizeEndpoint: string; - authorizeParams?: Record; - clientIdEnv: string; - clientSecretEnv: string; - scope?: string; - /** - * Treat a provider token response with `scope: ""` like an omitted scope and - * fall back to the requested scope string when storing the token. - * - * Enable this only for providers whose token responses cannot report OAuth - * scopes even though Junior needs a local requested-scope string for - * reauthorization checks. The built-in GitHub App plugin enables this because - * GitHub App user-to-server tokens always return an empty scope value — their - * effective access is enforced by GitHub App permissions, installation - * repository access, and the requesting user's own access, not OAuth scopes. - * - * Do not enable this for standard OAuth providers where an explicit empty - * `scope` means the provider granted no scopes. - */ - treatEmptyScopeAsUnreported?: boolean; - tokenAuthMethod?: "body" | "basic"; - tokenEndpoint: string; - tokenExtraHeaders?: Record; -} - -export interface JuniorPluginOAuthBearerCredentials { - apiHeaders?: Record; - authTokenEnv: string; - authTokenPlaceholder?: string; - domains: string[]; - type: "oauth-bearer"; -} - -export type JuniorPluginCredentials = JuniorPluginOAuthBearerCredentials; - -export interface JuniorPluginNpmRuntimeDependency { - package: string; - type: "npm"; - version: string; -} - -export interface JuniorPluginSystemRuntimeDependency { - package: string; - type: "system"; -} - -export interface JuniorPluginSystemRuntimeDependencyFromUrl { - sha256: string; - type: "system"; - url: string; -} - -export type JuniorPluginRuntimeDependency = - | JuniorPluginNpmRuntimeDependency - | JuniorPluginSystemRuntimeDependency - | JuniorPluginSystemRuntimeDependencyFromUrl; - -export interface JuniorPluginRuntimePostinstallCommand { - args?: string[]; - cmd: string; - sudo?: boolean; -} - -export interface JuniorPluginMcpConfig { - allowedTools?: string[]; - headers?: Record; - transport: "http"; - url: string; -} - -export interface JuniorPluginEnvVarDeclaration { - default?: string; - exposeToCommandEnv?: boolean; -} - -export interface JuniorPluginManifest { - apiHeaders?: Record; - capabilities?: string[]; - commandEnv?: Record; - configKeys?: string[]; - credentials?: JuniorPluginCredentials; - description: string; - displayName: string; - domains?: string[]; - envVars?: Record; - mcp?: JuniorPluginMcpConfig; - name: string; - oauth?: JuniorPluginOAuthConfig; - runtimeDependencies?: JuniorPluginRuntimeDependency[]; - runtimePostinstall?: JuniorPluginRuntimePostinstallCommand[]; - target?: { - commandFlags?: string[]; - configKey: string; - type: string; - }; -} - -export type JuniorPluginRegistrationInput = { - hooks?: AgentPluginHooks; - legacyStatePrefixes?: string[]; - manifest: JuniorPluginManifest; - name?: string; - packageName?: string; -}; - -export interface JuniorPluginRegistration extends JuniorPluginRegistrationInput { - name: string; -} - -const PLUGIN_NAME_RE = /^[a-z][a-z0-9-]*$/; - -/** Define one Junior plugin registration for app and build-time wiring. */ -export function defineJuniorPlugin( - plugin: JuniorPluginRegistrationInput, -): JuniorPluginRegistration { - if ("pluginConfig" in plugin) { - throw new Error( - "pluginConfig is no longer supported. Put runtime metadata in manifest and state prefixes on the plugin registration.", - ); - } - const manifest = plugin.manifest; - if (!manifest) { - throw new Error( - "defineJuniorPlugin() requires a manifest. Use a package name string in defineJuniorPlugins([...]) for plugin.yaml packages.", - ); - } - const name = plugin.name ?? manifest.name; - if (!name) { - throw new Error( - "Junior plugin registrations must include name or manifest.name.", - ); - } - if (!PLUGIN_NAME_RE.test(name)) { - throw new Error( - `Junior plugin registration name "${name}" must be a lowercase plugin identifier.`, - ); - } - if ( - typeof manifest.displayName !== "string" || - !manifest.displayName.trim() - ) { - throw new Error( - `Junior plugin "${name}" manifest.displayName is required.`, - ); - } - if ( - typeof manifest.description !== "string" || - !manifest.description.trim() - ) { - throw new Error( - `Junior plugin "${name}" manifest.description is required.`, - ); - } - if (plugin.name && manifest.name && plugin.name !== manifest.name) { - throw new Error( - `Junior plugin registration name "${plugin.name}" must match manifest.name "${manifest.name}".`, - ); - } - return { - ...plugin, - name, - }; -} +export * from "./schemas"; +export * from "./context"; +export * from "./state"; +export * from "./dispatch"; +export * from "./database"; +export * from "./tools"; +export * from "./operations"; +export * from "./credentials"; +export * from "./hooks"; +export * from "./manifest"; +export * from "./registration"; diff --git a/packages/junior-plugin-api/src/manifest.ts b/packages/junior-plugin-api/src/manifest.ts new file mode 100644 index 000000000..91ac0666d --- /dev/null +++ b/packages/junior-plugin-api/src/manifest.ts @@ -0,0 +1,87 @@ +export interface PluginOAuthConfig { + authorizeEndpoint: string; + authorizeParams?: Record; + clientIdEnv: string; + clientSecretEnv: string; + scope?: string; + /** + * Treat a provider token response with `scope: ""` like an omitted scope and + * fall back to the requested scope string when storing the token. + */ + treatEmptyScopeAsUnreported?: boolean; + tokenAuthMethod?: "body" | "basic"; + tokenEndpoint: string; + tokenExtraHeaders?: Record; +} + +export interface PluginOAuthBearerCredentials { + apiHeaders?: Record; + authTokenEnv: string; + authTokenPlaceholder?: string; + domains: string[]; + type: "oauth-bearer"; +} + +export type PluginCredentials = PluginOAuthBearerCredentials; + +export interface PluginNpmRuntimeDependency { + package: string; + type: "npm"; + version: string; +} + +export interface PluginSystemRuntimeDependency { + package: string; + type: "system"; +} + +export interface PluginSystemRuntimeDependencyFromUrl { + sha256: string; + type: "system"; + url: string; +} + +export type PluginRuntimeDependency = + | PluginNpmRuntimeDependency + | PluginSystemRuntimeDependency + | PluginSystemRuntimeDependencyFromUrl; + +export interface PluginRuntimePostinstallCommand { + args?: string[]; + cmd: string; + sudo?: boolean; +} + +export interface PluginMcpConfig { + allowedTools?: string[]; + headers?: Record; + transport: "http"; + url: string; +} + +export interface PluginEnvVarDeclaration { + default?: string; + exposeToCommandEnv?: boolean; +} + +export interface PluginManifest { + apiHeaders?: Record; + capabilities?: string[]; + commandEnv?: Record; + configKeys?: string[]; + credentials?: PluginCredentials; + description: string; + displayName: string; + domains?: string[]; + envVars?: Record; + mcp?: PluginMcpConfig; + name: string; + oauth?: PluginOAuthConfig; + runtimeDependencies?: PluginRuntimeDependency[]; + runtimePostinstall?: PluginRuntimePostinstallCommand[]; + target?: { + commandFlags?: string[]; + configKey: string; + type: string; + }; +} diff --git a/packages/junior-plugin-api/src/operations.ts b/packages/junior-plugin-api/src/operations.ts new file mode 100644 index 000000000..bca229df7 --- /dev/null +++ b/packages/junior-plugin-api/src/operations.ts @@ -0,0 +1,126 @@ +import type { PluginContext } from "./context"; +import type { PluginDb } from "./database"; +import type { Dispatch, DispatchOptions, DispatchResult } from "./dispatch"; +import type { PluginReadState, PluginState } from "./state"; + +export type PluginConversationStatus = + | "active" + | "completed" + | "failed" + | "hung" + | "superseded"; + +export interface PluginConversationSummary { + channelName?: string; + conversationId: string; + displayTitle: string; + lastActivityAt: string; + lastUpdatedAt: string; + source?: "api" | "internal" | "local" | "plugin" | "scheduler" | "slack"; + status: PluginConversationStatus; +} + +export interface PluginConversations { + listRecent(options?: { + limit?: number; + }): Promise; +} + +export interface HeartbeatHookContext extends PluginContext { + agent: { + dispatch(options: DispatchOptions): Promise; + get(id: string): Promise; + }; + nowMs: number; + state: PluginState; +} + +export interface HeartbeatResult { + dispatchCount?: number; +} + +export interface StorageMigrationResult { + existing: number; + migrated: number; + missing: number; + scanned: number; + skipped?: number; +} + +export interface StorageMigrationContext extends PluginContext { + db: PluginDb; + state: PluginState; +} + +export type PluginOperationalTone = "danger" | "good" | "neutral" | "warning"; + +export interface PluginOperationalMetric { + label: string; + tone?: PluginOperationalTone; + value: string; +} + +export interface PluginOperationalField { + key: string; + label: string; +} + +export interface PluginOperationalRecord { + id: string; + tone?: PluginOperationalTone; + values: Record; +} + +export interface PluginOperationalRecordSet { + fields?: PluginOperationalField[]; + emptyText?: string; + records?: PluginOperationalRecord[]; + title: string; +} + +export interface PluginOperationalReportContent { + generatedAt?: string; + metrics?: PluginOperationalMetric[]; + recordSets?: PluginOperationalRecordSet[]; + title?: string; +} + +export interface PluginOperationalReport extends PluginOperationalReportContent { + pluginName: string; +} + +export interface OperationalReportHookContext extends PluginContext { + conversations: PluginConversations; + nowMs: number; + state: PluginReadState; +} + +export type PluginRouteMethod = + | "GET" + | "POST" + | "PUT" + | "PATCH" + | "DELETE" + | "HEAD" + | "OPTIONS" + | "ALL"; + +export type PluginRouteHandler = { + bivarianceHack(request: Request): Promise | Response; +}["bivarianceHack"]; + +export interface PluginRoute { + handler: PluginRouteHandler; + method?: PluginRouteMethod | PluginRouteMethod[]; + path: string; +} + +export interface RouteRegistrationHookContext extends PluginContext {} + +export interface SlackConversationLink { + url: string; +} + +export interface SlackConversationLinkHookContext extends PluginContext { + conversationId: string; +} diff --git a/packages/junior-plugin-api/src/prompt.ts b/packages/junior-plugin-api/src/prompt.ts new file mode 100644 index 000000000..93f07b133 --- /dev/null +++ b/packages/junior-plugin-api/src/prompt.ts @@ -0,0 +1,59 @@ +import type { InvocationContext, PluginContext } from "./context"; +import type { + PluginSessionState, + PluginSessionStateAppend, + PluginState, +} from "./state"; + +export interface UserPromptContribution { + id: string; + text: string; +} + +export interface UserPromptContributionResult { + contributions?: UserPromptContribution[]; + sessionState?: PluginSessionStateAppend[]; +} + +export type UserPromptHookContext = PluginContext & + InvocationContext & { + isFirstPrompt: boolean; + session: PluginSessionState; + state: PluginState; + userText: string; + }; + +export interface PluginTaskEnqueueOptions { + idempotencyKey: string; + name: string; + payload?: unknown; +} + +export interface PluginTaskEnqueueResult { + id: string; + status: "created" | "already_exists"; +} + +export interface PluginTaskQueue { + enqueue(options: PluginTaskEnqueueOptions): Promise; +} + +export type TurnObservationHookContext = PluginContext & + InvocationContext & { + observationId: string; + tasks: PluginTaskQueue; + }; + +export interface PluginTaskContext extends PluginContext { + id: string; + name: string; + observation?: { + load(): Promise; + }; + payload?: unknown; + state: PluginState; +} + +export type PluginTaskHandler = ( + ctx: PluginTaskContext, +) => Promise | void; diff --git a/packages/junior-plugin-api/src/registration.ts b/packages/junior-plugin-api/src/registration.ts new file mode 100644 index 000000000..3a5653b20 --- /dev/null +++ b/packages/junior-plugin-api/src/registration.ts @@ -0,0 +1,62 @@ +import type { PluginDatabaseConfig } from "./database"; +import type { PluginHooks } from "./hooks"; +import type { PluginManifest } from "./manifest"; + +export type PluginRegistrationInput = { + database?: PluginDatabaseConfig; + hooks?: PluginHooks; + manifest: PluginManifest; + packageName?: string; +}; + +export interface PluginRegistration extends PluginRegistrationInput {} + +const PLUGIN_NAME_RE = /^[a-z][a-z0-9-]*$/; + +/** Define one Junior plugin registration for app and build-time wiring. */ +export function defineJuniorPlugin( + plugin: PluginRegistrationInput, +): PluginRegistration { + if ("pluginConfig" in plugin) { + throw new Error( + "pluginConfig is no longer supported. Put runtime metadata in manifest or plugin registration fields.", + ); + } + if ("name" in plugin) { + throw new Error("defineJuniorPlugin() uses manifest.name for identity."); + } + const manifest = plugin.manifest; + if (!manifest) { + throw new Error( + "defineJuniorPlugin() requires a manifest. Use a package name string in defineJuniorPlugins([...]) for plugin.yaml packages.", + ); + } + const name = manifest.name; + if (!name) { + throw new Error("Junior plugin manifest.name is required."); + } + if (!PLUGIN_NAME_RE.test(name)) { + throw new Error( + `Junior plugin registration name "${name}" must be a lowercase plugin identifier.`, + ); + } + if ( + typeof manifest.displayName !== "string" || + !manifest.displayName.trim() + ) { + throw new Error( + `Junior plugin "${name}" manifest.displayName is required.`, + ); + } + if ( + typeof manifest.description !== "string" || + !manifest.description.trim() + ) { + throw new Error( + `Junior plugin "${name}" manifest.description is required.`, + ); + } + return { + ...plugin, + }; +} diff --git a/packages/junior-plugin-api/src/schemas.ts b/packages/junior-plugin-api/src/schemas.ts new file mode 100644 index 000000000..4da59baaa --- /dev/null +++ b/packages/junior-plugin-api/src/schemas.ts @@ -0,0 +1,146 @@ +import { z } from "zod"; + +const slackTeamIdSchema = z.string().regex(/^T[A-Z0-9]+$/); +const slackConversationIdSchema = z.string().regex(/^(C|G|D)[A-Z0-9]+$/); +const localConversationIdSchema = z + .string() + .regex(/^local:[a-z0-9_-]+:[a-z0-9][a-z0-9_-]*$/); +const exactActorUserIdSchema = z + .string() + .min(1) + .refine( + (value) => value === value.trim() && value.toLowerCase() !== "unknown", + ); + +export const nonBlankStringSchema = z + .string() + .refine((value) => value.trim().length > 0); + +/** Runtime-owned Slack address for routing future work or side effects. */ +export const slackDestinationSchema = z + .object({ + platform: z.literal("slack"), + teamId: slackTeamIdSchema, + channelId: slackConversationIdSchema, + }) + .strict(); + +/** Runtime-owned local CLI conversation address. */ +export const localDestinationSchema = z + .object({ + platform: z.literal("local"), + conversationId: localConversationIdSchema, + }) + .strict(); + +/** Runtime-owned provider-neutral address for routing future work or side effects. */ +export const destinationSchema = z.discriminatedUnion("platform", [ + slackDestinationSchema, + localDestinationSchema, +]); + +/** Runtime-owned Slack coordinates for the inbound invocation. */ +export const slackSourceSchema = z + .object({ + platform: z.literal("slack"), + teamId: slackTeamIdSchema, + channelId: slackConversationIdSchema, + messageTs: nonBlankStringSchema.optional(), + threadTs: nonBlankStringSchema.optional(), + }) + .strict(); + +/** Runtime-owned local CLI coordinates for the inbound invocation. */ +export const localSourceSchema = localDestinationSchema; + +/** Runtime-owned provider-neutral coordinates for the inbound invocation. */ +export const sourceSchema = z.discriminatedUnion("platform", [ + slackSourceSchema, + localSourceSchema, +]); + +/** Stable user credential subject shape accepted from plugins. */ +export const pluginCredentialSubjectSchema = z + .object({ + type: z.literal("user"), + userId: exactActorUserIdSchema, + allowedWhen: z.literal("private-direct-conversation"), + }) + .strict(); + +/** Shared exact actor profile fields for platform-scoped requesters. */ +const requesterProfileSchema = { + email: nonBlankStringSchema.optional(), + fullName: nonBlankStringSchema.optional(), + userId: exactActorUserIdSchema, + userName: nonBlankStringSchema.optional(), +}; + +export const slackRequesterSchema = z + .object({ + ...requesterProfileSchema, + platform: z.literal("slack"), + teamId: slackTeamIdSchema, + }) + .strict(); + +export const localRequesterSchema = z + .object({ + ...requesterProfileSchema, + platform: z.literal("local"), + }) + .strict(); + +/** Runtime-provided requester identity visible to plugin hooks. */ +export const requesterSchema = z.discriminatedUnion("platform", [ + slackRequesterSchema, + localRequesterSchema, +]); + +const dispatchMetadataSchema = z + .record(z.string(), z.string()) + .superRefine((metadata, ctx) => { + const entries = Object.entries(metadata); + if (entries.length > 20) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: "Dispatch metadata has too many keys", + }); + return; + } + for (const [key, value] of entries) { + if (!key.trim()) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: "Dispatch metadata values must be strings", + path: [key], + }); + continue; + } + if (key.length > 128) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: "Dispatch metadata key exceeds the maximum length", + path: [key], + }); + } + if (value.length > 512) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: "Dispatch metadata value exceeds the maximum length", + path: [key], + }); + } + } + }); + +/** Plugin dispatch request accepted by Junior core. */ +export const dispatchOptionsSchema = z + .object({ + idempotencyKey: nonBlankStringSchema.pipe(z.string().max(512)), + credentialSubject: pluginCredentialSubjectSchema.optional(), + destination: slackDestinationSchema, + input: nonBlankStringSchema.pipe(z.string().max(32_000)), + metadata: dispatchMetadataSchema.optional(), + }) + .strict(); diff --git a/packages/junior-plugin-api/src/state.ts b/packages/junior-plugin-api/src/state.ts new file mode 100644 index 000000000..20ccb8990 --- /dev/null +++ b/packages/junior-plugin-api/src/state.ts @@ -0,0 +1,26 @@ +export interface PluginState { + delete(key: string): Promise; + get(key: string): Promise; + set(key: string, value: unknown, ttlMs?: number): Promise; + setIfNotExists(key: string, value: unknown, ttlMs?: number): Promise; + withLock( + key: string, + ttlMs: number, + callback: () => Promise, + ): Promise; +} + +export interface PluginReadState { + get(key: string): Promise; +} + +export interface PluginSessionStateAppend { + key: string; + value: unknown; +} + +export interface PluginSessionState { + list( + key: string, + ): Promise>; +} diff --git a/packages/junior-plugin-api/src/tools.ts b/packages/junior-plugin-api/src/tools.ts new file mode 100644 index 000000000..355f873ec --- /dev/null +++ b/packages/junior-plugin-api/src/tools.ts @@ -0,0 +1,130 @@ +import type { + PluginContext, + LocalInvocationContext, + Requester, + SlackInvocationContext, +} from "./context"; +import type { PluginCredentialSubject } from "./credentials"; +import type { PluginState } from "./state"; + +export interface PluginEnv { + get(key: string): string | undefined; + set(key: string, value: string): void; +} + +export interface PluginDecision { + deny(message: string): void; + replaceInput(input: Record): void; +} + +/** Thrown when a plugin tool rejects invalid model or user input. */ +export class PluginToolInputError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "PluginToolInputError"; + } +} + +export interface PluginSandbox { + juniorRoot: string; + root: string; + readFile(path: string): Promise; + run(input: { + args?: string[]; + cmd: string; + cwd?: string; + env?: Record; + sudo?: boolean; + }): Promise<{ + exitCode: number; + stderr: string; + stdout: string; + }>; + writeFile(input: { + content: string | Uint8Array; + mode?: number; + path: string; + }): Promise; +} + +export interface SandboxPrepareHookContext extends PluginContext { + requester?: Requester; + sandbox: PluginSandbox; +} + +export interface BeforeToolExecuteHookContext extends PluginContext { + decision: PluginDecision; + env: PluginEnv; + requester?: Requester; + tool: { + input: Record; + name: string; + }; +} + +export type PluginToolExecute = { + bivarianceHack( + input: TInput, + options: { experimental_context?: unknown }, + ): Promise | unknown; +}["bivarianceHack"]; + +export interface PluginToolDefinition { + annotations?: unknown; + description: string; + executionMode?: unknown; + inputSchema: unknown; + prepareArguments?: (args: unknown) => unknown; + /** + * @deprecated Put tool-selection and usage guidance directly in `description` + * and parameter descriptions. Retained for compatibility; may be removed in a + * future major version. + */ + promptGuidelines?: string[]; + /** + * @deprecated Put tool-selection and usage guidance directly in `description` + * and parameter descriptions. Retained for compatibility; may be removed in a + * future major version. + */ + promptSnippet?: string; + execute?: PluginToolExecute; +} + +export interface SlackToolRegistrationHookContext { + /** + * Capabilities of the source Slack conversation exposed to this plugin. + * Recomputed from `source.channelId`, not from `destination`. + */ + channelCapabilities: { + canAddReactions: boolean; + canCreateCanvas: boolean; + canPostToChannel: boolean; + }; + credentialSubject?: PluginCredentialSubject; +} + +interface BaseToolRegistrationHookContext extends PluginContext { + /** + * Opaque Junior conversation/session identity for this turn. + * Interactive Slack turns use `slack:{channelId}:{threadTs}`. + * Scheduled/API turns use an internal id such as `agent-dispatch:{id}`. + * Do not parse as Slack unless the value starts with `slack:`. + */ + conversationId?: string; + state: PluginState; + userText?: string; +} + +interface SlackToolRegistrationContext + extends BaseToolRegistrationHookContext, SlackInvocationContext { + slack: SlackToolRegistrationHookContext; +} + +interface LocalToolRegistrationContext + extends BaseToolRegistrationHookContext, LocalInvocationContext { + slack?: never; +} + +export type ToolRegistrationHookContext = + | LocalToolRegistrationContext + | SlackToolRegistrationContext; diff --git a/packages/junior-scheduler/drizzle.config.ts b/packages/junior-scheduler/drizzle.config.ts new file mode 100644 index 000000000..e94859d53 --- /dev/null +++ b/packages/junior-scheduler/drizzle.config.ts @@ -0,0 +1,8 @@ +import { defineConfig } from "drizzle-kit"; + +export default defineConfig({ + dialect: "postgresql", + out: "./migrations", + schema: "./src/db/schema.ts", + strict: true, +}); diff --git a/packages/junior-scheduler/migrations/0001_scheduler.sql b/packages/junior-scheduler/migrations/0001_scheduler.sql new file mode 100644 index 000000000..5ac100b15 --- /dev/null +++ b/packages/junior-scheduler/migrations/0001_scheduler.sql @@ -0,0 +1,50 @@ +CREATE TABLE IF NOT EXISTS junior_scheduler_tasks ( + id TEXT PRIMARY KEY, + team_id TEXT NOT NULL, + status TEXT NOT NULL, + next_run_at_ms BIGINT, + run_now_at_ms BIGINT, + created_at_ms BIGINT NOT NULL, + updated_at_ms BIGINT NOT NULL, + version INTEGER NOT NULL, + destination JSONB NOT NULL, + created_by JSONB NOT NULL, + conversation_access JSONB, + credential_subject JSONB, + execution_actor JSONB, + last_run_at_ms BIGINT, + original_request TEXT, + schedule JSONB NOT NULL, + status_reason TEXT, + task JSONB NOT NULL, + record JSONB NOT NULL +); + +CREATE INDEX IF NOT EXISTS junior_scheduler_tasks_team_status_idx + ON junior_scheduler_tasks (team_id, status, created_at_ms); + +CREATE INDEX IF NOT EXISTS junior_scheduler_tasks_due_idx + ON junior_scheduler_tasks (status, run_now_at_ms, next_run_at_ms); + +CREATE TABLE IF NOT EXISTS junior_scheduler_runs ( + id TEXT PRIMARY KEY, + task_id TEXT NOT NULL, + status TEXT NOT NULL, + claimed_at_ms BIGINT NOT NULL, + scheduled_for_ms BIGINT NOT NULL, + started_at_ms BIGINT, + completed_at_ms BIGINT, + dispatch_id TEXT, + error_message TEXT, + idempotency_key TEXT NOT NULL, + result_message_ts TEXT, + task_version INTEGER NOT NULL, + attempt INTEGER NOT NULL, + record JSONB NOT NULL +); + +CREATE INDEX IF NOT EXISTS junior_scheduler_runs_task_status_idx + ON junior_scheduler_runs (task_id, status, scheduled_for_ms); + +CREATE INDEX IF NOT EXISTS junior_scheduler_runs_status_idx + ON junior_scheduler_runs (status, scheduled_for_ms); diff --git a/packages/junior-scheduler/package.json b/packages/junior-scheduler/package.json index 9ff23a52d..4a6a48eea 100644 --- a/packages/junior-scheduler/package.json +++ b/packages/junior-scheduler/package.json @@ -19,17 +19,21 @@ }, "files": [ "dist", + "migrations", "src" ], "scripts": { "build": "tsup && tsc -p tsconfig.build.json --emitDeclarationOnly", + "db:generate": "pnpm dlx drizzle-kit@0.31.10 generate --config drizzle.config.ts", "prepare": "pnpm run build", "prepack": "pnpm run build", "typecheck": "tsc --noEmit" }, "dependencies": { "@sentry/junior-plugin-api": "workspace:*", - "@sinclair/typebox": "^0.34.49" + "@sinclair/typebox": "^0.34.49", + "drizzle-orm": "catalog:", + "zod": "catalog:" }, "devDependencies": { "@types/node": "^25.9.1", diff --git a/packages/junior-scheduler/src/db/schema.ts b/packages/junior-scheduler/src/db/schema.ts new file mode 100644 index 000000000..e05e4d5a6 --- /dev/null +++ b/packages/junior-scheduler/src/db/schema.ts @@ -0,0 +1,77 @@ +import { + bigint, + index, + integer, + jsonb, + pgTable, + text, +} from "drizzle-orm/pg-core"; +import type { ScheduledRun, ScheduledTask } from "../types"; + +export const juniorSchedulerTasks = pgTable( + "junior_scheduler_tasks", + { + id: text("id").primaryKey(), + teamId: text("team_id").notNull(), + status: text("status").notNull(), + nextRunAtMs: bigint("next_run_at_ms", { mode: "number" }), + runNowAtMs: bigint("run_now_at_ms", { mode: "number" }), + createdAtMs: bigint("created_at_ms", { mode: "number" }).notNull(), + updatedAtMs: bigint("updated_at_ms", { mode: "number" }).notNull(), + version: integer("version").notNull(), + destination: jsonb("destination").notNull(), + createdBy: jsonb("created_by").notNull(), + conversationAccess: jsonb("conversation_access"), + credentialSubject: jsonb("credential_subject"), + executionActor: jsonb("execution_actor"), + lastRunAtMs: bigint("last_run_at_ms", { mode: "number" }), + originalRequest: text("original_request"), + schedule: jsonb("schedule").notNull(), + statusReason: text("status_reason"), + task: jsonb("task").notNull(), + record: jsonb("record").$type().notNull(), + }, + (table) => [ + index("junior_scheduler_tasks_team_status_idx").on( + table.teamId, + table.status, + table.createdAtMs, + ), + index("junior_scheduler_tasks_due_idx").on( + table.status, + table.runNowAtMs, + table.nextRunAtMs, + ), + ], +); + +export const juniorSchedulerRuns = pgTable( + "junior_scheduler_runs", + { + id: text("id").primaryKey(), + taskId: text("task_id").notNull(), + status: text("status").notNull(), + claimedAtMs: bigint("claimed_at_ms", { mode: "number" }).notNull(), + scheduledForMs: bigint("scheduled_for_ms", { mode: "number" }).notNull(), + startedAtMs: bigint("started_at_ms", { mode: "number" }), + completedAtMs: bigint("completed_at_ms", { mode: "number" }), + dispatchId: text("dispatch_id"), + errorMessage: text("error_message"), + idempotencyKey: text("idempotency_key").notNull(), + resultMessageTs: text("result_message_ts"), + taskVersion: integer("task_version").notNull(), + attempt: integer("attempt").notNull(), + record: jsonb("record").$type().notNull(), + }, + (table) => [ + index("junior_scheduler_runs_task_status_idx").on( + table.taskId, + table.status, + table.scheduledForMs, + ), + index("junior_scheduler_runs_status_idx").on( + table.status, + table.scheduledForMs, + ), + ], +); diff --git a/packages/junior-scheduler/src/index.ts b/packages/junior-scheduler/src/index.ts index ff3806c1a..ef9def1aa 100644 --- a/packages/junior-scheduler/src/index.ts +++ b/packages/junior-scheduler/src/index.ts @@ -8,7 +8,11 @@ export { createSlackScheduleUpdateTaskTool, type SchedulerToolContext, } from "./schedule-tools"; -export { createSchedulerStore } from "./store"; +export { + createSchedulerOperationalSqlStore, + createSchedulerSqlStore, + migrateSchedulerStateToSql, +} from "./store"; export type { ScheduledCalendarFrequency, ScheduledLocalTime, diff --git a/packages/junior-scheduler/src/plugin.ts b/packages/junior-scheduler/src/plugin.ts index 3dd5ad2a1..860e946c3 100644 --- a/packages/junior-scheduler/src/plugin.ts +++ b/packages/junior-scheduler/src/plugin.ts @@ -1,15 +1,19 @@ import { defineJuniorPlugin, type Dispatch, - type AgentPluginToolDefinition, + type PluginDb, + type PluginToolDefinition, type PluginOperationalReportContent, + type PluginReadState, + type PluginState, type SlackDestination, type ToolRegistrationHookContext, } from "@sentry/junior-plugin-api"; import { buildScheduledTaskRunPrompt } from "./prompt"; import { - createSchedulerOperationalStore, - createSchedulerStore, + createSchedulerOperationalSqlStore, + createSchedulerSqlStore, + migrateSchedulerStateToSql, type SchedulerOperationalStore, type SchedulerStore, } from "./store"; @@ -31,6 +35,22 @@ import { const SCHEDULER_HEARTBEAT_LIMIT = 10; const DASHBOARD_TABLE_LIMIT = 5; +function schedulerStore(ctx: { db?: PluginDb }): SchedulerStore { + if (!ctx.db) { + throw new Error("Scheduler plugin requires ctx.db"); + } + return createSchedulerSqlStore(ctx.db); +} + +function schedulerOperationalStore(ctx: { + db?: PluginDb; +}): SchedulerOperationalStore { + if (!ctx.db) { + throw new Error("Scheduler plugin requires ctx.db"); + } + return createSchedulerOperationalSqlStore(ctx.db); +} + function shouldSkipRun( task: ScheduledTask, run: ScheduledRun, @@ -64,7 +84,7 @@ function createSchedulerToolContext( } : undefined, requester: ctx.requester?.platform === "slack" ? ctx.requester : undefined, - state: ctx.state, + store: schedulerStore(ctx), userText: ctx.userText, }; } @@ -73,7 +93,7 @@ async function applyDispatchResult(args: { dispatch: Dispatch; nowMs: number; run: ScheduledRun; - store: ReturnType; + store: SchedulerStore; }): Promise { if (args.dispatch.status === "completed") { const completed = await args.store.markRunCompleted({ @@ -362,19 +382,20 @@ async function buildSchedulerOperationalReport(args: { /** Create Junior's built-in trusted scheduler plugin. */ export function createSchedulerPlugin() { return defineJuniorPlugin({ + database: {}, manifest: { name: "scheduler", displayName: "Scheduler", description: "Scheduled Junior task management and heartbeat dispatch", }, - legacyStatePrefixes: ["junior:scheduler"], + packageName: "@sentry/junior-scheduler", hooks: { tools(ctx) { if ( ctx.source.platform !== "slack" || ctx.requester?.platform !== "slack" ) { - return {} as Record>; + return {} as Record>; } const context = createSchedulerToolContext(ctx); return { @@ -383,10 +404,10 @@ export function createSchedulerPlugin() { slackScheduleUpdateTask: createSlackScheduleUpdateTaskTool(context), slackScheduleDeleteTask: createSlackScheduleDeleteTaskTool(context), slackScheduleRunTaskNow: createSlackScheduleRunTaskNowTool(context), - } satisfies Record>; + } satisfies Record>; }, async heartbeat(ctx) { - const store = createSchedulerStore(ctx.state); + const store = schedulerStore(ctx); let processedCount = 0; let dispatchCount = 0; for (const run of await store.listIncompleteRuns()) { @@ -504,7 +525,16 @@ export function createSchedulerPlugin() { async operationalReport(ctx) { return buildSchedulerOperationalReport({ nowMs: ctx.nowMs, - store: createSchedulerOperationalStore(ctx.state), + store: schedulerOperationalStore(ctx), + }); + }, + async migrateStorage(ctx) { + if (!ctx.db) { + throw new Error("Scheduler storage migration requires ctx.db"); + } + return await migrateSchedulerStateToSql({ + db: ctx.db, + state: ctx.state, }); }, }, diff --git a/packages/junior-scheduler/src/schedule-tools.ts b/packages/junior-scheduler/src/schedule-tools.ts index 72d52cad6..11b9a7fbf 100644 --- a/packages/junior-scheduler/src/schedule-tools.ts +++ b/packages/junior-scheduler/src/schedule-tools.ts @@ -1,19 +1,18 @@ import { randomUUID } from "node:crypto"; import { Type } from "@sinclair/typebox"; import { - AgentPluginToolInputError, - agentPluginCredentialSubjectSchema, + PluginToolInputError, + pluginCredentialSubjectSchema, destinationSchema, isSlackDestination, - type AgentPluginCredentialSubject, - type AgentPluginState, - type AgentPluginToolDefinition, + type PluginCredentialSubject, + type PluginToolDefinition, type SlackDestination, type SlackRequester, } from "@sentry/junior-plugin-api"; import { buildCalendarRecurrence, parseScheduleTimestamp } from "./cadence"; import { sanitizeScheduledTaskPrincipal } from "./identity"; -import { createSchedulerStore } from "./store"; +import { type SchedulerStore } from "./store"; import { SCHEDULED_TASK_SYSTEM_ACTOR } from "./types"; import type { ScheduledCalendarFrequency, @@ -25,10 +24,10 @@ import type { } from "./types"; export interface SchedulerToolContext { - credentialSubject?: AgentPluginCredentialSubject; + credentialSubject?: PluginCredentialSubject; requester?: SlackRequester; source?: SlackDestination; - state: AgentPluginState; + store: SchedulerStore; userText?: string; } @@ -42,7 +41,7 @@ type SchemaIssue = { }; function throwToolInputError(error: string): never { - throw new AgentPluginToolInputError(error); + throw new PluginToolInputError(error); } function requireActiveConversation( @@ -99,8 +98,8 @@ function requireRequester( } function tool( - definition: AgentPluginToolDefinition, -): AgentPluginToolDefinition { + definition: PluginToolDefinition, +): PluginToolDefinition { return definition; } @@ -125,8 +124,8 @@ function getConversationAccess( function getCredentialSubject(args: { access: ScheduledTaskConversationAccess; - subject: AgentPluginCredentialSubject | undefined; -}): AgentPluginCredentialSubject | undefined { + subject: PluginCredentialSubject | undefined; +}): PluginCredentialSubject | undefined { if ( args.access.audience !== "direct" || args.access.visibility !== "private" @@ -136,7 +135,7 @@ function getCredentialSubject(args: { if (!args.subject) { return undefined; } - const subject = agentPluginCredentialSubjectSchema.safeParse(args.subject); + const subject = pluginCredentialSubjectSchema.safeParse(args.subject); if (!subject.success) { throwToolInputError("Active Slack credential subject is invalid."); } @@ -165,9 +164,7 @@ async function getWritableTask(args: { }): Promise { const destination = requireActiveConversation(args.context); - const task = await createSchedulerStore(args.context.state).getTask( - args.taskId, - ); + const task = await schedulerStore(args.context).getTask(args.taskId); if (!task || task.status === "deleted") { throwToolInputError( "Scheduled task was not found in the active Slack conversation.", @@ -224,6 +221,10 @@ function buildTaskId(): string { return `${TASK_ID_PREFIX}_${randomUUID()}`; } +function schedulerStore(context: SchedulerToolContext): SchedulerStore { + return context.store; +} + function normalizeStatus( value: string | undefined, ): ScheduledTaskStatus | undefined { @@ -427,7 +428,7 @@ export function createSlackScheduleCreateTaskTool( version: 1, }; - await createSchedulerStore(context.state).saveTask(task); + await schedulerStore(context).saveTask(task); return { ok: true, task: compactTask(task), @@ -448,7 +449,7 @@ export function createSlackScheduleListTasksTool( execute: async () => { const destination = requireActiveConversation(context); - const tasks = await createSchedulerStore(context.state).listTasksForTeam( + const tasks = await schedulerStore(context).listTasksForTeam( destination.teamId, ); const matching = tasks.filter((task) => @@ -574,7 +575,7 @@ export function createSlackScheduleUpdateTaskTool( version: lookup.version + 1, }; - await createSchedulerStore(context.state).saveTask(next); + await schedulerStore(context).saveTask(next); return { ok: true, task: compactTask(next), @@ -610,7 +611,7 @@ export function createSlackScheduleDeleteTaskTool( version: lookup.version + 1, }; - await createSchedulerStore(context.state).saveTask(next); + await schedulerStore(context).saveTask(next); return { ok: true, task: compactTask(next), @@ -650,7 +651,7 @@ export function createSlackScheduleRunTaskNowTool( version: lookup.version + 1, }; - await createSchedulerStore(context.state).saveTask(next); + await schedulerStore(context).saveTask(next); return { ok: true, task: compactTask(next), diff --git a/packages/junior-scheduler/src/store.ts b/packages/junior-scheduler/src/store.ts index 412cbc440..f44199897 100644 --- a/packages/junior-scheduler/src/store.ts +++ b/packages/junior-scheduler/src/store.ts @@ -1,10 +1,12 @@ import { - agentPluginCredentialSubjectSchema, + pluginCredentialSubjectSchema, destinationSchema, isSlackDestination, - type AgentPluginReadState, - type AgentPluginState, + type PluginDb, + type PluginReadState, + type PluginState, } from "@sentry/junior-plugin-api"; +import { z } from "zod"; import { getNextRunAtMs } from "./cadence"; import type { ScheduledRun, ScheduledTask } from "./types"; @@ -15,6 +17,101 @@ const CLAIM_TTL_MS = 6 * 60 * 60 * 1000; const PENDING_CLAIM_STALE_MS = 60_000; const MISSED_RUN_MAX_AGE_MS = 24 * 60 * 60 * 1000; const LOCK_TTL_MS = 10_000; +const SQL_INCOMPLETE_RUN_STATUSES = ["pending", "running"] as const; +const slackDestinationSchema = destinationSchema.refine(isSlackDestination); +const taskPrincipalSchema = z + .object({ + slackUserId: z.string(), + fullName: z.string().optional(), + userName: z.string().optional(), + }) + .strict(); +const recurrenceSchema = z + .object({ + dayOfMonth: z.number().optional(), + frequency: z.enum(["daily", "weekly", "monthly", "yearly"]), + interval: z.number(), + month: z.number().optional(), + startDate: z.string(), + time: z + .object({ + hour: z.number(), + minute: z.number(), + }) + .strict(), + weekdays: z.array(z.number()).optional(), + }) + .strict(); +const taskScheduleSchema = z + .object({ + description: z.string(), + kind: z.enum(["one_off", "recurring"]), + recurrence: recurrenceSchema.optional(), + timezone: z.string(), + }) + .strict(); +const taskSpecSchema = z + .object({ + text: z.string(), + }) + .strict(); +const taskRecordSchema = z + .object({ + id: z.string(), + conversationAccess: z + .object({ + audience: z.enum(["direct", "group", "channel"]), + visibility: z.enum(["private", "public", "unknown"]), + }) + .strict() + .optional(), + createdAtMs: z.number(), + createdBy: taskPrincipalSchema, + credentialSubject: pluginCredentialSubjectSchema.optional(), + destination: slackDestinationSchema, + executionActor: z + .object({ + type: z.literal("system"), + id: z.string(), + }) + .strict() + .optional(), + lastRunAtMs: z.number().optional(), + nextRunAtMs: z.number().optional(), + originalRequest: z.string().optional(), + runNowAtMs: z.number().optional(), + schedule: taskScheduleSchema, + status: z.enum(["active", "paused", "blocked", "deleted"]), + statusReason: z.string().optional(), + task: taskSpecSchema, + updatedAtMs: z.number(), + version: z.number(), + }) + .strict(); +const runRecordSchema = z + .object({ + id: z.string(), + attempt: z.number(), + claimedAtMs: z.number(), + completedAtMs: z.number().optional(), + dispatchId: z.string().optional(), + errorMessage: z.string().optional(), + idempotencyKey: z.string(), + resultMessageTs: z.string().optional(), + scheduledForMs: z.number(), + startedAtMs: z.number().optional(), + status: z.enum([ + "pending", + "running", + "completed", + "failed", + "blocked", + "skipped", + ]), + taskId: z.string(), + taskVersion: z.number(), + }) + .strict(); export interface SchedulerStore { claimDueRun(args: { nowMs: number }): Promise; @@ -107,7 +204,7 @@ function unique(values: string[]): string[] { } async function withLock( - state: AgentPluginState, + state: PluginState, key: string, callback: () => Promise, ): Promise { @@ -115,7 +212,7 @@ async function withLock( } async function addToIndex( - state: AgentPluginState, + state: PluginState, key: string, taskId: string, ): Promise { @@ -128,7 +225,7 @@ async function addToIndex( } async function removeFromIndex( - state: AgentPluginState, + state: PluginState, key: string, taskId: string, ): Promise { @@ -151,7 +248,7 @@ async function removeFromIndex( } async function getIndex( - state: AgentPluginReadState, + state: PluginReadState, key: string, ): Promise { const values = (await state.get(key)) ?? []; @@ -161,7 +258,7 @@ async function getIndex( } async function clearActiveRun( - state: AgentPluginState, + state: PluginState, taskId: string, runId: string, ): Promise { @@ -174,7 +271,7 @@ async function clearActiveRun( } async function clearStaleActiveRun( - state: AgentPluginState, + state: PluginState, taskId: string, nowMs: number, ): Promise { @@ -188,8 +285,7 @@ async function clearStaleActiveRun( return true; } - const activeRun = - (await state.get(runKey(active.runId))) ?? undefined; + const activeRun = parseStoredRun(await state.get(runKey(active.runId))); if (!isStaleActiveRun(active, activeRun, nowMs)) { return false; } @@ -358,27 +454,34 @@ function canFinishRun( return run.status === "running" && run.startedAtMs === startedAtMs; } +/** Decode retained scheduler task state, skipping invalid legacy records. */ function parseStoredTask(value: unknown): ScheduledTask | undefined { - if (!value || typeof value !== "object") { - return undefined; - } - const record = value as Partial; - const destination = destinationSchema.safeParse(record.destination); - if (!destination.success || !isSlackDestination(destination.data)) { - return undefined; + const parsed = taskRecordSchema.safeParse(parseJsonRecord(value)); + return parsed.success ? parsed.data : undefined; +} + +/** Decode retained scheduler run state, skipping invalid legacy records. */ +function parseStoredRun(value: unknown): ScheduledRun | undefined { + const parsed = runRecordSchema.safeParse(parseJsonRecord(value)); + return parsed.success ? parsed.data : undefined; +} + +function parseJsonRecord(value: unknown): T | undefined { + if (typeof value === "string") { + try { + return JSON.parse(value) as T; + } catch { + return undefined; + } } - const credentialSubject = - record.credentialSubject === undefined - ? undefined - : agentPluginCredentialSubjectSchema.safeParse(record.credentialSubject); - if (credentialSubject && !credentialSubject.success) { - return undefined; + if (value && typeof value === "object") { + return value as T; } - return { - ...(record as ScheduledTask), - destination: destination.data, - ...(credentialSubject ? { credentialSubject: credentialSubject.data } : {}), - }; + return undefined; +} + +function present(value: T | undefined): value is T { + return value !== undefined; } function requireStoredTask(task: ScheduledTask): ScheduledTask { @@ -390,14 +493,14 @@ function requireStoredTask(task: ScheduledTask): ScheduledTask { } async function getTaskFromState( - state: AgentPluginReadState, + state: PluginReadState, taskId: string, ): Promise { return parseStoredTask(await state.get(taskKey(taskId))); } async function listTasksFromState( - state: AgentPluginReadState, + state: PluginReadState, indexKey: string, ): Promise { const ids = await getIndex(state, indexKey); @@ -409,14 +512,14 @@ async function listTasksFromState( } async function getRunFromState( - state: AgentPluginReadState, + state: PluginReadState, runId: string, ): Promise { - return (await state.get(runKey(runId))) ?? undefined; + return parseStoredRun(await state.get(runKey(runId))); } async function listIncompleteRunsForTasksFromState( - state: AgentPluginReadState, + state: PluginReadState, tasks: ScheduledTask[], ): Promise { const runs: ScheduledRun[] = []; @@ -434,9 +537,9 @@ async function listIncompleteRunsForTasksFromState( } class PluginStateSchedulerOperationalStore implements SchedulerOperationalStore { - private readonly state: AgentPluginReadState; + private readonly state: PluginReadState; - constructor(state: AgentPluginReadState) { + constructor(state: PluginReadState) { this.state = state; } @@ -452,9 +555,9 @@ class PluginStateSchedulerOperationalStore implements SchedulerOperationalStore } class PluginStateSchedulerStore implements SchedulerStore { - private readonly state: AgentPluginState; + private readonly state: PluginState; - constructor(state: AgentPluginState) { + constructor(state: PluginState) { this.state = state; } @@ -903,13 +1006,747 @@ class PluginStateSchedulerStore implements SchedulerStore { } /** Create a scheduler store backed by this plugin's durable state namespace. */ -export function createSchedulerStore(state: AgentPluginState): SchedulerStore { +export function createSchedulerStore(state: PluginState): SchedulerStore { return new PluginStateSchedulerStore(state); } /** Create a read-only scheduler store for operational reporting. */ export function createSchedulerOperationalStore( - state: AgentPluginReadState, + state: PluginReadState, ): SchedulerOperationalStore { return new PluginStateSchedulerOperationalStore(state); } + +type SchedulerTaskRow = { + record: unknown; +}; + +type SchedulerRunRow = { + record: unknown; +}; + +/** Decode scheduler SQL task records and reject rows unsafe for scan paths. */ +function parseSqlTaskRecord(value: unknown): ScheduledTask | undefined { + const parsed = taskRecordSchema.safeParse(parseJsonRecord(value)); + return parsed.success ? parsed.data : undefined; +} + +function parseSqlTaskRow(row: SchedulerTaskRow): ScheduledTask | undefined { + return parseSqlTaskRecord(row.record); +} + +/** Decode scheduler SQL run records and reject rows unsafe for scan paths. */ +function parseSqlRunRow(row: SchedulerRunRow): ScheduledRun | undefined { + const parsed = runRecordSchema.safeParse(parseJsonRecord(row.record)); + return parsed.success ? parsed.data : undefined; +} + +function json(value: unknown): string { + return JSON.stringify(value); +} + +async function withSqlLock( + db: PluginDb, + key: string, + callback: (db: PluginDb) => Promise, +): Promise { + return await db.transaction(async (tx) => { + await tx.execute("SELECT pg_advisory_xact_lock(hashtext($1))", [key]); + return await callback(tx); + }); +} + +async function upsertSqlTask(db: PluginDb, task: ScheduledTask): Promise { + await db.execute( + ` +INSERT INTO junior_scheduler_tasks ( + id, + team_id, + status, + next_run_at_ms, + run_now_at_ms, + created_at_ms, + updated_at_ms, + version, + destination, + created_by, + conversation_access, + credential_subject, + execution_actor, + last_run_at_ms, + original_request, + schedule, + status_reason, + task, + record +) VALUES ( + $1, $2, $3, $4, $5, $6, $7, $8, + $9::jsonb, $10::jsonb, $11::jsonb, $12::jsonb, $13::jsonb, + $14, $15, $16::jsonb, $17, $18::jsonb, $19::jsonb +) +ON CONFLICT (id) DO UPDATE SET + team_id = EXCLUDED.team_id, + status = EXCLUDED.status, + next_run_at_ms = EXCLUDED.next_run_at_ms, + run_now_at_ms = EXCLUDED.run_now_at_ms, + created_at_ms = EXCLUDED.created_at_ms, + updated_at_ms = EXCLUDED.updated_at_ms, + version = EXCLUDED.version, + destination = EXCLUDED.destination, + created_by = EXCLUDED.created_by, + conversation_access = EXCLUDED.conversation_access, + credential_subject = EXCLUDED.credential_subject, + execution_actor = EXCLUDED.execution_actor, + last_run_at_ms = EXCLUDED.last_run_at_ms, + original_request = EXCLUDED.original_request, + schedule = EXCLUDED.schedule, + status_reason = EXCLUDED.status_reason, + task = EXCLUDED.task, + record = EXCLUDED.record +`, + [ + task.id, + task.destination.teamId, + task.status, + task.nextRunAtMs ?? null, + task.runNowAtMs ?? null, + task.createdAtMs, + task.updatedAtMs, + task.version, + json(task.destination), + json(task.createdBy), + task.conversationAccess ? json(task.conversationAccess) : null, + task.credentialSubject ? json(task.credentialSubject) : null, + task.executionActor ? json(task.executionActor) : null, + task.lastRunAtMs ?? null, + task.originalRequest ?? null, + json(task.schedule), + task.statusReason ?? null, + json(task.task), + json(task), + ], + ); +} + +async function upsertSqlRun(db: PluginDb, run: ScheduledRun): Promise { + await db.execute( + ` +INSERT INTO junior_scheduler_runs ( + id, + task_id, + status, + claimed_at_ms, + scheduled_for_ms, + started_at_ms, + completed_at_ms, + dispatch_id, + error_message, + idempotency_key, + result_message_ts, + task_version, + attempt, + record +) VALUES ( + $1, $2, $3, $4, $5, $6, $7, $8, + $9, $10, $11, $12, $13, $14::jsonb +) +ON CONFLICT (id) DO UPDATE SET + task_id = EXCLUDED.task_id, + status = EXCLUDED.status, + claimed_at_ms = EXCLUDED.claimed_at_ms, + scheduled_for_ms = EXCLUDED.scheduled_for_ms, + started_at_ms = EXCLUDED.started_at_ms, + completed_at_ms = EXCLUDED.completed_at_ms, + dispatch_id = EXCLUDED.dispatch_id, + error_message = EXCLUDED.error_message, + idempotency_key = EXCLUDED.idempotency_key, + result_message_ts = EXCLUDED.result_message_ts, + task_version = EXCLUDED.task_version, + attempt = EXCLUDED.attempt, + record = EXCLUDED.record +`, + [ + run.id, + run.taskId, + run.status, + run.claimedAtMs, + run.scheduledForMs, + run.startedAtMs ?? null, + run.completedAtMs ?? null, + run.dispatchId ?? null, + run.errorMessage ?? null, + run.idempotencyKey, + run.resultMessageTs ?? null, + run.taskVersion, + run.attempt, + json(run), + ], + ); +} + +async function getTaskFromSql( + db: PluginDb, + taskId: string, +): Promise { + const rows = await db.query( + "SELECT record FROM junior_scheduler_tasks WHERE id = $1", + [taskId], + ); + return rows[0] ? parseSqlTaskRow(rows[0]) : undefined; +} + +async function getRunFromSql( + db: PluginDb, + runId: string, +): Promise { + const rows = await db.query( + "SELECT record FROM junior_scheduler_runs WHERE id = $1", + [runId], + ); + return rows[0] ? parseSqlRunRow(rows[0]) : undefined; +} + +async function getClaimedRunSlotFromSql( + db: PluginDb, + runId: string, +): Promise { + const rows = await db.query( + "SELECT record FROM junior_scheduler_runs WHERE id = $1", + [runId], + ); + return rows[0] ? parseSqlRunRow(rows[0]) : undefined; +} + +async function listTasksFromSql(db: PluginDb): Promise { + const rows = await db.query( + ` +SELECT record +FROM junior_scheduler_tasks +WHERE status <> 'deleted' +ORDER BY created_at_ms ASC, id ASC +`, + ); + return rows.map(parseSqlTaskRow).filter(present); +} + +async function listTasksForTeamFromSql( + db: PluginDb, + teamId: string, +): Promise { + const rows = await db.query( + ` +SELECT record +FROM junior_scheduler_tasks +WHERE team_id = $1 + AND status <> 'deleted' +ORDER BY created_at_ms ASC, id ASC +`, + [teamId], + ); + return rows.map(parseSqlTaskRow).filter(present); +} + +async function listIncompleteRunsForTasksFromSql( + db: PluginDb, + tasks: ScheduledTask[], +): Promise { + if (tasks.length === 0) { + return []; + } + const rows = await db.query( + ` +SELECT record +FROM junior_scheduler_runs +WHERE task_id = ANY($1) + AND status = ANY($2) +ORDER BY scheduled_for_ms ASC, id ASC +`, + [tasks.map((task) => task.id), [...SQL_INCOMPLETE_RUN_STATUSES]], + ); + return rows.map(parseSqlRunRow).filter(present); +} + +class SqlSchedulerStore implements SchedulerStore, SchedulerOperationalStore { + constructor(private readonly db: PluginDb) {} + + async saveTask(task: ScheduledTask): Promise { + const next = requireStoredTask(task); + await withSqlLock(this.db, taskLockKey(task.id), async (db) => { + const current = await getTaskFromSql(db, task.id); + await this.saveTaskRecord(db, next, current); + }); + } + + private async saveTaskRecord( + db: PluginDb, + task: ScheduledTask, + current: ScheduledTask | undefined, + ): Promise { + // Reactivation intentionally forgets the blocked slot so authorization or + // configuration fixes can dispatch the same scheduled occurrence again. + if ( + current?.status === "blocked" && + task.status === "active" && + typeof task.nextRunAtMs === "number" && + Number.isFinite(task.nextRunAtMs) + ) { + await db.execute( + "DELETE FROM junior_scheduler_runs WHERE id = $1 AND status = 'blocked'", + [buildRunId(task.id, task.nextRunAtMs)], + ); + } + await upsertSqlTask(db, task); + } + + async getTask(taskId: string): Promise { + return await getTaskFromSql(this.db, taskId); + } + + async listTasks(): Promise { + return await listTasksFromSql(this.db); + } + + async listTasksForTeam(teamId: string): Promise { + return await listTasksForTeamFromSql(this.db, teamId); + } + + async claimDueRun(args: { + nowMs: number; + }): Promise { + return await withSqlLock(this.db, "junior:scheduler:claim", async (db) => { + const rows = await db.query( + ` +SELECT record +FROM junior_scheduler_tasks +WHERE status = 'active' + AND ( + (run_now_at_ms IS NOT NULL AND run_now_at_ms <= $1) + OR (next_run_at_ms IS NOT NULL AND next_run_at_ms <= $1) + ) +ORDER BY created_at_ms ASC, id ASC +`, + [args.nowMs], + ); + + for (const row of rows) { + const task = parseSqlTaskRow(row); + if (!task) { + continue; + } + const scheduledForMs = getDueRunAtMs(task, args.nowMs); + if (scheduledForMs === undefined) { + continue; + } + const runId = buildRunId(task.id, scheduledForMs); + const incompleteRuns = await listIncompleteRunsForTasksFromSql(db, [ + task, + ]); + const incompleteRun = incompleteRuns.find((run) => run.id === runId); + const blockingRun = incompleteRuns.find( + (run) => run.id !== runId && !isStalePendingRun(run, args.nowMs), + ); + if (blockingRun) { + continue; + } + if (incompleteRun) { + if (!isStalePendingRun(incompleteRun, args.nowMs)) { + continue; + } + const reclaimed = { + ...incompleteRun, + attempt: incompleteRun.attempt + 1, + claimedAtMs: args.nowMs, + }; + await upsertSqlRun(db, reclaimed); + return reclaimed; + } + const existingRun = await getClaimedRunSlotFromSql(db, runId); + if (existingRun) { + continue; + } + + if (isMissedRunTooOld({ nowMs: args.nowMs, scheduledForMs })) { + await this.skipMissedRun(db, { + nowMs: args.nowMs, + scheduledForMs, + task, + }); + continue; + } + + const run = buildScheduledRun({ + claimedAtMs: args.nowMs, + scheduledForMs, + task, + }); + await upsertSqlRun(db, run); + return run; + } + + return undefined; + }); + } + + private async skipMissedRun( + db: PluginDb, + args: { + nowMs: number; + scheduledForMs: number; + task: ScheduledTask; + }, + ): Promise { + const current = await getTaskFromSql(db, args.task.id); + if ( + !current || + current.status !== "active" || + getDueRunAtMs(current, args.nowMs) !== args.scheduledForMs + ) { + return; + } + + const duplicateOf = await this.findStaleRecoveryCanonicalTask(db, current); + const errorMessage = duplicateOf + ? `Duplicate stale scheduled task was skipped without dispatch. Canonical task: ${duplicateOf.id}.` + : "Scheduled occurrence was more than 24 hours late and was skipped without dispatch."; + await upsertSqlRun( + db, + buildSkippedScheduledRun({ + completedAtMs: args.nowMs, + errorMessage, + scheduledForMs: args.scheduledForMs, + task: current, + }), + ); + + const isRunNow = current.runNowAtMs === args.scheduledForMs; + let nextRunAtMs: number | undefined; + if (!duplicateOf) { + nextRunAtMs = + isRunNow && current.nextRunAtMs !== args.scheduledForMs + ? current.nextRunAtMs + : current.schedule.kind === "recurring" + ? getNextRunAtMs(current, args.scheduledForMs, args.nowMs) + : undefined; + } + const nextStatus = nextRunAtMs ? "active" : "paused"; + + await this.saveTaskRecord( + db, + { + ...current, + nextRunAtMs, + runNowAtMs: isRunNow ? undefined : current.runNowAtMs, + status: nextStatus, + statusReason: nextStatus === "paused" ? errorMessage : undefined, + updatedAtMs: args.nowMs, + version: current.version + 1, + }, + current, + ); + } + + private async findStaleRecoveryCanonicalTask( + db: PluginDb, + task: ScheduledTask, + ): Promise { + const fingerprint = taskDedupeFingerprint(task); + const tasks = await listTasksForTeamFromSql(db, task.destination.teamId); + return tasks + .filter((candidate) => candidate.id !== task.id) + .filter( + (candidate) => + candidate.status === "active" && + isEarlierTask(candidate, task) && + taskDedupeFingerprint(candidate) === fingerprint, + ) + .sort((a, b) => a.createdAtMs - b.createdAtMs || a.id.localeCompare(b.id)) + .at(0); + } + + async getRun(runId: string): Promise { + return await getRunFromSql(this.db, runId); + } + + async listIncompleteRuns(): Promise { + return await listIncompleteRunsForTasksFromSql( + this.db, + await this.listTasks(), + ); + } + + async listIncompleteRunsForTasks( + tasks: ScheduledTask[], + ): Promise { + return await listIncompleteRunsForTasksFromSql(this.db, tasks); + } + + async markRunDispatched(args: { + claimedAtMs: number; + dispatchId: string; + nowMs: number; + runId: string; + }): Promise { + return await this.updateRun(args.runId, (run) => + run.status === "pending" && run.claimedAtMs === args.claimedAtMs + ? { + ...run, + dispatchId: args.dispatchId, + startedAtMs: args.nowMs, + status: "running", + } + : undefined, + ); + } + + async markRunCompleted(args: { + completedAtMs: number; + resultMessageTs?: string; + runId: string; + startedAtMs: number; + }): Promise { + const next = await this.updateRun(args.runId, (run) => + canFinishRun(run, args.startedAtMs) + ? { + ...run, + completedAtMs: args.completedAtMs, + resultMessageTs: args.resultMessageTs, + status: "completed", + } + : undefined, + ); + return next; + } + + async markRunFailed(args: { + completedAtMs: number; + errorMessage: string; + startedAtMs?: number; + runId: string; + }): Promise { + return await this.updateRun(args.runId, (run) => + canFinishRun(run, args.startedAtMs) + ? { + ...run, + completedAtMs: args.completedAtMs, + errorMessage: args.errorMessage, + status: "failed", + } + : undefined, + ); + } + + async markRunSkipped(args: { + completedAtMs: number; + errorMessage: string; + runId: string; + }): Promise { + return await this.updateRun(args.runId, (run) => + run.status === "pending" + ? { + ...run, + completedAtMs: args.completedAtMs, + errorMessage: args.errorMessage, + status: "skipped", + } + : undefined, + ); + } + + async markRunBlocked(args: { + completedAtMs: number; + errorMessage: string; + runId: string; + startedAtMs?: number; + }): Promise { + return await this.updateRun(args.runId, (run) => + canFinishRun(run, args.startedAtMs) + ? { + ...run, + completedAtMs: args.completedAtMs, + errorMessage: args.errorMessage, + status: "blocked", + } + : undefined, + ); + } + + async updateTaskAfterRun(args: { + errorMessage?: string; + nowMs: number; + run: ScheduledRun; + status: "blocked" | "completed" | "failed"; + }): Promise { + await withSqlLock(this.db, taskLockKey(args.run.taskId), async (db) => { + const current = await getTaskFromSql(db, args.run.taskId); + if (!current || current.status === "deleted") { + return; + } + + const isRunNow = current.runNowAtMs === args.run.scheduledForMs; + if (isRunNow) { + let nextRunAtMs = current.nextRunAtMs; + if ( + args.status !== "blocked" && + typeof current.nextRunAtMs === "number" && + current.nextRunAtMs <= args.run.scheduledForMs + ) { + nextRunAtMs = getNextRunAtMs( + current, + current.nextRunAtMs, + args.nowMs, + ); + } + await this.saveTaskRecord( + db, + { + ...current, + lastRunAtMs: args.run.scheduledForMs, + nextRunAtMs, + runNowAtMs: undefined, + status: + args.status === "blocked" + ? "blocked" + : nextRunAtMs + ? current.status + : "paused", + statusReason: + args.status === "blocked" ? args.errorMessage : undefined, + updatedAtMs: args.nowMs, + version: current.version + 1, + }, + current, + ); + return; + } + + if ( + current.status !== "active" || + current.nextRunAtMs !== args.run.scheduledForMs + ) { + await this.saveTaskRecord( + db, + { + ...current, + lastRunAtMs: args.run.scheduledForMs, + updatedAtMs: args.nowMs, + version: current.version + 1, + }, + current, + ); + return; + } + + const nextRunAtMs = + args.status === "blocked" + ? undefined + : getNextRunAtMs(current, args.run.scheduledForMs, args.nowMs); + + await this.saveTaskRecord( + db, + { + ...current, + lastRunAtMs: args.run.scheduledForMs, + nextRunAtMs, + status: + args.status === "blocked" + ? "blocked" + : nextRunAtMs + ? "active" + : "paused", + statusReason: + args.status === "blocked" ? args.errorMessage : undefined, + updatedAtMs: args.nowMs, + version: current.version + 1, + }, + current, + ); + }); + } + + private async updateRun( + runId: string, + update: (run: ScheduledRun) => ScheduledRun | undefined, + ): Promise { + return await withSqlLock( + this.db, + indexLockKey(runKey(runId)), + async (db) => { + const current = await getRunFromSql(db, runId); + if (!current) { + return undefined; + } + const next = update(current); + if (!next) { + return undefined; + } + await upsertSqlRun(db, next); + return next; + }, + ); + } +} + +/** Create a scheduler store backed by the plugin SQL database. */ +export function createSchedulerSqlStore(db: PluginDb): SchedulerStore { + return new SqlSchedulerStore(db); +} + +/** Create a read-only scheduler operational store backed by SQL. */ +export function createSchedulerOperationalSqlStore( + db: PluginDb, +): SchedulerOperationalStore { + return new SqlSchedulerStore(db); +} + +/** Copy retained scheduler plugin-state records into the scheduler SQL tables. */ +export async function migrateSchedulerStateToSql(args: { + db: PluginDb; + state: PluginState; +}): Promise<{ + existing: number; + migrated: number; + missing: number; + scanned: number; +}> { + const store = createSchedulerSqlStore(args.db); + const ids = await getIndex(args.state, globalTaskIndexKey()); + let existing = 0; + let migrated = 0; + let missing = 0; + const migratedTasks: ScheduledTask[] = []; + + for (const id of ids) { + const task = await getTaskFromState(args.state, id); + if (!task) { + missing += 1; + continue; + } + migratedTasks.push(task); + if (await store.getTask(task.id)) { + existing += 1; + continue; + } + await store.saveTask(task); + migrated += 1; + } + + const runs = await listIncompleteRunsForTasksFromState( + args.state, + migratedTasks, + ); + for (const run of runs) { + if (await store.getRun(run.id)) { + existing += 1; + continue; + } + await upsertSqlRun(args.db, run); + migrated += 1; + } + + return { + existing, + migrated, + missing, + scanned: ids.length + runs.length, + }; +} diff --git a/packages/junior-scheduler/src/types.ts b/packages/junior-scheduler/src/types.ts index 42e242dd0..3b84cb42a 100644 --- a/packages/junior-scheduler/src/types.ts +++ b/packages/junior-scheduler/src/types.ts @@ -1,5 +1,5 @@ import type { - AgentPluginCredentialSubject, + PluginCredentialSubject, SlackDestination, } from "@sentry/junior-plugin-api"; @@ -71,7 +71,7 @@ export interface ScheduledTask { createdAtMs: number; createdBy: ScheduledTaskPrincipal; conversationAccess?: ScheduledTaskConversationAccess; - credentialSubject?: AgentPluginCredentialSubject; + credentialSubject?: PluginCredentialSubject; destination: SlackDestination; executionActor?: ScheduledTaskExecutionActor; lastRunAtMs?: number; diff --git a/packages/junior-test-fixtures/package.json b/packages/junior-test-fixtures/package.json index 9bd662c23..700b88a4b 100644 --- a/packages/junior-test-fixtures/package.json +++ b/packages/junior-test-fixtures/package.json @@ -11,7 +11,7 @@ }, "dependencies": { "@electric-sql/pglite": "^0.4.6", - "drizzle-orm": "^0.45.2" + "drizzle-orm": "catalog:" }, "devDependencies": { "@types/node": "^25.9.1", diff --git a/packages/junior-test-fixtures/src/pglite.ts b/packages/junior-test-fixtures/src/pglite.ts index f2785d577..54de78db5 100644 --- a/packages/junior-test-fixtures/src/pglite.ts +++ b/packages/junior-test-fixtures/src/pglite.ts @@ -35,6 +35,10 @@ class LocalPgliteExecutor implements LocalPgliteFixture { statement: string, params: readonly unknown[] = [], ): Promise { + if (params.length === 0) { + await this.queryClient().exec(statement); + return; + } await this.queryClient().query(statement, [...params]); } diff --git a/packages/junior/package.json b/packages/junior/package.json index bf8d77b7c..5942fe2c8 100644 --- a/packages/junior/package.json +++ b/packages/junior/package.json @@ -48,9 +48,10 @@ "build": "tsup && tsc -p tsconfig.build.json --emitDeclarationOnly", "lint": "oxlint --config .oxlintrc.json --deny-warnings src tests scripts bin tsup.config.ts", "lint:fix": "oxlint --config .oxlintrc.json --deny-warnings --fix src tests scripts bin tsup.config.ts", - "test": "pnpm run test:slack-boundary && pnpm run test:arch-boundary && vitest run --maxWorkers=4", + "test": "pnpm run test:slack-boundary && pnpm run test:package-boundary && pnpm run test:arch-boundary && vitest run --maxWorkers=4", "test:watch": "vitest", "test:slack-boundary": "node scripts/check-slack-test-boundary.mjs", + "test:package-boundary": "node scripts/check-package-boundary.mjs", "test:arch-boundary": "depcruise --config .dependency-cruiser.mjs src/chat", "typecheck": "tsc --noEmit", "skills:check": "node scripts/check-skills.mjs", @@ -76,13 +77,13 @@ "ai": "^6.0.190", "bash-tool": "^1.3.16", "chat": "4.29.0", - "drizzle-orm": "^0.45.2", + "drizzle-orm": "catalog:", "hono": "^4.12.22", "jose": "^6.2.3", "just-bash": "3.0.1", "node-html-markdown": "^2.0.0", "yaml": "^2.9.0", - "zod": "^4.4.3" + "zod": "catalog:" }, "devDependencies": { "@sentry/junior-scheduler": "workspace:*", diff --git a/packages/junior/scripts/check-package-boundary.mjs b/packages/junior/scripts/check-package-boundary.mjs new file mode 100644 index 000000000..a7952568a --- /dev/null +++ b/packages/junior/scripts/check-package-boundary.mjs @@ -0,0 +1,72 @@ +import fs from "node:fs/promises"; +import path from "node:path"; + +const packageRoot = process.cwd(); +const srcRoot = path.join(packageRoot, "src"); +const SOURCE_EXTENSIONS = new Set([ + ".ts", + ".tsx", + ".js", + ".jsx", + ".mjs", + ".cjs", +]); +const FORBIDDEN_PLUGIN_PACKAGE_RE = + /(?:from\s+["']|import\s*\(\s*["'])(@sentry\/junior-[^"']+)["']/g; +const ALLOWED_CORE_PACKAGES = new Set(["@sentry/junior-plugin-api"]); + +async function listFilesRecursive(dirPath) { + const entries = await fs.readdir(dirPath, { withFileTypes: true }); + const files = []; + + for (const entry of entries) { + const nextPath = path.join(dirPath, entry.name); + if (entry.isDirectory()) { + files.push(...(await listFilesRecursive(nextPath))); + continue; + } + files.push(nextPath); + } + + return files; +} + +function toRelative(filePath) { + return path.relative(packageRoot, filePath).split(path.sep).join("/"); +} + +function lineNumberForOffset(source, offset) { + return source.slice(0, offset).split("\n").length; +} + +async function main() { + const violations = []; + const files = (await listFilesRecursive(srcRoot)).filter((filePath) => + SOURCE_EXTENSIONS.has(path.extname(filePath)), + ); + + for (const filePath of files) { + const source = await fs.readFile(filePath, "utf8"); + for (const match of source.matchAll(FORBIDDEN_PLUGIN_PACKAGE_RE)) { + const packageName = match[1]; + if (ALLOWED_CORE_PACKAGES.has(packageName)) { + continue; + } + violations.push( + `${toRelative(filePath)}:${lineNumberForOffset(source, match.index ?? 0)} imports plugin package ${packageName}`, + ); + } + } + + if (violations.length > 0) { + console.error("Core package boundary check failed:"); + for (const violation of violations) { + console.error(`- ${violation}`); + } + process.exit(1); + } + + console.log("Core package boundary check passed."); +} + +await main(); diff --git a/packages/junior/src/api-reference.ts b/packages/junior/src/api-reference.ts index a998ad249..6fd3a4567 100644 --- a/packages/junior/src/api-reference.ts +++ b/packages/junior/src/api-reference.ts @@ -11,9 +11,9 @@ export type { } from "./plugins"; export { createJuniorReporting } from "./reporting"; export type { - AgentPluginConversationStatus, - AgentPluginConversations, - AgentPluginConversationSummary, + PluginConversationStatus, + PluginConversations, + PluginConversationSummary, ConversationFeed, ConversationReport, ConversationReportStatus, diff --git a/packages/junior/src/app.ts b/packages/junior/src/app.ts index 40303f927..0da4e13e0 100644 --- a/packages/junior/src/app.ts +++ b/packages/junior/src/app.ts @@ -13,17 +13,19 @@ import { setPluginCatalogConfig, } from "@/chat/plugins/registry"; import { - type AgentPluginRouteRegistration, - getAgentPluginRoutes, - setAgentPlugins, - validateAgentPlugins, + type PluginRouteRegistration, + getPluginRoutes, + setPlugins, + validatePlugins, } from "@/chat/plugins/agent-hooks"; +import { validatePluginDatabaseRequirements } from "@/chat/plugins/db"; import type { PluginCatalogConfig } from "@/chat/plugins/types"; import type { - AgentPluginRouteMethod, - JuniorPluginRegistration, + PluginRouteMethod, + PluginRegistration, } from "@sentry/junior-plugin-api"; import { + pluginCatalogConfigFromEnv, pluginCatalogConfigFromPluginSet, pluginHookRegistrationsFromPluginSet, type JuniorPluginSet, @@ -129,15 +131,6 @@ async function resolveVirtualConfig(): Promise< } } -/** Resolve plugin configuration from the env fallback. */ -function resolveEnvPluginCatalogConfig(): PluginCatalogConfig | undefined { - const packages = readEnvPluginPackages(); - if (packages) { - return { packages }; - } - return undefined; -} - function isMissingVirtualConfig(error: unknown): boolean { if (!(error instanceof Error)) { return false; @@ -151,33 +144,6 @@ function isMissingVirtualConfig(error: unknown): boolean { ); } -function readEnvPluginPackages(): string[] | undefined { - const env = process.env.JUNIOR_PLUGIN_PACKAGES; - if (!env) { - return undefined; - } - - let parsed: unknown; - try { - parsed = JSON.parse(env); - } catch (error) { - throw new Error("JUNIOR_PLUGIN_PACKAGES must be valid JSON", { - cause: error, - }); - } - - if ( - !Array.isArray(parsed) || - parsed.some((value) => typeof value !== "string" || !value.trim()) - ) { - throw new Error( - "JUNIOR_PLUGIN_PACKAGES must be a JSON array of package names", - ); - } - - return parsed; -} - function hasConfiguredPluginCatalog( config: PluginCatalogConfig | undefined, ): boolean { @@ -216,7 +182,7 @@ function validateBuildIncludesPluginPackages( } function validateBuildIncludesPluginHookRegistrations( - hookRegistrations: JuniorPluginRegistration[], + hookRegistrations: PluginRegistration[], virtualConfig: JuniorVirtualConfig | undefined, ): void { const bundledHookRegistrations = virtualConfig?.pluginHookRegistrations ?? []; @@ -224,7 +190,9 @@ function validateBuildIncludesPluginHookRegistrations( return; } - const registered = new Set(hookRegistrations.map((plugin) => plugin.name)); + const registered = new Set( + hookRegistrations.map((plugin) => plugin.manifest.name), + ); const missing = bundledHookRegistrations.filter( (pluginName) => !registered.has(pluginName), ); @@ -238,7 +206,7 @@ function validateBuildIncludesPluginHookRegistrations( } function validatePluginRegistrations( - registrations: JuniorPluginRegistration[], + registrations: PluginRegistration[], ): void { const loadedPlugins = getPluginProviders(); const loadedNames = new Set( @@ -246,19 +214,22 @@ function validatePluginRegistrations( ); for (const registration of registrations) { - if (!loadedNames.has(registration.name)) { + if (!loadedNames.has(registration.manifest.name)) { throw new Error( - `Plugin registration "${registration.name}" does not have a matching plugin manifest. Add an inline manifest, packageName, or app-local plugin.yaml with the same name.`, + `Plugin registration "${registration.manifest.name}" does not have a matching plugin manifest. Add an inline manifest, packageName, or app-local plugin.yaml with the same name.`, ); } } } function validatePluginEgressCredentialHooks( - registrations: JuniorPluginRegistration[], + registrations: PluginRegistration[], ): void { const plugins = new Map( - registrations.map((registration) => [registration.name, registration]), + registrations.map((registration) => [ + registration.manifest.name, + registration, + ]), ); for (const provider of getPluginProviders()) { @@ -305,18 +276,14 @@ function validatePluginEgressCredentialHooks( } /** Mount plugin HTTP handlers before core routes claim those paths. */ -function mountAgentPluginRoutes( - app: Hono, - routes: AgentPluginRouteRegistration[], -): void { +function mountPluginRoutes(app: Hono, routes: PluginRouteRegistration[]): void { for (const route of routes) { const handler = (c: Context) => route.handler(c.req.raw); const methods = Array.isArray(route.method) ? route.method : [route.method ?? "ALL"]; const explicitMethods = methods.filter( - (method): method is Exclude => - method !== "ALL", + (method): method is Exclude => method !== "ALL", ); if (methods.includes("ALL")) { @@ -331,24 +298,25 @@ function mountAgentPluginRoutes( export async function createApp(options?: JuniorAppOptions): Promise { const virtualConfig = await resolveVirtualConfig(); const configuredPlugins = options?.plugins ?? virtualConfig?.pluginSet; - const agentPlugins = pluginHookRegistrationsFromPluginSet(configuredPlugins); + const plugins = pluginHookRegistrationsFromPluginSet(configuredPlugins); const pluginConfig = configuredPlugins ? pluginCatalogConfigFromPluginSet(configuredPlugins) - : (virtualConfig?.plugins ?? resolveEnvPluginCatalogConfig()); + : (virtualConfig?.plugins ?? pluginCatalogConfigFromEnv()); if (configuredPlugins) { validateBuildIncludesPluginPackages(pluginConfig, virtualConfig); } - validateBuildIncludesPluginHookRegistrations(agentPlugins, virtualConfig); - validateAgentPlugins(agentPlugins); + validateBuildIncludesPluginHookRegistrations(plugins, virtualConfig); + validatePlugins(plugins); + validatePluginDatabaseRequirements(plugins); const shouldValidatePluginCatalog = hasConfiguredPluginCatalog(pluginConfig) || Boolean(configuredPlugins?.registrations.length) || Boolean(Object.keys(options?.configDefaults ?? {}).length); const previousPluginCatalogConfig = setPluginCatalogConfig(pluginConfig); - const previousAgentPlugins = setAgentPlugins(agentPlugins); + const previousPlugins = setPlugins(plugins); const previousConfigDefaults = getConfigDefaults(); const previousSlackReactionConfig = getSlackReactionConfig(); - let agentPluginRoutes: AgentPluginRouteRegistration[] = []; + let pluginRoutes: PluginRouteRegistration[] = []; let sandboxEgressTracePropagationDomains: string[] = []; try { sandboxEgressTracePropagationDomains = @@ -366,10 +334,10 @@ export async function createApp(options?: JuniorAppOptions): Promise { configuredPlugins?.registrations ?? [], ); } - agentPluginRoutes = getAgentPluginRoutes(); + pluginRoutes = getPluginRoutes(); } catch (error) { setPluginCatalogConfig(previousPluginCatalogConfig); - setAgentPlugins(previousAgentPlugins); + setPlugins(previousPlugins); setConfigDefaults(previousConfigDefaults); setSlackReactionConfig(previousSlackReactionConfig); throw error; @@ -407,7 +375,7 @@ export async function createApp(options?: JuniorAppOptions): Promise { await next(); }); - mountAgentPluginRoutes(app, agentPluginRoutes); + mountPluginRoutes(app, pluginRoutes); app.get("/", () => healthGET()); app.get("/health", () => healthGET()); diff --git a/packages/junior/src/build/copy-build-content.ts b/packages/junior/src/build/copy-build-content.ts index 27708d601..324add6f4 100644 --- a/packages/junior/src/build/copy-build-content.ts +++ b/packages/junior/src/build/copy-build-content.ts @@ -4,7 +4,7 @@ import { discoverInstalledPluginPackageContent } from "@/chat/plugins/package-di import { globToRegex } from "@/build/glob-to-regex"; import { isValidPackageName, resolvePackageDir } from "@/package-resolution"; -/** Copy app directory and plugin manifests into the server output. */ +/** Copy app and declared plugin package content into the server output. */ export function copyAppAndPluginContent( cwd: string, serverRoot: string, @@ -31,6 +31,16 @@ export function copyAppAndPluginContent( for (const root of packagedContent.skillRoots) { copyRootIntoServerOutput(cwd, serverRoot, root); } + + for (const pkg of packagedContent.packages) { + if (pkg.hasMigrationsDir) { + copyRootIntoServerOutput( + cwd, + serverRoot, + path.join(pkg.dir, "migrations"), + ); + } + } } /** Copy extra file patterns into server output for files the bundler cannot trace. */ diff --git a/packages/junior/src/build/virtual-config.ts b/packages/junior/src/build/virtual-config.ts index c8464c6aa..a31603cf2 100644 --- a/packages/junior/src/build/virtual-config.ts +++ b/packages/junior/src/build/virtual-config.ts @@ -61,7 +61,7 @@ export function injectVirtualConfig( plugins: pluginCatalogConfigFromPluginSet(pluginSet), pluginHookRegistrations: pluginHookRegistrationsFromPluginSet( pluginSet, - ).map((plugin) => plugin.name), + ).map((plugin) => plugin.manifest.name), }); }; } diff --git a/packages/junior/src/chat/agent-dispatch/context.ts b/packages/junior/src/chat/agent-dispatch/context.ts index b94c85f01..621ff489b 100644 --- a/packages/junior/src/chat/agent-dispatch/context.ts +++ b/packages/junior/src/chat/agent-dispatch/context.ts @@ -1,6 +1,10 @@ -import type { HeartbeatHookContext } from "@sentry/junior-plugin-api"; +import type { + HeartbeatHookContext, + PluginRegistration, +} from "@sentry/junior-plugin-api"; import { bindSlackDirectCredentialSubject } from "@/chat/credentials/subject"; -import { createAgentPluginLogger } from "@/chat/plugins/logging"; +import { getPluginDbForRegistration } from "@/chat/plugins/db"; +import { createPluginLogger } from "@/chat/plugins/logging"; import { createPluginState } from "@/chat/plugins/state"; import { createOrGetDispatch, @@ -64,18 +68,22 @@ function bindDispatchCredentialSubject( /** Build the plugin-scoped heartbeat context that gates durable dispatch access. */ export function createHeartbeatContext(args: { - legacyStatePrefixes?: string[]; nowMs: number; - plugin: string; + plugin: string | PluginRegistration; }): HeartbeatHookContext { + const pluginName = + typeof args.plugin === "string" ? args.plugin : args.plugin.manifest.name; + const db = + typeof args.plugin === "string" + ? undefined + : getPluginDbForRegistration(args.plugin); let dispatchCount = 0; return { - plugin: { name: args.plugin }, + plugin: { name: pluginName }, nowMs: args.nowMs, - state: createPluginState(args.plugin, { - legacyStatePrefixes: args.legacyStatePrefixes, - }), - log: createAgentPluginLogger(args.plugin), + ...(db ? { db } : {}), + state: createPluginState(pluginName), + log: createPluginLogger(pluginName), agent: { async dispatch(options) { validateDispatchOptions(options); @@ -85,7 +93,7 @@ export function createHeartbeatContext(args: { } await verifyDispatchCredentialSubjectAccess(dispatchOptions); const result = await createOrGetDispatch({ - plugin: args.plugin, + plugin: pluginName, options: dispatchOptions, nowMs: args.nowMs, }); @@ -103,7 +111,7 @@ export function createHeartbeatContext(args: { }, async get(id) { return await getPluginDispatchProjection({ - plugin: args.plugin, + plugin: pluginName, id, }); }, diff --git a/packages/junior/src/chat/agent-dispatch/heartbeat.ts b/packages/junior/src/chat/agent-dispatch/heartbeat.ts index 3cbfbae43..5a88e92e6 100644 --- a/packages/junior/src/chat/agent-dispatch/heartbeat.ts +++ b/packages/junior/src/chat/agent-dispatch/heartbeat.ts @@ -1,4 +1,4 @@ -import { getAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { getPlugins } from "@/chat/plugins/agent-hooks"; import { logException, logInfo } from "@/chat/logging"; import { recoverConversationWork } from "@/chat/task-execution/heartbeat"; import type { ConversationWorkQueue } from "@/chat/task-execution/queue"; @@ -147,7 +147,8 @@ export async function runPluginHeartbeats(args: { nowMs: number; }): Promise { let count = 0; - for (const plugin of getAgentPlugins()) { + for (const plugin of getPlugins()) { + const pluginName = plugin.manifest.name; if (count >= (args.limit ?? DEFAULT_PLUGIN_LIMIT)) { break; } @@ -161,8 +162,7 @@ export async function runPluginHeartbeats(args: { Promise.resolve( heartbeat( createHeartbeatContext({ - legacyStatePrefixes: plugin.legacyStatePrefixes, - plugin: plugin.name, + plugin, nowMs: args.nowMs, }), ), @@ -178,7 +178,7 @@ export async function runPluginHeartbeats(args: { {}, { "app.dispatch.count": result.dispatchCount, - "app.plugin.name": plugin.name, + "app.plugin.name": pluginName, }, "Plugin heartbeat dispatched agent work", ); @@ -188,7 +188,7 @@ export async function runPluginHeartbeats(args: { error, "plugin_heartbeat_failed", {}, - { "app.plugin.name": plugin.name }, + { "app.plugin.name": pluginName }, "Plugin heartbeat failed", ); } diff --git a/packages/junior/src/chat/agent-dispatch/store.ts b/packages/junior/src/chat/agent-dispatch/store.ts index be43f381b..a0b3db85c 100644 --- a/packages/junior/src/chat/agent-dispatch/store.ts +++ b/packages/junior/src/chat/agent-dispatch/store.ts @@ -1,7 +1,7 @@ import { createHash } from "node:crypto"; import type { Lock, StateAdapter } from "chat"; import { - agentPluginCredentialSubjectSchema, + pluginCredentialSubjectSchema, destinationSchema, isSlackDestination, type SlackDestination, @@ -52,7 +52,7 @@ const credentialSubjectBindingSchema = z signature: z.string().min(1), }) .strict(); -const boundCredentialSubjectSchema = agentPluginCredentialSubjectSchema +const boundCredentialSubjectSchema = pluginCredentialSubjectSchema .extend({ binding: credentialSubjectBindingSchema, }) diff --git a/packages/junior/src/chat/credentials/subject.ts b/packages/junior/src/chat/credentials/subject.ts index 5fcd4484b..1735394cc 100644 --- a/packages/junior/src/chat/credentials/subject.ts +++ b/packages/junior/src/chat/credentials/subject.ts @@ -1,5 +1,5 @@ import { createHmac, timingSafeEqual } from "node:crypto"; -import type { AgentPluginCredentialSubject } from "@sentry/junior-plugin-api"; +import type { PluginCredentialSubject } from "@sentry/junior-plugin-api"; import type { CredentialSubject } from "@/chat/credentials/context"; import { isDmChannel, normalizeSlackConversationId } from "@/chat/slack/client"; import { isActorUserId, parseActorUserId } from "@/chat/requester"; @@ -12,7 +12,7 @@ function getCredentialSubjectSecret(): string | undefined { } function buildPayload(input: { - allowedWhen: AgentPluginCredentialSubject["allowedWhen"]; + allowedWhen: PluginCredentialSubject["allowedWhen"]; channelId: string; teamId: string; userId: string; @@ -45,7 +45,7 @@ export function createSlackDirectCredentialSubject(input: { channelId: string | undefined; teamId: string | undefined; userId: string | undefined; -}): AgentPluginCredentialSubject | undefined { +}): PluginCredentialSubject | undefined { const channelId = normalizeSlackConversationId(input.channelId); const teamId = input.teamId?.trim(); const userId = parseActorUserId(input.userId); @@ -63,7 +63,7 @@ export function createSlackDirectCredentialSubject(input: { /** Bind a delegated user subject to the Slack DM destination being dispatched. */ export function bindSlackDirectCredentialSubject(input: { channelId: string; - subject: AgentPluginCredentialSubject; + subject: PluginCredentialSubject; teamId: string; }): CredentialSubject | undefined { const channelId = normalizeSlackConversationId(input.channelId); diff --git a/packages/junior/src/chat/plugins/agent-hooks.ts b/packages/junior/src/chat/plugins/agent-hooks.ts index 0c37af5ea..d2993fc04 100644 --- a/packages/junior/src/chat/plugins/agent-hooks.ts +++ b/packages/junior/src/chat/plugins/agent-hooks.ts @@ -1,18 +1,19 @@ import type { - AgentPluginConversations, - AgentPluginReadState, - AgentPluginRoute, - AgentPluginRouteMethod, - AgentPluginSandbox, + PluginConversations, + PluginReadState, + PluginRoute, + PluginRouteMethod, + PluginSandbox, PluginOperationalReport, PluginOperationalReportContent, PluginOperationalTone, SlackConversationLink, - JuniorPluginRegistration, + PluginRegistration, SlackToolRegistrationHookContext, } from "@sentry/junior-plugin-api"; import { logInfo } from "@/chat/logging"; -import { createAgentPluginLogger } from "@/chat/plugins/logging"; +import { getPluginDbForRegistration } from "@/chat/plugins/db"; +import { createPluginLogger } from "@/chat/plugins/logging"; import { createPluginState } from "@/chat/plugins/state"; import { SANDBOX_WORKSPACE_ROOT } from "@/chat/sandbox/paths"; import type { ToolDefinition } from "@/chat/tools/definition"; @@ -27,10 +28,10 @@ import { resolveChannelCapabilities } from "@/chat/tools/channel-capabilities"; import type { Requester } from "@/chat/requester"; /** Signal that a plugin intentionally denied a tool execution. */ -export class AgentPluginHookDeniedError extends Error { +export class PluginHookDeniedError extends Error { constructor(message: string) { super(message); - this.name = "AgentPluginHookDeniedError"; + this.name = "PluginHookDeniedError"; } } @@ -44,25 +45,25 @@ export interface ToolHookResult { input: Record; } -export interface AgentPluginRouteRegistration extends AgentPluginRoute { +export interface PluginRouteRegistration extends PluginRoute { pluginName: string; } -export interface AgentPluginHookRunner { +export interface PluginHookRunner { beforeToolExecute(input: ToolHookInput): Promise; prepareSandbox(sandbox: SandboxInstance): Promise; } -let agentPlugins: JuniorPluginRegistration[] = []; -const AGENT_PLUGIN_NAME_RE = /^[a-z][a-z0-9-]*$/; -const AGENT_PLUGIN_TOOL_NAME_RE = /^[a-z][A-Za-z0-9]*$/; +let registeredPlugins: PluginRegistration[] = []; +const PLUGIN_NAME_RE = /^[a-z][a-z0-9-]*$/; +const PLUGIN_TOOL_NAME_RE = /^[a-z][A-Za-z0-9]*$/; const OPERATIONAL_REPORT_MAX_METRICS = 8; const OPERATIONAL_REPORT_MAX_RECORD_SETS = 8; const OPERATIONAL_REPORT_MAX_FIELDS = 8; const OPERATIONAL_REPORT_MAX_RECORDS = 25; const OPERATIONAL_REPORT_MAX_LABEL_LENGTH = 80; const OPERATIONAL_REPORT_MAX_VALUE_LENGTH = 160; -const AGENT_PLUGIN_ROUTE_METHODS = new Set([ +const PLUGIN_ROUTE_METHODS = new Set([ "GET", "POST", "PUT", @@ -77,80 +78,61 @@ function isRecord(value: unknown): value is Record { return Boolean(value && typeof value === "object" && !Array.isArray(value)); } -function validateLegacyStatePrefixes(plugin: JuniorPluginRegistration): void { - const prefixes = plugin.legacyStatePrefixes; - if (prefixes === undefined) { - return; - } - if (!Array.isArray(prefixes)) { - throw new Error( - `Plugin "${plugin.name}" legacyStatePrefixes must be an array`, - ); - } - - const allowedPrefix = `junior:${plugin.name}`; - for (const rawPrefix of prefixes) { - const prefix = typeof rawPrefix === "string" ? rawPrefix.trim() : ""; - if (!prefix) { - throw new Error( - `Plugin "${plugin.name}" legacy state prefixes must be non-empty strings`, - ); - } - if (prefix !== allowedPrefix && !prefix.startsWith(`${allowedPrefix}:`)) { - throw new Error( - `Plugin "${plugin.name}" legacy state prefix "${prefix}" must stay under "${allowedPrefix}"`, - ); - } - } +function basePluginContext(plugin: PluginRegistration) { + const name = plugin.manifest.name; + const db = getPluginDbForRegistration(plugin); + return { + plugin: { name }, + log: createPluginLogger(name), + ...(db ? { db } : {}), + }; } /** Validate plugin identity before it can affect process-wide hooks. */ -export function validateAgentPlugins( - plugins: JuniorPluginRegistration[], -): void { +export function validatePlugins(plugins: PluginRegistration[]): void { const seen = new Set(); for (const plugin of plugins) { - if (!AGENT_PLUGIN_NAME_RE.test(plugin.name)) { + const name = plugin.manifest.name; + if (!PLUGIN_NAME_RE.test(name)) { throw new Error( - `Plugin name "${plugin.name}" must be a lowercase plugin identifier`, + `Plugin name "${name}" must be a lowercase plugin identifier`, ); } - if (seen.has(plugin.name)) { - throw new Error(`Duplicate plugin name "${plugin.name}"`); + if (seen.has(name)) { + throw new Error(`Duplicate plugin name "${name}"`); } - seen.add(plugin.name); - validateLegacyStatePrefixes(plugin); + seen.add(name); } } /** Replace runtime hook plugins and return the previous list for rollback. */ -export function setAgentPlugins( - plugins: JuniorPluginRegistration[], -): JuniorPluginRegistration[] { - validateAgentPlugins(plugins); - const previous = agentPlugins; - agentPlugins = [...plugins].sort((left, right) => - left.name.localeCompare(right.name), +export function setPlugins( + nextPlugins: PluginRegistration[], +): PluginRegistration[] { + validatePlugins(nextPlugins); + const previous = registeredPlugins; + registeredPlugins = [...nextPlugins].sort((left, right) => + left.manifest.name.localeCompare(right.manifest.name), ); return previous; } /** Return the current runtime hook plugins without exposing mutable state. */ -export function getAgentPlugins(): JuniorPluginRegistration[] { - return [...agentPlugins]; +export function getPlugins(): PluginRegistration[] { + return [...registeredPlugins]; } /** Collect turn-scoped tools exposed by plugins. */ -export function getAgentPluginTools( +export function getPluginTools( context: ToolRuntimeContext, ): Record> { const tools: Record> = {}; - for (const plugin of getAgentPlugins()) { + for (const plugin of getPlugins()) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.tools; if (!hook) { continue; } - const log = createAgentPluginLogger(plugin.name); const destination = context.destination; const slackToolContext = getSlackToolContext(context); const credentialSubject = slackToolContext @@ -172,8 +154,7 @@ export function getAgentPluginTools( const pluginContext = context.source.platform === "slack" ? { - plugin: { name: plugin.name }, - log, + ...basePluginContext(plugin), requester: context.requester?.platform === "slack" ? context.requester @@ -184,13 +165,10 @@ export function getAgentPluginTools( slack: slackContext!, source: context.source, userText: context.userText, - state: createPluginState(plugin.name, { - legacyStatePrefixes: plugin.legacyStatePrefixes, - }), + state: createPluginState(pluginName), } : { - plugin: { name: plugin.name }, - log, + ...basePluginContext(plugin), requester: context.requester?.platform === "local" ? context.requester @@ -200,20 +178,18 @@ export function getAgentPluginTools( destination?.platform === "local" ? destination : undefined, source: context.source, userText: context.userText, - state: createPluginState(plugin.name, { - legacyStatePrefixes: plugin.legacyStatePrefixes, - }), + state: createPluginState(pluginName), }; const pluginTools = hook(pluginContext); for (const [name, tool] of Object.entries(pluginTools)) { - if (!AGENT_PLUGIN_TOOL_NAME_RE.test(name)) { + if (!PLUGIN_TOOL_NAME_RE.test(name)) { throw new Error( - `Plugin tool "${name}" from plugin "${plugin.name}" must be a camelCase identifier`, + `Plugin tool "${name}" from plugin "${pluginName}" must be a camelCase identifier`, ); } if (tools[name]) { throw new Error( - `Duplicate plugin tool "${name}" from plugin "${plugin.name}"`, + `Duplicate plugin tool "${name}" from plugin "${pluginName}"`, ); } tools[name] = tool as unknown as ToolDefinition; @@ -224,9 +200,9 @@ export function getAgentPluginTools( /** Normalize route methods so JS plugins cannot register invalid verbs. */ function routeMethods( - route: AgentPluginRoute, + route: PluginRoute, pluginName: string, -): AgentPluginRouteMethod[] { +): PluginRouteMethod[] { const methods = Array.isArray(route.method) ? route.method : [route.method ?? "ALL"]; @@ -237,7 +213,7 @@ function routeMethods( } for (const method of methods) { - if (!AGENT_PLUGIN_ROUTE_METHODS.has(method)) { + if (!PLUGIN_ROUTE_METHODS.has(method)) { throw new Error( `Plugin route "${route.path}" from plugin "${pluginName}" has invalid method "${String(method)}"`, ); @@ -252,43 +228,42 @@ function routeMethods( } /** Collect route handlers exposed by plugins for app-level mounting. */ -export function getAgentPluginRoutes(): AgentPluginRouteRegistration[] { - const routes: AgentPluginRouteRegistration[] = []; +export function getPluginRoutes(): PluginRouteRegistration[] { + const routes: PluginRouteRegistration[] = []; const seen = new Set(); - const methodsByPath = new Map>(); + const methodsByPath = new Map>(); - for (const plugin of getAgentPlugins()) { + for (const plugin of getPlugins()) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.routes; if (!hook) { continue; } - const log = createAgentPluginLogger(plugin.name); const pluginRoutes = hook({ - plugin: { name: plugin.name }, - log, + ...basePluginContext(plugin), }); if (!Array.isArray(pluginRoutes)) { throw new Error( - `Plugin routes hook from plugin "${plugin.name}" must return an array`, + `Plugin routes hook from plugin "${pluginName}" must return an array`, ); } for (const route of pluginRoutes) { if (!isRecord(route)) { throw new Error( - `Plugin route from plugin "${plugin.name}" must be an object`, + `Plugin route from plugin "${pluginName}" must be an object`, ); } if (typeof route.path !== "string" || !route.path.startsWith("/")) { throw new Error( - `Plugin route "${route.path}" from plugin "${plugin.name}" must start with /`, + `Plugin route "${route.path}" from plugin "${pluginName}" must start with /`, ); } if (typeof route.handler !== "function") { throw new Error( - `Plugin route "${route.path}" from plugin "${plugin.name}" must provide a handler`, + `Plugin route "${route.path}" from plugin "${pluginName}" must provide a handler`, ); } - const methods = routeMethods(route, plugin.name); + const methods = routeMethods(route, pluginName); const pathMethods = methodsByPath.get(route.path) ?? new Set(); if ( pathMethods.has("ALL") || @@ -309,7 +284,7 @@ export function getAgentPluginRoutes(): AgentPluginRouteRegistration[] { methodsByPath.set(route.path, pathMethods); routes.push({ ...route, - pluginName: plugin.name, + pluginName, }); } } @@ -346,21 +321,20 @@ function trustedSlackConversationUrl( } /** Resolve the first plugin conversation URL for finalized Slack footers. */ -export function getAgentPluginSlackConversationLink( +export function getPluginSlackConversationLink( conversationId: string, ): SlackConversationLink | undefined { - for (const plugin of getAgentPlugins()) { + for (const plugin of getPlugins()) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.slackConversationLink; if (!hook) { continue; } - const log = createAgentPluginLogger(plugin.name); const link = hook({ - plugin: { name: plugin.name }, - log, + ...basePluginContext(plugin), conversationId, }); - const url = trustedSlackConversationUrl(plugin.name, link); + const url = trustedSlackConversationUrl(pluginName, link); if (url) { return { url }; } @@ -368,10 +342,10 @@ export function getAgentPluginSlackConversationLink( return undefined; } -function pluginReadState(state: { get: AgentPluginReadState["get"] }) { +function pluginReadState(state: { get: PluginReadState["get"] }) { return { get: state.get, - } satisfies AgentPluginReadState; + } satisfies PluginReadState; } function operationalReportText( @@ -551,24 +525,21 @@ function failedOperationalReport(args: { } /** Collect read-only operational summaries exposed by plugins. */ -export async function getAgentPluginOperationalReports( +export async function getPluginOperationalReports( nowMs: number, - conversations: AgentPluginConversations, + conversations: PluginConversations, ): Promise { const reports: PluginOperationalReport[] = []; - for (const plugin of getAgentPlugins()) { + for (const plugin of getPlugins()) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.operationalReport; if (!hook) { continue; } - const log = createAgentPluginLogger(plugin.name); try { - const state = createPluginState(plugin.name, { - legacyStatePrefixes: plugin.legacyStatePrefixes, - }); + const state = createPluginState(pluginName); const report = await hook({ - plugin: { name: plugin.name }, - log, + ...basePluginContext(plugin), conversations, nowMs, state: pluginReadState(state), @@ -578,15 +549,16 @@ export async function getAgentPluginOperationalReports( } reports.push( sanitizeOperationalReport({ - pluginName: plugin.name, + pluginName, report, }), ); } catch (error) { + const log = createPluginLogger(pluginName); log.error("Plugin operational report failed", { error: error instanceof Error ? error.message : String(error), }); - reports.push(failedOperationalReport({ nowMs, pluginName: plugin.name })); + reports.push(failedOperationalReport({ nowMs, pluginName })); } } return reports; @@ -605,7 +577,7 @@ function normalizeEnv(value: unknown): Record { return env; } -function createSandboxCapability(sandbox: SandboxInstance): AgentPluginSandbox { +function createSandboxCapability(sandbox: SandboxInstance): PluginSandbox { return { root: SANDBOX_WORKSPACE_ROOT, juniorRoot: `${SANDBOX_WORKSPACE_ROOT}/.junior`, @@ -637,17 +609,18 @@ function createSandboxCapability(sandbox: SandboxInstance): AgentPluginSandbox { } /** Create one runner over runtime hook plugins registered by the app. */ -export function createAgentPluginHookRunner( +export function createPluginHookRunner( input: { requester?: Requester; } = {}, -): AgentPluginHookRunner { - const loaded = getAgentPlugins(); +): PluginHookRunner { + const loaded = getPlugins(); return { async prepareSandbox(sandbox) { const sandboxCapability = createSandboxCapability(sandbox); for (const plugin of loaded) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.sandboxPrepare; if (!hook) { continue; @@ -655,12 +628,11 @@ export function createAgentPluginHookRunner( logInfo( "agent_plugin_hook_sandbox_prepare", {}, - { "app.plugin.name": plugin.name }, + { "app.plugin.name": pluginName }, "Running agent plugin sandbox prepare hook", ); await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), requester: input.requester, sandbox: sandboxCapability, }); @@ -671,6 +643,7 @@ export function createAgentPluginHookRunner( const env = normalizeEnv(nextInput.env); for (const plugin of loaded) { + const pluginName = plugin.manifest.name; const hook = plugin.hooks?.beforeToolExecute; if (!hook) { continue; @@ -678,8 +651,7 @@ export function createAgentPluginHookRunner( let replacement: Record | undefined; let denied: string | undefined; await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), requester: input.requester, tool: { name: tool.name, @@ -704,12 +676,12 @@ export function createAgentPluginHookRunner( }); if (denied) { - throw new AgentPluginHookDeniedError(denied); + throw new PluginHookDeniedError(denied); } if (replacement !== undefined) { if (!isRecord(replacement)) { throw new Error( - `Plugin "${plugin.name}" replaced tool input with a non-object value`, + `Plugin "${pluginName}" replaced tool input with a non-object value`, ); } nextInput = { ...replacement }; diff --git a/packages/junior/src/chat/plugins/credential-hooks.ts b/packages/junior/src/chat/plugins/credential-hooks.ts index a7686ecfc..0c8a78300 100644 --- a/packages/junior/src/chat/plugins/credential-hooks.ts +++ b/packages/junior/src/chat/plugins/credential-hooks.ts @@ -1,19 +1,20 @@ import { - agentPluginAuthorizationSchema, - agentPluginCredentialResultSchema, - agentPluginGrantSchema, - agentPluginProviderAccountSchema, - type AgentPluginAuthorization, - type AgentPluginCredentialResult, - type AgentPluginGrant, - type AgentPluginProviderAccount, + pluginAuthorizationSchema, + pluginCredentialResultSchema, + pluginGrantSchema, + pluginProviderAccountSchema, + type PluginAuthorization, + type PluginCredentialResult, + type PluginGrant, + type PluginProviderAccount, } from "@sentry/junior-plugin-api"; import type { StoredTokens, UserTokenStore, } from "@/chat/credentials/user-token-store"; -import { getAgentPlugins } from "@/chat/plugins/agent-hooks"; -import { createAgentPluginLogger } from "@/chat/plugins/logging"; +import { getPlugins } from "@/chat/plugins/agent-hooks"; +import { getPluginDbForRegistration } from "@/chat/plugins/db"; +import { createPluginLogger } from "@/chat/plugins/logging"; interface SafeSchema { safeParse(value: unknown): @@ -41,12 +42,12 @@ function parseSchema( function parseAuthorization( value: unknown, pluginName: string, -): AgentPluginAuthorization | undefined { +): PluginAuthorization | undefined { if (value === undefined) { return undefined; } const authorization = parseSchema( - agentPluginAuthorizationSchema, + pluginAuthorizationSchema, value, `Plugin "${pluginName}" grant authorization is invalid`, ); @@ -58,24 +59,34 @@ function parseAuthorization( return authorization; } -function parseGrant(value: unknown, pluginName: string): AgentPluginGrant { +function parseGrant(value: unknown, pluginName: string): PluginGrant { return parseSchema( - agentPluginGrantSchema, + pluginGrantSchema, value, `Plugin "${pluginName}" grantForEgress returned an invalid grant`, ); } -function agentPluginFor(provider: string) { - return getAgentPlugins().find((candidate) => candidate.name === provider); +function pluginFor(provider: string) { + return getPlugins().find((candidate) => candidate.manifest.name === provider); +} + +function basePluginContext(plugin: NonNullable>) { + const pluginName = plugin.manifest.name; + const db = getPluginDbForRegistration(plugin); + return { + plugin: { name: pluginName }, + log: createPluginLogger(pluginName), + ...(db ? { db } : {}), + }; } function parseCredentialResult( value: unknown, pluginName: string, -): AgentPluginCredentialResult { +): PluginCredentialResult { const result = parseSchema( - agentPluginCredentialResultSchema, + pluginCredentialResultSchema, value, `Plugin "${pluginName}" issueCredential result is invalid`, ); @@ -100,26 +111,27 @@ export interface EgressGrantInput { /** Ask a plugin which grant an outbound request needs. */ export async function selectPluginGrant( input: EgressGrantInput, -): Promise { - const plugin = agentPluginFor(input.provider); +): Promise { + const plugin = pluginFor(input.provider); const hook = plugin?.hooks?.grantForEgress; if (!plugin || !hook) { return undefined; } const result = await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), request: { ...(input.bodyText !== undefined ? { bodyText: input.bodyText } : {}), method: input.method, url: input.upstreamUrl.toString(), }, }); - return result === undefined ? undefined : parseGrant(result, plugin.name); + return result === undefined + ? undefined + : parseGrant(result, plugin.manifest.name); } export interface EgressResponseInput { - grant: AgentPluginGrant; + grant: PluginGrant; method: string; provider: string; response: { @@ -140,21 +152,20 @@ export interface EgressResponseEffects { export async function onPluginEgressResponse( input: EgressResponseInput, ): Promise { - const plugin = agentPluginFor(input.provider); + const plugin = pluginFor(input.provider); const hook = plugin?.hooks?.onEgressResponse; if (!plugin || !hook) { return {}; } let permissionDenied: { message: string } | undefined; await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), grant: input.grant, permissionDenied(message) { const trimmed = message.trim(); if (!trimmed) { throw new Error( - `Plugin "${plugin.name}" onEgressResponse permissionDenied message is empty`, + `Plugin "${plugin.manifest.name}" onEgressResponse permissionDenied message is empty`, ); } permissionDenied = { message: trimmed }; @@ -170,7 +181,7 @@ export async function onPluginEgressResponse( /** Return whether a plugin owns credential issuance for egress. */ export function hasEgressCredentialHooks(provider: string): boolean { - const hooks = agentPluginFor(provider)?.hooks; + const hooks = pluginFor(provider)?.hooks; return Boolean(hooks?.grantForEgress || hooks?.issueCredential); } @@ -188,7 +199,7 @@ export interface IssueCredentialInput { type: "user"; userId: string; }; - grant: AgentPluginGrant; + grant: PluginGrant; provider: string; userTokenStore: UserTokenStore; } @@ -197,31 +208,30 @@ export interface IssueCredentialInput { export async function resolvePluginOAuthAccount(input: { provider: string; tokens: StoredTokens; -}): Promise { - const plugin = agentPluginFor(input.provider); +}): Promise { + const plugin = pluginFor(input.provider); const hook = plugin?.hooks?.resolveOAuthAccount; if (!plugin || !hook) { return undefined; } const account = await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), tokens: input.tokens, }); return account === undefined ? undefined : parseSchema( - agentPluginProviderAccountSchema, + pluginProviderAccountSchema, account, - `Plugin "${plugin.name}" resolveOAuthAccount returned an invalid account`, + `Plugin "${plugin.manifest.name}" resolveOAuthAccount returned an invalid account`, ); } /** Ask a plugin to issue headers or describe why the selected grant is unavailable. */ export async function issuePluginCredential( input: IssueCredentialInput, -): Promise { - const plugin = agentPluginFor(input.provider); +): Promise { + const plugin = pluginFor(input.provider); const hook = plugin?.hooks?.issueCredential; if (!plugin || !hook) { throw new Error(`Plugin "${input.provider}" has no issueCredential hook`); @@ -230,8 +240,7 @@ export async function issuePluginCredential( input.actor.type === "user" ? input.actor.userId : undefined; const credentialSubjectUserId = input.credentialSubject?.userId; const result = await hook({ - plugin: { name: plugin.name }, - log: createAgentPluginLogger(plugin.name), + ...basePluginContext(plugin), actor: input.actor, grant: input.grant, ...(input.credentialSubject @@ -243,11 +252,14 @@ export async function issuePluginCredential( currentUser: { userId: currentUserId, get: async () => - await input.userTokenStore.get(currentUserId, plugin.name), + await input.userTokenStore.get( + currentUserId, + plugin.manifest.name, + ), set: async (tokens) => { await input.userTokenStore.set( currentUserId, - plugin.name, + plugin.manifest.name, tokens, ); }, @@ -261,12 +273,12 @@ export async function issuePluginCredential( get: async () => await input.userTokenStore.get( credentialSubjectUserId, - plugin.name, + plugin.manifest.name, ), set: async (tokens) => { await input.userTokenStore.set( credentialSubjectUserId, - plugin.name, + plugin.manifest.name, tokens, ); }, @@ -275,5 +287,5 @@ export async function issuePluginCredential( : {}), }, }); - return parseCredentialResult(result, plugin.name); + return parseCredentialResult(result, plugin.manifest.name); } diff --git a/packages/junior/src/chat/plugins/db.ts b/packages/junior/src/chat/plugins/db.ts new file mode 100644 index 000000000..8005523e4 --- /dev/null +++ b/packages/junior/src/chat/plugins/db.ts @@ -0,0 +1,260 @@ +import { createHash } from "node:crypto"; +import { readdirSync, readFileSync, statSync } from "node:fs"; +import path from "node:path"; +import type { PluginDb, PluginRegistration } from "@sentry/junior-plugin-api"; +import { z } from "zod"; +import { getChatConfig } from "@/chat/config"; +import type { JuniorSqlMigrationExecutor } from "@/chat/sql/db"; +import { createNeonJuniorSqlExecutor } from "@/chat/sql/neon"; + +const PLUGIN_SCHEMA_LOCK_NAME = "junior_plugin_schema"; +const MIGRATION_FILENAME_RE = /^[0-9]{4}_[a-z0-9_]+\.sql$/; + +const migrationRecordSchema = z + .object({ + id: z.string().min(1), + checksum: z.string().min(1), + }) + .strict(); + +export interface PluginMigration { + checksum: string; + filename: string; + id: string; + pluginName: string; + sql: string; +} + +export interface PluginMigrationRoot { + /** Absolute path to the plugin's migrations directory. */ + dir: string; + pluginName: string; +} + +export interface PluginMigrationResult { + existing: number; + migrated: number; + scanned: number; +} + +interface StoredMigrationRecord { + checksum: string; + id: string; +} + +let configuredPluginDb: + | { + databaseUrl: string; + db: PluginDb; + executor: JuniorSqlMigrationExecutor; + } + | undefined; + +function checksumSql(sql: string): string { + return createHash("sha256").update(sql).digest("hex"); +} + +function parseStoredMigrationRecord(value: unknown): StoredMigrationRecord { + return migrationRecordSchema.parse(value); +} + +function assertMigrationFilename(filename: string): void { + if ( + !filename || + filename !== path.basename(filename) || + !MIGRATION_FILENAME_RE.test(filename) + ) { + throw new Error(`Plugin migration filename "${filename}" is invalid`); + } +} + +function assertUniqueMigrationIds( + migrations: readonly PluginMigration[], +): void { + const seen = new Set(); + for (const migration of migrations) { + if (seen.has(migration.id)) { + throw new Error(`Duplicate plugin migration id ${migration.id}`); + } + seen.add(migration.id); + } +} + +function migrationId(pluginName: string, filename: string): string { + return `plugin:${pluginName}/${filename}`; +} + +function createMigrationTableSql(): string { + return ` +CREATE TABLE IF NOT EXISTS junior_schema_migrations ( + id TEXT PRIMARY KEY, + checksum TEXT NOT NULL, + applied_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP +) +`; +} + +function createPluginDb(executor: JuniorSqlMigrationExecutor): PluginDb { + const db = executor.db(); + const pluginDb: PluginDb = { + delete: db.delete.bind(db) as PluginDb["delete"], + execute: (statement, params) => executor.execute(statement, params), + insert: db.insert.bind(db) as PluginDb["insert"], + query: (statement: string, params?: readonly unknown[]) => + executor.query(statement, params), + select: db.select.bind(db) as PluginDb["select"], + transaction: async (callback) => + await executor.transaction( + async () => await callback(createPluginDb(executor)), + ), + update: db.update.bind(db) as PluginDb["update"], + }; + return pluginDb; +} + +function getConfiguredPluginDb(): PluginDb | undefined { + const databaseUrl = getChatConfig().sql.databaseUrl; + if (!databaseUrl) { + return undefined; + } + if (configuredPluginDb?.databaseUrl !== databaseUrl) { + const executor = createNeonJuniorSqlExecutor({ + connectionString: databaseUrl, + }); + configuredPluginDb = { + databaseUrl, + executor, + db: createPluginDb(executor), + }; + } + return configuredPluginDb.db; +} + +async function listAppliedMigrations( + executor: JuniorSqlMigrationExecutor, +): Promise> { + const rows = await executor.query( + "SELECT id, checksum FROM junior_schema_migrations ORDER BY id ASC", + ); + const records = new Map(); + for (const row of rows) { + const record = parseStoredMigrationRecord(row); + records.set(record.id, record); + } + return records; +} + +async function applyPluginMigration( + executor: JuniorSqlMigrationExecutor, + migration: PluginMigration, +): Promise { + await executor.transaction(async () => { + await executor.execute(migration.sql); + await executor.execute( + "INSERT INTO junior_schema_migrations (id, checksum) VALUES ($1, $2)", + [migration.id, migration.checksum], + ); + }); +} + +/** Adapt the shared Junior SQL executor to the plugin-facing DB surface. */ +export function createPluginDbForExecutor( + executor: JuniorSqlMigrationExecutor, +): PluginDb { + return createPluginDb(executor); +} + +/** Return a configured plugin DB only for plugins that declare database usage. */ +export function getPluginDbForRegistration( + registration: PluginRegistration, +): PluginDb | undefined { + if (!registration.database) { + return undefined; + } + return getConfiguredPluginDb(); +} + +/** Fail early when a plugin declares DB access without SQL config. */ +export function validatePluginDatabaseRequirements( + registrations: PluginRegistration[], +): void { + if (getChatConfig().sql.databaseUrl) { + return; + } + const databasePlugins = registrations + .filter((registration) => registration.database) + .map((registration) => registration.manifest.name); + if (databasePlugins.length > 0) { + throw new Error( + `Plugin database access requires JUNIOR_DATABASE_URL or DATABASE_URL for: ${databasePlugins.join(", ")}`, + ); + } +} + +/** Read committed SQL migration artifacts for one enabled plugin root. */ +export function readPluginMigrations( + root: PluginMigrationRoot, +): PluginMigration[] { + const migrationsDir = root.dir; + let stat: ReturnType; + try { + stat = statSync(migrationsDir); + } catch { + return []; + } + if (!stat.isDirectory()) { + throw new Error( + `Plugin "${root.pluginName}" migrations path is not a directory`, + ); + } + + return readdirSync(migrationsDir) + .filter((filename) => filename.endsWith(".sql")) + .sort((left, right) => left.localeCompare(right)) + .map((filename) => { + assertMigrationFilename(filename); + const sql = readFileSync(path.join(migrationsDir, filename), "utf8"); + if (!sql.trim()) { + throw new Error( + `Plugin "${root.pluginName}" migration "${filename}" is empty`, + ); + } + return { + checksum: checksumSql(sql), + filename, + id: migrationId(root.pluginName, filename), + pluginName: root.pluginName, + sql, + }; + }); +} + +/** Apply plugin-owned SQL migrations after core Junior migrations. */ +export async function migratePluginSchemas( + executor: JuniorSqlMigrationExecutor, + migrations: readonly PluginMigration[], +): Promise { + assertUniqueMigrationIds(migrations); + const result: PluginMigrationResult = { + existing: 0, + migrated: 0, + scanned: migrations.length, + }; + await executor.withLock(PLUGIN_SCHEMA_LOCK_NAME, async () => { + await executor.execute(createMigrationTableSql()); + const applied = await listAppliedMigrations(executor); + for (const migration of migrations) { + const existing = applied.get(migration.id); + if (existing) { + if (existing.checksum !== migration.checksum) { + throw new Error(`Plugin migration ${migration.id} checksum changed`); + } + result.existing++; + continue; + } + await applyPluginMigration(executor, migration); + result.migrated++; + } + }); + return result; +} diff --git a/packages/junior/src/chat/plugins/logging.ts b/packages/junior/src/chat/plugins/logging.ts index 9699dbd0d..60f8cafea 100644 --- a/packages/junior/src/chat/plugins/logging.ts +++ b/packages/junior/src/chat/plugins/logging.ts @@ -1,8 +1,8 @@ -import type { AgentPluginLogger } from "@sentry/junior-plugin-api"; +import type { PluginLogger } from "@sentry/junior-plugin-api"; import { logException, logInfo, logWarn } from "@/chat/logging"; /** Create the host logger exposed to plugin hooks. */ -export function createAgentPluginLogger(plugin: string): AgentPluginLogger { +export function createPluginLogger(plugin: string): PluginLogger { return { info(message, metadata) { logInfo( diff --git a/packages/junior/src/chat/plugins/package-discovery.ts b/packages/junior/src/chat/plugins/package-discovery.ts index 54c903b0d..f34906083 100644 --- a/packages/junior/src/chat/plugins/package-discovery.ts +++ b/packages/junior/src/chat/plugins/package-discovery.ts @@ -6,10 +6,11 @@ import { } from "@/package-resolution"; interface InstalledJuniorContentPackage { - name: string; dir: string; nodeModulesDir?: string; + packageName: string; hasRootPluginManifest: boolean; + hasMigrationsDir: boolean; hasPluginsDir: boolean; hasSkillsDir: boolean; } @@ -18,8 +19,9 @@ export interface InstalledPluginPackageContent { packageNames: string[]; packages: { dir: string; + hasMigrationsDir: boolean; hasSkillsDir: boolean; - name: string; + packageName: string; }[]; manifestRoots: string[]; skillRoots: string[]; @@ -105,18 +107,26 @@ function resolvePackageDirFromName( function readPluginPackageFlags(dir: string): { hasRootPluginManifest: boolean; + hasMigrationsDir: boolean; hasPluginsDir: boolean; hasSkillsDir: boolean; } | null { const hasRootPluginManifest = isFile(path.join(dir, "plugin.yaml")); + const hasMigrationsDir = isDirectory(path.join(dir, "migrations")); const hasPluginsDir = isDirectory(path.join(dir, "plugins")); const hasSkillsDir = isDirectory(path.join(dir, "skills")); - if (!hasRootPluginManifest && !hasPluginsDir && !hasSkillsDir) { + if ( + !hasRootPluginManifest && + !hasMigrationsDir && + !hasPluginsDir && + !hasSkillsDir + ) { return null; } return { hasRootPluginManifest, + hasMigrationsDir, hasPluginsDir, hasSkillsDir, }; @@ -149,15 +159,15 @@ function discoverDeclaredPackages( const pluginFlags = readPluginPackageFlags(resolved.dir); if (!pluginFlags) { throw new Error( - `Plugin package "${packageName}" was configured but does not contain plugin content; expected plugin.yaml, plugins/, or skills/ in ${resolved.dir}`, + `Plugin package "${packageName}" was configured but does not contain plugin content; expected plugin.yaml, migrations/, plugins/, or skills/ in ${resolved.dir}`, ); } seenPackageDirs.add(resolved.dir); discovered.push({ - name: packageName, dir: resolved.dir, nodeModulesDir: resolved.nodeModulesDir, + packageName, ...pluginFlags, }); } @@ -194,7 +204,7 @@ export function discoverInstalledPluginPackageContent( const tracingBasePath = pkg.nodeModulesDir ? pathForTracingInclude( resolvedCwd, - path.join(pkg.nodeModulesDir, ...pkg.name.split("/")), + path.join(pkg.nodeModulesDir, ...pkg.packageName.split("/")), ) : pathForTracingInclude(resolvedCwd, pkg.dir); if (pkg.hasRootPluginManifest) { @@ -203,6 +213,11 @@ export function discoverInstalledPluginPackageContent( tracingIncludes.push(`${tracingBasePath}/plugin.yaml`); } } + if (pkg.hasMigrationsDir) { + if (tracingBasePath) { + tracingIncludes.push(`${tracingBasePath}/migrations/**/*`); + } + } if (pkg.hasPluginsDir) { manifestRoots.push(path.join(pkg.dir, "plugins")); if (tracingBasePath) { @@ -219,12 +234,13 @@ export function discoverInstalledPluginPackageContent( return { packageNames: uniqueStringsInOrder( - discoveredPackages.map((pkg) => pkg.name), + discoveredPackages.map((pkg) => pkg.packageName), ), packages: discoveredPackages.map((pkg) => ({ dir: pkg.dir, + hasMigrationsDir: pkg.hasMigrationsDir, hasSkillsDir: pkg.hasSkillsDir, - name: pkg.name, + packageName: pkg.packageName, })), manifestRoots: uniqueStringsInOrder(manifestRoots), skillRoots: uniqueStringsInOrder(skillRoots), diff --git a/packages/junior/src/chat/plugins/registry.ts b/packages/junior/src/chat/plugins/registry.ts index aababdfbd..a961b2462 100644 --- a/packages/junior/src/chat/plugins/registry.ts +++ b/packages/junior/src/chat/plugins/registry.ts @@ -28,6 +28,7 @@ interface LoadedPluginState { packageSkillRoots: Set; pluginConfigKeys: Set; pluginDefinitions: PluginDefinition[]; + pluginMigrationRoots: Map; pluginsByName: Map; signature: string; } @@ -55,6 +56,7 @@ function createLoadedPluginState(signature: string): LoadedPluginState { return { signature, pluginDefinitions: [], + pluginMigrationRoots: new Map(), capabilityToPlugin: new Map(), domainToPlugin: new Map(), pluginConfigKeys: new Set(), @@ -77,6 +79,7 @@ function registerPluginManifest( manifest: PluginDefinition["manifest"], pluginDir: string, skillsDir?: string, + migrationsDir?: string, ): void { if (state.pluginsByName.has(manifest.name)) { throw new Error(`Duplicate plugin name "${manifest.name}"`); @@ -102,11 +105,15 @@ function registerPluginManifest( const definition: PluginDefinition = { manifest, dir: pluginDir, + ...(migrationsDir ? { migrationsDir } : {}), ...(skillsDir ? { skillsDir } : {}), }; state.pluginDefinitions.push(definition); state.pluginsByName.set(manifest.name, definition); + if (definition.migrationsDir) { + state.pluginMigrationRoots.set(manifest.name, definition.migrationsDir); + } for (const cap of manifest.capabilities) { state.capabilityToPlugin.set(cap, definition); @@ -125,6 +132,7 @@ function registerYamlPluginManifest( pluginDir: string, ): void { const manifest = parsePluginManifest(raw, pluginDir, pluginConfig); + // Declarative manifests are manifest-only; code registrations claim migrations. registerPluginManifest( state, manifest, @@ -167,6 +175,16 @@ function getPluginCatalogSource(): PluginCatalogSource { signature: JSON.stringify({ inlineManifests, manifestRoots, + packages: packagedContent.packages + .map((pkg) => ({ + dir: path.resolve(pkg.dir), + hasMigrationsDir: pkg.hasMigrationsDir, + hasSkillsDir: pkg.hasSkillsDir, + packageName: pkg.packageName, + })) + .sort((left, right) => + left.packageName.localeCompare(right.packageName), + ), packagedSkillRoots, packageNames: [...packagedContent.packageNames].sort(), pluginConfig: pluginConfig ?? {}, @@ -213,14 +231,19 @@ function clonePluginCatalogConfig( function packageContentByName( packagedContent: InstalledPluginPackageContent, packageName: string, -): { dir: string; hasSkillsDir: boolean } | undefined { - return packagedContent.packages.find((pkg) => pkg.name === packageName); +): + | { dir: string; hasMigrationsDir: boolean; hasSkillsDir: boolean } + | undefined { + return packagedContent.packages.find( + (pkg) => pkg.packageName === packageName, + ); } function registerInlineManifests( state: LoadedPluginState, source: PluginCatalogSource, ): void { + const migrationOwners = new Map(); for (const definition of source.inlineManifests) { const pkg = definition.packageName ? packageContentByName(source.packagedContent, definition.packageName) @@ -229,12 +252,28 @@ function registerInlineManifests( const skillsDir = pkg?.hasSkillsDir ? path.join(pkg.dir, "skills") : undefined; + const migrationsDir = + pkg?.hasMigrationsDir && + statSync(path.join(pkg.dir, "migrations"), { + throwIfNoEntry: false, + })?.isDirectory() + ? path.join(pkg.dir, "migrations") + : undefined; const manifest = parseInlinePluginManifest( definition.manifest, dir, pluginConfig, ); - registerPluginManifest(state, manifest, dir, skillsDir); + if (migrationsDir) { + const owner = migrationOwners.get(migrationsDir); + if (owner) { + throw new Error( + `Plugin "${manifest.name}" cannot share migrations directory with plugin "${owner}"`, + ); + } + migrationOwners.set(migrationsDir, manifest.name); + } + registerPluginManifest(state, manifest, dir, skillsDir, migrationsDir); } } @@ -419,6 +458,16 @@ export function getPluginProviders(): PluginDefinition[] { return [...ensurePluginsLoaded().pluginDefinitions]; } +export function getPluginMigrationRoots(): { + dir: string; + pluginName: string; +}[] { + const state = ensurePluginsLoaded(); + return [...state.pluginMigrationRoots.entries()] + .map(([pluginName, dir]) => ({ pluginName, dir })) + .sort((left, right) => left.pluginName.localeCompare(right.pluginName)); +} + export function getPluginMcpProviders(): PluginDefinition[] { return ensurePluginsLoaded().pluginDefinitions.filter((plugin) => Boolean(plugin.manifest.mcp), diff --git a/packages/junior/src/chat/plugins/state.ts b/packages/junior/src/chat/plugins/state.ts index ec38e4a24..4812176f3 100644 --- a/packages/junior/src/chat/plugins/state.ts +++ b/packages/junior/src/chat/plugins/state.ts @@ -1,18 +1,19 @@ import { createHash } from "node:crypto"; -import type { AgentPluginState } from "@sentry/junior-plugin-api"; +import type { PluginState } from "@sentry/junior-plugin-api"; +import type { StateAdapter } from "chat"; import { getStateAdapter } from "@/chat/state/adapter"; const MAX_PLUGIN_STATE_KEY_LENGTH = 512; -export interface PluginStateOptions { - legacyStatePrefixes?: string[]; -} - function hashKeyPart(value: string): string { return createHash("sha256").update(value).digest("hex").slice(0, 32); } function pluginStateKey(plugin: string, key: string): string { + const pluginPrefix = `junior:${plugin}`; + if (key === pluginPrefix || key.startsWith(`${pluginPrefix}:`)) { + return key; + } return `junior:plugin_state:${hashKeyPart(plugin)}:${hashKeyPart(key)}`; } @@ -25,68 +26,36 @@ function validatePluginStateKey(key: string): void { } } -function legacyStateKey( - key: string, - options: PluginStateOptions | undefined, -): string | undefined { - for (const prefix of options?.legacyStatePrefixes ?? []) { - const trimmed = prefix.trim(); - if (!trimmed) { - continue; - } - if (key === trimmed || key.startsWith(`${trimmed}:`)) { - return key; - } - } - return undefined; -} - /** Create a durable state namespace scoped to one plugin. */ export function createPluginState( plugin: string, - options?: PluginStateOptions, -): AgentPluginState { + adapter?: StateAdapter, +): PluginState { + const getAdapter = (): StateAdapter => adapter ?? getStateAdapter(); return { async delete(key) { validatePluginStateKey(key); - const state = getStateAdapter(); + const state = getAdapter(); await state.connect(); await state.delete(pluginStateKey(plugin, key)); - const legacyKey = legacyStateKey(key, options); - if (legacyKey) { - await state.delete(legacyKey); - } }, async get(key: string): Promise { validatePluginStateKey(key); - const state = getStateAdapter(); + const state = getAdapter(); await state.connect(); const value = await state.get(pluginStateKey(plugin, key)); - if (value !== null && value !== undefined) { - return value; - } - const legacyKey = legacyStateKey(key, options); - return legacyKey - ? ((await state.get(legacyKey)) ?? undefined) - : undefined; + return value ?? undefined; }, async set(key, value, ttlMs) { validatePluginStateKey(key); - const state = getStateAdapter(); + const state = getAdapter(); await state.connect(); await state.set(pluginStateKey(plugin, key), value, ttlMs); }, async setIfNotExists(key, value, ttlMs) { validatePluginStateKey(key); - const state = getStateAdapter(); + const state = getAdapter(); await state.connect(); - const legacyKey = legacyStateKey(key, options); - if (legacyKey) { - const existing = await state.get(legacyKey); - if (existing !== null && existing !== undefined) { - return false; - } - } return await state.setIfNotExists( pluginStateKey(plugin, key), value, @@ -95,10 +64,9 @@ export function createPluginState( }, async withLock(key, ttlMs, callback) { validatePluginStateKey(key); - const state = getStateAdapter(); + const state = getAdapter(); await state.connect(); - const lockKey = - legacyStateKey(key, options) ?? pluginStateKey(plugin, key); + const lockKey = pluginStateKey(plugin, key); const lock = await state.acquireLock(lockKey, ttlMs); if (!lock) { throw new Error(`Could not acquire plugin state lock for ${key}`); diff --git a/packages/junior/src/chat/plugins/types.ts b/packages/junior/src/chat/plugins/types.ts index 4d19e94f7..4e8d1938b 100644 --- a/packages/junior/src/chat/plugins/types.ts +++ b/packages/junior/src/chat/plugins/types.ts @@ -173,6 +173,7 @@ export interface PluginBrokerDeps { export interface PluginDefinition { manifest: PluginManifest; dir: string; + migrationsDir?: string; skillsDir?: string; } diff --git a/packages/junior/src/chat/respond.ts b/packages/junior/src/chat/respond.ts index 78f825187..48efca037 100644 --- a/packages/junior/src/chat/respond.ts +++ b/packages/junior/src/chat/respond.ts @@ -40,7 +40,7 @@ import { getPluginMcpProviders, getPluginProviders, } from "@/chat/plugins/registry"; -import { createAgentPluginHookRunner } from "@/chat/plugins/agent-hooks"; +import { createPluginHookRunner } from "@/chat/plugins/agent-hooks"; import { McpToolManager } from "@/chat/mcp/tool-manager"; import { inferActiveMcpProvidersFromPiMessages, @@ -769,7 +769,7 @@ export async function generateAssistantReply( ? context.credentialContext.actor.userId : undefined; const userTokenStore = createUserTokenStore(); - const agentPluginHooks = createAgentPluginHookRunner({ + const pluginHooks = createPluginHookRunner({ requester: actorRequester, }); sandboxExecutor = createSandboxExecutor({ @@ -779,7 +779,7 @@ export async function generateAssistantReply( traceContext: spanContext, tracePropagation: context.sandbox?.tracePropagation, credentialEgress: context.credentialContext, - agentHooks: agentPluginHooks, + agentHooks: pluginHooks, onSandboxAcquired: async (sandbox) => { lastKnownSandboxId = sandbox.sandboxId; lastKnownSandboxDependencyProfileHash = @@ -1233,7 +1233,7 @@ export async function generateAssistantReply( sandboxExecutor, pluginAuth, onToolCall, - agentPluginHooks, + pluginHooks, conversationPrivacy, ); advisorTools = createAgentTools( @@ -1244,7 +1244,7 @@ export async function generateAssistantReply( sandboxExecutor, pluginAuth, onToolCall, - agentPluginHooks, + pluginHooks, conversationPrivacy, ); // Keep Pi's native tool schema static for the whole turn. Ideally this diff --git a/packages/junior/src/chat/sandbox/egress-credentials.ts b/packages/junior/src/chat/sandbox/egress-credentials.ts index 41d3d4912..f0e612672 100644 --- a/packages/junior/src/chat/sandbox/egress-credentials.ts +++ b/packages/junior/src/chat/sandbox/egress-credentials.ts @@ -4,8 +4,8 @@ import { } from "@/chat/capabilities/factory"; import { CredentialUnavailableError } from "@/chat/credentials/broker"; import type { - AgentPluginAuthorization, - AgentPluginGrant, + PluginAuthorization, + PluginGrant, } from "@sentry/junior-plugin-api"; import { hasEgressCredentialHooks, @@ -28,11 +28,11 @@ const HTTP_READ_METHODS = new Set(["GET", "HEAD", "OPTIONS"]); export type SandboxEgressGrantSelection = | { - grant: AgentPluginGrant; + grant: PluginGrant; source: "plugin"; } | { - grant: AgentPluginGrant; + grant: PluginGrant; source: "broker"; }; @@ -40,14 +40,14 @@ export type SandboxEgressCredentialErrorKind = "auth_required" | "unavailable"; /** Signals that egress selected a grant but could not issue credential headers. */ export class SandboxEgressCredentialError extends Error { - readonly authorization?: AgentPluginAuthorization; - readonly grant: AgentPluginGrant; + readonly authorization?: PluginAuthorization; + readonly grant: PluginGrant; readonly kind: SandboxEgressCredentialErrorKind; readonly provider: string; constructor(input: { - authorization?: AgentPluginAuthorization; - grant: AgentPluginGrant; + authorization?: PluginAuthorization; + grant: PluginGrant; kind: SandboxEgressCredentialErrorKind; message: string; provider: string; @@ -65,7 +65,7 @@ function defaultGrantForProvider(input: { method: string; provider: string; }): SandboxEgressGrantSelection { - const access: AgentPluginGrant["access"] = HTTP_READ_METHODS.has( + const access: PluginGrant["access"] = HTTP_READ_METHODS.has( input.method.toUpperCase(), ) ? "read" @@ -82,7 +82,7 @@ function defaultGrantForProvider(input: { function oauthAuthorizationForProvider( provider: string, -): AgentPluginAuthorization | undefined { +): PluginAuthorization | undefined { const oauth = getPluginOAuthConfig(provider); return oauth ? { @@ -143,7 +143,7 @@ export async function selectSandboxEgressGrant(input: { export function authorizationForSandboxEgressGrant( provider: string, selection: SandboxEgressGrantSelection, -): AgentPluginAuthorization | undefined { +): PluginAuthorization | undefined { return selection.source === "broker" ? oauthAuthorizationForProvider(provider) : undefined; @@ -175,7 +175,7 @@ export async function sandboxEgressCredentialLease( let lease: { account?: SandboxEgressCredentialLease["account"]; - authorization?: AgentPluginAuthorization; + authorization?: PluginAuthorization; expiresAt: string; headerTransforms?: SandboxEgressCredentialLease["headerTransforms"]; }; diff --git a/packages/junior/src/chat/sandbox/egress-schemas.ts b/packages/junior/src/chat/sandbox/egress-schemas.ts index 2e85a6ea2..e5dfd9067 100644 --- a/packages/junior/src/chat/sandbox/egress-schemas.ts +++ b/packages/junior/src/chat/sandbox/egress-schemas.ts @@ -1,10 +1,10 @@ import { z } from "zod"; import { credentialContextSchema } from "@/chat/credentials/context"; import { - agentPluginAuthorizationSchema, - agentPluginCredentialHeaderTransformSchema, - agentPluginGrantSchema, - agentPluginProviderAccountSchema, + pluginAuthorizationSchema, + pluginCredentialHeaderTransformSchema, + pluginGrantSchema, + pluginProviderAccountSchema, } from "@sentry/junior-plugin-api"; const finiteNumberSchema = z.number().refine(Number.isFinite); @@ -12,7 +12,7 @@ const httpStatusSchema = z.number().int().min(100).max(599); const providerNameSchema = z.string().regex(/^[a-z][a-z0-9-]*$/); const credentialSignalKindSchema = z.enum(["auth_required", "unavailable"]); -export const sandboxEgressGrantSchema = agentPluginGrantSchema; +export const sandboxEgressGrantSchema = pluginGrantSchema; export const sandboxEgressCredentialContextSchema = z .object({ @@ -25,20 +25,18 @@ export const sandboxEgressCredentialContextSchema = z export const sandboxEgressCredentialLeaseSchema = z .object({ - account: agentPluginProviderAccountSchema.optional(), - authorization: agentPluginAuthorizationSchema.optional(), + account: pluginProviderAccountSchema.optional(), + authorization: pluginAuthorizationSchema.optional(), grant: sandboxEgressGrantSchema, provider: providerNameSchema, expiresAt: z.string().min(1), - headerTransforms: z - .array(agentPluginCredentialHeaderTransformSchema) - .min(1), + headerTransforms: z.array(pluginCredentialHeaderTransformSchema).min(1), }) .strict(); export const sandboxEgressAuthRequiredSignalSchema = z .object({ - authorization: agentPluginAuthorizationSchema.optional(), + authorization: pluginAuthorizationSchema.optional(), grant: sandboxEgressGrantSchema, kind: credentialSignalKindSchema.default("auth_required"), provider: providerNameSchema, @@ -61,7 +59,7 @@ export const sandboxEgressAuthRequiredSignalSchema = z export const sandboxEgressPermissionDeniedSignalSchema = z .object({ - account: agentPluginProviderAccountSchema.optional(), + account: pluginProviderAccountSchema.optional(), acceptedPermissions: z.string().optional(), grant: sandboxEgressGrantSchema, message: z.string().min(1), diff --git a/packages/junior/src/chat/sandbox/sandbox.ts b/packages/junior/src/chat/sandbox/sandbox.ts index ab8e214d9..c2035c12a 100644 --- a/packages/junior/src/chat/sandbox/sandbox.ts +++ b/packages/junior/src/chat/sandbox/sandbox.ts @@ -27,7 +27,7 @@ import { } from "@/chat/sandbox/errors"; import { SANDBOX_WORKSPACE_ROOT } from "@/chat/sandbox/paths"; import { createSandboxSessionManager } from "@/chat/sandbox/session"; -import type { AgentPluginHookRunner } from "@/chat/plugins/agent-hooks"; +import type { PluginHookRunner } from "@/chat/plugins/agent-hooks"; import { isHostFileMissingError, resolveHostDataPath, @@ -140,7 +140,7 @@ export function createSandboxExecutor(options?: { traceContext?: LogContext; tracePropagation?: SandboxEgressTracePropagationConfig; credentialEgress?: CredentialContext; - agentHooks?: AgentPluginHookRunner; + agentHooks?: PluginHookRunner; onSandboxAcquired?: (sandbox: SandboxAcquiredState) => void | Promise; runBashCustomCommand?: ( command: string, @@ -306,8 +306,10 @@ export function createSandboxExecutor(options?: { // side-channel from the network layer — not a property of shell exit status — // and `clearSandboxEgressSignals` runs before each execution to prevent // cross-command leakage. - const authRequired = await consumeSandboxEgressAuthRequiredSignal(activeEgressId); - const permissionDenied = await consumeSandboxEgressPermissionDeniedSignal(activeEgressId); + const authRequired = + await consumeSandboxEgressAuthRequiredSignal(activeEgressId); + const permissionDenied = + await consumeSandboxEgressPermissionDeniedSignal(activeEgressId); return { result: { diff --git a/packages/junior/src/chat/slack/footer.ts b/packages/junior/src/chat/slack/footer.ts index 7f1fe7271..23bb06599 100644 --- a/packages/junior/src/chat/slack/footer.ts +++ b/packages/junior/src/chat/slack/footer.ts @@ -1,5 +1,5 @@ import { buildSentryConversationUrl } from "@/chat/sentry-links"; -import { getAgentPluginSlackConversationLink } from "@/chat/plugins/agent-hooks"; +import { getPluginSlackConversationLink } from "@/chat/plugins/agent-hooks"; interface SlackMrkdwnTextObject { text: string; @@ -68,7 +68,7 @@ export function buildSlackReplyFooter(args: { value: conversationId, }; const conversationUrl = - getAgentPluginSlackConversationLink(conversationId)?.url ?? + getPluginSlackConversationLink(conversationId)?.url ?? buildSentryConversationUrl(conversationId); if (conversationUrl) { idItem.url = conversationUrl; diff --git a/packages/junior/src/chat/tools/agent-tools.ts b/packages/junior/src/chat/tools/agent-tools.ts index 1d250a16d..a8206e17a 100644 --- a/packages/junior/src/chat/tools/agent-tools.ts +++ b/packages/junior/src/chat/tools/agent-tools.ts @@ -21,7 +21,7 @@ import type { ToolDefinition } from "@/chat/tools/definition"; import { buildSandboxInput } from "@/chat/tools/execution/build-sandbox-input"; import { normalizeToolResult } from "@/chat/tools/execution/normalize-result"; import { handleToolExecutionError } from "@/chat/tools/execution/tool-error-handler"; -import type { AgentPluginHookRunner } from "@/chat/plugins/agent-hooks"; +import type { PluginHookRunner } from "@/chat/plugins/agent-hooks"; /** Wrap tool definitions into Pi Agent tool objects with logging, validation, and sandbox execution. */ export function createAgentTools( @@ -32,7 +32,7 @@ export function createAgentTools( sandboxExecutor?: SandboxExecutor, pluginAuthOrchestration?: PluginAuthOrchestration, onToolCall?: (toolName: string, params: Record) => void, - agentHooks?: AgentPluginHookRunner, + agentHooks?: PluginHookRunner, conversationPrivacy?: ConversationPrivacy, ): AgentTool[] { const shouldTrace = shouldEmitDevAgentTrace(); @@ -123,7 +123,9 @@ export function createAgentTools( const normalized = normalizeToolResult(result, isSandbox); if (isSandbox && pluginAuthOrchestration) { - await pluginAuthOrchestration.maybeHandleAuthSignal(normalized.details); + await pluginAuthOrchestration.maybeHandleAuthSignal( + normalized.details, + ); } const resultAttributeValue = normalized.details && diff --git a/packages/junior/src/chat/tools/execution/tool-error-handler.ts b/packages/junior/src/chat/tools/execution/tool-error-handler.ts index e00af98b0..1b4169ad2 100644 --- a/packages/junior/src/chat/tools/execution/tool-error-handler.ts +++ b/packages/junior/src/chat/tools/execution/tool-error-handler.ts @@ -5,7 +5,7 @@ import { setSpanAttributes, type LogContext, } from "@/chat/logging"; -import { AgentPluginToolInputError } from "@sentry/junior-plugin-api"; +import { PluginToolInputError } from "@sentry/junior-plugin-api"; import { GEN_AI_PROVIDER_NAME } from "@/chat/pi/client"; import type { ConversationPrivacy } from "@/chat/conversation-privacy"; import { getMcpAwareTelemetryMessage, McpToolError } from "@/chat/mcp/errors"; @@ -15,8 +15,8 @@ import { ToolInputError } from "@/chat/tools/execution/tool-input-error"; function isPluginToolInputError(error: unknown): boolean { return ( - error instanceof AgentPluginToolInputError || - (error instanceof Error && error.name === "AgentPluginToolInputError") + error instanceof PluginToolInputError || + (error instanceof Error && error.name === "PluginToolInputError") ); } diff --git a/packages/junior/src/chat/tools/index.ts b/packages/junior/src/chat/tools/index.ts index 799fbbefa..45842e759 100644 --- a/packages/junior/src/chat/tools/index.ts +++ b/packages/junior/src/chat/tools/index.ts @@ -38,7 +38,7 @@ import type { ToolRuntimeContext, ToolState, } from "@/chat/tools/types"; -import { getAgentPluginTools } from "@/chat/plugins/agent-hooks"; +import { getPluginTools } from "@/chat/plugins/agent-hooks"; import { createWebFetchTool } from "@/chat/tools/web/fetch-tool"; import { createWebSearchTool } from "@/chat/tools/web/search"; import { createWriteFileTool } from "@/chat/tools/sandbox/write-file"; @@ -162,9 +162,7 @@ export function createTools( } } - for (const [name, pluginTool] of Object.entries( - getAgentPluginTools(context), - )) { + for (const [name, pluginTool] of Object.entries(getPluginTools(context))) { if (tools[name]) { throw new Error(`Plugin tool "${name}" conflicts with a core tool`); } diff --git a/packages/junior/src/cli/upgrade.ts b/packages/junior/src/cli/upgrade.ts index b2f9b85a2..ab1ef6915 100644 --- a/packages/junior/src/cli/upgrade.ts +++ b/packages/junior/src/cli/upgrade.ts @@ -6,6 +6,9 @@ import { requireConversationSqlDatabaseUrl, sqlConversationMigration, } from "./upgrade/migrations/conversations-sql"; +import { pluginStorageMigration } from "./upgrade/migrations/plugin-storage"; +import { sqlPluginMigration } from "./upgrade/migrations/plugin-sql"; +import { resolveUpgradePlugins } from "./upgrade/migrations/upgrade-plugins"; import { redisConversationStateMigration } from "./upgrade/migrations/redis-conversation-state"; import type { MigrationContext, @@ -13,6 +16,7 @@ import type { UpgradeIo, UpgradeMigration, } from "./upgrade/types"; +import { type JuniorPluginSet } from "@/plugins"; const DEFAULT_IO: UpgradeIo = { info: console.log, @@ -21,8 +25,37 @@ const DEFAULT_IO: UpgradeIo = { const MIGRATIONS: UpgradeMigration[] = [ redisConversationStateMigration, sqlConversationMigration, + sqlPluginMigration, + pluginStorageMigration, ]; +function isMissingVirtualConfig(error: unknown): boolean { + if (!(error instanceof Error)) { + return false; + } + const code = (error as { code?: string }).code; + return ( + (code === "ERR_PACKAGE_IMPORT_NOT_DEFINED" || + code === "ERR_MODULE_NOT_FOUND" || + code === "MODULE_NOT_FOUND") && + error.message.includes("#junior/config") + ); +} + +async function resolveUpgradePluginSet(): Promise { + try { + const mod: { + pluginSet?: JuniorPluginSet; + } = await import("#junior/config"); + return mod.pluginSet; + } catch (error) { + if (!isMissingVirtualConfig(error)) { + throw error; + } + return undefined; + } +} + function formatMigrationResult(result: MigrationResult): string { const fields = [ `scanned=${result.scanned}`, @@ -40,12 +73,14 @@ function formatMigrationResult(result: MigrationResult): string { export async function runUpgradeMigrations( context: MigrationContext, ): Promise { - requireConversationSqlDatabaseUrl(context); + const plugins = await resolveUpgradePlugins(context); + const migrationContext = { ...context, ...plugins }; + requireConversationSqlDatabaseUrl(migrationContext); const results: MigrationResult[] = []; for (const migration of MIGRATIONS) { - context.io.info(`Running migration ${migration.name}...`); - const result = await migration.run(context); - context.io.info( + migrationContext.io.info(`Running migration ${migration.name}...`); + const result = await migration.run(migrationContext); + migrationContext.io.info( `Finished migration ${migration.name}: ${formatMigrationResult(result)}`, ); results.push(result); @@ -58,8 +93,14 @@ export async function runUpgrade(io: UpgradeIo = DEFAULT_IO): Promise { try { const { redisStateAdapter, stateAdapter } = await getConnectedStateContext(); + const pluginSet = await resolveUpgradePluginSet(); io.info("Running Junior upgrade migrations..."); - await runUpgradeMigrations({ io, redisStateAdapter, stateAdapter }); + await runUpgradeMigrations({ + io, + pluginSet, + redisStateAdapter, + stateAdapter, + }); io.info("Junior upgrade complete."); } finally { await disconnectStateAdapter(); diff --git a/packages/junior/src/cli/upgrade/migrations/plugin-sql.ts b/packages/junior/src/cli/upgrade/migrations/plugin-sql.ts new file mode 100644 index 000000000..bf7978c35 --- /dev/null +++ b/packages/junior/src/cli/upgrade/migrations/plugin-sql.ts @@ -0,0 +1,52 @@ +import { getChatConfig } from "@/chat/config"; +import { migratePluginSchemas, readPluginMigrations } from "@/chat/plugins/db"; +import { + getPluginMigrationRoots, + setPluginCatalogConfig, +} from "@/chat/plugins/registry"; +import { createNeonJuniorSqlExecutor } from "@/chat/sql/neon"; +import { resolveUpgradePlugins } from "./upgrade-plugins"; +import type { MigrationContext, MigrationResult } from "../types"; + +const REQUIRED_SQL_DATABASE_URL_MESSAGE = + "Junior SQL database URL is required for plugin schema migration. Set JUNIOR_DATABASE_URL or DATABASE_URL."; + +function requirePluginSqlDatabaseUrl(context: MigrationContext): string { + const databaseUrl = context.sqlDatabaseUrl ?? getChatConfig().sql.databaseUrl; + if (!databaseUrl) { + throw new Error(REQUIRED_SQL_DATABASE_URL_MESSAGE); + } + return databaseUrl; +} + +/** Apply SQL schema migrations owned by explicitly enabled plugins. */ +export async function migratePluginsToSql( + context: MigrationContext, +): Promise { + const databaseUrl = requirePluginSqlDatabaseUrl(context); + const { pluginCatalogConfig } = await resolveUpgradePlugins(context); + const previousConfig = setPluginCatalogConfig(pluginCatalogConfig); + const executor = createNeonJuniorSqlExecutor({ + connectionString: databaseUrl, + }); + try { + const migrations = getPluginMigrationRoots().flatMap((root) => + readPluginMigrations(root), + ); + const result = await migratePluginSchemas(executor, migrations); + return { + existing: result.existing, + migrated: result.migrated, + missing: 0, + scanned: result.scanned, + }; + } finally { + setPluginCatalogConfig(previousConfig); + await executor.close(); + } +} + +export const sqlPluginMigration = { + name: "migrate-plugin-sql", + run: migratePluginsToSql, +}; diff --git a/packages/junior/src/cli/upgrade/migrations/plugin-storage.ts b/packages/junior/src/cli/upgrade/migrations/plugin-storage.ts new file mode 100644 index 000000000..53b4c0daf --- /dev/null +++ b/packages/junior/src/cli/upgrade/migrations/plugin-storage.ts @@ -0,0 +1,112 @@ +import type { + PluginDb, + PluginRegistration, + StorageMigrationResult, +} from "@sentry/junior-plugin-api"; +import { pluginHookRegistrationsFromPluginSet } from "@/plugins"; +import { + createPluginDbForExecutor, + getPluginDbForRegistration, +} from "@/chat/plugins/db"; +import { createPluginLogger } from "@/chat/plugins/logging"; +import { createPluginState } from "@/chat/plugins/state"; +import { setPluginCatalogConfig } from "@/chat/plugins/registry"; +import { createNeonJuniorSqlExecutor } from "@/chat/sql/neon"; +import { resolveUpgradePlugins } from "./upgrade-plugins"; +import type { MigrationContext, MigrationResult } from "../types"; + +function emptyResult(): MigrationResult { + return { + existing: 0, + migrated: 0, + missing: 0, + scanned: 0, + }; +} + +function addResult( + left: MigrationResult, + right: StorageMigrationResult, +): MigrationResult { + return { + existing: left.existing + right.existing, + migrated: left.migrated + right.migrated, + missing: left.missing + right.missing, + scanned: left.scanned + right.scanned, + ...(left.skipped !== undefined || right.skipped !== undefined + ? { skipped: (left.skipped ?? 0) + (right.skipped ?? 0) } + : {}), + }; +} + +function dbForPlugin( + context: MigrationContext, + plugin: PluginRegistration, + sqlUrlDb: PluginDb | undefined, +): PluginDb | undefined { + if (!plugin.database) { + return undefined; + } + return context.pluginDb ?? sqlUrlDb ?? getPluginDbForRegistration(plugin); +} + +/** Run plugin-owned storage migrations after plugin SQL schemas are available. */ +export async function runPluginStorageMigrations( + context: MigrationContext, +): Promise { + const { pluginCatalogConfig, pluginSet } = + await resolveUpgradePlugins(context); + if (!pluginSet) { + return emptyResult(); + } + + const previousConfig = setPluginCatalogConfig(pluginCatalogConfig); + const ownedExecutor = + context.pluginDb || !context.sqlDatabaseUrl + ? undefined + : createNeonJuniorSqlExecutor({ + connectionString: context.sqlDatabaseUrl, + }); + const sqlUrlDb = ownedExecutor + ? createPluginDbForExecutor(ownedExecutor) + : undefined; + try { + let result = emptyResult(); + const plugins = pluginHookRegistrationsFromPluginSet(pluginSet) + .filter((plugin) => plugin.hooks?.migrateStorage) + .sort((left, right) => + left.manifest.name.localeCompare(right.manifest.name), + ); + for (const plugin of plugins) { + const pluginName = plugin.manifest.name; + const hook = plugin.hooks?.migrateStorage; + if (!hook) { + continue; + } + const db = dbForPlugin(context, plugin, sqlUrlDb); + if (!db) { + throw new Error( + `Plugin "${pluginName}" storage migration requires database access`, + ); + } + const pluginResult = await hook({ + db, + log: createPluginLogger(pluginName), + plugin: { name: pluginName }, + state: createPluginState(pluginName, context.stateAdapter), + }); + if (pluginResult) { + result = addResult(result, pluginResult); + } + } + return result; + } finally { + setPluginCatalogConfig(previousConfig); + await ownedExecutor?.close(); + } +} + +export const pluginStorageMigration = { + name: "run-plugin-storage-migrations", + run: runPluginStorageMigrations, +}; diff --git a/packages/junior/src/cli/upgrade/migrations/upgrade-plugins.ts b/packages/junior/src/cli/upgrade/migrations/upgrade-plugins.ts new file mode 100644 index 000000000..fe6ec84be --- /dev/null +++ b/packages/junior/src/cli/upgrade/migrations/upgrade-plugins.ts @@ -0,0 +1,116 @@ +import type { + InlinePluginManifestDefinition, + PluginCatalogConfig, +} from "@/chat/plugins/types"; +import { + defineJuniorPlugins, + pluginCatalogConfigFromEnv, + pluginCatalogConfigFromPluginSet, + type JuniorPluginSet, +} from "@/plugins"; +import type { MigrationContext } from "../types"; + +interface ResolvedUpgradePlugins { + pluginCatalogConfig?: PluginCatalogConfig; + pluginSet?: JuniorPluginSet; +} + +function unique(values: string[]): string[] { + return [...new Set(values)]; +} + +function baseCatalogConfig( + context: MigrationContext, +): PluginCatalogConfig | undefined { + return ( + context.pluginCatalogConfig ?? + (context.pluginSet + ? pluginCatalogConfigFromPluginSet(context.pluginSet) + : pluginCatalogConfigFromEnv()) + ); +} + +function inlinePluginName(definition: InlinePluginManifestDefinition): string { + return definition.manifest.name; +} + +function mergeInlineManifests( + left: InlinePluginManifestDefinition[] | undefined, + right: InlinePluginManifestDefinition[] | undefined, +): InlinePluginManifestDefinition[] | undefined { + const merged = new Map(); + for (const definition of [...(left ?? []), ...(right ?? [])]) { + merged.set(inlinePluginName(definition), definition); + } + return merged.size > 0 ? [...merged.values()] : undefined; +} + +function mergeCatalogConfig( + base: PluginCatalogConfig | undefined, + added: PluginCatalogConfig | undefined, +): PluginCatalogConfig | undefined { + if (!base) { + return added; + } + if (!added) { + return base; + } + const inlineManifests = mergeInlineManifests( + base.inlineManifests, + added.inlineManifests, + ); + const packages = unique([ + ...(base.packages ?? []), + ...(added.packages ?? []), + ]); + const manifests = + base.manifests || added.manifests + ? { ...base.manifests, ...added.manifests } + : undefined; + return { + ...(inlineManifests ? { inlineManifests } : {}), + ...(packages.length > 0 ? { packages } : {}), + ...(manifests ? { manifests } : {}), + }; +} + +function packageNamesFromContext( + context: MigrationContext, + catalog: PluginCatalogConfig | undefined, +): string[] { + return unique([ + ...(context.pluginSet?.packageNames ?? []), + ...(catalog?.packages ?? []), + ]); +} + +/** Resolve one effective plugin set and catalog for all upgrade migrations. */ +export async function resolveUpgradePlugins( + context: MigrationContext, +): Promise { + const catalog = baseCatalogConfig(context); + const packageNames = packageNamesFromContext(context, catalog); + const registrations = context.pluginSet?.registrations ?? []; + const manifests = + context.pluginSet?.manifests || catalog?.manifests + ? { + ...catalog?.manifests, + ...context.pluginSet?.manifests, + } + : undefined; + const pluginSet = + packageNames.length > 0 || registrations.length > 0 || context.pluginSet + ? defineJuniorPlugins( + [...packageNames, ...registrations], + manifests ? { manifests } : {}, + ) + : undefined; + + return { + pluginCatalogConfig: mergeCatalogConfig( + catalog, + pluginCatalogConfigFromPluginSet(pluginSet), + ), + ...(pluginSet ? { pluginSet } : {}), + }; +} diff --git a/packages/junior/src/cli/upgrade/types.ts b/packages/junior/src/cli/upgrade/types.ts index 6d826276a..185b48b37 100644 --- a/packages/junior/src/cli/upgrade/types.ts +++ b/packages/junior/src/cli/upgrade/types.ts @@ -1,5 +1,8 @@ import type { RedisStateAdapter } from "@chat-adapter/state-redis"; import type { StateAdapter } from "chat"; +import type { PluginDb } from "@sentry/junior-plugin-api"; +import type { PluginCatalogConfig } from "@/chat/plugins/types"; +import type { JuniorPluginSet } from "@/plugins"; export interface UpgradeIo { info: (line: string) => void; @@ -7,6 +10,9 @@ export interface UpgradeIo { export interface MigrationContext { io: UpgradeIo; + pluginDb?: PluginDb; + pluginCatalogConfig?: PluginCatalogConfig; + pluginSet?: JuniorPluginSet; sqlDatabaseUrl?: string; redisStateAdapter?: RedisStateAdapter; stateAdapter: StateAdapter; diff --git a/packages/junior/src/nitro.ts b/packages/junior/src/nitro.ts index 12d308205..4774433b7 100644 --- a/packages/junior/src/nitro.ts +++ b/packages/junior/src/nitro.ts @@ -183,7 +183,7 @@ async function loadPluginSetFromModule( function assertSerializableDirectPluginSet(pluginSet: JuniorPluginSet): void { const pluginHookNames = pluginHookRegistrationsFromPluginSet(pluginSet).map( - (plugin) => plugin.name, + (plugin) => plugin.manifest.name, ); if (pluginHookNames.length === 0) { return; @@ -311,7 +311,7 @@ export function juniorNitro(options: JuniorNitroOptions = {}): { pluginCatalogConfigFromPluginSet(directPluginSet); const pluginHookRegistrations = pluginHookRegistrationsFromPluginSet( directPluginSet, - ).map((plugin) => plugin.name); + ).map((plugin) => plugin.manifest.name); injectVirtualConfig(nitro, { ...(pluginModule ? { diff --git a/packages/junior/src/plugins.ts b/packages/junior/src/plugins.ts index d791ca09b..722db0be0 100644 --- a/packages/junior/src/plugins.ts +++ b/packages/junior/src/plugins.ts @@ -1,11 +1,11 @@ -import type { JuniorPluginRegistration } from "@sentry/junior-plugin-api"; +import type { PluginRegistration } from "@sentry/junior-plugin-api"; import type { InlinePluginManifestDefinition, PluginCatalogConfig, PluginManifestConfig, } from "./chat/plugins/types"; -export type JuniorPluginInput = JuniorPluginRegistration | string; +export type JuniorPluginInput = PluginRegistration | string; export interface JuniorPluginSetOptions { /** Install-level manifest overrides applied before validation. */ @@ -19,7 +19,7 @@ export interface JuniorPluginSet { /** Manifest-only plugin packages included by package name. */ packageNames: string[]; /** JavaScript plugin definitions included by package factories. */ - registrations: JuniorPluginRegistration[]; + registrations: PluginRegistration[]; } function cloneManifests( @@ -29,7 +29,7 @@ function cloneManifests( } function cloneInlineManifests( - registrations: JuniorPluginRegistration[], + registrations: PluginRegistration[], ): InlinePluginManifestDefinition[] | undefined { const inlineManifests = registrations.flatMap((plugin) => plugin.manifest @@ -41,11 +41,11 @@ function cloneInlineManifests( plugin.manifest.capabilities?.map((capability) => capability.includes(".") ? capability - : `${plugin.manifest!.name}.${capability}`, + : `${plugin.manifest.name}.${capability}`, ) ?? [], configKeys: plugin.manifest.configKeys?.map((key) => - key.includes(".") ? key : `${plugin.manifest!.name}.${key}`, + key.includes(".") ? key : `${plugin.manifest.name}.${key}`, ) ?? [], ...(plugin.manifest.target ? { @@ -66,15 +66,14 @@ function cloneInlineManifests( return inlineManifests.length > 0 ? inlineManifests : undefined; } -function assertUniquePluginNames( - registrations: JuniorPluginRegistration[], -): void { +function assertUniquePluginNames(registrations: PluginRegistration[]): void { const seen = new Set(); for (const plugin of registrations) { - if (seen.has(plugin.name)) { - throw new Error(`Duplicate plugin registration name "${plugin.name}"`); + const name = plugin.manifest.name; + if (seen.has(name)) { + throw new Error(`Duplicate plugin registration name "${name}"`); } - seen.add(plugin.name); + seen.add(name); } } @@ -90,7 +89,7 @@ function assertUniquePackageNames(packageNames: string[]): void { function normalizePluginInput(input: JuniorPluginInput): { packageName?: string; - registration?: JuniorPluginRegistration; + registration?: PluginRegistration; } { if (typeof input === "string") { return { packageName: input }; @@ -150,13 +149,50 @@ export function pluginCatalogConfigFromPluginSet( }; } +function readEnvPluginPackages( + env: NodeJS.ProcessEnv = process.env, +): string[] | undefined { + const value = env.JUNIOR_PLUGIN_PACKAGES; + if (!value) { + return undefined; + } + + let parsed: unknown; + try { + parsed = JSON.parse(value); + } catch (error) { + throw new Error("JUNIOR_PLUGIN_PACKAGES must be valid JSON", { + cause: error, + }); + } + + if ( + !Array.isArray(parsed) || + parsed.some((item) => typeof item !== "string" || !item.trim()) + ) { + throw new Error( + "JUNIOR_PLUGIN_PACKAGES must be a JSON array of package names", + ); + } + + return parsed; +} + +/** Build the manifest catalog config implied by plugin package env. */ +export function pluginCatalogConfigFromEnv( + env: NodeJS.ProcessEnv = process.env, +): PluginCatalogConfig | undefined { + const packages = readEnvPluginPackages(env); + return packages ? { packages } : undefined; +} + /** Return registrations that expose in-process runtime hooks. */ export function pluginHookRegistrationsFromPluginSet( pluginSet: JuniorPluginSet | undefined, -): JuniorPluginRegistration[] { +): PluginRegistration[] { return ( pluginSet?.registrations.filter( - (plugin) => plugin.hooks || plugin.legacyStatePrefixes, + (plugin) => plugin.database || plugin.hooks, ) ?? [] ); } diff --git a/packages/junior/src/reporting.ts b/packages/junior/src/reporting.ts index 2cd4b75dd..c5d9bfb56 100644 --- a/packages/junior/src/reporting.ts +++ b/packages/junior/src/reporting.ts @@ -4,7 +4,7 @@ import { getPluginPackageContent, getPluginProviders, } from "@/chat/plugins/registry"; -import { getAgentPluginOperationalReports } from "@/chat/plugins/agent-hooks"; +import { getPluginOperationalReports } from "@/chat/plugins/agent-hooks"; import { discoverSkills } from "@/chat/skills"; import { homeDir } from "@/chat/discovery"; import { GET as healthGET } from "@/handlers/health"; @@ -16,15 +16,15 @@ import { readConversationStatsReport, listRecentConversationSummaries, type ConversationFeed, - type AgentPluginConversationSummary, + type PluginConversationSummary, type ConversationReport, type ConversationStatsReport, } from "./reporting/conversations"; export type { - AgentPluginConversationStatus, - AgentPluginConversations, - AgentPluginConversationSummary, + PluginConversationStatus, + PluginConversations, + PluginConversationSummary, ConversationFeed, ConversationReport, ConversationReportStatus, @@ -67,8 +67,9 @@ export interface RuntimeInfoReport { export interface PluginPackageContentItemReport { dir: string; + hasMigrationsDir: boolean; hasSkillsDir: boolean; - name: string; + packageName: string; } export interface PluginPackageContentReport { @@ -103,7 +104,7 @@ export interface JuniorReporting { /** Read recent conversation summaries without transcript payloads. */ listRecentConversations?(options?: { limit?: number; - }): Promise; + }): Promise; /** Read sanitized operational summaries contributed by plugins. */ getPluginOperationalReports?(): Promise; /** @@ -152,7 +153,7 @@ export function createJuniorReporting(): JuniorReporting & { getConversationStats(): Promise; listRecentConversations(options?: { limit?: number; - }): Promise; + }): Promise; getPluginOperationalReports(): Promise; } { const conversationStore = getConfiguredConversationStore(); @@ -189,7 +190,7 @@ export function createJuniorReporting(): JuniorReporting & { return { source: "plugins", generatedAt: new Date(nowMs).toISOString(), - reports: await getAgentPluginOperationalReports(nowMs, { + reports: await getPluginOperationalReports(nowMs, { listRecent, }), }; diff --git a/packages/junior/src/reporting/conversations.ts b/packages/junior/src/reporting/conversations.ts index 97cc27e53..553359f1d 100644 --- a/packages/junior/src/reporting/conversations.ts +++ b/packages/junior/src/reporting/conversations.ts @@ -13,9 +13,9 @@ import { import type { PiMessage } from "@/chat/pi/messages"; import { buildSystemPrompt } from "@/chat/prompt"; import type { - AgentPluginConversationStatus, - AgentPluginConversations, - AgentPluginConversationSummary, + PluginConversationStatus, + PluginConversations, + PluginConversationSummary, Destination, } from "@sentry/junior-plugin-api"; import { @@ -47,9 +47,9 @@ import type { } from "@/chat/conversations/store"; export type { - AgentPluginConversationStatus, - AgentPluginConversations, - AgentPluginConversationSummary, + PluginConversationStatus, + PluginConversations, + PluginConversationSummary, }; const HUNG_TURN_PROGRESS_MS = 5 * 60 * 1000; @@ -1277,7 +1277,7 @@ export async function listRecentConversationSummaries( options: { limit?: number; } & ConversationReaderOptions = {}, -): Promise { +): Promise { const store = conversationStore(options); const nowMs = Date.now(); const limit = Math.max(0, Math.min(100, Math.floor(options.limit ?? 25))); diff --git a/packages/junior/tests/component/plugin-db-migrations.test.ts b/packages/junior/tests/component/plugin-db-migrations.test.ts new file mode 100644 index 000000000..6f31733a1 --- /dev/null +++ b/packages/junior/tests/component/plugin-db-migrations.test.ts @@ -0,0 +1,41 @@ +import { describe, expect, it } from "vitest"; +import { migratePluginSchemas, type PluginMigration } from "@/chat/plugins/db"; +import { createLocalJuniorSqlFixture } from "../fixtures/sql"; + +function migration(overrides: Partial = {}): PluginMigration { + return { + checksum: "checksum-1", + filename: "0001_init.sql", + id: "plugin:memory/0001_init.sql", + pluginName: "memory", + sql: "CREATE TABLE junior_memory_test (id TEXT PRIMARY KEY);", + ...overrides, + }; +} + +describe("plugin DB migrations", () => { + it("applies pending plugin migrations against local SQL", async () => { + const fixture = await createLocalJuniorSqlFixture(); + const pending = migration(); + + try { + const result = await migratePluginSchemas(fixture.executor, [pending]); + + expect(result).toEqual({ existing: 0, migrated: 1, scanned: 1 }); + await fixture.executor.execute( + "INSERT INTO junior_memory_test (id) VALUES ($1)", + ["row-1"], + ); + await expect( + fixture.executor.query("SELECT id FROM junior_memory_test"), + ).resolves.toEqual([{ id: "row-1" }]); + await expect( + fixture.executor.query( + "SELECT id, checksum FROM junior_schema_migrations ORDER BY id ASC", + ), + ).resolves.toEqual([{ id: pending.id, checksum: pending.checksum }]); + } finally { + await fixture.close(); + } + }); +}); diff --git a/packages/junior/tests/component/scheduler-sql-plugin.test.ts b/packages/junior/tests/component/scheduler-sql-plugin.test.ts new file mode 100644 index 000000000..b2e6d321e --- /dev/null +++ b/packages/junior/tests/component/scheduler-sql-plugin.test.ts @@ -0,0 +1,737 @@ +import path from "node:path"; +import { createMemoryState } from "@chat-adapter/state-memory"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { defineJuniorPlugin } from "@sentry/junior-plugin-api"; +import { + createSchedulerSqlStore, + schedulerPlugin, + type ScheduledTask, +} from "@sentry/junior-scheduler"; +import { createSchedulerStore } from "../../../junior-scheduler/src/store"; +import { defineJuniorPlugins } from "@/plugins"; +import { + createPluginDbForExecutor, + migratePluginSchemas, + readPluginMigrations, +} from "@/chat/plugins/db"; +import { createPluginState } from "@/chat/plugins/state"; +import { disconnectStateAdapter, getStateAdapter } from "@/chat/state/adapter"; +import { runPluginStorageMigrations } from "@/cli/upgrade/migrations/plugin-storage"; +import { migratePluginsToSql } from "@/cli/upgrade/migrations/plugin-sql"; +import { createLocalJuniorSqlFixture } from "../fixtures/sql"; + +const NEON = vi.hoisted(() => ({ + executor: undefined as + | Awaited>["executor"] + | undefined, +})); + +vi.hoisted(() => { + process.env.JUNIOR_STATE_ADAPTER = "memory"; +}); + +vi.mock("@/chat/sql/neon", () => ({ + createNeonJuniorSqlExecutor: vi.fn(() => { + if (!NEON.executor) { + throw new Error("Missing test SQL executor"); + } + return { + db: NEON.executor.db.bind(NEON.executor), + execute: NEON.executor.execute.bind(NEON.executor), + query: NEON.executor.query.bind(NEON.executor), + transaction: NEON.executor.transaction.bind(NEON.executor), + withLock: NEON.executor.withLock.bind(NEON.executor), + close: async () => {}, + }; + }), +})); + +const TEST_RUN_AT_MS = Date.parse("2026-05-26T12:00:00.000Z"); +const TEST_NOW_MS = Date.parse("2026-05-26T12:05:00.000Z"); + +function schedulerMigrationsDir(): string { + return path.resolve(process.cwd(), "../junior-scheduler/migrations"); +} + +async function migrateSchedulerSchema( + fixture: Awaited>, +) { + await migratePluginSchemas( + fixture.executor, + readPluginMigrations({ + dir: schedulerMigrationsDir(), + pluginName: "scheduler", + }), + ); +} + +function createTask(overrides: Partial = {}): ScheduledTask { + return { + id: "sched_sql_1", + createdAtMs: TEST_RUN_AT_MS, + createdBy: { slackUserId: "U123" }, + destination: { + platform: "slack", + teamId: "T123", + channelId: "C123", + }, + nextRunAtMs: TEST_RUN_AT_MS, + schedule: { + description: "Once at noon", + kind: "one_off", + timezone: "UTC", + }, + status: "active", + task: { + text: "Post a digest.", + }, + updatedAtMs: TEST_RUN_AT_MS, + version: 1, + ...overrides, + }; +} + +describe("scheduler SQL plugin storage", () => { + afterEach(async () => { + NEON.executor = undefined; + await disconnectStateAdapter(); + }); + + it("persists and claims scheduled runs through the plugin SQL database", async () => { + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const task = createTask(); + + await store.saveTask(task); + + await expect(store.listTasksForTeam("T123")).resolves.toMatchObject([ + { id: task.id }, + ]); + const run = await store.claimDueRun({ nowMs: TEST_NOW_MS }); + expect(run).toMatchObject({ + taskId: task.id, + scheduledForMs: TEST_RUN_AT_MS, + status: "pending", + }); + + const dispatched = await store.markRunDispatched({ + claimedAtMs: run!.claimedAtMs, + dispatchId: "dispatch_1", + nowMs: TEST_NOW_MS + 1, + runId: run!.id, + }); + expect(dispatched).toMatchObject({ status: "running" }); + + const completed = await store.markRunCompleted({ + completedAtMs: TEST_NOW_MS + 2, + resultMessageTs: "1718123456.000000", + runId: run!.id, + startedAtMs: dispatched!.startedAtMs!, + }); + expect(completed).toMatchObject({ status: "completed" }); + + await store.updateTaskAfterRun({ + nowMs: TEST_NOW_MS + 3, + run: completed!, + status: "completed", + }); + + await expect(store.getTask(task.id)).resolves.toMatchObject({ + id: task.id, + lastRunAtMs: TEST_RUN_AT_MS, + status: "paused", + }); + } finally { + await fixture.close(); + } + }, 15_000); + + it("claims later due runs when an older pending run is stale", async () => { + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const taskId = "sched_sql_stale_pending"; + const staleRunAtMs = TEST_NOW_MS - 2 * 60 * 1000; + const nextRunAtMs = TEST_NOW_MS - 30 * 1000; + const task = createTask({ + id: taskId, + nextRunAtMs: staleRunAtMs, + }); + + await store.saveTask(task); + const staleRun = await store.claimDueRun({ nowMs: staleRunAtMs }); + expect(staleRun).toMatchObject({ + id: `${taskId}:${staleRunAtMs}`, + status: "pending", + }); + + await store.saveTask({ + ...task, + nextRunAtMs, + updatedAtMs: TEST_NOW_MS, + }); + const nextRun = await store.claimDueRun({ nowMs: TEST_NOW_MS }); + + expect(nextRun).toMatchObject({ + id: `${taskId}:${nextRunAtMs}`, + scheduledForMs: nextRunAtMs, + status: "pending", + }); + await expect(store.getRun(staleRun!.id)).resolves.toMatchObject({ + status: "pending", + }); + } finally { + await fixture.close(); + } + }, 15_000); + + it("does not reclaim completed SQL run slots", async () => { + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const task = createTask({ id: "sched_sql_completed_slot" }); + + await store.saveTask(task); + const run = await store.claimDueRun({ nowMs: TEST_NOW_MS }); + expect(run).toMatchObject({ + id: `${task.id}:${TEST_RUN_AT_MS}`, + status: "pending", + }); + + const dispatched = await store.markRunDispatched({ + claimedAtMs: run!.claimedAtMs, + dispatchId: "dispatch_completed_slot", + nowMs: TEST_NOW_MS + 1, + runId: run!.id, + }); + await expect( + store.markRunCompleted({ + completedAtMs: TEST_NOW_MS + 2, + runId: run!.id, + startedAtMs: dispatched!.startedAtMs!, + }), + ).resolves.toMatchObject({ + id: run!.id, + status: "completed", + }); + + await expect(store.claimDueRun({ nowMs: TEST_NOW_MS + 3 })).resolves.toBe( + undefined, + ); + await expect(store.getRun(run!.id)).resolves.toMatchObject({ + id: run!.id, + status: "completed", + }); + } finally { + await fixture.close(); + } + }, 15_000); + + it("reclaims blocked SQL run slots after reactivation", async () => { + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const task = createTask({ id: "sched_sql_blocked_slot" }); + + await store.saveTask(task); + const run = await store.claimDueRun({ nowMs: TEST_NOW_MS }); + expect(run).toMatchObject({ + id: `${task.id}:${TEST_RUN_AT_MS}`, + status: "pending", + }); + + await expect( + store.markRunBlocked({ + completedAtMs: TEST_NOW_MS + 1, + errorMessage: "Missing provider authorization.", + runId: run!.id, + }), + ).resolves.toMatchObject({ + id: run!.id, + status: "blocked", + }); + + await store.updateTaskAfterRun({ + errorMessage: "Missing provider authorization.", + nowMs: TEST_NOW_MS + 2, + run: { + ...run!, + completedAtMs: TEST_NOW_MS + 1, + errorMessage: "Missing provider authorization.", + status: "blocked", + }, + status: "blocked", + }); + await expect(store.getTask(task.id)).resolves.toMatchObject({ + id: task.id, + status: "blocked", + }); + + await store.saveTask({ + ...task, + nextRunAtMs: TEST_RUN_AT_MS, + status: "active", + statusReason: undefined, + updatedAtMs: TEST_NOW_MS + 3, + version: task.version + 2, + }); + + await expect( + store.claimDueRun({ nowMs: TEST_NOW_MS + 4 }), + ).resolves.toMatchObject({ + id: `${task.id}:${TEST_RUN_AT_MS}`, + scheduledForMs: TEST_RUN_AT_MS, + status: "pending", + }); + } finally { + await fixture.close(); + } + }, 15_000); + + it("migrates existing scheduler plugin state into SQL idempotently", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const stateStore = createSchedulerStore( + createPluginState("scheduler", stateAdapter), + ); + const task = createTask({ id: "sched_state_sql" }); + await stateStore.saveTask(task); + const run = await stateStore.claimDueRun({ nowMs: TEST_NOW_MS }); + expect(run).toBeDefined(); + + const context = { + io: { info: () => {} }, + pluginDb: db, + pluginSet: defineJuniorPlugins([schedulerPlugin()]), + stateAdapter, + }; + + await expect(runPluginStorageMigrations(context)).resolves.toEqual({ + existing: 0, + migrated: 2, + missing: 0, + scanned: 2, + }); + await expect(runPluginStorageMigrations(context)).resolves.toEqual({ + existing: 2, + migrated: 0, + missing: 0, + scanned: 2, + }); + + const sqlStore = createSchedulerSqlStore(db); + await expect(sqlStore.getTask(task.id)).resolves.toMatchObject({ + id: task.id, + }); + await expect(sqlStore.getRun(run!.id)).resolves.toMatchObject({ + id: run!.id, + taskId: task.id, + }); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }, 15_000); + + it("skips malformed scheduler state records during SQL storage migration", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const state = createPluginState("scheduler", stateAdapter); + const stateStore = createSchedulerStore(state); + const task = createTask({ id: "sched_state_sql_valid_after_bad" }); + const badRunId = `${task.id}:${TEST_RUN_AT_MS}`; + await stateStore.saveTask(task); + await state.set( + "junior:scheduler:tasks", + ["sched_state_sql_bad", task.id], + 5 * 60 * 1000, + ); + await state.set( + "junior:scheduler:task:sched_state_sql_bad", + { + ...task, + id: "sched_state_sql_bad", + task: { text: 123 }, + }, + 5 * 60 * 1000, + ); + await state.set( + `junior:scheduler:active:${task.id}`, + { + claimedAtMs: TEST_NOW_MS, + runId: badRunId, + scheduledForMs: TEST_RUN_AT_MS, + }, + 5 * 60 * 1000, + ); + await state.set( + `junior:scheduler:run:${badRunId}`, + { id: badRunId }, + 5 * 60 * 1000, + ); + + await expect( + runPluginStorageMigrations({ + io: { info: () => {} }, + pluginDb: db, + pluginSet: defineJuniorPlugins([schedulerPlugin()]), + stateAdapter, + }), + ).resolves.toEqual({ + existing: 0, + migrated: 1, + missing: 1, + scanned: 2, + }); + + const sqlStore = createSchedulerSqlStore(db); + await expect(sqlStore.getTask(task.id)).resolves.toMatchObject({ + id: task.id, + }); + await expect(sqlStore.getTask("sched_state_sql_bad")).resolves.toBe( + undefined, + ); + await expect(sqlStore.getRun(badRunId)).resolves.toBe(undefined); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }, 15_000); + + it("does not load scheduler storage migration from package-only plugin set", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const stateStore = createSchedulerStore( + createPluginState("scheduler", stateAdapter), + ); + const task = createTask({ id: "sched_package_config" }); + await stateStore.saveTask(task); + const run = await stateStore.claimDueRun({ nowMs: TEST_NOW_MS }); + expect(run).toBeDefined(); + + await expect( + runPluginStorageMigrations({ + io: { info: () => {} }, + pluginDb: db, + pluginSet: defineJuniorPlugins(["@sentry/junior-scheduler"]), + stateAdapter, + }), + ).resolves.toEqual({ + existing: 0, + migrated: 0, + missing: 0, + scanned: 0, + }); + + const sqlStore = createSchedulerSqlStore(db); + await expect(sqlStore.getTask(task.id)).resolves.toBe(undefined); + await expect(sqlStore.getRun(run!.id)).resolves.toBe(undefined); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }, 15_000); + + it("does not apply scheduler SQL migrations from package-only config", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + NEON.executor = fixture.executor; + + try { + await expect( + migratePluginsToSql({ + io: { info: () => {} }, + pluginCatalogConfig: { packages: ["@sentry/junior-scheduler"] }, + sqlDatabaseUrl: "postgres://configured.example.test/neon", + stateAdapter, + }), + ).resolves.toEqual({ + existing: 0, + migrated: 0, + missing: 0, + scanned: 0, + }); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }); + + it("applies scheduler SQL migrations from registration-only config", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + NEON.executor = fixture.executor; + + try { + await expect( + migratePluginsToSql({ + io: { info: () => {} }, + pluginSet: defineJuniorPlugins([schedulerPlugin()]), + sqlDatabaseUrl: "postgres://configured.example.test/neon", + stateAdapter, + }), + ).resolves.toEqual({ + existing: 0, + migrated: 1, + missing: 0, + scanned: 1, + }); + + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const task = createTask({ id: "sched_schema_registration_config" }); + await store.saveTask(task); + await expect(store.getTask(task.id)).resolves.toMatchObject({ + id: task.id, + }); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }); + + it("does not duplicate scheduler SQL migrations for explicit registrations", async () => { + const stateAdapter = createMemoryState(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + NEON.executor = fixture.executor; + + try { + await expect( + migratePluginsToSql({ + io: { info: () => {} }, + pluginSet: defineJuniorPlugins([ + "@sentry/junior-scheduler", + schedulerPlugin(), + ]), + sqlDatabaseUrl: "postgres://configured.example.test/neon", + stateAdapter, + }), + ).resolves.toEqual({ + existing: 0, + migrated: 1, + missing: 0, + scanned: 1, + }); + } finally { + await stateAdapter.disconnect(); + await fixture.close(); + } + }); + + it("skips malformed SQL records while claiming due runs", async () => { + const fixture = await createLocalJuniorSqlFixture(); + + try { + await migrateSchedulerSchema(fixture); + const db = createPluginDbForExecutor(fixture.executor); + const store = createSchedulerSqlStore(db); + const task = createTask({ id: "sched_valid_after_bad_record" }); + + await db.execute( + ` +INSERT INTO junior_scheduler_tasks ( + id, + team_id, + status, + next_run_at_ms, + created_at_ms, + updated_at_ms, + version, + destination, + created_by, + schedule, + task, + record +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) +`, + [ + "sched_bad_record", + task.destination.teamId, + "active", + TEST_RUN_AT_MS, + TEST_RUN_AT_MS - 1, + TEST_RUN_AT_MS - 1, + 1, + JSON.stringify(task.destination), + JSON.stringify(task.createdBy), + JSON.stringify(task.schedule), + JSON.stringify(task.task), + JSON.stringify({ id: "sched_bad_record" }), + ], + ); + await store.saveTask(task); + await expect(store.getTask("sched_bad_record")).resolves.toBe(undefined); + await db.execute( + ` +INSERT INTO junior_scheduler_tasks ( + id, + team_id, + status, + next_run_at_ms, + created_at_ms, + updated_at_ms, + version, + destination, + created_by, + schedule, + task, + record +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) +`, + [ + "sched_bad_string_record", + task.destination.teamId, + "active", + TEST_RUN_AT_MS, + TEST_RUN_AT_MS - 1, + TEST_RUN_AT_MS - 1, + 1, + JSON.stringify(task.destination), + JSON.stringify(task.createdBy), + JSON.stringify(task.schedule), + JSON.stringify(task.task), + JSON.stringify("not-json"), + ], + ); + await expect(store.getTask("sched_bad_string_record")).resolves.toBe( + undefined, + ); + await db.execute( + ` +INSERT INTO junior_scheduler_runs ( + id, + task_id, + status, + claimed_at_ms, + scheduled_for_ms, + idempotency_key, + task_version, + attempt, + record +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) +`, + [ + "sched_bad_run", + task.id, + "pending", + TEST_NOW_MS - 120_000, + TEST_RUN_AT_MS - 60_000, + "sched_bad_run", + 1, + 1, + JSON.stringify({ id: "sched_bad_run" }), + ], + ); + await expect(store.getRun("sched_bad_run")).resolves.toBe(undefined); + await db.execute( + ` +INSERT INTO junior_scheduler_runs ( + id, + task_id, + status, + claimed_at_ms, + scheduled_for_ms, + idempotency_key, + task_version, + attempt, + record +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) +`, + [ + "sched_bad_string_run", + task.id, + "pending", + TEST_NOW_MS - 120_000, + TEST_RUN_AT_MS - 60_000, + "sched_bad_string_run", + 1, + 1, + JSON.stringify("not-json"), + ], + ); + await expect(store.getRun("sched_bad_string_run")).resolves.toBe( + undefined, + ); + + await expect( + store.claimDueRun({ nowMs: TEST_NOW_MS }), + ).resolves.toMatchObject({ + id: `${task.id}:${TEST_RUN_AT_MS}`, + taskId: task.id, + }); + } finally { + await fixture.close(); + } + }, 15_000); + + it("requires database access for plugin storage migrations", async () => { + const stateAdapter = getStateAdapter(); + await stateAdapter.connect(); + const fixture = await createLocalJuniorSqlFixture(); + + try { + const db = createPluginDbForExecutor(fixture.executor); + const plugin = defineJuniorPlugin({ + manifest: { + name: "stateless", + displayName: "Stateless", + description: "Storage migration without database access", + }, + hooks: { + migrateStorage() { + return { + existing: 0, + migrated: 0, + missing: 0, + scanned: 1, + }; + }, + }, + }); + + await expect( + runPluginStorageMigrations({ + io: { info: () => {} }, + pluginDb: db, + pluginSet: defineJuniorPlugins([plugin]), + stateAdapter, + }), + ).rejects.toThrow( + 'Plugin "stateless" storage migration requires database access', + ); + } finally { + await fixture.close(); + } + }, 15_000); +}); diff --git a/packages/junior/tests/integration/heartbeat.test.ts b/packages/junior/tests/integration/heartbeat.test.ts index e3f6d6f62..a989a1d22 100644 --- a/packages/junior/tests/integration/heartbeat.test.ts +++ b/packages/junior/tests/integration/heartbeat.test.ts @@ -1,16 +1,23 @@ +import path from "node:path"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { defineJuniorPlugin, + type PluginDb, type Destination, } from "@sentry/junior-plugin-api"; import { createHeartbeatContext } from "@/chat/agent-dispatch/context"; import { recoverStaleDispatches } from "@/chat/agent-dispatch/heartbeat"; import { - createSchedulerStore, + createSchedulerSqlStore, schedulerPlugin, type ScheduledTask, } from "@sentry/junior-scheduler"; -import { createPluginState } from "@/chat/plugins/state"; +import * as pluginDbModule from "@/chat/plugins/db"; +import { + createPluginDbForExecutor, + migratePluginSchemas, + readPluginMigrations, +} from "@/chat/plugins/db"; import { createOrGetDispatch, getDispatchRecord, @@ -26,10 +33,11 @@ import { persistThreadStateById } from "@/chat/runtime/thread-state"; import { getConversationWorkState } from "@/chat/task-execution/store"; import { scheduleAgentContinue } from "@/chat/services/agent-continue"; import type { PiMessage } from "@/chat/pi/messages"; -import { setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { setPlugins } from "@/chat/plugins/agent-hooks"; import { GET as heartbeat } from "@/handlers/heartbeat"; import { createSlackDirectCredentialSubject } from "@/chat/credentials/subject"; import { createConversationWorkQueueTestAdapter } from "../fixtures/conversation-work"; +import { createLocalJuniorSqlFixture } from "../fixtures/sql"; import { createWaitUntilCollector } from "../fixtures/wait-until"; import { getCapturedSlackApiCalls } from "../msw/handlers/slack-api"; @@ -45,8 +53,35 @@ const SLACK_DESTINATION = { channelId: "C123", } satisfies Destination; -function schedulerStore() { - return createSchedulerStore(createPluginState("scheduler")); +let schedulerSqlFixture: + | Awaited> + | undefined; +let schedulerPluginDb: PluginDb | undefined; + +function schedulerMigrationsDir(): string { + return path.resolve(process.cwd(), "../junior-scheduler/migrations"); +} + +async function migrateSchedulerSchema( + fixture: Awaited>, +) { + await migratePluginSchemas( + fixture.executor, + readPluginMigrations({ + dir: schedulerMigrationsDir(), + pluginName: "scheduler", + }), + ); +} + +async function useSchedulerSqlStore() { + schedulerSqlFixture = await createLocalJuniorSqlFixture(); + await migrateSchedulerSchema(schedulerSqlFixture); + schedulerPluginDb = createPluginDbForExecutor(schedulerSqlFixture.executor); + vi.spyOn(pluginDbModule, "getPluginDbForRegistration").mockImplementation( + (plugin) => (plugin.database ? schedulerPluginDb : undefined), + ); + return createSchedulerSqlStore(schedulerPluginDb); } function createTask(overrides: Partial = {}): ScheduledTask { @@ -170,13 +205,16 @@ describe("plugin heartbeat", () => { process.env.JUNIOR_SCHEDULER_SECRET = "heartbeat-secret"; process.env.JUNIOR_BASE_URL = "https://junior.example.com"; process.env.JUNIOR_SECRET = "dispatch-secret"; - setAgentPlugins([]); + setPlugins([]); await disconnectStateAdapter(); }); afterEach(async () => { global.fetch = originalFetch; - setAgentPlugins([]); + setPlugins([]); + await schedulerSqlFixture?.close(); + schedulerSqlFixture = undefined; + schedulerPluginDb = undefined; await disconnectStateAdapter(); delete process.env.JUNIOR_SCHEDULER_SECRET; delete process.env.CRON_SECRET; @@ -199,7 +237,7 @@ describe("plugin heartbeat", () => { it("runs plugin heartbeat hooks", async () => { const seen: number[] = []; - setAgentPlugins([ + setPlugins([ defineJuniorPlugin({ manifest: { name: "scheduler", @@ -467,6 +505,29 @@ describe("plugin heartbeat", () => { }); }); + it("exposes plugin DB access to heartbeat contexts for database plugins", () => { + const db = {} as any; + const spy = vi + .spyOn(pluginDbModule, "getPluginDbForRegistration") + .mockReturnValue(db); + const plugin = defineJuniorPlugin({ + database: {}, + manifest: { + name: "database-plugin", + displayName: "Database Plugin", + description: "Heartbeat database context test", + }, + }); + + const ctx = createHeartbeatContext({ + plugin, + nowMs: Date.parse("2026-05-26T12:00:00.000Z"), + }); + + expect(spy).toHaveBeenCalledWith(plugin); + expect(ctx.db).toBe(db); + }); + it("keeps plugin state isolated when plugin names and keys contain delimiters", async () => { const first = createHeartbeatContext({ plugin: "scheduler", @@ -484,30 +545,6 @@ describe("plugin heartbeat", () => { await expect(second.state.get("1")).resolves.toBe("second"); }); - it("claims scheduled tasks from the scheduler legacy state namespace", async () => { - const task = createTask({ id: "sched_legacy" }); - const state = getStateAdapter(); - await state.connect(); - await state.set("junior:scheduler:tasks", [task.id]); - await state.set("junior:scheduler:team:T123:tasks", [task.id]); - await state.set("junior:scheduler:task:sched_legacy", task); - - const store = createSchedulerStore( - createPluginState("scheduler", { - legacyStatePrefixes: ["junior:scheduler"], - }), - ); - - await expect(store.listTasksForTeam("T123")).resolves.toMatchObject([ - { id: task.id }, - ]); - await expect( - store.claimDueRun({ nowMs: TEST_NOW_MS }), - ).resolves.toMatchObject({ - taskId: task.id, - }); - }); - it("bounds dispatch fanout from one heartbeat context", async () => { const fetchMock = vi.fn(async () => { return new Response("Accepted", { status: 202 }); @@ -834,8 +871,8 @@ describe("plugin heartbeat", () => { return new Response("Accepted", { status: 202 }); }); global.fetch = fetchMock as typeof fetch; - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); await store.saveTask( createTask({ createdBy: { @@ -901,11 +938,11 @@ describe("plugin heartbeat", () => { lastRunAtMs: Date.parse("2026-05-26T12:00:00.000Z"), status: "paused", }); - }); + }, 30_000); it("exposes sanitized scheduler operational reports through Junior reporting", async () => { - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); await store.saveTask( createTask({ createdBy: { @@ -996,11 +1033,11 @@ describe("plugin heartbeat", () => { author: "Invalid Slack creator metadata", }); expect(JSON.stringify(feed)).not.toContain("Secret"); - }); + }, 30_000); it("counts all running scheduler runs in operational summaries", async () => { - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); for (let index = 0; index < 6; index += 1) { await store.saveTask( createTask({ @@ -1030,12 +1067,12 @@ describe("plugin heartbeat", () => { expect(runningSummary).toMatchObject({ value: "6" }); expect(runningSection?.records).toHaveLength(5); - }); + }, 30_000); it("carries scheduled task credential subjects into dispatch records", async () => { mockDispatchCallbackFetch(originalFetch); - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); await store.saveTask( createTask({ destination: { @@ -1079,15 +1116,15 @@ describe("plugin heartbeat", () => { }, }); expect(getCapturedSlackApiCalls("conversations.info")).toHaveLength(0); - }); + }, 30_000); it("fails scheduled runs when their dispatch record disappeared", async () => { const fetchMock = vi.fn(async () => { return new Response("Accepted", { status: 202 }); }); global.fetch = fetchMock as typeof fetch; - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); await store.saveTask(createTask()); const firstWaitUntil = createWaitUntilCollector(); @@ -1126,21 +1163,21 @@ describe("plugin heartbeat", () => { await expect(store.getTask("sched_plugin_1")).resolves.toMatchObject({ status: "paused", }); - }); + }, 30_000); it("blocks malformed scheduled tasks without stopping the scheduler plugin heartbeat", async () => { const fetchMock = vi.fn(async () => { return new Response("Accepted", { status: 202 }); }); global.fetch = fetchMock as typeof fetch; - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); await store.saveTask({ ...createTask(), id: "sched_plugin_malformed", task: { - text: undefined, - } as unknown as ScheduledTask["task"], + text: "", + }, }); const waitUntil = createWaitUntilCollector(); @@ -1170,15 +1207,15 @@ describe("plugin heartbeat", () => { ), }); expect(fetchMock).not.toHaveBeenCalled(); - }); + }, 30_000); it("skips old recurring occurrences and advances to the next future run", async () => { const fetchMock = vi.fn(async () => { return new Response("Accepted", { status: 202 }); }); global.fetch = fetchMock as typeof fetch; - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); const task = createDailyTask(); await store.saveTask(task); @@ -1203,15 +1240,15 @@ describe("plugin heartbeat", () => { nextRunAtMs: Date.parse("2026-05-27T12:00:00.000Z"), }); expect(fetchMock).not.toHaveBeenCalled(); - }); + }, 30_000); it("dedupes equivalent old recurring tasks during heartbeat recovery", async () => { const fetchMock = vi.fn(async () => { return new Response("Accepted", { status: 202 }); }); global.fetch = fetchMock as typeof fetch; - setAgentPlugins([schedulerPlugin()]); - const store = schedulerStore(); + setPlugins([schedulerPlugin()]); + const store = await useSchedulerSqlStore(); const first = createDailyTask({ id: "sched_plugin_duplicate_a", createdAtMs: Date.parse("2026-05-24T12:00:00.000Z"), @@ -1245,11 +1282,12 @@ describe("plugin heartbeat", () => { status: "active", nextRunAtMs: Date.parse("2026-05-27T12:00:00.000Z"), }); - await expect(store.getTask(duplicate.id)).resolves.toMatchObject({ + const duplicateTask = await store.getTask(duplicate.id); + expect(duplicateTask).toMatchObject({ status: "paused", - nextRunAtMs: undefined, statusReason: expect.stringContaining(first.id), }); + expect(duplicateTask).not.toHaveProperty("nextRunAtMs"); expect(fetchMock).not.toHaveBeenCalled(); - }); + }, 30_000); }); diff --git a/packages/junior/tests/integration/sandbox-egress-proxy.test.ts b/packages/junior/tests/integration/sandbox-egress-proxy.test.ts index 8a599ebfb..9b124372b 100644 --- a/packages/junior/tests/integration/sandbox-egress-proxy.test.ts +++ b/packages/junior/tests/integration/sandbox-egress-proxy.test.ts @@ -3,7 +3,7 @@ import path from "node:path"; import { defineJuniorPlugin, EgressAuthRequired, - type AgentPluginHooks, + type PluginHooks, } from "@sentry/junior-plugin-api"; import { http, HttpResponse } from "msw"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; @@ -112,8 +112,8 @@ function proxiedRequest(input: { async function registerManagedEgressPlugin(input?: { egressTracePropagationDomains?: string[]; - issueCredential?: NonNullable; - onEgressResponse?: NonNullable; + issueCredential?: NonNullable; + onEgressResponse?: NonNullable; }) { const { createApp, defineJuniorPlugins } = await import("@/app"); await createApp({ diff --git a/packages/junior/tests/integration/slack-schedule-tools.test.ts b/packages/junior/tests/integration/slack-schedule-tools.test.ts index 109c6eded..fb38981fe 100644 --- a/packages/junior/tests/integration/slack-schedule-tools.test.ts +++ b/packages/junior/tests/integration/slack-schedule-tools.test.ts @@ -1,10 +1,12 @@ +import path from "node:path"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { - AgentPluginToolInputError, - type AgentPluginToolDefinition, + PluginToolInputError, + type PluginDb, + type PluginToolDefinition, } from "@sentry/junior-plugin-api"; import { - createSchedulerStore, + createSchedulerSqlStore, createSlackScheduleCreateTaskTool, createSlackScheduleDeleteTaskTool, createSlackScheduleListTasksTool, @@ -15,18 +17,49 @@ import { } from "@sentry/junior-scheduler"; import { createSlackDirectCredentialSubject } from "@/chat/credentials/subject"; import { - getAgentPluginTools, - setAgentPlugins, -} from "@/chat/plugins/agent-hooks"; -import { createPluginState } from "@/chat/plugins/state"; + createPluginDbForExecutor, + migratePluginSchemas, + readPluginMigrations, +} from "@/chat/plugins/db"; +import * as pluginDbModule from "@/chat/plugins/db"; +import { getPluginTools, setPlugins } from "@/chat/plugins/agent-hooks"; import { disconnectStateAdapter } from "@/chat/state/adapter"; import { schedulerPlugin } from "@sentry/junior-scheduler"; +import { + createLocalJuniorSqlFixture, + type LocalJuniorSqlFixture, +} from "../fixtures/sql"; vi.hoisted(() => { process.env.JUNIOR_STATE_ADAPTER = "memory"; }); const TEST_TEAM_ID = `TSCHEDULE${Date.now()}`; +let currentFixture: LocalJuniorSqlFixture | undefined; +let currentSchedulerStore: SchedulerToolContext["store"] | undefined; + +function schedulerMigrationsDir(): string { + return path.resolve(process.cwd(), "../junior-scheduler/migrations"); +} + +async function useSchedulerSqlPlugin() { + const fixture = await createLocalJuniorSqlFixture(); + await migratePluginSchemas( + fixture.executor, + readPluginMigrations({ + dir: schedulerMigrationsDir(), + pluginName: "scheduler", + }), + ); + const db: PluginDb = createPluginDbForExecutor(fixture.executor); + vi.spyOn(pluginDbModule, "getPluginDbForRegistration").mockImplementation( + (plugin) => (plugin.database ? db : undefined), + ); + return { + fixture, + store: createSchedulerSqlStore(db), + }; +} function createContext( overrides: Partial & { @@ -53,7 +86,7 @@ function createContext( fullName: "David Cramer", }, userText: "schedule this weekly", - state: createPluginState("scheduler"), + store: schedulerStore(), ...contextOverrides, }; const credentialSubject = @@ -70,7 +103,7 @@ function createContext( } async function executeTool( - tool: AgentPluginToolDefinition, + tool: PluginToolDefinition, input: TInput, ) { if (typeof tool?.execute !== "function") { @@ -80,7 +113,22 @@ async function executeTool( } function schedulerStore() { - return createSchedulerStore(createPluginState("scheduler")); + if (!currentSchedulerStore) { + throw new Error("Scheduler SQL store is not initialized"); + } + return currentSchedulerStore; +} + +async function initializeSchedulerSqlStore(): Promise { + const plugin = await useSchedulerSqlPlugin(); + currentFixture = plugin.fixture; + currentSchedulerStore = plugin.store; +} + +async function cleanupSchedulerSqlStore(): Promise { + await currentFixture?.close(); + currentFixture = undefined; + currentSchedulerStore = undefined; } async function createTask( @@ -101,11 +149,14 @@ async function createTask( describe("Slack schedule tools", () => { beforeEach(async () => { await disconnectStateAdapter(); + await initializeSchedulerSqlStore(); }); afterEach(async () => { vi.useRealTimers(); delete process.env.JUNIOR_TIMEZONE; + await cleanupSchedulerSqlStore(); + vi.restoreAllMocks(); await disconnectStateAdapter(); }); @@ -224,7 +275,7 @@ describe("Slack schedule tools", () => { }), ); - await expect(rejected).rejects.toThrow(AgentPluginToolInputError); + await expect(rejected).rejects.toThrow(PluginToolInputError); await expect(rejected).rejects.toThrow( "No active Slack requester context is available.", ); @@ -243,7 +294,7 @@ describe("Slack schedule tools", () => { }, ); - await expect(rejected).rejects.toThrow(AgentPluginToolInputError); + await expect(rejected).rejects.toThrow(PluginToolInputError); await expect(rejected).rejects.toThrow( "Active Slack conversation workspace is invalid.", ); @@ -264,7 +315,7 @@ describe("Slack schedule tools", () => { }), ); - await expect(rejected).rejects.toThrow(AgentPluginToolInputError); + await expect(rejected).rejects.toThrow(PluginToolInputError); await expect(rejected).rejects.toThrow( "Active Slack conversation must not include unknown fields.", ); @@ -291,7 +342,7 @@ describe("Slack schedule tools", () => { }), ); - await expect(rejected).rejects.toThrow(AgentPluginToolInputError); + await expect(rejected).rejects.toThrow(PluginToolInputError); await expect(rejected).rejects.toThrow( "Active Slack credential subject is invalid.", ); @@ -467,7 +518,6 @@ describe("Slack schedule tools", () => { nextRunAtMs: Date.parse("2026-05-28T02:18:48.005Z"), schedule: { kind: "one_off", - recurrence: undefined, }, status: "active", }, @@ -612,7 +662,6 @@ describe("Slack schedule tools", () => { ).resolves.toMatchObject({ schedule: { kind: "one_off", - recurrence: undefined, }, }); }); @@ -642,7 +691,7 @@ describe("Slack schedule tools", () => { // same source conversation. // // In practice: a DM opened via Slack’s “Ask Junior” panel from #js-alerts - // has getAgentPluginTools build source.channelId = DDM rather than using + // has getPluginTools build source.channelId = DDM rather than using // the outbound assistant-context channel. Both creation and management // from that DM use DDM, so the stored task destination never drifts. const dmCtx = createContext({ channelId: "DDM" }); @@ -1091,7 +1140,7 @@ describe("Slack schedule tools", () => { }); }); -describe("Slack schedule tool wiring via getAgentPluginTools", () => { +describe("Slack schedule tool wiring via getPluginTools", () => { // These tests exercise the real agent-hooks.ts path where the runtime-owned // Destination is passed through to the scheduler plugin. @@ -1104,12 +1153,13 @@ describe("Slack schedule tool wiring via getAgentPluginTools", () => { }); it("scheduler tools bind to the runtime-owned source", async () => { - // Verifies that real getAgentPluginTools wiring passes Source through to + // Verifies that real getPluginTools wiring passes Source through to // the scheduler, which stores it as the task destination. - const previous = setAgentPlugins([schedulerPlugin()]); + const previous = setPlugins([schedulerPlugin()]); + const { fixture, store } = await useSchedulerSqlPlugin(); try { const TEAM_ID = `TWIRING${Date.now()}`; - const tools = getAgentPluginTools({ + const tools = getPluginTools({ source: { platform: "slack", teamId: TEAM_ID, @@ -1127,7 +1177,7 @@ describe("Slack schedule tool wiring via getAgentPluginTools", () => { userName: "alice", fullName: "Alice", }, - sandbox: {} as Parameters[0]["sandbox"], + sandbox: {} as Parameters[0]["sandbox"], }); expect(tools).toHaveProperty("slackScheduleCreateTask"); @@ -1145,9 +1195,7 @@ describe("Slack schedule tool wiring via getAgentPluginTools", () => { const taskId = (result as { task: { id: string } }).task.id; // Task destination must be the raw DM channel, NOT the assistant context. - const stored = await createSchedulerStore( - createPluginState("scheduler"), - ).getTask(taskId); + const stored = await store.getTask(taskId); expect(stored).toMatchObject({ destination: { channelId: "DDM", teamId: TEAM_ID }, conversationAccess: { audience: "direct", visibility: "private" }, @@ -1159,12 +1207,23 @@ describe("Slack schedule tool wiring via getAgentPluginTools", () => { allowedWhen: "private-direct-conversation", }); } finally { - setAgentPlugins(previous); + await fixture.close(); + vi.restoreAllMocks(); + setPlugins(previous); } }); }); describe("Slack schedule tool execution modes", () => { + beforeEach(async () => { + await initializeSchedulerSqlStore(); + }); + + afterEach(async () => { + await cleanupSchedulerSqlStore(); + vi.restoreAllMocks(); + }); + it("all write tools have executionMode sequential", () => { const context = createContext(); diff --git a/packages/junior/tests/integration/slack/outbound-normalization-contract.test.ts b/packages/junior/tests/integration/slack/outbound-normalization-contract.test.ts index 58ac5b9d1..907310d07 100644 --- a/packages/junior/tests/integration/slack/outbound-normalization-contract.test.ts +++ b/packages/junior/tests/integration/slack/outbound-normalization-contract.test.ts @@ -4,7 +4,7 @@ import { buildSlackReplyBlocks, buildSlackReplyFooter, } from "@/chat/slack/footer"; -import { setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { setPlugins } from "@/chat/plugins/agent-hooks"; import { addReactionToMessage, postSlackEphemeralMessage, @@ -25,7 +25,7 @@ describe("Slack contract: outbound normalization", () => { beforeEach(() => { process.env.SLACK_BOT_TOKEN = process.env.SLACK_BOT_TOKEN ?? "xoxb-test-token"; - setAgentPlugins([]); + setPlugins([]); resetSlackApiMockState(); }); @@ -82,9 +82,8 @@ describe("Slack contract: outbound normalization", () => { }); it("lets plugins replace the footer conversation link", async () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "dashboard", manifest: { name: "dashboard", displayName: "Dashboard", @@ -132,7 +131,7 @@ describe("Slack contract: outbound normalization", () => { }), ]); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); diff --git a/packages/junior/tests/unit/app-config.test.ts b/packages/junior/tests/unit/app-config.test.ts index 47fb2046d..b4c46a21c 100644 --- a/packages/junior/tests/unit/app-config.test.ts +++ b/packages/junior/tests/unit/app-config.test.ts @@ -8,7 +8,7 @@ import { getConfigDefaults, setConfigDefaults, } from "@/chat/configuration/defaults"; -import { getAgentPlugins, setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { getPlugins, setPlugins } from "@/chat/plugins/agent-hooks"; import { getPluginSkillRoots, getPluginProviders, @@ -58,7 +58,7 @@ async function writePluginPackage( afterEach(async () => { process.chdir(originalCwd); - setAgentPlugins([]); + setPlugins([]); setPluginCatalogConfig(undefined); setConfigDefaults(undefined); vi.doUnmock("#junior/config"); @@ -99,7 +99,7 @@ describe("createApp plugin config", () => { }); expect(getPluginProviders()).toEqual([]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); }); it("validates sandbox egress trace propagation domains from app options", async () => { @@ -138,7 +138,9 @@ describe("createApp plugin config", () => { expect(getPluginProviders().map((plugin) => plugin.manifest.name)).toEqual([ "base", ]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual(["base"]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([ + "base", + ]); }); it("loads package plugins with runtime hook plugins", async () => { @@ -176,7 +178,7 @@ describe("createApp plugin config", () => { "dashboard", "env", ]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([ + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([ "dashboard", ]); }); @@ -281,7 +283,9 @@ describe("createApp plugin config", () => { expect(getPluginProviders().map((plugin) => plugin.manifest.name)).toEqual([ "hooked", ]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual(["hooked"]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([ + "hooked", + ]); }); it("rejects incomplete plugin egress credential hooks", async () => { @@ -311,7 +315,7 @@ describe("createApp plugin config", () => { 'Plugin "example" egress credential hooks must include both grantForEgress and issueCredential.', ); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); @@ -347,7 +351,7 @@ describe("createApp plugin config", () => { 'Plugin "example" egress credential hooks require manifest.domains to list sandbox egress hosts.', ); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); @@ -379,7 +383,7 @@ describe("createApp plugin config", () => { 'Plugin "example" manifest.oauth without oauth-bearer credentials requires egress credential hooks.', ); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); @@ -411,7 +415,9 @@ describe("createApp plugin config", () => { expect(getPluginProviders().map((plugin) => plugin.manifest.name)).toEqual([ "example", ]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual(["example"]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([ + "example", + ]); }); it("does not assign app skills to runtime hook inline plugins", async () => { @@ -537,7 +543,7 @@ describe("createApp plugin config", () => { 'Plugin "invalid" manifest.domains requires egress credential hooks when no generic credentials or apiHeaders are configured.', ); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); @@ -577,7 +583,9 @@ describe("createApp plugin config", () => { expect(getPluginProviders().map((plugin) => plugin.manifest.name)).toEqual([ "hooked", ]); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual(["hooked"]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([ + "hooked", + ]); }); it("loads manifest-only package plugins by package name", async () => { @@ -600,7 +608,7 @@ describe("createApp plugin config", () => { plugins: defineJuniorPlugins(["@acme/full-plugin"]), }); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders().map((plugin) => plugin.manifest.name)).toEqual([ "full", ]); @@ -630,7 +638,7 @@ describe("createApp plugin config", () => { ]), ).toThrow('Duplicate plugin registration name "dupe"'); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); @@ -652,33 +660,20 @@ describe("createApp plugin config", () => { 'Junior plugin registration name "GitHub" must be a lowercase plugin identifier', ); - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); + expect(getPlugins().map((plugin) => plugin.manifest.name)).toEqual([]); expect(getPluginProviders()).toEqual([]); }); - it("rejects legacy state prefixes outside the plugin namespace", async () => { - await createApp({ - plugins: defineJuniorPlugins([]), - }); - - await expect( - createApp({ - plugins: defineJuniorPlugins([ - defineJuniorPlugin({ - manifest: { - name: "hooked", - displayName: "Hooked", - description: "Runtime plugin", - }, - legacyStatePrefixes: ["junior:scheduler"], - }), - ]), - }), - ).rejects.toThrow( - 'Plugin "hooked" legacy state prefix "junior:scheduler" must stay under "junior:hooked"', - ); - - expect(getAgentPlugins().map((plugin) => plugin.name)).toEqual([]); - expect(getPluginProviders()).toEqual([]); + it("rejects top-level plugin registration names", () => { + expect(() => + defineJuniorPlugin({ + name: "legacy", + manifest: { + name: "legacy", + displayName: "Legacy", + description: "Legacy plugin", + }, + } as Parameters[0] & { name: string }), + ).toThrow("defineJuniorPlugin() uses manifest.name for identity."); }); }); diff --git a/packages/junior/tests/unit/build/copy-build-content.test.ts b/packages/junior/tests/unit/build/copy-build-content.test.ts index 211213116..f7582156e 100644 --- a/packages/junior/tests/unit/build/copy-build-content.test.ts +++ b/packages/junior/tests/unit/build/copy-build-content.test.ts @@ -226,6 +226,7 @@ describe("copyAppAndPluginContent", () => { }); fs.mkdirSync(path.join(packageDir, "skills", "demo"), { recursive: true }); + fs.mkdirSync(path.join(packageDir, "migrations"), { recursive: true }); fs.writeFileSync( path.join(packageDir, "plugin.yaml"), "name: ancestor\ndescription: Ancestor plugin\n", @@ -236,6 +237,11 @@ describe("copyAppAndPluginContent", () => { "---\nname: demo\ndescription: Demo\n---\n", "utf8", ); + fs.writeFileSync( + path.join(packageDir, "migrations", "0001_init.sql"), + "CREATE TABLE junior_ancestor_items (id TEXT PRIMARY KEY);\n", + "utf8", + ); fs.mkdirSync(cwd, { recursive: true }); fs.writeFileSync( path.join(cwd, "package.json"), @@ -279,5 +285,17 @@ describe("copyAppAndPluginContent", () => { ), ), ).toBe(true); + expect( + fs.existsSync( + path.join( + serverRoot, + "node_modules", + "@acme", + "ancestor-plugin", + "migrations", + "0001_init.sql", + ), + ), + ).toBe(true); }); }); diff --git a/packages/junior/tests/unit/build/nitro-plugin-module.test.ts b/packages/junior/tests/unit/build/nitro-plugin-module.test.ts index fd2a1ccd9..a5b2f6d44 100644 --- a/packages/junior/tests/unit/build/nitro-plugin-module.test.ts +++ b/packages/junior/tests/unit/build/nitro-plugin-module.test.ts @@ -349,7 +349,6 @@ describe("juniorNitro plugin modules", () => { juniorNitro({ plugins: defineJuniorPlugins([ defineJuniorPlugin({ - name: "hooked", manifest: { name: "hooked", displayName: "Hooked", diff --git a/packages/junior/tests/unit/config/package-discovery.test.ts b/packages/junior/tests/unit/config/package-discovery.test.ts index d5e79a567..909111866 100644 --- a/packages/junior/tests/unit/config/package-discovery.test.ts +++ b/packages/junior/tests/unit/config/package-discovery.test.ts @@ -63,6 +63,12 @@ describe("plugin package discovery", () => { nodeModulesRoot, "@acme/junior-plugin-demo", ); + await fs.mkdir(path.join(packageRoot, "migrations")); + await fs.writeFile( + path.join(packageRoot, "migrations", "0001_init.sql"), + "CREATE TABLE plugin_demo (id TEXT PRIMARY KEY);\n", + "utf8", + ); await fs.writeFile( path.join(tempRoot, "package.json"), JSON.stringify({ name: "temp", private: true }), @@ -73,11 +79,22 @@ describe("plugin package discovery", () => { packageNames: ["@acme/junior-plugin-demo"], }); expect(discovered.packageNames).toContain("@acme/junior-plugin-demo"); + expect(discovered.packages).toEqual([ + { + dir: packageRoot, + hasMigrationsDir: true, + hasSkillsDir: true, + packageName: "@acme/junior-plugin-demo", + }, + ]); expect(discovered.manifestRoots).toContain(packageRoot); expect(discovered.skillRoots).toContain(path.join(packageRoot, "skills")); expect(discovered.tracingIncludes).toContain( "./node_modules/@acme/junior-plugin-demo/plugin.yaml", ); + expect(discovered.tracingIncludes).toContain( + "./node_modules/@acme/junior-plugin-demo/migrations/**/*", + ); expect(discovered.tracingIncludes).toContain( "./node_modules/@acme/junior-plugin-demo/skills/**/*", ); diff --git a/packages/junior/tests/unit/handlers/sandbox-egress-proxy.test.ts b/packages/junior/tests/unit/handlers/sandbox-egress-proxy.test.ts index 12e54782a..f952d6033 100644 --- a/packages/junior/tests/unit/handlers/sandbox-egress-proxy.test.ts +++ b/packages/junior/tests/unit/handlers/sandbox-egress-proxy.test.ts @@ -72,7 +72,7 @@ import { matchesSandboxEgressDomain, resolveSandboxCommandEnvironment, } from "@/chat/sandbox/egress-policy"; -import { setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { setPlugins } from "@/chat/plugins/agent-hooks"; import { isSandboxEgressForwardedRequest, proxySandboxEgressRequest, @@ -941,7 +941,7 @@ describe("sandbox egress proxy", () => { }, }; }); - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: githubPlugin().manifest, hooks: { @@ -1031,7 +1031,7 @@ describe("sandbox egress proxy", () => { sso: "required; url=https://github.com/orgs/getsentry/sso", }); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); diff --git a/packages/junior/tests/unit/plugins/agent-hooks.test.ts b/packages/junior/tests/unit/plugins/agent-hooks.test.ts index bd72078b0..099894e87 100644 --- a/packages/junior/tests/unit/plugins/agent-hooks.test.ts +++ b/packages/junior/tests/unit/plugins/agent-hooks.test.ts @@ -1,16 +1,16 @@ import { defineJuniorPlugin, - type AgentPluginConversations, + type PluginConversations, type ToolRegistrationHookContext, } from "@sentry/junior-plugin-api"; import { describe, expect, it } from "vitest"; import { - createAgentPluginHookRunner, - getAgentPluginOperationalReports, - getAgentPluginRoutes, - getAgentPluginSlackConversationLink, - getAgentPluginTools, - setAgentPlugins, + createPluginHookRunner, + getPluginOperationalReports, + getPluginRoutes, + getPluginSlackConversationLink, + getPluginTools, + setPlugins, } from "@/chat/plugins/agent-hooks"; import { createTools } from "@/chat/tools"; import { tool } from "@/chat/tools/definition"; @@ -29,7 +29,7 @@ const LOCAL_DESTINATION = { conversationId: "local:test:agent-hooks", } as const; -const EMPTY_CONVERSATIONS: AgentPluginConversations = { +const EMPTY_CONVERSATIONS: PluginConversations = { async listRecent() { return []; }, @@ -93,7 +93,7 @@ function fakeSandbox( describe("agent plugin hooks", () => { it("collects turn-scoped tools from configured plugins", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: { name: "agent-demo", @@ -115,7 +115,7 @@ describe("agent plugin hooks", () => { }), ]); try { - const tools = getAgentPluginTools({ + const tools = getPluginTools({ destination: SLACK_DESTINATION, requester: TEST_REQUESTER, source: SLACK_DESTINATION, @@ -124,12 +124,12 @@ describe("agent plugin hooks", () => { expect(tools).toHaveProperty("demoTool"); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects plugin tools with invalid names", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: { name: "agent-demo", @@ -151,19 +151,19 @@ describe("agent plugin hooks", () => { ]); try { expect(() => - getAgentPluginTools({ + getPluginTools({ destination: LOCAL_DESTINATION, source: LOCAL_DESTINATION, sandbox: {} as any, }), ).toThrow("must be a camelCase identifier"); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects plugin tools that conflict with core tools", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: { name: "agent-demo", @@ -196,14 +196,13 @@ describe("agent plugin hooks", () => { ), ).toThrow('Plugin tool "loadSkill" conflicts with a core tool'); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("collects route handlers from configured plugins", async () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -222,7 +221,7 @@ describe("agent plugin hooks", () => { }), ]); try { - const routes = getAgentPluginRoutes(); + const routes = getPluginRoutes(); expect(routes).toHaveLength(1); expect(routes[0]?.pluginName).toBe("agent-demo"); @@ -232,14 +231,13 @@ describe("agent plugin hooks", () => { ); await expect(response.text()).resolves.toBe("demo"); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects invalid route methods from configured plugins", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -259,18 +257,17 @@ describe("agent plugin hooks", () => { }), ]); try { - expect(() => getAgentPluginRoutes()).toThrow( + expect(() => getPluginRoutes()).toThrow( 'Plugin route "/demo" from plugin "agent-demo" has invalid method "TRACE"', ); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects routes that combine ALL with explicit methods", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -290,18 +287,17 @@ describe("agent plugin hooks", () => { }), ]); try { - expect(() => getAgentPluginRoutes()).toThrow( + expect(() => getPluginRoutes()).toThrow( 'Plugin route "/demo" from plugin "agent-demo" must not combine ALL with explicit methods', ); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects route paths that mix ALL and explicit method registrations", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -326,18 +322,17 @@ describe("agent plugin hooks", () => { }), ]); try { - expect(() => getAgentPluginRoutes()).toThrow( + expect(() => getPluginRoutes()).toThrow( 'Plugin route "/demo" conflicts with an ALL route for the same path', ); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("rejects unsafe Slack conversation links from configured plugins", () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -351,18 +346,17 @@ describe("agent plugin hooks", () => { }), ]); try { - expect(() => getAgentPluginSlackConversationLink("slack:C1:123")).toThrow( + expect(() => getPluginSlackConversationLink("slack:C1:123")).toThrow( 'Plugin "agent-demo" slackConversationLink must return an absolute http(s) URL', ); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("collects operational reports from configured plugins", async () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -386,7 +380,7 @@ describe("agent plugin hooks", () => { ]); try { await expect( - getAgentPluginOperationalReports(123, EMPTY_CONVERSATIONS), + getPluginOperationalReports(123, EMPTY_CONVERSATIONS), ).resolves.toEqual([ { pluginName: "agent-demo", @@ -395,14 +389,13 @@ describe("agent plugin hooks", () => { }, ]); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("passes conversation reader to operational reports", async () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -428,7 +421,7 @@ describe("agent plugin hooks", () => { ]); try { await expect( - getAgentPluginOperationalReports(123, { + getPluginOperationalReports(123, { async listRecent() { return [ { @@ -449,14 +442,13 @@ describe("agent plugin hooks", () => { }, ]); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("contains failed operational reports per plugin", async () => { - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ - name: "agent-demo", manifest: { name: "agent-demo", displayName: "Agent Demo", @@ -472,7 +464,6 @@ describe("agent plugin hooks", () => { }, }), defineJuniorPlugin({ - name: "broken-demo", manifest: { name: "broken-demo", displayName: "Broken Demo", @@ -487,7 +478,7 @@ describe("agent plugin hooks", () => { ]); try { await expect( - getAgentPluginOperationalReports(123, EMPTY_CONVERSATIONS), + getPluginOperationalReports(123, EMPTY_CONVERSATIONS), ).resolves.toEqual([ { pluginName: "agent-demo", @@ -508,13 +499,13 @@ describe("agent plugin hooks", () => { }, ]); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); it("runs sandbox and tool lifecycle hooks from configured plugins", async () => { const writes: Array<{ content: string | Uint8Array; path: string }> = []; - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: { name: "agent-demo", @@ -556,7 +547,7 @@ describe("agent plugin hooks", () => { }), ]); try { - const runner = createAgentPluginHookRunner({ + const runner = createPluginHookRunner({ requester: TEST_REQUESTER, }); @@ -585,12 +576,12 @@ describe("agent plugin hooks", () => { }); expect(before.env).toEqual({ AGENT_PLUGIN: "U123" }); } finally { - setAgentPlugins(previous); + setPlugins(previous); } }); }); -describe("getAgentPluginTools channel resolution", () => { +describe("getPluginTools channel resolution", () => { function capturePluginContext( context: ToolRuntimeContext = { destination: LOCAL_DESTINATION, @@ -599,7 +590,7 @@ describe("getAgentPluginTools channel resolution", () => { }, ) { let captured: ToolRegistrationHookContext | undefined; - const previous = setAgentPlugins([ + const previous = setPlugins([ defineJuniorPlugin({ manifest: { name: "capture", @@ -614,8 +605,8 @@ describe("getAgentPluginTools channel resolution", () => { }, }), ]); - getAgentPluginTools(context); - setAgentPlugins(previous); + getPluginTools(context); + setPlugins(previous); if (!captured) { throw new Error("capture plugin tools hook was not called"); } diff --git a/packages/junior/tests/unit/plugins/plugin-db-config.test.ts b/packages/junior/tests/unit/plugins/plugin-db-config.test.ts new file mode 100644 index 000000000..47797927c --- /dev/null +++ b/packages/junior/tests/unit/plugins/plugin-db-config.test.ts @@ -0,0 +1,81 @@ +import { defineJuniorPlugin } from "@sentry/junior-plugin-api"; +import { afterEach, describe, expect, it, vi } from "vitest"; + +const ORIGINAL_DATABASE_URL = process.env.DATABASE_URL; +const ORIGINAL_JUNIOR_DATABASE_URL = process.env.JUNIOR_DATABASE_URL; + +function restoreDatabaseEnv(): void { + if (ORIGINAL_DATABASE_URL === undefined) { + delete process.env.DATABASE_URL; + } else { + process.env.DATABASE_URL = ORIGINAL_DATABASE_URL; + } + if (ORIGINAL_JUNIOR_DATABASE_URL === undefined) { + delete process.env.JUNIOR_DATABASE_URL; + } else { + process.env.JUNIOR_DATABASE_URL = ORIGINAL_JUNIOR_DATABASE_URL; + } +} + +async function loadValidator() { + vi.resetModules(); + return await import("@/chat/plugins/db"); +} + +function dbPlugin() { + return defineJuniorPlugin({ + database: {}, + manifest: { + name: "database-plugin", + displayName: "Database Plugin", + description: "Plugin database config test", + }, + }); +} + +function statelessPlugin() { + return defineJuniorPlugin({ + manifest: { + name: "stateless-plugin", + displayName: "Stateless Plugin", + description: "Plugin database config test", + }, + }); +} + +afterEach(() => { + restoreDatabaseEnv(); + vi.resetModules(); +}); + +describe("plugin database config", () => { + it("fails database plugins when no SQL URL is configured", async () => { + delete process.env.DATABASE_URL; + delete process.env.JUNIOR_DATABASE_URL; + const { validatePluginDatabaseRequirements } = await loadValidator(); + + expect(() => validatePluginDatabaseRequirements([dbPlugin()])).toThrow( + "Plugin database access requires JUNIOR_DATABASE_URL or DATABASE_URL for: database-plugin", + ); + }); + + it("allows plugins without database declarations when no SQL URL is configured", async () => { + delete process.env.DATABASE_URL; + delete process.env.JUNIOR_DATABASE_URL; + const { validatePluginDatabaseRequirements } = await loadValidator(); + + expect(() => + validatePluginDatabaseRequirements([statelessPlugin()]), + ).not.toThrow(); + }); + + it("allows required database plugins when a SQL URL is configured", async () => { + delete process.env.DATABASE_URL; + process.env.JUNIOR_DATABASE_URL = "postgres://user:pass@example.test/neon"; + const { validatePluginDatabaseRequirements } = await loadValidator(); + + expect(() => + validatePluginDatabaseRequirements([dbPlugin()]), + ).not.toThrow(); + }); +}); diff --git a/packages/junior/tests/unit/plugins/plugin-db-migrations.test.ts b/packages/junior/tests/unit/plugins/plugin-db-migrations.test.ts new file mode 100644 index 000000000..4922c4155 --- /dev/null +++ b/packages/junior/tests/unit/plugins/plugin-db-migrations.test.ts @@ -0,0 +1,211 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { describe, expect, it } from "vitest"; +import { + migratePluginSchemas, + readPluginMigrations, + type PluginMigration, +} from "@/chat/plugins/db"; +import type { JuniorSqlMigrationExecutor } from "@/chat/sql/db"; + +class FakeSqlExecutor implements JuniorSqlMigrationExecutor { + readonly locks: string[] = []; + readonly statements: string[] = []; + readonly transactions: string[][] = []; + private activeTransaction: string[] | undefined; + private readonly applied = new Map(); + + constructor(applied?: Iterable) { + if (applied) { + this.applied = new Map(applied); + } + } + + db(): never { + throw new Error("Fake plugin migration executor does not support Drizzle"); + } + + async execute(statement: string, params: readonly unknown[] = []) { + const normalized = statement.trim(); + this.statements.push(normalized); + this.activeTransaction?.push(normalized); + if (normalized.startsWith("INSERT INTO junior_schema_migrations")) { + this.applied.set(String(params[0]), String(params[1])); + } + } + + async query(statement: string): Promise { + const normalized = statement.trim(); + this.statements.push(normalized); + if ( + normalized === + "SELECT id, checksum FROM junior_schema_migrations ORDER BY id ASC" + ) { + return [...this.applied.entries()].map(([id, checksum]) => ({ + id, + checksum, + })) as T[]; + } + throw new Error(`Unexpected query: ${statement}`); + } + + async transaction(callback: () => Promise): Promise { + const statements: string[] = []; + this.transactions.push(statements); + this.activeTransaction = statements; + try { + return await callback(); + } finally { + this.activeTransaction = undefined; + } + } + + async withLock(lockName: string, callback: () => Promise): Promise { + this.locks.push(lockName); + return await callback(); + } +} + +function migration(overrides: Partial = {}): PluginMigration { + return { + checksum: "checksum-1", + filename: "0001_init.sql", + id: "plugin:memory/0001_init.sql", + pluginName: "memory", + sql: "CREATE TABLE junior_memory_test (id TEXT PRIMARY KEY);", + ...overrides, + }; +} + +describe("plugin DB migrations", () => { + it("runs pending plugin migrations under the plugin schema lock", async () => { + const executor = new FakeSqlExecutor(); + + const result = await migratePluginSchemas(executor, [migration()]); + + expect(result).toEqual({ existing: 0, migrated: 1, scanned: 1 }); + expect(executor.locks).toEqual(["junior_plugin_schema"]); + expect(executor.statements[0]).toContain( + "CREATE TABLE IF NOT EXISTS junior_schema_migrations", + ); + expect(executor.transactions).toHaveLength(1); + expect(executor.transactions[0]).toEqual( + expect.arrayContaining([ + "CREATE TABLE junior_memory_test (id TEXT PRIMARY KEY);", + expect.stringContaining("INSERT INTO junior_schema_migrations"), + ]), + ); + }); + + it("does not reapply plugin migrations already recorded with the same checksum", async () => { + const applied = migration(); + const executor = new FakeSqlExecutor([[applied.id, applied.checksum]]); + + const result = await migratePluginSchemas(executor, [applied]); + + expect(result).toEqual({ existing: 1, migrated: 0, scanned: 1 }); + expect(executor.transactions).toHaveLength(0); + }); + + it("fails when an applied plugin migration checksum has changed", async () => { + const applied = migration(); + const executor = new FakeSqlExecutor([[applied.id, "old-checksum"]]); + + await expect(migratePluginSchemas(executor, [applied])).rejects.toThrow( + "Plugin migration plugin:memory/0001_init.sql checksum changed", + ); + }); + + it("reads sorted SQL files from a plugin migrations directory", () => { + const root = mkdtempSync(path.join(tmpdir(), "junior-plugin-migrations-")); + const migrationsDir = path.join(root, "migrations"); + mkdirSync(migrationsDir); + writeFileSync( + path.join(migrationsDir, "0002_second.sql"), + "CREATE TABLE junior_memory_second_plugin_table (id TEXT PRIMARY KEY);", + ); + writeFileSync( + path.join(migrationsDir, "0001_first.sql"), + "CREATE TABLE junior_memory_first_plugin_table (id TEXT PRIMARY KEY);", + ); + + try { + const migrations = readPluginMigrations({ + dir: migrationsDir, + pluginName: "memory", + }); + + expect(migrations.map((item) => item.id)).toEqual([ + "plugin:memory/0001_first.sql", + "plugin:memory/0002_second.sql", + ]); + expect(migrations[0]?.checksum).toMatch(/^[a-f0-9]{64}$/); + } finally { + rmSync(root, { force: true, recursive: true }); + } + }); + + it("accepts trusted plugin SQL without inspecting object ownership", () => { + const root = mkdtempSync(path.join(tmpdir(), "junior-plugin-migrations-")); + const migrationsDir = path.join(root, "migrations"); + mkdirSync(migrationsDir); + writeFileSync( + path.join(migrationsDir, "0001_init.sql"), + [ + "CREATE TABLE junior_memory_entries (id TEXT PRIMARY KEY);", + "CREATE INDEX junior_memory_entries_created_idx", + " ON junior_memory_entries (id);", + "INSERT INTO junior_memory_entries (id) VALUES ('seed');", + ].join("\n"), + ); + + try { + const migrations = readPluginMigrations({ + dir: migrationsDir, + pluginName: "memory", + }); + + expect(migrations).toHaveLength(1); + expect(migrations[0]?.sql).toContain("INSERT INTO"); + } finally { + rmSync(root, { force: true, recursive: true }); + } + }); + + it("rejects migration filenames outside the committed SQL pattern", () => { + const root = mkdtempSync(path.join(tmpdir(), "junior-plugin-migrations-")); + const migrationsDir = path.join(root, "migrations"); + mkdirSync(migrationsDir); + writeFileSync( + path.join(migrationsDir, "init.sql"), + "CREATE TABLE junior_memory_test (id TEXT PRIMARY KEY);", + ); + + try { + expect(() => + readPluginMigrations({ + dir: migrationsDir, + pluginName: "memory", + }), + ).toThrow('Plugin migration filename "init.sql" is invalid'); + } finally { + rmSync(root, { force: true, recursive: true }); + } + }); + + it("rejects duplicate plugin migration ids before applying SQL", async () => { + const executor = new FakeSqlExecutor(); + const pending = migration(); + + await expect( + migratePluginSchemas(executor, [ + pending, + migration({ checksum: "checksum-2" }), + ]), + ).rejects.toThrow( + "Duplicate plugin migration id plugin:memory/0001_init.sql", + ); + expect(executor.statements).toEqual([]); + }); +}); diff --git a/packages/junior/tests/unit/plugins/plugin-registry.test.ts b/packages/junior/tests/unit/plugins/plugin-registry.test.ts index aba08fb06..409d69f12 100644 --- a/packages/junior/tests/unit/plugins/plugin-registry.test.ts +++ b/packages/junior/tests/unit/plugins/plugin-registry.test.ts @@ -30,6 +30,7 @@ describe("plugin registry", () => { skillRoots: [], tracingIncludes: [], }), + normalizePluginPackageNames: (names: string[] | undefined) => names, })); const registry = await import("@/chat/plugins/registry"); @@ -57,8 +58,9 @@ describe("plugin registry", () => { packageNames: [] as string[], packages: [] as { dir: string; + hasMigrationsDir: boolean; hasSkillsDir: boolean; - name: string; + packageName: string; }[], manifestRoots: [] as string[], skillRoots: [] as string[], @@ -70,6 +72,7 @@ describe("plugin registry", () => { })); vi.doMock("@/chat/plugins/package-discovery", () => ({ discoverInstalledPluginPackageContent: () => packagedContent, + normalizePluginPackageNames: (names: string[] | undefined) => names, })); const registry = await import("@/chat/plugins/registry"); @@ -98,4 +101,262 @@ describe("plugin registry", () => { expect(registry.getPluginSkillRoots()).toContain(skillsRoot); expect(registry.isPluginProvider("demo")).toBe(true); }); + + it("does not register migrations from plugin yaml packages", async () => { + const tempRoot = await fs.mkdtemp( + path.join(os.tmpdir(), "junior-plugin-yaml-migrations-"), + ); + const pluginRoot = path.join(tempRoot, "demo-plugin"); + const migrationsRoot = path.join(pluginRoot, "migrations"); + await fs.mkdir(migrationsRoot, { recursive: true }); + await fs.writeFile( + path.join(pluginRoot, "plugin.yaml"), + ["name: demo", "display-name: Demo", "description: Demo plugin"].join( + "\n", + ), + "utf8", + ); + await fs.writeFile( + path.join(migrationsRoot, "0001_init.sql"), + "CREATE TABLE junior_demo_records (id text PRIMARY KEY);", + "utf8", + ); + + vi.doMock("@/chat/discovery", () => ({ + pluginRoots: () => [], + })); + vi.doMock("@/chat/plugins/package-discovery", () => ({ + discoverInstalledPluginPackageContent: () => ({ + packageNames: ["@acme/demo-plugin"], + packages: [ + { + dir: pluginRoot, + hasMigrationsDir: true, + hasSkillsDir: false, + packageName: "@acme/demo-plugin", + }, + ], + manifestRoots: [pluginRoot], + skillRoots: [], + tracingIncludes: [], + }), + normalizePluginPackageNames: (names: string[] | undefined) => names, + })); + + const registry = await import("@/chat/plugins/registry"); + + expect(registry.getPluginProviders()).toHaveLength(1); + expect(registry.getPluginProviders()[0]?.manifest.name).toBe("demo"); + expect(registry.getPluginMigrationRoots()).toEqual([]); + }); + + it("ignores package migrations without inline code registrations", async () => { + const tempRoot = await fs.mkdtemp( + path.join(os.tmpdir(), "junior-plugin-unowned-migrations-"), + ); + const pluginRoot = path.join(tempRoot, "code-plugin"); + await fs.mkdir(path.join(pluginRoot, "migrations"), { recursive: true }); + + vi.doMock("@/chat/discovery", () => ({ + pluginRoots: () => [], + })); + vi.doMock("@/chat/plugins/package-discovery", () => ({ + discoverInstalledPluginPackageContent: () => ({ + packageNames: ["@acme/code-plugin"], + packages: [ + { + dir: pluginRoot, + hasMigrationsDir: true, + hasSkillsDir: false, + packageName: "@acme/code-plugin", + }, + ], + manifestRoots: [], + skillRoots: [], + tracingIncludes: [], + }), + normalizePluginPackageNames: (names: string[] | undefined) => names, + })); + + const registry = await import("@/chat/plugins/registry"); + registry.setPluginCatalogConfig({ + packages: ["@acme/code-plugin"], + }); + + expect(registry.getPluginMigrationRoots()).toEqual([]); + }); + + it("registers named migrations from inline code plugin packages", async () => { + const tempRoot = await fs.mkdtemp( + path.join(os.tmpdir(), "junior-plugin-code-migrations-"), + ); + const pluginRoot = path.join(tempRoot, "code-plugin"); + const migrationsRoot = path.join(pluginRoot, "migrations"); + await fs.mkdir(migrationsRoot, { recursive: true }); + await fs.writeFile( + path.join(migrationsRoot, "0001_init.sql"), + "CREATE TABLE junior_code_plugin_records (id text PRIMARY KEY);", + "utf8", + ); + + vi.doMock("@/chat/discovery", () => ({ + pluginRoots: () => [], + })); + vi.doMock("@/chat/plugins/package-discovery", () => ({ + discoverInstalledPluginPackageContent: () => ({ + packageNames: ["@acme/code-plugin"], + packages: [ + { + dir: pluginRoot, + hasMigrationsDir: true, + hasSkillsDir: false, + packageName: "@acme/code-plugin", + }, + ], + manifestRoots: [], + skillRoots: [], + tracingIncludes: [], + }), + normalizePluginPackageNames: (names: string[] | undefined) => names, + })); + + const registry = await import("@/chat/plugins/registry"); + registry.setPluginCatalogConfig({ + packages: ["@acme/code-plugin"], + inlineManifests: [ + { + packageName: "@acme/code-plugin", + manifest: { + name: "code-plugin", + displayName: "Code Plugin", + description: "Code plugin", + capabilities: [], + configKeys: [], + }, + }, + ], + }); + + expect(registry.getPluginMigrationRoots()).toEqual([ + { pluginName: "code-plugin", dir: migrationsRoot }, + ]); + }); + + it("reloads inline migration roots when package metadata changes", async () => { + const tempRoot = await fs.mkdtemp( + path.join(os.tmpdir(), "junior-plugin-migration-reload-"), + ); + const pluginRoot = path.join(tempRoot, "code-plugin"); + const migrationsRoot = path.join(pluginRoot, "migrations"); + await fs.mkdir(pluginRoot, { recursive: true }); + + const packagedContent = { + packageNames: ["@acme/code-plugin"], + packages: [ + { + dir: pluginRoot, + hasMigrationsDir: false, + hasSkillsDir: false, + packageName: "@acme/code-plugin", + }, + ], + manifestRoots: [] as string[], + skillRoots: [] as string[], + tracingIncludes: [] as string[], + }; + + vi.doMock("@/chat/discovery", () => ({ + pluginRoots: () => [], + })); + vi.doMock("@/chat/plugins/package-discovery", () => ({ + discoverInstalledPluginPackageContent: () => packagedContent, + normalizePluginPackageNames: (names: string[] | undefined) => names, + })); + + const registry = await import("@/chat/plugins/registry"); + registry.setPluginCatalogConfig({ + packages: ["@acme/code-plugin"], + inlineManifests: [ + { + packageName: "@acme/code-plugin", + manifest: { + name: "code-plugin", + displayName: "Code Plugin", + description: "Code plugin", + capabilities: [], + configKeys: [], + }, + }, + ], + }); + + expect(registry.getPluginMigrationRoots()).toEqual([]); + + await fs.mkdir(migrationsRoot); + packagedContent.packages[0]!.hasMigrationsDir = true; + + expect(registry.getPluginMigrationRoots()).toEqual([ + { pluginName: "code-plugin", dir: migrationsRoot }, + ]); + }); + + it("rejects shared package migrations across inline registrations", async () => { + const tempRoot = await fs.mkdtemp( + path.join(os.tmpdir(), "junior-plugin-shared-migrations-"), + ); + const pluginRoot = path.join(tempRoot, "code-plugin"); + await fs.mkdir(path.join(pluginRoot, "migrations"), { recursive: true }); + + vi.doMock("@/chat/discovery", () => ({ + pluginRoots: () => [], + })); + vi.doMock("@/chat/plugins/package-discovery", () => ({ + discoverInstalledPluginPackageContent: () => ({ + packageNames: ["@acme/code-plugin"], + packages: [ + { + dir: pluginRoot, + hasMigrationsDir: true, + hasSkillsDir: false, + packageName: "@acme/code-plugin", + }, + ], + manifestRoots: [], + skillRoots: [], + tracingIncludes: [], + }), + normalizePluginPackageNames: (names: string[] | undefined) => names, + })); + + const registry = await import("@/chat/plugins/registry"); + registry.setPluginCatalogConfig({ + packages: ["@acme/code-plugin"], + inlineManifests: [ + { + packageName: "@acme/code-plugin", + manifest: { + name: "code-plugin", + displayName: "Code Plugin", + description: "Code plugin", + capabilities: [], + configKeys: [], + }, + }, + { + packageName: "@acme/code-plugin", + manifest: { + name: "other-plugin", + displayName: "Other Plugin", + description: "Other plugin", + capabilities: [], + configKeys: [], + }, + }, + ], + }); + + expect(() => registry.getPluginMigrationRoots()).toThrow( + 'Plugin "other-plugin" cannot share migrations directory with plugin "code-plugin"', + ); + }); }); diff --git a/packages/junior/tests/unit/slack/tool-registration.test.ts b/packages/junior/tests/unit/slack/tool-registration.test.ts index ab7d54e20..9e08085dc 100644 --- a/packages/junior/tests/unit/slack/tool-registration.test.ts +++ b/packages/junior/tests/unit/slack/tool-registration.test.ts @@ -1,8 +1,10 @@ -import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { PluginDb } from "@sentry/junior-plugin-api"; import { createTools } from "@/chat/tools"; import type { ToolRuntimeContext } from "@/chat/tools/types"; import { schedulerPlugin } from "@sentry/junior-scheduler"; -import { setAgentPlugins } from "@/chat/plugins/agent-hooks"; +import { setPlugins } from "@/chat/plugins/agent-hooks"; +import * as pluginDbModule from "@/chat/plugins/db"; const noopSandbox = {} as any; function ctx(): Extract; @@ -41,11 +43,12 @@ function ctx(channelId?: string): ToolRuntimeContext { describe("Slack tool registration", () => { beforeEach(() => { - setAgentPlugins([schedulerPlugin()]); + setPlugins([schedulerPlugin()]); }); afterEach(() => { - setAgentPlugins([]); + setPlugins([]); + vi.restoreAllMocks(); }); it("does not register channel-scope tools in DM context", () => { @@ -87,6 +90,9 @@ describe("Slack tool registration", () => { }); it("registers schedule tools only with complete Slack turn context", () => { + vi.spyOn(pluginDbModule, "getPluginDbForRegistration").mockReturnValue( + {} as PluginDb, + ); const incomplete = createTools([], {}, ctx("C12345")); const complete = createTools( [], diff --git a/packages/junior/vitest.config.ts b/packages/junior/vitest.config.ts index bd8b44bd7..01e709dff 100644 --- a/packages/junior/vitest.config.ts +++ b/packages/junior/vitest.config.ts @@ -34,6 +34,10 @@ export default defineConfig({ __dirname, "../junior-plugin-api/src/index.ts", ), + "@sentry/junior-scheduler": path.resolve( + __dirname, + "../junior-scheduler/src/index.ts", + ), }, }, test: { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 0ade0f9a6..898d6e2b9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -13,6 +13,12 @@ catalogs: "@sentry/starlight-theme": specifier: ^0.7.0 version: 0.7.0 + drizzle-orm: + specifier: ^0.45.2 + version: 0.45.2 + zod: + specifier: ^4.4.3 + version: 4.4.3 overrides: ai: 6.0.190 @@ -155,7 +161,7 @@ importers: version: 1.1.0 "@sentry/junior-plugin-api": specifier: workspace:* - version: link:../junior-plugin-api + version: file:packages/junior-plugin-api(@neondatabase/serverless@1.1.0) "@sentry/node": specifier: "catalog:" version: 10.53.1 @@ -184,7 +190,7 @@ importers: specifier: 4.29.0 version: 4.29.0(ai@6.0.190(zod@4.4.3))(zod@4.4.3) drizzle-orm: - specifier: ^0.45.2 + specifier: "catalog:" version: 0.45.2(@neondatabase/serverless@1.1.0) hono: specifier: ^4.12.22 @@ -202,7 +208,7 @@ importers: specifier: ^2.9.0 version: 2.9.0 zod: - specifier: ^4.4.3 + specifier: "catalog:" version: 4.4.3 devDependencies: "@emnapi/core": @@ -213,7 +219,7 @@ importers: version: 1.10.0 "@sentry/junior-scheduler": specifier: workspace:* - version: link:../junior-scheduler + version: file:packages/junior-scheduler(@neondatabase/serverless@1.1.0) "@sentry/junior-test-fixtures": specifier: workspace:* version: file:packages/junior-test-fixtures(@neondatabase/serverless@1.1.0) @@ -339,6 +345,9 @@ importers: "@sentry/junior-sentry": specifier: workspace:* version: link:../junior-sentry + "@sentry/junior-test-fixtures": + specifier: workspace:* + version: link:../junior-test-fixtures "@sentry/junior-testing": specifier: workspace:* version: link:../junior-testing @@ -374,8 +383,11 @@ importers: packages/junior-plugin-api: dependencies: + drizzle-orm: + specifier: "catalog:" + version: 0.45.2 zod: - specifier: ^4.4.3 + specifier: "catalog:" version: 4.4.3 devDependencies: oxlint: @@ -396,6 +408,12 @@ importers: "@sinclair/typebox": specifier: ^0.34.49 version: 0.34.49 + drizzle-orm: + specifier: "catalog:" + version: 0.45.2 + zod: + specifier: "catalog:" + version: 4.4.3 devDependencies: "@types/node": specifier: ^25.9.1 @@ -415,7 +433,7 @@ importers: specifier: ^0.4.6 version: 0.4.6 drizzle-orm: - specifier: ^0.45.2 + specifier: "catalog:" version: 0.45.2(@electric-sql/pglite@0.4.6) devDependencies: "@types/node": @@ -3868,6 +3886,9 @@ packages: "@sentry/junior-plugin-api@file:packages/junior-plugin-api": resolution: { directory: packages/junior-plugin-api, type: directory } + "@sentry/junior-scheduler@file:packages/junior-scheduler": + resolution: { directory: packages/junior-scheduler, type: directory } + "@sentry/junior-test-fixtures@file:packages/junior-test-fixtures": resolution: { directory: packages/junior-test-fixtures, type: directory } @@ -13855,6 +13876,7 @@ snapshots: - "@libsql/client" - "@libsql/client-wasm" - "@lynx-js/react" + - "@neondatabase/serverless" - "@netlify/blobs" - "@netlify/runtime" - "@node-rs/xxhash" @@ -13923,7 +13945,110 @@ snapshots: "@sentry/junior-plugin-api@file:packages/junior-plugin-api": dependencies: + drizzle-orm: 0.45.2 + zod: 4.4.3 + transitivePeerDependencies: + - "@aws-sdk/client-rds-data" + - "@cloudflare/workers-types" + - "@electric-sql/pglite" + - "@libsql/client" + - "@libsql/client-wasm" + - "@neondatabase/serverless" + - "@op-engineering/op-sqlite" + - "@opentelemetry/api" + - "@planetscale/database" + - "@prisma/client" + - "@tidbcloud/serverless" + - "@types/better-sqlite3" + - "@types/pg" + - "@types/sql.js" + - "@upstash/redis" + - "@vercel/postgres" + - "@xata.io/client" + - better-sqlite3 + - bun-types + - expo-sqlite + - gel + - knex + - kysely + - mysql2 + - pg + - postgres + - prisma + - sql.js + - sqlite3 + + "@sentry/junior-plugin-api@file:packages/junior-plugin-api(@neondatabase/serverless@1.1.0)": + dependencies: + drizzle-orm: 0.45.2(@neondatabase/serverless@1.1.0) + zod: 4.4.3 + transitivePeerDependencies: + - "@aws-sdk/client-rds-data" + - "@cloudflare/workers-types" + - "@electric-sql/pglite" + - "@libsql/client" + - "@libsql/client-wasm" + - "@neondatabase/serverless" + - "@op-engineering/op-sqlite" + - "@opentelemetry/api" + - "@planetscale/database" + - "@prisma/client" + - "@tidbcloud/serverless" + - "@types/better-sqlite3" + - "@types/pg" + - "@types/sql.js" + - "@upstash/redis" + - "@vercel/postgres" + - "@xata.io/client" + - better-sqlite3 + - bun-types + - expo-sqlite + - gel + - knex + - kysely + - mysql2 + - pg + - postgres + - prisma + - sql.js + - sqlite3 + + "@sentry/junior-scheduler@file:packages/junior-scheduler(@neondatabase/serverless@1.1.0)": + dependencies: + "@sentry/junior-plugin-api": file:packages/junior-plugin-api(@neondatabase/serverless@1.1.0) + "@sinclair/typebox": 0.34.49 + drizzle-orm: 0.45.2(@neondatabase/serverless@1.1.0) zod: 4.4.3 + transitivePeerDependencies: + - "@aws-sdk/client-rds-data" + - "@cloudflare/workers-types" + - "@electric-sql/pglite" + - "@libsql/client" + - "@libsql/client-wasm" + - "@neondatabase/serverless" + - "@op-engineering/op-sqlite" + - "@opentelemetry/api" + - "@planetscale/database" + - "@prisma/client" + - "@tidbcloud/serverless" + - "@types/better-sqlite3" + - "@types/pg" + - "@types/sql.js" + - "@upstash/redis" + - "@vercel/postgres" + - "@xata.io/client" + - better-sqlite3 + - bun-types + - expo-sqlite + - gel + - knex + - kysely + - mysql2 + - pg + - postgres + - prisma + - sql.js + - sqlite3 "@sentry/junior-test-fixtures@file:packages/junior-test-fixtures(@neondatabase/serverless@1.1.0)": dependencies: @@ -13970,7 +14095,7 @@ snapshots: "@logtape/logtape": 2.1.1 "@modelcontextprotocol/sdk": 1.29.0(zod@4.4.3) "@neondatabase/serverless": 1.1.0 - "@sentry/junior-plugin-api": file:packages/junior-plugin-api + "@sentry/junior-plugin-api": file:packages/junior-plugin-api(@neondatabase/serverless@1.1.0) "@sentry/node": 10.53.1 "@sinclair/typebox": 0.34.49 "@slack/web-api": 7.16.0 @@ -15698,6 +15823,8 @@ snapshots: domelementtype: 2.3.0 domhandler: 5.0.3 + drizzle-orm@0.45.2: {} + drizzle-orm@0.45.2(@electric-sql/pglite@0.4.6): optionalDependencies: "@electric-sql/pglite": 0.4.6 @@ -18824,17 +18951,16 @@ snapshots: source-map-js@1.2.1: {} - source-map@0.6.1: - optional: true + source-map@0.6.1: {} source-map@0.7.6: {} - space-separated-tokens@2.0.2: {} - split@0.2.10: dependencies: through: 2.3.8 + space-separated-tokens@2.0.2: {} + sprintf-js@1.1.3: {} sql.js@1.14.1: {} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 184610e59..75a855bb6 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -4,6 +4,9 @@ packages: catalog: "@sentry/node": 10.53.1 "@sentry/starlight-theme": ^0.7.0 + drizzle-kit: ^0.31.8 + drizzle-orm: ^0.45.2 + zod: ^4.4.3 syncInjectedDepsAfterScripts: - build minimumReleaseAge: 1440 diff --git a/specs/agent-prompt.md b/specs/agent-prompt.md index 1da6b2b2e..759a366c9 100644 --- a/specs/agent-prompt.md +++ b/specs/agent-prompt.md @@ -3,7 +3,7 @@ ## Metadata - Created: 2026-04-28 -- Last Edited: 2026-06-11 +- Last Edited: 2026-06-12 ## Purpose @@ -21,7 +21,10 @@ Define the canonical contract for Junior's platform-owned agent prompt so prompt - Defining Pi agent loop mechanics or terminal output assembly; see `./harness-agent.md`. - Defining Slack delivery transport behavior; see `./slack-agent-delivery.md` and `./slack-outbound-contract.md`. - Defining test-layer taxonomy; see `./testing.md`. -- Defining plugin-specific prompt overlays or provider workflows. Plugins own that guidance through their skills, tools, schemas, and tool guidance. +- Defining plugin prompt hook contracts; see `./plugin-prompt-hooks.md` for the + future target design. Those hooks are not implemented in the current plugin + API. +- Defining provider workflows. Plugins own provider guidance through their skills, tools, schemas, tool guidance, and prompt hooks. ## Contracts @@ -43,6 +46,8 @@ Turn context may disclose dynamic capability surfaces that the model can act on, Turn context is not a session-state cache. If prior tool use, loaded skills, MCP provider activation, or provider descriptors are already present in the agent session log, runtime must recover handles from that log and only disclose the currently actionable capability surface for this turn. Do not add prompt blocks whose purpose is to preserve or replay state that belongs in the session log. +Future plugin prompt contributions are governed by `./plugin-prompt-hooks.md`. Core prompt code owns where accepted plugin contributions render, and plugin-provided session append state is plugin-visible bookkeeping rather than model-visible prompt history. Those prompt contribution hooks are not implemented in the current plugin API. + The combined prompt surface must keep these concerns distinct: 1. Identity/personality. @@ -165,6 +170,7 @@ When debugging prompt behavior, use existing turn diagnostics, observed tool inv ## Related Specs - `./harness-agent.md` +- `./plugin-prompt-hooks.md` - `./harness-tool-context.md` - `./slack-agent-delivery.md` - `./slack-outbound-contract.md` diff --git a/specs/conversation-storage.md b/specs/conversation-storage.md index 47f53890b..ca62d8eae 100644 --- a/specs/conversation-storage.md +++ b/specs/conversation-storage.md @@ -10,9 +10,10 @@ Define Junior's first SQL-backed storage contract for queryable conversation records without moving transcript authorities into SQL. -This storage exists to support stats, dashboard lists, audit queries, -conversation configuration, durable source/destination/identity metadata, and -deploy-safe schema evolution. +This storage is the first feature-owned slice of Junior's shared SQL database. +It supports stats, dashboard lists, audit queries, conversation configuration, +durable source/destination/identity metadata, and deploy-safe schema evolution. +Plugin-owned SQL extensions are governed by `./plugin-database.md`. ## Scope @@ -40,7 +41,9 @@ deploy-safe schema evolution. ### Data Authorities SQL owns durable, queryable Junior data. This spec covers the first -feature-owned slice: conversation records and their long-term metadata. +feature-owned slice: conversation records and their long-term metadata. Plugin +tables may join the same shared database through the package migration contract +in `./plugin-database.md`. The transcript authorities from `./task-execution.md` remain unchanged: @@ -124,6 +127,8 @@ source-specific JSON extraction. Future slices may add feature-owned SQL tables for conversation configuration, artifact references, agent-run summaries, scheduler links, and other metadata concerns once their owning store interfaces are implemented. +Plugin-owned slices add tables through `./plugin-database.md` and must keep +their table names under their plugin-owned prefix. Opaque JSON columns are allowed for source-specific payloads that are not used for authorization, lock ownership, credential routing, or external side-effect @@ -270,6 +275,7 @@ normal runtime tests. - `./chat-architecture.md` - `./agent-session-resumability.md` - `./scheduler.md` +- `./plugin-database.md` - `./dashboard.md` - `./testing.md` diff --git a/specs/dashboard.md b/specs/dashboard.md index 43dc143bc..8b4e893f3 100644 --- a/specs/dashboard.md +++ b/specs/dashboard.md @@ -105,7 +105,7 @@ export interface JuniorDashboardPluginOptions { export function juniorDashboardPlugin( options?: JuniorDashboardPluginOptions, -): JuniorPluginRegistration; +): PluginRegistration; ``` The plugin factory is the normal dashboard integration path. When registered diff --git a/specs/index.md b/specs/index.md index 29d47f320..26660dd70 100644 --- a/specs/index.md +++ b/specs/index.md @@ -3,7 +3,7 @@ ## Metadata - Created: 2026-03-03 -- Last Edited: 2026-06-12 +- Last Edited: 2026-06-13 ## Purpose @@ -47,6 +47,10 @@ Define spec taxonomy, naming conventions, and canonical source-of-truth document - `specs/scheduler.md` - `specs/plugin-heartbeat.md` - `specs/plugin-dispatch.md` +- `specs/plugin-prompt-hooks.md` +- `specs/plugin-database.md` +- `specs/plugin-cli.md` +- `specs/memory-plugin/index.md` - `specs/harness-agent.md` - `specs/agent-session-resumability.md` - `specs/agent-execution.md` @@ -75,6 +79,9 @@ For chat/agent/Slack execution and response behavior: - `specs/chat-architecture.md` owns the end-to-end platform-event-to-agent-run data flow, platform adapter boundary, data authority map, and module boundaries. - `specs/task-execution.md` owns durable conversation mailbox execution, queue wake-up semantics, conversation leases, cooperative yield, and heartbeat repair. - `specs/conversation-storage.md` owns SQL-backed queryable conversation record, transcript-storage exclusions, and Vercel-safe migration/backfill behavior. +- `specs/plugin-database.md` owns plugin packaged SQL migration discovery/application and the trusted `ctx.db` hook surface. +- `specs/plugin-cli.md` owns future plugin-contributed host CLI command discovery, dispatch, admin context, and redaction contracts. +- `specs/memory-plugin/index.md` owns the long-term memory plugin's storage, recall, passive learning, tools, visibility, and lifecycle contracts. - `specs/local-agent.md` owns local CLI/local adapter user flows, identity, state, delivery, and verification contracts. - `specs/agent-turn-handling.md` owns user-message response policy: when Junior answers, stays silent, asks, uses tools, satisfies Slack side effects, handles resumed turns, and considers a turn complete. - `specs/agent-execution.md` owns coding-agent execution discipline and the repository-wide model-repairable tool failure contract. diff --git a/specs/memory-plugin/admin.md b/specs/memory-plugin/admin.md new file mode 100644 index 000000000..77417554d --- /dev/null +++ b/specs/memory-plugin/admin.md @@ -0,0 +1,137 @@ +# Memory Plugin Admin + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define operator/admin capabilities for inspecting and repairing memory state +outside model-visible tools. + +## Scope + +- Future plugin-contributed CLI command shape for memory. +- Admin operations for inspection, removal, repair, and embedding maintenance. +- Security and redaction rules for operator output. + +## Non-Goals + +- Requiring memory admin CLI in the first implementation slice. +- Letting the model invoke admin commands. +- Defining a dashboard UI for memory administration. +- Defining account deletion, legal export, or retention workflows. + +## Command Shape + +The memory plugin should reserve a future plugin CLI command: + +```txt +junior memory ... +``` + +This command is registered through the plugin CLI surface described in +[`../plugin-cli.md`](../plugin-cli.md). It is not a model-visible tool and is +not available inside sandbox command execution. + +Possible subcommands: + +- `junior memory stats` +- `junior memory list` +- `junior memory show ` +- `junior memory remove ` +- `junior memory repair` +- `junior memory rebuild-embeddings` + +The exact subcommand set can be narrowed during implementation. The broad need +is an operator surface for visibility debugging and repair that does not expand +the model tool surface. + +## Admin Context + +The command must run with a host/admin context, not as an inferred Slack or +local chat requester. + +Commands that operate on user-visible memory must require explicit selectors +such as requester identity, conversation identity, source platform, or memory +id. Selectors are resolved through the same storage visibility model used by +runtime code; display names and labels are not authorities. + +## Operations + +### stats + +Reports aggregate counts by scope type, memory type, sensitivity, archive +state, embedding status, repair status, and policy-hidden status. + +Default output must not include raw memory content. + +### list + +Lists memories for an explicit scope or query. + +Default output should include ids, type, sensitivity, timestamps, archive +state, and short redacted previews. Full content requires an explicit flag such +as `--show-content`. + +### show + +Shows one memory by id when the operator explicitly requests it. + +The output may include content, source attribution, lifecycle state, embedding +status, and bounded metadata. It must not include raw transcript payloads, +provider credentials, tokens, or raw extraction prompts. + +### remove + +Archives one memory by id with an admin archive reason. + +The command must not physically delete rows in V1. Account deletion and legal +retention flows need a separate retention/export spec. + +### repair + +Runs bounded consistency repair: + +- archive expired memories +- repair malformed lifecycle markers when deterministic +- identify missing or stale embedding rows +- enqueue embedding repair tasks + +Repair should report counts and task ids, not raw content. + +Repair must not silently make policy-hidden memories visible. If policy changes +make stored rows disallowed, repair should report counts and allow a future +operator workflow to archive them. + +### rebuild-embeddings + +Enqueues or runs bounded embedding rebuild work for selected memories. + +The command should prefer background task enqueueing for large work. If it runs +inline locally, it must use the same embedding provider boundary and dimension +checks as runtime storage. + +## Security Rules + +1. Admin commands are privileged host operations, not user-facing chat actions. +2. Default output must avoid raw private memory content. +3. Full-content output requires an explicit operator action. +4. Commands must not reveal secrets even with `--show-content`; secret + detection failures found during admin inspection should be reported as + repair findings and handled through a future deletion/retention workflow. +5. Commands must not accept model-style arbitrary actor, team, channel, thread, + or conversation ids as implicit authority. Selectors identify what to inspect; + deployment/operator authorization is a separate boundary. +6. Logs and spans for admin commands follow [`./security.md`](./security.md). + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./security.md` +- `./tools.md` +- `./verification.md` +- `../plugin-cli.md` diff --git a/specs/memory-plugin/extraction.md b/specs/memory-plugin/extraction.md new file mode 100644 index 000000000..85aeed2db --- /dev/null +++ b/specs/memory-plugin/extraction.md @@ -0,0 +1,242 @@ +# Memory Plugin Extraction + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define passive memory learning through completed-turn observation and plugin +background tasks. + +## What Belongs In Memory + +A stored memory is a self-contained assertion that can improve future +assistance without requiring the original conversation. + +A candidate may be stored only when all of these are true: + +1. Install-level memory policy allows this category, scope, and source. +2. It is a concrete fact, preference, relationship, durable project fact, + durable workflow preference, or explicit user request to remember something. +3. It is useful beyond the current turn or has an explicit expiration. +4. It is understandable without unresolved pronouns or hidden conversation + context. +5. It has a runtime-derived source actor and source conversation. +6. It has a runtime-derived visibility scope. +7. It contains no credential, token, private key, password, recovery code, + connection string with credentials, payment card number, or similar secret. +8. It is not merely an assistant claim, assistant action, tool result summary, + system capability, implementation detail, or prompt/routing rule. + +Examples that can be stored: + +- `User prefers concise technical answers.` +- `User's production deploy window is Mondays from 10:00 to 12:00 UTC.` +- `The #infra conversation uses Linear for incident follow-up.` +- `User wants Junior to remember that the Acme migration is paused.` + +Examples that must not be stored: + +- `The assistant searched GitHub.` +- `The user asked a question about the memory system.` +- `The OAuth token is xoxb-...` +- `The user is somewhere next week.` +- `The user has not decided what to do.` +- `Junior can use the scheduler plugin.` + +## Passive Learning + +The memory plugin observes completed turns through `observeTurn(ctx)`. + +The observation hook must: + +1. Run only after the user-visible turn is durably committed enough that + observation failure cannot fail delivery. +2. Enqueue one plugin background task for extraction from the completed turn. +3. Ignore assistant-authored claims as memory sources. +4. Skip task enqueueing when the source is not allowed to expose private turn + text to the trusted memory plugin. +5. Skip task enqueueing when install policy disables passive extraction for the + current source, scope, or requester. +6. Skip task enqueueing unless the source conversation is classified as + `public` by Junior's existing conversation privacy/destination visibility + contracts. +7. Use a stable idempotency key derived from the completed turn or source event. + +The observation hook does not perform extraction inline. It requests work from +core: + +```ts +await ctx.tasks.enqueue({ + name: "extractMemories", + idempotencyKey: ctx.observationId, + payload: { + observationId: ctx.observationId, + }, +}); +``` + +The payload must contain stable references and safe metadata only. It must not +contain raw private user text, raw assistant text, raw tool payloads, +credentials, or tokens. Core owns how the task is delivered: the existing +serverless queue, a signed callback, a future dedicated task worker, or a local +test worker are all valid implementations. + +Core must not require plugin code to know queue topic names, queue message +shape, Vercel-specific APIs, callback routes, visibility timeouts, or +acknowledgement semantics. + +## Extraction Task Handler + +The memory plugin's `extractMemories` task handler must: + +1. Reload the bounded observation payload for the referenced completed turn + through `ctx.observation.load()`. +2. Reload current install-level memory policy. +3. Process only that completed turn. +4. Extract candidate facts with a structured model output contract. +5. Ignore assistant-authored claims as memory sources. +6. Skip extraction when the bounded observation payload is unavailable, + expired, malformed, or no longer visible to the plugin. +7. Run policy adjudication for extracted candidates. +8. Reject malformed, low-confidence, incoherent, duplicate, unsafe, or + out-of-scope facts. +9. Reject facts disallowed by install policy, including workplace-sensitive + categories. +10. Convert relative times to absolute dates using `observed_at`. +11. Assign type, sensitivity, scope, and optional expiration. +12. Run centralized secret detection immediately before writing memory rows. +13. Insert accepted memories transactionally. +14. Generate or queue embeddings for accepted rows when configured and allowed + by policy. +15. Archive expired, superseded, or explicitly removed memories in bounded + batches. +16. Avoid storing raw extraction prompt, raw model output, or raw turn text + beyond the accepted memory records. + +Extraction tasks must be idempotent. If the same completed turn is observed or +delivered more than once, source idempotency fields and duplicate detection must +prevent duplicate memories. + +The task handler must be safe to run in a separate serverless invocation from +the original user turn. It must not depend on process memory, live Slack +clients, raw HTTP requests, provider tokens, or the model-visible prompt object +from the original run. + +## Extraction Rules + +Extraction must follow these rules: + +1. Extract only from user-authored text. +2. Prefer explicit "remember" requests over inferred passive learning. +3. Store facts, not conversation summaries. +4. Make content self-contained. +5. Reject unresolved references such as "that", "it", "the thing", "someone", + or "somewhere" when the referenced value is not present. +6. Reject negative knowledge such as "the user has not decided yet". +7. Reject assistant/system implementation details. +8. Reject low-utility facts that will not help 30 days later unless they have + explicit expiration. +9. Assign `context`, `event`, `task`, or `observation` for facts that should + decay. +10. Treat extraction confidence below the configured threshold as not stored. +11. Reject workplace-sensitive categories disallowed by install policy, such as + HR/performance, protected-class, health, legal, financial, gossip, or + coworker speculation. +12. In V1 passive extraction, prefer conversation-scoped operational knowledge + over personal memory. +13. Preserve provenance for third-party claims when the source matters for + correctness. +14. Store the minimum useful assertion rather than a direct quote or broad + summary. + +The plugin must have a deterministic post-extraction validation layer. The +extraction prompt is guidance, not the security boundary. + +## Policy Adjudication + +Policy enforcement may use a second model call after extraction. This should be +the default V1 shape when passive extraction is enabled: + +1. The extraction model proposes structured candidate memories from the bounded + observation payload. +2. A policy adjudicator, typically the configured fast/auxiliary model, reviews + each candidate against the installed memory policy and workplace guidance. +3. The deterministic validator applies hard rules and rejects anything unsafe or + malformed before storage. + +The policy adjudicator should receive only the candidate memory, the minimum +source context needed to judge it, and the installed policy guidance. It should +not receive unrestricted transcript history, raw tool payloads, provider +credentials, or unrelated conversation context. + +Policy adjudication output must be structured. It should include: + +- candidate id +- decision: `allow` or `reject` +- normalized rejection reason code when rejected +- optional adjusted memory type, sensitivity, scope, expiration, or content + rewrite +- confidence + +The adjudicator may narrow, rewrite, or reject extracted candidates, but it may +not override hard validators. If extraction and policy adjudication disagree, +the stricter outcome wins. If the policy adjudicator fails or returns malformed +output, the candidate is rejected unless it came from an explicit tool workflow +that can return a model-visible retryable error. + +## Secret Rejection + +Every entry point must call the same secret detector before writing memory +content: + +- `createMemory` +- passive extraction +- repair/import workflows +- tests and fixture helpers that create real memory records + +Every entry point must also run the same deterministic policy filter before +writing memory content. Explicit tools may use explicit user intent as a policy +input, but they do not bypass the filter. + +The detector must reject at least: + +- API keys and access tokens +- Slack tokens +- passwords and passphrases +- private keys +- recovery codes and MFA codes +- credit card numbers +- Social Security numbers +- connection strings with embedded credentials + +If a user explicitly asks Junior to remember a secret, the correct behavior is +a model-visible rejection, not storage with `sensitive`. + +## Duplicate And Supersession Rules + +Duplicate prevention is required before insertion: + +- same source observation id and same extracted fact index +- exact normalized content match in the same scope +- high lexical or embedding similarity to an active memory in the same scope + +Supersession is allowed when a new memory clearly replaces an old memory in the +same scope, such as a changed preference. Superseded memories remain archived +in place and are excluded from recall and list results unless explicitly +requested by an administrative repair workflow. + +V1 may implement conservative supersession only. If conflict is uncertain, +store the new fact without archiving the old one or skip the new fact; do not +guess. + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./security.md` +- `../plugin-prompt-hooks.md` +- `../data-redaction-policy.md` diff --git a/specs/memory-plugin/index.md b/specs/memory-plugin/index.md new file mode 100644 index 000000000..8c8931469 --- /dev/null +++ b/specs/memory-plugin/index.md @@ -0,0 +1,293 @@ +# Memory Plugin Spec + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define Junior's first long-term memory implementation as an explicitly enabled +runtime hook plugin with strict storage, recall, visibility, and deletion +contracts. + +## Implementation Status + +This spec describes the intended V1 memory plugin shape. It depends on future +plugin hook surfaces from `../plugin-prompt-hooks.md`; the current plugin API +does not yet export or invoke `userPrompt`, `observeTurn`, plugin prompt session +state, or plugin background task handlers. + +When automatic memory injection is enabled, the memory plugin makes relevant +facts available before each response without making recall depend on the model +choosing a search tool. When automatic memory injection is disabled, +model-visible recall is explicit through `searchMemories`. Other explicit tools +support user-directed memory management. + +## Scope + +- What is eligible for long-term memory. +- Install-level policy controls for workplace-safe extraction and recall. +- Memory plugin package shape and required plugin hooks. +- Plugin-owned SQL storage, retrieval indexes, embeddings, and model-provider + boundaries. +- Automatic recall through `userPrompt` when `autoInjectMemories` is enabled. +- Passive learning through `observeTurn` plus a plugin background task handler. +- Explicit `createMemory`, `removeMemory`, `listMemories`, and + `searchMemories` tools. +- Scope, attribution, sensitivity, lifecycle, tool, model, and secret rejection + rules. +- V1 implementation order and verification requirements. + +## Non-Goals + +- A core memory API outside the plugin system. +- A person graph, alias resolver, or multi-hop social retrieval. +- Cross-context recall between unrelated conversations. +- Requiring search tools when automatic memory injection is enabled. +- Storing conversation transcript history as memory. +- Storing credentials, secrets, raw OAuth data, or provider tokens. +- Letting model-supplied tool arguments choose actors, Slack workspaces, + channels, teams, or arbitrary visibility scopes. +- Exposing memory content through logs, traces, dashboard metadata, or plugin + operational reports for private conversations. + +## Spec Map + +Read these files as one canonical spec: + +- [storage.md](./storage.md): SQL storage model, retrieval indexes, pgvector, + embedding model provider, and operational storage rules. +- [policy.md](./policy.md): install-level controls for memory categories, + passive extraction, workplace-sensitive facts, model/provider use, and + retention. +- [security.md](./security.md): authority boundaries, multi-user visibility, + model/tool boundaries, task payload safety, and redaction rules. +- [retrieval.md](./retrieval.md): automatic recall, tool-mediated recall, + hybrid ranking, automatic injection mechanics, and performance strategy. +- [extraction.md](./extraction.md): passive observation, background extraction, + storable-fact policy, duplicate detection, and supersession. +- [tools.md](./tools.md): model-visible memory management and recall tools. +- [admin.md](./admin.md): future operator/admin CLI command shape for memory + inspection and repair. +- [verification.md](./verification.md): failure model, observability, and test + requirements. + +## Design Inputs + +The V1 shape is adapted from `~/src/ash/specs/memory/*`: use Ash's memory type +taxonomy, sensitivity split, centralized secret rejection, temporal rewriting, +and duplicate/supersession discipline, but omit Ash's person graph and +cross-context traversal until Junior has a stricter identity and disclosure +model for that behavior. + +External storage and retrieval assumptions are based on primary documentation: + +- [pgvector](https://github.com/pgvector/pgvector) for Postgres-native vector + columns, exact nearest-neighbor search, and HNSW/IVFFlat indexes. +- [Neon pgvector docs](https://neon.com/docs/extensions/pgvector) because + Junior's SQL adapter targets Neon-compatible Postgres. +- [Drizzle PostgreSQL extension docs](https://orm.drizzle.team/docs/extensions/pg) + for plugin-owned typed `vector` columns. +- [OpenAI embeddings docs](https://platform.openai.com/docs/guides/embeddings) + for current embedding model and dimension behavior. + +## Plugin Shape + +The V1 memory implementation is a trusted host plugin registered through +`defineJuniorPlugin({ manifest, database, hooks })`. + +The plugin uses the package name and plugin name `memory`. Plugin database +tables use the prefix: + +```txt +junior_memory_* +``` + +The V1 runtime plugin interface is: + +```ts +defineJuniorPlugin({ + manifest, + database: {}, + hooks: { + userPrompt, + observeTurn, + tasks: { + extractMemories, + embedMemories, + }, + tools, + }, +}); +``` + +`embedMemories` may be implemented as the same internal handler as extraction +backfill, but it is named separately so embedding repair can be queued without +pretending a completed turn needs to be re-extracted. + +The exact hook and task type names are owned by their generic plugin specs. The +memory plugin needs these broad V1 surfaces: optional automatic recall, +completed-turn observation, background task handling, model-visible memory +tools, SQL access, and host-owned embedding-provider access. A future admin CLI +surface is specified separately in [`./admin.md`](./admin.md). + +The plugin must also receive install-level memory policy before hooks execute. +Policy controls whether passive extraction is enabled, whether automatic memory +injection is enabled, what categories and scopes may be stored, which providers +may receive memory text, and which retention defaults apply. + +V1 passive extraction targets workplace knowledge from conversations classified +as `public` by Junior's existing conversation privacy/destination visibility +contracts. Private, direct, unknown, or unsupported sources can still use +explicit memory tools when policy allows them, but passive learning from those +sources is out of scope for V1. + +V1 uses the default extraction guidance in `policy.md`. Install-provided +extraction guidelines are out of scope for V1. + +The plugin owns: + +- its Drizzle table objects +- generated SQL migrations under `migrations/*.sql` +- a small memory store module around `ctx.db` +- extraction and retrieval policy +- install-level memory policy evaluation +- the `extractMemories` and embedding repair task handlers +- memory tool definitions +- future memory admin command definitions + +Core owns: + +- plugin loading and hook ordering +- prompt rendering and size limits +- plugin session append state +- database migration application +- runtime identity, source, and destination context +- plugin task enqueueing, retry, redelivery, and worker execution +- model and embedding provider credential custody +- tool schema validation and tool execution boundaries +- plugin config loading +- log, trace, and dashboard redaction + +## Memory Types + +The plugin stores one `type` for lifecycle and rendering policy: + +| Type | Meaning | Default TTL | +| -------------- | -------------------------------------------------- | ----------- | +| `preference` | Stable user or conversation preference | none | +| `identity` | Stable fact about the requester | none | +| `relationship` | Stable fact about a named person or relationship | none | +| `knowledge` | Durable project, workspace, or domain fact | none | +| `context` | Current situation that should decay | 7 days | +| `event` | Dated occurrence that may matter later | 30 days | +| `task` | Remembered obligation that is not a scheduled task | 14 days | +| `observation` | Low-durability observation | 3 days | + +Explicit scheduled work belongs to the scheduler plugin, not memory. A memory +of type `task` is only a remembered fact unless the user explicitly creates a +scheduled task through the scheduler workflow. + +V1 passive extraction must not create `identity` or `relationship` memories +about third parties. Those types are primarily for explicit personal memory, +such as the requester's own preferences, identity facts, or working +relationships that pass policy. + +## Scope Model + +V1 supports two visibility scopes: + +| Scope | Stored authority | Visible to | +| -------------- | ------------------------------------------------ | -------------------------------------------- | +| `personal` | current requester actor | same requester in compatible runtime context | +| `conversation` | current source/destination conversation identity | later turns in the same conversation | + +Rules: + +1. Scope is derived from runtime context. Model-visible tool arguments never + provide requester ids, team ids, channel ids, thread ids, or conversation ids. +2. Personal memory is the default for first-person facts in interactive turns. +3. Conversation memory may be created only when the user explicitly frames the + fact as shared team/channel/conversation knowledge or the passive extractor + can prove the fact is about the current conversation rather than a person. +4. V1 does not recall memories across unrelated conversations, even if display + names or Slack users appear to match. +5. Subject labels may be stored for later display and future person-graph work, + but they are not authorization principals in V1. + +## Sensitivity + +Every memory has a sensitivity: + +| Sensitivity | Meaning | V1 disclosure | +| ----------- | ------------------------------------------------ | ---------------------------------------------------------------------------- | +| `public` | Normal preference or operational fact | visible within stored scope | +| `personal` | Private detail that should not be shared broadly | personal scope only unless explicitly conversation-scoped by the source user | +| `sensitive` | health, financial, legal, employment, or similar | personal scope only | + +Sensitive memories must not be created as conversation-scoped passive memories. +If a user explicitly asks to store sensitive information as shared conversation +knowledge, the tool must reject the request with a model-visible input error +explaining that sensitive memories can only be stored personally. + +Secrets are not a sensitivity class. Secrets are rejected and never stored. + +## Store Boundary + +Hook bodies must not issue ad hoc SQL directly. The plugin should keep storage +behind a small store such as `MemoryStore`. + +The store boundary owns: + +- parsing database rows into memory records +- rejecting invalid enum values and malformed metadata +- visibility filtering +- create/archive/list operations +- duplicate detection +- extraction idempotency +- embedding row repair +- expiration and supersession updates + +Drizzle table objects may be imported inside the plugin package. They must not +be exported as part of Junior core. + +## Implementation Order + +Implement in this order: + +1. Core plugin hook surfaces needed by this spec: `userPrompt`, `observeTurn`, + plugin background tasks, `tools`, `ctx.db`, active-projection plugin session + state, host embedding provider access, and plugin config/policy access. +2. Memory plugin package with manifest, schema, migrations, store, and + install-level policy evaluator. +3. Explicit `createMemory`, `listMemories`, `searchMemories`, and + `removeMemory` tools with context-bound scope and secret rejection. +4. Automatic recall from stored memories through `userPrompt` when + `autoInjectMemories` is enabled, using lexical ranking before embeddings are + available. +5. Embedding provider integration, vector storage, and embedding repair tasks. +6. `observeTurn` task enqueueing and `extractMemories` task execution. +7. Deduplication, TTL archival, and conservative supersession. +8. Optional vector index tuning and hybrid ranking improvements. +9. Future admin CLI inspection and repair commands after redaction and access + rules are implemented. +10. Dashboard/admin UI only after a separate UI access-control contract exists. + +The first vertical slice should prove explicit memory create/list/remove/search +and optional automatic memory injection before adding automatic extraction. + +## Related Specs + +- `../plugin.md` +- `../plugin-runtime.md` +- `../plugin-prompt-hooks.md` +- `../plugin-database.md` +- `../plugin-cli.md` +- `./policy.md` +- `../task-execution.md` +- `../identity.md` +- `../credential-injection.md` +- `../data-redaction-policy.md` +- `../agent-prompt.md` +- `../testing.md` diff --git a/specs/memory-plugin/policy.md b/specs/memory-plugin/policy.md new file mode 100644 index 000000000..a57014adc --- /dev/null +++ b/specs/memory-plugin/policy.md @@ -0,0 +1,306 @@ +# Memory Plugin Policy + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define install-level memory policy controls, with specific attention to +workplace deployments where passive memory can create privacy, trust, and +compliance risks. + +## Scope + +- What must be tunable by the installing app or workspace. +- V1 passive extraction toggle and default extraction guidance. +- Default workplace extraction guidance. +- Workplace-sensitive information categories. +- How policy affects tools, passive extraction, retrieval, retention, models, + and admin output. + +## Non-Goals + +- Supporting install-provided extraction guidelines in V1. +- Defining legal retention, eDiscovery, or data subject export workflows. +- Creating per-jurisdiction legal compliance advice. +- Replacing the global data redaction policy. + +## Policy Model + +The memory plugin must evaluate an install-level policy before writing, +recalling, embedding, or displaying memory. + +Policy should be resolved from explicit plugin configuration and runtime +context. The model may not change policy through prompt text or tool arguments. + +V1 needs only a small required policy surface: + +- passive extraction toggle +- automatic memory injection toggle + +The V1 config shape is: + +```ts +interface MemoryPolicy { + passiveExtraction: boolean; + autoInjectMemories: boolean; +} +``` + +Plugin enablement is controlled by the normal plugin registration path. If an +install does not want memory at all, it should not enable the memory plugin. + +V1 uses the default workplace guidance in this spec. Configurable extraction +guidelines are a future extension. The deterministic validator enforces hard +rules: + +- no secrets +- runtime-derived scope only +- source visibility checks +- `public` conversation visibility for passive capture only in V1 +- policy toggle checks +- provider allowlist checks +- no raw transcript storage +- redaction and logging restrictions + +Memory policy must be loaded before hooks run and must be available to +extraction, tools, retrieval, storage, and admin code. + +## Conservative Defaults + +Workplace-safe defaults should be conservative: + +1. `passiveExtraction` defaults to `false`. +2. If passive extraction is enabled in V1, it learns only allowed workplace + knowledge from conversations classified as `public`. +3. `autoInjectMemories` defaults to `true` when the memory plugin is enabled. +4. Installs that do not want automatic memory injection can set + `autoInjectMemories` to `false`, requiring the model to use + `searchMemories` for recall. +5. Passive extraction from conversations classified as `direct`, `private`, + `unknown`, or unsupported is out of scope for V1. +6. Sensitive memory should be personal-only and should be disabled for passive + extraction by default. +7. Third-party personal facts about coworkers should not be passively stored by + default. +8. Retention should prefer shorter TTLs for `context`, `event`, `task`, and + `observation` memories. +9. Default admin output should be redacted. + +An install can choose whether to enable passive extraction and whether to enable +automatic memory injection, but V1 does not expose broader extraction behaviors. + +## Default Workplace Guidelines + +When `passiveExtraction` is `true`, the extractor should look for clean +workplace knowledge from conversations classified as `public`. + +Aim to extract: + +- durable project, product, repository, or operational facts +- team workflow preferences and conventions +- ownership and responsibility facts, such as who owns a project or migration +- explicit decisions, status changes, deadlines, launch windows, or deploy + windows +- channel-level norms, such as how a public channel tracks work or incidents +- explicit "remember this" requests that are appropriate for the current + channel scope + +Avoid extracting: + +- casual conversation, jokes, venting, or social commentary +- summaries of a discussion that are not useful without hidden context +- temporary troubleshooting details that will not matter later +- facts whose usefulness depends on remembering the whole transcript +- personal details about coworkers +- speculative claims about people +- sensitive workplace categories listed below + +The memory text should be the minimum useful assertion, not a transcript quote. +It should strip incidental names, Slack handles, timestamps, and context unless +they are needed for the memory to be correct. + +Future configurable extraction guidelines may narrow or redirect what the model +looks for, such as "only remember repository conventions and product +decisions." They are not part of V1, and when added they must not override hard +validators or allow passive extraction from non-public or otherwise disallowed +sources. + +## Third-Party Facts + +Third-party facts are allowed in V1 when they are operational knowledge from a +conversation classified as `public`, rather than personal claims. + +Useful third-party memories include: + +- `Priya owns the billing migration.` +- `Alex said the deploy freeze starts Friday, 2026-06-19.` +- `The infra team uses Linear for incident follow-up.` +- `The #frontend channel prefers PRs under 400 lines.` + +Unsafe third-party memories include: + +- `Bob is unreliable.` +- `Sam is interviewing elsewhere.` +- `Alice is dealing with a medical issue.` +- `Dana dislikes working with Chris.` + +When a memory is materially a person's claim rather than a direct public +conversation fact, preserve provenance in the content. Prefer +`Alex said the deploy freeze starts Friday` over laundering the claim into +`The deploy freeze starts Friday` unless the conversation context makes it an +accepted team fact. + +## Workplace-Sensitive Categories + +The extractor must be careful about information that can harm people if stored +or recalled out of context. + +The default workplace policy should reject passive storage of: + +- health, disability, medical, or family-care details +- legal issues, immigration status, or government identifiers +- compensation, performance review, promotion, discipline, or termination + details +- protected class, religion, politics, union activity, or similar affiliation +- financial hardship, personal relationships, or private life details +- passwords, credentials, tokens, keys, recovery codes, or secrets +- speculative claims about a coworker's intent, ability, mood, reliability, or + character +- jokes, venting, gossip, conflict, or interpersonal commentary +- raw conversation summaries whose future usefulness depends on hidden context + +Explicit user requests to remember sensitive personal details must still follow +scope and sensitivity rules. Some installs may choose to reject those requests +entirely. + +## Passive Extraction Policy + +Passive extraction must use policy as an input before model prompting and again +after structured extraction output. + +The extraction prompt may describe allowed categories for quality. Policy +enforcement should happen in a separate policy adjudication step after +extraction proposes candidate facts. The deterministic validator remains the +final enforcement point for hard safety rules. + +`passiveExtraction` is a boolean: + +| Value | Meaning | +| ------- | ----------------------------------------------------------------------- | +| `false` | Do not enqueue passive extraction tasks. | +| `true` | Learn allowed workplace knowledge from public conversations only in V1. | + +Explicit-only memory creation is not a passive extraction setting. It is the +normal tool path: when `passiveExtraction` is `false`, the only way to write +memory is through explicit tools such as `createMemory`. + +When `passiveExtraction` is `true`, policy allows passive extraction of: + +- explicitly requested durable user preferences about Junior's behavior +- durable project or repository facts +- operational workflow facts +- explicit dates or deployment windows +- explicit "remember this" requests + +Policy still disallows passive extraction by category, including: + +- personal facts about third parties +- identity or relationship facts about third parties +- non-operational conversation summaries +- sensitive facts +- low-confidence inferences +- facts without explicit durability + +For V1, passive extraction should store conversation-scoped operational +knowledge by default. Passive personal memory from public conversations requires +explicit remember language from the source user and must still be visible only +to that requester. + +## Automatic Injection Policy + +`autoInjectMemories` controls automatic memory reads. It is independent from +passive extraction: + +| Value | Meaning | +| ------- | -------------------------------------------------------------------------- | +| `true` | `userPrompt` injects relevant visible memories into model-visible prompts. | +| `false` | `userPrompt` does not inject memories; recall requires `searchMemories`. | + +When `autoInjectMemories` is `false`, the plugin may still expose memory tools +and may still perform passive extraction if `passiveExtraction` is `true`. The +model-visible recall path is explicit: the model must call `searchMemories`, +which applies the same visibility, policy, ranking, and redaction rules as +automatic memory injection. + +## Explicit Tools And Policy + +Explicit `createMemory` requests are still subject to install policy. + +For example, passive extraction is limited to public-conversation workplace +knowledge in V1, but users may still explicitly store personal preferences when +the requested memory passes policy. An install may disallow all sensitive memory +writes, including explicit requests. + +The explicit tool path must run the same deterministic policy filter as passive +extraction. Explicit user intent can make a fact eligible for storage under +install policy, but it cannot override secret rejection, source/scope rules, +workplace-sensitive category rejection, provider policy, or sensitivity +restrictions. + +Tool errors should explain policy rejection at a high level without revealing +hidden policy internals or sensitive content. + +Explicit `createMemory` may use the same policy adjudicator as passive +extraction when the policy decision is not deterministic. If adjudication fails +for an explicit tool request, the tool should return a retryable input error +rather than storing the memory. + +## Retrieval And Policy + +Retrieval must apply current policy as well as stored scope and lifecycle. + +If policy changes after a memory was created, the stricter current policy wins +for automatic memory injection and list/search results. The memory may remain +stored but hidden until an admin repair workflow decides whether to archive it. + +Policy changes must not make hidden memories visible merely because the model +asks for them. + +## Model And Provider Policy + +Some installs may restrict which providers can receive memory-related text. + +Host provider configuration must support disabling: + +- passive extraction model calls +- embedding model calls +- sending memory text to non-approved providers +- sending private conversation text to extraction models + +When embeddings are disabled by policy, lexical recall remains the fallback. + +## Admin Policy + +Admin commands must respect policy defaults: + +- redacted output by default +- explicit flags for content display +- no secret disclosure +- scope selectors required for user-visible records +- repair commands should report counts and ids before content + +Policy should also let installs disable full-content admin output if they need a +stricter workplace posture. + +## Related Specs + +- `./index.md` +- `./security.md` +- `./extraction.md` +- `./tools.md` +- `./retrieval.md` +- `./admin.md` +- `../data-redaction-policy.md` diff --git a/specs/memory-plugin/retrieval.md b/specs/memory-plugin/retrieval.md new file mode 100644 index 000000000..112484ae2 --- /dev/null +++ b/specs/memory-plugin/retrieval.md @@ -0,0 +1,160 @@ +# Memory Plugin Retrieval + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define how the memory plugin recalls active visible memories, optionally +injects them into model-visible user prompts, and exposes explicit recall +through `searchMemories`. + +## Automatic Injection Policy + +Install policy controls whether recall is automatic or tool-mediated: + +- `autoInjectMemories: true` enables `userPrompt` memory injection. +- `autoInjectMemories: false` disables memory injection; the model-visible recall + path is `searchMemories`. + +This setting does not control writes. Passive extraction and explicit creation +are governed by the extraction and tool policies in [`./policy.md`](./policy.md). + +## Automatic Recall + +The memory plugin recalls memories through `userPrompt(ctx)`. + +Core invokes the hook for every model-visible user prompt. When automatic memory +injection is disabled by policy, the plugin must return no memory contribution +and must not append injected memory session state. + +When automatic memory injection is enabled, the plugin must: + +1. Derive visible memory scopes from `ctx.requester`, `ctx.source`, + `ctx.destination`, and `ctx.conversationId`. +2. Read plugin session append state for already injected memory ids. +3. Query active visible memories relevant to `ctx.userText`. +4. Exclude memories already injected into the active session projection. +5. Include newly relevant memories even when earlier prompts already included + different memories. +6. Return one concise prompt contribution containing only accepted memory + content. +7. Append injected memory ids to plugin session state only when a contribution + is returned. + +The plugin session append state key should be: + +```txt +injected_memories +``` + +The value should be bounded JSON: + +```ts +interface InjectedMemoriesState { + memoryIds: string[]; +} +``` + +The prompt hook session state helper returns state from the current +model-visible session projection. If compaction removes a prior memory block +from the active projection, the plugin may inject that memory again. Hidden +bookkeeping must not make memory recall disappear. + +## Tool-Mediated Recall + +When automatic memory injection is disabled, `searchMemories` is the only +model-visible recall path. It must use the same visibility filter, policy +checks, ranking pipeline, and result budgets as automatic memory injection. + +`searchMemories` may return ids or short ids when useful for follow-up memory +management, but it should otherwise return concise memory content and avoid +private metadata. The tool must derive all authority-bearing scopes from +runtime context, not from model-supplied arguments. + +`searchMemories` should not suppress results merely because they were already +injected into the current session. That suppression is specific to automatic +injection, where repeated prompt blocks would waste context. + +### Visibility Filter + +Retrieval must filter by visibility before prompt rendering: + +- matching personal requester scope +- matching conversation scope +- current install policy allows recall for the memory type, scope, and + sensitivity +- `archived_at is null` +- `superseded_at is null` +- `expires_at is null or expires_at > now()` +- sensitivity allowed in the current scope + +The query planner, vector index, model, and ranker are not authorization +boundaries. + +If install policy changes after a memory was created, retrieval must apply the +current policy. Stricter current policy hides the memory from automatic memory +injection and normal list/search results even if the stored row is otherwise +visible. + +### Ranking Pipeline + +V1 uses hybrid retrieval without Ash's person graph: + +1. Build visible active candidate scopes. +2. Run lexical search against memory content and subject labels. +3. Run vector search when embeddings are configured and the user text can be + embedded. +4. Merge lexical and vector results with reciprocal-rank style fusion. +5. Apply small deterministic boosts for exact scope match, durable memory + types, high confidence, and recent observations. +6. For automatic injection only, drop memories already injected into the active + session projection. +7. Return the top memories within count and character budgets. + +Vector results should be overfetched before final filtering and prompt +formatting. Approximate vector search must be exact-reranked over visible +candidates before injection. + +### Exact Versus Indexed Vector Search + +The store should choose the simplest safe query for the visible candidate set: + +- If visible candidate count is small, rank exact cosine distance inside SQL. +- If visible candidate count is large and an HNSW index exists, use approximate + vector search with an overfetch multiplier, then re-rank exact visible + candidates. +- If embedding generation fails, skip vector search and continue with lexical, + recency, and type ranking. + +This keeps correctness independent of pgvector index tuning. + +### Prompt Rendering + +The memory prompt contribution should be short, stable, and clearly separated +from the user's request. + +Core owns the wrapper. The plugin owns the contribution text. The contribution +must: + +- include only active visible memories +- stay within configured count and character limits +- avoid raw provenance unless needed for disambiguation +- avoid ids +- not include secrets or archived facts +- not include facts whose scope is no longer visible + +Memory content is context, not instruction. The rendered contribution should +make clear that memories may be stale and should not override direct user +corrections or current repository evidence. + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./security.md` +- `../plugin-prompt-hooks.md` +- `../agent-prompt.md` diff --git a/specs/memory-plugin/security.md b/specs/memory-plugin/security.md new file mode 100644 index 000000000..2553f81a6 --- /dev/null +++ b/specs/memory-plugin/security.md @@ -0,0 +1,159 @@ +# Memory Plugin Security + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define the memory plugin's security boundaries for storage, retrieval, tools, +model calls, embeddings, logging, and multi-user visibility. + +## Security Invariants + +1. Runtime context, not model text, determines memory visibility. +2. Install-level policy determines which categories, scopes, and model providers + are allowed. +3. Secrets are rejected, not stored as sensitive memories. +4. Memory content may be model-visible only inside the stored scope and current + policy. +5. Retrieval ranking is not an authorization boundary. +6. Embeddings and lexical indexes are derived data and cannot grant visibility. +7. Provider credentials never enter plugin storage, prompt contributions, tool + schemas, task payloads, logs, or model-visible content. +8. Observation/task payloads use stable references and bounded safe metadata, + not raw private transcript text. +9. Every write path uses the same policy, validation, and secret rejection + layer. + +## Authority Boundaries + +The store must derive authority-bearing fields from Junior runtime context: + +- requester identity +- source platform +- tenant/workspace/org boundary when available +- destination or conversation identity +- source actor +- source event or observation id + +The model may request memory operations, but it cannot choose authority fields. +Tool arguments can express content, requested scope class, query text, limit, or +expiration. They cannot express actor ids, workspace ids, channel ids, thread +ids, arbitrary owner ids, arbitrary conversation ids, or arbitrary scope +overrides for `searchMemories`. + +Display names, subject labels, aliases, and model-extracted subject text are +metadata. They are useful for rendering and future graph work, but they are not +authorization principals. + +Admin CLI selectors also are not authorization by themselves. They identify the +records an operator wants to inspect or repair; deployment/operator +authorization is a separate host boundary. + +## Multi-User Visibility + +Personal memory is visible only to the same requester in a compatible runtime +context. + +Conversation memory is visible only in the same conversation identity. V1 does +not recall conversation memory across related channels, Slack workspaces, +threads, projects, or rooms. + +V1 passive extraction is limited to conversations classified as `public` by +Junior's existing conversation privacy/destination visibility contracts, and it +stores conversation-scoped workplace knowledge by default. Direct, private, +unknown, local CLI, and unsupported sources may still use explicit memory tools +when policy allows them, but they must not feed passive extraction. Visibility +classification must fail closed. + +Sensitive memory is personal-only. Passive extraction must never create a +conversation-scoped sensitive memory. An explicit tool request to store +sensitive shared memory must fail with a model-visible input error. + +For workplace installs, passive third-party personal facts should be rejected. +Third-party operational facts from public conversations may be stored only when +they are clean workplace knowledge under [`./policy.md`](./policy.md). + +## Model Boundaries + +Extraction, retrieval, and tool-calling models are helpers, not security +boundaries. + +The plugin must validate structured extraction output after model generation and +before storage. It must reject malformed, low-confidence, out-of-scope, +secret-like, or incoherent candidates even if the model marks them as valid. +If a second policy-adjudication model is used, its output is also guidance, not +the final security boundary. + +The embedding provider receives only memory text or retrieval query text needed +for the operation. It must not receive raw provider credentials, raw Slack +payloads, raw OAuth data, or unrestricted transcripts through the plugin API. +Install policy may disable extraction or embedding providers for private +conversation text. + +Embedding vectors inherit the same sensitivity, scope, lifecycle, policy, and +provider restrictions as their source memories. They must not be logged, +reported, exported, retained, or exposed under weaker rules than memory content. + +## Task Payloads + +Plugin background task payloads must contain stable references and bounded safe +metadata only. + +They must not contain: + +- raw private user text +- raw assistant text +- raw tool payloads +- provider credentials +- authorization URLs +- OAuth tokens +- Slack tokens +- memory content unless the task exists specifically to repair a memory id that + can be reloaded from storage + +Observation-backed tasks should reload bounded observation payloads through the +core-provided observation helper. + +## Logging And Reporting + +Logs, spans, dashboards, and plugin operational reports may include: + +- plugin name +- hook or task name +- memory operation name +- memory id or bounded id prefix +- scope type +- memory type and sensitivity enum +- embedding provider/model/dimensions +- extraction candidate counts +- rejection reason codes +- duration +- outcome + +They must not include: + +- raw memory content +- raw private conversation text +- extraction prompt text +- raw model extraction output +- SQL parameter values containing user data +- provider credentials +- authorization URLs +- Slack tokens +- raw private tool arguments or results + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./retrieval.md` +- `./extraction.md` +- `./tools.md` +- `./admin.md` +- `../identity.md` +- `../credential-injection.md` +- `../data-redaction-policy.md` diff --git a/specs/memory-plugin/storage.md b/specs/memory-plugin/storage.md new file mode 100644 index 000000000..4d00d91e0 --- /dev/null +++ b/specs/memory-plugin/storage.md @@ -0,0 +1,264 @@ +# Memory Plugin Storage + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define the memory plugin's broad SQL storage design, embedding storage +mechanism, model-provider boundary, and operational rules without prescribing +exact migrations or DDL. + +## Contracts + +### Storage Ownership + +The memory plugin owns its SQL schema, Drizzle table definitions, and packaged +migrations under [`../plugin-database.md`](../plugin-database.md). + +This spec defines the shape and invariants those migrations must satisfy. It +does not define exact migration filenames, full column lists, or generated SQL. + +### Data Classes + +The plugin stores two classes of data: + +1. **Memory records**: the durable source of truth for facts that may be + recalled later. +2. **Retrieval indexes**: derived data, such as embeddings and lexical indexes, + that can be deleted and rebuilt from memory records. + +The implementation may use one table per class or split them further if needed, +but V1 should keep the shape simple: + +- one authoritative memory-record table +- one derived embedding/vector table or equivalent vector index +- optional database-native lexical search support + +### Memory Record Shape + +Each memory record must contain enough information to enforce visibility and +lifecycle without consulting the original transcript. + +Required conceptual fields: + +- stable memory id +- self-contained memory content +- normalized content hash for duplicate detection +- memory type +- sensitivity +- runtime-derived visibility scope +- runtime-derived source attribution +- observation or tool idempotency marker when available +- optional subject/display labels that are not authorization principals +- extraction confidence when learned passively +- observed timestamp +- created timestamp +- optional expiration timestamp +- optional supersession link +- archive timestamp and reason +- bounded operational metadata + +Scope and source fields are authority-bearing. Display labels, subject labels, +model-generated summaries, and tool arguments are not. + +### Visibility Data + +The storage model must support these V1 visibility scopes: + +- personal memory owned by the current requester identity +- conversation memory owned by the current source/destination conversation + +The stored scope must be derived from runtime context before write. Model-visible +tool input cannot provide requester ids, actor ids, workspace ids, channel ids, +thread ids, or arbitrary conversation ids. + +The store must be able to filter active visible records by: + +- scope +- sensitivity +- current install policy +- archive state +- supersession state +- expiration + +### Idempotency And Duplicates + +Passive extraction must be idempotent across repeated observations, queue +redelivery, and task retry. The store needs a stable source marker for a +completed observation and the extracted fact's position or stable fact id inside +that observation. + +Duplicate suppression also needs active-scope content hashing and a later +semantic-similarity check when embeddings are available. + +### Lexical Search + +Lexical search is required because embeddings are optional operationally and can +fail independently of memory writes. + +The storage layer should use Postgres-native text search or an equivalent SQL +indexable mechanism. Retrieval must still apply the memory visibility predicate +before returning rows to prompt rendering or tools. + +### Embedding Storage + +Embeddings are derived retrieval data. They are not the authority for memory +existence, visibility, or deletion. + +Embedding rows or index entries must record: + +- memory id +- provider id +- model id +- dimensions +- distance metric +- content hash that was embedded +- vector value +- created/repaired timestamps + +The plugin should not store raw embedding-provider request or response payloads. + +Changing provider, model, dimensions, or metric requires re-embedding active +memories. Missing or stale embeddings degrade retrieval to lexical and recency +ranking. + +Vectors inherit the classification, scope, retention, deletion, and provider +policy of their source memory. Archiving or deleting a memory must remove or +invalidate derived vectors under the same rules as the memory content. + +### Vector Storage + +V1 should use Postgres-native vector storage through pgvector when embeddings +are enabled. The default should be a fixed-dimensional vector column compatible +with the configured embedding model. + +Use cosine distance by default. `text-embedding-3-small` at 1536 dimensions is +the expected default because it fits a common pgvector setup and matches the Ash +prototype's default. Larger native embedding models must either be configured to +return the stored dimension or wait for a migration/rebuild plan that changes +the stored vector dimension. + +### Vector Index Strategy + +The retrieval design should not assume approximate vector indexes are necessary +on day one. + +V1 should start with exact vector ranking over visible active candidates. If +production data shows that exact ranking is too slow, add an approximate +pgvector index such as HNSW and overfetch results before applying exact +visibility filtering and final reranking. + +Approximate vector search is a performance tool, not an authorization boundary. + +### Embedding Provider + +Core must keep provider credentials and expose only a narrow host capability to +trusted plugin hooks and tasks: + +```ts +interface PluginEmbeddingProvider { + embed(input: { + texts: string[]; + purpose: "memory"; + model?: string; + dimensions?: number; + }): Promise<{ + provider: string; + model: string; + dimensions: number; + vectors: number[][]; + }>; +} +``` + +Rules: + +1. The provider is host runtime code, not a model-visible tool. +2. The memory plugin never receives provider API keys. +3. The returned vector count must equal the input text count. +4. Empty or whitespace-only texts are rejected before provider calls. +5. The returned dimensions must match the configured vector storage. +6. Provider failures do not roll back accepted memory content. +7. Missing embeddings degrade recall to lexical and recency ranking. + +The default embedding configuration should be: + +```txt +provider = openai-compatible +model = text-embedding-3-small +dimensions = 1536 +metric = cosine +``` + +The exact provider name is deployment configuration. Stored embedding metadata +records the resolved provider and model used for each vector. + +### Write Path + +Memory creation follows this order: + +1. Validate content, scope, sensitivity, source, expiration, and metadata. +2. Run model-assisted policy adjudication when needed. +3. Run deterministic policy validation and centralized secret rejection. +4. Insert the memory record transactionally. +5. After the transaction commits, batch-generate embeddings for inserted records + when an embedding provider is configured. +6. Store or repair vector data only when provider output matches the configured + vector storage. + +Provider calls must not run inside the SQL transaction. + +If embedding generation fails, the memory remains active and can be found +through lexical/list retrieval. A later embedding repair task may repair missing +or stale embeddings. + +If install policy disables embeddings or a provider for a scope, the write path +must skip vector generation without failing the memory write. + +### Repair And Rebuild + +Embedding repair should run through plugin background work rather than request +handlers. + +The repair task finds active memories where: + +- no vector exists +- the vector was generated from an old content hash +- provider/model/dimensions differ from current config + +It processes bounded batches and is idempotent. + +### Removal And Lifecycle + +Memory removal archives in place: + +- set archive timestamp and reason +- exclude from recall and normal list results +- delete or ignore derived embedding/vector data + +Physical deletion is reserved for future retention, export, and account +deletion workflows. + +Memory maintenance must archive: + +- memories whose expiration is in the past +- ephemeral memories older than their type default TTL +- superseded memories after the supersession marker is committed + +V1 may perform this maintenance opportunistically during create, list, recall, +remove, extraction, and embedding repair paths. A future low-frequency +maintenance task may be specified separately if opportunistic cleanup is +insufficient. + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./security.md` +- `./retrieval.md` +- `./extraction.md` +- `../plugin-database.md` +- `../credential-injection.md` diff --git a/specs/memory-plugin/tools.md b/specs/memory-plugin/tools.md new file mode 100644 index 000000000..00a256cf1 --- /dev/null +++ b/specs/memory-plugin/tools.md @@ -0,0 +1,133 @@ +# Memory Plugin Tools + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define the explicit model-visible memory management and recall tools exposed by +the memory plugin. + +Operator/admin memory commands are covered by [`./admin.md`](./admin.md). They +must not be exposed through this model-visible tool surface. + +## Tool Surface + +The plugin exposes memory tools from `tools(ctx)`: + +```txt +createMemory +removeMemory +listMemories +searchMemories +``` + +Tool schemas must be context-bound. All tools derive requester, source, +destination, conversation, and tenant/workspace authority from runtime context. + +### createMemory + +`createMemory` may accept: + +- content +- optional scope enum: `personal` or `conversation` +- optional expiration duration/date +- optional sensitivity hint + +`createMemory` must not accept: + +- requester id +- actor id +- Slack team id +- Slack channel id +- Slack thread timestamp +- arbitrary conversation id +- arbitrary owner id +- raw source metadata + +The tool derives source, requester, destination, and scope from runtime context. +It runs the same validation, secret rejection, duplicate checks, and embedding +write path as passive extraction. Explicit tool requests are still subject to +install-level memory policy. + +`createMemory` must run the same deterministic policy filter as passive +extraction before writing. The fact that a user explicitly asked Junior to +remember something can satisfy the "explicit request" category, but it must not +bypass: + +- secret rejection +- source and scope rules +- workplace-sensitive category rejection +- sensitivity restrictions +- provider and embedding policy +- retention and lifecycle policy + +If policy rejects an explicit memory request, the tool should return a +model-visible input error that explains the rejection at a high level without +echoing sensitive content. + +### removeMemory + +`removeMemory` accepts a memory id or short id prefix and archives only a memory +visible in the current context. + +Ambiguous short prefixes must fail with a model-visible input error rather than +removing multiple rows. + +### listMemories + +`listMemories` lists only active memories visible in the current context. It +may accept an optional limit, but it must not search across unrelated users or +conversations. Current install policy must be applied before returning results. + +The tool may include ids or short ids because explicit removal workflows need a +handle. Normal automatic memory injection should avoid ids. + +### searchMemories + +`searchMemories` is the model-visible recall path when automatic memory +injection is disabled, and it can supplement automatic recall when the model +needs a targeted lookup. + +`searchMemories` may accept: + +- query text +- optional limit + +`searchMemories` must not accept: + +- requester id +- actor id +- Slack team id +- Slack channel id +- Slack thread timestamp +- arbitrary conversation id +- arbitrary owner id +- arbitrary scope override + +The tool derives visible scopes from runtime context, applies current install +policy, and runs the same retrieval pipeline as automatic memory injection. +Results must be active, visible, policy-allowed memories only. + +Unlike `listMemories`, `searchMemories` is relevance-ranked and does not need +to return every visible memory. It may omit ids unless the model needs a handle +for a follow-up `removeMemory` request. + +## Output Rules + +Tool output must be concise and must not reveal hidden private metadata. For +private conversations, tool output may contain memory content because it is +model-visible response context, but logs/traces/reporting for that tool must +redact content according to [`../data-redaction-policy.md`](../data-redaction-policy.md). + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./retrieval.md` +- `./admin.md` +- `../plugin-prompt-hooks.md` +- `../data-redaction-policy.md` diff --git a/specs/memory-plugin/verification.md b/specs/memory-plugin/verification.md new file mode 100644 index 000000000..d909fc1b3 --- /dev/null +++ b/specs/memory-plugin/verification.md @@ -0,0 +1,169 @@ +# Memory Plugin Verification + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define the memory plugin's failure model, observability rules, and verification +requirements. + +## Failure Model + +1. Missing required SQL database: startup and `junior upgrade` fail according + to [`../plugin-database.md`](../plugin-database.md). +2. Unapplied memory migrations: plugin hooks do not run; startup fails for the + required plugin. +3. Missing embedding provider: memory write and lexical recall still work; + vector recall and embedding repair are disabled. +4. Embedding provider failure: store the memory row, log safe metadata, and + leave the row eligible for repair. +5. Embedding dimension mismatch: reject the embedding row, log safe metadata, + and continue without vector recall for that memory. +6. `userPrompt` retrieval failure: omit memory contribution, log safe metadata, + and continue unless the failure indicates a broken required migration. +7. Prompt contribution validation failure: omit the contribution and do not + append injected memory session state. +8. `observeTurn` enqueue failure: log safe metadata and do not fail the + completed turn. +9. Task delivery failure: core retries according to the task runner policy. +10. Task retry bound exceeded or observation payload expired: mark or drop the + task with safe metadata; do not fail the completed user turn. +11. Duplicate post-turn observation or duplicate task delivery: task + idempotency, source idempotency, and duplicate detection suppress duplicate + stored memories. +12. Secret detection match: reject the write with a model-visible tool input + error for explicit tools or drop the passive fact with safe logging. +13. Visibility mismatch: fail closed and omit the memory. +14. Malformed stored rows: ignore the row for recall/list, log safe metadata, + and leave repair to a future administrative workflow. + +## Observability + +Logs and spans may include: + +- plugin name +- hook name +- memory operation name +- memory id or bounded id prefix +- scope type +- type and sensitivity enum +- embedding provider/model/dimensions +- extracted candidate fact count +- accepted/rejected fact counts +- rejection reason code +- duration +- outcome + +Logs and spans must not include: + +- raw memory content +- raw private conversation text +- extraction prompt text +- model extraction output +- SQL parameter values containing user data +- provider credentials +- authorization URLs +- Slack tokens +- raw tool arguments or results for private conversations + +Use `app.*` attributes for memory-specific telemetry when no OpenTelemetry +semantic key exists. + +## Verification + +Use integration tests for: + +- memory plugin packaged storage migrations are discovered and applied through + `junior upgrade` +- storage migrations provide the broad memory-record and derived-vector storage + mechanisms required by `storage.md` +- explicit memory creation stores a personal memory under the current requester +- explicit conversation memory stores under the current conversation without + accepting model-supplied Slack ids +- explicit memory creation is rejected when it violates install policy or + workplace-sensitive category rules +- install policy can disable passive extraction without disabling explicit + memory tools +- install policy can disable automatic memory injection without disabling + explicit memory tools +- install policy can reject workplace-sensitive passive facts +- stricter current policy hides previously stored memories from automatic memory + injection and list/search results +- `listMemories` returns only memories visible in the current context +- `searchMemories` returns only relevant memories visible in the current + context +- `searchMemories` cannot search across unrelated users or conversations +- `removeMemory` archives only visible memories +- `userPrompt` injects visible memories into every user prompt when + `autoInjectMemories` is `true` +- `userPrompt` returns no memory contribution and appends no injected-memory + state when `autoInjectMemories` is `false` +- injected memory ids are excluded only while their contribution remains in the + active session projection +- memory recall survives a follow-up prompt without requiring a search tool when + automatic memory injection is enabled +- memory recall works through `searchMemories` when automatic memory injection + is disabled +- lexical recall works when embeddings are unavailable +- vector recall works after embedding rows are created +- embedding failures leave memories listable and lexically recallable +- private conversation memory content is absent from logs, traces, and + dashboard reporting payloads +- passive `observeTurn` enqueues an extraction task without failing delivery +- extraction task payloads contain references rather than raw private text +- extraction task handlers can run in a separate worker invocation +- policy adjudication rejects extracted candidates that violate installed + workplace policy +- malformed or failed policy adjudication fails closed for passive extraction +- duplicate observation or task delivery of the same turn stores accepted + memories once + +When the future admin CLI is implemented, use integration tests for: + +- admin CLI commands default to redacted output +- full content display requires explicit operator flags +- admin repair reports counts and ids without making policy-hidden memories + visible + +Use unit tests for: + +- memory type, scope, and sensitivity parsers +- install policy parser and policy evaluation predicates +- secret detection +- storable-fact validation +- explicit-tool policy filtering +- policy adjudication output parsing +- TTL calculation +- visibility predicates +- duplicate detection +- prompt contribution formatting bounds +- tool schema rejection of actor, destination, team, channel, and conversation + fields +- embedding provider response validation +- lexical/vector result fusion + +Use evals for: + +- explicit "remember this" behavior +- later recall of stored preferences or facts +- use of `searchMemories` when automatic memory injection is disabled +- refusal to remember secrets +- explicit create rejection for policy-disallowed workplace-sensitive facts +- refusal or policy rejection for workplace-sensitive facts +- passive extraction quality once the extraction task is implemented +- model use of current user corrections over stale memories + +## Related Specs + +- `./index.md` +- `./policy.md` +- `./storage.md` +- `./security.md` +- `./retrieval.md` +- `./extraction.md` +- `./tools.md` +- `./admin.md` +- `../testing.md` diff --git a/specs/plugin-cli.md b/specs/plugin-cli.md new file mode 100644 index 000000000..d8576870f --- /dev/null +++ b/specs/plugin-cli.md @@ -0,0 +1,147 @@ +# Plugin CLI Spec + +## Metadata + +- Created: 2026-06-13 +- Last Edited: 2026-06-13 + +## Purpose + +Define the future shape for trusted plugins to contribute host CLI commands +without making those commands model-visible tools or sandbox commands. + +## Scope + +- Plugin-owned CLI command registration. +- Command discovery and conflict rules. +- Command context, IO, database access, task enqueueing, and admin boundaries. +- Security rules for local/operator execution. + +## Non-Goals + +- Implementing plugin CLI commands in V1. +- Letting `plugin.yaml` register executable CLI code. +- Exposing plugin CLI commands to the model. +- Running plugin CLI commands inside the agent sandbox. +- Replacing `junior chat`, `junior init`, `junior check`, `junior upgrade`, or + other core commands. + +## Contracts + +### Command Ownership + +Plugin CLI commands are trusted host code registered through app-code plugin +registration, not declarative `plugin.yaml`. + +The rough plugin shape is: + +```ts +defineJuniorPlugin({ + manifest, + cli: { + commands: [ + { + name: "memory", + summary: "Inspect and repair Junior memory state", + run, + }, + ], + }, +}); +``` + +The exact API may change before implementation. The required contract is that +commands are explicitly registered by enabled plugins and run with a narrow +host-provided context. + +### Command Namespace + +Plugin command names must be stable, lowercase, and unique across enabled +plugins and core commands. + +Core command names win. If a plugin command conflicts with a core command or +another enabled plugin command, startup or CLI bootstrap must fail before +dispatching the command. + +V1 should prefer one top-level command per plugin, such as: + +```txt +junior memory ... +``` + +Subcommands below that namespace are plugin-owned. + +### Command Context + +Plugin CLI command handlers may receive: + +- parsed argv for the plugin command +- stdout/stderr writers +- safe logger +- plugin metadata +- plugin config +- `ctx.db` when the plugin requires SQL and migrations are applied +- background task enqueue capability for repair/backfill work +- host embedding/model capabilities only when explicitly declared by the + plugin's command contract + +Handlers must not receive: + +- raw Slack clients or tokens +- raw HTTP request objects +- provider credentials +- model-visible tool contexts +- sandbox command handles +- cross-plugin mutable state + +### Admin Boundary + +Plugin CLI commands are operator/admin surfaces. They do not run as a Slack +requester or local chat requester unless the command explicitly accepts a +context selector and maps it through the same identity rules as runtime code. + +For production deployments, remote or hosted admin commands require a separate +admin authentication story before implementation. Local CLI access to a +configured database is not by itself a user-facing authorization model. + +### Output Rules + +Plugin CLI commands must be scriptable and redaction-aware: + +1. Default output should avoid raw private content when counts, ids, status, or + metadata are enough. +2. Commands that print private content must require an explicit flag or + subcommand. +3. Machine-readable output must not include secrets or provider credentials. +4. Errors must be concise and must not dump raw SQL parameters, provider + payloads, prompt text, or private transcripts. + +### Relationship To Model Tools + +Plugin CLI commands are not model-visible tools. The agent cannot call them +through the tool registry, and skills must not instruct the model to use CLI +commands for privileged memory administration during a normal turn. + +If an operation needs to be available to the model, expose it through the plugin +tool surface with context-bound schemas and model-safe output. If an operation +is administrative, expose it through CLI only. + +## Verification + +Required checks when implemented: + +- plugin CLI command discovery is explicit and deterministic +- command conflicts fail before dispatch +- disabled plugins do not expose commands +- plugin commands cannot access another plugin's state unless core provides an + explicit shared admin surface +- invalid arguments print usage and exit non-zero without side effects +- private content is omitted from default output + +## Related Specs + +- `./plugin.md` +- `./plugin-runtime.md` +- `./plugin-database.md` +- `./memory-plugin/admin.md` +- `./testing.md` diff --git a/specs/plugin-database.md b/specs/plugin-database.md new file mode 100644 index 000000000..6e8af5bbe --- /dev/null +++ b/specs/plugin-database.md @@ -0,0 +1,410 @@ +# Plugin Database Spec + +## Metadata + +- Created: 2026-06-12 +- Last Edited: 2026-06-13 + +## Purpose + +Define how explicitly enabled plugins extend Junior's shared SQL database with +packaged migrations and access that database from trusted runtime hooks without +requiring a memory-specific storage API or a globally merged plugin schema type. + +## Scope + +- Plugin package migration layout and discovery. +- Plugin-owned migration generation workflow. +- Migration ordering, checksums, and application through `junior upgrade`. +- Plugin-owned storage migration hooks for moving existing plugin state into + plugin SQL tables. +- The `ctx.db` surface exposed to trusted plugin hooks. +- Drizzle table ownership and typing boundaries for plugin code. +- Database behavior for plugins. + +## Non-Goals + +- Auto-discovering TypeScript schema files by convention. +- Generating plugin migrations from the host app. +- Applying migrations from request handlers or plugin hooks. +- Providing a database sandbox for untrusted plugin code. +- Exposing a globally typed Drizzle schema containing every installed plugin + table. +- Defining memory's concrete table schema. + +## Contracts + +### Package Shape + +Code plugin packages may include SQL migrations by convention: + +```txt +plugin-package/ +├── migrations/ +│ ├── 0001_init.sql +│ └── 0002_add_indexes.sql +└── src/ + └── db/ + └── schema.ts +``` + +`migrations/*.sql` is the runtime migration artifact. `src/db/schema.ts` is a +plugin-owned authoring and typing convention, not a file Junior auto-discovers +at runtime. + +Declarative `plugin.yaml` packages are a separate manifest-only shape. If they +are packaged next to `migrations/`, Junior treats those migration files as +inert. A database-backed code plugin package should expose JavaScript +registration and `migrations/` package content, not a same-plugin `plugin.yaml` +manifest that would also be loaded as a declarative plugin. Local `plugin.yaml` +roots do not contribute SQL migrations in V1. + +### Migration Discovery + +Junior applies migrations only for explicitly enabled code plugin registrations +that include a plugin `manifest.name` and an associated `packageName`. + +Package-name plugins and local `plugin.yaml` roots have an empty applied +migration list. This keeps the migration identity tied to the JavaScript +registration name that owns database access and storage migration hooks. + +Junior must never scan arbitrary `node_modules`, package dependencies, or +undeclared directories for migrations. + +Build packaging may copy or trace declared plugin-package `migrations/` +directories alongside plugin manifests and skills so `junior upgrade` can read +the same files in production output when a named code registration applies +them. Copying a migration directory does not make a declarative package apply +schema migrations by itself. + +### Migration Generation + +Plugin packages own their own schema authoring and migration generation. +Core owns migration application. Plugins publish committed SQL artifacts; Junior +does not let plugins run their own migration runner. + +A plugin that uses Drizzle should keep its table objects and Drizzle config in +the plugin package and generate SQL into that plugin's `migrations/` directory. +For example: + +```json +{ + "scripts": { + "db:generate": "drizzle-kit generate --config drizzle.config.ts" + } +} +``` + +Rules: + +1. Core does not generate plugin migrations. +2. Plugin migrations are generated from plugin-owned schema only. +3. Generated SQL files are committed and published as plugin package content. +4. Drizzle generation metadata may exist in the plugin package for future + plugin development, but Junior applies only `migrations/*.sql`. +5. A plugin package must not require the consuming app to run Drizzle Kit to use + the published plugin. + +### Schema Migration Application + +`junior upgrade` applies database migrations in this order: + +1. Core Junior migrations. +2. Plugin migrations, ordered by plugin name. +3. Migration files within each plugin, ordered lexically by filename. +4. Plugin storage migration hooks, ordered by plugin name. + +Plugin migration records use the shared `junior_schema_migrations` table. The +stored migration id is: + +```txt +plugin:/ +``` + +Core computes the checksum from the exact SQL file contents. If a migration id +already exists with a different checksum, upgrade must fail. + +Migration filenames must be stable, non-empty basenames matching +`NNNN_name.sql`, where `NNNN` is a zero-padded numeric prefix. This keeps +lexical filename ordering identical to migration order. Subdirectories are not +part of V1 migration discovery. + +### Storage Migration Hooks + +Schema migrations are not enough when an existing plugin has durable state in a +non-SQL store. A trusted runtime plugin may provide a storage migration hook: + +```ts +defineJuniorPlugin({ + manifest, + database: {}, + hooks: { + async migrateStorage(ctx) { + // Read old plugin-owned state through ctx.state. + // Write plugin-owned SQL records through ctx.db. + return { + scanned, + migrated, + existing, + missing, + }; + }, + }, +}); +``` + +The hook runs only as part of `junior upgrade`, not request handling. Core +invokes it only after core schema migrations and all discovered plugin SQL +migrations have completed successfully. This guarantees the plugin can write to +the tables created by its own `migrations/*.sql`. + +`junior upgrade` must resolve plugin registrations from the same configured +plugin set that runtime uses when that set is available. In deployed Nitro +output this means reading the virtual `#junior/config` plugin set; in tests or +programmatic callers this may be passed explicitly in the migration context. +Package-only declarative plugins do not contribute SQL schema migrations or +storage migration hooks. `@sentry/junior` core must not import plugin packages +to synthesize runtime registrations; database-backed plugins such as the +scheduler must be enabled through the same JavaScript registration module used +by runtime. + +The hook context is intentionally narrow: + +```ts +interface StorageMigrationContext extends PluginContext { + db: PluginDb; + state: PluginState; +} +``` + +Rules: + +1. `migrateStorage` hooks are JavaScript registration hooks. Declarative + `plugin.yaml` manifests cannot register upgrade behavior. +2. Core must not invoke a `migrateStorage` hook for a plugin registration that + was not explicitly enabled in the active plugin set. +3. `migrateStorage` hooks must be idempotent. Re-running `junior upgrade` must not + duplicate rows, corrupt state, or require deleting old state first. +4. `migrateStorage` hooks should read and write plugin-owned state and + plugin-owned SQL tables. Plugins are trusted host code; core does not + enforce this ownership boundary. +5. `migrateStorage` hooks must use `ctx.db` for SQL writes. A plugin with a + `migrateStorage` hook must declare database access and must fail upgrade + before the hook runs if no SQL database is configured. +6. `migrateStorage` hooks may read existing plugin state through `ctx.state`. This is + the only V1 bridge from pre-SQL plugin state into SQL. +7. `migrateStorage` hooks must return migration counters using the same result shape + as core migrations: `scanned`, `migrated`, `existing`, `missing`, and + optional `skipped`. +8. Core must run hooks sequentially in deterministic plugin-name order. V1 does + not provide dependency ordering between plugin storage migrations. +9. A thrown upgrade hook error fails `junior upgrade`. The new deployment should + not serve traffic until the failing plugin is fixed or disabled. +10. Storage migration hooks are not heartbeat hooks, background tasks, or admin commands. + They must not enqueue model work, dispatch agents, call provider APIs, or + depend on request-time context. +11. Storage migration hook logs must not include raw private conversation text, raw memory + content, credentials, SQL parameters, or existing state payloads. + +The scheduler plugin is the first expected consumer: it moves old +`junior:scheduler:*` plugin-state records into scheduler-owned SQL tables while +keeping the scheduler store interface stable. + +### Migration Safety + +Plugin migrations are privileged host code. The primary trust boundary is +explicit plugin installation and code review, not SQL sandboxing. + +V1 plugin migrations must be expand-only: + +- create plugin-owned tables +- add nullable columns to plugin-owned tables +- add indexes to plugin-owned tables +- add compatible constraints after existing data is clean + +Trusted plugin migrations should not: + +- drop tables or columns +- rewrite large tables synchronously +- mutate core tables +- mutate another plugin's tables +- create triggers or background jobs outside the plugin's ownership boundary +- depend on request-time execution + +Plugin-owned table names must use a deterministic prefix: + +```txt +junior__* +``` + +For plugin names containing hyphens, the SQL table prefix replaces hyphens with +underscores. For example, plugin `long-memory` owns +`junior_long_memory_*`. + +Core does not parse or validate plugin migration SQL for ownership. The prefix +is a convention for plugin authors and reviewers, not a runtime security +boundary. + +### Runtime DB Access + +Trusted runtime hook contexts may expose `ctx.db` when all of these are true: + +1. A Junior SQL database URL is configured. +2. The plugin is explicitly enabled. +3. The plugin declared database access through code registration. +4. The hook is running in host runtime code, not sandboxed model-controlled + code. + +Runtime does not validate plugin migration state before creating `ctx.db`. +`junior upgrade` is the only command that applies plugin migrations and checks +stored migration checksums. Deployments must run `junior upgrade` before serving +traffic for a build that enables or changes database-backed plugins. + +The V1 surface is a shared database connection/query capability: + +```ts +interface PluginDb { + select: JuniorDrizzleConnection["select"]; + insert: JuniorDrizzleConnection["insert"]; + update: JuniorDrizzleConnection["update"]; + delete: JuniorDrizzleConnection["delete"]; + execute(statement: string, params?: readonly unknown[]): Promise; + query( + statement: string, + params?: readonly unknown[], + ): Promise; + transaction(callback: (tx: PluginDb) => Promise): Promise; +} +``` + +Hook contexts should expose this as `ctx.db`, not `ctx.database` or a nested +`ctx.db.db`. + +`ctx.db` is not model-visible and must not be exposed to sandbox tools, skill +text, MCP tools, or tool input schemas. + +### Drizzle Typing Boundary + +Plugins own their table objects and row types. + +Plugin code can import its own Drizzle table objects and use them with +`ctx.db`: + +```ts +import { memories } from "./db/schema"; + +const rows = await ctx.db.select().from(memories); +``` + +The table object carries the row type for plugin queries. Core does not need to +merge plugin schemas into `juniorSqlSchema` for this query style to be typed. + +V1 does not support: + +- auto-importing `src/db/schema.ts` by convention +- `ctx.db.query.` relation helpers for plugin tables +- a public type that represents every installed plugin table + +If a future plugin needs globally composed Drizzle schema typing, that must be +added through an explicit code registration contract, not filesystem +auto-discovery. + +### Database Plugins + +Junior deployments require a SQL database. Plugins that use SQL still declare +database access through code registration so runtime contexts know whether to +expose `ctx.db`: + +```ts +defineJuniorPlugin({ + manifest, + database: {}, + hooks, +}); +``` + +Rules: + +1. Runtime and `junior upgrade` fail when Junior cannot resolve a SQL database + URL. +2. A plugin receives `ctx.db` only when it declares database access through code + registration. +3. Migration application and checksum validation happen only in `junior +upgrade`. +4. Declarative `plugin.yaml` cannot declare executable database behavior. + +### Store Boundaries + +Plugin hooks should not scatter ad hoc SQL throughout hook bodies. A plugin +should keep database access behind a small plugin-owned store module, such as a +memory store for the memory plugin. + +Plugin stores must parse database rows at their boundary before returning +domain records. Drizzle table types are compile-time help, not runtime +validation for data read from the database. + +## Failure Model + +1. Missing database URL: `junior upgrade` and startup fail. +2. Migration discovery failure for an enabled plugin: upgrade fails. +3. Migration checksum mismatch: upgrade fails. +4. Plugin migration SQL failure: upgrade fails before the new runtime serves + traffic. +5. Plugin storage migration hook failure: upgrade fails after schema migration and + before the new runtime serves traffic. +6. Plugin database query failure during a hook: the hook fails according to its + owning hook spec; prompt and observation hooks must fail closed with safe + logging. + +## Observability + +Plugin database logs and spans may include: + +- plugin name +- migration filename and migration id +- checksum prefix +- migration count +- migration outcome and duration +- database availability state +- plugin store operation name and duration +- plugin storage migration outcome and duration + +Logs and spans must not include raw private memory content, private +conversation text, credentials, authorization URLs, SQL parameter values that +may contain private user data, or raw query result payloads. + +## Verification + +Use integration tests with the local Postgres-compatible PGlite fixture for: + +- migration application from named code plugin registrations with package + `migrations/*.sql` +- no discovery from undeclared packages +- no migration application from package-name or local `plugin.yaml` plugins +- migration id/checksum recording in `junior_schema_migrations` +- deterministic plugin migration order +- checksum mismatch failure +- missing database URL failure +- plugins without database declarations do not receive `ctx.db` +- typed plugin table queries using plugin-owned Drizzle table objects +- plugin storage migration hooks run after plugin schema migrations +- plugin storage migration hooks are idempotent across repeated upgrade runs + +Use unit tests for: + +- migration filename validation +- table-prefix derivation from plugin names +- build/package bundling including `migrations/` +- `ctx.db` presence checks in hook context construction + +No evals are required for the database extension mechanism itself. + +## Related Specs + +- `./conversation-storage.md` +- `./plugin.md` +- `./plugin-runtime.md` +- `./plugin-prompt-hooks.md` +- `./memory-plugin/index.md` +- `./plugin-heartbeat.md` +- `./testing.md` diff --git a/specs/plugin-heartbeat.md b/specs/plugin-heartbeat.md index b01b7ff57..a523ddb7f 100644 --- a/specs/plugin-heartbeat.md +++ b/specs/plugin-heartbeat.md @@ -53,7 +53,7 @@ Plugins own only their domain logic: tools, heartbeat work discovery, durable pl Plugins may register turn-scoped tools: ```ts -interface AgentPluginHooks { +interface PluginHooks { tools?(ctx: ToolRegistrationContext): Record; } ``` @@ -101,7 +101,7 @@ The endpoint is a pulse, not a job runner. Plugins may implement: ```ts -interface AgentPluginHooks { +interface PluginHooks { heartbeat?(ctx: HeartbeatContext): Promise; } ``` diff --git a/specs/plugin-prompt-hooks.md b/specs/plugin-prompt-hooks.md new file mode 100644 index 000000000..0675cd9fe --- /dev/null +++ b/specs/plugin-prompt-hooks.md @@ -0,0 +1,426 @@ +# Plugin Prompt Hooks Spec + +## Metadata + +- Created: 2026-06-12 +- Last Edited: 2026-06-13 + +## Purpose + +Define the generic plugin hooks that let runtime hook plugins contribute prompt +text, observe completed turns, enqueue plugin background work, and keep +per-session append-only bookkeeping without exposing raw Junior internals or +creating memory-specific plugin APIs. + +## Implementation Status + +This is a target design for future plugin prompt, observation, session-state, +and background-task hooks. The current `@sentry/junior-plugin-api` package does +not export `userPrompt`, `observeTurn`, plugin prompt session state, or plugin +background task handlers, and Junior core does not invoke those hooks yet. + +## Scope + +- Plugin-provided system prompt and user prompt contributions. +- Prompt hook context and plugin-scoped session append state. +- Post-turn observation hook and plugin background task contract for passive + extraction workflows. +- Security and rendering boundaries for prompt contributions. +- V1 memory plugin usage of these generic hooks. + +## Non-Goals + +- A memory-specific retrieval or extraction hook. +- Plugin-owned prompt rendering. +- Cross-plugin session state access. +- A general event bus for every runtime lifecycle transition. +- Model-visible memory management as the only memory path. +- Storage schema for long-lived memory records. +- Exposing raw queue clients, queue topic names, callback routes, or worker + implementation details to plugins. + +## Contracts + +### Hook Surface + +Runtime hook plugins may provide prompt and observation hooks: + +```ts +interface PluginHooks { + systemPrompt?( + ctx: SystemPromptHookContext, + ): PromptContribution[] | Promise; + + userPrompt?( + ctx: UserPromptHookContext, + ): UserPromptContributionResult | Promise; + + observeTurn?(ctx: TurnObservationContext): void | Promise; + + tasks?: Record; +} +``` + +These hooks are app-code plugin hooks registered through +`defineJuniorPlugin({ manifest, hooks })`. Declarative `plugin.yaml` manifests +must not register prompt or observation hooks. + +### Prompt Contributions + +Prompt contributions are intentionally small: + +```ts +interface PromptContribution { + id: string; + text: string; +} +``` + +Rules: + +1. `id` is unique only within one plugin hook invocation. +2. `text` is plugin-provided prompt text after the plugin has applied its own + domain policy. +3. Core owns ordering between plugins, wrapper rendering, escaping where needed, + total size limits, and failure behavior. +4. Contributions are not durable state by themselves. If a plugin needs + deterministic continuity, it must use session append state. + +### System Prompt Hook + +`systemPrompt(ctx)` contributes stable plugin-level prompt text. + +System prompt contributions: + +1. Must not include requester-specific, conversation-specific, or private data. +2. Must not include provider credentials, authorization URLs, tokens, or raw + tool payloads. +3. Must be byte-stable for the same installed plugin configuration and source + platform. +4. Should be used sparingly for plugin behavior rules that cannot live in tool + descriptions, schemas, skills, or user prompt context. + +Core appends accepted system prompt contributions to the platform static prompt +after core-owned behavior rules and before the model receives the first user +message. Plugin system prompt text remains subordinate to core safety, +credential, tool, and output rules. + +### User Prompt Hook + +`userPrompt(ctx)` contributes dynamic request-scoped prompt text. Core invokes +the hook for every model-visible user prompt. + +```ts +interface UserPromptContributionResult { + contributions?: PromptContribution[]; + sessionState?: PluginSessionStateAppend[]; +} +``` + +Rules: + +1. User prompt contributions may depend on the current requester, source, + destination, conversation id, user text, plugin state, and plugin session + append state. +2. User prompt contributions must be inserted into the model-visible user + message, not the static system prompt. +3. The hook must not receive runtime implementation details such as timeout + continuation or auth-resume state. It receives product-level prompt facts + only. +4. Core commits returned `sessionState` appends only after it accepts the + corresponding contribution result for rendering. +5. If the hook returns no contributions, core must not append its returned + `sessionState`. + +### User Prompt Context + +`UserPromptHookContext` exposes only narrow runtime facts and helper surfaces: + +```ts +interface UserPromptHookContext { + conversationId?: string; + destination?: Destination; + isFirstPrompt: boolean; + log: PluginLogger; + plugin: PluginMetadata; + requester?: Requester; + session: PluginSessionState; + source: Source; + state: PluginState; + userText: string; +} +``` + +`isFirstPrompt` means this is the first model-visible user prompt in the +current agent session projection. It is the only prompt lifecycle flag exposed +in V1. + +The context must not expose: + +- raw Slack clients or tokens +- raw HTTP requests +- raw Pi internals +- continuation, resume, retry, or lease state +- cross-plugin state +- model messages outside the safe hook-specific context + +### Plugin Session Append State + +Prompt hooks may use per-session append state to track deterministic plugin +bookkeeping such as memories already injected into the model-visible prompt. + +```ts +interface PluginSessionStateAppend { + key: string; + value: unknown; +} + +interface PluginSessionState { + list( + key: string, + ): Promise>; +} +``` + +Rules: + +1. Session state is implicitly namespaced by plugin name. Plugin code never + supplies a plugin name. +2. Plugins can read only their own session append state. +3. Session state is append-only in V1. +4. Keys must be short validated strings. +5. Values must be bounded JSON-serializable data. +6. Session state is not an authorization source. Plugins must re-check current + visibility and access before reusing a stored id or fact. +7. Core appends session state in the same durable session-log stream used to + reconstruct model-visible session state. +8. Session state is plugin-visible bookkeeping, not automatically model-visible + prompt text. +9. `list` returns entries from the current model-visible session projection, + not every append ever written for the conversation. If compaction or another + projection change removes the prompt contribution associated with an append, + that append must not be returned to the plugin hook. + +The memory plugin can use this surface to record injected memory ids: + +```ts +const prior = await ctx.session.list<{ memoryIds: string[] }>( + "injected_memories", +); +``` + +### Turn Observation Hook + +`observeTurn(ctx)` lets plugins inspect a completed turn and enqueue bounded +post-turn work such as passive memory extraction. + +Core invokes observation hooks only after final turn state is committed far +enough that the hook cannot affect whether the user-visible turn succeeds. + +Observation context should include: + +- requester, source, destination, and conversation id +- bounded user-visible turn text needed by the plugin +- safe metadata about attachments and tool use +- plugin-scoped durable state and logger +- plugin-scoped background task enqueue capability + +The bounded observation payload is a runtime-owned projection, not a raw +transcript. Core may expose the same projection directly to `observeTurn(ctx)` +and later through `PluginTaskContext.observation.load()` for +observation-backed tasks. + +Observation hooks must not receive provider credentials, raw authorization URLs, +raw Slack clients, or unrestricted transcript history. For private +conversations, observation payloads must follow the same raw-payload restrictions +as runtime code: a plugin may receive private turn text only when it is an +explicitly enabled trusted host plugin whose contract requires that payload. + +Observation hooks must be best effort. A thrown observation error must be logged +with safe metadata and must not fail the already-completed user turn. + +### Plugin Background Tasks + +Observation hooks may enqueue plugin-owned background tasks through a +core-owned task capability: + +```ts +interface PluginTaskEnqueueOptions { + idempotencyKey: string; + name: string; + payload?: unknown; +} + +interface PluginTaskEnqueueResult { + id: string; + status: "created" | "already_exists"; +} + +interface PluginTaskQueue { + enqueue(options: PluginTaskEnqueueOptions): Promise; +} + +interface PluginTaskContext extends PluginContext { + id: string; + name: string; + payload?: unknown; + observation?: { + load(): Promise; + }; +} + +type PluginTaskHandler = (ctx: PluginTaskContext) => Promise | void; +``` + +The exact host implementation is not part of the plugin API. Core may run +plugin tasks with the existing queue infrastructure, a signed internal callback, +a future dedicated task worker, or a local in-process test worker. Plugin code +must observe the same contract in all cases. + +Task rules: + +1. Task names are resolved only inside the owning plugin. +2. Idempotency is scoped to plugin name and task name. +3. Task payloads must be bounded JSON-serializable data. +4. Task payloads should contain stable references and safe metadata, not raw + private prompt text, raw tool payloads, credentials, or tokens. +5. Task handlers run with plugin-scoped `ctx.db`, `ctx.state`, logger, and the + runtime-owned context needed by that task type. +6. Observation-backed tasks receive an `observation.load()` helper when core can + reconstruct a bounded observation payload from durable runtime state. +7. Task handlers must be idempotent because delivery is at least once. +8. Core owns queue acknowledgement, retry, redelivery, worker leases, callback + signing, and provider-specific visibility timeouts. +9. Plugins must not depend on task execution happening in the same process or + same request as `observeTurn`. + +For memory extraction, the observation hook should enqueue a task with stable +conversation/session/message references. The task worker reloads the bounded +observation payload from durable runtime state before invoking the plugin task +handler. Queue payloads must not become the authority for private conversation +text. + +### Memory Plugin V1 Usage + +The memory plugin should use the generic hooks as follows: + +1. `userPrompt(ctx)` retrieves memories visible to the current requester and + source, excludes memories already recorded in session append state, returns + a concise memory block, and appends injected memory ids to session state. +2. `observeTurn(ctx)` enqueues an idempotent memory extraction task for the + completed turn. +3. `tasks.extractMemories(ctx)` reloads the bounded observation payload, + validates accepted facts, and writes memories idempotently. +4. `tools(ctx)` may expose explicit memory tools such as `createMemory`, + `removeMemory`, `listMemories`, and `searchMemories`. + +When automatic memory injection is enabled, retrieval must not depend on the +model choosing a search tool. When automatic memory injection is disabled by +install policy, `searchMemories` is the explicit model-visible recall path. +Other tools are for explicit user management. + +### Memory Tool Constraints + +V1 memory tools are context-bound: + +1. Tool schemas must not expose model-supplied Slack team ids, channel ids, + user ids, or arbitrary visibility overrides. +2. Creation scope derives from runtime-owned requester, source, and + destination context. +3. Listing and removal must show or affect only memories visible in the current + context. +4. Tools must reject secrets, credentials, tokens, authorization URLs, and + private keys even when the user explicitly asks to remember them. +5. Tool failures caused by invalid user/model input must be model-visible tool + input errors. + +### Rendering And Ordering + +Core owns prompt rendering: + +1. Core calls plugins in deterministic plugin-name order. +2. Core wraps user prompt contributions inside the existing turn-context/user + prompt structure owned by `buildTurnContextPrompt(...)`. +3. Core applies per-contribution and total prompt extension size limits. +4. Core omits empty contributions. +5. Core records safe metadata about accepted contributions without exposing raw + private prompt text through logs, traces, or dashboard APIs. +6. Core must fail closed when prompt contribution rendering, validation, or + session-state append parsing fails. + +## Failure Model + +1. Invalid hook return shape: skip that plugin contribution, log safe metadata, + and continue unless startup validation can catch the problem earlier. +2. Oversized contribution: truncate only if the contribution contract supports + deterministic truncation; otherwise omit and log safe metadata. +3. Session append failure before prompt rendering: omit the corresponding + contribution or fail the turn before the model receives mismatched context. +4. Session append failure after prompt rendering has been accepted: fail the + turn before model execution or retry from the prior durable session state. +5. Observation hook failure: log safe metadata and do not change the completed + turn result. +6. Malformed stored session append entries: ignore entries for plugin helper + reads and log safe metadata; do not repair into guessed state. + +## Observability + +Prompt hook logs and spans may include: + +- plugin name +- hook name +- contribution count +- contribution ids +- contribution text character counts +- session append keys +- outcome and duration + +Prompt hook logs and spans must not include raw private prompt text, private +conversation text, provider credentials, tokens, authorization URLs, raw tool +arguments, raw tool results, or cross-plugin state. + +## Verification + +Use integration tests for: + +- plugin system prompt contributions appear in the static prompt without + exposing requester-specific data +- plugin user prompt contributions appear in model-visible user prompt context +- user prompt hooks run for every user prompt +- `isFirstPrompt` is true only for the first model-visible user prompt in the + current session projection +- plugin session append state is implicitly namespaced by plugin +- plugins cannot read another plugin's session state +- session appends commit only when the corresponding prompt contribution result + is accepted +- private conversation prompt contribution payloads are redacted from logs, + traces, and dashboard APIs + +Use unit tests for: + +- hook return-shape validation +- session state key and value bounds +- deterministic plugin ordering +- memory tool schema rejection of model-supplied actor or destination fields + +Use evals for: + +- automatic memory recall without explicit search tool use when automatic memory + injection is enabled +- explicit memory recall through `searchMemories` when automatic memory + injection is disabled +- explicit create/list/remove memory workflows +- duplicate memory injection avoidance across follow-up prompts +- secret rejection in explicit and passive memory paths + +## Related Specs + +- `./agent-prompt.md` +- `./plugin.md` +- `./plugin-runtime.md` +- `./task-execution.md` +- `./memory-plugin/index.md` +- `./plugin-heartbeat.md` +- `./identity.md` +- `./data-redaction-policy.md` +- `./harness-tool-context.md` diff --git a/specs/plugin-runtime.md b/specs/plugin-runtime.md index 71aeddb06..ea8e37659 100644 --- a/specs/plugin-runtime.md +++ b/specs/plugin-runtime.md @@ -3,7 +3,7 @@ ## Metadata - Created: 2026-05-28 -- Last Edited: 2026-05-30 +- Last Edited: 2026-06-13 ## Purpose @@ -21,7 +21,12 @@ Define how plugin manifests, skills, credentials, and MCP tool catalogs are load - Manifest field syntax; see [Plugin Manifest Spec](./plugin-manifest.md). - Provider credential issuance; see [Credential Injection Spec](./credential-injection.md). -- Plugin heartbeat/dispatch hooks; see [Plugin Heartbeat Spec](./plugin-heartbeat.md). +- Plugin prompt, background task, database, CLI, heartbeat, and dispatch hooks; see + [Plugin Prompt Hooks Spec](./plugin-prompt-hooks.md), + [Plugin Database Spec](./plugin-database.md), + [Plugin CLI Spec](./plugin-cli.md), + [Plugin Heartbeat Spec](./plugin-heartbeat.md), and + [Plugin Dispatch Spec](./plugin-dispatch.md). ## Discovery And Loading @@ -125,7 +130,7 @@ and validates that every registration has a matching manifest. Hook factories carry their manifest inline, so runtime code is not declared from `plugin.yaml`. -Hook contexts expose narrow capabilities rather than raw Junior internals. Plugin hook contracts are defined in [Plugin Heartbeat Spec](./plugin-heartbeat.md) and [Plugin Dispatch Spec](./plugin-dispatch.md). +Hook contexts expose narrow capabilities rather than raw Junior internals. Current hook contracts are defined in [Plugin Database Spec](./plugin-database.md), [Plugin CLI Spec](./plugin-cli.md), [Plugin Heartbeat Spec](./plugin-heartbeat.md), and [Plugin Dispatch Spec](./plugin-dispatch.md). [Plugin Prompt Hooks Spec](./plugin-prompt-hooks.md) is a future target design; its prompt, observation, session-state, and background-task hooks are not exported by `@sentry/junior-plugin-api` or invoked by Junior core yet. Plugin `migrateStorage` hooks are limited to `junior upgrade` storage backfills after SQL schema migration; they are not request-time runtime hooks and must not dispatch agent work. Plugins may provide `routes` to mount host-owned HTTP handlers inside `createApp()`. Route handlers receive only the web-standard `Request` and return a `Response`; plugin API types must not expose Hono internals. Core mounts plugin routes after sandbox-egress detection and before Junior's built-in health, webhook, OAuth, and internal routes. `ALL` route methods are exclusive for a path and must not be combined with explicit methods. Route plugins that serve package assets must keep those assets reachable through package-local code imports or static file references; manifest plugin declarations are not the asset-registration path for plugin routes. @@ -159,5 +164,8 @@ Plugins may also provide `slackConversationLink` to replace the finalized Slack - `./plugin-manifest.md` - `./credential-injection.md` - `./agent-prompt.md` +- `./plugin-prompt-hooks.md` +- `./plugin-database.md` +- `./plugin-cli.md` - `./plugin-heartbeat.md` - `./plugin-dispatch.md` diff --git a/specs/plugin.md b/specs/plugin.md index 67713ac71..a2d299735 100644 --- a/specs/plugin.md +++ b/specs/plugin.md @@ -3,7 +3,7 @@ ## Metadata - Created: 2026-03-01 -- Last Edited: 2026-06-08 +- Last Edited: 2026-06-13 ## Purpose @@ -13,7 +13,8 @@ Define the plugin model for provider integrations. Plugins package declarative r - Plugin package/directory shape. - Ownership boundaries between manifests, skills, runtime loading, credentials, and runtime hooks. -- Links to detailed contracts for manifests, runtime loading, credential injection, and plugin heartbeat/dispatch behavior. +- Links to detailed contracts for manifests, runtime loading, credential + injection, plugin CLI, and plugin heartbeat/dispatch behavior. ## Non-Goals @@ -51,6 +52,10 @@ plugins/sentry/ - [Credential Injection Spec](./credential-injection.md): credential-context-bound provider leases and sandbox egress auth. - [OAuth Flows Spec](./oauth-flows.md): OAuth challenge, callback, and agent continuation behavior. - [Sandbox Snapshots Spec](./sandbox-snapshots.md): runtime dependency snapshot build/reuse. +- [Plugin Prompt Hooks Spec](./plugin-prompt-hooks.md): future target design for prompt contribution, turn observation, plugin background tasks, and plugin session append state hooks. These hooks are not implemented in the current plugin API. +- [Plugin Database Spec](./plugin-database.md): packaged SQL migrations and `ctx.db` access for trusted runtime hook plugins. +- [Plugin CLI Spec](./plugin-cli.md): future plugin-contributed host CLI commands for operator/admin workflows. +- [Memory Plugin Spec](./memory-plugin/index.md): long-term memory implemented through prompt, observation, background task, database, and tool hooks. - [Plugin Heartbeat Spec](./plugin-heartbeat.md): heartbeat and tool hooks. - [Plugin Dispatch Spec](./plugin-dispatch.md): durable `ctx.agent.dispatch` contract. @@ -81,6 +86,10 @@ plugins/sentry/ - `./plugin-manifest.md` - `./plugin-runtime.md` - `./credential-injection.md` +- `./plugin-prompt-hooks.md` +- `./plugin-database.md` +- `./plugin-cli.md` +- `./memory-plugin/index.md` - `./plugin-heartbeat.md` - `./plugin-dispatch.md` - `./sandbox-snapshots.md` diff --git a/specs/scheduler.md b/specs/scheduler.md index e71054e4e..f42ccff39 100644 --- a/specs/scheduler.md +++ b/specs/scheduler.md @@ -20,7 +20,6 @@ Define the first scheduler contract for Junior: users can create durable tasks t ## Non-Goals - A generic event-rule engine for GitHub, Slack, Sentry, or webhook events. -- SQL-backed storage as a V1 requirement. - A full durable workflow runtime such as Temporal or Vercel Workflow. - Reusing agent continuation callbacks as the product scheduler. - Slack `chat.scheduleMessage` as the execution mechanism. @@ -132,16 +131,26 @@ This follows the router and turn-context pattern: background and state live in d ### Storage -V1 must not require SQL. The scheduler store should use the existing durable state dependency already required by Junior deployments. +The scheduler is a trusted runtime plugin and requires plugin SQL storage. Its +plugin package owns the scheduler migration files under `migrations/`, and +`junior upgrade` applies those migrations before scheduler storage hooks run. -The initial implementation may use the Chat SDK state adapter and a global task index: +The SQL store keeps task and run records in scheduler-owned tables: -- `junior:scheduler:task:{task_id}` stores the task record. -- `junior:scheduler:tasks` stores task ids for due scans. -- `junior:scheduler:team:{team_id}:tasks` stores task ids for workspace management. -- `junior:scheduler:run:{run_id}` stores run history. -- `junior:scheduler:active:{task_id}` stores the currently active run marker for task-level overlap prevention. -- `junior:scheduler:claim:{task_id}:{scheduled_for_ms}` is the idempotency claim. +- `junior_scheduler_tasks` stores current task state, destination fields, due + timestamps, schedule metadata, and the full task JSON record. +- `junior_scheduler_runs` stores run claims, dispatch ids, terminal status, + attempt metadata, and the full run JSON record. + +The scheduler store interface remains the stable boundary for tools, heartbeat, +and operational reporting. Runtime hook bodies use plugin SQL through `ctx.db`; +state-backed storage remains an internal compatibility path only for the +one-time storage migration. + +Existing state-backed scheduler records are migrated by the scheduler plugin's +`migrateStorage(ctx)` hook. The hook reads retained `junior:scheduler:*` plugin +state through `ctx.state`, writes scheduler-owned SQL rows through `ctx.db`, and +is idempotent across repeated `junior upgrade` runs. ### Run Idempotency @@ -165,7 +174,7 @@ The scheduler plugin uses two runtime hooks: Heartbeat flow: -1. Load due tasks from the scheduler plugin's namespaced state. +1. Load due tasks from the scheduler SQL store through `ctx.db`. 2. Reconcile previously dispatched runs with `ctx.agent.get(dispatchId)`. 3. Claim up to a small limit of due runs. 4. Mark each claimed run as pending dispatch.