From cddf02b8e397d0407e2b6ed6302043c01a53cbe5 Mon Sep 17 00:00:00 2001 From: Suleiman Shahbari Date: Sat, 27 Jun 2026 13:57:29 +0300 Subject: [PATCH] feat(ai-sdk)!: remove relocated /server provider + make:agent/ai:eval CLI and drop @rudderjs/* peers The /server provider, make:agent scaffolder, and ai:eval CLI command were relocated to the Rudder side and now ship in @rudderjs/ai@1.18.4. Delete them from the engine along with the @rudderjs/core and @rudderjs/console optional peers (and dev deps). The framework-agnostic engine no longer carries any @rudderjs/* peer dependency for these paths. Illustrative @rudderjs/* names in source doc-comments were rephrased to neutral wording. src/queue-job.ts still dynamically imports @rudderjs/queue / @rudderjs/broadcast; that is a separate BYO seam handled by a follow-up issue and is left untouched. --- .changeset/ai-sdk-remove-server-cli.md | 5 + packages/ai-sdk/README.md | 5 +- packages/ai-sdk/package.json | 22 - packages/ai-sdk/src/commands/ai-eval.ts | 465 ------------------- packages/ai-sdk/src/commands/make-agent.ts | 24 - packages/ai-sdk/src/eval-cli.test.ts | 401 ---------------- packages/ai-sdk/src/eval-html.test.ts | 311 ------------- packages/ai-sdk/src/eval/index.ts | 2 +- packages/ai-sdk/src/index.test.ts | 111 +---- packages/ai-sdk/src/isomorphic-check.test.ts | 9 +- packages/ai-sdk/src/observers.ts | 6 +- packages/ai-sdk/src/react/agent-run.ts | 3 +- packages/ai-sdk/src/react/index.ts | 4 +- packages/ai-sdk/src/react/useAgentRun.ts | 2 +- packages/ai-sdk/src/registry.ts | 4 +- packages/ai-sdk/src/server/index.ts | 1 - packages/ai-sdk/src/server/provider.ts | 185 -------- packages/ai-sdk/src/similarity-search.ts | 16 +- packages/ai-sdk/src/types.ts | 9 +- pnpm-lock.yaml | 94 ---- 20 files changed, 36 insertions(+), 1643 deletions(-) create mode 100644 .changeset/ai-sdk-remove-server-cli.md delete mode 100644 packages/ai-sdk/src/commands/ai-eval.ts delete mode 100644 packages/ai-sdk/src/commands/make-agent.ts delete mode 100644 packages/ai-sdk/src/eval-cli.test.ts delete mode 100644 packages/ai-sdk/src/eval-html.test.ts delete mode 100644 packages/ai-sdk/src/server/index.ts delete mode 100644 packages/ai-sdk/src/server/provider.ts diff --git a/.changeset/ai-sdk-remove-server-cli.md b/.changeset/ai-sdk-remove-server-cli.md new file mode 100644 index 0000000..c3bd6d4 --- /dev/null +++ b/.changeset/ai-sdk-remove-server-cli.md @@ -0,0 +1,5 @@ +--- +"@gemstack/ai-sdk": minor +--- + +Remove the relocated Rudder bindings from the engine: the `/server` provider, the `make:agent` scaffolder, and the `ai:eval` CLI command, plus the `@rudderjs/core` and `@rudderjs/console` optional peers. These now live in `@rudderjs/ai` (Rudder users pick them up there unchanged). The framework-agnostic engine no longer carries any `@rudderjs/*` peer dependency for these paths. Closes the ai-sdk/Rudder decouple epic. diff --git a/packages/ai-sdk/README.md b/packages/ai-sdk/README.md index 32e02a5..7f22fee 100644 --- a/packages/ai-sdk/README.md +++ b/packages/ai-sdk/README.md @@ -30,14 +30,13 @@ The core stands alone: `@gemstack/ai-sdk`'s only required runtime dependency is - `ConversationStore`, `UserMemory`, `BudgetStorage` ship in-memory defaults; bring your own backend by implementing the interface. - `CacheAdapter` (the suspendable run stores) and `StorageAdapter` (`ImageGenerator`/`AudioGenerator` `.store()`) are caller-supplied — no storage/cache package is bundled. -The ORM-backed implementations of those contracts (Prisma/Drizzle/native via `@rudderjs/orm`) are a Rudder binding and live in [`@rudderjs/ai`](https://www.npmjs.com/package/@rudderjs/ai) (`@rudderjs/ai/conversation-orm`, `/memory-orm`, `/budget-orm`, `/memory-embedding`), not here. A few remaining opt-in subpaths still carry optional Rudder peers (`/server` → `@rudderjs/core`; doctor + `make:agent` → `@rudderjs/console`). The version line stays `0.x` while the API settles toward `1.0.0`. +The engine is now fully framework-agnostic: it has **no `@rudderjs/*` peer dependency**. The ORM-backed implementations of those contracts (Prisma/Drizzle/native), the `/server` provider, and the `make:agent` scaffolder and `ai:eval` CLI command are all Rudder bindings and live in [`@rudderjs/ai`](https://www.npmjs.com/package/@rudderjs/ai) (`@rudderjs/ai/conversation-orm`, `/memory-orm`, `/budget-orm`, `/memory-embedding`, plus the provider and CLI), not here. Rudder users pick them up there unchanged. The version line stays `0.x` while the API settles toward `1.0.0`. ## Subpath exports | Subpath | What it provides | |---|---| | `.` | Core: `Agent`, `tool`, streaming, middleware, facade | -| `./server` | The server provider entry | | `./node` | Node-only entry | | `./computer-use` | Computer-use tool + executor | | `./eval` | Eval framework (`evalSuite`, metrics, reporters) | @@ -47,6 +46,8 @@ The ORM-backed implementations of those contracts (Prisma/Drizzle/native via `@r > **Moved in `0.3.0`:** the MCP bridge (`mcpClientTools` / `mcpServerFromAgent`), previously the `./mcp` subpath, is now its own package, [`@gemstack/ai-mcp`](https://github.com/gemstack-land/gemstack/tree/main/packages/ai-mcp). Update `@gemstack/ai-sdk/mcp` imports to `@gemstack/ai-mcp` and move the `@modelcontextprotocol/sdk` peer there. > > **Moved to `@rudderjs/ai`:** the ORM-backed stores (`./conversation-orm`, `./memory-orm`, `./budget-orm`, `./memory-embedding`) coupled the engine to `@rudderjs/orm`, so they now live in [`@rudderjs/ai`](https://www.npmjs.com/package/@rudderjs/ai) under the same subpath names. Update `@gemstack/ai-sdk/conversation-orm` imports to `@rudderjs/ai/conversation-orm` (etc.). They implement the same `ConversationStore` / `UserMemory` / `BudgetStorage` contracts, still exported from here. +> +> **Moved to `@rudderjs/ai`:** the `/server` provider (which carried a `@rudderjs/core` peer) and the `make:agent` scaffolder + `ai:eval` CLI command (which carried a `@rudderjs/console` peer) are Rudder bindings, so they now live in [`@rudderjs/ai`](https://www.npmjs.com/package/@rudderjs/ai). The engine no longer ships the `./server` / `./commands/*` subpaths or any `@rudderjs/*` peer. ## License diff --git a/packages/ai-sdk/package.json b/packages/ai-sdk/package.json index e1a4bfb..53760cb 100644 --- a/packages/ai-sdk/package.json +++ b/packages/ai-sdk/package.json @@ -51,18 +51,6 @@ "import": "./dist/node/index.js", "types": "./dist/node/index.d.ts" }, - "./server": { - "import": "./dist/server/index.js", - "types": "./dist/server/index.d.ts" - }, - "./commands/make-agent": { - "import": "./dist/commands/make-agent.js", - "types": "./dist/commands/make-agent.d.ts" - }, - "./commands/ai-eval": { - "import": "./dist/commands/ai-eval.js", - "types": "./dist/commands/ai-eval.d.ts" - }, "./observers": { "import": "./dist/observers.js", "types": "./dist/observers.d.ts" @@ -99,17 +87,9 @@ "zod": "^4.0.0" }, "peerDependencies": { - "@rudderjs/console": "^1.4.3", - "@rudderjs/core": "^1.13.3", "react": ">=19.2.0" }, "peerDependenciesMeta": { - "@rudderjs/console": { - "optional": true - }, - "@rudderjs/core": { - "optional": true - }, "react": { "optional": true } @@ -122,8 +102,6 @@ "openai": ">=4.70.0" }, "devDependencies": { - "@rudderjs/console": "^1.4.3", - "@rudderjs/core": "^1.13.3", "@types/node": "^20.0.0", "@types/react": "^19.2.0", "react": "^19.2.0", diff --git a/packages/ai-sdk/src/commands/ai-eval.ts b/packages/ai-sdk/src/commands/ai-eval.ts deleted file mode 100644 index 70263d1..0000000 --- a/packages/ai-sdk/src/commands/ai-eval.ts +++ /dev/null @@ -1,465 +0,0 @@ -/** - * `pnpm rudder ai:eval` — discover `evals/**\/*.eval.ts` suites, - * run each, and report. Console reporter by default; `--json` emits - * a machine-readable envelope to stdout for CI. - * - * Registered from the CLI loader (`packages/cli/src/index.ts`) - * — the AiProvider doesn't own this so it surfaces even when the - * user app fails to boot, matching the `command:list --json` - * graceful-degradation pattern from #349. - */ - -import { readdir } from 'node:fs/promises' -import path from 'node:path' -import { pathToFileURL } from 'node:url' -import { runSuite, reportConsole, evalSuite, stepsFromResponse } from '../eval/index.js' -import type { EvalSuite, EvalCase, Metric, SuiteReport } from '../eval/index.js' -import { reportJson } from '../eval/json-reporter.js' -import type { SuiteJson } from '../eval/json-reporter.js' -import { reportHtml } from '../eval/html-reporter.js' -import { defaultFixturesDir, readFixture, writeFixture } from '../eval/fixtures.js' -import { AiFake } from '../fake.js' -import type { AiFakeStep } from '../fake.js' -import type { Agent } from '../agent.js' -import type { AgentResponse } from '../types.js' - -type Rudder = { - command( - name: string, - handler: (args: string[]) => void | Promise, - ): { description(text: string): unknown } -} - -/** CLI flags + positional name filter. */ -export interface AiEvalOptions { - /** Substring filter (case-insensitive) applied to suite names. */ - filter?: string - /** Stop on the first failing suite. */ - bail: boolean - /** Emit `{ suites: [...] }` JSON to stdout. */ - json: boolean - /** - * Run against the real provider, capture each case's assistant - * turns to `evals/__fixtures__//.json`. Existing - * fixtures are overwritten — diff in your VCS to see what changed. - * Default `false`. - */ - record?: boolean - /** - * Swap the runtime with `AiFake.fake()` and feed each case its - * recorded fixture via `respondWithSequence`. Zero API calls, - * deterministic regression tests. Cases without a fixture fall - * through to a normal run with a stderr warning. Default `false`. - */ - replay?: boolean - /** - * Path for a self-contained HTML report (#A5 Phase 5). Pasteable - * into PR comments / Slack threads. Coexists with `--json` (JSON - * still goes to stdout, HTML goes to disk). - */ - html?: string -} - -/** - * Test seam — every external dependency gets an injectable - * override. The CLI handler defaults each to its real impl. - */ -export interface AiEvalDeps { - cwd?: string - stdout?: { write(s: string): boolean | void } - stderr?: { write(s: string): boolean | void } - /** Override the file walk (test harness returns a virtual list). */ - discover?: (cwd: string, pattern: string) => Promise - /** Override file → suite loader (test harness uses an in-memory map). */ - loadSuite?: (absPath: string) => Promise - /** Override config lookup (test harness skips `@rudderjs/core`). */ - configPattern?: () => string | null | Promise - /** - * Override fixtures directory (defaults to `/evals/__fixtures__`). - * Tests point to a tmpdir to keep round-trips off the source tree. - */ - fixturesDir?: string -} - -/** Register the `ai:eval` command on the rudder runner. */ -export function registerAiEvalCommand(rudder: Rudder): void { - rudder.command('ai:eval', async (rawArgs: string[]) => { - const code = await runEvalCli(parseArgs(rawArgs)) - if (code !== 0) process.exit(code) - }).description( - 'Run eval suites — pnpm rudder ai:eval [name-pattern] [--bail] [--json] [--record|--replay] [--html ]', - ) -} - -// ─── Args parser ───────────────────────────────────────── - -const VALUE_FLAGS = new Set(['--html']) - -/** - * Parse the rest-of-line. Recognizes: - * - boolean flags: `--bail`, `--json`, `--record`, `--replay` - * - value flags : `--html ` or `--html=` - * - one positional name filter (anything not consumed above) - */ -export function parseArgs(args: string[]): AiEvalOptions { - const positional: string[] = [] - const opts: AiEvalOptions = { bail: false, json: false } - - for (let i = 0; i < args.length; i++) { - const a = args[i]! - if (!a.startsWith('--')) { positional.push(a); continue } - - // `--flag=value` form - const eq = a.indexOf('=') - const name = eq >= 0 ? a.slice(0, eq) : a - const inline = eq >= 0 ? a.slice(eq + 1) : undefined - - if (name === '--bail') { opts.bail = true; continue } - if (name === '--json') { opts.json = true; continue } - if (name === '--record') { opts.record = true; continue } - if (name === '--replay') { opts.replay = true; continue } - if (VALUE_FLAGS.has(name)) { - const value = inline ?? args[i + 1] - if (!inline) i++ // consumed the next arg - if (!value) throw new Error(`[ai-sdk] ${name} requires a value`) - if (name === '--html') opts.html = value - continue - } - // unknown flag — surface as positional so the user sees the typo - positional.push(a) - } - - if (positional[0]) opts.filter = positional[0] - return opts -} - -// ─── Runner ────────────────────────────────────────────── - -/** - * Execute the CLI flow. Returns the process exit code (0 = all pass, - * 1 = at least one suite had a failure or no suites discovered). - * - * The handler is `process.exit`-free so tests can drive it directly. - */ -export async function runEvalCli(opts: AiEvalOptions, deps: AiEvalDeps = {}): Promise { - const cwd = deps.cwd ?? process.cwd() - const stdout = deps.stdout ?? process.stdout - const stderr = deps.stderr ?? process.stderr - - if (opts.record && opts.replay) { - stderr.write('[ai:eval] --record and --replay are mutually exclusive\n') - return 1 - } - - const pattern = await Promise.resolve((deps.configPattern ?? loadConfigPattern)()) ?? 'evals/**/*.eval.ts' - const discover = deps.discover ?? discoverSuiteFiles - const files = await discover(cwd, pattern) - - if (files.length === 0) { - stderr.write(`[ai:eval] no suites found matching ${pattern}\n`) - return opts.json ? emitJson(stdout, []) : 1 - } - - const loader = deps.loadSuite ?? defaultSuiteLoader - const fixturesDir = deps.fixturesDir ?? defaultFixturesDir(cwd) - const reports: SuiteJson[] = [] - const fullReports: SuiteReport[] = [] - let exitCode = 0 - - // `--replay` swaps the global runtime once, restored when we're done. - // The per-case fixture is set on the AiFake instance inside the - // wrapped agent factory just before each case's `agent.prompt()`. - let fake: AiFake | null = null - if (opts.replay) fake = AiFake.fake() - - try { - for (const file of files) { - let suite: EvalSuite | null - try { - suite = await loader(file) - } catch (err) { - stderr.write(`[ai:eval] failed to load ${path.relative(cwd, file)}: ${formatError(err)}\n`) - exitCode = 1 - if (opts.bail) break - continue - } - if (!suite) { - stderr.write(`[ai:eval] ${path.relative(cwd, file)} has no default eval suite — skipping\n`) - continue - } - - if (opts.filter && !suite.name.toLowerCase().includes(opts.filter.toLowerCase())) continue - - const decorated = await decorateForMode(suite, opts, { fixturesDir, stderr, fake }) - const report = await runSuite(decorated) - fullReports.push(report) - if (opts.json) { - reports.push(reportJson(report)) - } else { - reportConsole(report, { log: (s) => stdout.write(`${s}\n`) }) - } - - if (report.failed > 0) { - exitCode = 1 - if (opts.bail) break - } - } - } finally { - if (fake) fake.restore() - } - - if (opts.json) emitJson(stdout, reports) - if (opts.html) await writeHtmlReport(opts.html, fullReports, cwd, stderr) - return exitCode -} - -async function writeHtmlReport( - htmlPath: string, - reports: SuiteReport[], - cwd: string, - stderr: { write(s: string): boolean | void }, -): Promise { - const { writeFile, mkdir } = await import('node:fs/promises') - const abs = path.isAbsolute(htmlPath) ? htmlPath : path.resolve(cwd, htmlPath) - try { - await mkdir(path.dirname(abs), { recursive: true }) - await writeFile(abs, reportHtml(reports)) - stderr.write(`[ai:eval] wrote HTML report → ${path.relative(cwd, abs)}\n`) - } catch (err) { - stderr.write(`[ai:eval] failed to write HTML report ${abs}: ${formatError(err)}\n`) - } -} - -function emitJson(stdout: { write(s: string): boolean | void }, suites: SuiteJson[]): 0 { - stdout.write(`${JSON.stringify({ suites }, null, 2)}\n`) - return 0 -} - -function formatError(err: unknown): string { - return err instanceof Error ? err.message : String(err) -} - -// ─── Record / replay decoration ─────────────────────────── - -interface DecorateContext { - fixturesDir: string - stderr: { write(s: string): boolean | void } - fake: AiFake | null -} - -/** - * Wrap a suite so each case captures the response (`--record`) or - * pre-loads the fake's sequence (`--replay`) before running. A - * normal run returns the suite untouched. - * - * Implemented as a per-case `agent` / `assert` decoration so the - * runner stays unchanged — `runSuite` doesn't need to know about - * the fixture format. The original `agent`/`assert` for each case - * are still called; we just slip work in around them. - * - * For replay, fixtures load up-front (sync factory contract) so the - * AiFake is primed before each `agent.prompt()` runs. - */ -async function decorateForMode( - suite: EvalSuite, - opts: AiEvalOptions, - ctx: DecorateContext, -): Promise { - if (!opts.record && !opts.replay) return suite - - // Pre-load every fixture for replay so the per-case factory can - // call `respondWithSequence` synchronously. - const replaySteps = new Map() - if (opts.replay) { - for (let i = 0; i < suite.spec.cases.length; i++) { - const c = suite.spec.cases[i]! - const caseName = c.name ?? `case-${i}` - try { - const fixture = await readFixture(ctx.fixturesDir, suite.name, caseName) - if (fixture) replaySteps.set(caseName, fixture.steps) - else ctx.stderr.write( - `[ai:eval] no fixture for ${suite.name}/${caseName} — running against live provider\n`, - ) - } catch (err) { - ctx.stderr.write(`[ai:eval] fixture load error for ${suite.name}/${caseName}: ${formatError(err)}\n`) - } - } - } - - const wrapped = suite.spec.cases.map((c, i): EvalCase => { - const caseName = c.name ?? `case-${i}` - const baseFactory = c.agent ?? suite.spec.agent - const baseAssert = c.assert - - const factory = opts.replay - ? wrapReplayFactory(baseFactory, replaySteps.get(caseName), ctx.fake) - : baseFactory - - const assert: Metric = opts.record - ? wrapRecordAssert(baseAssert, suite.name, caseName, c.input, ctx) - : baseAssert - - const out: EvalCase = { - input: c.input, - assert, - agent: factory, - } - if (c.name) out.name = c.name - if (c.timeout !== undefined) out.timeout = c.timeout - if (c.skip !== undefined) out.skip = c.skip - return out - }) - - const newSpec: typeof suite.spec = { - agent: suite.spec.agent, - cases: wrapped, - } - if (suite.spec.timeout !== undefined) newSpec.timeout = suite.spec.timeout - return evalSuite(suite.name, newSpec) -} - -/** - * Replay path: before each case runs, prime the shared `AiFake` - * with the case's recorded steps. When the fixture is missing the - * factory still returns the agent — it'll hit whatever the AiFake - * is currently scripted to return (typically falling back to the - * default ambient response, which surfaces as an obvious diff in - * the case's assertion). - */ -function wrapReplayFactory( - base: () => Agent, - steps: AiFakeStep[] | undefined, - fake: AiFake | null, -): () => Agent { - return () => { - if (fake && steps) fake.respondWithSequence(steps) - return base() - } -} - -/** - * Record path: after each case's assertion runs, capture the - * agent response's assistant turns to the fixture file. Wrapping - * the assert is the cleanest hook — the runner already passes - * `response` into it, and the wrapped fn still returns the - * original assertion's result. - */ -function wrapRecordAssert( - base: Metric, - suite: string, - caseName: string, - input: string, - ctx: DecorateContext, -): Metric { - return async (response: AgentResponse, mctx) => { - try { - const file = await writeFixture(ctx.fixturesDir, suite, caseName, { - input, - steps: stepsFromResponse(response), - }) - ctx.stderr.write(`[ai:eval] recorded ${path.relative(process.cwd(), file)}\n`) - } catch (err) { - ctx.stderr.write(`[ai:eval] failed to record ${suite}/${caseName}: ${formatError(err)}\n`) - } - return base(response, mctx) - } -} - -// ─── File discovery ────────────────────────────────────── - -/** - * Recursive walk constrained to a `/**\/*` shape. - * Returns absolute paths sorted lexicographically for stable test - * output and predictable `--bail` ordering. - */ -export async function discoverSuiteFiles(cwd: string, pattern: string): Promise { - const { root, suffix } = parsePattern(pattern) - const absRoot = path.resolve(cwd, root) - const out: string[] = [] - await walk(absRoot, suffix, out) - return out.sort() -} - -/** - * Tiny pattern parser — supports `/**\/*` and bare - * `*` (current directory). Anything more elaborate is - * deferred to userland (run a custom script that imports `runSuite`). - * - * Examples: - * `evals/**\/*.eval.ts` → root=`evals`, suffix=`.eval.ts` - * `tests/agents/**\/*.ts` → root=`tests/agents`, suffix=`.ts` - * `*.eval.ts` → root=`.`, suffix=`.eval.ts` - */ -function parsePattern(pattern: string): { root: string; suffix: string } { - const doubleStar = pattern.indexOf('**') - let prefix: string - let postfix: string - if (doubleStar >= 0) { - prefix = pattern.slice(0, doubleStar).replace(/\/$/, '') - postfix = pattern.slice(doubleStar + 2).replace(/^\//, '') - } else { - const lastSlash = pattern.lastIndexOf('/') - prefix = lastSlash >= 0 ? pattern.slice(0, lastSlash) : '' - postfix = lastSlash >= 0 ? pattern.slice(lastSlash + 1) : pattern - } - if (!postfix.startsWith('*')) { - throw new Error( - `[ai-sdk] Unsupported eval pattern "${pattern}". ` + - `Expected /**/* or *.`, - ) - } - return { - root: prefix || '.', - suffix: postfix.slice(1), - } -} - -async function walk(dir: string, suffix: string, out: string[]): Promise { - let entries - try { - entries = await readdir(dir, { withFileTypes: true }) - } catch (err) { - if ((err as NodeJS.ErrnoException).code === 'ENOENT') return - throw err - } - for (const entry of entries) { - const p = path.join(dir, entry.name) - if (entry.isDirectory()) { - if (entry.name === 'node_modules' || entry.name.startsWith('.')) continue - await walk(p, suffix, out) - } else if (entry.isFile() && entry.name.endsWith(suffix)) { - out.push(p) - } - } -} - -// ─── Suite loader ──────────────────────────────────────── - -async function defaultSuiteLoader(file: string): Promise { - const mod = await import(pathToFileURL(file).href) as Record - const candidate = (mod['default'] ?? mod['suite']) as EvalSuite | undefined - if (!candidate || typeof candidate.name !== 'string' || !candidate.spec) return null - return candidate -} - -// ─── Config lookup ─────────────────────────────────────── - -/** - * Read `config('ai').eval.pattern` from the booted app. Returns - * `null` (default pattern) when `@rudderjs/core` isn't loadable - * or the app didn't boot — the CLI must still work in - * introspective mode (#349). - */ -async function loadConfigPattern(): Promise { - try { - // Dynamic import so the static graph doesn't pin `@rudderjs/core` - // (optional peer). Falls back to default when core isn't loadable - // or the app didn't boot. - const core = await import('@rudderjs/core') as { config?: (key: string) => T } - if (typeof core.config !== 'function') return null - const cfg = core.config<{ eval?: { pattern?: string } } | undefined>('ai') - return cfg?.eval?.pattern ?? null - } catch { - return null - } -} diff --git a/packages/ai-sdk/src/commands/make-agent.ts b/packages/ai-sdk/src/commands/make-agent.ts deleted file mode 100644 index c53f5d1..0000000 --- a/packages/ai-sdk/src/commands/make-agent.ts +++ /dev/null @@ -1,24 +0,0 @@ -import type { MakeSpec } from '@rudderjs/console' - -export const makeAgentSpec: MakeSpec = { - command: 'make:agent', - description: 'Create a new AI agent class', - label: 'Agent created', - suffix: 'Agent', - directory: 'app/Agents', - stub: (className) => `import { Agent } from '@gemstack/ai-sdk' -import type { HasTools, AnyTool } from '@gemstack/ai-sdk' - -export class ${className} extends Agent implements HasTools { - instructions(): string { - return 'You are a helpful assistant.' - } - - // model(): string | undefined { return 'anthropic/claude-sonnet-4-5' } - - tools(): AnyTool[] { - return [] - } -} -`, -} diff --git a/packages/ai-sdk/src/eval-cli.test.ts b/packages/ai-sdk/src/eval-cli.test.ts deleted file mode 100644 index 486bfb6..0000000 --- a/packages/ai-sdk/src/eval-cli.test.ts +++ /dev/null @@ -1,401 +0,0 @@ -/** - * `pnpm rudder ai:eval` CLI handler tests (#A5 Phase 2). - * - * Covers: - * - parseArgs: positional name filter + --bail / --json flags - * - discoverSuiteFiles: pattern parsing + recursive walk - * - runEvalCli: name-filter exclusion, --bail short-circuit, --json shape - * - * Real provider calls are stubbed via AiFake so suites complete in - * milliseconds without API keys. - */ - -import { describe, it, beforeEach, afterEach } from 'node:test' -import assert from 'node:assert/strict' -import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises' -import os from 'node:os' -import path from 'node:path' - -import { Agent } from './agent.js' -import { AiFake } from './fake.js' -import { evalSuite, exactMatch, type EvalSuite } from './eval/index.js' -import { - parseArgs, - discoverSuiteFiles, - runEvalCli, -} from './commands/ai-eval.js' - -class StubAgent extends Agent { - instructions() { return 'stub' } -} - -// ─── parseArgs ────────────────────────────────────────────── - -describe('parseArgs', () => { - it('parses --bail and --json flags', () => { - const o = parseArgs(['--bail', '--json']) - assert.equal(o.bail, true) - assert.equal(o.json, true) - assert.equal(o.filter, undefined) - }) - - it('treats the first positional as a name filter', () => { - const o = parseArgs(['support', '--json']) - assert.equal(o.filter, 'support') - assert.equal(o.json, true) - assert.equal(o.bail, false) - }) - - it('returns false flags when omitted', () => { - const o = parseArgs([]) - assert.equal(o.bail, false) - assert.equal(o.json, false) - }) -}) - -// ─── discoverSuiteFiles ───────────────────────────────────── - -describe('discoverSuiteFiles', () => { - let dir: string - beforeEach(async () => { dir = await mkdtemp(path.join(os.tmpdir(), 'ai-eval-disco-')) }) - afterEach(async () => { await rm(dir, { recursive: true, force: true }) }) - - it('walks evals/**/*.eval.ts by default', async () => { - await mkdir(path.join(dir, 'evals', 'agents'), { recursive: true }) - await writeFile(path.join(dir, 'evals', 'a.eval.ts'), 'export {}\n') - await writeFile(path.join(dir, 'evals', 'agents', 'b.eval.ts'), 'export {}\n') - await writeFile(path.join(dir, 'evals', 'helper.ts'), 'export {}\n') // wrong suffix - await writeFile(path.join(dir, 'unrelated.eval.ts'), 'export {}\n') // outside root - - const files = await discoverSuiteFiles(dir, 'evals/**/*.eval.ts') - const rels = files.map(f => path.relative(dir, f)) - assert.deepEqual(rels, [ - path.join('evals', 'a.eval.ts'), - path.join('evals', 'agents', 'b.eval.ts'), - ]) - }) - - it('skips node_modules and dotfile dirs', async () => { - await mkdir(path.join(dir, 'evals', 'node_modules', 'pkg'), { recursive: true }) - await mkdir(path.join(dir, 'evals', '.cache'), { recursive: true }) - await writeFile(path.join(dir, 'evals', 'a.eval.ts'), 'export {}\n') - await writeFile(path.join(dir, 'evals', 'node_modules', 'pkg', 'fake.eval.ts'), 'export {}\n') - await writeFile(path.join(dir, 'evals', '.cache', 'cached.eval.ts'), 'export {}\n') - - const files = await discoverSuiteFiles(dir, 'evals/**/*.eval.ts') - assert.equal(files.length, 1) - assert.match(files[0]!, /a\.eval\.ts$/) - }) - - it('returns [] when the root dir does not exist', async () => { - const files = await discoverSuiteFiles(dir, 'evals/**/*.eval.ts') - assert.deepEqual(files, []) - }) - - it('honors a custom /**/* pattern', async () => { - await mkdir(path.join(dir, 'tests', 'agents'), { recursive: true }) - await writeFile(path.join(dir, 'tests', 'agents', 'one.spec.ts'), 'export {}\n') - const files = await discoverSuiteFiles(dir, 'tests/**/*.spec.ts') - assert.equal(files.length, 1) - }) - - it('throws on patterns that aren\'t /**/*-shaped', async () => { - await assert.rejects( - () => discoverSuiteFiles(dir, 'evals/[abc]/file.ts'), - /Unsupported eval pattern/, - ) - }) -}) - -// ─── runEvalCli ───────────────────────────────────────────── - -interface CapturedStream { - write(s: string): boolean - read(): string -} -function capture(): CapturedStream { - const chunks: string[] = [] - return { - write(s: string): boolean { chunks.push(s); return true }, - read(): string { return chunks.join('') }, - } -} - -function makePassingSuite(name: string): EvalSuite { - return evalSuite(name, { - agent: () => new StubAgent(), - cases: [{ name: 'c1', input: 'hi', assert: exactMatch('hi') }], - }) -} - -function makeFailingSuite(name: string): EvalSuite { - return evalSuite(name, { - agent: () => new StubAgent(), - cases: [{ name: 'c1', input: 'hi', assert: exactMatch('not-hi') }], - }) -} - -describe('runEvalCli', () => { - let fake: AiFake - beforeEach(() => { fake = AiFake.fake() }) - - it('runs matching suites in console mode and returns 0 when all pass', async () => { - fake.respondWith('hi') - const stdout = capture() - const stderr = capture() - const code = await runEvalCli( - { bail: false, json: false }, - { - cwd: '/virtual', - stdout, - stderr, - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => makePassingSuite('A'), - }, - ) - assert.equal(code, 0) - assert.match(stdout.read(), /1 passed, 0 failed/) - }) - - it('returns 1 and prints failure when a case fails', async () => { - fake.respondWith('something else') - const stdout = capture() - const stderr = capture() - const code = await runEvalCli( - { bail: false, json: false }, - { - cwd: '/virtual', - stdout, - stderr, - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => makeFailingSuite('A'), - }, - ) - assert.equal(code, 1) - assert.match(stdout.read(), /0 passed, 1 failed/) - }) - - it('--bail stops on the first failing suite', async () => { - fake.respondWith('wrong') - - const seen: string[] = [] - const code = await runEvalCli( - { bail: true, json: false }, - { - cwd: '/virtual', - stdout: capture(), - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts', '/virtual/evals/b.eval.ts'], - loadSuite: async (file: string) => { - seen.push(path.basename(file)) - if (file.endsWith('a.eval.ts')) return makeFailingSuite('A') - if (file.endsWith('b.eval.ts')) return makePassingSuite('B') - return null - }, - }, - ) - assert.equal(code, 1) - assert.deepEqual(seen, ['a.eval.ts']) // never reaches b.eval.ts - }) - - it('--json emits a {suites: [...]} envelope and skips console output', async () => { - fake.respondWith('hi') - const stdout = capture() - const stderr = capture() - const code = await runEvalCli( - { bail: false, json: true }, - { - cwd: '/virtual', - stdout, - stderr, - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => makePassingSuite('A'), - }, - ) - assert.equal(code, 0) - const out = stdout.read() - // Console reporter would print "passed, failed" lines — JSON mode must not. - assert.doesNotMatch(out, /passed, .* failed/) - const parsed = JSON.parse(out) as { suites: Array<{ suite: string; passed: number; failed: number; cases: unknown[] }> } - assert.equal(parsed.suites.length, 1) - assert.equal(parsed.suites[0]!.suite, 'A') - assert.equal(parsed.suites[0]!.passed, 1) - assert.equal(parsed.suites[0]!.failed, 0) - assert.equal(parsed.suites[0]!.cases.length, 1) - }) - - it('positional filter excludes non-matching suite names (case-insensitive substring)', async () => { - fake.respondWith('hi') - const stdout = capture() - const stderr = capture() - const code = await runEvalCli( - { filter: 'support', bail: false, json: true }, - { - cwd: '/virtual', - stdout, - stderr, - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts', '/virtual/evals/b.eval.ts'], - loadSuite: async (file: string) => { - if (file.endsWith('a.eval.ts')) return makePassingSuite('SupportAgent') - if (file.endsWith('b.eval.ts')) return makePassingSuite('BillingAgent') - return null - }, - }, - ) - assert.equal(code, 0) - const parsed = JSON.parse(stdout.read()) as { suites: Array<{ suite: string }> } - assert.deepEqual(parsed.suites.map(s => s.suite), ['SupportAgent']) - }) - - it('emits an empty envelope (exit 0) in --json mode when no suites match', async () => { - const stdout = capture() - const stderr = capture() - const code = await runEvalCli( - { bail: false, json: true }, - { - cwd: '/no-suites', - stdout, - stderr, - configPattern: () => null, - discover: async () => [], - loadSuite: async () => null, - }, - ) - assert.equal(code, 0) - const parsed = JSON.parse(stdout.read()) as { suites: unknown[] } - assert.deepEqual(parsed.suites, []) - assert.match(stderr.read(), /no suites found/) - }) -}) - -// ─── --record / --replay round-trip (#A5 Phase 4) ───────── - -describe('runEvalCli — --record / --replay', () => { - let fake: AiFake - let fixturesDir: string - - beforeEach(async () => { - fake = AiFake.fake() - fixturesDir = await mkdtemp(path.join(os.tmpdir(), 'ai-eval-rec-')) - }) - afterEach(async () => { - fake.restore() - await rm(fixturesDir, { recursive: true, force: true }) - }) - - it('rejects --record + --replay together', async () => { - const stderr = capture() - const code = await runEvalCli( - { bail: false, json: false, record: true, replay: true }, - { cwd: '/v', stdout: capture(), stderr, configPattern: () => null, discover: async () => [] }, - ) - assert.equal(code, 1) - assert.match(stderr.read(), /mutually exclusive/) - }) - - it('--record writes one JSON fixture per case under fixturesDir//.json', async () => { - fake.respondWithSequence([{ text: 'A reply' }, { text: 'B reply' }]) - const suite: EvalSuite = evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [ - { name: 'first', input: 'a', assert: exactMatch('A reply') }, - { name: 'second', input: 'b', assert: exactMatch('B reply') }, - ], - }) - await runEvalCli( - { bail: false, json: true, record: true }, - { - cwd: '/virtual', - stdout: capture(), - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/sample.eval.ts'], - loadSuite: async () => suite, - fixturesDir, - }, - ) - const fs = await import('node:fs/promises') - const firstPath = path.join(fixturesDir, 'Sample', 'first.json') - const secondPath = path.join(fixturesDir, 'Sample', 'second.json') - const first = JSON.parse(await fs.readFile(firstPath, 'utf8')) as { steps: { text: string }[]; suite: string; case: string } - const second = JSON.parse(await fs.readFile(secondPath, 'utf8')) as { steps: { text: string }[]; case: string } - assert.equal(first.suite, 'Sample') - assert.equal(first.case, 'first') - assert.equal(first.steps[0]!.text, 'A reply') - assert.equal(second.case, 'second') - assert.equal(second.steps[0]!.text, 'B reply') - }) - - it('--replay primes AiFake per-case from fixtures (zero stray prompts)', async () => { - // Write a fixture by hand so replay has something to load. - const fs = await import('node:fs/promises') - await fs.mkdir(path.join(fixturesDir, 'Sample'), { recursive: true }) - await fs.writeFile( - path.join(fixturesDir, 'Sample', 'replayed.json'), - JSON.stringify({ - version: 1, - suite: 'Sample', - case: 'replayed', - input: 'a', - recordedAt: '2026-05-10T00:00:00.000Z', - steps: [{ text: 'fixture-text', finishReason: 'stop' }], - }), - ) - - // The CLI handler creates its OWN AiFake via `AiFake.fake()`, replacing - // the one we set up in beforeEach. We don't pre-script anything; the - // handler's per-case `respondWithSequence` is what we're testing. - fake.preventStrayPrompts() // guard: any unscripted prompt would throw - - const suite: EvalSuite = evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [ - { name: 'replayed', input: 'a', assert: exactMatch('fixture-text') }, - ], - }) - const stdout = capture() - const code = await runEvalCli( - { bail: false, json: true, replay: true }, - { - cwd: '/virtual', - stdout, - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/sample.eval.ts'], - loadSuite: async () => suite, - fixturesDir, - }, - ) - assert.equal(code, 0) - const parsed = JSON.parse(stdout.read()) as { suites: Array<{ passed: number; failed: number }> } - assert.equal(parsed.suites[0]!.passed, 1) - assert.equal(parsed.suites[0]!.failed, 0) - }) - - it('--replay warns on stderr when a fixture is missing', async () => { - const suite: EvalSuite = evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [{ name: 'no-fixture', input: 'x', assert: exactMatch('anything') }], - }) - const stderr = capture() - fake.respondWith('whatever') // fallback so the case can still run - await runEvalCli( - { bail: false, json: true, replay: true }, - { - cwd: '/virtual', - stdout: capture(), - stderr, - configPattern: () => null, - discover: async () => ['/virtual/evals/sample.eval.ts'], - loadSuite: async () => suite, - fixturesDir, - }, - ) - assert.match(stderr.read(), /no fixture for Sample\/no-fixture/) - }) -}) diff --git a/packages/ai-sdk/src/eval-html.test.ts b/packages/ai-sdk/src/eval-html.test.ts deleted file mode 100644 index 76a9823..0000000 --- a/packages/ai-sdk/src/eval-html.test.ts +++ /dev/null @@ -1,311 +0,0 @@ -/** - * HTML reporter + suite metadata tests for #A5 Phase 5. - */ - -import { describe, it, beforeEach, afterEach } from 'node:test' -import assert from 'node:assert/strict' -import { mkdtemp, readFile, rm } from 'node:fs/promises' -import os from 'node:os' -import path from 'node:path' - -import { Agent } from './agent.js' -import { AiFake } from './fake.js' -import { - evalSuite, - exactMatch, - runSuite, - reportHtml, - type SuiteReport, -} from './eval/index.js' -import { runEvalCli } from './commands/ai-eval.js' - -class StubAgent extends Agent { - instructions() { return 'stub' } -} - -// ─── reportHtml ────────────────────────────────────────── - -describe('reportHtml', () => { - it('renders suite + case names + stats', () => { - const reports: SuiteReport[] = [ - { - suite: 'SupportAgent', - cases: [ - { name: 'reset', status: 'passed', duration: 10, tokens: 50, cost: 0.001, input: 'reset?', responseText: 'go to /reset', metric: { pass: true, score: 1 } }, - { name: 'pricing', status: 'failed', duration: 20, tokens: 70, cost: 0.002, input: 'cost?', responseText: 'free!', metric: { pass: false, reason: 'expected $99' } }, - ], - passed: 1, failed: 1, skipped: 0, duration: 30, cost: 0.003, tokens: 120, - }, - ] - const html = reportHtml(reports, { generatedAt: '2026-05-10T00:00:00Z' }) - assert.match(html, //) - assert.match(html, /Eval Report<\/title>/) - assert.match(html, /SupportAgent/) - assert.match(html, /reset/) - assert.match(html, /go to \/reset/) - assert.match(html, /pricing/) - assert.match(html, /expected \$99/) - assert.match(html, /50%\s*pass/) - }) - - it('HTML-escapes user content (response, input, names)', () => { - const reports: SuiteReport[] = [ - { - suite: '<script>alert(1)</script>', - cases: [ - { - name: 'xss & friends', - status: 'failed', - duration: 1, - tokens: 0, - cost: 0, - input: '<img src=x onerror=alert(1)>', - responseText: 'a "quoted" \'string\' with <tags>', - metric: { pass: false, reason: '<bad>' }, - }, - ], - passed: 0, failed: 1, skipped: 0, duration: 1, cost: 0, tokens: 0, - }, - ] - const html = reportHtml(reports) - // No raw `<script>` from the suite name should leak into the document body. - const bodyStart = html.indexOf('<body>') - const body = html.slice(bodyStart) - assert.ok(!body.includes('<script>alert(1)</script>'), 'suite-name <script> must be escaped') - assert.ok(body.includes('<script>alert(1)</script>'), 'suite name should be escaped') - assert.ok(!body.includes('<img src=x'), 'input <img> must be escaped') - assert.match(body, /"quoted"/) - assert.match(body, /<tags>/) - }) - - it('renders metadata block when present, omits it when not', () => { - const withMeta: SuiteReport = { - suite: 'A', cases: [], passed: 0, failed: 0, skipped: 0, duration: 0, cost: 0, tokens: 0, - metadata: { owner: '@jane', lastReviewed: '2026-05-01', ticket: 'AI-42', custom: 'x' }, - } - const withoutMeta: SuiteReport = { - suite: 'B', cases: [], passed: 0, failed: 0, skipped: 0, duration: 0, cost: 0, tokens: 0, - } - const html = reportHtml([withMeta, withoutMeta]) - assert.match(html, /<dt>Owner<\/dt>\s*<dd>@jane<\/dd>/) - assert.match(html, /<dt>Last reviewed<\/dt>/) - assert.match(html, /<dt>Ticket<\/dt>\s*<dd>AI-42<\/dd>/) - assert.match(html, /<dt>Custom<\/dt>\s*<dd>x<\/dd>/) - // The without-meta suite section must not introduce a metadata <dl>. - const sections = html.split('<section class="suite">') - assert.equal(sections.length, 3) // [pre, A, B] - assert.ok(!sections[2]!.includes('<dl class="metadata">'), - 'second suite without metadata should have no <dl>') - }) - - it('renders <no response> when responseText is absent', () => { - const reports: SuiteReport[] = [ - { - suite: 'A', - cases: [{ name: 'agent-threw', status: 'failed', duration: 1, tokens: 0, cost: 0, input: 'q', metric: { pass: false, reason: 'crash' } }], - passed: 0, failed: 1, skipped: 0, duration: 1, cost: 0, tokens: 0, - }, - ] - const html = reportHtml(reports) - assert.match(html, /<no response — agent threw or skipped>/) - }) - - it('includes the script + click handler on the row', () => { - const html = reportHtml([{ - suite: 'S', - cases: [{ name: 'c', status: 'passed', duration: 1, tokens: 0, cost: 0, input: 'i', responseText: 'o', metric: { pass: true, score: 1 } }], - passed: 1, failed: 0, skipped: 0, duration: 1, cost: 0, tokens: 0, - }]) - assert.match(html, /addEventListener\('click'/) - assert.match(html, /aria-expanded="false"/) - }) - - it('handles empty reports list (all-skip / no-suites edge case)', () => { - const html = reportHtml([]) - assert.match(html, /0 suites/) - assert.match(html, /0 cases/) - assert.match(html, /0%\s*pass/) - }) -}) - -// ─── Suite metadata thread-through ─────────────────────── - -describe('evalSuite metadata', () => { - let fake: AiFake - beforeEach(() => { fake = AiFake.fake() }) - afterEach(() => fake.restore()) - - it('preserves metadata on the frozen suite + on SuiteReport', async () => { - fake.respondWith('hi') - const suite = evalSuite('S', { - agent: () => new StubAgent(), - cases: [{ input: 'x', assert: exactMatch('hi') }], - metadata: { owner: '@jane', ticket: 'AI-1' }, - }) - assert.deepEqual(suite.spec.metadata, { owner: '@jane', ticket: 'AI-1' }) - const report = await runSuite(suite) - assert.deepEqual(report.metadata, { owner: '@jane', ticket: 'AI-1' }) - }) - - it('omits metadata field when not provided (back-compat)', async () => { - fake.respondWith('hi') - const suite = evalSuite('S', { - agent: () => new StubAgent(), - cases: [{ input: 'x', assert: exactMatch('hi') }], - }) - const report = await runSuite(suite) - assert.equal('metadata' in report, false) - }) -}) - -// ─── CaseResult.input / responseText ───────────────────── - -describe('CaseResult — input + responseText', () => { - let fake: AiFake - beforeEach(() => { fake = AiFake.fake() }) - afterEach(() => fake.restore()) - - it('input is always populated, responseText is set on a successful run', async () => { - fake.respondWith('the response') - const suite = evalSuite('S', { - agent: () => new StubAgent(), - cases: [{ input: 'the input', assert: exactMatch('the response') }], - }) - const report = await runSuite(suite) - assert.equal(report.cases[0]!.input, 'the input') - assert.equal(report.cases[0]!.responseText, 'the response') - }) - - it('responseText is omitted on skipped cases (input still populated)', async () => { - const suite = evalSuite('S', { - agent: () => new StubAgent(), - cases: [{ input: 'q', assert: exactMatch('a'), skip: true }], - }) - const report = await runSuite(suite) - assert.equal(report.cases[0]!.input, 'q') - assert.equal(report.cases[0]!.responseText, undefined) - }) -}) - -// ─── CLI --html flag ───────────────────────────────────── - -describe('runEvalCli — --html', () => { - let fake: AiFake - let tmp: string - beforeEach(async () => { - fake = AiFake.fake() - tmp = await mkdtemp(path.join(os.tmpdir(), 'ai-eval-html-')) - }) - afterEach(async () => { - fake.restore() - await rm(tmp, { recursive: true, force: true }) - }) - - it('writes a self-contained HTML file at the given path', async () => { - fake.respondWith('hi') - const htmlPath = path.join(tmp, 'report.html') - const code = await runEvalCli( - { bail: false, json: false, html: htmlPath }, - { - cwd: '/virtual', - stdout: capture(), - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [{ name: 'first', input: 'x', assert: exactMatch('hi') }], - }), - }, - ) - assert.equal(code, 0) - const contents = await readFile(htmlPath, 'utf8') - assert.match(contents, /<title>Eval Report<\/title>/) - assert.match(contents, /Sample/) - assert.match(contents, /first/) - }) - - it('coexists with --json (HTML to file, JSON to stdout)', async () => { - fake.respondWith('hi') - const htmlPath = path.join(tmp, 'report.html') - const stdout = capture() - await runEvalCli( - { bail: false, json: true, html: htmlPath }, - { - cwd: '/virtual', - stdout, - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [{ name: 'first', input: 'x', assert: exactMatch('hi') }], - }), - }, - ) - // JSON envelope is on stdout - const parsed = JSON.parse(stdout.read()) as { suites: unknown[] } - assert.equal(parsed.suites.length, 1) - // HTML file is on disk - const html = await readFile(htmlPath, 'utf8') - assert.match(html, /<title>Eval Report<\/title>/) - }) - - it('creates intermediate directories for the HTML path', async () => { - fake.respondWith('hi') - const nested = path.join(tmp, 'reports', '2026', 'eval.html') - await runEvalCli( - { bail: false, json: false, html: nested }, - { - cwd: '/virtual', - stdout: capture(), - stderr: capture(), - configPattern: () => null, - discover: async () => ['/virtual/evals/a.eval.ts'], - loadSuite: async () => evalSuite('Sample', { - agent: () => new StubAgent(), - cases: [{ input: 'x', assert: exactMatch('hi') }], - }), - }, - ) - const html = await readFile(nested, 'utf8') - assert.match(html, /Sample/) - }) -}) - -// ─── parseArgs --html parsing ──────────────────────────── - -describe('parseArgs --html', () => { - it('supports --html=value form', async () => { - const { parseArgs } = await import('./commands/ai-eval.js') - assert.equal(parseArgs(['--html=out.html']).html, 'out.html') - }) - - it('supports --html value form', async () => { - const { parseArgs } = await import('./commands/ai-eval.js') - assert.equal(parseArgs(['--html', 'out.html']).html, 'out.html') - }) - - it('throws when --html has no value', async () => { - const { parseArgs } = await import('./commands/ai-eval.js') - assert.throws(() => parseArgs(['--html']), /requires a value/) - }) - - it('does not consume positional name filter as the --html value', async () => { - const { parseArgs } = await import('./commands/ai-eval.js') - const o = parseArgs(['support', '--html=out.html']) - assert.equal(o.filter, 'support') - assert.equal(o.html, 'out.html') - }) -}) - -// ─── helpers ───────────────────────────────────────────── - -interface CapturedStream { write(s: string): boolean; read(): string } -function capture(): CapturedStream { - const chunks: string[] = [] - return { - write(s) { chunks.push(s); return true }, - read() { return chunks.join('') }, - } -} diff --git a/packages/ai-sdk/src/eval/index.ts b/packages/ai-sdk/src/eval/index.ts index 76e7512..7be5efc 100644 --- a/packages/ai-sdk/src/eval/index.ts +++ b/packages/ai-sdk/src/eval/index.ts @@ -427,7 +427,7 @@ export function compose(...metrics: Metric[]): Metric { } } -/** Local cosine — kept inline so `eval/` doesn't pull in `memory-embedding` (which depends on `@rudderjs/orm`). */ +/** Local cosine — kept inline so `eval/` doesn't pull in `memory-embedding` (which depends on an ORM). */ function cosineSimilarity(a: number[], b: number[]): number { if (a.length !== b.length) return 0 let dot = 0 diff --git a/packages/ai-sdk/src/index.test.ts b/packages/ai-sdk/src/index.test.ts index c21e0b7..be8d51c 100644 --- a/packages/ai-sdk/src/index.test.ts +++ b/packages/ai-sdk/src/index.test.ts @@ -1897,112 +1897,6 @@ describe('AiFake', () => { }) }) -// ─── AiProvider ─────────────────────────────────────────── - -import { AiProvider } from './server/provider.js' -import * as _core from '@rudderjs/core' - -describe('AiProvider', () => { - it('is a ServiceProvider class', () => { - assert.ok(typeof AiProvider === 'function') - assert.ok(AiProvider.prototype) - }) -}) - -describe('AiProvider — empty apiKey skip-and-warn', () => { - // Fake `app` object — AiProvider only needs `.instance` and `.container.has`. - function makeFakeApp(): never { - return { - instance: () => undefined, - container: { has: () => false }, - make: () => undefined, - } as never - } - - async function bootWith(aiConfig: Record<string, unknown>): Promise<string[]> { - AiRegistry.reset() - const previousRepo = _core.getConfigRepository?.() - _core.drainBootNotices() // clear any buffered notices before this boot - - _core.setConfigRepository(new _core.ConfigRepository({ ai: aiConfig })) - try { - const provider = new AiProvider(makeFakeApp()) - await provider.boot() - } finally { - if (previousRepo) _core.setConfigRepository(previousRepo) - } - // Skipped providers now register a grouped boot notice (scope 'ai') instead - // of console.warn-ing inline; the framework flushes them after the tree. - return _core.drainBootNotices().map(n => `${n.scope}: ${n.message}`) - } - - it('boots cleanly when an apiKey-requiring provider has empty apiKey, warns once', async () => { - const warnings = await bootWith({ - default: 'anthropic/claude-sonnet-4-5', - providers: { - anthropic: { driver: 'anthropic', apiKey: '' }, - }, - }) - - assert.equal(warnings.length, 1, 'one notice for the skipped anthropic provider') - assert.match(warnings[0]!, /anthropic skipped/) - assert.match(warnings[0]!, /no API key/) - assert.throws( - () => AiRegistry.getFactory('anthropic'), - /Unknown AI provider "anthropic"/, - 'anthropic should not be registered', - ) - }) - - it('registers providers that DO have a key while skipping ones that don\'t', async () => { - const warnings = await bootWith({ - default: 'anthropic/claude-sonnet-4-5', - providers: { - anthropic: { driver: 'anthropic', apiKey: 'sk-real-key' }, - openai: { driver: 'openai', apiKey: '' }, - google: { driver: 'google', apiKey: '' }, - ollama: { driver: 'ollama', baseUrl: 'http://localhost:11434' }, - }, - }) - - assert.equal(warnings.length, 2, 'two notices — openai + google') - assert.ok(warnings.some(w => /openai skipped/.test(w)), 'openai noticed') - assert.ok(warnings.some(w => /google skipped/.test(w)), 'google noticed') - - assert.doesNotThrow(() => AiRegistry.getFactory('anthropic'), 'anthropic registered') - assert.doesNotThrow(() => AiRegistry.getFactory('ollama'), 'ollama registered (no apiKey needed)') - assert.throws(() => AiRegistry.getFactory('openai'), /Unknown AI provider "openai"/) - assert.throws(() => AiRegistry.getFactory('google'), /Unknown AI provider "google"/) - }) - - it('boots clean with zero providers configured', async () => { - const warnings = await bootWith({ - default: 'anthropic/claude-sonnet-4-5', - providers: {}, - }) - - assert.equal(warnings.length, 0, 'no warnings when no providers are configured') - }) - - it('boots clean when every apiKey-requiring provider is empty (matches fresh-scaffolded state)', async () => { - // Reproduces the scaffolder's default ai.ts: anthropic/openai/google all - // present, all reading from env vars that haven't been set yet, plus - // ollama (no apiKey needed). Pre-fix this would crash on the first one. - const warnings = await bootWith({ - default: 'anthropic/claude-sonnet-4-5', - providers: { - anthropic: { driver: 'anthropic', apiKey: '' }, - openai: { driver: 'openai', apiKey: '' }, - google: { driver: 'google', apiKey: '' }, - ollama: { driver: 'ollama', baseUrl: 'http://localhost:11434' }, - }, - }) - - assert.equal(warnings.length, 3, 'one warning per apiKey-requiring provider') - assert.doesNotThrow(() => AiRegistry.getFactory('ollama'), 'ollama still registers') - }) -}) - // ─── Client tools + tool approval ──────────────────────────── // // A scriptable mock provider lets each test queue up provider responses @@ -3450,9 +3344,8 @@ describe('parallel tool execution', () => { describe('AiRegistry store on globalThis', () => { it('state lives on globalThis so it survives a second copy of @gemstack/ai-sdk', () => { // Vite-bundled server apps inline `@gemstack/ai-sdk` (every agent path reads - // `AiRegistry.resolve(...)`) into entry.mjs, but `AiProvider.boot()` - // (loaded via `@gemstack/ai-sdk/server`) runs from a node_modules copy - // resolved via the provider auto-discovery manifest. Without a + // `AiRegistry.resolve(...)`) into entry.mjs, but provider registration can + // run from a separate node_modules copy resolved at boot. Without a // globalThis-routed store, factories registered from the externalized // copy would never be visible to agent resolution from inside the bundle // and every call would throw "Unknown AI provider". diff --git a/packages/ai-sdk/src/isomorphic-check.test.ts b/packages/ai-sdk/src/isomorphic-check.test.ts index b042bd2..e7fa831 100644 --- a/packages/ai-sdk/src/isomorphic-check.test.ts +++ b/packages/ai-sdk/src/isomorphic-check.test.ts @@ -10,14 +10,14 @@ const distDir = join(__dirname, '..', 'dist') /** * Files that must have ZERO Node-only static imports — these compose the runtime-agnostic - * main entry of @gemstack/ai-sdk. Anything in src/node/ or src/server/ is exempt. + * main entry of @gemstack/ai-sdk. Anything in src/node/ is exempt. */ const NODE_IMPORT_RE = /from ['"](node:|fs|path|os|crypto|child_process|fs\/promises|stream|buffer)['"]/ function listFiles(dir: string, prefix = ''): string[] { const out: string[] = [] for (const name of readdirSync(dir, { withFileTypes: true })) { - if (name.name === 'node' || name.name === 'server') continue + if (name.name === 'node') continue if (name.name.endsWith('.d.ts') || name.name.endsWith('.map')) continue const full = join(dir, name.name) const rel = prefix ? `${prefix}/${name.name}` : name.name @@ -49,7 +49,6 @@ test('main entry has no Node-only imports', () => { ) }) -test('/node and /server subpaths exist', () => { - assert.ok(existsSync(join(distDir, 'node', 'index.js')), 'dist/node/index.js missing') - assert.ok(existsSync(join(distDir, 'server', 'index.js')), 'dist/server/index.js missing') +test('/node subpath exists', () => { + assert.ok(existsSync(join(distDir, 'node', 'index.js')), 'dist/node/index.js missing') }) diff --git a/packages/ai-sdk/src/observers.ts b/packages/ai-sdk/src/observers.ts index e94d9f1..18db1ac 100644 --- a/packages/ai-sdk/src/observers.ts +++ b/packages/ai-sdk/src/observers.ts @@ -3,8 +3,8 @@ * made through `@gemstack/ai-sdk`. Any package can subscribe to be notified * about completed or failed agent runs. * - * Used today by `@rudderjs/telescope`'s AiCollector to record agent - * executions into the dashboard. The registry is defined here (inside + * Used by an observability collector to record agent + * executions into a dashboard. The registry is defined here (inside * `@gemstack/ai-sdk`) so the observer contract lives with the package that * owns the AI abstraction. */ @@ -132,7 +132,7 @@ export class AiObserverRegistry { reset(): void { this.observers = [] } } -// Process-wide singleton, like `httpObservers` in `@rudderjs/http`. +// Process-wide singleton. const _g = globalThis as Record<string, unknown> if (!_g['__rudderjs_ai_observers__']) { _g['__rudderjs_ai_observers__'] = new AiObserverRegistry() diff --git a/packages/ai-sdk/src/react/agent-run.ts b/packages/ai-sdk/src/react/agent-run.ts index 9e90c34..c7b2b1a 100644 --- a/packages/ai-sdk/src/react/agent-run.ts +++ b/packages/ai-sdk/src/react/agent-run.ts @@ -1,8 +1,7 @@ /** * Framework-free core of the `useAgentRun` React hook. * - * The React hook (`useAgentRun.ts`) is a thin wrapper around the pieces here — - * same posture as `@rudderjs/sync`'s `CollabRoomManager` vs `useCollabRoom`: + * The React hook (`useAgentRun.ts`) is a thin wrapper around the pieces here: * keeping the state machine and the stream-driving loop out of React lets us * exhaustively unit-test the client-tool round-trip and the approval resume * without a React testing harness (the framework intentionally ships none). diff --git a/packages/ai-sdk/src/react/index.ts b/packages/ai-sdk/src/react/index.ts index 75691e2..51edcef 100644 --- a/packages/ai-sdk/src/react/index.ts +++ b/packages/ai-sdk/src/react/index.ts @@ -4,8 +4,8 @@ * The agent framework, providers, and the runtime-agnostic agent-SSE protocol * (`readAgentStream` + the server framers) live in the main `@gemstack/ai-sdk` * entry. This subpath adds the React hook that drives a streamed agent run from - * a component — same split as `@rudderjs/sync/react` (the main entry stays - * React-free; React lives behind `/react`). + * a component — the same split where the main entry stays + * React-free and React lives behind `/react`. * * Peer requirement: `react@>=19.2.0`. */ diff --git a/packages/ai-sdk/src/react/useAgentRun.ts b/packages/ai-sdk/src/react/useAgentRun.ts index 3486b2d..c60134c 100644 --- a/packages/ai-sdk/src/react/useAgentRun.ts +++ b/packages/ai-sdk/src/react/useAgentRun.ts @@ -71,7 +71,7 @@ export interface UseAgentRunResult<TInput = unknown> { * approval requests with imperative `run` / `respond` / `approve` / `reject`. * * Lives behind the `@gemstack/ai-sdk/react` subpath so the main entry stays - * runtime-agnostic (same split as `@rudderjs/sync/react`). + * runtime-agnostic. * * @example * const { status, outputs, run } = useAgentRun({ diff --git a/packages/ai-sdk/src/registry.ts b/packages/ai-sdk/src/registry.ts index 3aecc14..0efc088 100644 --- a/packages/ai-sdk/src/registry.ts +++ b/packages/ai-sdk/src/registry.ts @@ -47,7 +47,7 @@ export async function tryWithFailover<T>( * throw "Unknown AI provider". * * Defensive migration per the #499 static-state singleton audit. Same pattern - * as PR #498 (`@rudderjs/orm` `ModelRegistry`), #500–#505 (pennant, cache, + * applied to other process-wide registries (model registry, pennant, cache, * queue, mail, storage, hash). */ interface AiRegistryStore { @@ -147,7 +147,7 @@ export class AiRegistry { throw new Error( `[ai-sdk] Provider "${providerName}" does not support hosted vector stores. ` + `Use a provider that implements createVectorStores() (e.g. openai). ` + - `For self-hosted RAG, use similaritySearch() against an @rudderjs/orm Model with a pgvector column.`, + `For self-hosted RAG, use similaritySearch() against an ORM Model with a pgvector column.`, ) } return factory.createVectorStores() diff --git a/packages/ai-sdk/src/server/index.ts b/packages/ai-sdk/src/server/index.ts deleted file mode 100644 index 1fede43..0000000 --- a/packages/ai-sdk/src/server/index.ts +++ /dev/null @@ -1 +0,0 @@ -export { AiProvider } from './provider.js' diff --git a/packages/ai-sdk/src/server/provider.ts b/packages/ai-sdk/src/server/provider.ts deleted file mode 100644 index 458858d..0000000 --- a/packages/ai-sdk/src/server/provider.ts +++ /dev/null @@ -1,185 +0,0 @@ -import { ServiceProvider, config, bootNotice } from '@rudderjs/core' -import { AiRegistry } from '../registry.js' -import { setConversationStore, setUserMemory } from '../agent.js' -import { GoogleCacheRegistry, type CacheStoreLike } from '../providers/google-cache-registry.js' -import type { AiConfig, AiProviderConfig, ProviderFactory } from '../types.js' - -/** - * Return the configured `apiKey`, or `null` when missing/empty. - * - * The config type lets `apiKey` be undefined (some drivers — ollama, bedrock — - * don't need one), so apiKey-requiring drivers use this gate. When the gate - * returns `null` the driver factory bails to `null` and `AiProvider.boot()` - * skips the provider with a warning instead of crashing — matches Laravel's - * "drivers as data, missing credentials don't kill the framework" pattern. - * - * Use-site (`AI.use('anthropic')`) will surface the standard - * "provider not registered" error so debugging stays actionable. - */ -function requireKey(_name: string, cfg: AiProviderConfig): string | null { - return cfg.apiKey || null -} - -type DriverDeps = { googleCacheRegistry: GoogleCacheRegistry } -type DriverBuilder = (name: string, cfg: AiProviderConfig, deps: DriverDeps) => Promise<ProviderFactory | null> - -const DRIVERS: Record<string, DriverBuilder> = { - anthropic: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { AnthropicProvider } = await import('../providers/anthropic.js') - return new AnthropicProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - openai: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { OpenAIProvider } = await import('../providers/openai.js') - return new OpenAIProvider({ - apiKey, - baseUrl: cfg.baseUrl, - organization: cfg['organization'] as string | undefined, - }) - }, - google: async (name, cfg, { googleCacheRegistry }) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { GoogleProvider } = await import('../providers/google.js') - return new GoogleProvider({ apiKey }, googleCacheRegistry) - }, - ollama: async (_name, cfg) => { - const { OllamaProvider } = await import('../providers/ollama.js') - return new OllamaProvider({ baseUrl: cfg.baseUrl }) - }, - deepseek: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { DeepSeekProvider } = await import('../providers/deepseek.js') - return new DeepSeekProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - xai: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { XaiProvider } = await import('../providers/xai.js') - return new XaiProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - groq: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { GroqProvider } = await import('../providers/groq.js') - return new GroqProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - mistral: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { MistralProvider } = await import('../providers/mistral.js') - return new MistralProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - azure: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - if (!cfg.baseUrl) { - throw new Error(`[ai-sdk] config('ai').providers.${name} is missing baseUrl (driver "azure" requires it).`) - } - const { AzureOpenAIProvider } = await import('../providers/azure.js') - return new AzureOpenAIProvider({ apiKey, baseUrl: cfg.baseUrl }) - }, - openrouter: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { OpenRouterProvider } = await import('../providers/openrouter.js') - return new OpenRouterProvider({ - apiKey, - baseUrl: cfg.baseUrl, - siteUrl: cfg['siteUrl'] as string | undefined, - siteName: cfg['siteName'] as string | undefined, - }) - }, - bedrock: async (_name, cfg) => { - const { BedrockProvider } = await import('../providers/bedrock.js') - const region = (cfg['region'] as string | undefined) ?? 'us-east-1' - const credentials = cfg['credentials'] as - | { accessKeyId: string; secretAccessKey: string; sessionToken?: string } - | undefined - return new BedrockProvider(credentials ? { region, credentials } : { region }) - }, - elevenlabs: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { ElevenLabsProvider } = await import('../providers/elevenlabs.js') - return new ElevenLabsProvider({ - apiKey, - ...(cfg.baseUrl ? { baseUrl: cfg.baseUrl } : {}), - ...(cfg['defaultTtsModelId'] ? { defaultTtsModelId: cfg['defaultTtsModelId'] as string } : {}), - }) - }, - voyage: async (name, cfg) => { - const apiKey = requireKey(name, cfg); if (apiKey === null) return null - const { VoyageProvider } = await import('../providers/voyage.js') - return new VoyageProvider({ - apiKey, - ...(cfg.baseUrl ? { baseUrl: cfg.baseUrl } : {}), - ...(cfg['defaultInputType'] ? { defaultInputType: cfg['defaultInputType'] as 'query' | 'document' } : {}), - }) - }, -} - -/** - * AI ServiceProvider — reads config from `config('ai')`. - * - * @example - * // bootstrap/providers.ts - * import { AiProvider } from '@gemstack/ai-sdk/server' - * export default [AiProvider, ...] - */ -export class AiProvider extends ServiceProvider { - register(): void {} - - async boot(): Promise<void> { - const cfg = config<AiConfig>('ai') - const googleCacheRegistry = this.buildGoogleCacheRegistry() - - for (const [name, providerConfig] of Object.entries(cfg.providers)) { - const driver = providerConfig.driver ?? name - const build = DRIVERS[driver] - if (!build) continue - const instance = await build(name, providerConfig, { googleCacheRegistry }) - if (instance === null) { - // Drivers that need an apiKey return null when it's missing/empty - // (see requireKey). Skip with a grouped boot notice so the app boots - // and `AI.use('${name}')` surfaces the standard "not registered" - // error at the use-site with a clear hint to set the env var. - bootNotice('ai', `${name} skipped, no API key (set it in .env)`) - continue - } - AiRegistry.register(instance) - } - - AiRegistry.setDefault(cfg.default) - AiRegistry.setModels(cfg.models ?? []) - this.app.instance('ai.registry', AiRegistry) - - // Register conversation store if provided in config - if (cfg.conversations) { - setConversationStore(cfg.conversations) - this.app.instance('ai.conversations', cfg.conversations) - } - - // Register user-memory store if provided in config (#A4) - if (cfg.memory) { - setUserMemory(cfg.memory) - this.app.instance('ai.memory', cfg.memory) - } - - // Register make:agent scaffolder - try { - const { registerMakeSpecs } = await import('@rudderjs/console') - const { makeAgentSpec } = await import('../commands/make-agent.js') - registerMakeSpecs(makeAgentSpec) - } catch { /* rudder not available */ } - } - - /** - * Build a `GoogleCacheRegistry` for Gemini's `cachedContent` resources. - * When `@rudderjs/cache` is installed and booted, the registered cache - * adapter is plumbed in for cross-process / cross-restart persistence. - * Otherwise the registry uses an in-process `Map` and warns once on - * first use. - */ - private buildGoogleCacheRegistry(): GoogleCacheRegistry { - if (this.app.container.has('cache')) { - const store = this.app.make<CacheStoreLike>('cache') - return new GoogleCacheRegistry({ store }) - } - return new GoogleCacheRegistry() - } -} diff --git a/packages/ai-sdk/src/similarity-search.ts b/packages/ai-sdk/src/similarity-search.ts index ae4617a..a53df83 100644 --- a/packages/ai-sdk/src/similarity-search.ts +++ b/packages/ai-sdk/src/similarity-search.ts @@ -1,6 +1,6 @@ /** * `similaritySearch({ model, column, embedWith, ... })` — agent-tool - * factory that wraps an `@rudderjs/orm` Model + a vector column into a + * factory that wraps an ORM Model + a vector column into a * drop-in `Tool` an agent can call to retrieve semantically similar * rows (#B7 Phase 2). * @@ -49,9 +49,9 @@ import type { ServerToolBuilder } from './tool.js' /** * Structural type for the model class similaritySearch accepts. * - * Declared locally instead of importing `Model` from `@rudderjs/orm` - * so the main entry stays free of orm runtime — the tool calls - * `model.query()` and never references the `@rudderjs/orm` package + * Declared locally instead of importing an ORM `Model` type + * so the main entry stays free of any ORM runtime — the tool calls + * `model.query()` and never references an ORM package * itself. The user's app brings its own Model class. */ export interface SimilaritySearchModel<TInstance> { @@ -61,8 +61,8 @@ export interface SimilaritySearchModel<TInstance> { /** * WhereOperator strings the `scope` callback may pass to `.where()`. - * Mirrors `@rudderjs/contracts`'s `WhereOperator`. Duplicated here so the - * main entry has no compile-time `@rudderjs/contracts` dep. + * Mirrors a query builder's where-operator set. Duplicated here so the + * main entry has no compile-time query-builder dep. */ export type SimilaritySearchWhereOperator = | '=' | '!=' | '>' | '>=' | '<' | '<=' @@ -71,7 +71,7 @@ export type SimilaritySearchWhereOperator = /** * Structural type for the QueryBuilder methods similaritySearch needs. - * Mirrors a subset of `@rudderjs/contracts`'s `QueryBuilder<T>` so apps + * Mirrors a subset of a query builder's `QueryBuilder<T>` so apps * writing a `scope` callback get autocomplete on the methods that actually * compose with `whereVectorSimilarTo` (#B7 Phase 2.5): * @@ -240,7 +240,7 @@ export function similaritySearch<TInstance>( if (typeof whereVec !== 'function' || typeof selectDist !== 'function') { throw new Error( `[ai-sdk] similaritySearch: ${model.name}'s ORM adapter does not implement vector queries. ` + - 'Use @rudderjs/orm-prisma against a Postgres + pgvector connection.', + 'Use a Postgres + pgvector connection.', ) } diff --git a/packages/ai-sdk/src/types.ts b/packages/ai-sdk/src/types.ts index c4f1eca..8fbea53 100644 --- a/packages/ai-sdk/src/types.ts +++ b/packages/ai-sdk/src/types.ts @@ -198,9 +198,8 @@ export interface CacheableConfig { */ messages?: number /** - * How long the cache entry should live. Duration string accepted by - * `@rudderjs/support`'s parser — `'30m'`, `'2h'`, `'1d'`, etc. Default - * `'1h'` when omitted. + * How long the cache entry should live. A duration string (`'30m'`, + * `'2h'`, `'1d'`, etc.). Default `'1h'` when omitted. * * **Google-only for now.** Anthropic's ephemeral cache and OpenAI's * automatic prefix cache have no per-call TTL knob; their adapters ignore @@ -624,8 +623,8 @@ export interface ServerTool<TInput = unknown, TOutput = unknown> extends Tool<TI /** * @deprecated Use {@link Tool}. A "client tool" is just a `Tool` whose - * `execute` is omitted; the browser handles execution via the - * `clientTools` registry in `@rudderjs/panels`. + * `execute` is omitted; the browser handles execution via a + * client-side tools registry. */ export type ClientTool<TInput = unknown, TOutput = unknown> = Tool<TInput, TOutput> diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 7118183..1158c0c 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -90,12 +90,6 @@ importers: specifier: ^4.0.0 version: 4.4.3 devDependencies: - '@rudderjs/console': - specifier: ^1.4.3 - version: 1.4.3 - '@rudderjs/core': - specifier: ^1.13.3 - version: 1.13.3 '@types/node': specifier: ^20.0.0 version: 20.19.43 @@ -334,14 +328,6 @@ packages: '@changesets/write@0.4.0': resolution: {integrity: sha512-CdTLvIOPiCNuH71pyDu3rA+Q0n65cmAbXnwWH84rKGiFumFzkmHNT8KHTMEchcxN+Kl8I54xGUhJ7l3E7X396Q==} - '@clack/core@1.4.2': - resolution: {integrity: sha512-0Ty/1Gfm+Kb07sXcuESjyKfwEhSy4Ns1AgeEisHb/bDY5fWme0tTeTkU14T1Gmcs17YIjB/teiDe4uaCghbYqQ==} - engines: {node: '>= 20.12.0'} - - '@clack/prompts@1.6.0': - resolution: {integrity: sha512-EYlRokl8szrP9Z25qT5aepMdBjzBvHF9ZEhzIiUBc9guz/T31EqRgvD0QSgZcpE93xiwrr+OkB4nz0BZyF6fSA==} - engines: {node: '>= 20.12.0'} - '@esbuild/aix-ppc64@0.28.1': resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==} engines: {node: '>=18'} @@ -577,26 +563,6 @@ packages: '@protobufjs/utf8@1.1.1': resolution: {integrity: sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==} - '@rudderjs/console@1.4.3': - resolution: {integrity: sha512-tCNCW6YlXr73MMWIeIEMc174MlZcpiSyez4X0SsBAO3vI04id11N2GouyT46mdlwuP2to4oOHNXZK8JQMcdJaQ==} - engines: {node: '>=22.12.0'} - - '@rudderjs/contracts@1.19.0': - resolution: {integrity: sha512-I9djlPm7os3n4Bxv8BUY7tbrWmj1bN+GDVWzDznZUc/FFIZN4LcfspNKrwddlkh+gdvVJToChRYEKMWZ5IvDgQ==} - engines: {node: '>=22.12.0'} - - '@rudderjs/core@1.13.3': - resolution: {integrity: sha512-hBJJsucZnr8xFGMCpsz77UwgxpRPDYb/m5iVvkfk5ywBh3kcIiUufT9b1hwPabQn+erDdH3hNtoH4KUT4ckStg==} - engines: {node: '>=22.12.0'} - - '@rudderjs/router@1.9.2': - resolution: {integrity: sha512-h+DMjZhbglyyHu00FuKxE4BA2jiFFUjkb0s/ue1uM2LijeOd4Y8rhAs4e91fjPuVik6Tc86ElHkAc0ygqD6cCQ==} - engines: {node: '>=22.12.0'} - - '@rudderjs/support@1.6.3': - resolution: {integrity: sha512-a1WjjyvA6FI7ZMWD4Grg1FXMKGqJOKl6ObM79SmpAMYl0NFp6KPP8ERhp8GMunhLe8AzIhucaXi4Drr5kS/YWQ==} - engines: {node: '>=22.12.0'} - '@smithy/core@3.26.0': resolution: {integrity: sha512-mLUktFAn+Pa2agl1J7VgtYNFWCX8/b4GMJSK1hCu4YCvtBfM6F8Os3EP4ry+DFFlXOf3wyvlgXhuUdFoy52D3g==} engines: {node: '>=18.0.0'} @@ -943,18 +909,9 @@ packages: fast-sha256@1.3.0: resolution: {integrity: sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==} - fast-string-truncated-width@3.0.3: - resolution: {integrity: sha512-0jjjIEL6+0jag3l2XWWizO64/aZVtpiGE3t0Zgqxv0DPuxiMjvB3M24fCyhZUO4KomJQPj3LTSUnDP3GpdwC0g==} - - fast-string-width@3.0.2: - resolution: {integrity: sha512-gX8LrtNEI5hq8DVUfRQMbr5lpaS4nMIWV+7XEbXk2b8kiQIizgnlr12B4dA3ZEx3308ze0O4Q1R+cHts8kyUJg==} - fast-uri@3.1.2: resolution: {integrity: sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==} - fast-wrap-ansi@0.2.2: - resolution: {integrity: sha512-7F2Fl+TjRSenLqlU3UjSH0iyqopqoZIu7eZVpEirP2g1GtWa2G/ecEmBdgz31+Mxr+ELclgg6sokpSFIQiZ02Q==} - fastq@1.20.1: resolution: {integrity: sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw==} @@ -1444,9 +1401,6 @@ packages: resolution: {integrity: sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==} engines: {node: '>=14'} - sisteransi@1.0.5: - resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} - slash@3.0.0: resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==} engines: {node: '>=8'} @@ -1970,18 +1924,6 @@ snapshots: human-id: 4.2.0 prettier: 2.8.8 - '@clack/core@1.4.2': - dependencies: - fast-wrap-ansi: 0.2.2 - sisteransi: 1.0.5 - - '@clack/prompts@1.6.0': - dependencies: - '@clack/core': 1.4.2 - fast-string-width: 3.0.2 - fast-wrap-ansi: 0.2.2 - sisteransi: 1.0.5 - '@esbuild/aix-ppc64@0.28.1': optional: true @@ -2164,30 +2106,6 @@ snapshots: '@protobufjs/utf8@1.1.1': optional: true - '@rudderjs/console@1.4.3': - dependencies: - '@clack/prompts': 1.6.0 - - '@rudderjs/contracts@1.19.0': {} - - '@rudderjs/core@1.13.3': - dependencies: - '@rudderjs/console': 1.4.3 - '@rudderjs/contracts': 1.19.0 - '@rudderjs/router': 1.9.2 - '@rudderjs/support': 1.6.3 - reflect-metadata: 0.2.2 - zod: 4.4.3 - - '@rudderjs/router@1.9.2': - dependencies: - '@rudderjs/contracts': 1.19.0 - reflect-metadata: 0.2.2 - - '@rudderjs/support@1.6.3': - dependencies: - zod: 4.4.3 - '@smithy/core@3.26.0': dependencies: '@aws-crypto/crc32': 5.2.0 @@ -2575,18 +2493,8 @@ snapshots: fast-sha256@1.3.0: optional: true - fast-string-truncated-width@3.0.3: {} - - fast-string-width@3.0.2: - dependencies: - fast-string-truncated-width: 3.0.3 - fast-uri@3.1.2: {} - fast-wrap-ansi@0.2.2: - dependencies: - fast-string-width: 3.0.2 - fastq@1.20.1: dependencies: reusify: 1.1.0 @@ -3107,8 +3015,6 @@ snapshots: signal-exit@4.1.0: {} - sisteransi@1.0.5: {} - slash@3.0.0: {} spawndamnit@3.0.1: