feat(embedding): add strategy-based fallback engine with provenance

README.md

@@ -90,9 +90,13 @@ Environment variables (BYOK):
 - `DUBSBOT_ANTHROPIC_MODEL`
 - `DUBSBOT_GOOGLE_MODEL` (defaults to `gemini-3.1-pro-preview`)
 - `DUBSBOT_OTEL_ENABLED=1` to enable telemetry export hooks
+- `DUBSBOT_EMBEDDING_STRATEGY_V2=1` to enable explicit embedding strategy resolution/fallback
+- `DUBSBOT_EMBEDDING_STRATEGY_CONFIG_JSON` to provide explicit strategy config
+- `DUBSBOT_EMBEDDING_PROVENANCE_LOG=1` to emit embedding provenance log lines
 
 ## Notes
 
 - Anthropic embeddings currently fall back to deterministic local vectors.
 - This project intentionally uses Biome only (no ESLint/Prettier).
 - Retrieval proofing benchmark schema/workflow docs: `docs/retrieval-proofing-benchmark-schema.md` and `docs/retrieval-proofing.md`.
+- Embedding strategy rollout guide: `docs/embedding-strategy-rollout.md`.

docs/embedding-strategy-rollout.md

@@ -0,0 +1,31 @@
+# Embedding Strategy V2 Rollout
+
+This rollout gates the explicit embedding strategy engine behind:
+
+- `DUBSBOT_EMBEDDING_STRATEGY_V2=1`
+
+Optional config override:
+
+- `DUBSBOT_EMBEDDING_STRATEGY_CONFIG_JSON` (JSON string matching schema version `1.0`)
+
+Optional provenance logging:
+
+- `DUBSBOT_EMBEDDING_PROVENANCE_LOG=1`
+
+## Enable (staged)
+
+1. Set `DUBSBOT_EMBEDDING_STRATEGY_V2=1` in a non-production environment.
+2. Start with default legacy-mapped config (no custom JSON).
+3. Run indexing and retrieval checks.
+4. If needed, provide explicit strategy JSON to control Anthropic fallback paths.
+5. Verify fallback/provenance behavior with tests:
+   - `pnpm test -- embedding-strategy`
+
+## Rollback
+
+1. Unset or set `DUBSBOT_EMBEDDING_STRATEGY_V2=0`.
+2. Restart CLI/daemon processes.
+3. System returns to legacy embedding execution path.
+
+Rollback is safe because provenance fields are additive and read-compatible.
+

openspec/changes/embedding-parity-hardening/tasks.md → openspec/changes/archive/2026-03-04-embedding-parity-hardening/tasks.md

@@ -1,24 +1,24 @@
 ## 1. Strategy Configuration Foundation
 
-- [ ] 1.1 Introduce typed `embeddingStrategy` config schema with provider/model primary and ordered fallback entries
-- [ ] 1.2 Add startup validation for unknown providers, missing models, and cyclic fallback paths with structured errors
-- [ ] 1.3 Add backward-compatible default mapping from legacy embedding settings to explicit strategy definitions
+- [x] 1.1 Introduce typed `embeddingStrategy` config schema with provider/model primary and ordered fallback entries
+- [x] 1.2 Add startup validation for unknown providers, missing models, and cyclic fallback paths with structured errors
+- [x] 1.3 Add backward-compatible default mapping from legacy embedding settings to explicit strategy definitions
 
 ## 2. Runtime Strategy Resolution and Anthropic Policy
 
-- [ ] 2.1 Implement deterministic strategy resolver that requires a valid strategy id for each embedding request
-- [ ] 2.2 Implement Anthropic native-first execution path with explicit fallback eligibility based on configured failure categories
-- [ ] 2.3 Enforce configured fallback order and terminal failure behavior when fallback is disallowed or exhausted
+- [x] 2.1 Implement deterministic strategy resolver that requires a valid strategy id for each embedding request
+- [x] 2.2 Implement Anthropic native-first execution path with explicit fallback eligibility based on configured failure categories
+- [x] 2.3 Enforce configured fallback order and terminal failure behavior when fallback is disallowed or exhausted
 
 ## 3. Provenance Envelope and Data Plumbing
 
-- [ ] 3.1 Define a normalized embedding result envelope including strategy id, provider/model attempt path, fallback state, and failure category
-- [ ] 3.2 Propagate provenance fields through indexing writes and retrieval/query responses
-- [ ] 3.3 Update logging/metrics hooks to include provenance identifiers for debugging and parity analysis
+- [x] 3.1 Define a normalized embedding result envelope including strategy id, provider/model attempt path, fallback state, and failure category
+- [x] 3.2 Propagate provenance fields through indexing writes and retrieval/query responses
+- [x] 3.3 Update logging/metrics hooks to include provenance identifiers for debugging and parity analysis
 
 ## 4. Verification and Rollout Safety
 
-- [ ] 4.1 Add conformance tests for valid/invalid strategy config loading and runtime resolution behavior
-- [ ] 4.2 Add Anthropic policy tests covering success, non-fallbackable failures, fallbackable failures, and no-fallback scenarios
-- [ ] 4.3 Add provenance completeness tests for both successful and terminal-failure embedding outcomes
-- [ ] 4.4 Gate rollout behind a feature flag and document enable/rollback procedure for staged deployment
+- [x] 4.1 Add conformance tests for valid/invalid strategy config loading and runtime resolution behavior
+- [x] 4.2 Add Anthropic policy tests covering success, non-fallbackable failures, fallbackable failures, and no-fallback scenarios
+- [x] 4.3 Add provenance completeness tests for both successful and terminal-failure embedding outcomes
+- [x] 4.4 Gate rollout behind a feature flag and document enable/rollback procedure for staged deployment

openspec/specs/anthropic-embedding-fallback-and-provenance/spec.md

@@ -0,0 +1,39 @@
+# anthropic-embedding-fallback-and-provenance Specification
+
+## Purpose
+Define expected Anthropic-primary embedding behavior, including native-first execution,
+failure-category-gated fallback sequencing, and required provenance metadata for both successful
+embedding results and terminal failures.
+## Requirements
+### Requirement: Anthropic Native-First Execution Policy
+For strategies configured with Anthropic as primary, the system SHALL attempt Anthropic native embedding first and SHALL only consider fallback providers explicitly listed in that strategy.
+
+#### Scenario: Anthropic primary succeeds
+- **WHEN** a request resolves to a strategy with Anthropic as primary and Anthropic returns embeddings successfully
+- **THEN** the system returns the Anthropic embedding result without invoking fallback providers
+
+#### Scenario: Anthropic primary fails with non-fallbackable error
+- **WHEN** Anthropic returns an error outside configured fallbackable categories
+- **THEN** the system returns a terminal embedding error and MUST NOT invoke fallback providers
+
+### Requirement: Controlled Anthropic Fallback Behavior
+The system SHALL invoke fallback providers for Anthropic strategies only for configured fallbackable failure categories and in configured fallback order.
+
+#### Scenario: Fallback is invoked in configured order
+- **WHEN** Anthropic primary fails with a fallbackable error category and fallback providers are configured
+- **THEN** the system attempts fallback providers sequentially in strategy order until one succeeds or all fail
+
+#### Scenario: No fallback configured
+- **WHEN** Anthropic primary fails with a fallbackable error category but no fallback providers are configured
+- **THEN** the system returns a structured failure indicating no fallback path was available
+
+### Requirement: Embedding Provenance Metadata
+The system SHALL attach provenance metadata to every embedding result and terminal failure outcome, including strategy id, attempt provider/model path, and fallback usage state.
+
+#### Scenario: Provenance is emitted on success
+- **WHEN** any provider successfully returns embeddings
+- **THEN** the result includes provenance fields for strategy id, resolved provider/model, attempt path, and whether fallback was used
+
+#### Scenario: Provenance is emitted on terminal failure
+- **WHEN** all attempts fail or fallback is disallowed
+- **THEN** the error payload includes provenance fields for attempted providers/models, failure category, and terminal resolution reason

openspec/specs/embedding-strategy-configuration/spec.md

@@ -0,0 +1,28 @@
+# embedding-strategy-configuration Specification
+
+## Purpose
+Define how embedding strategies are configured and resolved across providers and models,
+including named strategy IDs, primary provider/model selection, ordered fallback chains, and
+deterministic runtime resolution with startup validation of invalid or inconsistent configurations.
+## Requirements
+### Requirement: Provider-Configurable Embedding Strategy
+The system SHALL support explicit embedding strategy configuration per embedding use-case, including primary provider/model selection and an ordered fallback list.
+
+#### Scenario: Valid strategy is loaded
+- **WHEN** the service starts with a strategy configuration where each strategy has a primary provider/model and valid fallback entries
+- **THEN** the system initializes successfully and registers the strategy for runtime resolution
+
+#### Scenario: Invalid strategy is rejected
+- **WHEN** the service starts with a strategy configuration that references an unknown provider, missing model, or cyclic fallback path
+- **THEN** the system MUST fail validation and return a configuration error that identifies the invalid strategy entry
+
+### Requirement: Deterministic Runtime Strategy Resolution
+The system SHALL resolve embedding strategies deterministically for each request using the configured strategy identifier and SHALL NOT use implicit provider defaults.
+
+#### Scenario: Strategy id resolves to configured primary
+- **WHEN** an embedding request specifies a known strategy id
+- **THEN** the system uses the configured primary provider/model for the first execution attempt
+
+#### Scenario: Unknown strategy id is rejected
+- **WHEN** an embedding request specifies a strategy id not present in configuration
+- **THEN** the system returns a structured error and MUST NOT attempt embedding generation

src/cli/runtime.ts

@@ -1,5 +1,6 @@
 import { AgentOrchestrator } from '../agent/orchestrator';
 import { loadAgentsConfig } from '../config/agents-loader';
+import { loadEmbeddingStrategyConfig } from '../context/embedding/config';
 import { createDb } from '../db/client';
 import { runMigrations } from '../db/migrate';
 import { OptionalOtelExporter } from '../observability/otel';
@@ -13,6 +14,7 @@ import { ToolRegistry } from '../tools/registry';
 export async function createRuntime() {
   await runMigrations();
   const db = await createDb();
+  const embeddingStrategyConfig = loadEmbeddingStrategyConfig();
   const agentsConfig = await loadAgentsConfig(process.cwd());
   const provider = createProviderAdapter(detectProvider());
   const policyEngine = new DefaultPolicyEngine(createDefaultApprovalPolicy());
@@ -24,6 +26,7 @@ export async function createRuntime() {
 
   return {
     db,
+    embeddingStrategyConfig,
     provider,
     policyEngine,
     orchestrator,

src/context/embedding/config.ts

@@ -0,0 +1,93 @@
+import { detectProvider, type ProviderName } from '../../providers';
+import {
+  type EmbeddingStrategyConfig,
+  EmbeddingStrategyConfigError,
+  parseEmbeddingStrategyConfig,
+} from './strategy';
+
+export function loadEmbeddingStrategyConfig(): EmbeddingStrategyConfig {
+  const rawFromEnv = process.env.DUBSBOT_EMBEDDING_STRATEGY_CONFIG_JSON;
+  const raw = rawFromEnv ? parseJsonConfigFromEnv(rawFromEnv) : buildLegacyDefaultConfig();
+  const parsed = parseEmbeddingStrategyConfig(raw);
+  if (!parsed.config) {
+    throw new EmbeddingStrategyConfigError(parsed.issues);
+  }
+  return parsed.config;
+}
+
+export function isEmbeddingStrategyV2Enabled(): boolean {
+  return process.env.DUBSBOT_EMBEDDING_STRATEGY_V2 === '1';
+}
+
+function buildLegacyDefaultConfig(): EmbeddingStrategyConfig {
+  const primaryProvider = detectProvider();
+  const primary = toPrimaryStrategy(primaryProvider, 'default-primary');
+  const strategies = [primary];
+
+  if (primaryProvider === 'anthropic') {
+    strategies.push(toPrimaryStrategy('openai', 'fallback-openai'));
+    strategies.push(toPrimaryStrategy('google', 'fallback-google'));
+    primary.fallback = [
+      {
+        strategyId: 'fallback-openai',
+        onFailure: ['rate_limit', 'timeout', 'service_unavailable'],
+      },
+      {
+        strategyId: 'fallback-google',
+        onFailure: ['rate_limit', 'timeout', 'service_unavailable'],
+      },
+    ];
+  }
+
+  return {
+    version: '1.0',
+    defaults: {
+      indexing: 'default-primary',
+      query: 'default-primary',
+    },
+    strategies,
+  };
+}
+
+function toPrimaryStrategy(provider: ProviderName, id: string) {
+  return {
+    id,
+    provider,
+    model: defaultEmbeddingModel(provider),
+    fallback: [] as Array<{
+      strategyId: string;
+      onFailure: Array<
+        'rate_limit' | 'timeout' | 'service_unavailable' | 'auth' | 'invalid_request' | 'unknown'
+      >;
+    }>,
+  };
+}
+
+function defaultEmbeddingModel(provider: ProviderName): string {
+  switch (provider) {
+    case 'openai':
+      return process.env.DUBSBOT_OPENAI_EMBEDDING_MODEL ?? 'text-embedding-3-small';
+    case 'google':
+      return process.env.DUBSBOT_GOOGLE_EMBEDDING_MODEL ?? 'text-embedding-004';
+    case 'anthropic':
+      return process.env.DUBSBOT_ANTHROPIC_EMBEDDING_MODEL ?? 'local-deterministic';
+    default:
+      return 'local-deterministic';
+  }
+}
+
+function parseJsonConfigFromEnv(rawFromEnv: string): unknown {
+  try {
+    return JSON.parse(rawFromEnv);
+  } catch (error) {
+    if (error instanceof SyntaxError) {
+      throw new EmbeddingStrategyConfigError([
+        {
+          code: 'schema_invalid',
+          detail: `DUBSBOT_EMBEDDING_STRATEGY_CONFIG_JSON is invalid JSON: ${error.message}`,
+        },
+      ]);
+    }
+    throw error;
+  }
+}