Summary
router.stats.baselineModel is parsed, but createRouterRuntime overrides it with router.scenarios.default whenever a default scenario exists.
Why it matters
The purpose of router.stats.baselineModel is to define the no-router baseline used for saved-cost and baseline-cost calculations. Most router configs have a default scenario, so the explicit baseline is silently ignored in normal use. The dashboard or stats output can then report savings against the routed default model instead of the user-selected baseline model.
Evidence
src/router/config/schema.ts:61 documents baselineModel as the provider/model ref used for saved-cost baseline calculation.
src/router/config/parseRouterConfig.ts:491 enters parseStats.
src/router/config/parseRouterConfig.ts:532 parses raw.baselineModel.
src/router/config/parseRouterConfig.ts:538 returns the parsed baselineModel.
src/router/RouterRuntime.ts:90 constructs TokenStatsCollector.
src/router/RouterRuntime.ts:93 prefers config.scenarios.default as the collector baseline whenever a default scenario exists.
src/router/RouterRuntime.ts:95 only falls back to config.stats?.baselineModel when there is no default scenario.
src/router/stats/TokenStatsCollector.ts:68 stores the collector baseline from config?.baselineModel.
src/router/stats/TokenStatsCollector.ts:317 uses that baseline during baseline-cost calculation.
src/router/stats/TokenStatsCollector.ts:321 selects the baseline provider.
src/router/stats/TokenStatsCollector.ts:322 selects the baseline model.
Validation
Validation level: dynamic reproduction.
Minimal reproduction:
- Create router config with:
router.scenarios.default = cheap/c
router.stats.baselineModel = exp/e
- Call
createRouterRuntime(...).
- Inspect the created stats collector baseline.
Key output:
- Parsed config preserves the explicit baseline.
- Runtime stats collector receives
{ provider: "cheap", model: "c" }.
- Expected explicit baseline was
{ provider: "exp", model: "e" }.
Boundary: this repro does not need to perform a real model call. It validates the runtime construction path where stats collector config is derived. It does not change scenario routing behavior; it only observes which baseline is passed to stats.
Expected behavior
An explicitly configured router.stats.baselineModel should take precedence over router.scenarios.default. The default scenario should only be used as a fallback baseline when no explicit stats baseline is configured.
Existing coverage checked
No matching fix was found.
Checked adjacent work:
Search terms checked included router.stats.baselineModel, TokenStatsCollector baselineModel, and savedCost baseline.
Suggested fix
Change the stats collector construction priority in createRouterRuntime so it uses:
config.stats?.baselineModel, when explicitly present.
config.scenarios?.default, only as fallback.
- No baseline override otherwise.
This keeps routing behavior unchanged while making the stats contract match the parser/schema contract.
Suggested tests
- Explicit
stats.baselineModel wins when scenarios.default also exists.
scenarios.default is used as fallback when stats.baselineModel is absent.
- Explicit
stats.baselineModel works when no default scenario exists.
- Baseline-cost calculation uses the explicit baseline provider/model after runtime construction.
Submitted with Codex.
Summary
router.stats.baselineModelis parsed, butcreateRouterRuntimeoverrides it withrouter.scenarios.defaultwhenever a default scenario exists.Why it matters
The purpose of
router.stats.baselineModelis to define the no-router baseline used for saved-cost and baseline-cost calculations. Most router configs have a default scenario, so the explicit baseline is silently ignored in normal use. The dashboard or stats output can then report savings against the routed default model instead of the user-selected baseline model.Evidence
src/router/config/schema.ts:61documentsbaselineModelas the provider/model ref used for saved-cost baseline calculation.src/router/config/parseRouterConfig.ts:491entersparseStats.src/router/config/parseRouterConfig.ts:532parsesraw.baselineModel.src/router/config/parseRouterConfig.ts:538returns the parsedbaselineModel.src/router/RouterRuntime.ts:90constructsTokenStatsCollector.src/router/RouterRuntime.ts:93prefersconfig.scenarios.defaultas the collector baseline whenever a default scenario exists.src/router/RouterRuntime.ts:95only falls back toconfig.stats?.baselineModelwhen there is no default scenario.src/router/stats/TokenStatsCollector.ts:68stores the collector baseline fromconfig?.baselineModel.src/router/stats/TokenStatsCollector.ts:317uses that baseline during baseline-cost calculation.src/router/stats/TokenStatsCollector.ts:321selects the baseline provider.src/router/stats/TokenStatsCollector.ts:322selects the baseline model.Validation
Validation level: dynamic reproduction.
Minimal reproduction:
router.scenarios.default = cheap/crouter.stats.baselineModel = exp/ecreateRouterRuntime(...).Key output:
{ provider: "cheap", model: "c" }.{ provider: "exp", model: "e" }.Boundary: this repro does not need to perform a real model call. It validates the runtime construction path where stats collector config is derived. It does not change scenario routing behavior; it only observes which baseline is passed to stats.
Expected behavior
An explicitly configured
router.stats.baselineModelshould take precedence overrouter.scenarios.default. The default scenario should only be used as a fallback baseline when no explicit stats baseline is configured.Existing coverage checked
No matching fix was found.
Checked adjacent work:
RouterRuntimefor resilience, retry, and provider health tracking, but does not change stats baseline precedence and does not add tests for explicitstats.baselineModel.Search terms checked included
router.stats.baselineModel,TokenStatsCollector baselineModel, andsavedCost baseline.Suggested fix
Change the stats collector construction priority in
createRouterRuntimeso it uses:config.stats?.baselineModel, when explicitly present.config.scenarios?.default, only as fallback.This keeps routing behavior unchanged while making the stats contract match the parser/schema contract.
Suggested tests
stats.baselineModelwins whenscenarios.defaultalso exists.scenarios.defaultis used as fallback whenstats.baselineModelis absent.stats.baselineModelworks when no default scenario exists.Submitted with Codex.