feat(providers): pin Gemini and Grok models via config.json - #162
Conversation
Adds `providers.gemini.model` and `providers.grok.model` so both providers
can be pinned from deliberation's own config instead of only through env
vars and each CLI's private settings.
Gemini: agy >= 1.0.9 has a `--model` flag, so the bridge now passes the
model through to argv instead of accepting it and dropping it. A run no
longer inherits whatever `~/.gemini/settings.json` happens to hold. The
built-in default stays the portable `auto-gemini-3` router alias, because
concrete agy ids differ per install and a pinned id that does not exist
locally fails every call. Pinning a concrete id is still preferable where
possible: the alias routes server side and agy's catalog also carries
Claude and GPT-OSS entries, so a routed call can seat a non-Gemini model
in the Gemini slot and silently collapse cross-model independence in
ask-all and consensus.
Grok: the documented default `grok-4.3` no longer matches the served
model, so the built-in moves to `grok-4.5`. The standalone bridge reads
the same config key as the unified server rather than staying env-only.
Precedence for both, uniform: per-call `model` > `providers.<name>.model`
> env var > built-in default. The key is read once at MCP start, so a
change needs a restart. Codex is deliberately excluded, it resolves its
model from `~/.codex/config.toml` and deliberation never overrides it.
`resolveProviders` previously projected `providers` down to `{ enabled }`,
which silently dropped the new key on the way to the composition root, so
a valid config edit would have been accepted and done nothing. It now
carries `model`, drops blank or non-string values rather than forwarding a
bogus id, and ignores the key for openrouter, whose models map owns
selection.
Tests: `M1` asserted the old "model never reaches argv" contract and is
rewritten for the new one, plus `M1b` for the default path and `PM1`-`PM4`
for the config projection. 595 pass.
Docs: TECHNICAL.md, SETUP.md, rules/model-selection.md, and the ask-*
commands. Removed the hardcoded model id from the ask-gemini example,
since pinning a concrete id in a shared command is the same portability
trap. Host artifacts regenerated.
Claude-Session: https://claude.ai/code/session_01Q7Zv2sYjAsBhacqakr1ftJ
antonbabenko
left a comment
There was a problem hiding this comment.
Review verdict: REQUEST CHANGES
(Posted as a comment - GitHub blocks request-changes on your own PR.)
Config plumbing is sound and the resolveProviders projection fix is well covered by PM1-PM4. Two HIGH issues block merge.
Self-review caveat: authored and reviewed in the same session. Findings below were verified by execution, not by re-reading intent.
HIGH
H1 - M1b fails whenever GEMINI_DEFAULT_MODEL is set (environment-dependent test)
DEFAULT_MODEL = process.env.GEMINI_DEFAULT_MODEL || "auto-gemini-3" is computed at module load; the spawned bridge inherits the parent env and no test clears the var. Proven:
$ GEMINI_DEFAULT_MODEL="gemini-3.1-pro-low" node --test test/bridge.test.js
✖ M1b: absent model falls back to the pinned default, not settings.json
actual: 'gemini-3.1-pro-low'
expected: 'auto-gemini-3'
Not hypothetical: GROK_DEFAULT_MODEL=grok-4.5 is already exported in this dev environment, so the sibling var being set is the normal case. Any contributor or CI runner with GEMINI_DEFAULT_MODEL set gets a red suite from an unrelated change.
Fix: pass a scrubbed child env to startBridge for M1b so the test pins the constant, not the machine.
H2 - Unconditional --model breaks agy < 1.0.9, no version guard
buildAgyArgs now always pushes --model. agy rejects unknown flags outright (agy models --json -> Error: flags provided but not defined: -json). Before this PR the bridge never passed it, so older installs worked; now every Gemini call fails at argv parse. TECHNICAL.md states ">= 1.0.9" in prose but nothing enforces or degrades.
I could not install an older agy to reproduce directly, so this is inferred from the same binary's parser behavior plus the removed model ... never reaches argv comment. The change is unguarded either way.
Fix: server/gemini/index.js:811 already shells out to agy --help for the health check. Capture it once, detect --model, and skip the flag or fail with an actionable upgrade message.
MEDIUM
M1 - Seeding model into config.default.json kills the env var for new installs. commands/setup.md:88 copies that file to the user config, and precedence is config > env, so GROK_DEFAULT_MODEL / GEMINI_DEFAULT_MODEL become dead for anyone who runs setup. This undercuts the "configurable via both" goal. Values match the built-ins so behavior is unchanged today, but consider omitting the key from the seeded default and documenting it in SETUP.md only.
M2 - Composition-root expression is ambiguous and double-reads config. (getConfig().providers && getConfig().providers.gemini || {}).model parses correctly as (a && b) || {} but reads poorly and calls getConfig() twice. Prefer getConfig()?.providers?.gemini?.model.
LOW
L1 - providers.*.model is read once at startup while the models map hot-reloads. Documented in code, SETUP.md, and the PR body, so a note rather than a defect. Reading it inside ask() (as the Grok bridge's configuredModel() already does) would drop the restart requirement and make the two paths consistent.
Verified as non-issues
runWithFilesdelegates torunGrok, so the config pin applies to file-bearing Grok calls.configuredModel()is try/catch wrapped, returnsundefinedon failure, adds no throw path.- Blank / non-string
modeldropped rather than forwarded (PM3);openroutercorrectly ignores the key (PM4). - No circular import between the Grok bridge and the config module.
- No secrets, injection surface, or auth changes;
--modelis a distinct argv element (no shell) and the-p <prompt>tail invariant holds.
Validation
| Check | Result |
|---|---|
| Type check | Pass |
| Tests | Pass (595 pass, 0 fail) |
| Lint / Build | Skipped (no such scripts) |
| Host-artifact drift | Pass |
The suite is green on this machine only because GEMINI_DEFAULT_MODEL is unset. See H1.
…dent Addresses the two HIGH findings from the review of #162, and adds a configurable Grok reasoning effort. H1: `DEFAULT_MODEL` reads `GEMINI_DEFAULT_MODEL` at module load, so M1b asserted whatever the machine exported rather than the built-in constant. Any contributor or CI runner with that variable set got a red suite from an unrelated change. The test now blanks the variable in the child env. Verified: the whole suite stays green with GEMINI_DEFAULT_MODEL, GROK_DEFAULT_MODEL, and GROK_REASONING_EFFORT all set to foreign values. H2: `buildAgyArgs` pushed `--model` unconditionally, but agy rejects unknown flags outright, so every Gemini call would have failed on agy older than 1.0.9. `agySupportsModelFlag()` now probes `agy --help` once (the same call the startup health check already makes), caches the result for the process, and omits the flag with a one-time stderr warning when absent. Degrading to the pre-pin behavior beats failing every call. A probe that throws is treated as supported, so a transient failure never silently drops a pin the operator asked for. `fake-agy-nomodel.sh` errors if it ever sees `--model`, so M1c fails loudly on a regression. M2: the composition root hoists the provider config blocks once instead of calling `getConfig()` twice inside a mixed `&&`/`||` expression. Grok reasoning effort is now configurable via `providers.grok.reasoningEffort`, on the same ladder as the model pin: per-call > config > `GROK_REASONING_EFFORT` > `high`. Gemini gets no equivalent key because agy fuses effort into the model id itself. The first attempt put that lookup inside `resolveReasoningEffort`, which made the pure resolver read the real user config and broke the existing G20 test on any machine with a Grok effort pinned - the same environment-dependence as H1. Config injection now happens at the call sites (composition root for the core adapter, handler for the bridge) and the resolver stays pure. 600 pass, and the suite is now verified hermetic against both a polluted environment and a populated config.json. Claude-Session: https://claude.ai/code/session_01Q7Zv2sYjAsBhacqakr1ftJ
Review follow-up: H1, H2, M2 fixed (+ Grok effort config)Pushed H1 — fixed. M1b now blanks H2 — fixed. M2 — fixed. Composition root hoists M1 — still open by design. L1 — partly addressed. The new New:
|
The PR review artifact belongs on the PR, not in version control. Its content is preserved as a review comment on #162. Claude-Session: https://claude.ai/code/session_01Q7Zv2sYjAsBhacqakr1ftJ
What
Adds
providers.gemini.modelandproviders.grok.modelso both providers can be pinned from deliberation's own config, instead of only through env vars and each CLI's private settings file.Precedence is uniform for both: per-call
model>providers.<name>.model> env var > built-in default. The key is read once at MCP start (unlike the hot-reloadingmodelsmap), so a change needs a restart.Codex is deliberately excluded. It resolves its model from
~/.codex/config.tomland deliberation never overrides it, so it keeps theenabled-onlysimpleProvidershape. Gemini needed its owngeminiProviderdef to carry a model.Gemini
agy>= 1.0.9 has a--modelflag. The bridge previously accepted amodelparam and dropped it on the floor (model is accepted but never reaches argv); it now passes it through, so a run no longer inherits whatever~/.gemini/settings.jsonhappens to hold.The built-in default stays the portable
auto-gemini-3router alias. Concrete agy ids differ per install, and a pinned id that does not exist locally fails every call.Pinning a concrete id is still preferable where you can: the alias routes server-side, and
agy modelsshows the catalog also carries Claude Opus/Sonnet and GPT-OSS entries. A routed call can therefore seat a non-Gemini model in the Gemini slot, which quietly collapses cross-model independence in/ask-alland/consensuswhile Claude is already the arbiter. Nothing in the result envelope would show it.Model ids fuse family and reasoning effort (
gemini-3.6-flash-high); a bare family is rejected withrequires --effort.Grok
The documented default
grok-4.3no longer matches the served model, so the built-in moves togrok-4.5. The standalone bridge now reads the same config key as the unified server rather than staying env-only, so/ask-grokand thedeliberationserver cannot drift apart.The bug this nearly shipped with
resolveProvidersinserver/openrouter/config.jsprojectedprovidersdown to{ enabled }, silently dropping the new key before it reached the composition root. A valid config edit would have been accepted and done nothing:It now carries
model, drops blank or non-string values rather than forwarding a bogus id, and ignores the key foropenrouter(whosemodelsmap owns selection).PM1-PM4cover the projection so it cannot quietly narrow again.Test plan
npm run check(typecheck + suite): 595 pass, 0 failM1rewritten. It asserted the exact contract this PR reverses (model param accepted, never reaches argv) and failed as expected before being updatedM1badded for the default-pin pathPM1-PM4added for the config projection["--sandbox","--model","gemini-3.6-flash-high","-p"]gemini-3.6-flash-highresolves,gemini-3.6-flasherrors withrequires --effortscripts/sync-hosts.jsDocs
TECHNICAL.md(bridge section + both env rows),SETUP.md(example block + new precedence section),rules/model-selection.md,commands/ask-gemini.md,commands/ask-grok.md,commands/ask-all.md.Removed the hardcoded
model:id from theask-geminiexample. Pinning a concrete id in a shared command is the same portability trap the default avoids.https://claude.ai/code/session_01Q7Zv2sYjAsBhacqakr1ftJ