Skip to content

Move model capability tables into ai-config, add inferModelCapabilities#13

Merged
wch merged 8 commits into
mainfrom
model-capabilities-in-ai-config
Jul 16, 2026
Merged

Move model capability tables into ai-config, add inferModelCapabilities#13
wch merged 8 commits into
mainfrom
model-capabilities-in-ai-config

Conversation

@melissa-barca

@melissa-barca melissa-barca commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Moves the LLM model-capability tables (Anthropic, DeepSeek, Gemini, Gemma, OpenAI, Posit AI) from ai-provider-bridge into ai-config, and adds inferModelCapabilities(providerId, modelId) as the single entry point for merging the generic baseline with provider-family inference. This lets any ai-config consumer (in particular, Positron's authentication extension) synthesize model capabilities without taking the bridge's dependency tree.

The Gemini Interactions API allowlist (getGeminiInteractionsProfile, isInteractionsEligible) stays in the bridge since it's SDK-routing logic rather than a dependency-free capability table. The bridge re-exports every capability helper it used to own directly from ai-config, so its public export surface is unchanged.

Also adds tsx as an explicit ai-config devDependency -- its prebuild script already invoked tsx but relied on hoisting from the workspace root rather than declaring its own dependency.

Closes #9.

Test plan

  • npm run check-types / npm test / npm run lint / npm run format:check all pass for every package
  • Root shape guard (typechecks/) passes
  • git diff main -- packages/ai-config/src/model-capabilities confirms the moved tables are byte-identical apart from import lines
  • packages/ai-provider-bridge/dist/index.d.ts still declares every previously-exported capability name
  • grep -rn "model-capabilities" packages/ai-provider-bridge/src shows only gemini-interactions references

@melissa-barca
melissa-barca requested a review from wch July 15, 2026 19:07

@wch wch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! Some automated review feedback:

Findings

  • must-fixpackages/ai-config/src/model-capabilities/infer.ts:152: Posit AI Gemma inference spreads requiresChatTemplateKwargs into the returned object, but the strict customModelSchema does not accept that key (schema.ts:65) and customModelToModelInfo cannot carry it (resolve-models.ts:192). Since this API was added to synthesize models.custom entries during migration, the natural { id, name, ...inferModelCapabilities(...) } result is rejected for Gemma 4; add the field to the model schemas and resolver pipeline (with a schema-to-resolver regression test), or deliberately strip it if persisted custom Gemma thinking is unsupported.
  • must-fixpackages/ai-config/README.md:160: The README says ai-provider-bridge re-exports inferModelCapabilities, but its root export list only re-exports the five pre-existing helpers plus openaiMaxInputTokens (ai-provider-bridge/src/index.ts:75). Either add the advertised export or correct the claim so consumers are not directed to an import that does not exist.

Suggestions

  • packages/ai-provider-bridge/src/model-capabilities/gemini-interactions.ts:25: Splitting the capability table into ai-config leaves each eligible model's thinking levels duplicated across packages and explicitly requires maintainers to update both. Keep only the fail-closed eligible-ID set here and derive each profile's thinkingLevels from getGeminiModelCapabilities; then eligibility remains bridge routing policy while the allowed levels have one source of truth.

@melissa-barca

Copy link
Copy Markdown
Collaborator Author

Thanks @wch — went through all three. Fixes pushed in 8c19152.

Item 1 (Gemma requiresChatTemplateKwargs) — fixed. Confirmed the diagnosis: both the strict customModelSchema and customModelToModelInfo drop the key, so { id, name, ...inferModelCapabilities(...) } would be rejected for Gemma 4. Rather than persist it, I dropped it from inferModelCapabilities' output — it's a runtime request-shaping flag (tells the vLLM client to send chat_template_kwargs), re-derived from the model id at request time by the bridge's positai path, so it isn't durable disk metadata. The capability tables still carry it for that runtime use; it's stripped only at the migration seam. inferModelCapabilities now returns Omit<InferredModelCapabilities, "requiresChatTemplateKwargs">, and there's a regression test that parses { id, name, ...inferModelCapabilities(...) } through customModelSchema for a representative id per provider family (Gemma included).

Item 2 (README re-export claim) — fixed. You're right, the bridge root doesn't re-export inferModelCapabilities (only the five helpers + openaiMaxInputTokens, unchanged from before this PR). Corrected the README to say so and to direct consumers to import inferModelCapabilities from ai-config directly.

Item 3 (derive Interactions thinkingLevels from the table) — respectfully declining. The two lists look redundant but are conceptually different data, and today's agreement is a coincidence rather than a shared source:

  • profile.thinkingLevels (interactions allowlist) is the wire-level Interactions API validation set — GeminiClient.buildInteractionsOptions uses it to clamp an unrecognized thinkingLevel to "medium". It deliberately excludes "off", because "off" is handled as a separate branch (thinkingEffort !== "off") that sends no thinkingLevel at all.
  • The capability table's thinkingEffortLevels is the user-facing effort menu, which does include "off" for several models.

Deriving one from the other would mean filtering out "off" (and any table-only level) in the bridge, which re-introduces exactly the coupling the split was meant to avoid: eligibility + wire-level validation is bridge routing policy, and it'd now depend on the exact contents of a dependency-free ai-config table. I'd rather keep the allowlist self-contained and fail-closed. Happy to reconsider if you feel the duplication risk outweighs that.

@melissa-barca
melissa-barca requested a review from wch July 16, 2026 13:38

@wch wch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@wch
wch merged commit e5537a0 into main Jul 16, 2026
1 check passed
@wch
wch deleted the model-capabilities-in-ai-config branch July 16, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose model capability inference for reuse outside the bridge

2 participants