fix(ai,coding-agent): correct Copilot context window discovery and await it during startup#34
Conversation
7bf7eef to
450369b
Compare
|
CI failure here is unrelated to this PR — |
…ait it during startup GitHub Copilot's /models endpoint surfaces 1M-context Anthropic variants that aren't in the static registry. Three bugs made these effectively unusable: 1. Discovery hit api.individual.githubcopilot.com unconditionally, which returns HTTP 421 Misdirected Request for some accounts. Discovery silently no-op'd, the 1M ids were never merged, and resolution fell back to a smaller-window sibling. 2. When discovery did succeed, contextWindow was set to max_context_window_tokens (total input+output+cache budget), not max_prompt_tokens (real input cap). The compaction threshold (window - 16k reserve) never tripped, so the provider rejected oversized prompts before compaction could fire. 3. discoverAnthropicCapabilities() was fire-and-forget at session startup, racing initial model resolution. If the saved default model wasn't in the static registry, resolution lost the race and fell back to a sibling with a smaller window. Fixes: - discoverCopilot now tries the configured host first, then falls back through api.githubcopilot.com / .business / .enterprise. The host that responded is used as the registered model's baseUrl so chat calls hit the same proxy that served /models. - Both the capability cache and the discovered registry entry now use max_prompt_tokens (fall back to max_context_window_tokens only when missing). Compaction thresholds now match what the provider will actually accept. - createAgentSession and createAgentSessionServices now await discovery before resolving the initial model when the saved default isn't in the static snapshot. Adds at most one /models round-trip to startup in that case; fire-and-forget path is preserved when the saved default already resolves.
450369b to
2e8e740
Compare
|
This CI failure is the same set of pre-existing failures present on None of the failing tests exercise the files this PR touches ( |
Summary
Three bugs combined to make Copilot's 1M-context Anthropic variants effectively unusable: the model resolved to a smaller-window sibling and compaction fired well below the real ceiling.
Bugs fixed
Discovery host hardcoded.
discoverCopilothitapi.individual.githubcopilot.com/modelsunconditionally, which returns HTTP 421 Misdirected Request for some accounts (business/enterprise tiers, and at least one individual account). Discovery silently no-op'd (if (!res.ok) return;) → 1M ids never merged into the registry → resolution fell back to a smaller-window sibling.contextWindowused the wrong field. Discovery wrotemax_context_window_tokens(total input+output+cache budget) tocontextWindow. The compaction threshold (window − 16k reserve) should compare against the input budget (max_prompt_tokens). The threshold never tripped until the provider rejected the request.Discovery races initial model resolution.
discoverAnthropicCapabilities()was fire-and-forget at session startup. When the saved default model isn't in the static registry (true for any discovered-only id), resolution loses the race and falls back to a sibling with a smaller window.Fix
discoverCopilottries the configured host first, then falls back throughapi.githubcopilot.com→.business→.enterprise. The host that responded is also used as the registered model'sbaseUrlso chat hits the same proxy that served/models.max_prompt_tokens(fall back tomax_context_window_tokensonly when missing).createAgentSessionandcreateAgentSessionServicesawait discovery before resolving the initial model only when the saved default isn't already in the static snapshot. Adds at most one/modelsround-trip to startup in that case; the fast path is preserved otherwise.Verification
Direct simulation of
shouldCompactagainst the patched code with a hypothetical 936k-prompt-cap model:Live session confirmed: with this patch, resolution returns the discovered entry with the correct prompt-cap window from the first turn instead of compacting at the static-registry sibling's much smaller threshold.
Files
packages/ai/src/providers/anthropic-discovery.ts— host fallback +max_prompt_tokenspackages/coding-agent/src/core/sdk.ts— await discovery when saved default isn't in snapshotpackages/coding-agent/src/core/agent-session-services.ts— same change for the services factory