Skip to content

fix(ai,coding-agent): correct Copilot context window discovery and await it during startup#34

Open
spencerharmon wants to merge 1 commit into
JuliusBrussee:mainfrom
spencerharmon:fix/copilot-discovery-context-window
Open

fix(ai,coding-agent): correct Copilot context window discovery and await it during startup#34
spencerharmon wants to merge 1 commit into
JuliusBrussee:mainfrom
spencerharmon:fix/copilot-discovery-context-window

Conversation

@spencerharmon

@spencerharmon spencerharmon commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Three bugs combined to make Copilot's 1M-context Anthropic variants effectively unusable: the model resolved to a smaller-window sibling and compaction fired well below the real ceiling.

Bugs fixed

  1. Discovery host hardcoded. discoverCopilot hit api.individual.githubcopilot.com/models unconditionally, which returns HTTP 421 Misdirected Request for some accounts (business/enterprise tiers, and at least one individual account). Discovery silently no-op'd (if (!res.ok) return;) → 1M ids never merged into the registry → resolution fell back to a smaller-window sibling.

  2. contextWindow used the wrong field. Discovery wrote max_context_window_tokens (total input+output+cache budget) to contextWindow. The compaction threshold (window − 16k reserve) should compare against the input budget (max_prompt_tokens). The threshold never tripped until the provider rejected the request.

  3. Discovery races initial model resolution. discoverAnthropicCapabilities() was fire-and-forget at session startup. When the saved default model isn't in the static registry (true for any discovered-only id), resolution loses the race and falls back to a sibling with a smaller window.

Fix

  • discoverCopilot tries the configured host first, then falls back through api.githubcopilot.com.business.enterprise. The host that responded is also used as the registered model's baseUrl so chat hits the same proxy that served /models.
  • Both the capability cache and the registered model entry use max_prompt_tokens (fall back to max_context_window_tokens only when missing).
  • createAgentSession and createAgentSessionServices await discovery before resolving the initial model only when the saved default isn't already in the static snapshot. Adds at most one /models round-trip to startup in that case; the fast path is preserved otherwise.

Verification

Direct simulation of shouldCompact against the patched code with a hypothetical 936k-prompt-cap model:

contextWindow: 936000  reserve: 16384  threshold: 919616
  tokens= 125529 → compact? false   ← previously a false-trigger
  tokens= 919000 → compact? false
  tokens= 920000 → compact? true    ← real ceiling

Live session confirmed: with this patch, resolution returns the discovered entry with the correct prompt-cap window from the first turn instead of compacting at the static-registry sibling's much smaller threshold.

Files

  • packages/ai/src/providers/anthropic-discovery.ts — host fallback + max_prompt_tokens
  • packages/coding-agent/src/core/sdk.ts — await discovery when saved default isn't in snapshot
  • packages/coding-agent/src/core/agent-session-services.ts — same change for the services factory
  • CHANGELOG entries in both packages

@spencerharmon
spencerharmon force-pushed the fix/copilot-discovery-context-window branch from 7bf7eef to 450369b Compare June 4, 2026 07:57
@spencerharmon

Copy link
Copy Markdown
Contributor Author

CI failure here is unrelated to this PR — models.generated.ts is regenerated each CI run, and several test files reference model IDs that upstream providers no longer return (gpt-4o on github-copilot, qwen-3-235b-a22b-instruct-2507 on cerebras, google/gemini-2.0-flash-001 on openrouter). All 17 type errors are in test files this PR does not touch. Fix split out into #35.

…ait it during startup

GitHub Copilot's /models endpoint surfaces 1M-context Anthropic variants
that aren't in the static registry. Three bugs made these effectively
unusable:

1. Discovery hit api.individual.githubcopilot.com unconditionally, which
   returns HTTP 421 Misdirected Request for some accounts. Discovery
   silently no-op'd, the 1M ids were never merged, and resolution fell
   back to a smaller-window sibling.

2. When discovery did succeed, contextWindow was set to
   max_context_window_tokens (total input+output+cache budget), not
   max_prompt_tokens (real input cap). The compaction threshold
   (window - 16k reserve) never tripped, so the provider rejected
   oversized prompts before compaction could fire.

3. discoverAnthropicCapabilities() was fire-and-forget at session
   startup, racing initial model resolution. If the saved default model
   wasn't in the static registry, resolution lost the race and fell
   back to a sibling with a smaller window.

Fixes:

- discoverCopilot now tries the configured host first, then falls back
  through api.githubcopilot.com / .business / .enterprise. The host
  that responded is used as the registered model's baseUrl so chat
  calls hit the same proxy that served /models.
- Both the capability cache and the discovered registry entry now use
  max_prompt_tokens (fall back to max_context_window_tokens only when
  missing). Compaction thresholds now match what the provider will
  actually accept.
- createAgentSession and createAgentSessionServices now await discovery
  before resolving the initial model when the saved default isn't in
  the static snapshot. Adds at most one /models round-trip to startup
  in that case; fire-and-forget path is preserved when the saved
  default already resolves.
@spencerharmon
spencerharmon force-pushed the fix/copilot-discovery-context-window branch from 450369b to 2e8e740 Compare June 7, 2026 06:38
@spencerharmon

Copy link
Copy Markdown
Contributor Author

This CI failure is the same set of pre-existing failures present on main itself (run 27038697661 on d9a441e, the current main HEAD). The 9 failing suites — ws17.test.ts, package-manager.test.ts, settings-manager.test.ts, resource-loader.test.ts, etc. — all reference a .pi directory that no longer exists (the project uses .cave), and reproduce on a clean origin/main checkout with no changes from this PR:

$ git checkout origin/main
$ cd packages/coding-agent && npx vitest --run test/settings-manager.test.ts
# 3 failed | 15 passed

None of the failing tests exercise the files this PR touches (anthropic-discovery.ts, sdk.ts, agent-session-services.ts). Happy to file a separate cleanup PR for the stale .pi references if useful, but it's a much bigger surface area than the upstream-drift fix in #35.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant