Add ChatGPT subscription LLM support

[neubig]

Summary

• Add ChatGPT subscription auth constants, service wrappers, query/mutation hooks, and MSW mocks for planned agent-server subscription endpoints.
• Extend LLM settings/profile UI with API-key vs ChatGPT subscription mode, subscription model selection, and connect/status/logout controls.
• Shape subscription conversation/profile payloads to omit API key/base URL and gate conversation start/profile switch on subscription connection status.

Verification

• npm run typecheck
• npx vitest run __tests__/api/llm-subscription-service.test.ts __tests__/api/agent-server-adapter.test.ts __tests__/routes/llm-settings.test.tsx
• npm run check-translation-completeness
• npm test
• npm run lint
• npm run build

Notes

• Live ChatGPT subscription verification still requires real browser-login credentials/session and the planned agent-server subscription endpoints.

This PR was created by an AI agent (OpenHands) on behalf of the user.

---

🐳 Docker images for this PR

• GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component	Value
Image	ghcr.io/openhands/agent-canvas
Architectures	amd64, arm64
Agent Server	ghcr.io/openhands/agent-server:1.23.0-python
Automation	openhands-automation==1.0.0a3
Commit	83e343c3bef1e1da25b168af210fc6196d03d569

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-83e343c

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-83e343c

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-83e343c-amd64
ghcr.io/openhands/agent-canvas:add-llm-subscription-support-amd64
ghcr.io/openhands/agent-canvas:pr-744-amd64
ghcr.io/openhands/agent-canvas:sha-83e343c-arm64
ghcr.io/openhands/agent-canvas:add-llm-subscription-support-arm64
ghcr.io/openhands/agent-canvas:pr-744-arm64
ghcr.io/openhands/agent-canvas:sha-83e343c
ghcr.io/openhands/agent-canvas:add-llm-subscription-support
ghcr.io/openhands/agent-canvas:pr-744

About Multi-Architecture Support

• Each tag (e.g., sha-83e343c) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., sha-83e343c-amd64) are also available if needed

Companion PRs / temporary pins

• Backend / agent-server endpoints: Add LLM subscription auth endpoints software-agent-sdk#3367
• Typed client methods: Add LLM subscription client methods typescript-client#178
• Temporary dependency pin: @openhands/typescript-client points at git+https://github.com/OpenHands/typescript-client.git#add-llm-subscription-client until the client PR is merged and released.
• Temporary live backend ref: use OH_AGENT_SERVER_GIT_REF=add-llm-subscription-endpoints for manual live testing until the SDK PR is merged and released.

This PR description update was created by an AI agent (OpenHands) on behalf of the user.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agent-canvas	Ready	Preview, Comment	May 23, 2026 3:43pm

[all-hands-bot]

Review Summary

🟡 Acceptable with critical issues - The subscription auth feature is well-structured overall, but has fundamental issues that must be addressed before merging.

[CRITICAL ISSUES]

• [__tests__/api/llm-subscription-service.test.ts] Test Quality: The core test suite violates repository guidelines by mocking the entire LLMMetadataClient HTTP client instead of testing real code paths with MSW. This means tests won't catch actual regressions when the HTTP layer, auth headers, or server contract changes. Rewrite to use the MSW handlers already available in settings-handlers.ts.

• [src/components/features/settings/llm-profiles/llm-settings-local-view.tsx:170] Complexity: The handleSave function has >4 levels of nesting (try-catch → rename if → auth type if-else → subscription model if → API key if-else → base_url if → wasActive if), violating the repository's '3 levels max' guideline. Extract helpers: buildLlmConfigForAuthType(), handleProfileRename(), handleProfileReactivation().

• [src/api/llm-subscription-service.ts:78] Type Safety: Unsafe casting (client as unknown as LLMMetadataClientWithTransport).client to access internal transport. If @openhands/typescript-client changes its internal structure, this fails at runtime with cryptic errors. Add runtime validation that the transport exists and has the expected methods before using it.

[IMPORTANT ISSUES]

• [src/components/features/settings/llm-settings/openai-subscription-auth-card.tsx:61] User Experience: When polling returns connected: false, an error toast is shown with a "PENDING" message. This is misleading—users think something failed when they just need to wait. Use an info/warning toast or inline status text instead.

• [src/components/features/settings/llm-settings/openai-subscription-auth-card.tsx:36] State Management: Users can get stuck in the challenge state if they start device login but close the tab or navigate away. No way to clear the challenge except completing login or logging out (which fails if not connected). Add a "Cancel" button to reset challenge state.

• [src/routes/llm-settings.tsx:219] Data Flow: Model selection is silently changed when switching auth types. If a user has 'gpt-4o', switches to subscription to explore, then switches back, their original selection is lost forever (replaced with defaultModel). Store previous models in component state and restore them, or at minimum show a toast explaining the change.

• [src/api/llm-subscription-service.ts:109] Validation: Missing validation for empty strings in normalizeDeviceChallenge. A buggy server could return empty strings that pass the !deviceCode check. Add .trim().length > 0 validation for deviceCode, userCode, and verificationUri.

• [__tests__/api/llm-subscription-service.test.ts] Test Coverage: Missing critical test cases: error handling (401/403, 500, timeouts, malformed responses), validation (incomplete device challenges), end-to-end integration (full device flow), and edge cases (empty strings, null values, concurrent requests).

• [src/api/llm-subscription-service.ts:87] Error Handling: Network errors from the HTTP client propagate without context. Wrap errors to provide more specific messages about which subscription operation failed.

[SUGGESTIONS]

• [src/api/agent-server-adapter.ts:820] Documentation: Add docstring to assertSubscriptionAuthReady noting that it's NOT called when resuming conversations or sending additional messages, so subscriptions could expire mid-conversation. The agent-server must handle expired tokens gracefully.

• [src/api/llm-subscription-service.ts:136] Data Structure: The expiresAt field tries both string and number fallbacks, including expires_in which is typically a relative duration in seconds, not an absolute timestamp. Mixing relative and absolute time values in the same field could cause confusion. Document the expected behavior or convert relative durations to absolute timestamps.

• [src/hooks/query/use-llm-subscription-status.ts:14] Query Configuration: staleTime of 30 seconds seems short for auth status that rarely changes. Consider 5 minutes to reduce unnecessary network requests, especially since mutations properly invalidate this query.

[RISK ASSESSMENT]

🟡 MEDIUM RISK - This PR adds new authentication flows and modifies conversation creation logic. The main risks are:

• Untested error paths could cause poor UX when auth fails
• Complex nested logic increases maintenance burden
• Type safety issues could cause runtime failures with SDK updates

Not blocking auto-merge based on risk alone, but the critical issues above should be addressed.

VERDICT:

❌ Needs rework - Address the critical test quality and complexity issues, then fix the important UX and validation gaps.

KEY INSIGHT:

The subscription auth implementation follows OAuth device flow patterns correctly, and backward compatibility is preserved (API key flows still work). However, the test suite doesn't provide confidence that the integration will work in production, and the complex nested save logic creates maintenance risk. Focus first on rewriting tests to use MSW and refactoring handleSave.

---

Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/OpenHands/agent-canvas/actions/runs/26336381445

[github-actions]

📸 Snapshot Test Report

Warning

Snapshot comparison step crashed (timeout, OOM, or runner error) — diff results below may be incomplete or absent.
Check the CI logs for the full error output (look for the "Run snapshot comparison" step).

✅ 1 snapshot changed — acknowledged via the update-snapshots label. New baselines will be uploaded when this PR merges.

Category	Count
🔴 Changed	1
🆕 New	0
✅ Unchanged	72
Total	73

🔴 Changed snapshots (1)

onboarding

onboarding-step-2-setup-llm

Expected (main)	Actual (PR)	Diff

✅ Unchanged snapshots (72)

archived-conversation

• conversation-panel-with-archived-badges
• conversation-view-archived
• conversation-view-sandbox-error

automations

• automations-delete-modal
• automations-list-active-inactive
• automations-no-automations
• automations-search-no-results

backends-extended

• backend-add-blank-disabled
• backend-add-cloud-advanced-open
• backend-add-cloud-no-key-disabled
• backend-add-cloud-with-key-enabled
• backend-add-form-partially-filled
• backend-add-invalid-url-disabled
• backend-add-local-ready
• backend-add-name-only-disabled
• backend-add-two-column-layout
• backend-add-whitespace-host-disabled
• backend-after-switch
• backend-cancel-nothing-saved
• backend-dropdown-two-backends
• backend-edit-prefilled
• backend-manage-after-removal
• backend-manage-two-listed
• backend-remove-cancelled
• backend-remove-confirmation
• backend-switch-overlay

backends

• backend-add-modal
• backend-manage-modal
• backend-selector-open

changes-tab

• changes-deleted-file
• changes-diff-viewer
• changes-empty

collapsible-thinking

• reasoning-content-collapsed
• reasoning-content-expanded
• think-action-collapsed
• think-action-expanded

mcp-page

• mcp-custom-server-1-editor-open
• mcp-custom-server-2-url-filled
• mcp-custom-server-3-all-filled
• mcp-custom-server-4-installed
• mcp-custom-server-editor
• mcp-empty-installed
• mcp-search-filtered
• mcp-slack-install-1-marketplace
• mcp-slack-install-2-modal
• mcp-slack-install-3-filled
• mcp-slack-install-4-installed

onboarding

• onboarding-step-0-choose-agent
• onboarding-step-1-check-backend
• onboarding-step-3-say-hello

projects-workspace-browser

• projects-workspace-browser

settings-page

• add-backend-modal
• analytics-consent-modal
• home-screen
• settings-app-page
• settings-page

settings-secrets

• secrets-add-form-filled
• secrets-add-form
• secrets-after-save
• secrets-delete-confirm
• secrets-list

settings-verification

• condenser-settings
• verification-settings-off
• verification-settings-on

sidebar

• sidebar-collapsed
• sidebar-conversation-panel
• sidebar-filter-menu

skills-page

• skills-empty
• skills-loaded
• skills-no-match
• skills-search-filtered
• skills-type-filter

---

Generated by the Snapshot Tests workflow. This comment was created by an AI agent (OpenHands) on behalf of the repo maintainers.