feat: add Vertex AI Grok support with ADC#286
Closed
AmeerJ97 wants to merge 5 commits into
Closed
Conversation
Package tarballs should contain the current built CLI and public package metadata, not local agent state, editor rules, hooks, stale build outputs, or generated test artifacts. Tighten npm ignore rules and clean dist before TypeScript builds so normal installs are smaller and do not leak workspace runtime or branch-stale files. Constraint: npm pack includes untracked files unless .npmignore excludes them, and tsc does not remove stale dist files from previously built branches. Rejected: Leaving package hygiene to developer worktrees | local runtime files and stale branch outputs can be accidentally shipped by any pack/install-from-folder flow. Confidence: high Scope-risk: narrow Directive: Keep package contents explicit; update this ignore list whenever new local agent/runtime directories are introduced, and keep dist cleaning ahead of package builds. Tested: git diff --check; bun run build; npm pack --dry-run --json with zero .omx/.cursor/.codex/.agents/.husky/test-artifact/stale-vertex matches. Not-tested: Published npm release from CI. Co-authored-by: OmX <omx@oh-my-codex.dev>
AmeerJ97
force-pushed
the
feature/vertex-grok-headless
branch
2 times, most recently
from
a5e3f08 to
1776c4b
Compare
superagent-security
Bot
added
the
contributor:flagged
|
| Dimension | Score | What it measures |
|---|---|---|
| Identity | 30 | Account age, contribution history, GPG keys, org memberships |
| Behavior | 80 | PR patterns, unsolicited contribution ratio, activity cadence |
| Content | 100 | PR body substance, issue linkage, contribution quality |
| Graph | 30 | Cross-repo trust, co-contributor relationships |
Analyzed by Brin · Full profile
superagent-security
Bot
added
the
pr:flagged
Brin PR Security ScanThis PR has findings that should be reviewed.
Findings:
Analyzed by Brin |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Vertex AI-backed Grok path that reuses the existing xAI/OpenAI-compatible client flow by translating chat-completions requests to Vertex generateContent/streamGenerateContent, while keeping native xAI API-key mode available.
Changes:
- Added Vertex auth/config plumbing (
GROK_USE_VERTEX, project/location/base URL settings, ADC token acquisition, auth-state helpers). - Added a Vertex request/response adapter for chat completions, streaming, model ID mapping, and tool schema sanitization.
- Updated CLI/agent/tooling/docs/tests to allow Vertex mode and to block native xAI-only features when Vertex is active.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.config.ts | Adds shared test setup file for vitest. |
| test-vertex-integration.ts | Adds live Vertex smoke test script. |
| src/utils/settings.ts | Adds Vertex settings, auth-mode detection, and user-settings path override. |
| src/utils/settings-vertex.test.ts | Adds tests for Vertex settings/auth helpers. |
| src/test/setup.ts | Adds global test env setup for isolated user-settings path. |
| src/telegram/headless-bridge.ts | Lets Telegram headless mode accept Vertex auth state. |
| src/index.ts | Updates CLI auth checks and blocks batch mode in Vertex mode. |
| src/grok/vertex-auth.ts | Adds ADC token acquisition and auth error formatting. |
| src/grok/vertex-auth.test.ts | Adds tests for Vertex auth behavior. |
| src/grok/vertex-adapter.ts | Adds xAI-to-Vertex request/stream/tool translation layer. |
| src/grok/vertex-adapter.test.ts | Adds adapter/unit streaming/schema tests. |
| src/grok/tools.ts | Blocks xAI-only tools when Vertex mode is enabled. |
| src/grok/client.ts | Switches provider creation/runtime selection for Vertex mode. |
| src/grok/client.test.ts | Adds Vertex-mode runtime test. |
| src/audio/stt/engine.ts | Rejects Telegram STT in Vertex-only auth mode. |
| src/agent/auth.test.ts | Adds auth-state tests for incomplete/complete Vertex config. |
| src/agent/agent.ts | Treats configured Vertex mode as a provider-backed auth path. |
| README.md | Documents Vertex setup, limits, and troubleshooting. |
| package.json | Adds clean build step, Vertex test script, and google-auth-library dependency. |
| CHANGELOG.md | Documents new Vertex support. |
| bun.lock | Locks new google-auth-library dependency tree. |
| .npmignore | Excludes more local/test artifacts from published package. |
| .env.example | Documents Vertex-related env vars. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+447
to
+454
| const vertexEnabled = isVertexModeEnabled(); | ||
| const vertexMissing = vertexSettings.projectId ? [] : ["GROK_VERTEX_PROJECT_ID"]; | ||
| const vertexConfigured = vertexEnabled && vertexMissing.length === 0; | ||
| const xaiConfigured = Boolean(getApiKey()); | ||
|
|
||
| return { | ||
| activeMode: vertexEnabled ? "vertex" : "xai", | ||
| configured: vertexEnabled ? vertexConfigured : xaiConfigured, |
| const xaiRequest = (await readJsonRequest(input, init)) as XaiChatRequest; | ||
| const vertexSettings = requireVertexSettings(); | ||
| const isStreaming = xaiRequest.stream === true; | ||
| const vertexRequest = convertXaiChatRequestToVertex(xaiRequest); |
Comment on lines
+5
to
+17
|
|
||
| export async function getVertexAccessToken(): Promise<string> { | ||
| const auth = new GoogleAuth({ | ||
| scopes: VERTEX_AUTH_SCOPES, | ||
| clientOptions: { | ||
| transporterOptions: { | ||
| fetchImplementation, | ||
| }, | ||
| }, | ||
| }); | ||
| let token: string | null | undefined; | ||
| try { | ||
| token = await auth.getAccessToken(); |
Comment on lines
+304
to
+309
| const unionSchema = pickUnionSchema(schema); | ||
| if (unionSchema && unionSchema !== schema) { | ||
| const unionResult = sanitizeVertexSchemaValue(unionSchema); | ||
| if (isRecord(unionResult)) { | ||
| return copySchemaMetadata(schema, unionResult); | ||
| } |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Ensure pack and publish paths rebuild from a clean dist directory under Bun and keep platform-specific standalone binaries out of the npm tarball.\n\nConstraint: Review feedback showed npm packaging could include stale generated files when contributors skipped build or had binary artifacts in dist.\nRejected: Relying on reviewer discipline to run build manually | publish should enforce the package boundary.\nConfidence: high\nScope-risk: narrow\nDirective: Keep npm package contents generated from tsc output; standalone binaries belong in release artifacts, not the npm package.\nTested: bun run build; bun run build:binary; npm pack --dry-run --ignore-scripts --json; npm pack --dry-run --json\nNot-tested: actual npm publish
Grok on Vertex uses Google ADC, the global Vertex API host, and Google-native generateContent endpoints instead of xAI API-key chat completions. Add a fetch adapter that preserves the existing xAI provider surface while translating auth, URLs, messages, streaming chunks, and local tool schemas for Vertex. Constraint: Grok partner models require https://aiplatform.googleapis.com as the host while keeping a normal location path such as us-central1; Vertex also rejects OpenAI message/tool schemas that are not converted. Rejected: Using regional Vertex hosts | Grok partner models return bad requests there. Rejected: Replacing the xAI provider wholesale | the CLI already depends on @ai-sdk/xai behavior, so a fetch adapter keeps the blast radius smaller. Confidence: medium Scope-risk: broad Directive: Keep native xAI behavior gated behind GROK_USE_VERTEX=false; keep Vertex smoke tests deterministic by disabling tools for exact text-response checks and use separate tool smokes for local tool access. Tested: git diff --check; bun run typecheck; bun run test; bun run lint; bun run build; bun run test:vertex; Vertex targeted tests; npm pack --dry-run --json; normal tarball install Not-tested: Google Cloud projects without xAI partner model access; manual OpenTUI auth modal walkthrough Co-authored-by: OmX <omx@oh-my-codex.dev>
Address review findings around ADC reuse, unsupported request handling, nullable schema translation, and auth validation paths while keeping the native xAI flow unchanged.\n\nConstraint: Vertex mode must bridge OpenAI-style Vercel AI SDK requests into Google generateContent without leaking conversion exceptions into fetch callers.\nRejected: Validating ADC during simple configuration checks | token refresh can require network and belongs in request-time/test-auth paths.\nConfidence: high\nScope-risk: moderate\nDirective: Keep Vertex compatibility shims returning OpenAI-shaped JSON errors; do not let adapter conversion exceptions escape fetch.\nTested: bunx vitest run src/grok/vertex-auth.test.ts src/grok/vertex-adapter.test.ts src/telegram/headless-bridge.test.ts src/agent/auth.test.ts; bun run build\nNot-tested: live Vertex request after these review fixes
AmeerJ97
force-pushed
the
feature/vertex-grok-headless
branch
from
1776c4b to
3fa5c8f
Compare
Vertex streams may interleave text and function-call parts. Count only function calls when emitting OpenAI streaming tool_call indexes so downstream AI SDK parsing can correlate deltas correctly. Constraint: Cursor review identified part-position indexes as incompatible with OpenAI streaming tool-call semantics. Rejected: Keeping Vertex part positions in delta indexes | text parts can create sparse indexes and dropped tool calls. Confidence: high Scope-risk: narrow Directive: Preserve OpenAI tool-call index semantics independently from Vertex part array positions. Tested: bunx vitest run src/grok/vertex-adapter.test.ts; bun run build Not-tested: Live Vertex stream containing multiple function calls in one candidate.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ae95313. Configure here.
| case "MAX_TOKENS": | ||
| return "length"; | ||
| case "MALFORMED_FUNCTION_CALL": | ||
| return "tool_calls"; |
There was a problem hiding this comment.
MALFORMED_FUNCTION_CALL mapped to "tool_calls" without tool calls
Low Severity
mapVertexFinishReason maps MALFORMED_FUNCTION_CALL to "tool_calls", but this value is only used when extractFunctionCalls returns an empty array (the toolCalls.length > 0 ternary takes priority otherwise). The result is a response with finish_reason: "tool_calls" but no tool_calls field in the message, which is inconsistent for OpenAI-compatible consumers. The AI SDK may expect tool calls to execute when it sees this finish reason. Mapping to "stop" would be safer when no actual function call parts are present.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit ae95313. Configure here.
Author
|
Closing to reduce review noise from the accidental split. Consolidating the actual Vertex work into the remaining PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add native Google Cloud Vertex AI support for Grok using Application Default Credentials while preserving the existing native xAI API-key flow.
Stack / Review Diff
Stacked after #285 (
fix/npm-package-hygiene). GitHub shows this againstmainbecause the head branch is on a fork.Recommended review diff:
Local branch diff if both branches are present:
Merge order:
mainbefore merge.Scope
GROK_USE_VERTEX=1auth mode usinggoogle-auth-libraryand Google ADC.https://aiplatform.googleapis.comwith a configurable location path defaulting tous-central1.generateContent/streamGenerateContentrequests.system/assistant/tool messages to Vertex-compatibleuser/modelcontents.invalid_rapt/invalid_grantfailures.test:vertexlive smoke script.Review Notes
Primary files:
src/grok/vertex-adapter.tssrc/grok/vertex-auth.tssrc/grok/client.tssrc/utils/settings.tssrc/grok/tools.tssrc/agent/agent.tssrc/index.tstest-vertex-integration.tsRisk
Moderate to broad. The Vertex path touches provider setup, auth, request/response translation, streaming, and tool-calling. Native xAI remains gated on
GROK_USE_VERTEXbeing unset.Review focus:
Tested
git diff --checkbun run typecheckbun run test(47 files, 257 tests)bun run lint(passes with existing unrelated warnings)bun run buildbun run test -- src/grok/vertex-adapter.test.ts src/grok/vertex-auth.test.ts src/utils/settings-vertex.test.ts src/agent/auth.test.ts src/grok/client.test.ts(5 files, 38 tests)bun run test:vertexpassed against refreshed ADC.npm pack --dry-run --jsonreported 292 files, 348 KB package size, and zero local-runtime/test artifact matches.npm install -g <packed tgz>succeeded.which grokresolves to/home/aj/.local/bin/grok.grok --versionreturns1.1.5.installed-vertex-ok.pwd && ls -1 | head -5and returned local repo contents.Not Tested
Note
Medium Risk
Adds a new request/stream translation layer and alternate auth path (Google ADC) that changes how providers, tool calling, and headless flows behave when
GROK_USE_VERTEX=1, with potential edge cases in streaming/tool schema conversion. Native xAI flows remain mostly gated, but misconfiguration handling and feature availability branching could impact CLI behavior.Overview
Adds Google Cloud Vertex AI as an alternate runtime for Grok chat completions behind
GROK_USE_VERTEX=1, usinggoogle-auth-libraryApplication Default Credentials and a newvertex-adapterthat translates xAI/OpenAI-style chat requests/responses (including SSE streaming) to/from VertexgenerateContentAPIs.Updates auth/config handling (
src/utils/settings.ts,src/index.ts,src/agent/agent.ts) to treat Vertex project+ADC as valid “model auth”, provide clearer error messages, and block native xAI-only features in Vertex mode (e.g.--batch-api, X/web search tools, image/video generation, Telegram STT) with explicit guidance.Adds Vertex-focused tests and a live smoke script (
test:vertex), plus docs/env updates for the new configuration knobs (GROK_VERTEX_PROJECT_ID,GROK_VERTEX_LOCATION,GROK_VERTEX_BASE_URL,GROK_VERTEX_DISABLE_TOOLS) and packaging hygiene updates to exclude local/dev artifacts from npm.Reviewed by Cursor Bugbot for commit ae95313. Bugbot is set up for automated code reviews on this repo. Configure here.