feat: add LiteLLM AI gateway adapter#729
Open
RheagalFire wants to merge 2 commits into
Open
Conversation
Contributor
📝 WalkthroughWalkthroughAdds a new ChangesLiteLLM Integration Package
Sequence DiagramsequenceDiagram
participant User
participant litellmText as litellmText()
participant getLiteLLMApiKeyFromEnv as getLiteLLMApiKeyFromEnv()
participant createLitellmText as createLitellmText()
participant withLiteLLMDefaults as withLiteLLMDefaults()
participant LiteLLMTextAdapter as LiteLLMTextAdapter
User->>litellmText: litellmText(model, config?)
litellmText->>getLiteLLMApiKeyFromEnv: read LITELLM_API_KEY
getLiteLLMApiKeyFromEnv-->>litellmText: apiKey
litellmText->>createLitellmText: createLitellmText(model, apiKey, config)
createLitellmText->>withLiteLLMDefaults: withLiteLLMDefaults(config)
withLiteLLMDefaults-->>createLitellmText: config with baseURL
createLitellmText->>LiteLLMTextAdapter: new LiteLLMTextAdapter(config, model)
LiteLLMTextAdapter-->>createLitellmText: adapter instance
createLitellmText-->>litellmText: LiteLLMTextAdapter
litellmText-->>User: LiteLLMTextAdapter
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (3)
packages/ai-litellm/package.json (1)
52-52: ⚡ Quick winInconsistent workspace protocol for internal peer dependency.
The guideline specifies using
workspace:*for internal package dependencies, but this peer dependency usesworkspace:^. For consistency with the regular dependencies (lines 56-57) and the coding guidelines, consider usingworkspace:*here as well.📝 Suggested change for consistency
- "`@tanstack/ai`": "workspace:^", + "`@tanstack/ai`": "workspace:*",As per coding guidelines, "Use the
workspace:*protocol for internal package dependencies in package.json".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/package.json` at line 52, The peer dependency entry "`@tanstack/ai`": "workspace:^" is using the wrong workspace protocol; update this dependency to use the internal workspace protocol "workspace:*" so it matches the other internal dependencies and the project guideline (look for the "`@tanstack/ai`" key in packages/ai-litellm/package.json and replace the version string).Source: Coding guidelines
packages/ai-litellm/src/adapters/text.ts (1)
24-26: 💤 Low valueConsider using
unknowninstead ofanyfor provider options.Line 26 uses
Record<string, any>as a type parameter. While this provides maximum flexibility, usingRecord<string, unknown>would provide better type safety by requiring explicit type assertions when accessing provider options, without losing functionality.💡 Suggested type safety improvement
export class LiteLLMTextAdapter extends OpenAIBaseChatCompletionsTextAdapter< string, - Record<string, any>, + Record<string, unknown>, readonly [], Record<string, never>, readonly []🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/src/adapters/text.ts` around lines 24 - 26, The generic parameter on LiteLLMTextAdapter uses Record<string, any> which weakens type safety; update the adapter declaration that extends OpenAIBaseChatCompletionsTextAdapter (the type parameter currently written as Record<string, any>) to use Record<string, unknown> so provider options are typed as unknown and require explicit assertions or narrowing when accessed.packages/ai-litellm/src/utils/client.ts (1)
15-21: 💤 Low valueConsider narrowing the catch clause or re-throwing unexpected errors.
The empty catch clause catches all errors from
getApiKeyFromEnv, not just the expected missing environment variable error. While this is unlikely to cause issues in practice (sincegetApiKeyFromEnvhas a single failure mode), catching all errors without inspection can mask unexpected failures.Consider either checking the error type before re-throwing, or documenting that all errors from
getApiKeyFromEnvare treated as missing API key errors.🛡️ Alternative approach with error inspection
export function getLiteLLMApiKeyFromEnv(): string { try { return getApiKeyFromEnv('LITELLM_API_KEY') - } catch { + } catch (error) { + // getApiKeyFromEnv only throws when env var is missing, but be explicit throw new Error( 'LITELLM_API_KEY is required. Please set it in your environment variables or use createLitellmText() with an explicit API key.', ) } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/src/utils/client.ts` around lines 15 - 21, The current catch block around getApiKeyFromEnv('LITELLM_API_KEY') swallows all errors; change it to catch the error as a variable, inspect it (e.g., check instanceof or error.message content that indicates "missing API key" from getApiKeyFromEnv), and only convert that specific case to the user-facing Error about LITELLM_API_KEY; for any other unexpected error re-throw the original error so failures in getApiKeyFromEnv are not masked. Ensure references to getApiKeyFromEnv and the existing message mentioning createLitellmText() remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/ai-litellm/package.json`:
- Line 52: The peer dependency entry "`@tanstack/ai`": "workspace:^" is using the
wrong workspace protocol; update this dependency to use the internal workspace
protocol "workspace:*" so it matches the other internal dependencies and the
project guideline (look for the "`@tanstack/ai`" key in
packages/ai-litellm/package.json and replace the version string).
In `@packages/ai-litellm/src/adapters/text.ts`:
- Around line 24-26: The generic parameter on LiteLLMTextAdapter uses
Record<string, any> which weakens type safety; update the adapter declaration
that extends OpenAIBaseChatCompletionsTextAdapter (the type parameter currently
written as Record<string, any>) to use Record<string, unknown> so provider
options are typed as unknown and require explicit assertions or narrowing when
accessed.
In `@packages/ai-litellm/src/utils/client.ts`:
- Around line 15-21: The current catch block around
getApiKeyFromEnv('LITELLM_API_KEY') swallows all errors; change it to catch the
error as a variable, inspect it (e.g., check instanceof or error.message content
that indicates "missing API key" from getApiKeyFromEnv), and only convert that
specific case to the user-facing Error about LITELLM_API_KEY; for any other
unexpected error re-throw the original error so failures in getApiKeyFromEnv are
not masked. Ensure references to getApiKeyFromEnv and the existing message
mentioning createLitellmText() remain unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 60ddb427-349a-40ad-b1f4-0a7e5b16d8db
📒 Files selected for processing (7)
packages/ai-litellm/package.jsonpackages/ai-litellm/src/adapters/text.tspackages/ai-litellm/src/index.tspackages/ai-litellm/src/utils/client.tspackages/ai-litellm/src/utils/index.tspackages/ai-litellm/tsconfig.jsonpackages/ai-litellm/vite.config.ts
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
testing/e2e/src/lib/feature-support.ts (1)
11-149:⚠️ Potential issue | 🔴 CriticalFix LiteLLM E2E wiring:
provider: 'litellm'is marked supported, but the harness has no LiteLLM adapter.
testing/e2e/src/lib/feature-support.ts+testing/e2e/tests/test-matrix.tsincludelitellmfor the added feature keys, buttesting/e2e/src/lib/providers.ts’screateTextAdapter()has nolitellmentry indefaultModelsor thefactoriesmap;testing/e2e/src/routes/api.chat.tsalways callscreateTextAdapter(provider, ...), so this will fail at runtime forprovider === 'litellm'.testing/e2e/src/lib/llmock-server.ts’sLLMockrecord config doesn’t includelitellm, so recording/fixture generation for LiteLLM can’t work either.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@testing/e2e/src/lib/feature-support.ts` around lines 11 - 149, The test harness marks provider 'litellm' supported but the runtime wiring is missing; add a LiteLLM adapter and fixtures: update createTextAdapter (in testing/e2e/src/lib/providers.ts) by adding 'litellm' to defaultModels and the factories map (implement or reference a LiteLLM TextAdapter factory) so createTextAdapter(provider, ...) in routes/api.chat.ts can instantiate it, and update the LLMock record config (in testing/e2e/src/lib/llmock-server.ts) to include a 'litellm' entry so recording/fixture generation works for LiteLLM; ensure the adapter follows the same interface used by other providers and reuse existing adapter patterns for naming and behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@testing/e2e/src/lib/feature-support.ts`:
- Around line 11-149: The test harness marks provider 'litellm' supported but
the runtime wiring is missing; add a LiteLLM adapter and fixtures: update
createTextAdapter (in testing/e2e/src/lib/providers.ts) by adding 'litellm' to
defaultModels and the factories map (implement or reference a LiteLLM
TextAdapter factory) so createTextAdapter(provider, ...) in routes/api.chat.ts
can instantiate it, and update the LLMock record config (in
testing/e2e/src/lib/llmock-server.ts) to include a 'litellm' entry so
recording/fixture generation works for LiteLLM; ensure the adapter follows the
same interface used by other providers and reuse existing adapter patterns for
naming and behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f15d9991-7dff-4f86-98ea-d2a7d90976cb
📒 Files selected for processing (4)
.changeset/add-litellm-adapter.mdtesting/e2e/src/lib/feature-support.tstesting/e2e/src/lib/types.tstesting/e2e/tests/test-matrix.ts
✅ Files skipped from review due to trivial changes (1)
- testing/e2e/tests/test-matrix.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Adds
@tanstack/ai-litellm, a tree-shakeable adapter for the LiteLLM AI gateway proxy. Gives users access to 100+ LLM providers (OpenAI, Anthropic, Google, Azure, AWS Bedrock, Ollama, Groq, Mistral, and more) through a single adapter.Follows the same pattern as
ai-groqandai-grok: extendsOpenAIBaseChatCompletionsTextAdapterwith abaseURLoverride pointing at the LiteLLM proxy. New package atpackages/ai-litellm/withLiteLLMTextAdapter,createLitellmText(), andlitellmText()factory functions. ReadsLITELLM_API_KEYfrom env, defaults proxy URL tohttp://localhost:4000/v1.LiteLLM added to E2E feature-support matrix and test-matrix for all OpenAI-compatible features (chat, tool-calling, structured output, etc.).
Example usage:
✅ Checklist
pnpm run test:pr. (pnpm installblocked by upstreamchokidar@4.0.3trust downgrade in the svelte example, unrelated to this change.)🚀 Release Impact