feat: add LiteLLM as AI gateway provider

[RheagalFire]

Summary

Add LiteLLM as a built-in provider, giving users access to 100+ LLM providers (Anthropic, Azure, Bedrock, Vertex, Groq, Ollama, etc.) through a single OpenAI-compatible proxy.

LiteLLM proxy runs locally and translates requests to any supported provider's native format. Since it speaks OpenAI protocol, it reuses the existing OPENAI_CHAT transport with zero new transport code.

Changes:

• src/row_bot/providers/catalog.py - added litellm ProviderDefinition (base_url: http://localhost:4000/v1, transport: OPENAI_CHAT)
• src/row_bot/providers/runtime.py - added litellm to configured provider list
• tests/test_litellm_provider.py - 3 unit tests

Type of change

• Bug fix
• Feature
• Docs
• Refactor
• Test-only change
• Build / CI / release tooling

Risk area

• Agent / prompts / tools
• Designer
• Memory / knowledge graph
• Channels / external integrations
• Installers / auto-update
• UI only
• Other

Testing

• I ran python tests/test_suite.py
• I added or updated tests
• I manually tested the affected user flow
• Not applicable, docs-only change

python tests/test_suite.py: 1143 pass, 103 fail (all pre-existing failures from missing optional deps nicegui and langgraph.checkpoint.sqlite, unrelated to this change).

tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_registered PASSED
tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_definition PASSED
tests/test_litellm_provider.py::TestLiteLLMProvider::test_provider_uses_openai_compatible_transport PASSED
3 passed in 0.26s

Existing provider catalog tests: 41 passed (1 pre-existing nicegui failure).

Release notes

• User-visible change, release notes needed

Users can now select LiteLLM as a provider and point it at their LiteLLM proxy (http://localhost:4000) to access any of 100+ supported LLM providers.

Checklist

• Branch is based on latest main
• No direct secrets, API keys, local paths, or private data included
• The change is focused and does not include unrelated cleanup
• Windows/macOS behavior considered where relevant

[RheagalFire]

@siddsachar

[siddsachar]

Thanks for the PR. I like the direction, but this is not enough for LiteLLM to be a first-class Row-Bot provider yet.

Right now the change only adds a provider definition, includes it in list_configured_provider_ids(), and adds registration tests. That creates a provider-shaped entry, but it does not wire the provider through the actual Row-Bot provider lifecycle.

A proper first-class provider needs all of the following:

1. Runtime construction

   • Add an explicit litellm branch in src/row_bot/providers/runtime.py.
   • It should use ChatOpenAICompatible, not fall through to the OpenRouter fallback.
   • Pass an endpoint profile like:
     • provider_id: "litellm"
     • display_name: "LiteLLM"
     • configurable base_url
     • transport: "openai_chat"
     • profile: "litellm"

2. Auth and configuration

   • Add a provider secret mapping, likely LITELLM_API_KEY, in src/row_bot/providers/auth_store.py.
   • LiteLLM can run with or without proxy auth, so first-class setup should support both API-key and no-auth/local proxy modes.
   • The base URL must be configurable. http://localhost:4000/v1 is a useful default, but it cannot be the only supported URL.
   • LiteLLM virtual keys and custom key headers should be considered, since LiteLLM commonly uses proxy keys rather than upstream provider keys.

3. Settings and setup wizard

   • Add user-facing controls in Settings -> Providers.
   • Add setup-wizard support if this is intended to appear beside OpenAI/OpenRouter/Atlas/etc.
   • Saving the key/base URL should trigger model catalog refresh.

4. Model discovery and catalog refresh

   • Add fetch_cloud_models("litellm").
   • Add LiteLLM to refresh_cloud_models().
   • Populate provider-qualified cache rows: model:litellm:<model-id>.
   • Use /v1/models for model discovery.
   • Where available, use LiteLLM metadata endpoints such as /model_group/info or /model/info to capture context windows, vision support, tool support, etc.
   • Preserve or prune stale LiteLLM cache rows safely.

5. Capability and readiness behavior

   • Do not assume every LiteLLM-routed model is agent/tool ready.
   • LiteLLM can route to many upstream providers and model types, so Row-Bot needs metadata or a probe before advertising Agent readiness.
   • Unknown tool support should fall back to Chat Only, similar to our OpenRouter/custom endpoint safety posture.
   • If a probe is added, it should verify chat, structured tool calling, tool-result replay, and ideally streaming tool-call behavior.

6. Tests
   Please add coverage comparable to the existing Atlas Cloud, OpenCode, and custom endpoint provider tests:

   • provider definition
   • API key/no-auth status
   • settings/setup wiring
   • runtime construction uses ChatOpenAICompatible
   • model refresh from LiteLLM /v1/models
   • provider-qualified cache rows
   • capability classification from LiteLLM metadata
   • readiness: agent-ready when tools are proven, chat-only when tool support is unknown/false
   • model catalog/status-card behavior

Until this is implemented, LiteLLM is already supported in Row-Bot through the existing Custom OpenAI-compatible endpoint flow. Users can go to Settings -> Providers -> Custom Endpoints, add their LiteLLM base URL, and choose the LiteLLM compatibility profile. That path already supports configurable base URLs, optional auth, model refresh, LiteLLM metadata probing, capability snapshots, and endpoint probes.

So I’m going to request changes for now. Happy to review a fuller version that makes LiteLLM a true first-class provider rather than a thinner duplicate of the existing custom endpoint profile.