Skip to content

Add vLLM local OpenAI-compatible server support#6

Draft
Emilien-Etadam wants to merge 6 commits into
YellowSakura:mainfrom
Emilien-Etadam:claude/cool-euler-AEPqA
Draft

Add vLLM local OpenAI-compatible server support#6
Emilien-Etadam wants to merge 6 commits into
YellowSakura:mainfrom
Emilien-Etadam:claude/cool-euler-AEPqA

Conversation

@Emilien-Etadam

Copy link
Copy Markdown

Summary

This PR adds support for vLLM as an LLM provider, allowing users to connect
to a local (or remote) vLLM inference server via its OpenAI-compatible API.

Changes

  • src/ts/llmProviders/impl/vllmProvider.ts — New provider implementing
    all standard methods (summarize, translate, explain, etc.) plus static
    getModels() to list available models from the server. Supports optional
    API key for secured deployments.
  • src/ts/llmProviders/providerFactory.ts — Register vllm in the
    provider map.
  • src/ts/helpers/configType.ts — Add vllm: { serviceUrl, model, apiKey }
    to the config interface.
  • src/html/partials/options-vllm.html — Options UI with service URL,
    optional API key, and a "Load models" button.
  • src/ts/options/optionsVllm.ts — Options handler: loads models on
    provider change or button click.
  • src/ts/options/options.ts — Wire up vLLM config save/restore.
  • src/html/options.html — Add vLLM to the provider dropdown and include
    the partial.
  • src/sass/options.scss — Show the vLLM fieldset when selected; include
    vLLM in the local-LLM timeout warning (consistent with Ollama/LMS).
  • src/manifest.json — Add http://localhost/* and http://127.0.0.1/*
    permissions so the extension can reach a local vLLM server.
  • src/locales/ — Add EN/IT strings for the vLLM section.

Notes

  • vLLM exposes the same /v1/chat/completions and /v1/models endpoints as
    OpenAI, so the implementation closely mirrors the existing LM Studio provider.
  • The API key field is optional — most local deployments run without one.
  • Default service URL is http://localhost:8000 (vLLM default port).

claude added 6 commits May 28, 2026 14:59
Integrates vLLM as a new LLM provider using its OpenAI-compatible
/v1/chat/completions API. Supports configurable service URL, optional
Bearer API key, and dynamic model listing via /v1/models.

- VllmProvider class following the same pattern as LmsProvider
- Config type, factory registration, options save/restore
- HTML partial with URL, optional API key and model selector
- Dynamic model loading script (optionsVllm.ts)
- EN and IT i18n messages
- Add http://localhost/* and http://127.0.0.1/* to manifest permissions
  so Thunderbird allows requests to local vLLM server without CORS issues
- VllmProvider, options UI, and locales were already implemented
- Include ai-mail-support.xpi for direct installation (forced past .gitignore)

https://claude.ai/code/session_015XBtaaGrf1fLtGWaqd6vYi
The vLLM provider section was hidden because the CSS data-provider selector
list was missing the vllm entry. Also added vllm to the local LLM timeout
warning, consistent with other local providers (lms, ollama).

https://claude.ai/code/session_015XBtaaGrf1fLtGWaqd6vYi
copyToEmailTop() was directly manipulating document.body DOM which is
unreliable in Thunderbird's compose editor and bypasses its internal state.
Now sends a message to the background script which uses the proper
messenger.compose.setComposeDetails() API, with support for both HTML
and plain-text compose modes.

https://claude.ai/code/session_015XBtaaGrf1fLtGWaqd6vYi
After receiving an AI suggestion, a pencil icon (✏) appears in the
action bar. Clicking it opens a small textarea where the user can type
a refinement instruction (e.g. "make it shorter", "be more formal").
Submitting sends the last AI response + the instruction back to the LLM
via applyCustomPrompt(), replacing the current response in the overlay.
Enter submits, Shift+Enter adds a newline.

https://claude.ai/code/session_015XBtaaGrf1fLtGWaqd6vYi
…al flow

actionsContainer was position:absolute which caused it to overlap
refineContainer when the latter was shown. Both are now wrapped in a
#amsFooter div in the normal document flow, eliminating the overlap.

https://claude.ai/code/session_015XBtaaGrf1fLtGWaqd6vYi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants