Skip to content

feat(llm): extract BaseAnthropicLLM, add base_url, export extension points#566

Open
matteomedioli wants to merge 8 commits into
matteo/anthropic-kwargs-bugfixfrom
matteo/base-anthropic-llm
Open

feat(llm): extract BaseAnthropicLLM, add base_url, export extension points#566
matteomedioli wants to merge 8 commits into
matteo/anthropic-kwargs-bugfixfrom
matteo/base-anthropic-llm

Conversation

@matteomedioli

@matteomedioli matteomedioli commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Description

Stacked on top of #565 (the http_client sync/async fix), this PR extends the same "shared base class, thin provider subclass" shape already used for OpenAI (BaseOpenAILLMOpenAILLM/AzureOpenAILLM) to Anthropic.

AnthropicLLM today mixes two concerns in one class: provider-shape logic (message building, JSON-schema conversion, response parsing) and SDK client construction. There's no supported way to reuse the provider-shape logic against a different Anthropic-compatible endpoint without copying the whole class.

Changes:

  • Extract BaseAnthropicLLM(LLMBase, abc.ABC) holding all of AnthropicLLM's message-building, schema-conversion, and response-parsing logic, moved verbatim — no behavior change.
  • AnthropicLLM becomes a thin subclass of BaseAnthropicLLM whose __init__ only constructs anthropic.Anthropic/anthropic.AsyncAnthropic (reusing the sync/async http_client split from fix(llm): route http_client to the matching sync/async Anthropic client #565).
  • Add an explicit, documented base_url: str | None = None keyword parameter to AnthropicLLM, passed through to both SDK clients — previously only reachable as an undocumented **kwargs passthrough accident.
  • Export both BaseAnthropicLLM and BaseOpenAILLM from neo4j_graphrag.llm and add them to __all__BaseOpenAILLM already existed but was never exported, so this also fixes that gap for OpenAI as a side effect of applying one consistent policy to both families.
  • Add a new docs page (docs/source/llm_extensibility.rst, linked from the docs index and API reference) documenting the http_client/base_url contract shared by both base classes, with a worked subclassing example.

Every provider SDK already supports pointing at a custom endpoint over a custom transport — the limitation was never at the SDK level, only in how these classes exposed (or didn't expose) that as a documented constructor contract. With this change, anyone running a model behind an Anthropic-compatible endpoint — a self-hosted gateway, an internal proxy, a compatibility layer, or simply a different region/deployment — can write a small subclass instead of forking AnthropicLLM or relying on **kwargs accidents:

from neo4j_graphrag.llm import BaseAnthropicLLM
import anthropic

class MyCustomAnthropicLLM(BaseAnthropicLLM):
    def __init__(self, model_name, **kwargs):
        super().__init__(model_name=model_name, **kwargs)
        self.client = anthropic.Anthropic(
            base_url="https://my-endpoint.example.com",
            http_client=my_configured_httpx_client,
        )
        self.async_client = anthropic.AsyncAnthropic(
            base_url="https://my-endpoint.example.com",
            http_client=my_configured_async_httpx_client,
        )

All message-building, schema conversion, and response parsing is inherited — nothing provider-shape-related needs reimplementing.

Only 2 of our 8 LLM providers follow this shape today — AnthropicLLM/BaseAnthropicLLM (this PR) and OpenAILLM/AzureOpenAILLM/BaseOpenAILLM (pre-existing). The other six — CohereLLM, MistralAILLM, OllamaLLM, BedrockLLM, GeminiLLM, and VertexAILLM — still mix provider-shape logic and SDK client construction in one class, with no Base<X>LLM split and no exported extension point. Worth a follow-up pass to align each of them where the provider's SDK actually has a client object to factor out; a couple, like Bedrock (boto3 session/endpoint_url, not an httpx client) and VertexAI (no per-instance client at all, driven by global vertexai.init()), will need a different-shaped extension point. Not proposing that fix here, just flagging the inconsistency as worth scoping separately.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Documentation update
  • Project configuration change

Complexity

Complexity: Medium

How Has This Been Tested?

  • Unit tests
  • E2E tests
  • Manual tests

BaseAnthropicLLM holds all previously-AnthropicLLM logic, unchanged in behavior; base_url verified passed through to both anthropic.Anthropic and anthropic.AsyncAnthropic; BaseAnthropicLLM/BaseOpenAILLM verified importable from neo4j_graphrag.llm and present in __all__; full tests/unit/llm/ suite green; mypy/ruff clean; new docs page builds without new Sphinx warnings.

Checklist

The following requirements should have been met (depending on the changes in the branch):

  • Documentation has been updated
  • Unit tests have been updated
  • E2E tests have been updated
  • Examples have been updated
  • New files have copyright header
  • CLA (https://neo4j.com/developer/cla/) has been signed
  • CHANGELOG.md updated if appropriate

…d schema logic

Moves the message-building, schema-conversion, and response-parsing logic
that lived directly on AnthropicLLM into a new BaseAnthropicLLM base class,
mirroring the BaseOpenAILLM/OpenAILLM split. AnthropicLLM becomes a subclass
responsible only for constructing the SDK clients, paving the way for
alternate Anthropic-compatible client implementations.
Allows routing Anthropic requests to a custom Anthropic-compatible
endpoint by passing base_url through to both the sync and async SDK
clients, alongside the existing http_client routing logic.
Add both base classes to neo4j_graphrag.llm's imports and __all__ so
they are documented, supported entry points for subclassing custom
LLM clients, with a test verifying the exports.
Covers the AnthropicLLM/BaseAnthropicLLM split: confirms AnthropicLLM
subclasses BaseAnthropicLLM, exercises invoke/schema logic through a
minimal BaseAnthropicLLM subclass, and verifies the new base_url
parameter reaches both the sync and async Anthropic SDK clients
(alone and combined with an explicit http_client).
…sses

Adds a Sphinx docs page explaining what BaseAnthropicLLM and BaseOpenAILLM
are for and how the http_client/base_url injection contract works, with a
worked example of subclassing BaseAnthropicLLM to reach a custom endpoint.
Links the page and adds API reference entries for both base classes.
Adds a changelog entry covering the new BaseAnthropicLLM base class, the
base_url parameter on AnthropicLLM, the BaseAnthropicLLM/BaseOpenAILLM
exports, and the new LLM extensibility docs page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant