feat(model-runtime): thread request_metadata through to model runtimes

[ryuta-kobayashi-ug]

What

Thread request_metadata from the public model invoke() surfaces down to model_runtime.invoke_* for all six model types, and add it to the corresponding runtime protocols.

Why

Dify needs to pass request-level metadata such as the originating app_id through Graphon to model runtimes and plugins.

That enables provider-side cost attribution and observability for providers that accept request metadata, including Amazon Bedrock requestMetadata, OpenAI and Azure OpenAI metadata, and Vertex AI labels.

Changes

• Added request_metadata: Mapping[str, object] | None = None to the model runtime protocol methods for LLM, text embedding, rerank, TTS, speech-to-text, and moderation.
• Forwarded request_metadata from the public model wrappers to the corresponding model_runtime.invoke_* calls.
• Forwarded request_metadata through the LLM callback path so observability hooks receive the same request metadata.
• Kept request_metadata keyword-only on the public model wrapper surfaces to avoid accidental positional argument shifts.
• Updated tests and runtime stubs to use the request_metadata name consistently.

Compatibility

request_metadata defaults to None, so existing callers that do not pass request metadata continue to behave the same.

Runtime implementations need to accept the new keyword parameter if they implement the updated protocol surface.

Out of scope

The structured-output runtime entry invoke_llm_with_structured_output() is intentionally not touched here and can follow separately if needed.

Tests

• uv run ruff format src/graphon/model_runtime tests/model_runtime/test_model_dispatch.py
• uv run ruff check src/graphon/model_runtime tests/model_runtime/test_model_dispatch.py
• uv run ty check
• uv run pytest tests/model_runtime/test_model_dispatch.py tests/test_protocol_abstract_contracts.py tests/test_protocols_exports.py tests/model_runtime/test_text_embedding_model.py
• git diff --check

Related

• feat(api): pass app_id to model plugins for provider-side cost attribution dify#35859

[github-actions]

All contributors on this pull request have signed the CLA.
_{Posted by the CLA Assistant Lite bot.}

[ryuta-kobayashi-ug]

I have read the CLA Document and I hereby sign the CLA

[Copilot]

Pull request overview

This PR threads the existing invocation_context metadata from the public model invoke() methods down into each model runtime protocol surface, enabling runtimes/providers to receive per-invocation metadata for attribution/observability.

Changes:

• Added invocation_context: Mapping[str, object] | None = None to the six model runtime protocols (invoke_llm, invoke_text_embedding, invoke_multimodal_embedding, invoke_rerank, invoke_multimodal_rerank, invoke_tts, invoke_speech_to_text, invoke_moderation).
• Updated the corresponding base model wrappers to forward invocation_context into model_runtime.invoke_* calls (and for LLM, also through callback plumbing).
• Extended dispatch tests to ensure invocation_context reaches the runtime and defaults to None.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/model_runtime/test_model_dispatch.py	Updates runtime stubs and adds assertions that invocation_context is forwarded and defaults to None.
src/graphon/model_runtime/protocols/llm_runtime.py	Adds invocation_context to the LLM runtime protocol method overloads/abstract method.
src/graphon/model_runtime/protocols/text_embedding_runtime.py	Adds invocation_context to text and multimodal embedding runtime protocol methods.
src/graphon/model_runtime/protocols/rerank_runtime.py	Adds invocation_context to rerank runtime protocol methods.
src/graphon/model_runtime/protocols/tts_runtime.py	Adds invocation_context to the TTS runtime protocol method.
src/graphon/model_runtime/protocols/speech_to_text_runtime.py	Adds invocation_context to the speech-to-text runtime protocol method.
src/graphon/model_runtime/protocols/moderation_runtime.py	Adds invocation_context to the moderation runtime protocol method.
src/graphon/model_runtime/model_providers/base/large_language_model.py	Threads invocation_context through runtime invocation and callback triggers.
src/graphon/model_runtime/model_providers/base/text_embedding_model.py	Forwards invocation_context into embedding runtime invocations.
src/graphon/model_runtime/model_providers/base/rerank_model.py	Forwards invocation_context into rerank runtime invocations (text + multimodal).
src/graphon/model_runtime/model_providers/base/tts_model.py	Forwards invocation_context into TTS runtime invocation.
src/graphon/model_runtime/model_providers/base/speech2text_model.py	Forwards invocation_context into speech-to-text runtime invocation.
src/graphon/model_runtime/model_providers/base/moderation_model.py	Forwards invocation_context into moderation runtime invocation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.