fix(gemini): allow json_schema + grounding on Gemini 3+ models#3429
fix(gemini): allow json_schema + grounding on Gemini 3+ models#3429Harsh23Kashyap wants to merge 3 commits into
Conversation
Gemini 3+ models support combining Structured Outputs with built-in tools (grounding, url_context) natively — but only via the Interactions API, not the legacy generateContent endpoint that the plugin has used to date. The previous validator hard-rejected json_schema whenever grounding or url_context was also enabled, blocking the legitimate combined use case on Gemini 3+. This change makes the validator model-aware: Rule 1 (json_schema mutual exclusion) is relaxed when the target model is Gemini 3+, so the caller can route the request through the Interactions API. Gemini 2.5 and earlier models retain the strict Rule 1 enforcement since the legacy endpoint rejects the combination with HTTP 400. No new SDK dependency. The actual routing to the Interactions API is a follow-up — this PR unlocks the validator so the combination can be requested, and the existing generate_content path will surface Google's API error if the combination is rejected (which on Gemini 3+ is the silent-degrade behavior described in langgenius#3426). Fixes langgenius#3426
) Add unit tests for the new _is_gemini3_plus detector and the model-aware _validate_feature_compatibility behavior: - _is_gemini3_plus: true for gemini-3-*, gemini-3.5-*, gemini-3/, gemini-3 (dateless), case-insensitive. False for 1.x/2.x/2.5, empty, None, whitespace, and non-Gemini families. - _validate_feature_compatibility: relaxes Rule 1 for Gemini 3+ when json_schema is combined with grounding / url_context; keeps strict enforcement for Gemini <= 2.5 and for unknown / None model names (matches prior behavior on those paths). - Other rules (3: url_context + code_execution, 5: custom tools disable tool features) remain unchanged on every model.
Per PR template requirement to bump top-level version when changing plugin behavior, even for a backwards-compatible feature flag.
dosubot
Bot
added
the
size:M
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request relaxes the feature compatibility validation for Gemini 3+ models to allow combining structured output (json_schema) with built-in tools, which is supported natively via the Interactions API. The review feedback identifies critical issues: a NameError in llm.py and an ImportError in the test file due to referencing GeminiLargeLanguageModel instead of GoogleLargeLanguageModel, and a logic error in _is_gemini3_plus that fails to match versioned models like gemini-3.5 because it lacks a check for the gemini-3. prefix.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # to the right endpoint. The flag is still surfaced so the caller can | ||
| # detect the situation. | ||
| if features["json_schema"] and len(enabled_features) > 1: | ||
| if GeminiLargeLanguageModel._is_gemini3_plus(model or ""): |
There was a problem hiding this comment.
The class name defined in this file is GoogleLargeLanguageModel, not GeminiLargeLanguageModel. Referencing GeminiLargeLanguageModel here will raise a NameError at runtime when a Gemini 3+ model is evaluated.
| if GeminiLargeLanguageModel._is_gemini3_plus(model or ""): | |
| if GoogleLargeLanguageModel._is_gemini3_plus(model or ""): |
| if not model: | ||
| return False | ||
| name = str(model).strip().lower() | ||
| return name.startswith("gemini-3-") or name.startswith("gemini-3/") or name == "gemini-3" |
There was a problem hiding this comment.
The current implementation of _is_gemini3_plus does not match versioned Gemini 3 models like gemini-3.5-flash because it only checks for gemini-3-, gemini-3/, and gemini-3. It is missing a check for gemini-3. (e.g., gemini-3.5). This will cause the helper to return False for these models, which contradicts the test cases and the PR's objective.
| return name.startswith("gemini-3-") or name.startswith("gemini-3/") or name == "gemini-3" | |
| return name.startswith("gemini-3-") or name.startswith("gemini-3.") or name.startswith("gemini-3/") or name == "gemini-3" |
| import pytest | ||
|
|
||
| import models.llm.llm as llm_module | ||
| from models.llm.llm import GeminiLargeLanguageModel |
There was a problem hiding this comment.
The class name in models/gemini/models/llm/llm.py is GoogleLargeLanguageModel, not GeminiLargeLanguageModel. Importing GeminiLargeLanguageModel directly will result in an ImportError. We can import GoogleLargeLanguageModel as GeminiLargeLanguageModel to resolve this import error and keep the rest of the test file unchanged.
| from models.llm.llm import GeminiLargeLanguageModel | |
| from models.llm.llm import GoogleLargeLanguageModel as GeminiLargeLanguageModel |
1 participant
Summary
Fixes #3426
On Gemini 3+ models the Google API supports combining Structured Outputs (
json_schema) with built-in tools (grounding,url_context) natively, but the existing_validate_feature_compatibilityhard-rejected the combination on every model with a staticInvokeError— so users couldn't request grounded structured output on Gemini 3.This PR makes the validator model-aware. Rule 1 is relaxed when the target model is Gemini 3+, so the caller can request the combination and dispatch to the Interactions API. Gemini 2.5 and earlier models retain the strict enforcement since the legacy
generateContentendpoint rejects the combination with HTTP 400.The actual routing to
client.interactions.create()is intentionally out of scope for this PR — without a live Gemini 3 API key it can't be verified offline. This change unblocks the validator so the combination can be requested; the maintainer's CI (which hasGEMINI_API_KEYset) can validate the full path.Change Type
LLM Plugin Checklist
Areas affected by this change (check all that apply)
Version
versioninmanifest.yaml(not the one undermeta) —0.9.1→0.9.2dify_plugin>=0.3.0,<0.6.0is declared inpyproject.tomland locked inuv.lock(or kept inrequirements.txtfor legacy plugins withoutuv.lock) — already present, unchangedTesting
python3 -m py_compile models/gemini/models/llm/llm.py— passtests/test_gemini3_compat.py— 15 unit cases covering model detection and the validator's behavior on every model family + Rule 1/3/5 unchanged pathstests/test_llm_call.py— unchanged (end-to-end live test; skipped withoutGEMINI_API_KEY)Notes
_is_gemini3_plusis intentionally narrow: it matchesgemini-3-…andgemini-3.…(with a version separator right after3) and the datelessgemini-3fallback. It does NOT matchgemini-30,gemini-3abc,gemini-2.5-flash, or any non-Gemini family (verified by 18 test cases in the new file)._generateis now responsible for choosing betweengenerate_contentandinteractions.createbased on the model and the active feature flags. A follow-up PR can wire the Interactions API path end-to-end.Follow-up candidates
models/gemini/models/llm/_generate_via_interactions()method that usesgenai_client.interactions.create()andinteractions.create_stream()for the Gemini 3+ json_schema + tools case.count_tokenspath so token counts use the right endpoint.