fix(provider): gemma-4 tool calls 500 on nullable (array-typed) schema fields by Gajesh2007 · Pull Request #310 · Layr-Labs/d-inference

Gajesh2007 · 2026-06-11T00:14:13Z

Consumer-reported — reproduced on prod

A consumer running a daily tool-call matrix found every gemma-4 + tool definitions job failing (Runtime error: upper filter requires string, surfacing as 408/500), while gpt-oss and no-tool gemma runs pass.

Reproduced live: any tool parameter using the JSON-Schema nullable array form crashes gemma:

"city": { "type": ["string", "null"] }   →  HTTP 500: upper filter requires string

This is the standard shape Pydantic emits for every Optional[...] tool parameter, so any Python agent framework with optional fields hits it. (The coordinator retries 3×, then the consumer sees the failure — sometimes as a 408.)

Root cause — second class of DAR-130

The gemma template renders {{ value['type'] | upper }} per parameter. The DAR-130 normalizer (#303) only fills missing types (dict["type"] == nil); a type that is present but not a string (array form, or malformed values) passes through untouched and the Jinja upper filter throws on the non-string. My earlier DAR-130 verification passed because it only covered the missing-type/anyOf/additionalProperties shapes.

Fix

ToolSchemaNormalization.injectDefaultTypes now collapses any non-string type to a single renderable string:

["string","null"] → "string" (first concrete member, order-independent: ["null","integer"] → "integer")
["null"] → "null" (honest, still renders)
malformed (e.g. numeric) → structural inference (object/array/union-member/string — extracted into inferredType(for:), shared with the existing missing-type branch)

The key is never deleted — a node whose only content is its type wouldn't be refilled by the missing-type branch and would crash anyway.

Tests

5 new cases (nullable collapse, null-first ordering, null-only, nested object+items, malformed numeric) in ToolSchemaNormalizationTests; suite 6 → 11, full provider run 708 green.

Rollout

Provider-side → ships with the next provider release; fleet picks it up via auto-update. Until then, affected consumers can work around by avoiding nullable types in tool schemas (e.g. Pydantic Optional → required-with-default).

🤖 Generated with Claude Code

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

…0 class Consumer-reported and reproduced on prod: gemma-4 tool calls 500 with 'Runtime error: upper filter requires string' whenever a tool parameter uses the JSON-Schema nullable array form — "type": ["string","null"] — which Pydantic emits for every Optional[...] field. The DAR-130 normalizer only filled MISSING types (dict["type"] == nil); a present-but-non-string type sailed through to the template's {{ value['type'] | upper }} and crashed. The coordinator retries 3x and the consumer ultimately sees a 408/500. Fix: collapse any non-string type to one renderable string — first concrete (non-"null") member of an array type, the lone "null" when that is all the array declares, else structural inference (object/array/union-member/string, extracted into inferredType and shared with the missing-type branch). The key is never deleted: a node whose only content is its type would not be refilled and would crash anyway. 5 new tests: nullable collapse, null-first ordering, null-only, nested object/items, malformed numeric type. Full suite 708 green.

vercel · 2026-06-11T00:14:17Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
d-inference	Ready	Preview	Jun 11, 2026 12:36am
d-inference-console-ui-dev	Ready	Preview	Jun 11, 2026 12:36am
d-inference-landing	Ready	Preview	Jun 11, 2026 12:36am

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: REQUEST_CHANGES

Security — ✅ No issues found

Performance — 1 finding(s) (1 blocking)

🟡 [MEDIUM] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:86-87 — collapsedType calls inferredType which duplicates structural analysis already done in the main function
- Suggestion: Cache the structural analysis result or refactor to avoid duplicate checks for properties/items/unions

Type_diligence — ✅ No issues found

Additive_complexity — 1 finding(s)

🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:106-118 — inferredType duplicates existing logic from the missing-type branch
- Suggestion: Extract the existing structural inference logic (lines 95-105) into inferredType and reuse it in both places to eliminate duplication

2 finding(s) total, 1 blocking. Verdict: REQUEST_CHANGES.

🤖 Automated review by Centaur · DAR-186

ethenotethan · 2026-06-11T00:14:28Z

+        if let t = dict["type"], !(t is String) {
+            dict["type"] = collapsedType(t, in: dict)


🟡 [MEDIUM] ⚡ collapsedType calls inferredType which duplicates structural analysis already done in the main function

💡 Suggestion: Cache the structural analysis result or refactor to avoid duplicate checks for properties/items/unions

📊 Score: 2×4 = 8 · Category: Repeated work

Not accurate as written: the main function no longer performs structural analysis — that logic was extracted INTO inferredType (this diff did the extraction). The collapse branch (type present, non-string) and the missing-type branch (type == nil) are mutually exclusive, so inferredType runs at most once per node. No repeated work on any path.

ethenotethan · 2026-06-11T00:14:28Z

+            if let concrete = members.first(where: { $0 != "null" }) {
+                return concrete
+            }
+            if let nullOnly = members.first {
+                return nullOnly
+            }
+        }
+        return inferredType(for: dict)
+    }
+
+    /// Structural default for a schema node's `type`: object when it has
+    /// properties, array when it has items, a union member's type when it is an
+    /// anyOf/oneOf/allOf (skipping "null" — mislabelling a union as a string


🔵 [INFO] 🧩 inferredType duplicates existing logic from the missing-type branch

💡 Suggestion: Extract the existing structural inference logic (lines 95-105) into inferredType and reuse it in both places to eliminate duplication

📊 Score: 2×3 = 6 · Category: duplicate logic

This is exactly what the diff already does — inferredType IS the extraction of the former inline missing-type logic, reused by both branches.

…key (review) Reviewer finding: the gemma template natively renders the standard `nullable` key, so collapsing ["string","null"] -> "string" can be LOSSLESS — set nullable:true when a null member is collapsed away (never clobbering an explicit value). Also adds three ordering/coverage tests from review: union member with array type drives parent inference, top-level parameters node collapse, additionalProperties schema collapse. Suite 11 -> 14 green.

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: COMMENT

Security — ✅ No issues found

Performance — 2 finding(s)

🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:89-96 — Array type casting performed twice for same value
- Suggestion: Cache the compactMap result: let members = (t as? [Any])?.compactMap { $0 as? String } ?? []; then reuse members in both conditions
🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:113-123 — Duplicate array casting and compactMap in collapsedType function
- Suggestion: Extract the array casting and compactMap to a helper function or cache the result to avoid redundant operations

Type_diligence — ✅ No issues found

Additive_complexity — 3 finding(s)

🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:89-96 — Complex nullable detection logic could be simplified
- Suggestion: Consider extracting the nullable detection into a simple helper function or inline the logic more directly
🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:113-126 — collapsedType and inferredType have overlapping structural inference logic
- Suggestion: Consider having collapsedType call inferredType directly for the fallback case instead of duplicating the structural inference pattern
🔵 [INFO] provider-swift/Tests/ProviderCoreTests/ToolSchemaNormalizationTests.swift:105-108 — Verbose multi-line comment could be more concise
- Suggestion: Condense the comment to focus on the key issue: 'Array-typed nullable fields crash gemma template's | upper filter'

5 finding(s) total, 0 blocking. Verdict: COMMENT.

🤖 Automated review by Centaur · DAR-186

ethenotethan · 2026-06-11T00:22:57Z

+        if let t = dict["type"], !(t is String) {
+            let members = (t as? [Any])?.compactMap { $0 as? String } ?? []
+            if members.contains("null"), members.contains(where: { $0 != "null" }),
+                dict["nullable"] == nil {
+                dict["nullable"] = true
+            }
+            dict["type"] = collapsedType(t, in: dict)
+        }


🔵 [INFO] ⚡ Array type casting performed twice for same value

💡 Suggestion: Cache the compactMap result: let members = (t as? [Any])?.compactMap { $0 as? String } ?? []; then reuse members in both conditions

📊 Score: 2×3 = 6 · Category: Repeated work

Valid — fixed: members is computed once at the call site and passed into collapsedType(members:in:), so the nullable check and the collapse share a single compactMap.

ethenotethan · 2026-06-11T00:22:57Z

+        if let members = (value as? [Any])?.compactMap({ $0 as? String }) {
+            if let concrete = members.first(where: { $0 != "null" }) {
+                return concrete
+            }
+            if let nullOnly = members.first {
+                return nullOnly
+            }
+        }
+        return inferredType(for: dict)
+    }
+


🔵 [INFO] ⚡ Duplicate array casting and compactMap in collapsedType function

💡 Suggestion: Extract the array casting and compactMap to a helper function or cache the result to avoid redundant operations

📊 Score: 2×2 = 4 · Category: Repeated work

Fixed in the same commit — collapsedType now receives the pre-extracted members and does no casting of its own.

ethenotethan · 2026-06-11T00:22:57Z

+        if let t = dict["type"], !(t is String) {
+            let members = (t as? [Any])?.compactMap { $0 as? String } ?? []
+            if members.contains("null"), members.contains(where: { $0 != "null" }),
+                dict["nullable"] == nil {
+                dict["nullable"] = true
+            }
+            dict["type"] = collapsedType(t, in: dict)
+        }


🔵 [INFO] 🧩 Complex nullable detection logic could be simplified

💡 Suggestion: Consider extracting the nullable detection into a simple helper function or inline the logic more directly

📊 Score: 2×3 = 6 · Category: over-abstraction

Addressed by the same restructure: one members extraction feeding both the nullable check and the collapse; no helper needed beyond that.

ethenotethan · 2026-06-11T00:22:57Z

+        if let members = (value as? [Any])?.compactMap({ $0 as? String }) {
+            if let concrete = members.first(where: { $0 != "null" }) {
+                return concrete
+            }
+            if let nullOnly = members.first {
+                return nullOnly
+            }
+        }
+        return inferredType(for: dict)
+    }
+
+    /// Structural default for a schema node's `type`: object when it has
+    /// properties, array when it has items, a union member's type when it is an
+    /// anyOf/oneOf/allOf (skipping "null" — mislabelling a union as a string


🔵 [INFO] 🧩 collapsedType and inferredType have overlapping structural inference logic

💡 Suggestion: Consider having collapsedType call inferredType directly for the fallback case instead of duplicating the structural inference pattern

📊 Score: 2×2 = 4 · Category: duplicate logic

collapsedType already calls inferredType directly for its fallback (last line of the function) — there is no duplicated structural inference between them.

ethenotethan · 2026-06-11T00:22:57Z

+    // MARK: - Array-typed (nullable) `type` values — the second DAR-130 class.
+    // `"type": ["string","null"]` is what Pydantic emits for Optional[...] tool
+    // parameters; the gemma template's `| upper` crashed on the list ("upper
+    // filter requires string", reproduced on prod 2026-06-10).


🔵 [INFO] 🧩 Verbose multi-line comment could be more concise

💡 Suggestion: Condense the comment to focus on the key issue: 'Array-typed nullable fields crash gemma template's | upper filter'

📊 Score: 1×2 = 2 · Category: over-abstraction

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c59aff871

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-11T00:26:31Z

+            if let concrete = members.first(where: { $0 != "null" }) {
+                return concrete


Preserve non-null union schemas

When a valid schema uses a multi-type array such as "type": ["integer", "string"] rather than the nullable two-member form, this collapse keeps only the first concrete type. ProviderLoop.decodeOpenAIRequest applies this normalizer unconditionally before the request is rendered into any model's chat template, so every model sees a narrowed tool schema and can be prompted to emit the wrong argument type for those union-typed parameters. Please limit this rewrite to nullable arrays or preserve non-null unions another way.

Useful? React with 👍 / 👎.

Half-agree, but the suggested gating would reintroduce the crash: gemma's template throws on ANY array-valued type — ["integer","string"] crashes | upper exactly like the nullable form — so non-null unions must be collapsed too; we cannot leave them intact. On fidelity: true multi-member non-null unions are rare in practice (OpenAI strict schemas only permit the null-union form), the first concrete member plus the preserved enum/description keeps the prompt useful, and nullable unions are now lossless via the template-native nullable key. Per-template gating (only normalize for templates that need it) is a reasonable follow-up, noted — but the decode boundary currently has no knowledge of the target model's template, and a crash-proof default is the right trade for a 500 consumers are hitting daily.

collapsedType now takes the pre-extracted string members instead of re-casting the raw value — the nullable check and the collapse share one compactMap.

ethenotethan

Automated Code Review — Layr-Labs/d-inference#

Verdict: COMMENT

Security — ✅ No issues found

Performance — 1 finding(s)

🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:89-96 — compactMap creates new array for every type field without pre-allocation
- Suggestion: Pre-allocate array capacity or use lazy evaluation if members array is typically small

Type_diligence — ✅ No issues found

Additive_complexity — 3 finding(s)

🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:89-96 — Complex nullable detection logic could be simplified with early return
- Suggestion: Extract nullable detection to a separate function or use early return to reduce nesting and improve readability
🔵 [INFO] provider-swift/Sources/ProviderCore/Inference/ToolSchemaNormalization.swift:113-125 — Type inference logic duplicated between collapsedType and inferredType functions
- Suggestion: Consider having collapsedType delegate directly to inferredType for the fallback case instead of duplicating the structural inference logic
🔵 [INFO] provider-swift/Tests/ProviderCoreTests/ToolSchemaNormalizationTests.swift:105-108 — Verbose multi-line comment could be more concise
- Suggestion: Condense the comment to focus on the key issue: array-typed fields crash gemma template's upper filter

4 finding(s) total, 0 blocking. Verdict: COMMENT.

🤖 Automated review by Centaur · DAR-186

ethenotethan · 2026-06-11T00:36:04Z

+        if let t = dict["type"], !(t is String) {
+            let members = (t as? [Any])?.compactMap { $0 as? String } ?? []
+            if members.contains("null"), members.contains(where: { $0 != "null" }),
+                dict["nullable"] == nil {
+                dict["nullable"] = true
+            }
+            dict["type"] = collapsedType(members: members, in: dict)
+        }


🔵 [INFO] ⚡ compactMap creates new array for every type field without pre-allocation

💡 Suggestion: Pre-allocate array capacity or use lazy evaluation if members array is typically small

📊 Score: 2×3 = 6 · Category: Unbounded allocations

ethenotethan · 2026-06-11T00:36:04Z

+        if let t = dict["type"], !(t is String) {
+            let members = (t as? [Any])?.compactMap { $0 as? String } ?? []
+            if members.contains("null"), members.contains(where: { $0 != "null" }),
+                dict["nullable"] == nil {
+                dict["nullable"] = true
+            }
+            dict["type"] = collapsedType(members: members, in: dict)
+        }


🔵 [INFO] 🧩 Complex nullable detection logic could be simplified with early return

💡 Suggestion: Extract nullable detection to a separate function or use early return to reduce nesting and improve readability

📊 Score: 2×3 = 6 · Category: over-abstraction

ethenotethan · 2026-06-11T00:36:04Z

+    private static func collapsedType(members: [String], in dict: [String: Any]) -> String {
+        if let concrete = members.first(where: { $0 != "null" }) {
+            return concrete
+        }
+        if let nullOnly = members.first {
+            return nullOnly
+        }
+        return inferredType(for: dict)
+    }
+
+    /// Structural default for a schema node's `type`: object when it has
+    /// properties, array when it has items, a union member's type when it is an
+    /// anyOf/oneOf/allOf (skipping "null" — mislabelling a union as a string


🔵 [INFO] 🧩 Type inference logic duplicated between collapsedType and inferredType functions

💡 Suggestion: Consider having collapsedType delegate directly to inferredType for the fallback case instead of duplicating the structural inference logic

📊 Score: 2×2 = 4 · Category: duplicate logic

ethenotethan · 2026-06-11T00:36:04Z

+    // MARK: - Array-typed (nullable) `type` values — the second DAR-130 class.
+    // `"type": ["string","null"]` is what Pydantic emits for Optional[...] tool
+    // parameters; the gemma template's `| upper` crashed on the list ("upper
+    // filter requires string", reproduced on prod 2026-06-10).


🔵 [INFO] 🧩 Verbose multi-line comment could be more concise

💡 Suggestion: Condense the comment to focus on the key issue: array-typed fields crash gemma template's upper filter

📊 Score: 1×2 = 2 · Category: over-abstraction

…malization, capability floors Root-caused from the OpenRouter partner incident (vision+tools requests 500ing fleet-wide, 'provider disconnected' after a bare role delta): - Deferred commit: dispatch no longer commits on boilerplate chunks (role-only delta / response.created). Pre-content provider errors and disconnects retry invisibly on another provider; the speculative TTFT race now measures real first tokens instead of being defeated by instant boilerplate. In-band SSE errors only after content has flowed. - Inference-error circuit breaker: 2x 5xx within 60s puts the (provider, model) pair on a 5-min routing cooldown; success clears. - Version-diverse retry: after a 5xx, the next attempt soft-prefers a provider running a different binary version (never fail-closed). - Capability floors: tool-bearing requests route only to providers >= 0.6.3 (provider-side schema normalization, #310) whose models do not report template_render_ok=false. - Coordinator-side tool-schema normalization (Go port of the Swift ToolSchemaNormalization): nullable array types collapsed, missing types inferred, applied pre-encryption so lagging providers never see template-crashing schemas. - Provider scan-time chat-template render self-check (Swift): renders each model's template against canonical fixtures (multimodal parts, normalized tools, tool responses) and advertises template_render_ok tri-state on the wire. - DD: inference.in_band_error, routing.cooldown_entered, inference.dispatches{status:retry_precontent}. Coordinator: go test ./... green (16 pkgs), -race clean on api+registry, linux/amd64 cross-compile OK. Provider: full non-live Swift suite green (695 swift-testing + XCTest incl. 18 new).

Gajesh2007 requested a deployment to benchmarks June 11, 2026 00:14 — with GitHub Actions Waiting

ethenotethan requested changes Jun 11, 2026

View reviewed changes

vercel Bot deployed to Preview – d-inference-landing June 11, 2026 00:14 View deployment

vercel Bot deployed to Preview – d-inference June 11, 2026 00:15 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev June 11, 2026 00:15 View deployment

Gajesh2007 requested a deployment to benchmarks June 11, 2026 00:22 — with GitHub Actions Waiting

ethenotethan reviewed Jun 11, 2026

View reviewed changes

vercel Bot deployed to Preview – d-inference-landing June 11, 2026 00:22 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev June 11, 2026 00:23 View deployment

vercel Bot deployed to Preview – d-inference June 11, 2026 00:23 View deployment

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

refactor(provider): compute array-type members once (review nit)

1d3df1e

collapsedType now takes the pre-extracted string members instead of re-casting the raw value — the nullable check and the collapse share one compactMap.

Gajesh2007 requested a deployment to benchmarks June 11, 2026 00:35 — with GitHub Actions Waiting

ethenotethan reviewed Jun 11, 2026

View reviewed changes

vercel Bot deployed to Preview – d-inference-landing June 11, 2026 00:36 View deployment

vercel Bot deployed to Preview – d-inference-console-ui-dev June 11, 2026 00:36 View deployment

vercel Bot deployed to Preview – d-inference June 11, 2026 00:36 View deployment

Gajesh2007 merged commit 08f800c into master Jun 11, 2026
71 of 74 checks passed

Gajesh2007 deleted the fix/tool-schema-array-type branch June 11, 2026 00:55

Gajesh2007 mentioned this pull request Jun 11, 2026

feat(routing): pre-content failover, error cooldowns, tool-schema normalization, capability floors #322

Merged

		if let t = dict["type"], !(t is String) {
		dict["type"] = collapsedType(t, in: dict)

		if let concrete = members.first(where: { $0 != "null" }) {
		return concrete

Conversation

Gajesh2007 commented Jun 11, 2026 • edited by blacksmith-sh Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Consumer-reported — reproduced on prod

Root cause — second class of DAR-130

Fix

Tests

Rollout

Uh oh!

vercel Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — 1 finding(s) (1 blocking)

Type_diligence — ✅ No issues found

Additive_complexity — 1 finding(s)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — 2 finding(s)

Type_diligence — ✅ No issues found

Additive_complexity — 3 finding(s)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ethenotethan left a comment

Choose a reason for hiding this comment

Automated Code Review — Layr-Labs/d-inference#

Security — ✅ No issues found

Performance — 1 finding(s)

Type_diligence — ✅ No issues found

Additive_complexity — 3 finding(s)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Gajesh2007 commented Jun 11, 2026 •

edited by blacksmith-sh Bot

Loading

vercel Bot commented Jun 11, 2026 •

edited

Loading