Add WarnExtraModel: log warnings for unknown fields in API request models#2015
Draft
imapotato123 wants to merge 1 commit into
Draft
Add WarnExtraModel: log warnings for unknown fields in API request models#2015imapotato123 wants to merge 1 commit into
imapotato123 wants to merge 1 commit into
Conversation
Pydantic v2 BaseModel defaults to extra="ignore", silently dropping unknown fields on request bodies. For internal data structures this is fine, but for public API request bodies it makes mistyped or out-of-spec fields invisible: the request is accepted, the offending field is dropped, and the request runs with whatever defaults the matching declared fields had. We hit this in cluster validation: a mistyped instance_meta field on /place_instance was silently dropped, falling back to the default MlxRing instead of the requested MlxJaccl, with no log line. Symptom was lower decode throughput, no error. This change adds a WarnExtraModel base class that keeps the existing extra="ignore" semantics (so it's non-breaking) but logs a warning the first time a given (model_name, unknown_field) pair is seen, rate-limited per pair to avoid log-spam from repeat requests. Public API request models updated: ChatCompletionRequest, BenchChatCompletionRequest (via inheritance), PlaceInstanceParams, CreateInstanceParams, AddCustomModelParams, DeleteInstanceTaskParams, InstanceLinkBody, ImageGenerationTaskParams, ImageEditsTaskParams, ResponsesRequest, OllamaChatRequest, OllamaGenerateRequest, OllamaShowRequest, ClaudeMessagesRequest. A future change can flip extra="forbid" once the warning has had time to surface miswired clients in the wild.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warn on dropped extra fields in API request models
Problem
Pydantic v2
BaseModeldefaults toextra="ignore", silently acceptingand dropping unknown JSON fields on request bodies. For internal data
structures this is fine, but for public API request bodies it makes
mistyped or out-of-spec fields invisible.
We hit this on a 2x Mac Studio M4 Max cluster (TB5 RDMA, MLX-Jaccl
backend) when validating
/place_instance: a mistypedinstance_metafield was silently dropped, falling back to the default
Pipeline + MlxRingplacement instead of the intendedTensor + MlxJaccl. The request returned 200 OK, placement lookedsuccessful, and the only symptom was ~1.5x lower decode throughput.
There was no log line, no warning, no 4xx — debugging required diffing
the placement response against the request payload by hand.
This is the kind of footgun where the API silently does the wrong
thing. A 4xx would be the right end state, but a hard switch to
extra="forbid"is breaking for any client today that's relying onextras being dropped (including future-spec fields the server simply
hasn't learned about yet).
Change
Adds a
WarnExtraModelbase class (insrc/exo/utils/extra_fields_warner.py)that keeps
extra="ignore"semantics — so behavior is unchanged — butattaches a
model_validator(mode="before")that compares the inputdict's keys against the model's declared fields (and aliases / alias
choices / alias paths) and logs a one-line warning per unknown key.
Warnings are rate-limited per
(model_name, field_name)pair to onceper 60s. A burst of 100 mistyped requests produces one log line per
distinct field, not one hundred.
Then applies
WarnExtraModelas the base class for the public APIrequest models. Existing
BaseModel(frozen=True)and inheritedclasses are preserved exactly — only the base swaps in the validator.
Models updated
src/exo/api/types/api.py:ChatCompletionRequest,BenchChatCompletionRequest(transitively),PlaceInstanceParams,CreateInstanceParams,AddCustomModelParams,DeleteInstanceTaskParams,InstanceLinkBody,ImageGenerationTaskParams,BenchImageGenerationTaskParams(transitively),
ImageEditsTaskParamssrc/exo/api/types/openai_responses.py:ResponsesRequestsrc/exo/api/types/ollama_api.py:OllamaChatRequest,OllamaGenerateRequest,OllamaShowRequestsrc/exo/api/types/claude_api.py:ClaudeMessagesRequestFrozenModeldescendants (e.g.StartDownloadParams,CancelDownloadParams,DeleteTracesRequest) already useextra="forbid"and are intentionally untouched — they 4xx today andshould keep doing so.
Test plan
src/exo/utils/tests/(30 tests) pass unchangedsrc/exo/api/tests/(42 tests) pass unchangedsrc/exo/utils/tests/test_extra_fields_warner.py(10 cases) cover:
Field(alias=...)) are recognizedAliasChoicesaccepts any declared name without warningAliasPathrecognises the top-level path entryBaseModelsubclasses are unaffected (mixin is opt-in)instanceMetaonPlaceInstanceParamsnow logs:
behavior preserved).
Migration notes
and the same default behavior. Only the log gains a warning.
extra="forbid"(or move to a stricter baseclass) in a follow-up PR after a deprecation window long enough for
the warning to surface miswired clients in operator logs.
by a threading lock). Multi-worker deployments will independently
rate-limit per-process; this is the desired behavior for log-noise
control.
Repro