Skip to content

Add WarnExtraModel: log warnings for unknown fields in API request models#2015

Draft
imapotato123 wants to merge 1 commit into
exo-explore:mainfrom
imapotato123:feat/warn-dropped-extra-fields
Draft

Add WarnExtraModel: log warnings for unknown fields in API request models#2015
imapotato123 wants to merge 1 commit into
exo-explore:mainfrom
imapotato123:feat/warn-dropped-extra-fields

Conversation

@imapotato123
Copy link
Copy Markdown

@imapotato123 imapotato123 commented May 2, 2026

Warn on dropped extra fields in API request models

Problem

Pydantic v2 BaseModel defaults to extra="ignore", silently accepting
and dropping unknown JSON fields on request bodies. For internal data
structures this is fine, but for public API request bodies it makes
mistyped or out-of-spec fields invisible.

We hit this on a 2x Mac Studio M4 Max cluster (TB5 RDMA, MLX-Jaccl
backend) when validating /place_instance: a mistyped instance_meta
field was silently dropped, falling back to the default
Pipeline + MlxRing placement instead of the intended
Tensor + MlxJaccl. The request returned 200 OK, placement looked
successful, and the only symptom was ~1.5x lower decode throughput.
There was no log line, no warning, no 4xx — debugging required diffing
the placement response against the request payload by hand.

This is the kind of footgun where the API silently does the wrong
thing. A 4xx would be the right end state, but a hard switch to
extra="forbid" is breaking for any client today that's relying on
extras being dropped (including future-spec fields the server simply
hasn't learned about yet).

Change

Adds a WarnExtraModel base class (in src/exo/utils/extra_fields_warner.py)
that keeps extra="ignore" semantics — so behavior is unchanged — but
attaches a model_validator(mode="before") that compares the input
dict's keys against the model's declared fields (and aliases / alias
choices / alias paths) and logs a one-line warning per unknown key.

Warnings are rate-limited per (model_name, field_name) pair to once
per 60s. A burst of 100 mistyped requests produces one log line per
distinct field, not one hundred.

Then applies WarnExtraModel as the base class for the public API
request models. Existing BaseModel(frozen=True) and inherited
classes are preserved exactly — only the base swaps in the validator.

Models updated

  • src/exo/api/types/api.py:
    ChatCompletionRequest, BenchChatCompletionRequest (transitively),
    PlaceInstanceParams, CreateInstanceParams, AddCustomModelParams,
    DeleteInstanceTaskParams, InstanceLinkBody,
    ImageGenerationTaskParams, BenchImageGenerationTaskParams
    (transitively), ImageEditsTaskParams
  • src/exo/api/types/openai_responses.py: ResponsesRequest
  • src/exo/api/types/ollama_api.py: OllamaChatRequest,
    OllamaGenerateRequest, OllamaShowRequest
  • src/exo/api/types/claude_api.py: ClaudeMessagesRequest

FrozenModel descendants (e.g. StartDownloadParams,
CancelDownloadParams, DeleteTracesRequest) already use
extra="forbid" and are intentionally untouched — they 4xx today and
should keep doing so.

Test plan

  • Existing src/exo/utils/tests/ (30 tests) pass unchanged
  • Existing src/exo/api/tests/ (42 tests) pass unchanged
  • New tests in src/exo/utils/tests/test_extra_fields_warner.py
    (10 cases) cover:
    • Known fields parse without warning
    • Unknown field is dropped + warned, request still parses
    • Repeat unknown field on same model is rate-limited
    • Distinct unknown fields each warn once
    • Aliased fields (Field(alias=...)) are recognized
    • AliasChoices accepts any declared name without warning
    • AliasPath recognises the top-level path entry
    • Subclasses inherit the validator + warning names the subclass
    • Non-dict inputs (model instances) pass through cleanly
    • Plain BaseModel subclasses are unaffected (mixin is opt-in)
  • End-to-end repro: mistyped instanceMeta on PlaceInstanceParams
    now logs:
    WARNING | exo.utils.extra_fields_warner:_warn_unknown_fields - Dropping unknown field 'instanceMeta' on PlaceInstanceParams request — this field is not declared on the model and will be ignored. Check for a typo or an out-of-spec field. (Subsequent occurrences of this same field on this model are rate-limited to once per 60s.)
    
    while still placing on the default backend (existing fallback
    behavior preserved).

Migration notes

  • Non-breaking. Clients sending unknown fields still get 200 OK
    and the same default behavior. Only the log gains a warning.
  • The intent is to flip extra="forbid" (or move to a stricter base
    class) in a follow-up PR after a deprecation window long enough for
    the warning to surface miswired clients in operator logs.
  • The rate-limit cache is process-local (a module-level dict guarded
    by a threading lock). Multi-worker deployments will independently
    rate-limit per-process; this is the desired behavior for log-noise
    control.

Repro

# Trigger the warning by sending a typo on /place_instance:
curl -sS -X POST http://<exo-host>:52415/place_instance \
  -H 'Content-Type: application/json' \
  -d '{"model_id":"qwen-3.6-35b","sharding":"Tensor","instanceMeta":"MlxJaccl"}'

# Check logs:
tail -f ~/.local/share/exo/exo.log | grep "Dropping unknown field"

Pydantic v2 BaseModel defaults to extra="ignore", silently dropping
unknown fields on request bodies. For internal data structures this
is fine, but for public API request bodies it makes mistyped or
out-of-spec fields invisible: the request is accepted, the offending
field is dropped, and the request runs with whatever defaults the
matching declared fields had.

We hit this in cluster validation: a mistyped instance_meta field on
/place_instance was silently dropped, falling back to the default
MlxRing instead of the requested MlxJaccl, with no log line. Symptom
was lower decode throughput, no error.

This change adds a WarnExtraModel base class that keeps the existing
extra="ignore" semantics (so it's non-breaking) but logs a warning
the first time a given (model_name, unknown_field) pair is seen,
rate-limited per pair to avoid log-spam from repeat requests.

Public API request models updated: ChatCompletionRequest,
BenchChatCompletionRequest (via inheritance), PlaceInstanceParams,
CreateInstanceParams, AddCustomModelParams, DeleteInstanceTaskParams,
InstanceLinkBody, ImageGenerationTaskParams, ImageEditsTaskParams,
ResponsesRequest, OllamaChatRequest, OllamaGenerateRequest,
OllamaShowRequest, ClaudeMessagesRequest.

A future change can flip extra="forbid" once the warning has had time
to surface miswired clients in the wild.
@imapotato123 imapotato123 marked this pull request as draft May 3, 2026 20:06
@imapotato123 imapotato123 changed the title feat: warn on dropped extra fields in public API request models Add WarnExtraModel: log warnings for unknown fields in API request models May 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant