Skip to content

feat(vanity-gateway): expose dropped shadow request metric #428

Description

@FamousDirector

Problem

The vanity gateway records dropped shadow dispatches in trace attributes and a debug log, but it does not expose a Prometheus metric. Operators cannot quantify shadow drops or compare delivered shadow traffic with drops for each configured shadow model using metrics alone.

Proposed change

  • Add a monotonic counter for dropped shadow dispatch attempts.
  • Label the counter by openai_model_name and bounded drop reason: concurrency_limit, body_read_error, or body_rewrite_error.
  • Pre-initialize every configured shadow-model and reason combination to zero so its series exists before the first drop.
  • Add focused tests for counter initialization and per-model increments.

Acceptance criteria

  • Prometheus exposes nvcf_ai_api_gateway_shadow_requests_dropped_total.
  • Drops can be queried by openai_model_name and reason.
  • Each recorded drop increments exactly one model and reason series.
  • Multi-target failures increment the corresponding target-model series.
  • Existing shadow dispatch and tracing behavior remains unchanged.
  • Focused vanity-gateway tests pass.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions