Summary
Add a /metrics endpoint that exposes Prometheus-compatible metrics for monitoring inference performance in production.
Details
trio-core's FastAPI server (src/trio_core/api/server.py) currently has no observability endpoints. Adding Prometheus metrics would let users monitor inference latency, request counts, and error rates using standard tools (Grafana, Datadog, etc).
What to Implement
- Add
prometheus_client as an optional dependency in pyproject.toml (under an [observability] extra)
- Create a
/metrics endpoint in the API server
- Expose at minimum:
trio_request_latency_seconds — Histogram for request duration, labeled by endpoint
trio_requests_total — Counter for total requests, labeled by endpoint and status
trio_inference_latency_seconds — Histogram specifically for model inference time
trio_active_requests — Gauge for currently-in-flight requests
Acceptance Criteria
Files to Modify
pyproject.toml — add optional dependency
src/trio_core/api/server.py — add middleware and endpoint
tests/test_api.py — add test
Resources
Summary
Add a
/metricsendpoint that exposes Prometheus-compatible metrics for monitoring inference performance in production.Details
trio-core's FastAPI server (
src/trio_core/api/server.py) currently has no observability endpoints. Adding Prometheus metrics would let users monitor inference latency, request counts, and error rates using standard tools (Grafana, Datadog, etc).What to Implement
prometheus_clientas an optional dependency inpyproject.toml(under an[observability]extra)/metricsendpoint in the API servertrio_request_latency_seconds— Histogram for request duration, labeled by endpointtrio_requests_total— Counter for total requests, labeled by endpoint and statustrio_inference_latency_seconds— Histogram specifically for model inference timetrio_active_requests— Gauge for currently-in-flight requestsAcceptance Criteria
GET /metricsreturns Prometheus text formatprometheus_clientis installed (graceful degradation)/metricsreturns 200 with expected metric namesFiles to Modify
pyproject.toml— add optional dependencysrc/trio_core/api/server.py— add middleware and endpointtests/test_api.py— add testResources