Observability

[ashum9]

Observability Stack + Engine Integration

New mofa-observability crate, event schema, async metrics collector (18 metric families), Prometheus renderer, mock harness, 3 Grafana dashboards, Docker Compose stack.

Engine integration: ObservabilityConfig in config schema, GET /metrics endpoint, observability bridge with startup gauge seeding, OpenTelemetry OTLP tracing support. Zero engine-core logic changed all wiring and new modules.

Verified live: real engine traffic flows through to Grafana dashboards.

[Copilot]

Pull request overview

This PR introduces a new mofa-observability crate (event schema + async in-memory metrics collector + Prometheus text renderer) and wires it into the engine so the SDK can expose a /metrics endpoint and the app can optionally enable OTLP tracing. It also adds a Docker Compose-based local observability stack (Prometheus + Grafana) plus prebuilt dashboards.

Changes:

• Add mofa-observability crate: event types, metrics collector pipeline, Prometheus text exposition renderer, and a mock harness example.
• Integrate engine: ObservabilityConfig in config schema, start observability pipeline + bridge, and add GET /metrics endpoint.
• Add dockerized Prometheus/Grafana stack and provisioned dashboards for engine/preflight/memory views.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
test_obs_config.toml	Adds a sample config enabling observability.
mofa-observability/src/lib.rs	Exposes observability modules (events/collector/prometheus).
mofa-observability/src/events.rs	Defines the observability event schema + tests.
mofa-observability/src/collector.rs	Implements the in-memory metrics state and async collector pipeline.
mofa-observability/src/prometheus.rs	Renders metrics state into Prometheus text exposition format + tests.
mofa-observability/examples/mock_harness.rs	Example app generating synthetic events and serving /metrics.
mofa-observability/Cargo.toml	Adds crate dependencies for observability components and example.
mofa-observability/docker/Dockerfile	Builds/runs the mock harness in a container.
mofa-observability/docker/docker-compose.yml	Compose stack for mock harness + Prometheus + Grafana.
mofa-observability/docker/prometheus/prometheus.yml	Prometheus scrape configuration for harness/engine.
mofa-observability/docker/grafana/provisioning/datasources/prometheus.yml	Grafana datasource provisioning.
mofa-observability/docker/grafana/provisioning/dashboards/dashboards.yml	Grafana dashboard provisioning.
mofa-observability/dashboards/engine_overview.json	Grafana dashboard for overall engine health/perf metrics.
mofa-observability/dashboards/memory_lifecycle.json	Grafana dashboard for memory/model lifecycle metrics.
mofa-observability/dashboards/preflight_routing.json	Grafana dashboard for preflight/routing/failover metrics.
mofa-engine-core/src/config.rs	Adds ObservabilityConfig to engine config schema.
mofa-engine-core/src/engine.rs	Updates tests/config construction to include observability defaults.
mofa-engine-sdk/src/server.rs	Adds optional metrics state to server and implements /metrics.
mofa-engine-sdk/src/observability_bridge.rs	Translates engine events into observability events + seeds gauges.
mofa-engine-sdk/src/lib.rs	Exposes the observability bridge module.
mofa-engine-sdk/Cargo.toml	Adds dependency on mofa-observability.
mofa-engine-app/src/main.rs	Initializes tracing (optionally OTLP) + starts metrics + bridge.
mofa-engine-app/Cargo.toml	Adds OTLP/tracing dependencies and mofa-observability.
Cargo.toml	Adds mofa-observability workspace member + OTLP deps in workspace.
Cargo.lock	Locks new crate/dependency graph changes.
.dockerignore	Adds docker ignore rules for build context.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.