gdev-agent is a governed local support-triage reference workload for
game-studio scenarios. It receives signed support webhooks, blocks unsafe input
before any model call, classifies and extracts structured data, routes risky
actions into human approval, and records an audit/cost trail behind one HTTP
API.
The supported proof boundary is a tested local Docker Compose workload with synthetic demo/eval data and database-enforced tenant isolation. Its 55-case conformance set passes, while the harder published 100-case Eval Lab gate fails. There is no claimed pilot, external deployment, live customer traffic, production readiness, or production SLO.
- This repository owns application behavior, signed ingress, approval/audit flows, tenant controls, and candidate fixes.
- Eval Ground Truth Lab owns external workflow-quality gates. Its canonical 100-case result for this revision is FAIL and remains authoritative for those challenge outcomes.
- Agent Runtime Grid is an optional execution layer. It does not replace gdev lifecycle/security tests or Eval Lab quality decisions.
- AI Workflow Playbook is an independent governance companion, not a runtime dependency.
- The thin umbrella pins compatible revisions and runs integration evidence; it does not absorb component code or history.
The repository demonstrates one bounded local support workflow. It is not a hosted support platform, customer-data product, generic agent framework, production multi-tenant SaaS, or proof that the demo classifier generalizes.
For a claim-by-claim proof map, start with docs/EVIDENCE_INDEX.md.
| Claim to inspect | Start here | Bounded status |
|---|---|---|
| Evidence question map | docs/EVIDENCE_INDEX.md#evidence-questions | One-click route for problem, architecture, controls, quality, failure behavior, metrics, demo, limits, and production changes |
| Three-project stack map | docs/STACK_OVERVIEW.md | Explains how gdev-agent, Eval Ground Truth Lab, and Agent Runtime Grid fit together as workflow, quality, and runtime layers |
| One-page engineering story | docs/CASE_STUDY.md | Evidence-backed case study for problem, architecture, controls, eval, load, trade-offs, and production changes |
| Architecture and workflow boundaries | docs/ARCHITECTURE.md, docs/architecture-diagram.md | Implemented local stack with documented gaps and ADRs |
| Human approval decision path | docs/APPROVAL_FLOW.md | Code-and-test-backed risk, tenant, TTL, reject, and one-time execution flow; not an operator UI or routing-quality claim |
| Agent harness boundary | docs/HARNESS_CARD.md, docs/TRACE_SCHEMA.md, AGENTS.md | Model + prompt/tool loop + guards + approvals + trace + eval are reviewed as one bounded harness |
| Repeatable demo path | docs/DEMO.md | Local Compose demo with deterministic/free mode |
| Evaluation discipline | docs/EVALUATION.md, docs/EVAL_REPORT.md, docs/EVAL_SCOPE_RECONCILIATION.md | 180-case internal smoke eval, 55-case Eval Lab conformance baseline, and a canonical 100-case challenge run whose stricter gate failed |
| Canonical failure preview | docs/evidence/GDEV_EVAL_FAILURE_PREVIEW_2026-07-13.md | Rendered from Eval Lab v0.2.0 content-addressed raw JSON; five failed thresholds remain visible and limitations are explicit |
| Observability | docs/observability.md | Metrics, traces, logs, and alerting design for local evidence |
| Load profile | docs/load-profile.md, docs/LOAD_TEST_REPORT.md | Local deterministic/synthetic report and scenario targets; not production capacity claims |
| Tenant isolation and security | docs/TENANT_ISOLATION.md, docs/data-map.md#6-tenant-isolation-model, docs/ARCHITECTURE.md#7-security-model | RLS, tenant-scoped JWT, webhook signature, secrets, approval, and cost ledger boundaries |
| Tests | Current State, maintainer-surface verification | P0 code baseline: 310 passing tests at 0e4c5f0; 2026-07-13 maintainer-surface verification: 312 passing tests after two documentation contract tests |
| Failure modes and SLO/runbook | docs/FAILURE_MODES.md, docs/SLO_RUNBOOK.md, docs/observability.md#alert-runbooks | Local taxonomy and runbook evidence; external incident evidence is out of scope |
| Deployment readiness boundaries | docs/DEPLOYMENT_READINESS.md, Known Limits | Secrets checklist, backup/restore notes, local production-like config, and known limitations without production readiness claims |
| Known limits and production changes | Known Limits, docs/DEPLOYMENT_READINESS.md | Explicitly bounded as local reference evidence, not production SaaS readiness |
The image is a preview of pinned negative evidence, not a current quality-pass badge. Follow it to the exact Eval Lab tag, raw JSON hashes, rendering command, and interpretation boundary.
The synthetic reference scenario covers billing disputes, account-access
incidents, bug reports, moderation signals, and repetitive gameplay questions.
gdev-agent exercises the orchestration boundary between inbound support
traffic and downstream tools: routine paths move through deterministic controls,
while low-confidence or risky actions require review.
flowchart LR
caller[Webhook caller\nTelegram / n8n / Make / HTTP client] --> webhook[POST /webhook]
webhook --> sig[Signature + rate-limit middleware]
sig --> guard[Input guard\nlength and injection checks]
guard --> llm[LLM tool loop\nclassify + extract + draft]
llm --> policy[Policy + output guard\nrisk, confidence, secret and URL checks]
policy --> decision{Risky action?}
decision -->|Yes| pending[Store pending approval\nRedis + reviewer callback]
pending --> approve[POST /approve]
approve --> execute[Execute approved action\nticket + reply + audit]
decision -->|No| execute
execute --> result[HTTP response + audit trail\nlogs, metrics, cost ledger]
The current stack includes FastAPI, Redis, PostgreSQL with Row-Level Security, pgvector-backed embeddings, a clean service layer (WebhookService, ApprovalService, AuthService, EvalService), RCA clustering jobs, and an n8n integration path for orchestration and approvals. Architecture detail lives in docs/ARCHITECTURE.md.
| Area | What ships today |
|---|---|
| Ingress | POST /webhook entrypoint with WebhookService (tenant resolution, dedup, OTel tracing); per-tenant HMAC verification, rate limiting |
| AI pipeline | Claude tool_use classification and extraction, guarded draft generation, configurable auto-approve threshold |
| Safety | Input injection guard, output secret scan, URL allowlist enforcement, approval workflow with ApprovalService (HMAC + cross-tenant enforcement) |
| Execution | Tool registry for ticketing and reply actions, dedup cache for idempotent replays, pending approval storage with TTL |
| Multi-tenancy | PostgreSQL FORCE RLS on all 16 tenant-scoped tables, non-owner gdev_app runtime role, tenant registry, per-tenant encrypted secrets |
| Operations | Cost ledger with daily budget enforcement, structured JSON logs, Prometheus metrics (OTel child spans on all endpoints), Grafana/Loki/Tempo stack |
| Analytics | Eval runner with budget check, eval API, tenant learning metrics from approval latency/overrides, RCA clustering job (DBSCAN + pgvector), cluster read endpoints with DB-backed membership |
| Admin | gdev-admin CLI for tenant/budget/RCA operations; separate maintenance role with BYPASSRLS |
| Platform | Docker Compose full stack; Python 3.12 CI; ruff, full pytest, eval, Compose RLS, and deterministic demo gates |
This path is aligned to docker-compose.yml and is the fastest way to get a healthy local stack.
git clone https://github.com/ashishki/gdev-agent.git
cd gdev-agent
cp .env.example .env
docker compose up --buildWhat starts:
| Service | URL / Port | Purpose |
|---|---|---|
| agent | http://localhost:8000 |
FastAPI application |
| postgres | localhost:5432 |
Primary database |
| redis | localhost:6379 |
Rate limit, dedup, approval state |
| n8n | http://localhost:5678 |
Workflow orchestration |
| prometheus | http://localhost:9090 |
Metrics scrape |
| grafana | http://localhost:3000 |
Dashboards |
| tempo | http://localhost:3200 |
Trace backend |
| loki | http://localhost:3100 |
Log backend |
The migrate service runs Alembic and seeds the database before the API starts.
Compose creates a bootstrap/migration owner (gdev_owner) and a distinct
non-superuser request role (gdev_app). Their local-only passwords come from
GDEV_OWNER_PASSWORD and GDEV_APP_PASSWORD in the copied .env; Compose has
no password fallback. LLM_MODE defaults to deterministic demo mode.
Verify the stack:
curl -i http://localhost:8000/health
bash scripts/verify_compose_rls.shExpected response:
HTTP/1.1 200 OK{"status":"ok","app":"gdev-agent"}If you want to exercise live LLM behavior, set LLM_MODE=live and ANTHROPIC_API_KEY in .env before startup.
Copy .env.example and adjust only what you need for your environment. Deployment-readiness boundaries, required secrets, backup, restore, and known limitations are documented in docs/DEPLOYMENT_READINESS.md.
| Variable | Required | Notes |
|---|---|---|
ANTHROPIC_API_KEY |
Yes for LLM_MODE=live |
Live provider calls fail startup without it |
ANTHROPIC_MODEL |
No | Defaults to claude-sonnet-4-6 |
VOYAGE_API_KEY |
No | Needed for embedding-backed features outside stub mode |
EMBEDDING_MODEL |
No | Defaults to voyage-3-lite |
KB_BASE_URL |
Recommended | FAQ links should also be present in URL_ALLOWLIST |
REDIS_URL |
Yes | Approval store, rate limiting, dedup, caching |
DATABASE_URL |
Yes for Postgres features | Compose provides it automatically |
GDEV_OWNER_PASSWORD / GDEV_APP_PASSWORD |
Yes for Compose | Separate migration-owner and request-role passwords; local examples are in .env.example |
TEST_DATABASE_URL |
No | Test-only override |
DB_POOL_SIZE / DB_MAX_OVERFLOW |
No | Async Postgres pool sizing |
WEBHOOK_SECRET |
Optional legacy path | Global webhook secret; per-tenant secret storage is the main design |
WEBHOOK_SECRET_ENCRYPTION_KEY |
Recommended | Fernet key for encrypted per-tenant webhook secrets |
JWT_SECRET |
Yes outside demo mode | JWT signing secret |
JWT_ALGORITHM |
No | Defaults to HS256 |
JWT_TOKEN_EXPIRY_HOURS |
No | Access token lifetime |
APPROVE_SECRET |
Recommended | Shared secret for approval callbacks |
RATE_LIMIT_RPM / RATE_LIMIT_BURST |
No | Request and burst limits |
AUTH_RATE_LIMIT_ATTEMPTS |
No | Login throttling |
MAX_INPUT_LENGTH |
No | Input guard length cap |
AUTO_APPROVE_THRESHOLD |
No | Confidence threshold for auto execution |
EXEMPLAR_GUARD_ENABLED |
No | Enables curated-example consistency checks before auto execution |
EXEMPLAR_GUARD_THRESHOLD / EXEMPLAR_GUARD_TOP_K |
No | Similarity threshold and nearest-example count |
EXEMPLAR_GUARD_EXAMPLES_PATH |
No | Optional JSONL override; empty uses eval/exemplars/triage_v1.jsonl |
APPROVAL_CATEGORIES |
No | Comma-separated category list |
APPROVAL_TTL_SECONDS |
No | Pending approval expiry |
OUTPUT_GUARD_ENABLED |
No | Enables output checks |
URL_ALLOWLIST |
No | Comma-separated hostname allowlist |
OUTPUT_URL_BEHAVIOR |
No | strip or reject |
RCA_LOOKBACK_HOURS |
No | RCA clustering window |
RCA_BUDGET_PER_RUN_USD |
No | Budget cap per RCA run |
LINEAR_API_KEY / LINEAR_TEAM_ID |
Optional | Ticket creation integration |
TELEGRAM_BOT_TOKEN / TELEGRAM_APPROVAL_CHAT_ID |
Optional | Messaging and approval notifications |
GOOGLE_SHEETS_CREDENTIALS_JSON / GOOGLE_SHEETS_ID |
Optional | Audit export integration |
SQLITE_LOG_PATH |
Optional | Enables local SQLite event logging |
OTLP_ENDPOINT / OTEL_SERVICE_NAME |
Optional | OpenTelemetry export |
APP_NAME / APP_ENV / LOG_LEVEL |
No | App identity and logging controls |
LLM_INPUT_RATE_PER_1K / LLM_OUTPUT_RATE_PER_1K |
No | Cost ledger rates |
| Endpoint | Purpose |
|---|---|
POST /webhook |
Main ingestion path for support messages; returns executed, pending, or guard-blocked |
POST /approve |
Human decision endpoint for pending actions |
GET /health |
Application liveness check used by Docker health checks |
GET /metrics |
Prometheus scrape endpoint |
| Endpoint | Purpose |
|---|---|
POST /auth/token |
Login and issue JWT |
POST /auth/logout |
Blocklist a token |
POST /auth/refresh |
Refresh an access token |
POST /eval/run |
Start an eval run |
GET /eval/runs |
List eval history |
| Endpoint | Purpose |
|---|---|
GET /tickets and GET /tickets/{ticket_id} |
Ticket history and detail |
GET /audit |
Audit log history |
GET /metrics/cost |
Cost ledger readout |
GET /metrics/learning |
Tenant approval latency, override, rejection, and reviewed-volume metrics |
GET /agents and PUT /agents/{agent_id} |
Agent config inspection and versioned updates |
GET /clusters |
RCA cluster list |
GET /clusters/{cluster_id} |
RCA cluster detail |
GET /clusters/{cluster_id}/tickets |
Tickets associated with a cluster |
Most endpoints outside /health, /webhook, and /metrics require JWT auth plus tenant context.
- A caller sends a support event to
POST /webhook. - Middleware verifies the request signature, applies rate limits, and assigns request correlation.
- The agent blocks oversized or injection-shaped input before any model call.
- The LLM classifies the message, extracts entities, and drafts a response.
- Policy and output-guard checks decide whether the action can auto-execute or must wait for review.
- The service either executes the tool path immediately or stores a pending approval for
POST /approve. - Audit rows, metrics, and cost tracking capture the outcome.
- docs/EVIDENCE_INDEX.md: evidence question map and claim-by-claim proof table.
- docs/STACK_OVERVIEW.md: three-project stack map and provider strategy.
- docs/EVAL_SCOPE_RECONCILIATION.md: reconciles the internal 180-case smoke, Eval Lab 55-case conformance baseline, canonical 100-case challenge failure, and Runtime Grid proofs.
- docs/ARCHITECTURE.md: system structure, service boundaries, request flow, deployment view.
- docs/HARNESS_CARD.md: agent harness boundary across model, tools, memory, retries, permissions, HITL, trace, and eval.
- docs/TRACE_SCHEMA.md: trace completeness contract for debugging, eval, audit, and approval retrospectives.
- docs/architecture-diagram.md: GitHub-rendered architecture diagram for the main workflow.
- docs/APPROVAL_FLOW.md: implemented human-approval decision path with focused regression commands and limits.
- docs/evidence/GDEV_EVAL_FAILURE_PREVIEW_2026-07-13.md: content-addressed preview of the canonical failed Eval Lab challenge.
- docs/evidence/GDEV_MAINTAINER_SURFACE_2026-07-13.md: local receipt for the maintainer, visual, full-suite, eval, and Compose gates.
- docs/CASE_STUDY.md: concise evidence-backed engineering case study.
- docs/spec.md: product scope, API intent, and behavioral contract.
- docs/N8N.md: n8n integration and approval workflow blueprint.
- docs/observability.md: metrics, tracing, and logging conventions.
- docs/agent-registry.md: agent configuration model and governance.
- docs/llm-usage.md: prompt/versioning and model-usage rules.
- docs/load-profile.md: load targets and performance assumptions.
- docs/TENANT_ISOLATION.md: canonical tenant-isolation proof with exact tests and migrations.
- docs/DEPLOYMENT_READINESS.md: local production-like config, secrets checklist, backup/restore notes, and known limitations.
- docs/data-map.md: schema, Redis keys, and tenant-boundary rules.
- n8n/README.md: workflow assets committed in this repository.
- AGENTS.md: operating rules for coding/ops agents working in this repo.
- Suspected vulnerabilities must follow the private path in SECURITY.md; do not disclose them in a public issue.
- Supported local defects use the reproducible bug form with an exact commit, sanitized environment, and deterministic commands.
- There is no generic feature or hosted-product roadmap intake. The maintained boundary is the current local reference workload described in Current Maturity.
- The project is a tested local reference workload. It has no claimed external deployment, production SaaS readiness, live tenant traffic, or real customer operations.
- Demo, eval, and load evidence is synthetic unless a later report explicitly says otherwise. The current load report is deterministic/local evidence, not live capacity proof.
- Eval metrics have multiple scopes. The internal 180-case smoke report exposes
broad demo-mode routing gaps, while the external Eval Lab 55-case baseline is
an integration/conformance pass over the configured
/webhookadapter. A canonical Eval Lab run of the separate 100-case challenge against exact revision0e4c5f0fd50382bbf12ffd35cfca4632384fb0ccfailed its stricter gate: reconciled pass rate0.32, classification accuracy0.244444, 68 unexpected failures, and 58 blocking failures. This is published negative evidence, not a threshold pass. See docs/EVAL_SCOPE_RECONCILIATION.md. - Live load measurements remain out of scope for the current local evidence. Deployment readiness notes are local-only and explicitly do not prove production readiness.
- Live LLM behavior requires a real Anthropic API key and budget controls; the local stack is the supported review path today.
The local stack demonstrates the governed request pipeline. It includes the tenant-scoped storage foundation, JWT/RBAC boundary, approval hardening, eval APIs with budget enforcement, auth service flows, embedding persistence, RCA clustering with persisted cluster membership, service-layer separation for the main write/auth/eval workflows with read-route extraction still tracked as architecture drift, Dockerized observability, admin CLI, and n8n workflow artifacts for local demonstrations.
At revision 0e4c5f0fd50382bbf12ffd35cfca4632384fb0cc, the dated 2026-07-13
local baseline is 310 tests passed (unit + integration,
including migration up/down, role flags, FORCE RLS topology, cross-tenant
rejection, eval metric validators, load fixtures, observability signals, and
cluster membership persistence). The same repair run also passed the 180-case
demo eval gate and a clean Compose auth/approval demo. Exact commands and
bounded outputs are recorded in
docs/evidence/GDEV_P0_TRUTH_REPAIR_2026-07-13.md.
The later maintainer/evidence-surface tranche added two focused documentation contracts and reran the complete local suite at 312 passing tests, along with Ruff, the non-writing 180-case eval gate, and an isolated default-Compose RLS and approval demo. Its bounded receipt is docs/evidence/GDEV_MAINTAINER_SURFACE_2026-07-13.md.
The main value is the governed request pipeline: webhook in → guardrails → LLM-assisted triage → human approval where needed → auditable execution throughout, with tenant isolation enforced at the database layer and observable at every step.