From 4ab6db2d02a548a1caf2bffeaed6ad707cb12a7c Mon Sep 17 00:00:00 2001 From: Preetam Dwivedi Date: Fri, 5 Jun 2026 15:53:22 -0700 Subject: [PATCH] docs(workflow): document queue and database ownership by service MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary ### Why? Issue #211 (follow-up from PR #205) asks for a single place that records the submitqueue topology at a high level: which service owns its data and how the two services communicate. The workflow RFC already covers the cross-queue flow, so ownership belongs alongside it. ### What? Append an "Ownership by service" section to doc/rfc/submitqueue/workflow.md, described at a conceptual level rather than enumerating individual tables and topics: - Gateway — RPC entry point and owner of the request log; the only service that reads or writes that record. - Orchestrator — runs the pipeline and owns its working state (requests, batches, builds); the only service that writes it. - Messaging queue — the shared, pluggable infrastructure the two services communicate through, kept in its own database separate from application data. A closing "Request-log ownership invariant" section captures the rule: the orchestrator only emits log events, the gateway is the sole consumer and the only writer of the request log. Documentation only; no code, schema, or proto changes. ## Test Plan - ✅ `make lint` (clean tree) ## Issue Closes #211 --- doc/rfc/submitqueue/workflow.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/doc/rfc/submitqueue/workflow.md b/doc/rfc/submitqueue/workflow.md index 52ad4e3..dc1e045 100644 --- a/doc/rfc/submitqueue/workflow.md +++ b/doc/rfc/submitqueue/workflow.md @@ -14,8 +14,8 @@ The pipeline has two cycles: `speculate → build → buildsignal → speculate` │ LandRequest ▼ ┌──────────────────────┐ ┌──────────────────────────────────┐ - │ log (gateway sink) │◄───│ start │ - │ Persist RequestLog │ │ Persist Request, emit Started │ + │ gateway: log │◄───│ start │ + │ Persist request log │ │ Persist Request, emit Started │ └──────────────────────┘ └────────────────┬─────────────────┘ ▲ │ RequestID │ ▼ @@ -80,3 +80,25 @@ The pipeline has two cycles: `speculate → build → buildsignal → speculate` | **merge** | BatchID | conclude, speculate | Merge the batch and advance the queue | | **conclude** | BatchID | — | Map terminal batch state to request state | | **log** | RequestLog | — | Gateway-owned sink: persists request log events to storage | + +## Ownership by service + +Each service owns its own data; the gateway and orchestrator never touch each other's, and the only thing they share is the messaging queue. + +### Gateway + +The gateway is the RPC entry point and the owner of the request log. It accepts requests, hands them to the orchestrator over the queue, and owns the record of what happened to each request — the only service that reads or writes the request log. It writes that record both directly, as requests arrive, and by consuming the log events the orchestrator emits. + +### Orchestrator + +The orchestrator runs the pipeline that advances a request from acceptance to a terminal state. It owns the working state of that pipeline — requests, batches, builds, and their bookkeeping — and is the only service that writes it. It drives a request through a series of internal stages, re-entering speculation as CI results arrive and as batches advance. + +### Shared: the messaging queue + +The two services communicate only through the messaging queue. It is pluggable infrastructure kept in its own database, separate from either service's application data: the gateway publishes incoming requests for the orchestrator to consume, and the orchestrator publishes log events for the gateway to consume. + +## Request-log ownership invariant + +The request log has exactly one owner: the **gateway**. The orchestrator only emits log events onto the queue; it never persists them. The gateway is the sole consumer of those events and the only writer of the request log. + +This keeps all request-log writes in one service: the orchestrator stays a pure pipeline that emits events, and the gateway owns the request log end to end.