Skip to content

Commit 2460337

Browse files
sbalabanov-zzsbalabanov-zz
committed
docs(rfc): Add orchestrator workflow diagram and controller summary
Documents the queue-driven controller pipeline from gateway entry through batching, scoring, build, merge, and conclude — with an ASCII diagram, a per-controller summary table, and a short overview of the two cycles in the pipeline (CI feedback and merge → speculate).
1 parent 8e3e4af commit 2460337

2 files changed

Lines changed: 83 additions & 0 deletions

File tree

‎doc/rfc/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ Design documents and technical proposals for SubmitQueue.
55
## Index
66

77
- [SQL-Based Distributed Queue](sql-queue-rfc.md) - MySQL-based distributed message queue with partition leasing and at-least-once delivery
8+
- [Orchestrator Workflow](workflow.md) - Queue-driven controller pipeline from gateway entry through batching, scoring, build, merge, and conclude

‎doc/rfc/workflow.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Orchestrator Workflow
2+
3+
The orchestrator processes land requests through a queue-driven pipeline of small, single-purpose controllers. The gateway accepts a request over RPC and hands it off asynchronously; from there each controller consumes one topic, advances the request or batch, and publishes to the next topic. Most hops carry only an ID — the controller fetches the entity from storage — while a few entry points (`start`, `buildsignal`, `log`) carry the full payload because there is no row to fetch yet.
4+
5+
The pipeline has two cycles: `speculate → build → buildsignal → speculate` (CI feedback loop) and `merge → speculate` (advance the next batch). `conclude` is the only stage that transitions a request to a terminal state; `log` is an append-only sink that any controller can publish to via `core/request.PublishLog`.
6+
7+
## Diagram
8+
9+
```
10+
┌──────────────────────────────────┐
11+
│ gateway:Land (RPC entry) │
12+
│ Accept, mint ID, hand off async │
13+
└────────────────┬─────────────────┘
14+
│ LandRequest
15+
▼
16+
┌──────────────────────┐ ┌──────────────────────────────────┐
17+
│ log (terminal sink) │◄───│ start │
18+
│ Append RequestLog │ │ Persist Request, emit Started │
19+
└──────────────────────┘ └────────────────┬─────────────────┘
20+
▲ │ RequestID
21+
│ ▼
22+
│ ┌──────────────────────────────────┐
23+
│ │ validate │
24+
│ │ Check mergeability + change meta │
25+
│ └────────────────┬─────────────────┘
26+
│ │ RequestID
27+
│ ▼
28+
│ ┌──────────────────────────────────┐
29+
│ │ batch │
30+
│ │ Group request into a Batch │
31+
│ └────────────────┬─────────────────┘
32+
│ │ BatchID
33+
│ ▼
34+
│ ┌──────────────────────────────────┐
35+
├─────────────────│ score │
36+
│ RequestLog×N │ Score the batch, persist score │
37+
│ └────────────────┬─────────────────┘
38+
│ │ BatchID
39+
│ ▼
40+
│ ┌──────────────────────────────────┐
41+
│ ┌───►│ speculate (stub) │◄────┐
42+
│ │ │ Decide CI verify vs. land │ │
43+
│ │ └──────┬─────────────────┬─────────┘ │
44+
│ │ BatchID │ │ BatchID │
45+
│ │ ▼ ▼ │
46+
│ │ ┌──────────────────┐ ┌──────────────────┐ │
47+
│ │ │ build │ │ merge │ │
48+
│ │ │ Trigger CI build │ │ Merge + advance │─┤
49+
│ │ └────────┬─────────┘ └────────┬─────────┘ │
50+
│ │ Build │ │ BatchID │
51+
│ │ ▼ │ │
52+
│ │ ┌──────────────────┐ │ │
53+
│ └──│ buildsignal │ │ │
54+
│ BatchID │ Feed CI result │ │ │
55+
│ │ back to spec. │ │ │
56+
│ └──────────────────┘ │ │
57+
│ ▲ │ BatchID │
58+
│ │ Build (ext. CI) ▼ │
59+
│ │ ┌──────────────────┐ │
60+
│ │ │ conclude │ │
61+
│ │ │ Map batch state │ │
62+
│ │ │ → request state │ │
63+
│ │ └──────────────────┘ │
64+
│ │ │
65+
└─── any controller via core/request.PublishLog ──────┘
66+
```
67+
68+
## Per-controller summary
69+
70+
| Controller | In | Out | One-line role |
71+
|---|---|---|---|
72+
| **gateway/Land** | RPC | start | Accept request, mint ID, log Accepted, hand off async |
73+
| **start** | LandRequest | validate, log | Persist Request and emit Started log |
74+
| **validate** | RequestID | batch | Check mergeability and fetch change metadata |
75+
| **batch** | RequestID | score | Group request into a Batch with dependencies |
76+
| **score** | BatchID | speculate, log | Score the batch (∏ per-request scores), persist score |
77+
| **speculate** | BatchID | build, merge | (stub) Decide whether to verify via CI or land |
78+
| **build** | BatchID | buildsignal | Trigger CI build for the batch |
79+
| **buildsignal** | Build | speculate | Feed CI result back into speculation |
80+
| **merge** | BatchID | conclude, speculate | Merge the batch and advance the queue |
81+
| **conclude** | BatchID | — | Map terminal batch state to request state |
82+
| **log** | RequestLog | — | Append-only sink for request log events |

0 commit comments

Comments
 (0)