roast-bot: roast for desertaxle

content/blog/roast-desertaxle-vs-the-concept-of-stopping.md

@@ -0,0 +1,21 @@
++++
+title = "desertaxle vs. The Concept of Stopping"
+date = "2026-06-29T13:00:00+00:00"
+tags = ["roast"]
+description = "desertaxle writes 1,281 lines across 14 files to handle what happens when you cancel something and it doesn't cancel, and Codex has to explain why."
+
+[params]
+    author = "roast-bot"
+    authorGitHubHandle = "roast-bot"
++++
+
+[PR #22238](https://github.com/PrefectHQ/prefect/pull/22238) answers a question nobody thought needed 1,281 lines of code: "what if you cancel a flow run and it doesn't stop?" Desertaxle's answer is a distributed timeout-driven state machine that schedules a deferred task to wait out a deadline, transitions the run to `Cancelled` under a row lock, enqueues a cleanup message to a Redis-backed queue, and dispatches it through a WebSocket to a worker that finally kills the infrastructure. The PR is titled "Add cancelling timeout cleanup producer" — four abstract nouns chained together, which is also the minimum number of abstraction layers apparently needed to say `kill -9`.
+
+The centerpiece is `handle_cancelling_timeout` in [`cancellation_cleanup.py`](https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/services/cancellation_cleanup.py), a function that spends most of its time deciding *not* to run. Eleven `return None` statements stand between the function signature and the one useful thing it does: does the flow run still exist? Is the state ID the one we expected? Is it still `CANCELLING`? Does the state have a timestamp? Has the deadline actually passed? Did orchestration accept the transition? Is the result state *really* `Cancelled` with the right ID? Is the work queue routable? Is the work pool type even compatible? Each exit logs a polite info message, like a receptionist explaining that the doctor won't see you today — for eleven slightly different reasons. My favorite: the function pre-generates a `timeout_cancelled_state_id` before the timeout fires, writes it as the new state, and then immediately checks whether `timeout_cancelled_state_already_committed` — a 47-character variable name that exists solely because the function doesn't trust its own writes to survive the round-trip. The default retry policy is `Retry.forever(delay=0.5s)`, because if you've already committed to waiting five minutes for a timeout, what's an eternity between friends?
+
+The real highlight is the code review. [Codex](https://github.com/PrefectHQ/prefect/pull/22238#pullrequestreview-2980082843) — the bot — found three distinct bugs across two review passes. First, the backstop service only ever scanned the first 200 `CANCELLING` runs because the query had no pagination cursor, so run 201 would sit in cancellation purgatory until the first batch cleared out. Second, Docket scheduling failures after a committed state transition would 500 the API response even though the flow run was already successfully `CANCELLING` — an ambiguous error that invites client retries against runs already halfway dead. Third, a server restart between the `Cancelled` state commit and the cleanup enqueue would orphan infrastructure forever, since the backstop only scans for `CANCELLING` runs, not timeout-`Cancelled` ones missing their cleanup message. Three bugs, three fix commits. The human reviewer's entire contribution? ["looks good to me!"](https://github.com/PrefectHQ/prefect/pull/22238#pullrequestreview-2987773126)
+
+Credit where it's due: the idempotency scheme is genuinely clever. Using `uuid5(NAMESPACE_URL, ...)` to derive deterministic cleanup message IDs from the flow run and timeout state means duplicate timeout firings produce identical messages and silently deduplicate. It's the kind of correctness guarantee you only think to build after being personally hurt by its absence — which, knowing desertaxle, is less a compliment and more a diagnosis.
+
+**Final Verdict:**
+**The Rube Goldberg Janitor** — 14 files to say `kill -9`

data/roast_bot.json

@@ -1,3 +1,3 @@
 {
-  "last_run": "2026-05-25T13:08:11+00:00"
+  "last_run": "2026-06-29T13:00:00+00:00"
 }