Durability Window Lab

An interactive POSIX awk simulation of the gap between a transaction returning success and its write-ahead log becoming crash-durable.

All four default strategies report 100% commit success. Their recovery evidence is very different:

Strategy	Acknowledgement boundary	Default incident
Buffered acknowledgement	WAL copied into the OS cache	1,740 acknowledged transactions lost
Timed fdatasync	WAL flushed later on an application timer	140 acknowledged transactions lost
Local group commit	Shared local WAL flush completes	Process-crash safe; 15 transactions exposed to primary loss
Quorum durable	Local and synchronous replica flush complete	Process- and primary-loss safe, with higher latency

The lab demonstrates why successful responses alone cannot establish a durability SLO.

Run it

Requirements:

• A POSIX-compatible awk (awk, mawk, or gawk)
• Python 3.9+
• jq, Node.js, and curl for the test suite

make run

Open http://localhost:8080. Every control change reruns the awk model through a small Python standard-library HTTP adapter.

Or use Docker:

docker build -t durability-window-lab .
docker run --rm -p 8080:8080 durability-window-lab

Exercise the model directly

awk -f src/model.awk | jq '.strategies[].summary'

Parameters are integer environment variables:

DUR_BACKGROUND_FLUSH_SECONDS=5 \
DUR_APP_FLUSH_MS=500 \
DUR_CRASH_SECOND=73 \
DUR_CRASH_OFFSET_MS=475 \
  awk -f src/model.awk | jq

The output includes a per-second timeline for all strategies and summaries for:

• acknowledged but non-durable transactions
• acknowledged transactions lost at the configured crash
• recovered durability coverage
• WAL buffer size
• acknowledgement p99
• fsync operations
• remote durable lag
• transactions exposed to primary loss

What is modeled

This is a deterministic teaching model, not a storage benchmark.

The workload emits a fixed number of transactions per second. A crash occurs at the configured second and millisecond offset:

• Buffered acknowledgement returns success after a write enters the operating-system cache. A background writer flushes the WAL every configured number of seconds.
• Timed fdatasync bounds the exposure with an application-level flush interval but continues acknowledging transactions inside that interval.
• Local group commit queues transactions behind a shared fsync and only responds after local durability. This survives a process or operating-system crash but does not prove another host owns the commit.
• Quorum durable waits for both local durability and a synchronous replica flush before responding.

The model intentionally keeps apparent_commit_success_percent at 100 for all policies. Durable recovery coverage uses:

(acknowledged transactions - acknowledged transactions lost) /
acknowledged transactions

The telemetry that resolves the incident

Join the client promise to the WAL custody chain:

transaction_id
  + acknowledged_lsn
  + local_durable_lsn
  + replica_flush_lsn
  + recovery_end_lsn
  + commit_mode

This lets an investigation answer:

1. Which exact WAL position was promised to the client?
2. Had that position reached durable local storage?
3. Had it crossed the primary host boundary?
4. Where did crash recovery actually end?
5. Which commit mode created each promise?

telemetry.sh makes that high-cardinality custody chain queryable without reducing the incident to component-local averages.

Implementation

• src/model.awk — portable deterministic model and JSON encoder
• bin/server.py — dependency-free HTTP and process adapter
• public/ — responsive UI and native SVG timeline without frontend dependencies
• t/model-test.sh — deterministic replay, scenario invariants, and boundary checks
• tests/ — HTTP, UI contract, and production-container checks

Run all native checks:

make check

Exercise the production image:

make container-test

References

• A successful write() does not guarantee that data has reached disk: Linux write(2) manual
• fsync() blocks until the storage device reports the transfer complete: Linux fsync(2) manual
• PostgreSQL documents the same acknowledgement-versus-durability tradeoff for asynchronous commit, including the bounded WAL writer risk window: PostgreSQL — Asynchronous Commit
• The model uses portable awk language facilities: GNU Awk User’s Guide

License

MIT