Skip to content

feat(boxen): C M1 #809 -- input_decoder PTY-replay harness + stub#816

Merged
jsavin merged 2 commits into
developfrom
phase-c-m1-decoder-harness
Jun 30, 2026
Merged

feat(boxen): C M1 #809 -- input_decoder PTY-replay harness + stub#816
jsavin merged 2 commits into
developfrom
phase-c-m1-decoder-harness

Conversation

@jsavin

@jsavin jsavin commented Jun 30, 2026

Copy link
Copy Markdown
Owner

Summary

Phase C input-decoder milestone M1: PTY-replay test harness + decoder
stub. Pure infrastructure — no production input-path behavior changes.
backend_tb2.c is untouched (the M6 cutover wires the decoder; M1 only
adds the leaf module and the harness against it).

Plan: planning/phase_c/INPUT_DECODER_PLAN.md
(see section 7 for M1 spec, section 6 for testing strategy).

What this adds

  • frontier-cli/boxen/input_decoder.h — private API (typedef,
    lifecycle, poll, mouse / Kitty enable, INPUT_DECODER_TEST_SEAM-guarded
    inject + buffered-bytes accessor). Header is private to the boxen
    subsystem, sibling to boxen_internal.h.
  • frontier-cli/boxen/input_decoder.c — minimal skeleton. 4 KiB linear
    scratch buffer; NULL-safe everywhere. poll() always returns
    BOXEN_ERR_TIMEOUT (the state machine that drains the ring is M2).
    Compiled into the frontier-cli production binary via BOXEN_SOURCES so
    the build stays green, but NOT yet wired into tb2_poll_event (that is
    M6).
  • tests/input_decoder_tests.c — TR_RUN harness, 61 tests:
    • 12 M1 behavioral tests that pass today (create / destroy round-trip,
      NULL safety on every entry point, mouse-enable round-trip, inject
      deposits exactly N bytes, poll-on-empty returns TIMEOUT and zeroes
      the out parameter, etc.).
    • 49 SKIP stubs ranged by milestone: M2 cursor keys + modifier
      params + ESC + control chars + UTF-8 (27), M3 SGR mouse + X10 +
      burst-read (10), M4 bracketed paste (4), M5 Kitty CSI-u (4). Each
      stub prints [SKIP] <protocol entry> to stderr so deferrals are
      visible in the log; each will be rewritten as a behavioral assert
      when its milestone lands.
  • tests/Makefile — new input_decoder_tests target. Link slice:
    input_decoder.c only — no termbox2, no Frontier runtime, no boxen
    core. Built with -DINPUT_DECODER_TEST_SEAM to expose the inject path.
  • frontier-cli/Makefileinput_decoder.c added to BOXEN_SOURCES.

SKIP mechanism note

test_report.h does not define TR_SKIP. The plan section 7 left this
open ("assert(false && "not yet implemented") or TR_SKIP equivalent")
but assert(false) would fail the test. The harness instead uses an
early-return helper tr_skip(const char *reason) that prints
[SKIP] <reason> to stderr and returns — TR_RUN then records the
surrounding test as passing.

Trade-off: the JSON tally (tests/tmp/unit/input_decoder_tests.json)
reports 61/61 green rather than 12/12 + 49 skipped. The deferrals are
recoverable from the stderr log, not the JSON. This was the only path
that keeps a single test binary green without claiming protocol behavior
the decoder has not yet implemented. M2 will replace each stub with the
real behavioral assert, so the tally naturally tightens as milestones
land. If a structured SKIP signal in the JSON becomes important, that's
a separate refactor against test_report.h itself (out of M1 scope).

Done criterion (per plan section 7, M1)

make -C tests input_decoder_tests && ./tests/input_decoder_tests
  → compiles, passes 61/61 (12 behavioral + 49 SKIP stubs)

Test plan

  • Unit: ./tools/run_headless_tests.sh882/882 pass
    (pre-existing 821 + 61 new input_decoder_tests, no regressions).
  • Integration: cd tests && make test-integration — 2186 pass /
    21 fail — matches the pre-existing baseline (failures in html/tcp/startup,
    none touching boxen / input decoder / REPL).
  • Build: make -C frontier-cli — clean, no new warnings.
  • New test binary: make -C tests input_decoder_tests && ./tests/input_decoder_tests
    — 61/61 pass (12 real + 49 SKIP).
  • Manual: none required — M1 is harness-only, no user-visible
    behavior change. Manual smoke begins at M2 (Option-Left/Right
    recognition per plan section 6.4).

What is explicitly out of scope for M1

Listed here so reviewers can confirm the PR honors the milestone boundary
rather than leaking into later work:

  • The escape-sequence state machine (GROUND / ESC_RECEIVED / CSI_COLLECTING
    / SS3_RECEIVED) — M2.
  • SGR mouse, X10 fallback, burst-read — M3.
  • Bracketed paste, BOXEN_EV_PASTE ABI add — M4.
  • Kitty CSI-u decode + probe — M5.
  • Cutover in backend_tb2.cM6.
  • /mouse on/off slash command — M7.

backend_tb2.c, ws_server.c, ws_server.h, databases/Virgin.root,
and the rendering path (tb_set_cell, tb_present, tb_clear,
tb_width, tb_height) are all untouched. No call to
tb_set_input_mode(TB_INPUT_MOUSE) is added.

Files

frontier-cli/Makefile               |   9 +-
frontier-cli/boxen/input_decoder.c  | 199 ++++++++++++++
frontier-cli/boxen/input_decoder.h  | 124 +++++++++
tests/Makefile                      |  15 +-
tests/input_decoder_tests.c         | 531 +++++++++++++++++++++++++++
5 files changed, 876 insertions(+), 2 deletions(-)

Closes #809

jsavin added 2 commits June 29, 2026 18:42
Phase C input-decoder milestone M1.  Pure infrastructure: a stub decoder
module and a 61-test harness that exercises the M1 API surface (create /
destroy / inject-bytes / poll / mouse-enable / kitty-enable).

What this adds:
- frontier-cli/boxen/input_decoder.h -- private API (typedef, lifecycle,
  poll, mouse / kitty enable, INPUT_DECODER_TEST_SEAM-guarded inject /
  buffered-bytes accessor).
- frontier-cli/boxen/input_decoder.c -- minimal skeleton: 4 KiB linear
  scratch buffer, NULL-safe everywhere.  poll() always returns
  BOXEN_ERR_TIMEOUT (the state machine lands in M2).  Wired into the
  frontier-cli production build so the CLI continues to link, but NOT yet
  wired into backend_tb2.c (the M6 cutover).
- tests/input_decoder_tests.c -- TR_RUN harness.  12 M1 behavioral tests
  (create / destroy round-trip, NULL safety, mouse-enable round-trip,
  inject deposits bytes, poll-on-empty returns TIMEOUT, etc.).  49 SKIP
  stubs ranged by milestone (M2: cursor keys + UTF-8, M3: mouse, M4:
  paste, M5: Kitty).  Each SKIP stub prints "[SKIP] <protocol entry>" to
  stderr so deferrals are visible in the test log; the M2-M5 sub-agents
  rewrite each stub as a behavioral assert when its milestone lands.
- tests/Makefile -- new input_decoder_tests target (link slice:
  input_decoder.c only, no termbox2, no Frontier runtime; uses
  -DINPUT_DECODER_TEST_SEAM to expose the inject seam).
- frontier-cli/Makefile -- input_decoder.c added to BOXEN_SOURCES.

SKIP mechanism note (no TR_SKIP exists in test_report.h): the SKIP stubs
return without firing assert(), so TR_RUN records them as passing.  This
keeps the 61/61 green count accurate (we are not claiming protocol
behavior we have not implemented).  The early-return + stderr SKIP line
makes the deferral visible in the log without polluting the JSON tally.

Done criterion (per plan section 7, M1):
  make -C tests input_decoder_tests && ./tests/input_decoder_tests
  --> compiles, passes (61/61 green, of which 49 print SKIP).
  Full unit suite: 882/882 (was 821 pre-existing, +61 new).

Out of scope for M1 (per plan section 7): the state machine (M2), SGR
mouse (M3), bracketed paste with BOXEN_EV_PASTE ABI add (M4), Kitty
keyboard protocol (M5), backend_tb2 cutover (M6), /mouse toggle (M7).
No production input-path behavior changes; backend_tb2.c is untouched.

Plan: planning/phase_c/INPUT_DECODER_PLAN.md
Closes #809
Round 2 of M1: fold in the three P1 findings from the local /gate review
(security + concurrency reviewers) plus the cheap P2s that are part of
the same edits.  No behavior change to the existing 12 M1 tests; one new
test exercises the back-pressure surface.

Concurrency P1 -- input_decoder.h threading contract was wrong:
- Previous text said "single-threaded, GIL-held" then "releases the GIL
  before entering the read loop", which contradict each other and would
  have misled the M6 sub-agent.
- Rewrite the threading block to state the single-owner invariant
  explicitly: exactly one decoder per process, GIL held at every API
  entry/exit, GIL dropped ONLY inside the M6 blocking read inside
  input_decoder_poll, mutations during that window are safe because no
  other thread holds a pointer to the decoder.

Concurrency P1 -- set_mouse / kitty_enable owner-thread invariant was
implicit:
- Document that these must be called from GIL-held context AND only when
  no poll is in progress.  The REPL event loop satisfies this naturally
  (slash commands dispatch between polls), but it needs to be a
  documented invariant rather than an accident, because M3+ will turn
  these into TTY writes that would race the in-flight read(2) otherwise.

Security P1 -- silent drop-on-overflow contract:
- Previous inject_bytes returned void and silently truncated.  M2/M3
  read-loop overflow would have been an invisible parser-desync surface.
- Change inject_bytes to return the number of bytes accepted (size_t).
- Add a monotonic dropped_bytes counter on the struct, exposed via
  input_decoder_dropped_bytes() (test seam, mirrors buffered_bytes).
- Add behavioral test test_inject_overflow_increments_dropped_counter
  that floods 5 KiB into the 4 KiB ring and asserts accepted+dropped =
  attempted, then verifies a second inject when full drops everything.

Cheap P2s folded in:
- Rename head_len -> fill_len (bar-raiser P2 #6: "head" implied an index,
  it's actually a fill level).  Will keep the M2 state-machine commit
  honest about what's an index vs a count.
- assert(fill_len <= INPUT_DECODER_BUFFER_SIZE) at top of inject_bytes
  (bar-raiser P2 #4): cheap insurance for the M2 drain refactor.
- assert(bytes != NULL) when len > 0 (security P2): test seam should
  catch caller bugs, not silently no-op.
- Add input_decoder_kitty_enabled() symmetric query (bar-raiser P2 #7):
  the field was write-only with no observer; tests can now assert the
  bit flipped, and M5 has the seam pre-built.
- Document tty_fd ownership: caller owns the fd, decoder does not
  validate or close it (bar-raiser P2 #5/8).
- Document SIGWINCH non-interaction (concurrency P2 #12).
- Unify the threading comment between header and .c (concurrency P2 #11).

Test-harness P2 (bar-raiser #5):
- Rename all 49 SKIP-stub functions test_* -> test_skip_* so a grep over
  tests/tmp/unit/input_decoder_tests.json discloses the deferred set
  without parsing stderr.
- Add g_skip_count counter bumped by tr_skip(); main() prints
  "[skip-summary] 49 of 62 tests are SKIP stubs deferred to M2-M5"
  alongside the green tally so the 62/62 doesn't conceal unimplemented
  behavior.

Test count:
  Before: 61/61 (12 real + 49 SKIP).
  After:  62/62 (13 real + 49 SKIP).
  Net suite: 882 -> 883 (no other tests touched).

Deferred to follow-up issues (P2s not folded in here):
- Forward-compat note about future M2 signed/unsigned hazard in drain
  arithmetic (security P2 #10) -- belongs in the M2 PR review checklist.
- "Document the implementation's actual behavior of bytes==NULL" --
  resolved by switching to assert (no separate doc needed now).

Plan: planning/phase_c/INPUT_DECODER_PLAN.md
PR: #816
@jsavin

jsavin commented Jun 30, 2026

Copy link
Copy Markdown
Owner Author

/gate review round 1 — branch HEAD aa8edf0 (initial commit)

Three reviewers ran per .claude/gate.yaml: bar-raiser (always), security
(always), concurrency (auto: matched path glob frontier-cli/** + content
pattern GIL). swiftui excluded (pure C codebase).

Verdict: PASS WITH NOTES

  • P0: 0
  • P1: 3 (all addressed in commit c886b52e)
  • P2: 10 (6 folded into c886b52e, 4 deferred — see below)

P1 findings (fixed in c886b52e)

# Source Finding Resolution
1 concurrency input_decoder.h:9-10 threading contract says "single-threaded, GIL-held" then "releases the GIL before entering the read loop" — contradictory; would mislead the M6 sub-agent. Rewrote the threading block: single-owner invariant explicit (one decoder per process, GIL held at entry/exit, GIL dropped only inside the M6 blocking read, no other thread holds a pointer during that window).
2 concurrency set_mouse / kitty_enable owner-thread invariant not documented across the M6/M7 boundary — M3+ would turn these into TTY writes that race in-flight read(2). Documented: must be called from GIL-held context AND only when no poll is in progress. REPL event loop satisfies this naturally (dispatch between polls).
3 security input_decoder.c:166-190 silent drop-on-overflow contract — M2/M3 read-loop overflow would have been invisible parser-desync surface (CWE-130, CWE-400). Changed inject_bytes to return size_t (accepted count). Added monotonic dropped_bytes counter on struct + input_decoder_dropped_bytes() accessor. New behavioral test test_inject_overflow_increments_dropped_counter floods 5 KiB into 4 KiB ring, asserts accepted+dropped == attempted.

P2 findings folded in (cheap fixes in same commit)

  • head_lenfill_len rename (bar-raiser): "head" implied an index; it's a fill level.
  • assert(fill_len <= INPUT_DECODER_BUFFER_SIZE) at top of inject_bytes (bar-raiser).
  • assert(bytes != NULL) when len > 0 (security): test seam should catch caller bugs.
  • Added input_decoder_kitty_enabled() symmetric query (bar-raiser): write-only field gained an observer; M5 has the seam pre-built.
  • Documented tty_fd ownership (bar-raiser): caller owns; decoder does not validate or close.
  • Documented SIGWINCH non-interaction (concurrency).
  • Unified threading comment between header and .c file (concurrency).
  • Renamed 49 SKIP-stub functions test_*test_skip_* (bar-raiser): grep over JSON tally now discloses deferred set without parsing stderr.
  • Added g_skip_count + [skip-summary] print line: green 62/62 tally no longer conceals 49 unimplemented stubs.

P2 findings deferred (will track if they recur)

Test results at c886b52e

  • Unit suite: 883/883 (was 882; +1 from new test_inject_overflow_increments_dropped_counter).
  • New test binary: 62/62 (13 real M1 behavioral + 49 SKIP stubs; "[skip-summary] 49 of 62 ..." visible in log).
  • Integration suite at aa8edf0: 2186 pass / 21 fail — matches pre-existing baseline (failures in html/tcp/startup, none touching boxen/input decoder/REPL). Not re-run at c886b52e because no production-path code touched.
  • CLI build (make -C frontier-cli): clean.

Reviewer raw outputs

Full transcripts available locally at:

  • tasks/a43d9232de6dc6044.output (bar-raiser)
  • tasks/a86547a20462b49a0.output (security)
  • tasks/a4dba6f39ec0d6dcc.output (concurrency)

Local /gate review was the canonical pre-merge review per Frontier's
local-first posture (no external review bot configured). No P0/P1
remaining; ready for human approval to merge.

@jsavin jsavin merged commit 2d08170 into develop Jun 30, 2026
1 check passed
@jsavin jsavin deleted the phase-c-m1-decoder-harness branch June 30, 2026 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase C input decoder M1: PTY-replay harness + decoder stub

1 participant