feat(boxen): C M4 #812 -- input_decoder bracketed paste + BOXEN_EV_PASTE

[jsavin]

Summary

Milestone M4 of the Phase C input decoder: bracketed-paste support and a new BOXEN_EV_PASTE event. This is the protocol that solves the "Cmd-V into the REPL under mouse mode shreds the paste into per-keystroke events" problem (the core pain that PR #808 surfaced and that prior milestones were prerequisites for).

The decoder now buffers everything between \e[200~ and \e[201~ into a heap-grown paste buffer (geometric growth from 256 bytes up to a 256 KiB hard cap), normalizes CR/LF line endings to LF at emit time, and emits a single BOXEN_EV_PASTE event with the consumer-owned data pointer. The REPL's boxen_repl_run_one_tick() inserts the paste at the input bar's cursor position, mirroring the typed-char insertion path.

Plan reference: planning/phase_c/INPUT_DECODER_PLAN.md sections 3.8 (bracketed paste), 5.3 (paste as the copy/paste bridge), 4 (PASTE_ACTIVE state), 7 (M4 spec).

ABI Change (contained to this milestone)

boxen.h:

• New event type: BOXEN_EV_PASTE
• New union arm: struct { char *data; size_t len; } paste;

data is heap-allocated by the decoder; ownership transfers to the event consumer, which must free(ev.paste.data). This is the only boxen_event_t variant that transfers heap ownership. The existing event-dispatch sites in boxen.c already have a default: arm and don't need updating; the REPL is the first (and currently only) consumer.

Implementation Notes

• Streaming close-match: the decoder tracks how many bytes of \x1b[201~ have matched independently of paste_buf. Once the truncation cap is hit, we stop appending bytes to the buffer but keep scanning the byte stream for the close marker — otherwise a paste larger than the cap would wedge the decoder in PASTE_ACTIVE forever. Bytes that partially match the close marker are held back and only committed to paste_buf if the match aborts (so the close marker never appears in the emitted payload).
• ESC bytes inside paste are literal: the PASTE_ACTIVE branch runs before the state switch and consumes every byte verbatim. A paste containing \e[A is delivered as the literal 3 bytes, not a KEY_UP event.
• CR/LF normalization at emit time (not during append): keeps the close-marker scan comparing against literal bytes.
• One BOXEN_LOG_W per truncated paste, not per dropped byte (paste_truncated latches).
• Realloc-OOM is safe: on a failed realloc mid-paste we latch truncation, log the warning, and keep whatever bytes we already buffered — the paste still emits as a partial event so the user sees something rather than nothing.

Test Plan

• Unit tests: ./tools/run_headless_tests.sh — 953/953 passed (was 946 + 11 new tests; 4 M4 SKIP stubs removed).
• Integration tests: cd tests && make test-integration — 21 failures, matches pre-existing baseline. All failing tests are unrelated to M4 (html, tcp, startup).
• input_decoder_tests: 6 new behavioral tests (simple, CR/LF normalization, 256 KiB truncation + BOXEN_LOG_W, embedded ESC literal, empty paste, split-across-inject).
• boxen_repl_tests: 5 new dispatch tests (empty input, mid-buffer insert, oversize truncation, empty paste no-op, palette debounce cancellation).
• CLI build (make -C frontier-cli): clean.
• Manual smoke (Cmd-V into a live REPL session): deferred to M6 cutover — the decoder is not yet wired into backend_tb2.c.

What's Next

• M5: Kitty keyboard protocol (CSI-u decoding).
• M6: cutover — replace tb2_poll_event with the custom decoder in backend_tb2.c. This is the milestone that makes paste actually work in a running REPL.
• M7: /mouse toggle + on-demand mouse-mode policy.

Closes #812

[jsavin]

Gate Review: phase-c-m4-bracketed-paste

Verdict: PASS (gate-fixes applied)

Tests

Layer	Result
unit	passed (953/953) at 4956d9e7e
integration	passed (21 pre-existing failures match baseline) at 4956d9e7e
e2e / smoke / lint	none (project has no such layers)

Reviewers

Reviewer	Status
bar-raiser	ran (always)
security	ran (always)
concurrency	ran (auto: matched path frontier-cli/**)
swiftui	skipped (excluded — pure C codebase)

Reviewers ran in the main session (no Task tool surface in this /auto invocation). Each reviewer's checklist was applied against the full diff.

---

P0 — Blocking

None.

P1 — Important

None.

P2 — Nice to Have (FIXED in second commit)

#	Finding	File	Flagged by	Resolution
1	paste_buf_append re-enters realloc on every dropped byte after OOM (latches paste_truncated but doesn't gate the realloc retry). Sustained OOM → thousands of malloc attempts for nothing.	frontier-cli/boxen/input_decoder.c:874	security	Gated the realloc path behind dec->paste_truncated so the OOM/cap path is single-attempt per paste.
2	Redundant big[2048 - 1] = 'q'; write after memset(big, 'q', 2048).	tests/boxen_repl_tests.c	bar-raiser	Dropped the redundant write.

Both fixes applied in commit c64a688c4 (fix(boxen): C M4 #812 gate-fix).

---

Detailed Reviewer Notes

bar-raiser (code quality, bugs, test coverage)

• Test coverage is solid: 6 behavioral tests for the decoder (simple, CR/LF, 256 KiB truncation + warning, embedded-ESC literal, empty, split-across-inject) and 5 dispatch tests for the REPL handler (empty input, mid-buffer insert, oversize truncation, empty no-op, palette debounce cancellation).
• Streaming close-match (paste_close_match independent of paste_buf) is the right call — without it, pastes exceeding the cap would wedge the decoder forever. The mismatch-with-recheck pattern (if (b == PASTE_CLOSE_MARKER[0]) after flushing) handles the corner case where a partial close abort is itself the start of a new close attempt.
• The default: arm in boxen_dispatch_event's switch already covers BOXEN_EV_PASTE (events not handled by the central dispatcher), so the ABI extension doesn't break the existing dispatch sites. REPL is the first and currently only consumer.
• One pre-existing pattern propagated: the input-bar truncation matches the typed-char path (silently drop on full buffer). Acceptable for consistency.

security (memory safety, input validation, IO bounds)

• Bounded allocation: paste buffer caps at 256 KiB per paste; cap is reset on every paste open. A hostile terminal cannot exhaust memory.
• Geometric growth math: INPUT_DECODER_PASTE_INITIAL (256) × 2^10 = INPUT_DECODER_PASTE_CAP (262144) exactly. No integer-overflow risk in size_t arithmetic at any growth step.
• Caller-owned heap pointer: BOXEN_EV_PASTE documents that consumers own ev.paste.data and must free it. Both consumers (the M4 REPL handler and the new test cases) do so unconditionally; free(NULL) is a no-op on the empty-paste branch.
• Decoder destroy frees dec->paste_buf if a paste is in flight — no leak on early shutdown.
• CR/LF normalization runs after close detection, in place with a two-pointer scan where w <= r always holds. No buffer overrun possible.
• ESC-inside-paste literal handling is correct: the PASTE_ACTIVE branch runs before the state switch, so no byte in the paste body re-enters the CSI parser. Verified by test_bracketed_paste_embedded_esc_is_literal.
• realloc OOM preserves the original dec->paste_buf per the standard, and the gate-fix now latches truncation so we don't retry on every subsequent dropped byte.

concurrency (threading, races, GIL contract)

• M4 adds new mutable fields (paste_buf, paste_len, paste_cap, paste_truncated, paste_close_match) to input_decoder_t. All five live under the same single-owner invariant documented in input_decoder.h (lines 13-39) that governs the existing decoder state. No new threading surface introduced.
• BOXEN_LOG_W calls the installed log hook synchronously from inside input_decoder_poll → same owner thread; no new race.
• input_decoder_destroy's free(dec->paste_buf) is called from the owner thread per the existing contract; M6 will enforce no-concurrent-poll-during-shutdown when the real read loop lands. For M4 the test seam is single-threaded and production cutover is deferred.
• The ABI change is a pure addition (new enum value + new union arm); doesn't affect the existing GIL contract for KEY/MOUSE/RESIZE events.

---

Stats

• Files changed: 6 (boxen.h, input_decoder.c, boxen_repl.c, Makefile, two test files)
• Lines: +874 / -39 (net + new tests + ~120-line PASTE_ACTIVE branch + 72-line REPL handler)
• Findings: 0 P0, 0 P1, 2 P2 (both fixed)
• ABI change: contained to this PR; documented in PR body and on boxen.h

Next Steps

Branch is ready to merge from a /gate perspective. Manual smoke (Cmd-V into a live REPL) is deferred to M6 cutover — the decoder is not yet wired into backend_tb2.c.