feat: support OSC envelope for RGP (fixes #67, unblocks RGP on Windows)#68
Open
ahostbr wants to merge 4 commits into
Open
feat: support OSC envelope for RGP (fixes #67, unblocks RGP on Windows)#68ahostbr wants to merge 4 commits into
ahostbr wants to merge 4 commits into
Conversation
The Ratty Graphics Protocol previously rode an APC envelope (`\x1b_ratty;g;…\x1b\`). Windows ConPTY strips APC sequences before they reach the host process in every shipping conhost — verified by a portable-pty-backed probe that sends `printf '\x1b_…\x1b\'` and reads the PTY master. APC body never arrives; OSC payloads at arbitrary numbers (1337, 9001, 99999, 52, 0) survive intact. Changes to make RGP work over OSC end-to-end: * `rgp.rs`: add `RGP_OSC_START = b"\x1b]8901;ratty;g;"`. Refactor `consume_sequence` to strip either envelope (APC ends on ST/C1-ST; OSC also accepts BEL) and dispatch the common body grammar. * `inline.rs`: extend the PTY scan loop with an `EnvelopeKind` discriminator that picks the next APC *or* the specific RGP OSC prefix. Unrelated OSC traffic (window titles, hyperlinks, prompt marks) still flows straight through to vt100. * `inline.rs` + `systems.rs`: fix a pre-existing scroll-tracking false-positive that the OSC migration exposes. When an `\x1b]…\x1b\` Place arrives mid-frame, `infer_upward_scroll` can return a large shift (e.g. 25 rows on a 32-row screen) because the chunk that carried the Place also cleared and repainted the terminal. The old `apply_scroll` then evicts the very anchor that chunk just set. Track which object ids had a Place inside the chunk (`placed_this_chunk`), drain it after consume, and scroll only the intersection of pre-chunk anchors minus the freshly-placed set. The public `apply_scroll` API is unchanged; new `apply_scroll_to_existing` takes the exemption set. OSC 8901 choice: 4-digit private number with no known assignment. Numbers explicitly avoided: 0,1,2,4,7,8,9,10,11,12,52,104,110-119 (recognised by conhost), 99 (kitty notifications), 133 (FinalTerm / Warp), 633 (VS Code shell integration), 777 (urxvt), 1337 (iTerm2), 9001 (Microsoft Terminal's `WTAction` in `src/terminal/parser/OutputStateMachineEngine.hpp` — dispatching is numeric-first, so the `;ratty;` discriminator does not protect against a numeric collision). Existing APC senders keep working: the parser tries both envelopes per sequence. The widget switches to OSC in a follow-up commit. Verified on Windows 11 26200.6457 + RTX 5090: `big_rat.exe` running inside `ratty.exe` registers, places, and renders the SpinyMouse GLB end-to-end.
Updates `ratatui-ratty`'s seven emit sites (register path, register payload init/chunk/empty, place, update, delete) from APC (`\x1b_ratty;g;…\x1b\`) to OSC 8901 (`\x1b]8901;ratty;g;…\x1b\`). The body grammar is unchanged: `<verb>;<key=value>;…`. Existing parsers that handle the old APC form keep working unchanged because the matching Ratty parser commit accepts both envelopes during the migration. Required on Windows because ConPTY strips APC mid-line. Verified by the matching Ratty parser commit on Win11 26200.6457 — `big_rat` and `document` examples register, place, and render their inline 3D objects end-to-end inside `ratty.exe`. ST (`\x1b\`) is emitted as the OSC terminator; the parser also accepts BEL for compatibility with terminals that prefer the older form.
Owner
|
Does it require any updates in the protocol document? |
Adds a Transport subsection covering the OSC compatibility envelope: when to use it, the number selection rationale, and the collision avoidance table. Both envelopes carry identical body grammar. Agent-Name: Sentinel Agent-ID: bf13c4e3-4681-4470-9d0e-663aa2f1d3c9 Agent-Tier: orchestrator
Contributor
Author
|
Good call — pushed
|
Agent-Name: Sentinel Agent-ID: bf13c4e3-4681-4470-9d0e-663aa2f1d3c9 Agent-Tier: orchestrator
Owner
|
Thanks for clarifying. I probably need to test this on Windows before I go ahead and review it. I also appreciate more eyes on this PR 🙏🏼 If anyone uses Windows and tries Ratty, feel free to chime in on this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OSC envelope (
ESC ] 8901 ; ratty ; g ; … ESC \) as an alternative to the existing APC envelope for the Ratty Graphics Protocol, and fixes a related scroll-tracking false-positive. Together these unblock RGP-based widgets — including the bundledbig_rat,draw, anddocumentexamples — on Windows. Fixes #67.Why
Windows ConPTY strips APC sequences (
ESC _ … ESC \) before they reach the host process. Verified with a minimalportable-pty-backed probe: a child runningprintf '\x1b_PROBE;hello=world\x1b\'arrives at the PTY master with only the bash init CSIs (\x1b[?9001h\x1b[?1004h) — APC body and ST terminator gone. Equivalent OSC payloads at arbitrary numbers (1337, 9001, 99999, 52, 0) survive intact end-to-end.This is structural in conhost, not a configuration issue:
PSEUDOCONSOLE_PASSTHROUGH_MODEisn't a real public flag. The value0x8in currentwinconpty.hisPSEUDOCONSOLE_GLYPH_WIDTH_GRAPHEMES(not passthrough). The#[allow(dead_code)]constant inportable-ptytraces back to a 2019 proposal that never shipped.microsoft/terminalv1.24conpty.dll+OpenConsole.exedoes not change the APC behavior (empirically tested). The 1.22 "passthrough" PR (#17510) was an internal conhost refactor that enables conhost to render unrecognised sequences (Sixel) in its own renderer — not a forwarding feature for third-party hosts.Changes
src/rgp.rs— addsRGP_OSC_START = b"\x1b]8901;ratty;g;".consume_sequenceis refactored to recognise either envelope and strip the right terminator (APC ends on ST/C1-ST; OSC also accepts BEL). The body grammar (<verb>;<key=value>;…) is unchanged, so APC and OSC paths share the same parser.src/inline.rs—consume_pty_output's scan loop gains anEnvelopeKinddiscriminator; it picks the next APC (\x1b_) or the specific RGP OSC prefix and dispatches each through the right path. Unrelated OSC traffic (window titles, hyperlinks, prompt marks) flows straight through to vt100 as before.src/inline.rs+src/systems.rs— fixes a pre-existing scroll-tracking false-positive that the OSC migration exposes. When aPlacearrives in the same PTY chunk that clears and repaints the terminal,infer_upward_scrollcan return a large shift (25 rows on a 32-row screen in this repro) and the oldapply_scrollevicts the very anchor that chunk just set. Tracked via a newplaced_this_chunk: HashSet<u32>onTerminalInlineObjects;pump_pty_outputdrains it after consume and scrolls onlypre_anchor_ids \ placed_this_chunk. Publicapply_scroll(rows)API is unchanged; newapply_scroll_to_existing(rows, exempt)is the underlying helper.widget/src/lib.rs—ratatui-ratty's seven emit sites (register path, register payload init/chunk/empty, place, update, delete) switch from APC to OSC 8901. ST terminator emitted; parser also accepts BEL.OSC 8901 — number choice
Picked as a 4-digit private number with no known assignment in any surveyed terminal codebase, registry, or shell-integration spec. Numbers explicitly avoided:
WTActioninsrc/terminal/parser/OutputStateMachineEngine.hpp— dispatching is numeric-first, so the;ratty;discriminator does not protect against a numeric collisionThe
;ratty;discriminator after the number namespaces the payload for the protocol family, leaving room forg;(graphics) plus future verb-spaces if you want them.Verification
Windows 11 Pro 26200.6457 / RTX 5090 / Vulkan backend.
cargo fmt --check,cargo check,cargo clippy --all-targets --all-features -- -D warningsall green.Wire-level probe (vendored
portable-pty 0.8.1shelling out tobash.exe):End-to-end inside
ratty.exe—ratty -e big_rat.exe:The full RGP register → place → spawn → render chain fires, and the SpinyMouse GLB renders inside Ratty's viewport for the first time on Windows. Screenshot below.
Compatibility
Existing APC senders keep working. The parser tries both envelopes per sequence, and
ratatui-ratty's on-the-wire behavior is the only change for downstream widget consumers — they get a one-time API recompile but nothing breaks in the body grammar.Open follow-ups (separate work)
placeis already self-describing (row,col,w,hcarry absolute coordinates), so reordering does not silently mis-anchor the object, but a freshness key (seq=orframe=) would defeat stale-arrival cases. Worth a follow-up if it ever bites in practice.\ePtmux;…\e\passthrough wrapper if a user runs RGP-emitting apps through them. Out of scope here.