Skip to content

feat(tui): active terminal graphics detection (kitty query probe) + SENPI_IMAGES override #1

Description

@minpeter

Summary

detectCapabilities() in packages/tui/src/terminal-image.ts detects terminal image support exclusively from environment variables. Over SSH / EternalTerminal (ET) those variables never reach the remote host, so senpi misjudges a fully kitty-graphics-capable terminal as "unknown" and silently degrades images to a one-line text fallback — even though the transport passes the graphics escapes through transparently.

This issue proposes two complementary fixes:

  1. Active capability probing via the kitty graphics protocol query (transport- and env-agnostic).
  2. SENPI_IMAGES env override as an explicit escape hatch.

Problem

Detection currently keys off client-side terminal identity envs:

Variable Detects
KITTY_WINDOW_ID, TERM_PROGRAM=kitty kitty
GHOSTTY_RESOURCES_DIR, TERM contains ghostty ghostty
WEZTERM_PANE, TERM_PROGRAM=wezterm wezterm
WARP_SESSION_ID, TERM_PROGRAM=warpterminal warp
ITERM_SESSION_ID, TERM_PROGRAM=iterm.app iTerm2

None of these survive a remote session:

  • SSH forwards only TERM by default. Anything else needs SendEnv on every client and AcceptEnv on every server's sshd_config.
  • ET parses ssh config itself and ignores some directives (e.g. SetEnv — see MisterTea/EternalTerminal#679), so ssh-style env forwarding is not reliable there.
  • TERM is frequently sanitized to xterm-256color for terminfo compatibility (a common Ghostty setup — see ghostty-org/ghostty#3161). Even when TERM does survive, the TERM-substring check only covers ghostty; xterm-kitty is not matched at all.
  • No active probe exists. The only terminal query in use is the cell-metrics query (CSI 16 t, packages/tui/src/tui.ts:866), which is used for sizing, not protocol detection.

Result: fallback to { images: null, ... }imageFallback() text in packages/tui/src/components/image.ts.

Reproduced live

Session over ET 7.0.0, local terminal Ghostty, remote shell env:

TERM=xterm-256color
TERM_PROGRAM=<unset>
ET_VERSION=7.0.0

detectCapabilities() → {"images":null,"trueColor":false,"hyperlinks":false}

After adding export TERM_PROGRAM=ghostty to the remote shell rc:

detectCapabilities() → {"images":"kitty","trueColor":true,"hyperlinks":true}

… and kitty-protocol images render correctly through the ET pipe. Detection is the only blocker; transport is fine.

(Note: mosh is a different story — its state-synchronization design drops all non-grid sequences, so no client-side fix is possible. Out of scope here; see mobile-shell/mosh#1248.)

Proposal

1. Active probe: kitty graphics query

Probe the terminal directly at TUI startup, next to the existing CSI 16 t cell-size query infrastructure in packages/tui/src/tui.ts:

→  \x1b_Gi=31,s=1,v=1,a=q,t=d,f=24;AAAA\x1b\\
←  \x1b_Gi=31;OK\x1b\\        (kitty-protocol terminals)
  • Any \x1b_Gi=31;... APC response means images: "kitty".
  • Terminals without kitty support silently ignore APC sequences → bounded timeout (~100–300 ms) → keep the env-based result.
  • Must not block startup: run alongside the existing query round-trip, or resolve lazily before the first image render; the env-based result remains the pre-timeout default.
  • Response parsing must live in the stdin sequence parser without consuming unrelated query responses (e.g. the CSI 16 t reply).

Spec: kitty graphics protocol — query support.

2. Escape hatch: SENPI_IMAGES

Value Effect
kitty force kitty protocol
iterm2 force iTerm2 protocol
off force disable images

Checked before all other detection. Covers cases a probe cannot (non-TTY output, exotic multiplexers, CI).

Detection precedence

  1. SENPI_IMAGES (explicit user override, including off)
  2. tmux/screen branch → images: null (unchanged; probe never runs there)
  3. Kitty query probe → "kitty"
  4. Existing env-based detection (unchanged fallback)

Acceptance criteria

  • Over SSH or ET with no terminal identity envs forwarded and TERM=xterm-256color, senpi on a kitty-protocol terminal renders images with zero shell-rc workarounds.
  • No visible probe artifacts and no startup delay regression on terminals without kitty support (timeout budget respected).
  • SENPI_IMAGES=kitty|iterm2|off wins over env detection and probe result in all combinations.
  • tmux/screen behavior unchanged (images: null).
  • True-color/hyperlink detection paths untouched.
  • Tests: probe-response parsing, timeout fallback, override precedence — headless coverage using the existing virtual terminal harness (packages/tui/test/virtual-terminal.ts, pattern after packages/tui/test/terminal-image.test.ts).

Non-goals (follow-up issues)

  • tmux passthrough: DCS passthrough wrapping + allow-passthrough probing. Currently hard-disabled by design ("Image protocols are unreliable under tmux").
  • sixel support.
  • iTerm2 active probing (no standard query; env detection is adequate for local iTerm2).

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions