stdout & stderr output escaping

[artem-zinnatullin]

This is a tracking/discussion issue for stdout and stderr output escaping.

This is pretty generic problem, however normally not that many programs face it.

My example

In Mainframer we pipe output from ssh to the Rust program's stdout and stderr respectively.

If ssh is forced to run in interactive mode (-t) where pseudo-terminal is allocated and the program we're running via ssh does some complicated progress output (Gradle build system) it can lead to Terminal state being corrupted.

Generic example

If you cat some binary file, there is a good chance it'll break Terminal state.

To my understanding, some unexpected control symbols leak through stdout/stderr and break Terminal state.

Hacky Solutions

In some cases (not my though) running:

$ reset

After the Rust program will fix Terminal.

In my case, running:

$ stty sane

Fixes the Terminal.

[Screwtapello]

Terminals are very, very stateful things, but as a general rule, if you remove or escape all control characters (bytes in 0x00..=0x1F, in Rust syntax), you should be fine. Traditionally that's done by printing ^ followed by the control character + 32 (so byte 0x03 becomes "^C"), but other schemes are possible.

While safe, the result is not always pretty, since terminal escape sequences like "change text colour" will be output literally, leaving ^]]42;36m gibberish in the output. Unfortunately, recognising terminal control sequences so they can be stripped out is difficult; there's rather a lot of them.

A streaming lexer for terminal control sequences would be a cool thing to have, though.

[artem-zinnatullin]

Do you think we should document how programs need to respect TTY vs piping in this book?

There are cases when programs need to respect that, an example could be "interactive progress output", when run without TTY (for example on CI) — it should be very basic or even omitted