Errors raised in argparse are colorized when redirecting stderr to a file

[mpkocher]

Bug report

Bug description:

(Some) Errors raised in argparse will be colorized when redirecting stderr to a file.

Requirements:

1. Errors redirected to a file are not colorized
2. Unittest added to cover the case of error and redirecting of stderr

The most minimal example can be taken from calendar from the stdlib.

Works as expected with the color displayed correctly.

./python.exe -m calendar 2020 01 01

Redirecting stderr

./python.exe -m calendar 2020 01 01 2> err
cat -v err
^[[1;34musage: ^[[0m^[[1;35mpython.exe -m calendar^[[0m [^[[32m-h^[[0m] [^[[32m-w ^[[33mWIDTH^[[0m] [^[[32m-l ^[[33mLINES^[[0m] [^[[32m-s ^[[33mSPACING^[[0m] [^[[32m-m ^[[33mMONTHS^[[0m] [^[[32m-c ^[[33mCSS^[[0m] [^[[32m-L ^[[33mLOCALE^[[0m] [^[[32m-e ^[[33mENCODING^[[0m] [^[[32m-t ^[[33m{text,html}^[[0m] [^[[32m-f ^[[33mFIRST_WEEKDAY^[[0m] ^[[32m[year]^[[0m ^[[32m[month]^[[0m
python.exe -m calendar: error: unrecognized arguments: 01

Redirecting stdout will trigger the "correct" behavior

./python.exe -m calendar 2020 01 01 >out 2> err
(zatters-314) mkocher@zudio cpython % cat -v err
usage: python.exe -m calendar [-h] [-w WIDTH] [-l LINES] [-s SPACING]
                              [-m MONTHS] [-c CSS] [-L LOCALE] [-e ENCODING]
                              [-t {text,html}] [-f FIRST_WEEKDAY]
                              [year] [month]
python.exe -m calendar: error: unrecognized arguments: 01

Observations

1. Calling can_colorize(file=None) assumes that the stdout is used when file=None This implicit use of can_colorize() makes the code hard to reason about.
2. Calling _set_color always assumes stdout.
3. Calling self.print_usage(_sys.stderr) is breaking the assumption that stdout is being used.
4. global ENV var design choice make data flow in the code confusing to debug. Depending on how your argparse code is written and the pattern/style of raising errors, some errors raised are being colorized properly by traceback layer. For example, using parser.error(...) or raise ArgumentTypeError(...) vs a raise ValueError() within your application code.
5. I think users are going to be a bit confused when setting ArgumentParser(color=False) and the error (might) still be colorized (because of the traceback layer determining if it should be colorized). It is documented that configure for traceback is done via ENV vars, but it's not particular clear how the config of these components overlap.
6. the specific error is not colorized (python.exe -m calendar: error: unrecognized arguments: 01) but the usage is?

Stand alone example for debugging:

import sys
from argparse import ArgumentParser, ArgumentTypeError


def validate_lower(sx: str) -> str:
    if any(not s.islower() for s in sx):
        # this will be handled from within argparse.
        # redirecting stderr will still have colorized ascii
        raise ArgumentTypeError(f"{sx} contains a lowercase string")
    return sx


def get_parser() -> ArgumentParser:
    p = ArgumentParser(description="Testing", color=True)
    f = p.add_argument
    f('-m', '--max-records', type=int, help="Max number of records", required=True)
    f('-n', '--name', type=validate_lower, help="User Name", required=True)
    f('-t', '--trigger-error', action='store_true', help="Trigger error", required=False)
    return p

def main(argv: list[str]) -> int:
    p = get_parser()
    pa = p.parse_args(argv)
    if pa.trigger_error:
        # will be colorized by traceback, must be configured by ENV vars to disable
        raise ValueError("Trigger an error")

    # this is also a common usage to raise an error
    #p.error("Error case")
    print(f"Completed running {pa}")
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

• gh-139946: distinguish stdout or stderr when colorizing output in argparse #140495
• GH-139946: Colorize error and warning messages in argparse #140695
• [3.14] GH-139946: Document argparse includes color codes when redirecting to stderr to file #142398

[picnixz]

cc @savannahostrowski I don't know if you want a separate issue for that or an umbrella issue with argparse and colors.

[savannahostrowski]

I'll take a look through this list and figure out what the best course of action is for each last this week. Thanks for filing this!

[savannahostrowski]

Thanks for filing this! Points 1-3 will be addressed by #140495!

Regarding your points about environment variables, honoring environment variables is standard practice and necessary - many users rely on things like NO_COLOR being respected globally. We do document this in https://docs.python.org/3/library/argparse.html#color, which links to how you can control color in your local environment. However, I agree that the order of precedence could be documented more clearly.

Regarding tracebacks, this is working as expected. ArgumentParser(color=False) should only control argparse's output. If a user wants to disable color in their tracebacks, I'd expect them to set the correct environment variables to do that.

Regarding point 6, good catch. The error/warning keywords should also be colorized for consistency.