Fork server doesn't flush stdout/stderr after preloading modules, potentially leaving buffered data to be inherited by child processes

[duaneg]

Bug report

Bug description:

Files

a/__init__.py

import os
import time
print(f"init {os.getpid()} at {time.clock_gettime_ns(time.CLOCK_MONOTONIC)}")

repro.py

import multiprocessing
import os
import time

if __name__ == '__main__':
    print(f"run main {os.getpid()}")
    multiprocessing.set_forkserver_preload(['a'])
    for _ in range(2):
        p = multiprocessing.Process()
        p.start()
        p.join()
else:
    print(f"re-import main {os.getpid()} at {time.clock_gettime_ns(time.CLOCK_MONOTONIC)}")

Reproduction

1. Create a new module a containing the __init__.py file above
2. Run the repro.py script above, ensuring the module created is on PYTHONPATH

Result

> ./python repro.py
run main 1056488
init 1056490 at 151009034069836
re-import main 1056491 at 151009045273212
re-import main 1056492 at 151009051787587
> ./python repro.py | tee /dev/null
run main 1056607
init 1056610 at 151113770440639
re-import main 1056611 at 151113781130002
init 1056610 at 151113770440639
re-import main 1056612 at 151113787814593
init 1056610 at 151113770440639

Expected

The output to be the same when stdout is redirected as when it is not.

Analysis

This is due to fork server preloading a module that writes to stdout, but not flushing it. When a child process is spawned it inherits the buffered data and spuriously outputs it when it flushes its stdout. Note that #126631 prevents __main__ from being preloaded, so at present this will not be triggered by printing from __main__.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-135335: flush stdout/stderr in forkserver after preloading modules #135338
• [3.14] gh-135335: flush stdout/stderr in forkserver after preloading modules (GH-135338) #135670
• [3.13] gh-135335: flush stdout/stderr in forkserver after preloading modules (GH-135338) #135671
• gh-135335: Simplify preload regression test using __main__ #138686
• [3.14] gh-135335: Simplify preload regression test using __main__ (GH-138686) #141886
• [3.13] gh-135335: Simplify preload regression test using __main__ (GH-138686) #141887

[efimov-mikhail]

Confirmed, this bug can be reproduced.