gh-144503: Pass sys.argv to forkserver as real argv elements by gpshead · Pull Request #148194 · python/cpython

gpshead · 2026-04-07T03:51:35Z

Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError.

Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. (well, almost, we consume two more arguments for the -c blah so the forkserver only works if there are slightly less than the maximum consumption of command line arguments the platform allows, 2MiB total a.k.a. 1/4 stack size for argv+envp space on Linux in most default configs... a far less common scenario than sys.argv repr'ing to >128KiB)

Regression introduced by gh-143706 / 298d544 which landed as a bugfix in 3.14.3 and 3.13.12.

This PR is an alternative to #144508, perhaps less risky to backport as a bugfix.

Issue: multiprocessing.Pool fails if command has a long list of arguments (regression in 3.14.3, 3.13.12) #144503

Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError. Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. Regression introduced by pythongh-143706 / 298d544.

bedevere-bot · 2026-04-07T04:17:44Z

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 77028cd 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F148194%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

gpshead · 2026-04-07T04:34:35Z

buildbot result notes (editing as i see things):

buildbot/x86 Debian Installed with X PR - unrelated - timeout in a multiprocessing fork interrupt test. not forkserver related.

JelleZijlstra

Yes, this makes sense and feels safer than the alternative.

miss-islington-app · 2026-04-07T05:41:06Z

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

miss-islington-app · 2026-04-07T05:41:09Z

Sorry, @gpshead, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 5e9d90b615b94469081b39a7b0808fea86c417be 3.14

miss-islington-app · 2026-04-07T05:41:14Z

Sorry, @gpshead, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 5e9d90b615b94469081b39a7b0808fea86c417be 3.13

…ents (pythonGH-148194) Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError. Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. Regression introduced by pythongh-143706 / 298d544. (cherry picked from commit 5e9d90b) Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>

bedevere-app · 2026-04-07T05:51:44Z

GH-148195 is a backport of this pull request to the 3.14 branch.

…ents (pythonGH-148194) Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError. Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. Regression introduced by pythongh-143706 / 298d544. (cherry picked from commit 5e9d90b) Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>

bedevere-app · 2026-04-07T05:53:33Z

GH-148196 is a backport of this pull request to the 3.13 branch.

bedevere-bot · 2026-04-07T05:54:44Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Debian Non-Debug with X 3.x (no tier) has failed when building commit 5e9d90b.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1245/builds/7959) and take a look at the build logs.
Check if the failure is related to this commit (5e9d90b) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1245/builds/7959

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/contextlib.py"�[0m, line �[35m85�[0m, in �[35minner�[0m
    return func(*args, **kwds)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m649�[0m, in �[35mtest_interrupt�[0m
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/contextlib.py"�[0m, line �[35m85�[0m, in �[35minner�[0m
    return func(*args, **kwds)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m630�[0m, in �[35m_kill_process�[0m
    self.assertEqual(�[31mjoin�[0m�[1;31m()�[0m, None)
                     �[31m~~~~�[0m�[1;31m^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m303�[0m, in �[35m__call__�[0m
    return �[31mself.func�[0m�[1;31m(*args, **kwds)�[0m
           �[31m~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/process.py"�[0m, line �[35m156�[0m, in �[35mjoin�[0m
    res = self._popen.wait(timeout)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/popen_fork.py"�[0m, line �[35m44�[0m, in �[35mwait�[0m
    return �[31mself.poll�[0m�[1;31m(os.WNOHANG if timeout == 0.0 else 0)�[0m
           �[31m~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/popen_fork.py"�[0m, line �[35m28�[0m, in �[35mpoll�[0m
    pid, sts = �[31mos.waitpid�[0m�[1;31m(self.pid, flag)�[0m
               �[31m~~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m626�[0m, in �[35mhandler�[0m
    raise RuntimeError('join took too long: %s' % p)
�[1;35mRuntimeError�[0m: �[35mjoin took too long: <Process name='Process-158' pid=10431 parent=6517 started daemon>�[0m


Traceback (most recent call last):
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/contextlib.py"�[0m, line �[35m85�[0m, in �[35minner�[0m
    return func(*args, **kwds)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m649�[0m, in �[35mtest_interrupt�[0m
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/contextlib.py"�[0m, line �[35m85�[0m, in �[35minner�[0m
    return func(*args, **kwds)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m630�[0m, in �[35m_kill_process�[0m
    self.assertEqual(�[31mjoin�[0m�[1;31m()�[0m, None)
                     �[31m~~~~�[0m�[1;31m^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m303�[0m, in �[35m__call__�[0m
    return �[31mself.func�[0m�[1;31m(*args, **kwds)�[0m
           �[31m~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/process.py"�[0m, line �[35m156�[0m, in �[35mjoin�[0m
    res = self._popen.wait(timeout)
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/popen_fork.py"�[0m, line �[35m44�[0m, in �[35mwait�[0m
    return �[31mself.poll�[0m�[1;31m(os.WNOHANG if timeout == 0.0 else 0)�[0m
           �[31m~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/multiprocessing/popen_fork.py"�[0m, line �[35m28�[0m, in �[35mpoll�[0m
    pid, sts = �[31mos.waitpid�[0m�[1;31m(self.pid, flag)�[0m
               �[31m~~~~~~~~~~�[0m�[1;31m^^^^^^^^^^^^^^^^�[0m
  File �[35m"/buildbot/buildarea/3.x.ware-debian-x86.nondebug/build/Lib/test/_test_multiprocessing.py"�[0m, line �[35m626�[0m, in �[35mhandler�[0m
    raise RuntimeError('join took too long: %s' % p)
�[1;35mRuntimeError�[0m: �[35mjoin took too long: <Process name='Process-1' pid=29842 parent=29840 started daemon>�[0m

…H-148194) (#148195) Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError. Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. Regression introduced by gh-143706 / 298d544. (cherry picked from commit 5e9d90b)

…H-148194) (#148196) Avoid embedding the parent's sys.argv into the forkserver -c command string via repr(). When sys.argv is large (e.g. thousands of file paths from a pre-commit hook), the resulting single argument could exceed the OS per-argument length limit (MAX_ARG_STRLEN on Linux, typically 128 KiB), causing posix_spawn to fail and the parent to observe a BrokenPipeError. Instead, append the argv entries as separate command-line arguments after -c; the forkserver child reads them back as sys.argv[1:]. This cannot exceed any limit the parent itself did not already satisfy. Regression introduced by gh-143706 / 298d544. (cherry picked from commit 5e9d90b)

Python 3.15.0a8 introduced a change to `forkserver` spawn mode of `multiprocessing`, and that change was also back-ported to python 3.13.13 and 3.14.4. See: python/cpython#148194 In the modified version, the `sys.argv` elements passed to the `forkserver` process have been moved from the command string (that follows `-c` switch) itself into additional command-line arguments. This breaks our current argument-matching implementation in `multiprocessing` run-time hook, which assumes that the command string is the last passed argument. In addition, the command string itself changed a bit (there is an added `import sys; ` at the beginning). Therefore, revise the argument matching to find the index of `-c` argument, and use the following argument (if available) as the command string to match. Add the pattern for the new forkserver command to ensure match under these new python versions.

gpshead self-assigned this Apr 7, 2026

bedevere-app bot added the awaiting core review label Apr 7, 2026

bedevere-app bot mentioned this pull request Apr 7, 2026

multiprocessing.Pool fails if command has a long list of arguments (regression in 3.14.3, 3.13.12) #144503

Open

gpshead added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 7, 2026

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 7, 2026

gpshead added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Apr 7, 2026

gpshead added this to Multiprocessing issues Apr 7, 2026

gpshead moved this to In Progress in Multiprocessing issues Apr 7, 2026

gpshead added the release-blocker label Apr 7, 2026

github-project-automation bot added this to Release and Deferred blockers 🚫 Apr 7, 2026

github-project-automation bot moved this to Todo in Release and Deferred blockers 🚫 Apr 7, 2026

gpshead removed this from Multiprocessing issues Apr 7, 2026

JelleZijlstra approved these changes Apr 7, 2026

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Apr 7, 2026

gpshead merged commit 5e9d90b into python:main Apr 7, 2026
94 of 103 checks passed

github-project-automation bot moved this from Todo to Done in Release and Deferred blockers 🚫 Apr 7, 2026

bedevere-app bot removed the awaiting merge label Apr 7, 2026

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Apr 7, 2026

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Apr 7, 2026

rokm mentioned this pull request Apr 10, 2026

Fix multiprocessing with forkserver in python 3.13.13 and 3.14.4 pyinstaller/pyinstaller#9423

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

gh-144503: Pass sys.argv to forkserver as real argv elements#148194

gh-144503: Pass sys.argv to forkserver as real argv elements#148194
gpshead merged 1 commit intopython:mainfrom
gpshead:gh-144503-argv-passthrough

gpshead commented Apr 7, 2026 •

edited

Loading

Uh oh!

bedevere-bot commented Apr 7, 2026

Uh oh!

gpshead commented Apr 7, 2026 •

edited

Loading

Uh oh!

JelleZijlstra left a comment

Uh oh!

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

bedevere-app bot commented Apr 7, 2026

Uh oh!

bedevere-app bot commented Apr 7, 2026

Uh oh!

bedevere-bot commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

gpshead commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Apr 7, 2026

Uh oh!

gpshead commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

miss-islington-app bot commented Apr 7, 2026

Uh oh!

bedevere-app bot commented Apr 7, 2026

Uh oh!

bedevere-app bot commented Apr 7, 2026

Uh oh!

bedevere-bot commented Apr 7, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gpshead commented Apr 7, 2026 •

edited

Loading

gpshead commented Apr 7, 2026 •

edited

Loading