Document that "multiprocessing.get_context()" is also setting the context globally

[Delgan]

Documentation

Hi.

It's not clear from the documentation that calling multiprocessing.get_context() (with method=None) has the side effect of also setting the default context globally, preventing future usage of multiprocessing.set_start_method().

import multiprocessing

default_context = multiprocessing.get_context()  # "Fork" on Linux.
multiprocessing.set_start_method("spawn")  # Error.

Which causes:

Traceback (most recent call last):
  File "/home/delgan/test.py", line 5, in <module>
    multiprocessing.set_start_method("spawn")  # Error raised.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/context.py", line 247, in set_start_method
    raise RuntimeError('context has already been set')
RuntimeError: context has already been set

This is a surprising behavior because calling multiprocessing.get_context("fork") then multiprocessing.set_start_method("spawn") works perfectly fine. I assumed calling get_context(method=None) would just create a context using the default method ("fork" on Linux, "spawn" otherwise) but there is visibly more to it than that.

---

I have a library allowing user to specify the multiprocessing.Context of their choice:

def function(context=None):
    if context is None:
        context = multiprocessing.get_context()
    pipe = context.Pipe()
    ...

This recipe looks correct, but it is not. It will cause a RuntimeError if the user or another third-party library tries to call multiprocesing.set_start_method() after using my function.

I'll probably need to refactor it to something like that:

def function(context=None):
    if context is None:
        start_method = multiprocessing.get_start_method(allow_none=True)
        if start_method is not None:
            context = multiprocessing.get_context(start_method)
        elif os.name == "posix":
            context = multiprocessing.get_context("fork")
        else:
            context = multiprocessing.get_context("spawn")
    pipe = context.Pipe()
    ...

I wonder if there shouldn't be a method to retrieve the current context without setting one if there is none?
I would like to be able to use a Context just like if I was using multiprocessing directly.

Linked PRs

• gh-109070: Document that multiprocessing.get_context() sets the global context #109488
• gh-109070: multiprocessing.get_context will not set the start method globally #135678
• gh-109070: Document that get_context in multiprocessing have side effect #136341
• [3.14] gh-109070: Document that get_context in multiprocessing have side effect (GH-136341) #136343
• [3.13] gh-109070: Document that get_context in multiprocessing have side effect (GH-136341) #136344

[rajveer43]

Would like to work on this

[gpshead]

Would like to work on this

Feel free to go ahead and propose documentation updates in a PR. Regardless of if we're able to change the side effect behavior in the future (the other issue), we should document the current actual behavior.

[rajveer43]

sure

[rajveer43]

Would like to work on this

Feel free to go ahead and propose documentation updates in a PR. Regardless of if we're able to change the side effect behavior in the future (the other issue), we should document the current actual behavior.

can you suugest a path where i need to make changes?

[aisk]

Maybe we can fix the side effect of this function, instead of document it.

[gpshead]

I think we still need to just document it rather than try to change it. We depend on this otherwise unseemly default setting behavior. #135678 (comment)