Add concurrent.futures.InterpreterPoolExecutor

[ericsnowcurrently]

Feature or enhancement

Proposal:

While we wait on PEP 734, there's nothing blocking us from adding a new executor for multiple interpreters. Doing so would allow people to start trying them out. (I wish I had thought of this for 3.13!)

My planned design:

• mostly built on top of ThreadPoolExecutor
• callables (and args) passed as initializer or to submit() or map() will be pickled (same as ProcessPoolExecutor)
• initializer and submit() arg can be a script instead of a callable (a script for map() doesn't make sense since there are no args)
• __init__() will take a new "shared" arg that corresponds to what Interpreter.prepare_main() takes in PEP 734, to allow each worker interpreter to share some data

In the future we could support a more efficient "sharing" scheme than pickle. When (or if) PEP 734 is accepted, we can refactor the executor to use the interpreters module rather than using _interpreters directly.

Open questions:

• reconstitute and raise uncaught exceptions from running in the interpreter (like we do for ProcessPoolExecutor)?
• (optionally?) exec functions against main instead of the their original module, i.e. use the body of the function as a script?

CC @brianquinlan

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-124694: Add concurrent.futures.InterpreterPoolExecutor #124548
• gh-124694: In test_interpreter_pool, Restore the Asyncio Event Loop Policy During Cleanup #125708
• gh-124694: Document missing ctxkwargs argument in ThreadPoolExecutor #134321
• gh-124694: Add versionadded note to InterpreterPoolExecutor #140468
• [3.14] gh-124694: Add versionadded note to InterpreterPoolExecutor (GH-140468) #140605