Allow top-level statements in a global pre-setup phase for `timeit`

[inventshah]

Bug report

Bug description:

timeit crashes when using wildcard imports in either the "setup" or "statement" fields.

$ python -m timeit -s "from collections import Counter" -- "Counter([1, 2])" # allowed
1000000 loops, best of 5: 277 nsec per loop
$ python -m timeit -s "from collections import *" -- "Counter([1, 2])" # cashes
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "Lib/timeit.py", line 378, in <module>
    sys.exit(main())
             ~~~~^^
  File "Lib/timeit.py", line 316, in main
    t = Timer(stmt, setup, timer)
  File "Lib/timeit.py", line 132, in __init__
    code = compile(src, dummy_src_name, "exec")
  File "<timeit-src>", line 3
SyntaxError: import * only allowed at module level

This is surprising as this syntax is allowed in other contexts where code strings are executed (namely exec and python -c). The cause is code strings are placed inside a wrapper function that performs the timing here.

The git history of the timeit tests show 2bef585 explicitly added a test for this behavior, but based on the commit message / corresponding issue (GH #62718) I'm not sure if this is indeed the intended behavior. The comments seem to focus on statements that would effect a timing result (e.g., break ending a loop early or return changing the timed value), which I don't think applies to wild card imports.

Extra history: #46779 discussed this briefly in the context of from __main__ import * and the performance implications of wildcard imports when namespace arguments were first introduced.

Allowing wildcard imports in both stages would be great to bring unity with the other exec-likes and make CLI usage nicer, but the argument for statement is weaker than setup.

Potential options

1. move setup to outside inner and update globals and locals to be the same when exec is called on the code-object.
   • doesn't allow wildcards in statement
   • any variable created in setup requires a global namespace lookup which adds overhead
   • setup would be ran exactly once, so breaks methods that repeat with randomization
2. remove inner and instead call exec where inner was called, and either move the timing code bits to outside exec or extract the timed value from the exec namespace.
   • supports wildcard imports in both code strings
   • variable access seems to be through LOAD_NAME instead of LOAD_FAST which may have a overhead?
   • significantly more invasive to the existing code
3. do nothing and leave the mismatch. PEP8 / the docs discourage wildcard imports anyway.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-137578: support top-level setup statements in Timer objects #137587

[picnixz]

I thought I would treat it as a feature, but actually it's more than just from X import * that is not being supported. What's not supported neither is python -m timeit -s 'from __future__ import annotations' -- '...'. I'm surprised no one has ever complained about it. But I think it's important that futures are allowed during the setup phase.

[serhiy-storchaka]

As you can see, any proposed solution to this problem will add overhead in the benchmarked code, because it will no longer be able to use fast locals access.

Yet one option, which is backward compatible, -- add a new argument, pre-setup, which will be compiled and executed only once in the global scope.

[picnixz]

I think it's better to have a new option. Like -G for "global statements to be executed exactly once before anything else". I'll see what I can do.

[serhiy-storchaka]

That's what I said. I have no strict preference for the keyword argument name and the CLI option name.