gh-149335: Avoid JIT trace buffer asserts with overhead above FITNESS_INITIAL

[cocolato]

The trace buffer now reserves a small fixed overhead above FITNESS_INITIAL for entry and loop-closing uops.

• Issue: JIT can trigger assert(uop_buffer_remaining_space(trace) > space_needed); for widened uops #149335

[markshannon]

I think this is the wrong approach.
We want to assert that the buffer hasn't overflowed, not handle the overflow.

We need to either reduce the fitness or increase the buffer size.
If we aren't correctly tracking the amount of buffer used, we should also fix that.

Since the fitness numbers should be tailored for giving us good traces, I'd increase the buffer size a bit.

Maybe define the fitness and then set the buffer size to that plus a small bit to account for loop closure/entry overheads.
So instead of

#define FITNESS_INITIAL      UOP_MAX_TRACE_LENGTH

use

#define UOP_MAX_TRACE_LENGTH (FITNESS_INITIAL + OVERHEAD)

The OVERHEAD only needs to a small number, maybe 8 or 10.

[cocolato]

thanks for review! updated.

[markshannon]

Looks good, thanks

[vstinner]

This change should be backported to Python 3.15 which is also affected by the issue.

[miss-islington-app]

Thanks @cocolato for the PR, and @markshannon for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-150245 is a backport of this pull request to the 3.15 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.15 (tier-1) has failed when building commit 22c994c.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/2019/builds/85) and take a look at the build logs.
4. Check if the failure is related to this commit (22c994c) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/2019/builds/85

Failed tests:

• test_embed

Failed subtests:

• test_thread_state_ensure - test.test_embed.AuditingTests.test_thread_state_ensure

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 948, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.FileIO name=11 mode='wb' closefd=True>


Traceback (most recent call last):
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/test_embed.py", line 2000, in test_thread_state_ensure
    self.run_embedded_interpreter("test_thread_state_ensure")
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/test_embed.py", line 138, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "bad returncode %d, stderr is %r" %
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     (p.returncode, err))
                     ^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0 : bad returncode -6, stderr is "_testembed: ./Programs/_testembed.c:2736: test_thread_state_ensure: Assertion `_Py_atomic_load_int(&data.done) == 1' failed.\n"