Skip to content

gh-151763: Fix OOM cleanup of initial thread state#152165

Open
zainnadeem786 wants to merge 1 commit into
python:mainfrom
zainnadeem786:fix/oom-0020-threadstate-cleanup
Open

gh-151763: Fix OOM cleanup of initial thread state#152165
zainnadeem786 wants to merge 1 commit into
python:mainfrom
zainnadeem786:fix/oom-0020-threadstate-cleanup

Conversation

@zainnadeem786

Copy link
Copy Markdown
Contributor

Summary

This PR addresses OOM-0020 from gh-151763.

It fixes a free-threaded crash during interpreter creation when memory allocation fails after the preallocated initial thread state has been selected but before init_threadstate() initializes its interpreter back-pointer.

Issue

In the free-threaded build, _interpreters.create(reqrefs=True) can fail during early thread-state allocation setup.

The affected path is:

_interpreters.create(reqrefs=True)
-> _interpreters_create_impl()
-> _PyXI_NewInterpreter()
-> Py_NewInterpreterFromConfig()
-> new_interpreter()
-> _PyThreadState_New()
-> new_threadstate()
-> alloc_threadstate()
-> _Py_qsbr_reserve() / _Py_ReserveTLBCIndex()
-> free_threadstate()

For the initial thread state, alloc_threadstate() obtains memory embedded in PyInterpreterState rather than heap-allocated memory.

However, if a later free-threaded allocation fails before init_threadstate() runs, cleanup reaches free_threadstate() while tstate->base.interp is still unset.

As a result, free_threadstate() fails to recognize the preallocated initial thread state and may incorrectly call PyMem_RawFree() on embedded memory.

This can result in:

  • _PyMem_DebugRawFree: bad ID in debug builds
  • access violation in release builds

Fix

Initialize the interpreter back-pointer earlier in alloc_threadstate():

tstate->base.interp = interp;

This ensures that cleanup paths can correctly identify the preallocated initial thread state even if a later allocation fails before init_threadstate() is reached.

The successful path remains unchanged because init_threadstate() already sets the same value later.

Validation

Validated locally on Windows.

Builds completed successfully:

PCbuild\build.bat -p x64 -c Debug --disable-gil
PCbuild\build.bat -p x64 -c Release --disable-gil
PCbuild\build.bat -p x64 -c Debug

Before the fix, the OOM-0020 reproducer crashed in the free-threaded build with:

_PyMem_DebugRawFree: bad ID

After the fix, the same allocation-failure path reaches MemoryError instead of crashing.

Focused tests passed:

PCbuild/amd64t/python_d.exe -m test test__interpreters
PCbuild/amd64/python_d.exe -m test test__interpreters

git diff --check also passes.

Regression Test

No regression test is included.

The reproducer depends on _testcapi.set_nomemory() in a free-threaded build, and the exact allocation index is build-sensitive. This follows the recent maintainer direction for these OOM-path fixes, where a focused code fix is preferable to fragile allocation-index regression tests.

Compatibility

This is a localized initialization-order fix for an OOM cleanup path.

Normal successful interpreter creation keeps the same final state, because init_threadstate() already assigns the same interpreter pointer later.

Addresses OOM-0020 from gh-151763.

Comment thread Python/pystate.c
}
reset_threadstate(tstate);
}
tstate->base.interp = interp;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should happen in new_threadstate, not in the allocator.

@bedevere-app

bedevere-app Bot commented Jun 25, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants