Improvements to test.support.interpreters.Interpreter #132775

ericsnowcurrently · 2025-04-21T16:08:49Z

Interpreter.call() should be able to support arbitrary callables, full args, and return values.

Linked PRs

The text was updated successfully, but these errors were encountered:

…h-132974) This change consists of adding tests and moving code around, with some renaming thrown in.

…eUTF8() (gh-132979) They are derived from the existing `PyModule_GetFilenameObject(). They are used by a later change related to pickle and handling __main__.

We replace it with _Py_GetMainModule(), and add _Py_CheckMainModule(), but both in the internal-only C-API. We also add _PyImport_GetModulesRef(), which is the equivalent of _PyImport_GetModules(), but which increfs before the lock is released. This is used by a later change related to pickle and handling __main__.

This is the base for several other XIData wrappers, like pickle and marshal. It is essentially a refactor of the existing bytes XIData code.

Note that the bulk of this change is tests.

The function indicates whether or not the function has a return statement. This is used by a later change related treating some functions like scripts.

markshannon · 2025-04-29T14:31:20Z

Could you elaborate what the "minor issues" are?
Either as a task list or sub issues?

It isn't clear to me why, for example, #132981 is needed, or if it is the right solution to whatever the problem is.

This helper is useful in a variety of ways, including in demonstrating how the different counts relate to one another. It will be used in a later change to help identify if a function is "stateless", meaning it doesn't have any free vars or globals. Note that a majority of this change is tests.

There's some extra complexity due to making sure we we get things right when handling functions and classes defined in the __main__ module. This is also reflected in the tests, including the addition of extra functions in test.support.import_helper.

This reverts commit 94b4fcd.

The change broke the s390 builds, so I'm reverting it while I investigate. This reverts commit 94b4fcd.

This reverts commit 811edcf (pythongh-133128).

…gh-134515)" This reverts commit bbf8048.

This reverts commit 09e72cf, AKA gh-134511. We are reverting due to refleaks on free-threaded builds.

…ta()" (gh-134600) This reverts commit bbf8048, AKA gh-134515. We are reverting due to refleaks on free-threaded builds.

…-133482) It now supports a "full" fallback to _PyFunction_GetXIData() and then `_PyPickle_GetXIData()`. There's also room for other fallback modes if that later makes sense.

…Stateless() (pythongh-134439)

…4440) This change includes some semi-related refactoring of queues and channels.

This is mostly a refactor to clean things up a bit, most notably the "XI namespace" code. Making the session opaque requires adding the following internal-only functions: * _PyXI_NewSession() * _PyXI_FreeSession() * _PyXI_GetMainNamespace()

…PyCode GetScriptXIData()" (pythongh-134600)" This reverts commit 7476f90.

This reverts commit 8a793c4, AKA gh-134599. This effectively re-applies commit 09e72cf (gh-134511)

This reverts commit 7476f90, AKA gh-134600. This effectively unreverts commit bbf8048 (gh-134515).

…4758) This is a small follow-up to gh-133481. There's a corner case in the behavior of PyImport_ImportModuleAttrString(), where it expects __builtins__ to be set if __globals__ is set.

…ythongh-134758) This is a small follow-up to pythongh-133481. There's a corner case in the behavior of PyImport_ImportModuleAttrString(), where it expects __builtins__ to be set if __globals__ is set. (cherry picked from commit 9b5e800) Co-authored-by: Eric Snow <[email protected]>

…gh-134794) This is a small follow-up to gh-133481. There's a corner case in the behavior of PyImport_ImportModuleAttrString(), where it expects __builtins__ to be set if __globals__ is set. (cherry picked from commit 9b5e800, AKA gh-134758) Co-authored-by: Eric Snow <[email protected]>

The problem we're fixing here is that we were using PyDict_Size() on "defaults", which it is actually a tuple. We're also adding some explicit type checks. This is a follow-up to gh-133221/gh-133528.

The problem we're fixing here is that we were using PyDict_Size() on "defaults", which it is actually a tuple. We're also adding some explicit type checks. This is a follow-up to pythongh-133221/pythongh-133528. (cherry picked from commit dafd141) Co-authored-by: Eric Snow <[email protected]>

gh-132775: Fix _PyFunctIon_VerifyStateless() () The problem we're fixing here is that we were using PyDict_Size() on "defaults", which it is actually a tuple. We're also adding some explicit type checks. This is a follow-up to gh-133221/gh-133528. (cherry picked from commit dafd141, AKA gh-134900) Co-authored-by: Eric Snow <[email protected]>

It now supports most callables, full args, and return values.

…h-133484) It now supports most callables, full args, and return values. (cherry picked from commit 52deabe) Co-authored-by: Eric Snow <[email protected]>

) It now supports most callables, full args, and return values. (cherry picked from commit 52deabe, AKA gh-133484) Co-authored-by: Eric Snow [email protected]

ericsnowcurrently added the type-bug An unexpected behavior, bug, or error label Apr 21, 2025

ericsnowcurrently self-assigned this Apr 21, 2025

ericsnowcurrently added this to Subinterpreters Apr 21, 2025

github-project-automation bot moved this to Todo in Subinterpreters Apr 21, 2025

ericsnowcurrently added stdlib Python modules in the Lib dir topic-subinterpreters 3.14 bugs and security fixes labels Apr 21, 2025

ericsnowcurrently added a commit that referenced this issue Apr 28, 2025

gh-132775: Cleanup Related to crossinterp.c Before Further Changes (g…

6f04325

…h-132974) This change consists of adding tests and moving code around, with some renaming thrown in.

bedevere-app bot mentioned this issue Apr 28, 2025

gh-132775: Add _PyBytes_GetXIData() #133101

Merged

ericsnowcurrently added a commit that referenced this issue Apr 28, 2025

gh-132775: Add _PyBytes_GetXIData() (gh-133101)

606003f

This is the base for several other XIData wrappers, like pickle and marshal. It is essentially a refactor of the existing bytes XIData code.

This was referenced Apr 28, 2025

gh-132775: Add _PyPickle_GetXIData() #133107

Merged

gh-132775: Add _PyMarshal_GetXIData() #133108

Merged

ericsnowcurrently added a commit that referenced this issue Apr 28, 2025

gh-132775: Add _PyMarshal_GetXIData() (gh-133108)

bdd23c0

Note that the bulk of this change is tests.

ericsnowcurrently added a commit that referenced this issue Apr 29, 2025

gh-132775: Add _PyCode_ReturnsOnlyNone() (gh-132981)

96a7fb9

The function indicates whether or not the function has a return statement. This is used by a later change related treating some functions like scripts.

bedevere-app bot mentioned this issue Apr 29, 2025

gh-132775: Add _PyCode_GetVarCounts() #133128

Merged

bedevere-app bot mentioned this issue Apr 30, 2025

gh-132775: Add _PyCode_VerifyStateless() #133221

Merged

ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 1, 2025

Revert "pythongh-132775: Add _PyCode_GetVarCounts() (pythongh-133128)"

1740ecb

This reverts commit 94b4fcd.

ericsnowcurrently added a commit that referenced this issue May 1, 2025

Revert "gh-132775: Add _PyCode_GetVarCounts() (gh-133128)" (gh-133232)

811edcf

The change broke the s390 builds, so I'm reverting it while I investigate. This reverts commit 94b4fcd.

ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 1, 2025

pythongh-132775: Unrevert "Add _PyCode_GetVarCounts()"

d9c3672

This reverts commit 811edcf (pythongh-133128).

bedevere-app bot mentioned this issue May 1, 2025

gh-132775: Unrevert "Add _PyCode_GetVarCounts()" #133265

Merged

ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 23, 2025

Revert "[3.14] pythongh-132775: Use _PyCode GetScriptXIData() (python…

250b775

…gh-134515)" This reverts commit bbf8048.

ericsnowcurrently added a commit that referenced this issue May 23, 2025

gh-134557: Revert "gh-132775: Use _PyCode GetScriptXIData()" (gh-134599)

8a793c4

This reverts commit 09e72cf, AKA gh-134511. We are reverting due to refleaks on free-threaded builds.

ericsnowcurrently added a commit that referenced this issue May 23, 2025

[3.14] gh-134557: Revert "[3.14] gh-132775: Use _PyCode GetScriptXIDa…

7476f90

…ta()" (gh-134600) This reverts commit bbf8048, AKA gh-134515. We are reverting due to refleaks on free-threaded builds.

lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025

pythongh-132775: Use _PyFunction_VerifyStateless() and _PyCode_Verify…

3a65bbd

…Stateless() (pythongh-134439)

lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025

pythongh-132775: Use _PyObject_GetXIData (With Fallback) (pythongh-13…

a106405

…4440) This change includes some semi-related refactoring of queues and channels.

lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025

pythongh-132775: Use _PyCode GetScriptXIData() (pythongh-134511)

6d84fb3

lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025

pythongh-132775: Fix Recently Introduced Warnings (pythongh-134530)

790b299

ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this issue May 26, 2025

Revert "[3.14] pythongh-134557: Revert "[3.14] pythongh-132775: Use _…

1a5a117

…PyCode GetScriptXIData()" (pythongh-134600)" This reverts commit 7476f90.

This was referenced May 26, 2025

gh-132775: Unrevert "Use _PyCode GetScriptXIData()" #134735

Merged

[3.14] gh-132775: Unrevert "Use _PyCode GetScriptXIData()" #134736

Merged

ericsnowcurrently added a commit that referenced this issue May 26, 2025

gh-132775: Unrevert "Use _PyCode GetScriptXIData()" (gh-134735)

56743af

This reverts commit 8a793c4, AKA gh-134599. This effectively re-applies commit 09e72cf (gh-134511)

ericsnowcurrently added a commit that referenced this issue May 26, 2025

[3.14] gh-132775: Unrevert "Use _PyCode GetScriptXIData()" (gh-134736)

e86efab

This reverts commit 7476f90, AKA gh-134600. This effectively unreverts commit bbf8048 (gh-134515).

bedevere-app bot mentioned this issue May 26, 2025

gh-132775: Always Set __builtins__ In _PyFunction_FromXIData() #134758

Merged

bedevere-app bot mentioned this issue May 27, 2025

[3.14] gh-132775: Always Set __builtins__ In _PyFunction_FromXIData() (gh-134758) #134794

Merged

bedevere-app bot mentioned this issue May 29, 2025

gh-132775: Fix _PyFunctIon_VerifyStateless() #134900

Merged

bedevere-app bot mentioned this issue May 29, 2025

[3.14] gh-132775: Fix _PyFunctIon_VerifyStateless() (GH-134900) #134901

Merged

ericsnowcurrently added a commit that referenced this issue May 30, 2025

gh-132775: Expand the Capability of Interpreter.call() (gh-133484)

52deabe

It now supports most callables, full args, and return values.

bedevere-app bot mentioned this issue May 30, 2025

[3.14] gh-132775: Expand the Capability of Interpreter.call() (gh-133484) #134933

Merged

ericsnowcurrently closed this as completed May 30, 2025

github-project-automation bot moved this from Todo to Done in Subinterpreters May 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Improvements to test.support.interpreters.Interpreter #132775

Improvements to test.support.interpreters.Interpreter #132775

ericsnowcurrently commented Apr 21, 2025 •

edited by bedevere-app bot

Loading

markshannon commented Apr 29, 2025

Uh oh!

Uh oh!

Improvements to test.support.interpreters.Interpreter #132775

Improvements to test.support.interpreters.Interpreter #132775

Comments

ericsnowcurrently commented Apr 21, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked PRs

markshannon commented Apr 29, 2025

Uh oh!

ericsnowcurrently commented Apr 21, 2025 •

edited by bedevere-app bot

Loading