gh-122712: Test `CALL_ALLOC_AND_ENTER_INIT` handles reassignment of `code` #122713

mpage · 2024-08-06T00:09:31Z

Add a test to verify CALL_ALLOC_AND_ENTER_INIT handles the case where the __init__ function's code object is reassigned.

Issue: Specializing interpreter may crash if the code object of a class's __init__ method is reassigned #122712

mpage · 2024-08-06T01:47:20Z

Failing hypothesis tests are a known issue: #122686

mpage · 2024-08-06T02:20:54Z

JIT test failures look like unrelated issues with qemu:

ERROR:../../plugins/core.c:221:qemu_plugin_vcpu_init_hook: assertion failed: (success)
Bail out! ERROR:../../plugins/core.c:221:qemu_plugin_vcpu_init_hook: assertion failed: (success)
aarch64-binfmt-P: ./include/qemu/rcu.h:101: rcu_read_unlock: Assertion `p_rcu_reader->depth != 0' failed.

This reverts commit 2d52d28.

This reverts commit ef0420e.

This reverts commit fdd055e.

This reverts commit 545a120.

mpage · 2024-08-07T23:43:54Z

@brandtbucher @markshannon - I've updated this to perform the necessary checks inline in CALL_ALLOC_AND_ENTER_INIT rather than use function versions. Would you please have a look?

markshannon

This looks like the correct fix. Thanks.

I'd like the DEOPT test streamlined a bit, but it looks good otherwise.

Python/bytecodes.c

bedevere-app · 2024-08-15T10:30:07Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

brandtbucher

Agreed with @markshannon's suggestion. Otherwise, looks good!

This is the preferred style

mpage · 2024-08-20T18:58:00Z

Looks like this was fixed on main by gh-123140. The underlying issue is still present in 3.13, however. @markshannon how would you like to proceed with fixing on 3.13?

bedevere-app · 2024-08-20T22:09:22Z

GH-123184 is a backport of this pull request to the 3.13 branch.

mpage · 2024-08-20T22:14:18Z

Closing this since gh-123140 fixes it on main.

…ND_ENTER_INIT (GH-122713) (GH-123184)

markshannon · 2024-08-21T12:18:48Z

Sorry. I meant to merge this first, so we got the test before changing CALL_ALLOC_AND_ENTER_INIT. It appears I forgot.

Do you want to re-open this PR with just the test, or make a new one?

bedevere-bot · 2024-08-21T12:43:20Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.13 has failed when building commit 50a595b.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1386/builds/496) and take a look at the build logs.
Check if the failure is related to this commit (50a595b) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1386/builds/496

Failed tests:

test_import

Failed subtests:

test_basic_multiple_interpreters_deleted_no_reset - test.test_import.SinglephaseInitTests.test_basic_multiple_interpreters_deleted_no_reset

Summary of the results of the build (if available):

==

Click to see traceback logs

TracebackTests.test_exec_failure_nested) ... ok


TracebackTests.test_nonexistent_module_nested) ... ok


TracebackTests.test_syntax_error) ... ok


TracebackTests.test_nonexistent_module) ... ok


TracebackTests.test_unencodable_filename) ... skipped 'need TESTFN_UNENCODABLE'


TracebackTests.test_broken_parent_from) ... ok


TracebackTests.test_broken_from) ... ok


TracebackTests.test_exec_failure) ... ok


TracebackTests.test_broken_parent) ... ok


TracebackTests.test_import_bug) ... ok


TracebackTests.test_broken_submodule) ... ok


Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 136, in wrapper
    func(self)
    ~~~~^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 3031, in test_basic_multiple_interpreters_deleted_no_reset
    self.check_copied(loaded_interp1, base)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 2694, in check_copied
    self.assertNotEqual(snap.id, base.snapshot.id)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4607884272 == 4607884272

ericsnowcurrently · 2024-08-21T15:33:58Z

======================================================================
FAIL: test_basic_multiple_interpreters_deleted_no_reset (test.test_import.SinglephaseInitTests.test_basic_multiple_interpreters_deleted_no_reset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 136, in wrapper
    func(self)
    ~~~~^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 3031, in test_basic_multiple_interpreters_deleted_no_reset
    self.check_copied(loaded_interp1, base)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/163B47BC-A1C1-4382-9772-4D79687530BB/data/Containers/Bundle/Application/FDBABAF6-E02D-45C5-A42C-5FCCA078C9B2/iOSTestbed.app/python/lib/python3.13/test/test_import/__init__.py", line 2694, in check_copied
    self.assertNotEqual(snap.id, base.snapshot.id)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4607884272 == 4607884272

----------------------------------------------------------------------

ericsnowcurrently · 2024-08-21T15:34:51Z

Also, the failure is relative to the 3.13 backport: gh-123184.

mpage · 2024-08-21T17:30:57Z

That failure seems like it should be unrelated to this change. I also haven't been able to reproduce it locally.

mpage · 2024-08-21T17:42:08Z

@markshannon - No worries. Reopened this with just the test.

markshannon · 2024-08-22T07:38:13Z

Thanks

mpage added 5 commits August 5, 2024 11:01

Add repro

40c094e

Add fix

2d52d28

Regenerate code

ef0420e

Fix test_dis

fdd055e

Fix test_monitoring

545a120

mpage added the skip news label Aug 6, 2024

bedevere-app bot mentioned this pull request Aug 6, 2024

Specializing interpreter may crash if the code object of a class's __init__ method is reassigned #122712

Closed

mpage marked this pull request as ready for review August 6, 2024 02:22

mpage requested review from ericsnowcurrently and markshannon as code owners August 6, 2024 02:22

bedevere-app bot added the awaiting review label Aug 6, 2024

mpage requested review from brandtbucher and colesbury August 7, 2024 04:46

mpage added 7 commits August 7, 2024 13:15

Revert "Add fix"

1742966

This reverts commit 2d52d28.

Revert "Regenerate code"

b450d26

This reverts commit ef0420e.

Revert "Fix test_dis"

cf9112e

This reverts commit fdd055e.

Revert "Fix test_monitoring"

1eef216

This reverts commit 545a120.

Check properties inline rather than using function versions

b4d2e75

Regenerate files

f11d984

Merge branch 'main' into pythongh-122712-fix-call-alloc

f483286

markshannon requested changes Aug 15, 2024

View reviewed changes

Python/bytecodes.c Outdated Show resolved Hide resolved

bedevere-app bot added awaiting changes and removed awaiting review labels Aug 15, 2024

brandtbucher approved these changes Aug 15, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting changes labels Aug 15, 2024

mpage added 4 commits August 19, 2024 13:16

Merge branch 'main' into pythongh-122712-fix-call-alloc

796ce3b

Revert refactoring

2d2fe41

Inline checks for simple functions into the bytecode body

8b7000e

This is the preferred style

Merge branch 'main' into pythongh-122712-fix-call-alloc

169e465

mpage closed this Aug 20, 2024

mpage mentioned this pull request Aug 20, 2024

[3.13] gh-122712: Guard against __code__ reassignment in CALL_ALLOC_AND_ENTER_INIT (GH-122713) #123184

Merged

markshannon pushed a commit that referenced this pull request Aug 21, 2024

[3.13] gh-122712: Guard against __code__ reassignment in CALL_ALLOC_A…

50a595b

…ND_ENTER_INIT (GH-122713) (GH-123184)

Merge branch 'main' into pythongh-122712-fix-call-alloc

1d01624

mpage reopened this Aug 21, 2024

mpage changed the title ~~gh-122712: Guard against __code__ reassignment in CALL_ALLOC_AND_ENTER_INIT~~ gh-122712: Test CALL_ALLOC_AND_ENTER_INIT handles reassignment of __code__ Aug 21, 2024

Update comment

39f8ba2

markshannon merged commit 79ddf75 into python:main Aug 22, 2024
32 checks passed

bedevere-app bot removed the awaiting merge label Aug 22, 2024

Uh oh!

gh-122712: Test CALL_ALLOC_AND_ENTER_INIT handles reassignment of __code__ #122713

gh-122712: Test CALL_ALLOC_AND_ENTER_INIT handles reassignment of __code__ #122713

Uh oh!

Conversation

mpage commented Aug 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mpage commented Aug 6, 2024

Uh oh!

mpage commented Aug 6, 2024

Uh oh!

mpage commented Aug 7, 2024

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-app bot commented Aug 15, 2024

Uh oh!

brandtbucher left a comment

Choose a reason for hiding this comment

Uh oh!

mpage commented Aug 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented Aug 20, 2024

Uh oh!

mpage commented Aug 20, 2024

Uh oh!

markshannon commented Aug 21, 2024

Uh oh!

bedevere-bot commented Aug 21, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

ericsnowcurrently commented Aug 21, 2024

Uh oh!

ericsnowcurrently commented Aug 21, 2024

Uh oh!

mpage commented Aug 21, 2024

Uh oh!

mpage commented Aug 21, 2024

Uh oh!

Uh oh!

markshannon commented Aug 22, 2024

Uh oh!

Uh oh!

gh-122712: Test `CALL_ALLOC_AND_ENTER_INIT` handles reassignment of `code` #122713

gh-122712: Test `CALL_ALLOC_AND_ENTER_INIT` handles reassignment of `code` #122713

mpage commented Aug 6, 2024 •

edited

Loading

mpage commented Aug 20, 2024 •

edited

Loading