Skip to content

Add notes on nogil & reinitialization to the Opt-Out section in Module Isolation HOWTO #134141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 22, 2025
Merged

Add notes on nogil & reinitialization to the Opt-Out section in Module Isolation HOWTO #134141

merged 2 commits into from
May 22, 2025

Conversation

encukou
Copy link
Member

@encukou encukou commented May 17, 2025 •
edited by github-actions bot
Loading

@bedevere-app bedevere-app bot added docs Documentation in the Doc dir skip news awaiting core review labels May 17, 2025
@github-project-automation github-project-automation bot moved this to Todo in Docs PRs May 17, 2025
ZeroIntensity
ZeroIntensity reviewed May 17, 2025
View reviewed changes
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't free-threading perform a stop-the-world when importing modules?

@encukou
Copy link
Member Author

encukou commented May 17, 2025

A better question is: do we guarantee that it will always stop the world?

colesbury
colesbury reviewed May 17, 2025
View reviewed changes
@colesbury
Copy link
Contributor

colesbury commented May 17, 2025 •
edited
Loading

We stop the world, temporarily enable the GIL, then resume (but with only one thread now holding the GIL), and the run the C extension initialization code.

I think you might still need the locks in this example because of isolated sub interpreters each with their own GIL.

We haven't explicitly guaranteed the stop the world behavior, but it'll probably be difficult to change in the future.

@ZeroIntensity
Copy link
Member

I think you might still need the locks in this example because of isolated sub interpreters each with their own GIL.

Subinterpreters switch to the main interpreter to import an extension, so the GIL synchronizes it.

encukou
encukou commented May 17, 2025
View reviewed changes
encukou
encukou commented May 17, 2025
View reviewed changes
@encukou
Copy link
Member Author

encukou commented May 17, 2025

Subinterpreters switch to the main interpreter to import an extension, so the GIL synchronizes it.

Well, that's something I'd like to change for 3.15 :)

Let's play it safe and keep our options open?
If you know the current implementation details, it's easy to leave this out.

@ZeroIntensity
Copy link
Member

I'll yield, but I'm worried that it would break too much code to be feasible (or worth doing). I think all third-party extensions currently assume no other threads, right? Maybe we could make this opt-in.

@encukou encukou merged commit 1f0a294 into python:main May 22, 2025
24 checks passed
@github-project-automation github-project-automation bot moved this from Todo to Done in Docs PRs May 22, 2025
@encukou encukou deleted the isolating-opt-out branch May 22, 2025 07:36
@encukou encukou added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels May 22, 2025
@miss-islington-app
Copy link

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 22, 2025
@encukou @colesbury @miss-islington
Add notes on nogil & reinitialization to the Opt-Out section in Modul…
a570d76 
…e Isolation HOWTO (pythonGH-134141)

(cherry picked from commit 1f0a294)

Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Sam Gross <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 22, 2025
@encukou @colesbury @miss-islington
Add notes on nogil & reinitialization to the Opt-Out section in Modul…
0513cf7 
…e Isolation HOWTO (pythonGH-134141)

(cherry picked from commit 1f0a294)

Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Sam Gross <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented May 22, 2025

GH-134491 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label May 22, 2025
@bedevere-app
Copy link

bedevere-app bot commented May 22, 2025

GH-134492 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label May 22, 2025
encukou added a commit that referenced this pull request May 22, 2025
@miss-islington @encukou @colesbury
[3.14] Add notes on nogil & reinitialization to the Opt-Out section i…
9cef5c1 
…n Module Isolation HOWTO (GH-134141) (GH-134492)

Add notes on nogil & reinitialization to the Opt-Out section in Module Isolation HOWTO (GH-134141)
(cherry picked from commit 1f0a294)

Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Sam Gross <[email protected]>
@encukou encukou mentioned this pull request May 23, 2025
Open
encukou added a commit that referenced this pull request May 23, 2025
@miss-islington @encukou @colesbury
[3.13] Add notes on nogil & reinitialization to the Opt-Out section i…
c6497d6 
…n Module Isolation HOWTO (GH-134141) (GH-134491)

Add notes on nogil & reinitialization to the Opt-Out section in Module Isolation HOWTO (GH-134141)
(cherry picked from commit 1f0a294)

Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Sam Gross <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@colesbury colesbury colesbury left review comments

@ZeroIntensity ZeroIntensity ZeroIntensity left review comments

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently

Assignees
No one assigned
Labels
docs Documentation in the Doc dir skip issue skip news topic-C-API
Projects
Docs PRs
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@encukou @colesbury @ZeroIntensity