https://github.com/python/cpython/pull/116338 Skip to content Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} python / cpython Public * * Notifications * Fork 28.5k * Star 58.8k * Code * Issues 5k+ * Pull requests 1.5k * Actions * Projects 27 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights New issue Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community. Pick a username [ ] Email Address [ ] Password [ ] [ ] Sign up for GitHub By clicking "Sign up for GitHub", you agree to our terms of service and privacy statement. We'll occasionally send you account related emails. Already on GitHub? Sign in to your account Jump to bottom gh-116167: Allow disabling the GIL with PYTHON_GIL=0 or -X gil=0 # 116338 Merged colesbury merged 11 commits into python:main from swtaarrs: cpython-disable-gil Mar 11, 2024 Merged gh-116167: Allow disabling the GIL with PYTHON_GIL=0 or -X gil=0 # 116338 colesbury merged 11 commits into python:main from swtaarrs: cpython-disable-gil Mar 11, 2024 +163 -1 Conversation 20 Commits 11 Checks 31 Files changed 12 Conversation This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters swtaarrs Copy link Member @swtaarrs swtaarrs commented Mar 4, 2024 * edited In free-threaded builds, running with PYTHON_GIL=0 or -X gil=0 will now disable the GIL. #116322 and #116329 track follow-up work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default, respectively. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag at startup that causes take_gil() and drop_gil() to return early. With PYTHON_GIL=0 set, I spot-checked a few tests and small programs that don't use threads. They all seem to run fine, and very basic threaded programs work, sometimes. Trying to run the full test suite crashes pretty quickly, in test_asyncio. * Issue: Add a mechanism to disable the GIL #116167 Sorry, something went wrong. 29 colesbury, corona10, erlend-aasland, philippjfr, avinayak, tri-chu, tomwojcik, Josua996, jotamudo, alexfromapex, and 19 more reacted with hooray emoji 25 itamaro, Eclips4, corona10, erlend-aasland, philippjfr, avinayak, tomwojcik, alexfromapex, jacoblearned, tizz98, and 15 more reacted with rocket emoji All reactions * 29 reactions * 25 reactions @swtaarrs swtaarrs added the topic-free-threading label Mar 4, 2024 @swtaarrs swtaarrs requested a review from colesbury March 4, 2024 22:58 @bedevere-app bedevere-app bot mentioned this pull request Mar 4, 2024 Add a mechanism to disable the GIL #116167 Open @swtaarrs swtaarrs added the skip news label Mar 4, 2024 @swtaarrs Copy link Member Author swtaarrs commented Mar 4, 2024 !buildbot nogil All reactions Sorry, something went wrong. @bedevere-bot Copy link bedevere-bot commented Mar 4, 2024 New build scheduled with the buildbot fleet by @swtaarrs for commit ac24233 The command will test the builders whose names match following regular expression: nogil The builders matched are: * x86-64 MacOS Intel ASAN NoGIL PR * x86-64 MacOS Intel NoGIL PR * ARM64 MacOS M1 Refleaks NoGIL PR * ARM64 MacOS M1 NoGIL PR * AMD64 Ubuntu NoGIL Refleaks PR * AMD64 Ubuntu NoGIL PR * AMD64 Windows Server 2022 NoGIL PR All reactions Sorry, something went wrong. @swtaarrs Allow disabling the GIL with PYTHON_GIL=0 ... 9e03999 In free-threaded builds, running with `PYTHON_GIL=0` will now disable the GIL. Follow-up issues track work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag that causes `take_gil()` and `drop_gil()` to return early. With `PYTHON_GIL=0` set, I spot-checked a few tests and small programs that don't use threads. They all seem to run fine, and very basic threaded programs work, sometimes. Trying to run the full test suite crashes pretty quickly, in `test_asyncio`. @swtaarrs swtaarrs force-pushed the cpython-disable-gil branch from ac24233 to 9e03999 Compare March 5, 2024 20:38 @swtaarrs Copy link Member Author swtaarrs commented Mar 5, 2024 !buildbot nogil All reactions Sorry, something went wrong. @bedevere-bot Copy link bedevere-bot commented Mar 5, 2024 New build scheduled with the buildbot fleet by @swtaarrs for commit 9e03999 The command will test the builders whose names match following regular expression: nogil The builders matched are: * x86-64 MacOS Intel ASAN NoGIL PR * ARM64 MacOS M1 Refleaks NoGIL PR * AMD64 Windows Server 2022 NoGIL PR * AMD64 Ubuntu NoGIL PR * x86-64 MacOS Intel NoGIL PR * AMD64 Ubuntu NoGIL Refleaks PR * ARM64 MacOS M1 NoGIL PR All reactions Sorry, something went wrong. corona10 corona10 reviewed Mar 5, 2024 View reviewed changes Copy link Member @corona10 corona10 left a comment * edited There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment Good to add test case in this file too. cpython/Lib/test/test_cmd_line.py Line 896 in 72714c0 def test_int_max_str_digits(self): Sorry, something went wrong. All reactions @corona10 Copy link Member corona10 commented Mar 5, 2024 * edited [DEL:Ah ignore above comments, this PR only referencing the environment variable so not proper for this case.:DEL] You can just add the testcase for the environment variable, still worth to adding it. 1 swtaarrs reacted with thumbs up emoji All reactions * 1 reaction Sorry, something went wrong. @corona10 Copy link Member corona10 commented Mar 5, 2024 At not for this PR, since this feature is not officially announced, but in some moment, you should write about the environment variable at https://github.com/python/cpython/blob/ 72714c0266ce6d39c7c7fb63f617573b8f5a3cb2/Doc/using/cmdline.rst to get user feedback. 1 swtaarrs reacted with thumbs up emoji All reactions * 1 reaction Sorry, something went wrong. swtaarrs added 3 commits March 5, 2024 13:40 @swtaarrs Don't add 'enable_gil' to test_embed in normal builds a53ed2e @swtaarrs Set enable_gil properly when PYTHON_GIL=1 e0b8484 @swtaarrs Add test for PYTHON_GIL in test_cmd_line 4f43a87 @swtaarrs Copy link Member Author swtaarrs commented Mar 5, 2024 !buildbot nogil All reactions Sorry, something went wrong. @bedevere-bot Copy link bedevere-bot commented Mar 5, 2024 New build scheduled with the buildbot fleet by @swtaarrs for commit 4f43a87 The command will test the builders whose names match following regular expression: nogil The builders matched are: * x86-64 MacOS Intel ASAN NoGIL PR * ARM64 MacOS M1 NoGIL PR * AMD64 Ubuntu NoGIL PR * x86-64 MacOS Intel NoGIL PR * ARM64 MacOS M1 Refleaks NoGIL PR * AMD64 Ubuntu NoGIL Refleaks PR * AMD64 Windows Server 2022 NoGIL PR All reactions Sorry, something went wrong. @swtaarrs swtaarrs marked this pull request as ready for review March 5, 2024 22:56 @swtaarrs swtaarrs requested a review from ericsnowcurrently as a code owner March 5, 2024 22:56 @bedevere-app bedevere-app bot added the awaiting review label Mar 5, 2024 @swtaarrs swtaarrs requested a review from vstinner March 5, 2024 23:16 @swtaarrs Copy link Member Author swtaarrs commented Mar 5, 2024 you should write about the environment variable ... to get user feedback What would be a good time to do that? Maybe once the GIL is disabled by default, so the description doesn't have to change? All reactions Sorry, something went wrong. colesbury colesbury reviewed Mar 5, 2024 View reviewed changes Python/initconfig.c Outdated Show resolved Hide resolved @swtaarrs Fix else brace formatting 9e9d925 @colesbury Copy link Contributor colesbury commented Mar 6, 2024 * edited I would document the environment variable now, when it's added. For reference, we already document the --disable-gil configure flag. I don't think it needs to be extensive, but should cover: * Only available in free-threaded builds * 0 forces the GIL to be disabled * 1 forces the GIL to be enabled * New in version 3.13 The behavior when then environment variable is not set is what's going to change in the future. We can document that later. 4 swtaarrs, ambv, corona10, and erlend-aasland reacted with thumbs up emoji All reactions * 4 reactions Sorry, something went wrong. @swtaarrs Document PYTHON_GIL environment variable 5053661 @swtaarrs Copy link Member Author swtaarrs commented Mar 7, 2024 After discussion in Discord, I'm also going to add a -X option to go with the environment variable. All reactions Sorry, something went wrong. @swtaarrs swtaarrs marked this pull request as draft March 7, 2024 14:31 @bedevere-app bedevere-app bot removed the awaiting review label Mar 7, 2024 swtaarrs added 2 commits March 7, 2024 12:25 @swtaarrs Merge remote-tracking branch 'upstream/main' into HEAD 6d62fbb @swtaarrs Add -X gil option, add to sys.flags, modify test to cover env var... ... 971a9a1 ... and option @swtaarrs swtaarrs changed the title [DEL:gh-116167: Allow disabling the GIL with PYTHON_GIL=0:DEL] [INS:gh-116167: Allow disabling the GIL with PYTHON_GIL=0 or -X gil=0:INS] Mar 7, 2024 swtaarrs added 2 commits March 7, 2024 12:49 @swtaarrs Fix link to -X gil 89b1150 @swtaarrs Fix PYTHON_GIL versionchanged line 16fbd44 @swtaarrs swtaarrs marked this pull request as ready for review March 7, 2024 22:29 @swtaarrs swtaarrs requested a review from gpshead as a code owner March 7, 2024 22:29 @bedevere-app bedevere-app bot added the awaiting review label Mar 7, 2024 @colesbury colesbury requested a review from erlend-aasland March 7, 2024 22:47 ericsnowcurrently ericsnowcurrently approved these changes Mar 8, 2024 View reviewed changes Copy link Member @ericsnowcurrently ericsnowcurrently left a comment There was a problem hiding this comment. Choose a reason for hiding this comment The reason will be displayed to describe this comment to others. Learn more. [Choose a reason] Hide comment LGTM I've left one minor comment about an unclear test. It really isn't a blocker, as the test is passing, but it would still be nice to have a response. Sorry, something went wrong. All reactions Lib/test/_test_embed_set_config.py Outdated Show resolved Hide resolved @bedevere-app bedevere-app bot added awaiting merge and removed awaiting review labels Mar 8, 2024 erlend-aasland erlend-aasland approved these changes Mar 9, 2024 View reviewed changes corona10 corona10 approved these changes Mar 10, 2024 View reviewed changes @swtaarrs Clarify test_flags in normal builds 8d39bbc colesbury colesbury approved these changes Mar 11, 2024 View reviewed changes Hide details View details @colesbury colesbury merged commit 2731913 into python:main Mar 11, 2024 34 checks passed @bedevere-app bedevere-app bot removed the awaiting merge label Mar 11, 2024 @vstinner Copy link Member vstinner commented Mar 11, 2024 This is both exciting and very scary. Congrats @swtaarrs and @colesbury :-) 2 s373r and edpyt reacted with hooray emoji [?] 4 swtaarrs, s373r, matej-mnoucek-adsk, and edpyt reacted with heart emoji 6 jezdez, cpvandehey, nkhitrov, edpyt, K0IN, and zaphodschmidt reacted with rocket emoji All reactions * 2 reactions * [?] 4 reactions * 6 reactions Sorry, something went wrong. @alikud This comment was marked as off-topic. Sign in to view @edpyt This comment was marked as off-topic. Sign in to view @fractalvision This comment was marked as off-topic. Sign in to view @python python locked as spam and limited conversation to collaborators Mar 11, 2024 Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in. Reviewers @colesbury colesbury colesbury approved these changes @ericsnowcurrently ericsnowcurrently ericsnowcurrently approved these changes @corona10 corona10 corona10 approved these changes @erlend-aasland erlend-aasland erlend-aasland approved these changes @vstinner vstinner Awaiting requested review from vstinner @gpshead gpshead Awaiting requested review from gpshead gpshead is a code owner Assignees No one assigned Labels skip news topic-free-threading Projects None yet Milestone No milestone Development Successfully merging this pull request may close these issues. None yet 10 participants @swtaarrs @bedevere-bot @corona10 @colesbury @vstinner @alikud @edpyt @fractalvision @ericsnowcurrently @erlend-aasland Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later. Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.