freenode
AnalysisLanguages & Toolchains

Free-threaded Python forces the concurrency contract into the open

PEP 805's object-state checks and a push to write down distinct memory models mark the shift from GIL-hidden races to explicit programmer assumptions.

Free-threaded CPython has removed the Global Interpreter Lock's quiet guarantee that most Python code could not race against itself. That removal has turned long-deferred questions about shared object state and concurrent visibility into immediate design problems, now playing out in Mark Shannon's PEP 805 and a simultaneous call to document separate concurrency models for GIL and free-threading builds.

Shannon opened the Safe Parallel Python proposal after a year of intermittent work, stating that parallel execution of code is race free by default: objects must be explicitly declared to be safe to be shared between parallel threads, or such sharing is prohibited. The PEP adds per-object state so the runtime can check, at low cost, whether an operation is safe and raise an exception when it is not. It positions itself as a unification that offers better safety than PEP 703, better sharing than PEP 734, and better performance than either of them. Ownership and freezing become first-class notions: an object starts local to its creating thread; only an explicit freeze or share step makes it visible elsewhere. The check is meant to catch accidental cross-thread mutation before a data race can occur.

That design immediately drew scrutiny over the depth of the freeze. ValorZard asked whether shallow freezing was really really bad, noting that a supposedly immutable object could still hold references to mutable objects living on other threads and that a method call might then race. The counter-argument, voiced by dpdani, is that the first access by a non-owner thread to a still-local object raises an IllegalAccessException, so mutation never begins and the race never materializes. Shannon has also indicated willingness to simplify finalizer behavior if cycles grow too intricate, observing that the language already permits finalizers never to run; Antoine Pitrou replied that CPython users often hold stricter expectations than the spec's minimum. The exchange shows the tension between making the safety net cheap enough for production and making it strong enough that programmers can reason about it without surprise.

At the same time Shannon has argued that the documentation gap itself is now untenable. We have an object model but there is no mention of concurrent access, he wrote, and therefore We should write down concurrency models for both with-GIL and free-threading Python. The models need not be ultra-rigorous at first, only reasonably complete and unambiguous: what the GIL actually guarantees, which common operations are atomic under it, how non-atomic operations behave, and how much latitude other implementations may take. Free-threading requires a different description because its implementation does not maintain sequential consistency and does not even maintain release-acquire ordering in general. Read-modify-write operations such as dict.setdefault receive special attention; users expect them to preserve a consistent modification order even when plain loads and stores use weaker atomics. Lists illustrate the point in concrete terms: some mutating methods synchronize and release on the underlying array while treating size with relaxed ordering, while certain readers may observe sequential consistency or synchronized regular loads depending on the path. The result is a surface that looks familiar to Python programmers yet rests on a weaker foundation than the GIL's mutual exclusion.

Extension authors and anyone shipping threaded native code sit at the sharp end of both proposals. Under the GIL they could often treat Python objects as sequentially consistent by default and rely on the lock to paper over missing happens-before edges. Free-threading removes that paper. PEP 805's runtime state bits give them an explicit way to declare which objects are intentionally shared, but only if the surrounding C code respects the same ownership rules and does not bypass the checks. Documenting the two memory models would at least tell them what the interpreter itself promises, rather than leaving them to reverse-engineer atomic operations from the free-threading build. A related thread-sanitizer effort, originally academic and now being hardened for practical use, aims to surface exactly those missing happens-before relationships, lock inversions, and check-then-act patterns without flooding developers in false positives. Interest from numeric-library maintainers underscores how high the stakes have become once the GIL no longer hides the races.

The two threads reinforce each other. PEP 805 supplies a concrete object-state machine that can enforce a race-free default; the memory-model discussion supplies the vocabulary needed to explain what that machine, and the existing free-threading atomics, actually guarantee. Neither is finished. Shannon's PEP is still early, finalizer and deep-versus-shallow questions remain open, and the documentation effort has only just been framed as necessary rather than drafted. What is clear is that free-threading has ended the era in which the GIL could serve as an implicit concurrency model. The ecosystem is now obliged to say, in runtime checks and in prose, what programmers may assume when objects cross thread boundaries. How much of that contract will be enforced by the interpreter, how much left to careful coding, and how the two builds will be described side by side are the questions still on the table.