Python Sub-Interpreters and Per-Interpreter GIL: Why Splitting One Lock Took a Decade
In The 30-Year Evolution of Python’s GIL and The 20-Year Evolution of Python Async, we looked at how CPython has grappled with the limits of a single Global Interpreter Lock (GIL): async coroutines solved high-concurrency I/O waits, while PEP 703 set out to remove the GIL altogether so multiple threads can share memory directly.
Removing the GIL, however, comes at a steep price. To protect individual objects, free-threading must introduce biased reference counting, thread-safe memory pools, and per-object critical sections. Beyond the underlying complexity, this imposes roughly a 1% to 8% single-threaded performance penalty (varying by CPU platform; early experimental builds of Python 3.13 saw slowdowns of up to ~40% when the specializing adaptive interpreter introduced in 3.11 was temporarily disabled).
To saturate multi-core CPUs without sacrificing the raw single-threaded speed of the GIL, core developer Eric Snow pursued a different path starting in 2014: keep the lock, but shrink its scope from the entire OS process down to a single interpreter. After nearly a decade of refactoring, this effort landed in the Python 3.12 C-API as PEP 684 (Per-Interpreter GIL) and entered the Python 3.14 standard library as concurrent.interpreters via PEP 734.
Sub-Interpreters Existed in 1997—Why Couldn’t They Run in Parallel for 25 Years?
Sub-interpreters are not a Python 3.12 invention—the C-API has provided Py_NewInterpreter() and Py_EndInterpreter() since Python 1.5 in 1997.
They were originally designed for namespace isolation within a single process, not multi-core parallelism. The classic examples were mod_python and mod_wsgi on Apache: hosting multiple Python websites within the same worker process so they could share CPython’s dynamically linked code segment to save resident memory (RSS), while giving each site its own sys.modules and global variables.
Yet anyone who tried to spin up multiple threads to run CPU-bound workloads across separate sub-interpreters hit the exact same wall: in Python 3.11 and earlier, the GIL was bound to the OS process, not the interpreter.
Python 3.11 and earlier: Sub-interpreters share one GIL
- Process-Wide GIL
-
- Main Interp[Thread 1]
- Sub-Interp 1[Thread 2]
- Sub-Interp 2[Thread 3]
In the legacy architecture, the GIL was a process-global lock defined in Python/ceval.c (and later packaged into _PyRuntimeState). Regardless of how many sub-interpreters you created or how many OS threads (pthread) you spawned, every thread had to acquire the exact same GIL before executing bytecode.
As long as Thread 1 in the main interpreter was crunching numbers, Thread 2 in Sub-Interpreter 1 sat blocked on a condition variable, serializing execution across all CPU cores.
Scattered Global Statics: Why Did Splitting One Lock Take a Decade?
If sharing a single lock was the bottleneck, why not just give each sub-interpreter its own GIL right away? Because hundreds of thousands of lines of CPython C code were written under the assumption that only one lock existed per process—leaving the codebase littered with C static global variables.
The moment two sub-interpreters ran simultaneously on different CPU cores, these unisolated shared structures would collide:
| Global Static Structure | Consequence of Unisolated Multi-Core Contention |
|---|---|
| Small Integer Cache | Cached integers from -5 to 256 across the entire process; concurrent reference count updates (Py_INCREF / Py_DECREF) from two interpreters trigger data races |
| Interned String Dictionary | Cached variable names and short strings in a global dictionary; two interpreters parsing code and inserting identical strings simultaneously corrupt the underlying hash buckets |
| Memory Pool Allocator (PyMalloc) | Global pointers managing Arena, Pool, and Block free lists (such as usedpools) lacked lock protection, causing memory corruption |
| Built-in Type Objects | Static structs like PyLong_Type and PyUnicode_Type had their reference counts and internal method lookup caches mutated concurrently across interpreters, overwriting each other |
| Garbage Collector (GC) | A single process-wide doubly linked circular list tracked generational GC objects; one interpreter scanning while another allocated or freed objects broke pointers and caused segfaults |
Giving each interpreter its own lock meant uprooting thousands of these static variables from every corner of the runtime. That foundational cleanup is why Eric Snow’s multi-core initiative, launched in 2014, took nearly ten years to complete.
PEP 684’s Three-Tier State Hierarchy: Moving State Back into Each Interpreter
After purging the static variables, PEP 684 reorganized the CPython runtime into a strict Three-Tier State Hierarchy, moving state that once lived at the top of the process into each individual interpreter.
- Layer 1: _PyRuntimeState (Process Global)OS Signals, Interpreter List, Global rwmutex
-
- Layer 2: InterpState AOwn GIL & Own GC; Own obmalloc & Cache
- Layer 3: ThreadState 1Call Stack & Frame
- Layer 2: InterpState BOwn GIL & Own GC; Own obmalloc & Cache
- Layer 3: ThreadState 2Call Stack & Frame
Under this hierarchy, the process-global layer (_PyRuntimeState) steps into the background, primarily retaining OS signal routing (such as forwarding SIGINT to the main interpreter) and lifecycle management for the active interpreter list. At the bottom, the thread layer (PyThreadState) holds the call stack and exception context for a single OS thread.
The linchpin of multi-core parallelism is the middle tier: the interpreter layer (PyInterpreterState). Runtime machinery that used to sit at the process level—including a dedicated GIL, an isolated memory pool (obmalloc), a generational garbage collector (gc), as well as sys.modules, builtins, the small integer cache, and the interned string table—is now private to each interpreter.
As a result, both memory allocation and Stop-the-World garbage collection pauses are completely localized: when Interpreter A triggers cyclic garbage collection, only threads inside Interpreter A pause, while Interpreter B on another CPU core keeps running at full speed. Meanwhile, the C-API preserves the legacy shared-GIL configuration mode so existing embedded applications like older mod_wsgi deployments keep working unchanged.
Why Can’t Sub-Interpreters Share Objects Directly? Reference Counting and Immortal Objects
Since multiple sub-interpreters live inside the same OS process, why can’t they just pass Python object pointers around and skip copying data altogether?
The answer lies in Python’s most fundamental memory mechanism: in CPython, even “reading” an object triggers a memory write.
Reference Counting and Cache-Line Bouncing
When you write a trivial assignment like x = None, CPython runs Py_INCREF(Py_None) under the hood to increment the reference count by 1, and runs Py_DECREF to decrement it by 1 when the variable leaves scope.
Each interpreter’s GIL only governs threads inside that interpreter—Interpreter A’s GIL has zero authority over Interpreter B. When two interpreters run simultaneously on separate CPU cores, passing an object pointer across them means reference counts once protected by a single GIL are mutated concurrently from both sides, hitting two distinct hardware bottlenecks:
- Data races from non-atomic updates: A standard
ob_refcnt++compiles down to three CPU operations: read, increment, and write back. If two cores increment or decrement the same object’s reference count simultaneously, the count becomes corrupted—causing either premature deallocation and crashes, or memory that can never be reclaimed. - Cache-line bouncing: Even if every reference count update used atomic CPU instructions (such as
LOCK XADDon x86), having two cores frequently mutateob_refcnton the same object forces the 64-byte cache line containing that object to be continuously invalidated and synchronized between L1/L2 caches under the MESI cache-coherency protocol. The resulting bus synchronization latency wipes out the speedup gained from multi-core parallelism.
Consequently, as long as an object’s reference count can change, its pointer cannot be shared directly across interpreters. Each sub-interpreter must strictly isolate its own dynamic heap: a list or dict created in Interpreter A can only be managed by Interpreter A’s memory pool and GC.
PEP 683 Immortal Objects: Pinning the Reference Count
If every sub-interpreter had to instantiate its own copy of immutable foundational constants like None, True, and False, both memory usage and startup time would suffer; yet sharing them across interpreters requires avoiding the reference count writes that trigger cache-line bouncing.
To solve this, Python 3.12 introduced PEP 683 (Immortal Objects) alongside PEP 684, pinning the reference counts of compile-time read-only core singletons so they can return to process-global memory and be safely shared across all interpreters, while keeping dynamically changing objects and cache tables private to each interpreter.
The mechanism behind immortal objects is straightforward: set the ob_refcnt of core constants to a high-bit sentinel constant (around 3.2 billion initially on 64-bit systems, where any value above the threshold is treated as immortal). Whenever Py_INCREF or Py_DECREF sees this marker, it skips the increment or decrement entirely and performs no memory write at all.
Because no memory write occurs, the cache lines holding None, True, False, Ellipsis, the empty tuple (), common single-character ASCII strings, and global built-in types remain in a read-only Shared state across every CPU core’s L1 cache. Sub-interpreters can safely share these foundational singletons without triggering a single cache-line bounce.
Python 3.14 in Practice: How Do You Pass Data Across Interpreters?
If dynamic objects cannot share pointers, how do multiple sub-interpreters coordinate work?
The architecture follows a philosophy familiar from the Actor model and Go’s CSP: do not communicate by sharing memory; instead, share data by communicating. Python 3.12 exposed the low-level C-API first, and in Python 3.14, PEP 734 brought concurrent.interpreters and InterpreterPoolExecutor into the standard library.
1. Standard Library API and InterpreterPoolExecutor
The easiest way to adopt sub-interpreters is as a drop-in executor in concurrent.futures. InterpreterPoolExecutor is a subclass of ThreadPoolExecutor and, together with ProcessPoolExecutor, implements the same Executor interface:
from concurrent.futures import InterpreterPoolExecutor
def cpu_bound_worker(chunk: list[int]) -> int:
return sum(x * x for x in chunk)
if __name__ == "__main__":
data_chunks = [list(range(i, i + 1_000_000)) for i in range(4)]
# Create 4 sub-interpreters, each running in parallel with its own GIL
with InterpreterPoolExecutor(max_workers=4) as executor:
results = list(executor.map(cpu_bound_worker, data_chunks))
print("Total sum:", sum(results))
For finer-grained lifecycle control and bidirectional communication, you can use concurrent.interpreters directly to create sub-interpreters and cross-interpreter queues:
from concurrent import interpreters
interp = interpreters.create()
queue = interpreters.create_queue()
# Bind the queue into the sub-interpreter's __main__ namespace
interp.prepare_main({"task_queue": queue})
queue.put(10)
interp.exec("""
import math
data = task_queue.get()
task_queue.put(math.factorial(data))
""")
print(queue.get()) # Output: 3628800
interp.close()
2. How Is Data Safely Transported Under the Hood?
When passing data across interpreters, concurrent.interpreters strips the source interpreter’s PyObject down to raw C data, then reconstructs a brand-new Python object using the target interpreter’s memory pool. Developers can check whether an object supports direct cross-interpreter sharing via interpreters.is_shareable(obj). Under the hood, CPython uses different transport strategies depending on the data type:
| Data Category | Cross-Interpreter Transport Mechanism | Memory & Communication Characteristics |
|---|---|---|
Shareable TypesNone, booleans, int, float, str, bytes, and tuples of shareable elements | Stripped down to raw C data and reconstructed as a new PyObject in the receiving interpreter’s memory pool | No general-purpose serialization needed; passing a string transfers only the internal character buffer pointer (with kind and length metadata) so the receiver allocates a fresh string |
Mutable Containers & Complex Objectslist, dict, and other mutable objects | Objects that are neither shareable nor pickle-able raise NotShareableError; queues and executors default to pickle copying | Completed entirely in-memory within the same OS process, with no OS pipe or socket overhead |
Large Binary Buffersmemoryview and contiguous binary data | Shares the underlying contiguous C memory (raw buffer) directly across interpreters via the Buffer Protocol | Zero-copy; shares neither Python headers nor reference counts, avoiding cache-line bouncing while tracking references at the source to prevent premature release |
The Biggest Pain Point: Why Can’t C Extensions Like NumPy Be Used Directly Yet?
Pure Python code runs at native speed inside sub-interpreters, but the moment you run import numpy or load many third-party packages inside a sub-interpreter, you may hit ImportError: module <name> does not support loading in subinterpreters.
The root cause is identical to what CPython’s own core had to fix: traditional C extension modules also declare plenty of C static global variables.
Legacy Python C extensions used “Single-Phase Initialization,” where modules routinely declared cached dictionaries, counters, or custom types directly as static global variables in their C source files.
If two sub-interpreters load such a module simultaneously, both read and write the exact same process data segment at the C level, triggering immediate data races; once one sub-interpreter shuts down and frees resources tied to those types, another sub-interpreter still running will segfault on the spot when it touches that type.
To address this, Python defined multi-phase initialization in PEP 489 (Multi-Phase Extension Module Initialization) and extended it with support flags in PEP 684, requiring C extensions to do three things to safely support sub-interpreters:
- Eliminate C static variables: Move all module state into a dedicated struct, dynamically allocating an isolated copy each time a sub-interpreter imports the module.
- Switch to heap types: Create custom types dynamically on the heap so type lifecycles follow each individual interpreter.
- Declare explicit support: Explicitly mark support for a per-interpreter GIL (
Py_MOD_PER_INTERPRETER_GIL_SUPPORTED) in the module’s slot definitions.
IMPORTANT
Unconverted C Extensions Are Blocked by Default: To prevent legacy C extensions from silently corrupting memory in the background, sub-interpreters with a per-interpreter GIL enforce an allowlist—unless a module implements PEP 489 multi-phase initialization and declares the Py_MOD_PER_INTERPRETER_GIL_SUPPORTED flag, importing it inside a sub-interpreter is blocked outright by CPython with an ImportError. This protects process stability, but it also means scientific computing and machine learning libraries built on complex C/C++ extensions still need time to finish isolating their internal state.
Three Paths to Parallelism: Choosing Between Multiprocessing, Sub-Interpreters, and Free-Threading
By Python 3.14, backend and data engineers facing CPU-bound workloads have three architecturally distinct parallel tools at their disposal. None renders the others obsolete; each occupies a distinct point on the engineering trade-off spectrum:
| Dimension | Multiprocessing (multiprocessing) | Sub-Interpreters (Per-Interpreter GIL) | Free-Threading |
|---|---|---|---|
| GIL & Execution Unit | Multiple OS processes, one GIL per process | Single process, multiple threads, one GIL per interpreter | Single process, multiple threads, no GIL |
| Single-Threaded Speed | 0% penalty (native speed) | 0% penalty (native speed) | ~1%–8% slower (varies by platform; ~40% slower in early 3.13) |
| Memory Model | Physically isolated process address spaces | Shared process address space, isolated Python heaps | Shared process address space, fully shared heap |
| Memory & Startup Cost | High (slow process startup, CoW easily broken) | Low (shares process space & immortal objects, but separate imports) | Lowest (creates threads only, shares a single heap) |
| Cross-Unit Data Transfer | OS IPC (Pipe/Socket + Pickle) | In-process C data reconstruction / Buffer zero-copy | Direct object pointer sharing (zero communication cost) |
| Data Race Risk | None | None (no shared mutable Python objects) | High (requires explicit locking against race conditions) |
| C Extension Compatibility | 100% compatible with all existing packages | Requires PEP 489 & declared support flag | Requires free-threaded compatibility (otherwise falls back to GIL) |
| Crash Isolation | High (child crash does not affect parent process) | Low (C-level crash takes down the entire process) | Low (C-level crash takes down the entire process) |
1. Why Sub-Interpreters Address the Pain Points of multiprocessing
For years, multiprocessing was the standard answer for bypassing the GIL, but in large backend services it suffers from two chronic pain points:
- Copy-on-Write (CoW) invalidation: When you spawn multiple worker processes with
forkon Linux, they nominally share memory pages with the parent process. Yet even with PEP 683 immortal objects protecting core constants, user modules, class dictionaries, and dynamic objects remain mortal; the moment a worker reads those objects, it modifiesob_refcnt, triggering OS Copy-on-Write to duplicate the entire memory page and often causing Gunicorn or Celery cluster memory usage to spike within minutes of startup. - High cross-process IPC cost:
multiprocessing.Queuemust pickle data, write it into a kernel pipe or socket buffer, and read and unpickle it in the receiving process, incurring two context switches and heavy copying overhead.
Although sub-interpreters still need to import application modules individually and build isolated namespaces, they completely bypass OS fork and Copy-on-Write.
Running as multiple threads within a single process, all sub-interpreters share dynamically linked code segments (.text) and global immortal objects. Each interpreter’s memory footprint is predictable and never spikes from CoW page duplication, while cross-interpreter communication stays entirely out of the OS kernel—and interpreters can exchange raw binary data with zero copying via the Buffer Protocol.
2. Sub-Interpreters vs. Free-Threading: Isolation Safety vs. Shared-Memory Flexibility
Now that Python 3.14 officially supports python3.14t (free-threading), why do we still need sub-interpreters?
- No single-threaded slowdown: Most web backends and data pipelines spend a large share of their time running single-threaded logic. Inside each sub-interpreter, CPython keeps its lean, traditional execution engine without paying a global performance tax for fine-grained locks and biased reference counting.
- No multi-threaded debugging nightmares: Under free-threading, built-in
dictandlistoperations have internal critical section protection so concurrent access won’t corrupt C memory directly, but two threads reading and writing the same application state simultaneously will still trigger logical race conditions. Sub-interpreters enforce a Shared-Nothing architecture by default, giving developers parallel speedups without having to manage multi-threaded locks by hand.
In practice, three clear criteria separate the three models:
TIP
Three Practical Decision Criteria:
- Free-Threading: When multiple threads must frequently read and write the same large mutable state (such as large in-memory graphs or shared cache trees).
- Multiprocessing: When heavily relying on legacy C extensions that do not yet support PEP 489, or when process-level crash isolation is required.
- Sub-Interpreters (Per-Interpreter GIL): For pure-Python CPU-bound workloads and staged data pipelines with clear task boundaries, combining native single-threaded speed with memory isolation between interpreters.