The GIL, Processes, and Shared State

Why more threads do not speed up a calculation

Advanced 14 min

In this lesson

Two lessons ago you were told that threads do nothing for calculation, and asked to take it on trust. Here is the reason, and it is not a flaw in your code.

This lesson also covers the failure mode unique to concurrent programs — the one where nothing crashes, no exception is raised, and the answer is simply wrong.

Explain it like I’m 5

The GIL is a single microphone in a room. Extra speakers do not help when only one person may talk at a time. For genuinely parallel talking, you need separate rooms.

The GIL, in one paragraph

CPython (the Python you downloaded) protects its internals with a Global Interpreter Lock. Only one thread may run Python code at any single instant. Threads take turns holding it, swapping many times a second, so they interleave beautifully; they just never run Python code at the same moment.

For waiting, this costs nothing at all: a thread releases the GIL the moment it blocks on the network or the disk, which is exactly why thread pools work so well for I/O. For calculation, it is decisive. Four threads doing arithmetic take turns on one microphone and finish in about the time one thread would, plus the cost of the swapping.

Example · cpu_bound.py 
import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def count_primes(limit):
    """Pure calculation: no waiting anywhere."""
    total = 0
    for n in range(2, limit):
        if all(n % d for d in range(2, int(n ** 0.5) + 1)):
            total += 1
    return total

JOBS = [1_000_000] * 4

if __name__ == "__main__":
    start = time.perf_counter()
    [count_primes(j) for j in JOBS]
    print(f"one at a time: {time.perf_counter() - start:.1f}s")

    start = time.perf_counter()
    with ThreadPoolExecutor(max_workers=4) as pool:
        list(pool.map(count_primes, JOBS))
    print(f"4 threads:     {time.perf_counter() - start:.1f}s")

    start = time.perf_counter()
    with ProcessPoolExecutor(max_workers=4) as pool:
        list(pool.map(count_primes, JOBS))
    print(f"4 processes:   {time.perf_counter() - start:.1f}s")
Output 
one at a time: 5.7s
4 threads:     5.8s
4 processes:   1.6s
Four threads: no faster. Four processes: nearly four times faster.

What processes cost

Processes are not free, which is why they are not simply the default.

They take time to start. Each one is a fresh Python interpreter. That is tens of milliseconds each, irrelevant next to six seconds of work, ruinous if each job takes a millisecond.

They share no memory. Arguments and return values are pickled, shipped between processes, and unpickled. Send a large DataFrame to four workers and the copying can cost more than the calculation. It also means anything you send must be picklable, which rules out open files, database connections, and lambdas.

The practical rule: reach for processes when the jobs are chunky and the data passed is small. Four jobs of two seconds each is ideal. Ten thousand jobs of one millisecond is a disaster.

The bug that does not crash

Now the failure mode unique to concurrency. When two workers change the same variable, you can lose data without a single error appearing.

counter += 1 looks indivisible. It is not. It is three steps: read the value, add one, write it back. A worker can be interrupted between any two of them. If both workers read 5 before either writes, both write 6, and one increment vanishes. That is a race condition, and its defining nastiness is that it depends on timing — so it passes your tests, and fails in production.

Real races are unpredictable, so here it is with the interleaving written out explicitly. This is one way two threads can genuinely land. Both workers want to add 1, so the counter should reach 2. Fill in the four steps in the order the comments give, and see what you actually get.

counter = 0

# Two workers each want to add 1 to the counter.
# An increment is really three steps: read, add, write.
# Here is one interleaving that a real thread swap can produce:

# TODO: 1. worker A reads the counter into a_read
# TODO: 2. worker B reads the counter into b_read  (A has not written yet!)
# TODO: 3. worker A writes back a_read + 1
# TODO: 4. worker B writes back b_read + 1

print("two increments, counter =", counter)
print("should have been:        ", 2)

Locks, and the advice that usually beats them

The textbook fix is a lock: with lock: around the read-modify-write, so no other worker can interleave. It works, and you should know it exists.

The better everyday advice is simpler: do not share. Give each worker its own data, have it return a result, and combine the results yourself when everything is finished. A pool already does this: map and submit hand you return values, and if you only ever use those, races cannot occur.

Locks are also easy to get wrong in ways that are worse than the original bug: forget one and the race remains, take two in the wrong order and the program hangs forever with no error at all. Returning results has none of these failure modes, which is why it is the default worth reaching for.

Common mistake: Blaming your code when threads do not speed up a calculation

Why it happens:

The same pattern made network calls five times faster, so the code looks right.

How to fix it:

It is the GIL, not your code. Switch to ProcessPoolExecutor for CPU-bound work.

Common mistake: Sharing a mutable list or counter across workers

Why it happens:

It works every time on small test data, where the timing rarely lines up badly.

How to fix it:

Have each worker return its own result and combine them afterwards. If you truly must share, guard every read-modify-write with a lock.

Common mistake: Reaching for processes when the work was I/O-bound

Why it happens:

Processes were introduced as the more powerful option, so they sound like the better one.

How to fix it:

For waiting, threads are lighter and need no pickling. Use processes only when the CPU is genuinely the bottleneck.

Common mistake: Forgetting the if __name__ == "__main__" guard

Why it happens:

Nothing else in Python needs it, so it looks like boilerplate to skip.

How to fix it:

Worker processes re-import your file. Without the guard they re-run your script and spawn pools endlessly.

Why do threads not speed up CPU-bound Python?

What does a process pool cost you?

What is the simplest way to avoid a race condition?

Mini exercise (medium)

Two workers, each adding 1 to a counter three times, so the total should be 6. racy_total() is supplied and shares one counter with the bad interleaving. Write safe_total() the other way: let each worker count its own three increments independently, then add the two results together.

Take the wheel. Complete the code, hit Run, and check your output right here.

def racy_total():
    """Two workers share one counter. Each adds 1, three times over."""
    counter = 0
    for _ in range(3):
        a_read = counter        # both workers read before either writes
        b_read = counter
        counter = a_read + 1
        counter = b_read + 1    # A's increment is overwritten
    return counter

def safe_total():
    """The same six increments, with nothing shared."""
    return 0   # TODO: give each worker its own total, then add them up

print("shared counter:", racy_total())
print("each returns:  ", safe_total())
print("should be:     ", 6)

What to learn next

You found out why: the GIL lets one thread run Python bytecode at a time, which costs nothing while waiting and everything while calculating. ProcessPoolExecutor is the answer for real work, at the price of startup time and pickled data. You also produced a race condition on purpose and saw two increments become one — then fixed it the way that always works, by not sharing at all.

Time to put the unit together. The Unit 11 Project makes the Unit 7 fetch several times faster without getting you rate-limited.