Context Managers and the with Statement
Cleanup that happens even when things go wrong
In this lesson
You have written with open(...) since Unit 3 without asking what with does. It runs a context manager: an object with a setup step and a cleanup step, where the cleanup runs whatever happens — including when your code raises halfway through.
Explain it like I’m 5
A context manager is a door that closes itself behind you, even if you leave in a hurry because something caught fire.
Two methods, and a guarantee
A context manager is any object with __enter__ and __exit__. with calls __enter__ before the block, and __exit__ after it, on the way out, no matter how you leave.
Whatever __enter__ returns is what as binds. That is where the file object in with open(...) as f comes from.
class Tracked:
def __enter__(self):
print("enter")
return self
def __exit__(self, exc_type, exc_value, traceback):
print("exit")
return False # do not swallow exceptions
with Tracked():
print("body")
enter body exit
The demonstration that matters
Everything above is also true of ordinary code written in order. The reason context managers exist is what happens when the block fails: __exit__ still runs.
Cleanup written at the end of a block is skipped by an exception. Cleanup in __exit__ is not. That is the whole value.
class Tracked:
def __enter__(self):
print("enter")
return self
def __exit__(self, exc_type, exc_value, traceback):
print("exit (exception was:", exc_type.__name__ if exc_type else None, ")")
return False
try:
with Tracked():
print("body starts")
raise ValueError("boom")
print("never reached")
except ValueError as err:
print("caught outside:", err)
enter body starts exit (exception was: ValueError ) caught outside: boom
The shorter way to write one
from contextlib import contextmanager
@contextmanager
def tracked(label):
print(f"enter {label}")
try:
yield label # the with-block runs here
finally:
print(f"exit {label}") # runs even if the block raises
with tracked("job") as name:
print("working on", name)
try:
with tracked("risky"):
raise ValueError("boom")
except ValueError:
print("handled outside")
enter job working on job exit job enter risky exit risky handled outside
Write a context manager that announces entering and leaving a section. Print "start: NAME" on the way in and "end: NAME" on the way out, including when the block raises. Use try/finally around the yield.
from contextlib import contextmanager
@contextmanager
def section(name):
pass # TODO: print "start: NAME", yield, then print "end: NAME" in a finally
with section("load"):
print("loading data")
try:
with section("save"):
raise ValueError("disk full")
except ValueError as err:
print("caught:", err)Print the start line, then try: yield name followed by finally: print(f"end: {name}"). The finally is what makes the end line appear even for the block that raises.
from contextlib import contextmanager
@contextmanager
def section(name):
print(f"start: {name}")
try:
yield name
finally:
print(f"end: {name}")
with section("load"):
print("loading data")
try:
with section("save"):
raise ValueError("disk full")
except ValueError as err:
print("caught:", err)Where you will actually use them
Anything acquired then released is a candidate: files, database connections, locks, temporary directories, timers. If you find yourself writing ‘open it, use it, remember to close it’, that last part belongs in a context manager rather than in your memory.
The database connection from Unit 8 is the obvious example. with conn: commits on success and rolls back on failure. That is a context manager doing exactly this job.
from contextlib import contextmanager
import sqlite3
@contextmanager
def database(path=":memory:"):
conn = sqlite3.connect(path)
try:
yield conn
conn.commit() # success: save the work
except Exception:
conn.rollback() # failure: undo it
raise
finally:
conn.close() # always: release the file
with database() as conn:
conn.execute("CREATE TABLE books (title TEXT)")
conn.execute("INSERT INTO books VALUES (?)", ("Dune",))
print("rows:", conn.execute("SELECT COUNT(*) FROM books").fetchone()[0])
print("connection closed cleanly")
rows: 1 connection closed cleanly
Common mistake: Putting cleanup at the end of the block instead of in __exit__
It reads in the right order and works whenever nothing fails.
An exception skips the rest of the block, so that cleanup never runs. Put it in __exit__ or a finally.
Common mistake: Forgetting yield in a @contextmanager function
It looks like an ordinary function.
Without yield it is not a generator and with raises an error. There must be exactly one yield, marking the boundary between setup and cleanup.
Common mistake: Returning True from __exit__ by accident
Returning something feels tidier than returning nothing.
A truthy return suppresses the exception entirely. Return False or nothing unless you deliberately want it swallowed.
Common mistake: Leaving the yield outside a try/finally
The cleanup line is right there after the yield, so it looks guaranteed.
If the block raises, the exception travels back through the yield and skips the rest. Only a finally guarantees it runs.
When does __exit__ run?
That guarantee is the entire reason context managers exist.
In @contextmanager, what separates setup from cleanup?
Everything before yield is setup, everything after is cleanup, and the yielded value is what as binds.
What happens if __exit__ returns True?
A truthy return means ‘handled’, which silently swallows the error, rarely what you want.
Mini exercise (medium)
Write a collecting context manager that gathers lines. On entry it yields an empty list for the block to append to; on exit it prints "collected N lines", even if the block raises. Prove it by using it twice, once cleanly and once with an exception.
Take the wheel. Complete the code, hit Run, and check your output right here.
from contextlib import contextmanager
@contextmanager
def collecting():
pass # TODO: yield an empty list, and print "collected N lines" on the way out
with collecting() as lines:
lines.append("one")
lines.append("two")
try:
with collecting() as lines:
lines.append("partial")
raise ValueError("stopped early")
except ValueError as err:
print("caught:", err)Create the list before the yield, yield it so the block can append, and put the print in a finally. The caller appends through the name bound by as.
from contextlib import contextmanager
@contextmanager
def collecting():
lines = []
try:
yield lines
finally:
print(f"collected {len(lines)} lines")
with collecting() as lines:
lines.append("one")
lines.append("two")
try:
with collecting() as lines:
lines.append("partial")
raise ValueError("stopped early")
except ValueError as err:
print("caught:", err)