Error Handling That Holds Up
Catch what you meant, and let the rest through
In this lesson
Unit 3 taught you try/except. This lesson is about using it deliberately: catching the failure you actually anticipated, letting everything else through, and making sure the ones you do catch leave enough evidence to fix them. Handling errors badly is worse than not handling them, because it hides the problem.
Explain it like I’m 5
A bare except is a bucket that catches everything, including the thing you were trying to find. Catch the specific drip instead.
Catch the exception you expected
except Exception: catches everything: the ValueError you anticipated, and also the typo in your variable name, the KeyboardInterrupt, and the bug you have not found yet. All of them get the same treatment, which is almost never right.
Name the exception you are actually prepared to handle. Everything else should be free to travel up and crash the program, loudly, where you will see it.
def parse_year(text):
try:
return int(text)
except ValueError: # the failure we actually expect
return None
print(parse_year("1965"))
print(parse_year("nineteen"))
1965 None
Crashing is a valid strategy
There is a strong instinct that catching an error is always more professional than letting it happen. It is not. If your script cannot reach the database, it cannot do its job, and continuing with a broken connection produces nonsense results that look like real ones.
Handle an error when you have a genuine plan: a default that is actually correct, another source to try, or a record you can legitimately skip. Otherwise let it crash. A stack trace at the point of failure is the most useful bug report you will ever get.
Custom exceptions and keeping the cause
When your code detects a problem that is meaningful to your program rather than to Python, raise your own exception type. It lets callers catch exactly that case, and the class name documents what went wrong.
When you raise one in response to another, use raise ... from err. That keeps the original exception attached, so the traceback shows both the thing you noticed and the underlying cause. Without it, the original is lost and you are debugging with half the evidence.
class InvalidRecord(Exception):
"""A record we cannot turn into usable data."""
def clean(record):
if not record.get("title"):
raise InvalidRecord("missing title")
try:
year = int(record["year"])
except (KeyError, ValueError) as err:
raise InvalidRecord(f"bad year: {record.get('year')!r}") from err
return {"title": record["title"], "year": year}
for record in [{"title": "Dune", "year": "1965"},
{"title": "", "year": "1984"},
{"title": "Neuromancer", "year": "nineteen"}]:
try:
print("ok:", clean(record))
except InvalidRecord as err:
print("skipped:", err)
ok: {'title': 'Dune', 'year': 1965}
skipped: missing title
skipped: bad year: 'nineteen'
Replace the bare except with something deliberate. Raise an InvalidPrice for values that cannot be parsed, and let the caller report them. Negative prices are also invalid.
class InvalidPrice(Exception):
pass
def parse_price(text):
try:
return float(text)
except Exception: # TODO: catch ValueError and raise InvalidPrice from it
return 0.0
# TODO: reject negative prices too
for value in ["12.50", "free", "-3.00"]:
try:
print("ok:", parse_price(value))
except InvalidPrice as err:
print("rejected:", err)Assign the parsed value first, catch ValueError as err and raise InvalidPrice(...) from err. Then check price < 0 after the try block and raise again. Use {text!r} so the bad value appears in quotes.
class InvalidPrice(Exception):
pass
def parse_price(text):
try:
price = float(text)
except ValueError as err:
raise InvalidPrice(f"not a number: {text!r}") from err
if price < 0:
raise InvalidPrice(f"negative price: {price}")
return price
for value in ["12.50", "free", "-3.00"]:
try:
print("ok:", parse_price(value))
except InvalidPrice as err:
print("rejected:", err)Retrying what is worth retrying
Network calls fail for reasons that pass. A timeout or a 503 is worth trying again after a pause; a 404 never is, because the answer will be the same forever.
The pattern is a loop with a wait that grows between attempts, and a final failure that gives up honestly rather than looping forever. (Unit 10 shows how to wrap this up so you write it once instead of at every call site.)
import time
def fetch_with_retry(fetch, attempts=3):
for attempt in range(1, attempts + 1):
try:
return fetch()
except TimeoutError:
if attempt == attempts:
raise # out of tries: let it fail
wait = 2 ** attempt # 2s, then 4s
print(f"attempt {attempt} timed out, retrying in {wait}s")
time.sleep(wait)
Common mistake: Catching Exception when you meant one specific error
It reliably stops the crash, which looks like success.
Name the exception you planned for. A broad catch also swallows typos and logic bugs, turning a clear crash into a mystery.
Common mistake: except Exception: pass
The script needs to keep going and this is the shortest way.
At minimum log it with logging.exception(). Silence means the failure surfaces later as wrong data with nothing pointing at the cause.
Common mistake: Returning a default that hides the failure
return 0 or return None keeps the caller working.
Only return a default when it is genuinely correct. A price of 0.0 for an unparseable value will end up in a total and nobody will know why it is wrong.
Common mistake: Retrying an error that cannot succeed
Retry logic gets applied to every failure equally.
Retry timeouts and 5xx responses. A 404 or a validation error will fail identically three more times, just slower.
What is wrong with except Exception: pass?
It catches everything and reports nothing, so problems surface far from their cause with no evidence.
What does raise NewError(...) from err preserve?
from err chains the exceptions so the underlying cause stays visible.
Which failure is worth retrying?
Retry transient failures. A missing resource and invalid input will fail the same way every time.
Mini exercise (medium)
Write load_all(rows) that turns raw rows into clean records. Raise a custom InvalidRow for rows that cannot be used, catch it per row so one bad row does not stop the batch, log a warning for each rejection, and return the good records plus the count rejected.
Try it yourself. Complete the snippet and hit Run; it executes in your browser, no setup required.
import logging, sys
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s",
stream=sys.stdout, force=True)
class InvalidRow(Exception):
pass
ROWS = [("Dune", "1965"), ("", "1984"), ("Neuromancer", "nineteen")]
def to_record(row):
# TODO: raise InvalidRow for a missing title or an unparseable year
title, raw_year = row
return {"title": title, "year": int(raw_year)}
def load_all(rows):
good, rejected = [], 0
# TODO: convert each row, warn and count rejections
return good, rejected
records, rejected = load_all(ROWS)
print(records)
print("rejected:", rejected)Handle the two failures separately inside the converter: a missing field and an unparseable number. Use raise ... from err for the second so the original ValueError stays attached.
import logging, sys
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s",
stream=sys.stdout, force=True)
class InvalidRow(Exception):
pass
ROWS = [("Dune", "1965"), ("", "1984"), ("Neuromancer", "nineteen")]
def to_record(row):
title, raw_year = row
if not title:
raise InvalidRow("missing title")
try:
year = int(raw_year)
except ValueError as err:
raise InvalidRow(f"bad year {raw_year!r}") from err
return {"title": title, "year": year}
def load_all(rows):
good, rejected = [], 0
for row in rows:
try:
good.append(to_record(row))
except InvalidRow as err:
logging.warning("rejected %s: %s", row, err)
rejected += 1
return good, rejected
records, rejected = load_all(ROWS)
print(records)
print("rejected:", rejected)