Tests That Catch Real Bugs: assert to pytest
The bug you just fixed should never come back
In this lesson
Unit 5 introduced assert and mentioned pytest in passing. Now it earns a lesson of its own, because a test is the only tool that catches the bug the previous lesson called expensive: the one that runs cleanly and returns the wrong answer.
The goal is not full coverage. It is a handful of tests on the parts that would actually hurt, running in under a second, so that you notice a break within moments of causing it.
Explain it like I’m 5
A test is a question you already know the answer to, written down so the computer can keep asking it for you.
A test is three lines: arrange, act, assert
Every test has the same shape, and naming the parts makes them much easier to write.
- Arrange — set up the input.
- Act — call the function you are testing, once.
- Assert — state what the result must be.
If a test is hard to write, that is information: usually the unit test is telling you the function does too many things, or it needs the network, or the clock, or a real file. Splitting out the pure part is both the easier test and the better design.
def split_name(full_name):
"""Split 'Ada Lovelace' into ('Ada', 'Lovelace')."""
parts = full_name.strip().split()
return parts[0], " ".join(parts[1:])
# Arrange, act, assert - three times over.
assert split_name("Ada Lovelace") == ("Ada", "Lovelace")
assert split_name(" Grace Brewster Hopper ") == ("Grace", "Brewster Hopper")
assert split_name("Prince") == ("Prince", "")
print("3 checks passed")
3 checks passed
pytest: the same asserts, organized
Bare asserts stop scaling once you have twenty of them: the first failure stops the file, so you never see the other nineteen. pytest fixes exactly that and asks almost nothing in return.
The rules are small enough to memorize. Put tests in files named test_*.py, write functions named test_*, and use plain assert. Run pytest in the project folder and it finds them.
What you get back is worth the install: every test runs even when one fails, and a failure prints the values on both sides of the comparison rather than a bare AssertionError.
# test_names.py - a real pytest file.
from names import split_name
def test_simple_two_part_name():
assert split_name("Ada Lovelace") == ("Ada", "Lovelace")
def test_extra_whitespace_is_ignored():
assert split_name(" Grace Hopper ") == ("Grace", "Hopper")
def test_single_word_name_has_no_surname():
assert split_name("Prince") == ("Prince", "")
def test_empty_name_raises():
import pytest
with pytest.raises(IndexError):
split_name("")
# Run with: pytest # 4 tests are collected and run, and each reports pass or fail # independently, so one failure never hides the other three.
$ pip install pytest
$ pytest -q
.... [100%]
4 passed in 0.00s
$ pytest -q # after someone breaks split_name
..F. [100%]
================================== FAILURES ==================================
____________________ test_single_word_name_has_no_surname ____________________
def test_single_word_name_has_no_surname():
> assert split_name("Prince") == ("Prince", "")
E AssertionError: assert ('Prince', 'Unknown') == ('Prince', '')
E
E At index 1 diff: 'Unknown' != ''
E Use -v to get more diff
test_names.py:14: AssertionError
========================== short test summary info ===========================
FAILED test_names.py::test_single_word_name_has_no_surname - AssertionError...
1 failed, 3 passed in 0.01s
Test the edges, not the middle
New testers write three versions of the case that obviously works. Bugs do not live there. They live at the boundaries:
- Empty — an empty list, an empty string, no rows. This one finds the most bugs of any single case.
- One — a single item often breaks code written for “many”.
- Boundary — exactly at the limit, and one either side of it.
- Wrong shape — the missing key, the string where a number goes.
- Duplicates and ties — two items with the same score; which wins?
When several cases test the same behavior with different values, @pytest.mark.parametrize writes them once and runs them separately, so each value gets its own pass or fail.
import pytest
from grades import letter
@pytest.mark.parametrize("score, expected", [
(100, "A"),
(70, "A"), # exactly on the boundary
(69, "B"), # one below it
(50, "B"),
(49, "C"),
(0, "C"),
])
def test_letter_grades(score, expected):
assert letter(score) == expected
def test_negative_score_is_rejected():
with pytest.raises(ValueError):
letter(-1)
# pytest runs test_letter_grades six separate times, once per row, # and names each one in the output, e.g. test_letter_grades[70-A]. # A failure at exactly 70 therefore points straight at the boundary.
Here is letter() with a genuine off-by-one bug: 70 should be an A, and it is not. Write the boundary checks that expose it, then fix the function so all four pass. Test first, then fix — that order is the point.
def letter(score):
"""A for 70+, B for 50-69, C below 50."""
if score > 70:
return "A"
if score >= 50:
return "B"
return "C"
# TODO: four checks - 71, 70, 69 and 50 - then fix the bug they reveal.
checks = []
for score, expected in checks:
got = letter(score)
print(f"{score}: expected {expected}, got {got}")
assert got == expected
print(f"{len(checks)} checks passed")Fill checks with the four pairs (71, "A"), (70, "A"), (69, "B") and (50, "B"). Run it and watch 70 fail. The docstring says “70+”, so the comparison must be >= rather than >.
def letter(score):
"""A for 70+, B for 50-69, C below 50."""
if score >= 70:
return "A"
if score >= 50:
return "B"
return "C"
checks = [(71, "A"), (70, "A"), (69, "B"), (50, "B")]
for score, expected in checks:
got = letter(score)
print(f"{score}: expected {expected}, got {got}")
assert got == expected
print(f"{len(checks)} checks passed")The regression test: never fix a bug twice
Here is the habit that makes testing pay for itself. When you find a bug, do not fix it first. Write the test that fails because of it, watch it fail, and only then fix the code and watch it pass.
Two things come free. You have proved the test can actually detect the problem: a test that has never failed has never been checked. And the bug can never silently return, because something now asks about it on every run.
The smallest failing example you shrank down in the previous lesson is exactly the input this test wants. That is why the debugging step said to save it.
def average(numbers):
"""The mean of some numbers; 0.0 for an empty list."""
if not numbers:
return 0.0
return sum(numbers) / len(numbers)
# The bug report was: "the dashboard crashes for new users".
# The smallest failing example was an empty list.
assert average([]) == 0.0 # <- the regression test
assert average([10]) == 10.0
assert average([1, 2, 3, 4]) == 2.5
print("regression test in place")
regression test in place
Common mistake: Testing only the case that obviously works
It is the case you had in mind while writing the function, so it is the one that comes to mind while testing it.
Write the empty case, the single-item case, and both sides of every boundary. That is where the bugs are.
Common mistake: Writing the fix before the test
You can see the fix, and writing a test for a bug you have already fixed feels like going backwards.
A test that has never failed has never been verified. Write it, watch it go red, then fix. It takes an extra minute and proves the test works.
Common mistake: Tests that depend on each other or on run order
Sharing a list or a file between tests avoids repeating the setup.
Each test must set up its own data. Otherwise one test's leftovers make another pass or fail for no reason, and the suite becomes untrustworthy.
Common mistake: Testing against the clock, the network, or real files
The function genuinely uses them, so the test does too.
That is a slow, flaky test that fails on a train. Split the pure decision out of the function and test that; pytest's tmp_path covers the cases that really do need a file.
Common mistake: Chasing 100% coverage
Coverage is a number, and numbers feel like progress.
Coverage measures which lines ran, not whether they were checked for correctness. Ten tests on the code that decides things beat two hundred on getters.
Why write the test before fixing the bug?
A test that has never gone red might assert nothing useful. Watching it fail is the only proof it works.
Which case finds the most bugs for the least effort?
Code is written with “some data” in mind, so zero items is the case most often forgotten.
What does @pytest.mark.parametrize give you over a loop inside one test?
A loop stops at the first failing value; parametrize reports every case independently, so you see how many boundaries broke.
Your test needs the current date and it fails at midnight. What is the fix?
Taking the date as a parameter makes the function both testable and more flexible. Hidden dependencies on the clock are the classic source of flaky tests.
Mini exercise (medium)
A shopping-basket discount rule with a real bug. discount(total) should give 10% off from $50 and 20% off from $100, but it gets the boundaries wrong. Write the six checks that pin the rule down, including both sides of each boundary, then fix the function so every check passes.
Try it yourself. Complete the snippet and hit Run; it executes in your browser, no setup required.
def discount(total):
"""10% off from 50, 20% off from 100."""
if total > 50:
return total * 0.9
if total > 100:
return total * 0.8
return total
# TODO: six (total, expected) pairs covering both sides of BOTH boundaries.
# Run them first and watch which ones fail, then fix discount().
CHECKS = []
for total, expected in CHECKS:
got = round(discount(total), 2)
print(f"{total:6.2f} -> {got:6.2f} (expected {expected:6.2f})")
assert got == expected, f"{total} should come to {expected}"
print(f"{len(CHECKS)} checks passed")The boundaries are 50 and 100, so test 49.99, 50, 99.99 and 100, plus a small total and a zero. “From $50” means >=, not >, and the 20% rule must be checked before the 10% one or every large total stops at 10%.
def discount(total):
"""10% off from 50, 20% off from 100."""
if total >= 100:
return total * 0.8
if total >= 50:
return total * 0.9
return total
CHECKS = [
(0, 0.00),
(49.99, 49.99),
(50, 45.00),
(99.99, 89.99),
(100, 80.00),
(150, 120.00),
]
for total, expected in CHECKS:
got = round(discount(total), 2)
print(f"{total:6.2f} -> {got:6.2f} (expected {expected:6.2f})")
assert got == expected, f"{total} should come to {expected}"
print(f"{len(CHECKS)} checks passed")