Regular Expressions: Finding Patterns in Text
Describe what you are looking for, and let Python find it
In this lesson
Half of real data arrives as messy text: log files, scraped pages, exports from something old. A regular expression is a compact way to describe a pattern so Python can find, extract, or replace every place it occurs. This lesson is the practical subset, plus the equally important matter of when to use something else.
Explain it like I’m 5
A regular expression is a description of what you are looking for, handed to a very literal search dog. It finds exactly what you described, including the things you did not mean to describe.
Three functions cover most of the work
The re module is in the standard library. Three functions do nearly everything:
re.search(pattern, text), find the first match, orNone.re.findall(pattern, text), get every match as a list.re.sub(pattern, replacement, text), replace every match.
Always write patterns as raw strings with an r prefix: r"\d+". Without it, Python processes the backslashes before re ever sees them, and you end up debugging the wrong layer.
import re
text = "Order 1043 shipped, order 2287 delayed"
print(re.findall(r"\d+", text))
print(re.search(r"\d+", text).group())
print(re.sub(r"\d+", "####", text))
['1043', '2287'] 1043 Order #### shipped, order #### delayed
The pieces worth memorizing
Regex has a large vocabulary and you need surprisingly little of it. This is most of what you will use:
- Character classes:
\da digit,\wa letter, digit or underscore,\swhitespace. Capitalize to invert:\Dis ‘not a digit’. - Your own class:
[aeiou]any one of those,[A-Z]any capital,[^0-9]anything except a digit. - Quantifiers:
+one or more,*zero or more,?optional,{3}exactly three. - Anchors:
^start of the line,$end of it. - Any character:
.matches anything except a newline.
Build patterns up one piece at a time and check what each addition matches, and what it now wrongly matches. That second question catches most regex bugs.
import re
log = "09:14 ERROR E501 disk full\n09:15 INFO backup ok\n10:02 ERROR E404 missing file"
print(re.findall(r"E\d{3}", log))
print(re.findall(r"^\d{2}:\d{2}", log, re.MULTILINE))
['E501', 'E404'] ['09:14', '09:15', '10:02']
Capture groups pull out the pieces
Parentheses mark a capture group: a part of the match you want back separately. That turns regex from ‘find the line’ into ‘pull the four fields out of the line’, which is what makes it useful for parsing.
match.group(1) gets the first group, and match.groups() returns them all as a tuple, which lines up neatly with database columns.
import re
line = "2026-07-27 10:02:11 ERROR disk full"
pattern = r"^(\d{4}-\d{2}-\d{2}) (\d{2}:\d{2}:\d{2}) (\w+) (.*)$"
match = re.match(pattern, line)
print(match.groups())
print("level:", match.group(3))
('2026-07-27', '10:02:11', 'ERROR', 'disk full')
level: ERROR
Pull the error codes and timestamps out of the log. Print the list of codes (letter E plus three digits), then the list of times at the start of each line.
import re
log = "09:14 ERROR E501 disk full\n09:15 INFO backup ok\n10:02 ERROR E404 missing file"
codes = [] # TODO: every E followed by three digits
times = [] # TODO: the HH:MM at the start of each line
print(codes)
print(times)E\d{3} for the codes. For the times, anchor with ^ and pass re.MULTILINE so it applies to every line rather than only the first.
import re
log = "09:14 ERROR E501 disk full\n09:15 INFO backup ok\n10:02 ERROR E404 missing file"
codes = re.findall(r"E\d{3}", log)
times = re.findall(r"^\d{2}:\d{2}", log, re.MULTILINE)
print(codes)
print(times)Greediness, and knowing when to stop
Quantifiers are greedy: .* takes as much as it possibly can, then backs off only if the rest of the pattern fails. That is why a pattern meant to match one tag swallows the whole line. Adding ? makes it lazy: .*? takes as little as possible.
And the more important judgment. Regex is the wrong tool for structured formats. JSON, CSV, and HTML each have a proper parser that handles quoting, nesting, and escaping correctly. Reach for regex when the text has no parser, log lines being the classic case. When a plain str.startswith() or .split() would do, use that instead, it is far easier to read six months later.
import re
text = "<b>bold</b> and <i>italic</i>"
print(re.findall(r"<.*>", text)) # greedy: swallows everything
print(re.findall(r"<.*?>", text)) # lazy: one tag at a time
['<b>bold</b> and <i>italic</i>'] ['<b>', '</b>', '<i>', '</i>']
Common mistake: Forgetting the r prefix on a pattern
It looks like an ordinary string, and simple patterns work without it.
Always write r"...". Without it Python interprets backslash sequences first, so re receives something other than what you typed.
Common mistake: Greedy .* swallowing far more than intended
.* reads as ‘some characters’ but means ‘as many as possible’.
Use .*? for the shortest match, or better, be specific: [^>]* says ‘anything that isn’t a closing bracket’ and states the intent.
Common mistake: Parsing HTML, CSV, or JSON with regex
A small pattern works on the sample data you happen to have.
Use the proper parser: json, the csv module, or an HTML library. They handle quoting, escaping, and nesting that no pattern will get right.
Common mistake: Writing one enormous pattern nobody can read
It grows a piece at a time and each addition seems small.
Split the work into steps, name the pattern in a variable near where it is used, and add a comment showing a sample line it matches. Future you will need it.
What does \d{3} match?
\d is a digit and {3} means exactly three of them.
Why write patterns as raw strings, r"\d+"?
Without the r, Python processes escape sequences first and re sees something different from what you wrote.
Which job is regex the wrong tool for?
JSON has a real parser that handles nesting and escaping. Unstructured text like logs is where regex belongs.
Mini exercise (medium)
Write redact(text) that replaces every email address in a string with [redacted] and returns the result, plus count_codes(text) that returns how many error codes (E followed by three digits) appear.
Try it yourself. Complete the snippet and hit Run; it executes in your browser, no setup required.
import re
text = "E501 raised, mail [email protected]; E404 raised, mail [email protected]"
def redact(text):
return text # TODO: replace email addresses with [redacted]
def count_codes(text):
return 0 # TODO: how many E### codes appear
print(redact(text))
print(count_codes(text))A workable email pattern for this exercise is r"[\w.]+@[\w.]+". Use re.sub for the redaction and len(re.findall(...)) for the count.
import re
text = "E501 raised, mail [email protected]; E404 raised, mail [email protected]"
def redact(text):
return re.sub(r"[\w.]+@[\w.]+", "[redacted]", text)
def count_codes(text):
return len(re.findall(r"E\d{3}", text))
print(redact(text))
print(count_codes(text))