Search and Summarise a Log File

Turn a 9,000-line log into a three-line summary: scan it line by line, keep only the errors, count them by type, and write a report — the everyday shape of an ops or security script.

Intermediate 35–50 minutes

About this project

Log files are where programs write down what happened, one line per event, often thousands of lines a day. When something breaks, nobody scrolls through by hand. They run a small script that finds the important lines and summarises them. That’s exactly what you’ll build here, on a realistic sample log.

Why it’s worth building: ‘read a big text file, keep the lines that matter, count them, write a report’ is one of the most reusable patterns in practical Python. It’s the backbone of error triage, security log review, and quick data spelunking, and it never loads the whole file into memory, so it works on files far too big to open in an editor.

The whole job: scan a big file line by line, filter, count, and report.

Build it step by step

We’ll build it the way you’d build the real thing: get the matching lines first, then count them, then wrap it in a friendly command-line tool. Each step runs on its own.

The sample log

Save this as app.log next to your script (or download it below). It mixes routine INFO lines with the WARNING and ERROR lines we care about, just like a real one, only shorter.

Example · app.log 
2026-07-01 09:00:01 INFO started up
2026-07-01 09:00:04 ERROR db timeout
2026-07-01 09:00:05 INFO request ok
2026-07-01 09:00:09 WARNING slow query
2026-07-01 09:00:12 ERROR null response
2026-07-01 09:00:15 INFO request ok
2026-07-01 09:00:19 ERROR db timeout
2026-07-01 09:00:21 INFO request ok
Eight lines standing in for thousands: three of them are errors.

Step 1: Read only the lines that matter

The key move is to walk the file one line at a time rather than loading it whole, a generator function does this cleanly and uses almost no memory no matter how big the file is. We keep a line when it contains the text "ERROR".

Example 
from pathlib import Path


def error_lines(path):
    """Yield each line in the log that reports an ERROR."""
    for line in Path(path).read_text(encoding="utf-8").splitlines():
        if "ERROR" in line:
            yield line


for line in error_lines("app.log"):
    print(line)
Output 
2026-07-01 09:00:04 ERROR db timeout
2026-07-01 09:00:12 ERROR null response
2026-07-01 09:00:19 ERROR db timeout
Three lines survive the filter.

Step 2: Count the errors by type

Three errors, but which ones? The interesting number is how many of each kind. We take the message (everything after the word ERROR) and let collections.Counter tally them. It’s a dictionary built for exactly this.

Example 
from collections import Counter

lines = [
    "09:00:04 ERROR db timeout",
    "09:00:12 ERROR null response",
    "09:00:19 ERROR db timeout",
]

messages = [line.split("ERROR", 1)[1].strip() for line in lines]
counts = Counter(messages)
print(counts.most_common())
Output 
[('db timeout', 2), ('null response', 1)]
Counter turns a list of messages into ranked totals in one line.

Step 3: Make it a tool you can point at any log

Finally we wrap it up: take the log’s path as a command-line argument, write the matching lines to errors.txt for the record, print the summary, and fail politely if the file is missing.

Example 
import sys
from pathlib import Path

# (error_lines and Counter from the steps above)

if len(sys.argv) < 2:
    print("Usage: python summarize_log.py <logfile>")
    raise SystemExit(1)

path = Path(sys.argv[1])
sys.argv[1] is the first thing typed after the script name.

The finished script

The whole tool: scan → filter → count → report, wrapped in functions and driven by a command-line argument. Run it as python summarize_log.py app.log.

Example · summarize_log.py 
from collections import Counter
from pathlib import Path
import sys


def error_lines(path):
    """Yield each line in the log that reports an ERROR."""
    for line in Path(path).read_text(encoding="utf-8").splitlines():
        if "ERROR" in line:
            yield line


def summarize(path):
    lines = list(error_lines(path))
    # The message is everything after the word ERROR.
    messages = [line.split("ERROR", 1)[1].strip() for line in lines]
    return lines, Counter(messages)


def main():
    if len(sys.argv) < 2:
        print("Usage: python summarize_log.py <logfile>")
        raise SystemExit(1)

    path = Path(sys.argv[1])
    try:
        lines, counts = summarize(path)
    except FileNotFoundError:
        print(f"No such file: {path}")
        raise SystemExit(1)

    Path("errors.txt").write_text("\n".join(lines), encoding="utf-8")

    print(f"{len(lines)} errors found")
    for message, number in counts.most_common():
        print(f"{number}x {message}")


if __name__ == "__main__":
    main()
Output 
$ python summarize_log.py app.log
3 errors found
2x db timeout
1x null response
It also drops the raw error lines into errors.txt for the record.

Keep going, make it your own

The core is solid. Each upgrade below mirrors something a real log tool does.

Count warnings and errors separately. Add a second filter (or check the severity word on each line) so your report shows both WARNING and ERROR totals. Most triage starts by comparing the two.

Example 
from collections import Counter

severities = Counter()
for line in Path("app.log").read_text().splitlines():
    for level in ("ERROR", "WARNING"):
        if level in line:
            severities[level] += 1
print(severities)

Match case-insensitively. Some logs write error in lower case. Compare line.upper() against "ERROR" so nothing slips through.

Example 
if "ERROR" in line.upper():
    ...

Extract just the timestamp of each error. Slice the front of the line (or split on the first space) to build a timeline of when things went wrong, the first question anyone asks.

Example 
stamp = line.split(" INFO")[0]  # crude but effective for a fixed format

Download the files

Download the sample log and the finished script, then run <code>python summarize_log.py app.log</code> and match the summary above.

Mini exercise (medium)

Write the heart of the tool: count_errors(lines) that takes a list of log lines and returns a Counter of the error messages (the text after ERROR), ignoring any line without an error.

Your turn. Fill in the code below and press Run to test it right here, nothing to install.

from collections import Counter


def count_errors(lines):
    # keep ERROR lines, take the message after "ERROR", count them
    return Counter()


sample = ["INFO ok", "ERROR db timeout", "ERROR db timeout", "ERROR null response"]
print(count_errors(sample).most_common())

Where to go next

This ‘read → filter → count → report’ shape reappears constantly. The CSV Cleaner applies it to structured data, and argparse upgrades the command line with proper flags like --level ERROR.