Packaging: Turn Your Script Into an Installable Tool

In this lesson

You have a script that works. To use it you cd to the right folder and type python tidyup.py, and if you send it to a colleague they have to be told which packages to install first.

Packaging ends both problems. It turns the script into something you install once and then run by name from anywhere, and it is the final topic on the Advanced roadmap.

Explain it like I’m 5

Packaging is putting your script in a box with a label saying what it needs, so any computer can unpack it and know how to run it.

Reproducing an environment vs shipping an artifact

Unit 5 covered virtual environments and requirements.txt. It is worth being precise about how that differs from what happens here, because the two get confused constantly.

requirements.txt reproduces an environment. It is a list of what to install so this project runs on another machine. It says nothing about your code: your code is just the files sitting there.

pyproject.toml ships an artifact. It describes your project as an installable thing: its name, its version, what it needs, and what commands it provides. After it exists, pip install works on your code the same way it works on requests.

One is for the people who develop the project. The other is for the people who merely want to use it.

Where the code goes: the src layout

Before the config, the folders. A packaged project puts the importable code inside a src/ directory:

The reason is subtle and worth knowing. Without src/, your project folder is on the path whenever you run Python there, so import tidyup finds the local files whether or not the package installed correctly. Your tests then pass against code that would fail for everyone else. Moving the code into src/ forces the tests to use the installed copy — so what you test is what you ship.

Example

tidyup/
├── pyproject.toml        # the label on the box
├── README.md            # what it does, how to use it
├── src/
│   └── tidyup/          # the importable package
│       ├── __init__.py  # marks it as a package
│       ├── cli.py       # the command-line entry point
│       └── sorter.py    # the actual work
└── tests/
    └── test_sorter.py

The standard layout. Two folders and one config file.

pyproject.toml, line by line

This one file replaces the setup.py, setup.cfg and MANIFEST.in you will see in older projects. It is TOML, a config format designed to be readable: sections in square brackets, key = value underneath.

There are only three parts to understand: which tool builds the package, what the package is, and what commands it provides.

Example · pyproject.toml

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "tidyup"
version = "0.1.0"
description = "Sort a messy downloads folder into dated subfolders."
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "rich>=13.0",
]

[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff"]

[project.scripts]
tidyup = "tidyup.cli:main"

A complete, working pyproject.toml. Every line is doing something.

What this code does

[build-system] names the tool that turns your folder into an installable file. hatchling is a good modern default; setuptools is the older one you will meet in existing projects. You will never call it yourself. pip reads this and installs it behind the scenes.

name is what people type after pip install, and it must be unique on PyPI if you ever publish. version is the number you must remember to raise: pip will not reinstall an unchanged version, which is the single most common packaging frustration.

dependencies is the important difference from requirements.txt. Pip installs these automatically when someone installs your tool, so rich>=13.0 means “anything from 13 upwards”, deliberately loose, because your users have other packages to satisfy too. Pin exactly in an application, keep ranges in a library.

[project.optional-dependencies] holds the things only contributors need. pip install -e ".[dev]" gets them; a normal user never downloads pytest.

[project.scripts] is the line that creates the terminal command, and it gets the whole next section.

Entry points: the line that makes it a command

tidyup = "tidyup.cli:main" reads as: create a command called tidyup that imports the module tidyup.cli and calls the function main. The colon separates the module from the function inside it.

On install, pip writes a tiny launcher script into the environment's bin/ folder. Because that folder is already on your PATH, the command works from any directory — which is the entire difference between a script and a tool.

The function it points at takes no arguments and returns nothing. It reads sys.argv itself, usually via argparse from Unit 5.

Example · src/tidyup/cli.py

# src/tidyup/cli.py - the function the entry point names.
import argparse


def main():
    """The tidyup command. Reads arguments, does the work, prints a summary."""
    parser = argparse.ArgumentParser(prog="tidyup",
                                     description="Sort a folder into dated subfolders.")
    parser.add_argument("folder", help="the folder to tidy")
    parser.add_argument("--dry-run", action="store_true",
                        help="show what would happen, change nothing")
    args = parser.parse_args(["~/Downloads", "--dry-run"])   # normally: parse_args()

    print(f"folder:  {args.folder}")
    print(f"dry run: {args.dry_run}")


if __name__ == "__main__":
    main()

Output

folder:  ~/Downloads
dry run: True

A normal argparse function: the entry point just names it.

Read a pyproject.toml the way pip does. tomllib is in the standard library, so parsing TOML needs no install. Pull out the name, version, dependency count and command, then split the entry point "tidyup.cli:main" into the module and the function it names.

import tomllib

PYPROJECT = """
[project]
name = "tidyup"
version = "0.3.1"
dependencies = ["rich>=13.0", "click"]

[project.scripts]
tidyup = "tidyup.cli:main"
"""

data = tomllib.loads(PYPROJECT)

name = ""          # TODO
version = ""       # TODO
deps = 0           # TODO: how many dependencies
command = ""       # TODO: the name of the command it installs
target = ""        # TODO: what that command points at
module, function = "", ""   # TODO: split target on the colon

print(f"{name} {version}")
print(f"dependencies: {deps}")
print(f"command: {command} -> import {module}, call {function}()")

import tomllib

PYPROJECT = """
[project]
name = "tidyup"
version = "0.3.1"
dependencies = ["rich>=13.0", "click"]

[project.scripts]
tidyup = "tidyup.cli:main"
"""

data = tomllib.loads(PYPROJECT)

name = data["project"]["name"]
version = data["project"]["version"]
deps = len(data["project"]["dependencies"])
command = list(data["project"]["scripts"])[0]
target = data["project"]["scripts"][command]
module, function = target.split(":")

print(f"{name} {version}")
print(f"dependencies: {deps}")
print(f"command: {command} -> import {module}, call {function}()")

Installing your own project, and building the box

With the file in place, install it into your virtual environment in editable mode:

pip install -e . — the . means “the project in this folder” and -e means editable: pip links to your source rather than copying it, so edits take effect immediately with no reinstall. This is how you work on a package day to day.

To give it to someone else, build a wheel: a .whl file, which is a zip of your package plus its metadata. It is the format pip installs, and building one takes a single command.

Example

$ pip install -e .
Obtaining file:///home/ada/tidyup
  Installing build dependencies ... done
Successfully installed tidyup-0.1.0

$ tidyup ~/Downloads --dry-run     # the command now exists, from anywhere
folder:  /home/ada/Downloads
dry run: True

$ pip install build
$ python -m build
Successfully built tidyup-0.1.0.tar.gz and tidyup-0.1.0-py3-none-any.whl

$ ls dist/
tidyup-0.1.0-py3-none-any.whl  tidyup-0.1.0.tar.gz

Install it, run it by name, then build the file you can hand over.

PyPI, and why you probably do not need it

PyPI is the public index pip install reads from. Publishing there means pip install twine, an account, an API token, and twine upload dist/*. It is genuinely that short.

The honest advice is that most projects should not. Publishing makes you responsible for a name forever, for anyone who comes to depend on it, and for the security of the account that can push updates. Meanwhile the alternatives cover nearly every real need:

Hand over the wheel — pip install ./tidyup-0.1.0-py3-none-any.whl. Works offline, no account.
Install from Git — pip install git+https://github.com/you/tidyup. Works with private repositories too.
Install from a folder — pip install . on a shared drive.

Publish when strangers should be able to find and install it by name. Until then, packaging has already given you everything useful.

Common mistake: Forgetting to raise the version number

Why it happens:

The code changed, so it feels like the package changed.

How to fix it:

Pip skips a reinstall when the version matches what is already there, so your fix silently does not arrive. Raise version in pyproject.toml with every release.

Common mistake: A dependency that works only because it is installed globally

Why it happens:

You installed it once months ago, so your imports work and it never occurs to you to list it.

How to fix it:

Create a fresh virtual environment, install only your package, and run it. Every ModuleNotFoundError is a line missing from dependencies.

Common mistake: Choosing a name already taken on PyPI

Why it happens:

The obvious name for a tool is obvious to everyone.

How to fix it:

Search PyPI before naming the project. Changing it later means changing the folder, the imports, the entry point and every document.

Common mistake: Expecting requirements.txt to make the package installable

Why it happens:

It lists the dependencies, so it looks like it is doing the same job.

How to fix it:

It only reproduces an environment. Without pyproject.toml there is no package, no version, and no command: pip has nothing to install.

Common mistake: Putting the package at the top level instead of in src/

Why it happens:

It is one less folder and everything appears to work.

How to fix it:

It works because the current directory is on the path, so your tests never exercise the installed copy. A missing file then only shows up for your users. The src/ layout makes that impossible.

What does [project.scripts] do?

Why install your own project with pip install -e . ?

What is the difference between requirements.txt and pyproject.toml?

What is a wheel?

Mini exercise (hard)

Write the check that saves a release. problems(data) takes a parsed pyproject.toml and returns a sorted list of what is wrong with it: a missing name, a missing version, an entry point with no colon in it, and an entry point whose module does not start with the package name. Return an empty list when the file is fine.

Try it yourself. Complete the snippet and hit Run; it executes in your browser, no setup required.

import tomllib

GOOD = """
[project]
name = "tidyup"
version = "0.3.1"
[project.scripts]
tidyup = "tidyup.cli:main"
"""

BROKEN = """
[project]
name = "tidyup"
[project.scripts]
tidyup = "tidyup.cli.main"
sortit = "helpers.tools:run"
"""

def problems(data):
    """Everything wrong with a parsed pyproject.toml, sorted."""
    found = []
    # TODO: "missing name" / "missing version"
    # TODO: for each entry point, no colon -> f"{command}: entry point needs module:function"
    # TODO: module outside the package -> f"{command}: {module} is not inside {package}"
    return sorted(found)

print("good:  ", problems(tomllib.loads(GOOD)) or "ok")
for problem in problems(tomllib.loads(BROKEN)):
    print("broken:", problem)
print("empty: ", problems({}))

import tomllib

GOOD = """
[project]
name = "tidyup"
version = "0.3.1"
[project.scripts]
tidyup = "tidyup.cli:main"
"""

BROKEN = """
[project]
name = "tidyup"
[project.scripts]
tidyup = "tidyup.cli.main"
sortit = "helpers.tools:run"
"""

def problems(data):
    """Everything wrong with a parsed pyproject.toml, sorted."""
    project = data.get("project", {})
    package = project.get("name", "")
    found = []

    if not package:
        found.append("missing name")
    if not project.get("version"):
        found.append("missing version")

    for command, target in project.get("scripts", {}).items():
        if ":" not in target:
            found.append(f"{command}: entry point needs module:function")
            continue
        module = target.split(":", 1)[0]
        if module.split(".")[0] != package:
            found.append(f"{command}: {module} is not inside {package}")

    return sorted(found)

print("good:  ", problems(tomllib.loads(GOOD)) or "ok")
for problem in problems(tomllib.loads(BROKEN)):
    print("broken:", problem)
print("empty: ", problems({}))

What to learn next

You closed the roadmap. You know why the code goes in src/, what every line of a pyproject.toml is doing, how an entry point turns tidyup.cli:main into a command on your PATH, what pip install -e . gives you while developing, and how to build a wheel you can simply hand to someone. You also know why most projects should never touch PyPI.

Shipping raises a new question: does it still build and pass on a machine that is not yours? Continuous Integration with GitHub Actions answers it automatically on every push. To do it end to end on a real tool, Package and Ship a Command-Line Tool walks the whole path from one loose script to an installable wheel.