Python Modules and Packages for Ops Fleets — the reuse boundary

A useful script in ~/bin is a private habit. A package on PyPI is a shared library.

The package is the boundary that turns useful code into shared code. Cross it once; the fleet compounds.

§ IFrame

Every Python fleet starts the same way. Someone writes a useful script, drops it in ~/bin, and shares it around. Six months later there are twenty scripts, each in a different developer's home directory, each with its own copy of the same YAML parser and its own copy of the same retry decorator, and no one is sure whether the version of deploy.py on the ops-lead's laptop matches the version on the CI runner. This is the point where a fleet needs its own Python package.

The package is the boundary that turns useful code into shared code. When retry, YAML loading, service-registry lookup, and every other cross-cutting piece lives in one importable library, every script the fleet writes gets the improvements automatically and every fix lands in one place. This lesson names how to structure that library correctly, when to reach for a namespace package instead of a regular one, and how to expose scripts through entry points so the fleet's tools live at PATH instead of python -m invocations.

The reason to do this in Ops rather than pure-Dev is that the daily consequence of getting it wrong is a fleet full of divergent copies. The Dev lesson today drills the packaging machinery in depth (pyproject.toml, PEP 621, build backends, wheels). This lesson names the operational shape that machinery produces: a hedronite_ops package on every operator's laptop, versioned, installable via pip install hedronite-ops, extended by every team that needs to add tools to it, exposing CLI tools at the shell where the ops person is already typing.

§ IIFoundations — Three Shapes a Package Takes

Python offers three shapes for organizing importable code, and choosing wrong is where fleets accumulate friction.

A module is a single .py file. retry.py is a module. Import it as import retry. Modules are the atomic unit; a script that does one thing well can stay a module forever. Lutz opens the modules chapter that grounds this lesson exactly here: a module is a namespace, and importing one binds its attributes to a name in the caller's scope.

A regular package is a directory with __init__.py. hedronite_ops/ containing __init__.py, retry.py, registry.py, and secrets.py is a package. Import as import hedronite_ops.retry or from hedronite_ops import retry. The __init__.py runs when the package is first imported and can define the public API by re-exporting names: from .retry import with_backoff in __init__.py lets callers write hedronite_ops.with_backoff without knowing which submodule it lives in. Regular packages are the default choice for any fleet library that lives in one repository owned by one team.

A namespace package spans multiple directories without an __init__.py. Introduced by PEP 420, this shape lets multiple installed packages contribute to the same top-level namespace. If hedronite-ops-core installs hedronite_ops/retry.py and hedronite-ops-billing installs hedronite_ops/billing.py, both are importable as hedronite_ops.retry and hedronite_ops.billing, and neither package needs to know about the other. Reach for namespace packages when multiple teams need to extend one fleet-wide vocabulary without coordinating a single repository. Reach for regular packages otherwise, because namespace packages hide subtle bugs that regular packages catch (a typo'd import in a namespace package silently falls through to whatever else is on sys.path).

§ IIIMechanism — Import Path, Package Discovery, and Entry Points

Three surfaces sit between "someone typed import" and "code from another file runs." Read them in the order Python does.

sys.path is the search order. When Python encounters import hedronite_ops, it walks sys.path in order and returns the first match. sys.path is populated at interpreter start from the current directory, PYTHONPATH environment variable, and the interpreter's own site-packages directory. The exam and daily debugging both benefit from knowing this: python -c "import sys; print(sys.path)" shows you exactly where imports resolve, and an unexpected import is almost always a directory earlier in sys.path shadowing the intended one.

The finder picks a match; the loader materializes it. For a regular package, Python looks for a directory with __init__.py at each sys.path entry. For a namespace package, Python looks for a directory without __init__.py at each entry and merges every match into one virtual package. For a module, Python looks for a .py file. The finder returns the first match by kind (packages before modules of the same name, typically). This ordering is what makes namespace packages work: multiple entries on sys.path each contribute part of the namespace.

Entry points are how packages register tools with the outside world. In pyproject.toml, a package declares scripts under [project.scripts]:

[project.scripts]
hedronite-sweep = "hedronite_ops.sweep:main"
hedronite-registry = "hedronite_ops.registry:cli"

At pip install time, the packaging system creates executable shims at the interpreter's bin directory (hedronite-sweep, hedronite-registry) that, when invoked, import the named module and call the named function. The fleet operator types hedronite-sweep at the shell and Python runs hedronite_ops.sweep.main(). No python -m prefix, no PATH fiddling.

Beyond CLI scripts, entry points also power plugin discovery. A framework declares an entry-point group (say, hedronite_ops.plugins) and reads entries other packages register under it. Every installed package in the environment that registered under that group becomes discoverable without the framework knowing about it in advance. This is how pytest, black, and dozens of other tools accept third-party extensions.

§ IVWorked Example — hedronite_ops as a Fleet Library

The specimen is a regular package structured for a small ops team, with two CLI tools exposed and one plugin surface open for extension.

Directory layout:

hedronite-ops/
├── pyproject.toml
├── src/
│   └── hedronite_ops/
│       ├── __init__.py
│       ├── retry.py
│       ├── registry.py
│       ├── sweep.py
│       └── plugins.py
└── tests/
    └── test_retry.py

The src/ layout is the recommended shape because it forces every test to install the package first, catching the "imports work locally but not in CI" class of bug that flat layouts hide. pyproject.toml declares the package and its two entry points; the Dev lesson today walks the full contents.

__init__.py defines the public API:

"""Hedronite ops library — shared fleet tools."""
from .retry import with_backoff
from .registry import ServiceRegistry
from .plugins import discover

__version__ = "0.4.2"
__all__ = ["with_backoff", "ServiceRegistry", "discover"]

Callers write from hedronite_ops import with_backoff without knowing that with_backoff lives in retry.py. If the retry implementation moves to a different module tomorrow, the caller's import is unchanged. This is the whole point of the public-API pattern: __init__.py is a stable façade over an internal structure that can move.

sweep.py provides one of the two CLI tools:

import argparse
from .retry import with_backoff
from .registry import ServiceRegistry

@with_backoff(retries=3, factor=2.0)
def probe_one(registry, host):
    return registry.probe(host)

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--namespace", required=True)
    parser.add_argument("--service", required=True)
    args = parser.parse_args()
    reg = ServiceRegistry(args.namespace, args.service)
    for host in reg.snapshot():
        print(probe_one(reg, host))

After pip install -e . in the repo, hedronite-sweep --namespace prod --service api runs hedronite_ops.sweep.main() at the shell. The retry decorator applied to probe_one is the fleet's shared retry policy, so every tool that reaches for it gets the same behavior, and improving the policy improves every tool.

plugins.py opens a plugin surface for other teams:

from importlib.metadata import entry_points

def discover(group: str = "hedronite_ops.plugins") -> dict:
    return {ep.name: ep.load() for ep in entry_points(group=group)}

Any other package in the environment that declares an entry point under hedronite_ops.plugins becomes discoverable via discover(). This is how the billing team can ship hedronite-billing-ops as a separate package that plugs into hedronite-ops without either team's repo needing to know about the other's contents. Loose coupling; strict contract; the shape every extensible ops tool uses.

§ VConnection to Prior Lessons

Two weeks ago the context-manager lesson taught that a with-block is a scope with an exit guarantee. This lesson names where those with-blocks belong: inside a package's public API, imported once and reused by every tool the fleet writes. Yesterday's Kubernetes lesson wrote a ServiceRegistry as one class; today it lives inside hedronite_ops.registry and every script that needs endpoint discovery imports it, so a bug fix in the registry propagates automatically. Sunday's concurrent-sweep lesson likewise belongs in hedronite_ops.sweep, not in every operator's ~/bin. The reuse boundary is what turns individually-good code into a compounding library.

§ VIConnection to Today's Dev and Cert Lessons

The Dev lesson today goes into pyproject.toml, PEP 621, build backends (setuptools, hatchling, poetry-core, flit), and the difference between wheels and sdists. Where today's Ops lesson names the shape a fleet library should have, the Dev lesson names the file that describes it to Python's packaging system. The Cert lesson lifts the same surface into the PCAP Modules and Packages objective, where the exam tests import semantics, __init__.py, the difference between regular and namespace packages, and the standard library layout. The three lessons interlock: the shape, the metadata that describes the shape, and the exam-tier vocabulary for both.

§ VIIClosing

A module is a file. A regular package is a directory with __init__.py. A namespace package is a directory without one that multiple installed packages can extend. Entry points let a package register CLI tools and plugin extensions with the outside world so the fleet's shared library becomes shell-usable and third-party-extensible. The failure mode is never Python's import system. It is twenty divergent copies of the same script in twenty developers' home directories because nobody owned the reuse boundary.

Pick a script your team runs. Move it into a package with a real pyproject.toml, define one entry point for it, and run pip install -e .. When the shell finds the tool at PATH instead of a filesystem hunt, the boundary this lesson names is the boundary you have just crossed.