A Kubernetes Operator in Python with kopf — the loop you no longer write

Three lessons ago you wrote the watch, the queue, and the loop by hand. Today you keep only the part that was ever yours: the delta.

§ IFrame

The Python-touching-K8s arc has built three machines: the raw controller (LIST then WATCH, work queue, level-triggered reconcile), the admission webhook standing in the request path, and the watch-fed registry keeping a thread-safe snapshot. Each spent most of its lines on connection lifecycles, resync, retry, error routing.

An operator is a controller that manages a domain you defined yourself, expressed as a custom resource, so your intent becomes an API object and your logic becomes the loop that honors it. Newman files operators beside Helm and CRDs as the deployment machinery of mature platforms (Building Microservices, ch. 8, p. 338); Poulton's API chapter explains why the move is cheap: the API is uniform and discoverable, and a CRD extends it (The Kubernetes Book, ch. 14, pp. 201-202). kopf is the Python seat at that table.

§ IIThe Custom Resource — Intent as an API Object

Today's domain is the Ops lesson's allow-list discipline: every namespace that opts in gets the default-deny pair and the DNS door, forever.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: namespacebaselines.hedronite.io
spec:
  group: hedronite.io
  scope: Cluster
  names:
    kind: NamespaceBaseline
    plural: namespacebaselines
    shortNames: [nsb]
  versions:
    - name: v1alpha1
      served: true
      storage: true
      subresources:
        status: {}
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: [namespaceLabel]
              properties:
                namespaceLabel:
                  type: string
                allowDNS:
                  type: boolean
                  default: true
            status:
              type: object
              x-kubernetes-preserve-unknown-fields: true

One instance declares: every namespace labeled baseline=guarded shall carry the wall. The status subresource lets the operator report without bumping metadata.generation. kubectl get nsb now works on a kind that did not exist an hour ago.

§ IIIHandlers — Where kopf Puts Your Code

import kopf
from kubernetes import client

@kopf.on.create("hedronite.io", "v1alpha1", "namespacebaselines")
@kopf.on.update("hedronite.io", "v1alpha1", "namespacebaselines")
@kopf.on.resume("hedronite.io", "v1alpha1", "namespacebaselines")
def reconcile_baseline(spec, name, logger, **_):
    label = spec["namespaceLabel"]
    dns = spec.get("allowDNS", True)
    matched = guarded_namespaces(label)
    applied = 0
    for ns in matched:
        applied += ensure_baseline(ns, dns, logger)
    logger.info(f"baseline held in {len(matched)} namespaces, {applied} objects applied")
    return {"namespaces": len(matched), "objectsApplied": applied}

@kopf.timer("hedronite.io", "v1alpha1", "namespacebaselines", interval=300.0)
def drift_check(spec, logger, **_):
    return reconcile_baseline(spec=spec, name=None, logger=logger)

Three decisions carry the pattern's weight. The same function serves create, update, and resume: resume fires when the operator restarts and finds resources already in place, the 07-23 level-triggered lesson wearing framework clothes. The return value is written into the resource's status under the handler's name, so kubectl get nsb -o yaml becomes the operational dashboard. And the timer is the drift answer: handlers fire on changes to the custom resource, and nothing here fires when someone deletes a NetworkPolicy out of a guarded namespace, so the five-minute timer re-runs the reconcile against reality. Watches for the fast path, timers for the truth.

Failure routing completes the contract: raise kopf.TemporaryError("...", delay=30) and the framework re-queues with backoff; raise kopf.PermanentError and it stops retrying and records the failure in status. Transient and permanent take different exits, legibly.

§ IVThe Reconcile — Idempotent by Construction

NETWORKING = client.NetworkingV1Api()
CORE = client.CoreV1Api()

def guarded_namespaces(label: str) -> list[str]:
    selector = f"{label.split('=')[0]}={label.split('=')[1]}"
    return [ns.metadata.name for ns in CORE.list_namespace(label_selector=selector).items]

def ensure_baseline(ns: str, dns: bool, logger) -> int:
    desired = [deny_all_manifest(ns)]
    if dns:
        desired.append(dns_egress_manifest(ns))
    applied = 0
    for manifest in desired:
        try:
            NETWORKING.read_namespaced_network_policy(manifest["metadata"]["name"], ns)
            NETWORKING.patch_namespaced_network_policy(
                manifest["metadata"]["name"], ns, manifest)
        except client.ApiException as exc:
            if exc.status == 404:
                NETWORKING.create_namespaced_network_policy(ns, manifest)
            elif exc.status in (429, 500, 503):
                raise kopf.TemporaryError(f"API pressure in {ns}", delay=30) from exc
            else:
                raise
        applied += 1
    return applied

deny_all_manifest and dns_egress_manifest return plain dicts holding exactly the two objects from the Ops lesson's §III with the namespace stamped in. The read-patch-create dance is the client-library idiom from K8sUR's language-clients chapter (ch. 13, p. 218); the 429/500/503 branch routes API pressure into kopf's retry machinery, and the bare raise refuses to swallow a real bug. The from exc chain keeps the traceback honest, the 07-24 exception lesson holding rank inside the operator.

Deletion rides ownership: kopf.adopt() can stamp owner references so baseline policies garbage-collect with the NamespaceBaseline. Whether you want that is a policy question, because a dead operator taking the walls down with it is exactly the failure a security baseline should refuse. Leaving policies orphaned on delete is the fail-closed choice, the 07-26 webhook temperament at the gate.

§ VRunning It — RBAC, Liveness, and the Operator's Own Perimeter

The operator runs as a Deployment with a ServiceAccount whose RBAC is its blast radius: list/watch on namespaces, full verbs on networkpolicies, read-write on namespacebaselines and status, events. Nothing else. An operator that can write NetworkPolicies everywhere is itself a target, so it lives in its own guarded namespace behind the very baseline it ships, with one extra egress door to the API server. kopf run baseline_operator.py serves development; in-cluster, the same entry point runs with --liveness=http://0.0.0.0:8080/healthz and the Deployment probes it.

Hand-applied discipline decays at the rate teams onboard. Reconciled discipline decays at the rate someone edits the CRD, in a pull request, with a reviewer watching, and the timer un-does everything else within five minutes.

§ VIConnection to Today's Ops and Cert Lessons

The Ops lesson wrote the wall's grammar and proved it with a probe pod. This lesson made the wall self-installing: intent in a CRD, delta in a decorated function, drift on a timer, failure routed by class. The Cert lesson closes the trio at exam altitude: CKS Cluster Setup, default-deny against the clock, kube-bench auditing the ground. Grammar, installer, audit.

§ VIIClosing

An operator is the controller pattern with your name on the kind. Let kopf own the watch, the queue, the retry, and the restart; keep the CRD schema, the idempotent ensure, and the decision about which failures retry. Report through status. Re-derive on resume. Heal on a timer.

Take any manual runbook you executed twice this month. Write its intent as a CRD schema of five fields or fewer, and its execution as one idempotent function. If you can name the schema and the function, you have an operator; kopf is only the rig that keeps them running.