Python terraform state JSON census — ephemeral and write-only secret absence

Plans can hide secrets. State must prove they never landed.

<!-- hal:authoritative:yaml -->

Plans can hide secrets. State must prove they never landed. Walk the JSON. Assert absence. Exit zero when the ledger is clean.

§I — Frame

09-14 counted import, move, and no-op actions on a saved plan. 09-05 counted create/update/delete/replace risk. Keep those roles.

Ops today generates a password, writes it through value_wo, and feeds MySQL through administrator_password_wo. The PR sentence you need is different: after apply, does state JSON contain the password string, and do write-only attribute slots stay null or missing?

That sentence is today's census. Call terraform show -json (state) and optionally terraform show -json tfplan. Parse. Search. Print a markdown report. Exit zero unless Terraform failed or a forbidden secret substring appears where it should not.

§II — Language idiom: subprocess plus defensive JSON

Python for DevOps spends subprocess.run as the standard-library way to call CLI tools and capture stdout.

import json
import subprocess
import sys
from pathlib import Path

FORBIDDEN_MARKERS = ("administrator_password", "value_wo", "password_wo")


def show_json(target: str | None = None) -> dict:
    cmd = ["terraform", "show", "-json"]
    if target:
        cmd.append(target)
    proc = subprocess.run(cmd, check=False, capture_output=True, text=True)
    if proc.returncode != 0:
        sys.stderr.write(proc.stderr)
        sys.exit(proc.returncode or 1)
    return json.loads(proc.stdout)


def walk(obj, path="$"):
    if isinstance(obj, dict):
        for k, v in obj.items():
            yield from walk(v, f"{path}.{k}")
    elif isinstance(obj, list):
        for i, v in enumerate(obj):
            yield from walk(v, f"{path}[{i}]")
    else:
        yield path, obj

Treat schema defensively. Terraform version skew changes exact keys. Prefer scanning string leaves and known attribute names over hard-coding one provider schema forever.

§III — Absence checks that match Ops

1. No ephemeral instances in state resources. Ephemeral resources should not appear as ordinary managed entries in values.root_module.resources the way azurerm_key_vault_secret does.
2. Write-only attributes null or absent. For resources that used value_wo / administrator_password_wo, the corresponding sensitive plaintext must not appear as a string leaf under values / attributes.
3. Version companions may exist. value_wo_version and administrator_password_wo_version are ordinary tracked numbers. Seeing them is success, not leakage.
4. Optional plan scan. On a plan file, confirm write-only values are not echoed as plaintext changes. Prefer asserting absence over printing redacted blobs into CI logs.

def census(state: dict, banned_substrings: list[str]) -> dict:
    hits = []
    for path, val in walk(state):
        if not isinstance(val, str):
            continue
        for needle in banned_substrings:
            if needle and needle in val:
                hits.append({"path": path, "needle": needle})
    resources = (
        state.get("values", {})
        .get("root_module", {})
        .get("resources", [])
    )
    types = sorted({r.get("type", "") for r in resources})
    return {
        "resource_types": types,
        "secret_substring_hits": hits,
        "absence_ok": len(hits) == 0,
    }

Wire banned_substrings from a CI secret the pipeline already knows (the password just applied) or from a test fixture in Maghrib lab time. Never commit the live password into the census script.

§III.b — Full script skeleton

def main() -> None:
    banned = [s for s in Path("banned.txt").read_text().splitlines() if s.strip()]
    state = show_json()  # current state
    report = census(state, banned)
    print("# Secret-absence census")
    print(f"- Resource types: {', '.join(report['resource_types']) or '(none)'}")
    print(f"- Secret substring hits: {len(report['secret_substring_hits'])}")
    for hit in report["secret_substring_hits"][:20]:
        print(f"  - {hit['path']} matched {hit['needle']!r}")
    print(f"- Absence verdict: {'yes' if report['absence_ok'] else 'no'}")
    if not report["absence_ok"]:
        sys.exit(2)

if __name__ == "__main__":
    main()

Keep banned.txt out of git. Generate it in CI from the same secret store Ops just wrote, or from a lab fixture password known only to the runner.

When Maghrib points at Lab 24, run the sensitive lab first so you feel redaction, then apply the Ops write-only stack and run this census so you feel absence. Do not merge the two success criteria into one script flag.

§IV — Report shape

Print markdown a reviewer can skim:

# Secret-absence census
- Resource types: azurerm_key_vault, azurerm_key_vault_secret, azurerm_mysql_flexible_server, ...
- Write-only version attrs observed: yes/no
- Secret substring hits: 0
- Absence verdict: yes

Exit zero on clean absence. Exit non-zero only when Terraform fails or hits are found. Do not turn this reporter into the 08-09 policy gate unless Maghrib asks for a hard fail mode.

§V — Contrast with Lab 24

Lab 24 grades sensitive marking and redaction. A sensitive value can still serialize into state. Today's census fails closed if the password string is present at all under state JSON leaves. Different finish lines. Run Lab 24 to feel redaction. Run Ops apply plus this census to feel absence.

§VI — Close

Ops erases on purpose. Python proves the erase. Pair the census with the Azure Key Vault / MySQL write-only apply. No Go lesson in this re-author. Maghrib quiz will fold any Go-flavored absence questions into Dev if needed.

Re-authored on the lab Mac SoT 2026-09-18 after vault mirror clobber of Bot-only #123.