Python for the Terraform Cloud API — workspace, run, state

The CLI is one client. Python is another. Anything the browser can do, a script can do at fleet scale.

A workspace is a Python object. A run is a Python future. A state version is a JSON document you fetch, parse, and route.

§ IFrame

The Terraform CLI is one client of the HCP Terraform (formerly Terraform Cloud) API. The API is what the CLI is calling under the covers when it kicks off a run, uploads a plan for review, and downloads the last state file. Anything the CLI does through the browser or over terraform apply is a REST call, and anything a REST call can do, a Python script can do too, which means the fleet's Terraform operations become programmable at the layer above a single apply. A workspace becomes a Python object; a run becomes a Python future; a state version becomes a JSON document you fetch, parse, and route.

The reason to reach for the API is that a fleet running dozens or hundreds of workspaces cannot be operated by hand. When a new region rolls out, thirty workspaces need their AWS_DEFAULT_REGION variable set. When a compliance sweep asks how many workspaces are running an out-of-support Terraform version, no one is going to click through the console. This lesson builds the shape of a Python client against three HCP Terraform surfaces most fleets touch: workspaces, runs, and state versions. The client goes through terrasnek, the maintained Python SDK for the HCP Terraform API, because a raw requests-plus-signed-URL loop is the wrong altitude for daily work when the SDK already knows the endpoint layout, the pagination rules, and the token-auth handshake.

§ IILanguage Idiom

The API is a paginated REST surface with JSON:API-shaped responses. Three idioms in the Python translation are worth naming before any code lands.

Token auth as a first-class dependency. The client authenticates with a personal or team token passed as an Authorization: Bearer header. In Python that becomes a constructor argument, and the constructor is where the client's whole surface is bound. Load the token from an environment variable, never a literal, and if the environment variable is missing, raise immediately with a message naming the variable so the caller can fix it in one line rather than debugging a 401 five stack frames deep.

Pagination as an iterator, not a page counter. The API returns twenty items per page by default and every list endpoint accepts page[number] and page[size] query parameters. The terrasnek SDK exposes both a list call and a list_all call; the latter walks pagination internally and returns the concatenated set. Prefer list_all for correctness, and reach for the paged form only when the collection is large enough that memory or rate limit becomes a real concern. When you do page manually, wrap the loop in a generator function so the caller consumes items one at a time and the rate-limit checks live in one place.

Every mutation returns a resource, not a status. POST, PATCH, and DELETE calls that the API accepts return the updated resource in the response body, or a 204 with no body for deletes. This is convenient because the returned object carries the server-authoritative fields, including the numeric identifiers your code needs for follow-up calls. Assign the response, never discard it. ws = api.workspaces.create({...}) binds the new workspace's ID to ws["data"]["id"] for the next call in the same transaction.

§ IIICode Worked Example

The specimen is a three-step operation the fleet runs weekly: enumerate workspaces in an organization, upsert a AWS_DEFAULT_REGION variable in each one, and read the latest state version's serial number so a report can flag any workspace whose state has not moved in more than thirty days.

First, the client. One place for the token, one place for the base URL, no scattered secrets.

import os
from terrasnek.api import TFC

def make_client(org: str) -> TFC:
    token = os.environ.get("TFC_TOKEN")
    if not token:
        raise RuntimeError("TFC_TOKEN is not set; export it before running")
    api = TFC(token, url="https://app.terraform.io")
    api.set_org(org)
    return api

The set_org call scopes every subsequent workspace, run, and variable call to one organization, so downstream code stops passing the org name explicitly.

Second, enumerate. list_all handles pagination internally, so the caller gets one flat list and never sees a page number.

def all_workspaces(api: TFC) -> list[dict]:
    resp = api.workspaces.list_all()
    return resp["data"]

Twenty or two thousand workspaces, the call shape is the same. If the organization is large enough that holding every workspace in memory is a real concern, api.workspaces.list(page_size=100, page_number=n) gives you the loop back and lets you process each page and drop it.

Third, upsert a workspace variable. The HCP API does not offer PUT-with-upsert, so the fleet-side pattern is find-then-create-or-update, done under a scope that treats the pair as one operation:

def upsert_var(api: TFC, workspace_id: str, key: str, value: str) -> dict:
    existing = api.workspace_vars.list(workspace_id=workspace_id)
    for var in existing["data"]:
        if var["attributes"]["key"] == key:
            payload = {"data": {"type": "vars", "attributes": {"value": value}}}
            return api.workspace_vars.update(workspace_id, var["id"], payload)
    payload = {
        "data": {
            "type": "vars",
            "attributes": {
                "key": key,
                "value": value,
                "category": "env",
                "sensitive": False,
            },
        }
    }
    return api.workspace_vars.create(workspace_id, payload)

Two things are load-carrying here. The category must be "env" for environment variables that Terraform sees as AWS_DEFAULT_REGION, versus "terraform" for HCL input variables the module reads through var.region; a mismatched category writes to the wrong scope silently. And sensitive: True on a variable makes it write-only afterward, which is what you want for tokens and what you do not want for a region name you may later need to audit.

Fourth, the state read. State versions live behind the workspace and carry the same JSON:API shape:

from datetime import datetime, timezone, timedelta

def days_since_last_state(api: TFC, workspace_id: str) -> int:
    latest = api.state_versions.get_current(workspace_id)
    ts = latest["data"]["attributes"]["created-at"]
    created = datetime.fromisoformat(ts.replace("Z", "+00:00"))
    return (datetime.now(timezone.utc) - created).days

get_current returns metadata about the latest state version. The state file itself is behind a separate download URL if you need to parse it, but for a staleness report the metadata is enough. Days since last state is the operational signal: a workspace whose state has not moved in ninety days is either finished work no one wrote down, or drifted infrastructure no one is planning against.

The three functions compose into the weekly sweep:

def sweep(org: str, region: str, stale_days: int = 30) -> list[dict]:
    api = make_client(org)
    stale = []
    for ws in all_workspaces(api):
        ws_id = ws["id"]
        upsert_var(api, ws_id, "AWS_DEFAULT_REGION", region)
        days = days_since_last_state(api, ws_id)
        if days > stale_days:
            stale.append({"workspace": ws["attributes"]["name"], "days": days})
    return stale

The sweep is deliberately sequential, not concurrent. Yesterday's Python lesson taught the fan-out budget, the deadline, and the ledger; a fleet sweep against a rate-limited API is exactly the case for that discipline. When this script grows to two hundred workspaces, the loop becomes an asyncio.gather under a Semaphore(10), each upsert_var becomes an async coroutine, and each row of the returned list becomes a ledger entry, exactly as yesterday walked. The single-threaded version is where you start; the concurrent version is where you land once the workspace count crosses the threshold where serial takes minutes.

§ IVConnection to Today's Ops Lesson

The Ops lesson stopped at what a provider block looks like in one module. This lesson runs the layer where thirty modules share the same provider posture. A workspace variable of category env is how a provider block's region = var.region becomes a per-environment value the CLI never has to remember. The lockfile the Ops lesson named lives in the workspace's configuration; HCP Terraform reads it on every run and installs the exact provider version the team committed, so the version pin the Ops lesson wrote and the sweep this lesson runs are the two ends of the same pipe. Take away either end and the promise breaks. Take away both and no one knows what version of the AWS provider is running against production.

§ VPrior-Lesson Reach

Last TF day's Dev lesson tested a module's plan through Python-driven pytest against a local backend. This lesson is the same posture at the fleet altitude: what pytest asserts about one module's plan, the API sweep observes across every workspace's actual state. Yesterday's structured-concurrency lesson supplies the async form the sweep grows into. And the context-manager lesson from two weeks ago is the model for how the token-loading client should really look: with make_client(org) as api: if the SDK grew a proper cleanup hook, though terrasnek today leaves session close to the caller and the try/finally on the requests.Session underneath is a small addition worth writing where the sweep runs long enough to matter.

§ VIClosing

The HCP Terraform API is a REST surface with a shape that maps one-to-one to Python: a client, a set of resource calls, and JSON:API-shaped responses. Reach for it when the fleet is too large to click through, when a policy sweep needs mechanical enumeration, and when a report on state staleness must run the same way every week and diff cleanly. The failure mode is never the API. It is a token loaded from a hard-coded literal, a sensitive: True on a field that later needs to be read, or a serial sweep against ten thousand workspaces because the concurrency lesson from the day before was not carried across.

Pick one manual HCP Terraform operation your team does with the browser. Write the Python equivalent. Set the token from an environment variable, use list_all for enumeration, and end the script with a printed count. When you commit it, you have replaced one recurring human task with one file the next teammate can extend.