Rate Limits and Cost Governance in the Claude Agent Harness — Context Management and Agentic Architecture Through the Budget-Governor Lens

The agent harness is a client of a metered upstream. Everything the Ops lesson said about protecting the Upstream Budget, the harness must say to itself.

§ IFrame

Sunday is Anthropic Day, and today the whole trio shares one theme: rate limits, finite budgets, and the honest refusal. The Ops lesson protected an Upstream Budget with an edge limiter. The Dev lesson made that limiter true across a fleet. This lesson turns the same eye inward, onto the agent harness, because the harness is itself a client of a metered upstream — the Claude API — and it carries two distinct budgets that the Claude Certified Associate Foundations exam expects a builder to govern.

The first budget is external: the API's own rate limits, measured in requests per minute and tokens per minute, and its per-token cost. The second is internal and subtler: the context window, a hard ceiling on how much the model can attend to in a single turn. Both are finite. Both overspend silently if nothing governs them. This lesson names the thing that governs both the Budget-Governor, and shows that it is the same discipline the Ops lesson built at the edge, moved into the loop.

Two CCA-F domains meet here. Agentic Architecture, the largest domain at roughly a quarter of the exam, owns the agent loop and the tool-calling structure where API spend accrues. Context Management, a smaller but distinct domain, owns the window budget. The exam tests whether a builder can keep an agent inside both.

§ IIDomain Foundations — Two Budgets, One Discipline

An agent harness runs a loop. It sends a request to the model, receives a response that may call a tool, runs the tool, appends the result to the conversation, and sends again. Each pass through the loop is one API call, and each call spends against both budgets at once.

The external budget is the API rate limit and the bill. Anthropic meters the Claude API on requests per minute, input tokens per minute, and output tokens per minute, and it bills per token. Cross a rate limit and the API returns the same 429 the Ops lesson built, with the same Retry-After header naming how long to wait. This is the reveal that ties the trio: a builder calling Claude is in the exact position of the origin service in the Ops lesson, and Anthropic's API is the metered upstream. The 429 contract the Ops lesson taught you to serve is the contract the harness must learn to obey.

The internal budget is the context window. The window is the maximum tokens the model can hold in one turn — the system prompt, the full conversation history, tool definitions, tool results, and the room left for the response, all counted together. It does not refill. An agent loop that appends every tool result to the history grows the window pass by pass until it hits the ceiling, at which point the call fails or, worse, silently truncates the oldest context and the agent forgets what it was doing. Context management is budget governance for a resource you cannot mint more of.

§ IIIAgentic Architecture Flavor (Domain One) — Governing the External Budget

The agent loop is where API spend accrues, so the loop is where the Budget-Governor sits. Three moves the exam expects.

Obey the 429 with backoff, do not fight it. When the Claude API returns 429, the naive harness retries immediately and gets 429 again, tightening the very overload it caused. The correct harness reads Retry-After, waits exactly that long, then retries, and if no header is present it backs off exponentially with jitter. This is the same lesson as the Ops limiter viewed from the client seat: a caller that reads its refusal and returns on schedule is a caller the upstream keeps serving. The SRE overload discipline states the principle directly — a client that retries without backoff converts a transient overload into a sustained one, so backoff is not politeness, it is what keeps the system from amplifying its own failure.

Cap the loop, because an unbounded agent is an unbounded bill. An agent that can call tools can loop forever if a tool keeps returning results that prompt another call. Every iteration is API spend. The Budget-Governor sets a hard ceiling on iterations per task and a token budget per task, and when either is hit the harness stops and reports rather than spending into the dark. This is the burst parameter from the Ops lesson applied to a single task: how much are you willing to spend on one job before you refuse to spend more.

Batch and parallelize against the rate limit, not around it. When an agent has independent sub-tasks, running them concurrently is faster but multiplies the per-minute token rate, exactly as three Express nodes multiplied the limiter's rate in the Dev lesson. The governor tracks the fleet-wide spend rate and holds concurrency to what the tokens-per-minute limit allows, which is the shared-counter problem from the Dev lesson wearing a different hat.

§ IVContext Management Flavor (Domain Five) — Governing the Internal Budget

The context window is the budget that does not refill, so managing it is pure conservation. Three moves.

Count the window as a budget, not a limit you hope not to hit. The 06-14 lesson framed the window as a spend, and that framing is the exam's. Before each call the harness knows, or can estimate, how many tokens the request carries. A governor that treats the window as a live budget — system prompt plus history plus tool definitions plus expected output — sees the ceiling approaching before it arrives, the same way the Ops limiter's arithmetic guaranteed the bound before the request could breach it.

Compact the history rather than let it truncate itself. When the running conversation approaches the window ceiling, the harness has one honest move and one dishonest one. The dishonest one is to let the API silently drop the oldest turns, so the agent loses its earliest context without knowing it. The honest one is to compact deliberately: summarize the older turns into a compact synthesis, keep the recent turns verbatim, and continue. Deliberate compaction is the internal analog of the 429 with Retry-After — an honest, controlled response to a budget ceiling instead of a silent failure.

Spend the window on what earns its place. Tool definitions, few-shot examples, and long system prompts all consume the budget every single turn. The governor asks of each token whether it earns its recurring cost. A tool the agent never calls still costs its definition on every request. Pruning the context to what the task uses is the same instinct as the Ops lesson's key-expiry: bound the state to what is active, or it grows until it breaks the thing it lives in.

§ VWorked Scenario — A Research Agent Under Both Budgets

Suppose a harness runs a research agent that searches, reads, and synthesizes. Trace both budgets through one task.

The agent starts with a system prompt and four tool definitions, a fixed context cost paid on every call. It searches, appends ten results, reads three of them, and appends their full text. By the fifth loop the history has grown large enough that the next call would approach the window ceiling. The Budget-Governor sees this from its token estimate before sending. It compacts: the ten search results and three documents become a structured summary of findings, the recent reasoning stays verbatim, and the loop continues under a restored window budget. No context was silently lost; it was deliberately condensed.

Meanwhile the external budget runs in parallel. The agent's rapid loop pushes the tokens-per-minute rate toward the API limit. On the eighth call the API returns 429 with Retry-After: 12. The governor does not retry immediately. It waits twelve seconds, then resumes. The task takes twelve seconds longer and completes; the naive harness would have hammered the API, earned a longer cooldown, and possibly failed the task outright.

One task, two budgets, one governor holding both. The external budget is protected by obeying the 429 and capping the loop. The internal budget is protected by counting the window and compacting before truncation. The Ops lesson built this governor at the edge of a service; the harness runs it inside the agent. Same discipline, different seat.

§ VIConnection to Today's Ops and Dev Lessons

The Ops lesson is this lesson viewed from the server side of the same 429. There, you serve the refusal to protect your Upstream Budget; here, you obey the refusal because you are the client of Anthropic's. The Dev lesson's shared-counter problem is this lesson's concurrency governance: three Express nodes multiplying a rate is the same shape as three parallel agent sub-tasks multiplying the tokens-per-minute spend, and both are fixed by counting against one shared budget rather than N private ones. The trio is one principle seen from three seats: the edge that refuses, the fleet that must agree on the count, and the agent that must obey the refusal and govern its own window.

§ VIIPractice Questions

§ VIIIClosing

The agent harness is a client of a metered upstream, and everything the Ops lesson said about protecting a budget, the harness must say to itself twice: once for the external budget of API rate and cost, once for the internal budget of the context window. The Budget-Governor is one discipline serving both — obey the 429 with backoff, cap the loop, count the window, compact before truncation. The 429 contract you learned to serve at the edge is the contract you must obey in the loop, and the shared-counter lesson from Node is the concurrency lesson for parallel agents. A harness that governs both budgets finishes its tasks; one that governs neither runs until the bill or the window stops it, whichever comes first.

Open your agent loop. Find the retry logic and the iteration cap. If the retry ignores Retry-After or there is no iteration cap, you have written an agent that cannot be trusted with a budget. Add the backoff and the cap before you give it a paid key.

This lesson prepares CCA-F Agentic Architecture and Context Management domains. Public CCA-F access is undated; prep accumulates against the opening.

Paired lessons → Ops 01-Earth-DevOps/.../2026-07-12-rate-limiting-at-the-edge-... · Dev Polyglot-Dev/Web/2026-07-12-rate-limiting-middleware-in-node-and-express-...