Why cap each agent run with a cost per task budget?
A hard runtime cap on tokens, tool calls, or dollars per task. Without one, runaway loops turn $0.10 tasks into $100 bills because token cost grows quadratically with step count.
Imagine giving an assistant a credit card to buy office supplies. Most days they spend twenty dollars. One day they get stuck in a loop trying to find the perfect stapler, comparing models, ordering and returning, ordering and returning, and rack up a thousand-dollar bill before lunch. Setting a daily card limit of fifty dollars stops the disaster before it happens. A cost per task budget is that limit for an agent. Each task gets a hard ceiling on how many tokens, tool calls, or dollars it can use. The moment the agent hits the ceiling, the task stops, even if the answer is not finished, because the alternative is a five hundred dollar bill on what should have been a ten cent task.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A cost per task budget is a hard cap on what a single agent task is allowed to consume, enforced by the runtime as a kill switch when the cap is reached. The unit can be input tokens, output tokens, tool calls, dollars, or wall-clock time, and most production systems enforce several of these at once.
The reason this primitive exists is that agent cost is not well-behaved. A typical agent task on a typical day looks cheap. A task that should cost ten cents costs ten cents, and you forget the budget machinery is even there. But agents have a specific failure mode that no other class of system shares: they can loop on themselves, generate ever-longer prompts, and chew through tokens at a quadratic rate. Without a runtime cap, a single buggy planner can produce a four-figure bill on a workload nobody knew was running.
This explanation builds the picture in four parts: why agent cost compounds non-linearly, what dimensions to actually cap and why one cap is not enough, where the cap must be enforced for it to actually work, and how to tune budget values from production data so they catch runaways without breaking legitimate long tasks.
Why agent cost compounds non-linearly
The non-obvious thing about agent economics is that step count alone does not bound cost. Each model call the agent makes carries a prompt that includes everything the agent has done so far. At step 1 the prompt holds only the user message. At step 5 it holds the user message plus four turns of tool calls and results. At step 20 it holds twenty turns. The input-token cost of step k is roughly proportional to k.
Sum that across the task. Total input tokens across N steps grow like 1 plus 2 plus ... plus N, which is N(N+1)/2, or order N-squared in the limit. Output tokens grow linearly with N because each step produces a roughly constant output. So the input-token term dominates once N is more than a handful of steps, and total cost scales quadratically with step count in the worst case.
A concrete example clarifies the magnitude. Suppose your normal task converges in 5 steps with an average input size of 5,000 tokens and an output of 300 tokens. On Claude Opus 4.7 at $5 per million input tokens and $25 per million output tokens, that is about $0.13 per task. Now imagine the same agent loops to 30 steps with the same history accumulation. Total input tokens are around 30 times the average step's input, but the average step input has also grown six-fold. Total tokens cross 750,000 input plus 9,000 output, and the bill is closer to $4.00 per task, a 30-times jump from a 6-times jump in step count.
Worse cases compound further. If the agent fetches a 20,000-token document and never compresses it, every subsequent step carries that document in its prompt. Output tokens stay small, input tokens explode. The economics flip from 'cheap automated work' to 'unbounded liability' within seconds, and any system without a cost per task budget is exposed to that flip whenever a planner misbehaves.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- LangGraph supports per-graph and per-run step caps and integrates with cost-tracking middleware to enforce token and dollar limits at the runtime layer.
- Helicone and Langfuse report per-task token and dollar cost from their tracing data, making per-task budgets observable and alertable in production dashboards.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does agent cost grow quadratically with step count instead of linearly?
Each step's prompt includes the prior history. Input tokens at step k include the tokens added by steps 1 through k minus 1. Summed across N steps, total input tokens grow like N squared in the worst case. Output tokens grow linearly. The squared term dominates once N is more than a handful of steps.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Believing token cost per step is the right unit to budget. Cost grows non-linearly because each step's prompt includes every prior step's history, so the right unit is total tokens or dollars per task, not per step.
60 second bullets to scan on the way to the call.
Define a cost per task budget as a hard cap enforced by the runtime.
Explain why agent cost grows non-linearly with step count (prompt accumulation).
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.