Define cost per task budgeting for an agent. Describe the enforcement mechanism and explain what happens when the budget is exceeded mid task.
A cost per task budget caps the dollars one agent run may spend on tokens plus tool fees, enforced by a running counter that injects a graceful stop signal before the cap.
Imagine you give a worker a prepaid card and one job to finish. Every phone call they make and every paid lookup they buy comes off that card. You do not want them to keep spending forever, so you watch the balance. When the card is nearly empty, you do not snatch the phone away mid-sentence and lose everything they were doing. Instead you tap them on the shoulder and say the money is almost gone, please wrap up and tell me what you found so far. They write a short summary and stop. An agent budget works the same way. The card balance is the running cost, the tap on the shoulder is a polite message added to its notes, and the summary is a partial answer instead of a crash that throws away all the work.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A cost per task budget is a runtime spending limit applied to a single agent task, from the first model call to the final answer. It exists because agentic systems do not have a fixed cost. An agent loop can take three steps or thirty, and each step is a fresh model call plus possibly a paid tool call. Cost compounds with step count, and a confused agent that loops on a hallucinated assumption can burn an unbounded amount of money before anyone notices.
The contrast with a single chat completion is what makes this hard. One prompt and response has a cost you can predict from token counts before you even send it. An agent loop is open-ended by design, so the only way to bound spend is to measure it as the run unfolds and stop deliberately. The budget is therefore not a static config value, it is a live control loop running alongside the agent.
The budget answers two questions. First, what counts as cost for one task. Second, how does the runtime stop the loop before that cost is exceeded, without throwing away the work already done. The second question is where most naive implementations get it wrong, and it is the part a senior interviewer is really probing.
What cost per task actually sums
The cost of a task is not the cost of one call. It is the sum of every charge incurred across the whole run. The dominant term is usually tokens. Every model call bills for input tokens, which include the entire growing transcript, plus output tokens for the response. Because the transcript grows with each turn, input token cost climbs as the loop runs, so late steps cost more than early ones even for the same model. This super-linear growth is the reason a ten-step run can cost far more than ten times a one-step run.
The second term is external tool fees. A web search tool, a hosted retrieval service, or a sandboxed code execution environment each charges per call. These are easy to forget because they do not show up in the model provider's bill, but on a search-heavy agent they can dominate token cost entirely. A complete accounting also tracks cached versus uncached input tokens, since prompt caching can change the effective price of the growing transcript by a large factor, and any reasoning tokens the model bills separately.
The metric that matters in production is cost per successful task, not cost per call and not even cost per task. Failed runs and retries still cost money. If you divide total spend by successful completions, you see the true unit economics. A cheap per-call price can hide an expensive retry storm where the agent fails repeatedly before it succeeds. Teams that only watch the per-call price are routinely surprised when the monthly bill is several times what a naive multiplication predicted, because the multiplier hiding in the gap is the average number of steps and retries per successful task.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
def step_guard(state, budget):
spent = state.token_cost + state.tool_cost
next_estimate = budget.per_call + budget.per_tool
if spent + next_estimate > budget.cap:
# soft stop: model sees this and summarises
state.observations.append(
{"role": "tool", "name": "budget_exceeded",
"content": f"Budget cap ${budget.cap} reached. "
"Return a partial answer now."}
)
return "stop_signal_injected"
return "ok"| Stop strategy | How it triggers | Effect on completed work |
|---|---|---|
| Hard termination | Runtime raises an exception when the cap is hit | All in-progress work is discarded; user gets a crash |
| Graceful stop | Runtime injects a synthetic budget exceeded observation | Model summarises progress and returns a partial answer |
| Tiered degrade | Spend crosses a soft threshold below the cap | Loop switches to a cheaper model to extend the run |
Real products, models, and research that use this idea.
- Anthropic's Claude Agent SDK and the Claude Opus 4.7 tool use loop expose per-step token usage, letting a runtime accumulate spend and enforce a dollar ceiling across a task.
- LangGraph lets you store a running cost field in typed graph state and add a conditional edge that routes to a summarise-and-stop node once spend crosses a threshold.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you estimate the cost of the next step before you have made the call?
Use a conservative estimate from historical step costs for this agent or a percentile of recent output lengths. Reserve a cushion equal to one model call plus the most expensive likely tool, so the pre-flight check never lets a single turn overshoot the cap.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Enforcing the cap by throwing an exception that kills the loop. A hard crash discards every step already paid for, instead of letting the model summarise and exit cleanly.
60 second bullets to scan on the way to the call.
Define cost per task as token cost plus external tool fees over one run.
Explain why the unit is cost per successful task, not cost per call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.