A $0.50 task budget is a constraint across the **whole task**, not on any one call. An agent might make 20 LLM calls in a single task. If each one is under max_tokens but the total adds up to $2, you blew the budget.
The right design tracks **cumulative cost** in the runtime. Every LLM call adds to a running sum: input tokens times the input price, plus output tokens times the output price. The runtime knows at any moment how much has been spent and how much is left.
Before each call, the runtime projects the cost: input size now, expected output size at current pricing. If projected cost is comfortably under remaining budget, the call goes ahead normally. If projected cost is close to the limit, the runtime takes action. It can switch the next call to a cheaper model (Claude Opus 4.7 to Haiku, GPT-5.5 to mini) which preserves the call at lower spend. Or it can inject an early-finalize prompt asking the agent to wrap up with what it has now.
On budget exhaustion, the task hard-kills with a structured failure so the caller knows what happened. Silent truncation produces an answer that looks fine but might be missing the actual conclusion.
max_tokens alone caps one call. A step cap is a rough proxy but breaks when later steps grow expensive. Worst-case pre-compute is impossible for an open-ended agent.
Per-task budget enforcement is **per-task, not per-call**. The runtime maintains a running sum of token cost across every LLM call the task makes, computed at the current provider rate.
Before each call, the runtime projects the next call's cost: input tokens at the input rate plus expected output tokens at the output rate. Compare against remaining budget. Three states emerge.
Far from limit: call proceeds normally. Close to limit: trigger one of two strategies. **Tier downgrade** swaps the next call to a cheaper model, preserving the call structure but at a lower spend. **Early finalize** injects a system prompt asking the agent to produce its best answer with what it has now. Both extend reachable runtime under the same budget.
At exhaustion, the task hard-kills with a structured failure: a typed error containing budget consumed, budget allowed, last step reached, and a partial answer if available. Silent truncation is the antipattern. The caller has no way to tell that the response is incomplete, and downstream pipelines treat truncated output as the final answer.
The other options each fail. max_tokens bounds one call; the task makes many. Worst-case pre-compute is unknowable because agent step counts are open-ended. A step cap is a proxy that breaks when context grows quadratically and later steps become expensive.
Production systems instrument every LLM call with the running cost and emit it to a tracing tool so you can see budget burn per task in real time.
Quick reference
TL;DR
Track cumulative cost across all calls, check projected next-call cost against remaining budget, downgrade or early-finalize when close, hard-kill on exhaustion. The runtime owns the budget, not a per-call cap.
Imagine a road trip with 50 dollars in your pocket. You do not control the trip by saying every snack must cost less than 5 dollars. That would let you spend 5 dollars on snacks ten times and run out. You control it by carrying a running tally: how much have I spent so far, how much is left, how much is the next stop probably going to cost. When you are getting close to empty, you stop at a cheaper place or skip a stop. When you hit zero, you stop the trip and call home. An agent task budget works the same way. You do not just cap each LLM call. You watch the running total, predict the next call, and adjust before you run out instead of after.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
A single-shot one-call task with a known input size and predictable output bound. max_tokens plus an output cap is sufficient when there is no loop accumulating cost across calls.
A development or staging environment with unrestricted spend allowed for exploration. Budget enforcement adds friction; in non-production environments the cost of an occasional runaway task is lower than the cost of constantly tuning budgets.
A task where partial answers are unacceptable, such as a binary go/no-go decision in a safety-critical pipeline. Early finalize or tier downgrade would produce a partial answer; the right design is to refuse the task at submission if the budget is too low for a full run, rather than degrade quality mid-task.
Quick scan · 2–4 min
Everything important, quickly.
Lead with the framing that budget is a per-task runtime concern, not a per-call model parameter. Describe the running-sum cost track and the per-call projection step. Cover the two close to limit strategies, tier downgrade and early finalize, and explain why each works. Hard-kill on exhaustion with a structured failure carrying the partial answer and reason code. Close with observability: per-task cost emission, p99 alerts, prompt caching as a multiplier on effective budget.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's docs recommend tracking cumulative cost per task and exposing it on the response so applications can enforce budgets at the runtime layer rather than relying on per-call caps.
2LangSmith and Langfuse both expose per-trace token cost broken down by call, which production teams use to set per-tenant or per-task budget alerts.
3OpenAI's Assistants API and Anthropic's tool-use loop both leave budget enforcement to the application, which is why frameworks like LangGraph ship explicit cost-tracking primitives.
4Coding-agent products like Cursor and Cline expose per-conversation cost and degrade gracefully when usage approaches the user's plan limit, often by switching to a cheaper model tier.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you estimate next-call output tokens accurately enough to project cost?
Sign in or go Pro to see the approach for this follow-up.
QTier downgrade trades cost for capability. How do you decide when the cheaper model is good enough?
Sign in or go Pro to see the approach for this follow-up.
QWhat if the agent is in a multi-turn user conversation and the per-task budget is actually a per-turn budget?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Setting max_tokens per call and hoping the total stays under budget, when max_tokens has no idea what the other calls in the task have already spent.