Why does token cost in a naive agent loop grow quadratically with step count?

Attempt it

Quick reference

TL;DR

Every loop turn re-sends the whole growing transcript, so per-turn input scales with step number and the summed cost over N turns is quadratic.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine you hire someone with no memory between minutes. Each time you ask them to do the next part of a job, you must re-read aloud everything that happened so far before they can act. On step one you read one paragraph. On step ten you read all ten paragraphs again, even though only the last one is new. The further you get, the longer your re-reading takes, because the story keeps growing and you always start from the top. An agent loop works exactly like this. The model forgets nothing only because you resend the entire transcript on every turn. Early turns are cheap, late turns are expensive, and the total reading time across a long task piles up far faster than the number of turns alone would suggest.

Key concepts

Cost Compounding Context Window

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

Start from the stateless model and append-only transcript, show per-turn input scaling with the step index, derive the triangular sum and simplify it to O(N²), rule out pricing and parameter count, then cover prompt caching and the context-trimming techniques that pull cost back toward linear.

Real-world examples

Real products, models, and research that use this idea.

Anthropic and OpenAI both ship prompt caching specifically because agent loops resend a growing prefix; cache reads cut the per-turn dollar cost of that resent context by a large margin.
LangGraph and the Anthropic SDK expose summarisation and message-trimming hooks so long-running loops can hold per-turn input near constant instead of letting it climb with step count.
Coding agents like Cursor and Cline keep full file contents in a side store and inject only diffs or referenced spans, avoiding resending entire files on every edit-test turn.
Tracing tools such as LangSmith and Langfuse attribute token cost per loop step, which is how teams spot the quadratic climb and decide where to trim context.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QPrompt caching means resent tokens are cheap. Does the quadratic problem go away?

Sign in or go Pro to see the approach for this follow-up.

QSummarisation holds per-turn input near constant. What does it cost you?

Sign in or go Pro to see the approach for this follow-up.

QPer-turn cost is linear in the step index, yet total cost is quadratic. Reconcile these.

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Blaming per-token pricing or model size. Cost grows because each turn resends the full accumulated context, not because tokens get more expensive as the window fills.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.