Same topic, related formats. Practice these next.
An agent loop performs 5 sequential tool calls before final answer. Walk through where the latency comes from and explain why total wall-clock is ~6× a single completion. Identify three mitigations and what each costs.
An agent loop chains 6 model turns plus 5 tool round-trips, and context grows every turn, so prefill, decode, and network all stack into roughly 6x a single completion.
Imagine cooking a recipe where you must phone a friend after every step to ask what comes next. Each call has the same overhead: you read the whole recipe so far aloud, your friend thinks, you wait on the line, they answer, then you act. As the recipe grows, reading it aloud each time takes longer and longer. Five questions means six phone calls plus five waits for your friend to act, and every call re-reads more text than the last. The cooking itself is fast. The phone tag is what eats the clock. You speed it up by asking several independent questions in one call, by not re-reading the parts your friend already heard, and by starting the next step before they finish replying when you are confident.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: decompose one round (decode, network, tool, prefill) + why prefill grows monotonically + the 6 turns 5 trips count + three mitigations each mapped to an axis + the cost of each.
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Blaming tool execution or the network for the latency while ignoring that the growing context forces a larger prefill on every single turn of the loop.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.