An agent system has two major components: the LLM and the runtime. Describe the responsibility of each, then explain why confusing them leads to incorrect debugging assumptions.
The LLM only decides and emits text; the runtime parses that text, executes tools, and feeds results back. Knowing who does what tells you where a bug actually lives.
Imagine a brilliant chess coach who is locked in a soundproof booth. The coach can see the board through glass and can shout out moves, but the coach can never touch a piece. A second person, the assistant, stands at the board, hears the move, moves the piece, and then describes the new board back through the glass. The coach is the brain that decides. The assistant is the hands that act. If a piece ends up in the wrong square, you have to ask the right question. Did the coach call a bad move, or did the assistant hear it wrong and move the wrong piece? An agent works the same way. The model is the coach in the booth, and the runtime is the assistant at the board.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
An agent is often described as an LLM in a loop with tools. That phrase hides the most important architectural fact: the loop contains two fundamentally different kinds of component. One is a probabilistic text generator. The other is ordinary deterministic code. The first decides. The second acts. Keeping them straight is what lets you reason about cost, safety, and failure.
The model is the brain. The runtime is the harness wrapped around it. The brain consumes context and emits a decision as text. The harness reads that text, does something real in the world, and reports back. Almost every confusing agent bug traces to someone forgetting which side of this line a behavior lives on.
The split is not a pedantic distinction. It maps directly onto two different teams, two different skill sets, and two different debugging toolkits. Prompt engineers tune the brain. Backend engineers harden the runtime. When the line between them blurs, both teams start editing the wrong thing, and the same bug gets misdiagnosed twice.
What the LLM actually does
The LLM is a pure function in the mathematical sense. Given a context window, it produces a probability distribution over the next tokens, and the output is sampled text. That is the whole contract. It has no file handle, no network socket, and no ability to run code. It cannot even remember the previous turn unless that turn is present in the context you hand it.
When we say the model made a tool call, we are using a convenient shorthand that hides the truth. The model emitted a chunk of structured text that describes a tool call: a name and a set of arguments. With provider tool-calling APIs this arrives as a typed object, but underneath it is still generated tokens. The model is asking for an action; it has not performed one. The same is true of a final answer: it is just text the runtime chooses to treat as terminal.
This is why the model can hallucinate a tool that does not exist, or supply arguments of the wrong type, or invent a plausible-looking result it never received. It is simply predicting plausible text. Nothing validates that text until it leaves the model and reaches the runtime. Treating the model as a stateless text to text function, rather than an actor with memory and hands, is the foundation everything else rests on.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- The Anthropic SDK returns a tool_use content block from Claude Opus 4.7; your code, not the model, must execute it and post back a tool_result block to close the loop.
- OpenAI function calling returns a tool_calls array as JSON text. The model never runs the function; the developer's runtime parses the arguments and dispatches the real call.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf the model emits a tool call but the result never appears in the next turn, where do you look first?
Trace the runtime path: did parsing the tool-call object succeed, did validation pass, did the tool throw, and did the result get serialized and appended to context? The model cannot tell you because it never sees its own emission again.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying the model 'called' a tool. The model only emits a tool-call object as text. The runtime parses it and decides whether to execute it, so a silent failure can hide there.
60 second bullets to scan on the way to the call.
State that the LLM only produces text and cannot execute anything itself.
Explain that a tool call in the output is a request, not a real invocation.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.