Spot the error in this agent runtime's tool-call handling code
Click any words you think contain an error. Click again to unmark.
The runtime trusts the model's tool call blindly. It never checks the tool exists, so a hallucinated name returns None and crashes on .execute instead of becoming a recoverable error observation.
Imagine a waiter who takes any order a customer shouts and runs to the kitchen to make it, even if the dish is not on the menu. When the kitchen has no recipe, the waiter freezes mid-step and the whole restaurant stops. A better waiter checks the menu first. If the dish does not exist, they calmly tell the customer it is not available and ask them to pick something real. The customer can then choose again. This code is the bad waiter. It takes whatever tool name the model invents and tries to run it. When the name is not on the menu, the lookup hands back nothing, and the next line breaks because you cannot run nothing. The fix is to check the menu before cooking.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
State the bug precisely, that an unguarded execute on a None lookup crashes the loop. Then explain why a tool call is untrusted input, lay out the validate then reject and reprompt pattern over both name and schema, and close with error message quality and the try block around execution itself.
Real products, models, and research that use this idea.
- The Anthropic SDK and OpenAI tool use both return a structured tool result message on failure, so a bad call from Claude Opus 4.7 or GPT-5.5 becomes an observation the model reasons about rather than a crash.
- LangGraph's prebuilt ReAct agent routes an unknown tool name back as a ToolMessage error, keeping the state graph alive so the model can pick a valid tool on the next node.
- Pydantic AI validates tool arguments against a typed model before execution, and a validation error is fed back to the model as a retry prompt automatically.
- MCP servers declare a JSON schema per tool, and a compliant host validates the model's arguments against that schema before dispatching the call to the server.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you design the error observation so the model is most likely to self correct on the next turn?
QWhen does validation alone fail to protect you, and what else must the runtime enforce on a tool call?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming the model only ever names tools that exist. Models hallucinate function names and malformed arguments, so the runtime must validate every call against the registry and schema before executing.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.