Similar questions
Same topic, related formats. Practice these next.
Next question
What is the KV cache in transformer inference?
Flashcard·Easy
Same topic, related formats. Practice these next.
A chat app exposes one tool (`get_weather`) with a 180-token JSON schema. A user asks 'what is the weather in Tokyo?' (8 input tokens, system prompt 50 tokens). Two paths: Path A, inline answer (no tools): model replies directly with a 25-token guess. Path B, function calling round-trip: Turn 1: prompt includes system + tool schemas + user message; model emits a `tool_call` message (40 output tokens: function name + JSON args). Tool executes externally (no model cost). Turn 2: prompt now includes everything from Turn 1 plus the tool_call assistant message plus a 60-token tool_result message; model emits the final 30-token natural-language answer. Using OpenAI-style accounting where the full conversation is re-sent on every turn, compute: 1. Total billed input tokens across Path B (sum across both turns) 2. Total billed output tokens across Path B 3. The Path B / Path A token-cost ratio (assuming equal input and output unit price) Report all three numbers.
A two-turn tool round-trip re-sends the schema and the prior assistant + tool messages on Turn 2, so a one-tool workflow with a 180-token schema costs about 7.8x what an inline answer would, dominated by input
Imagine asking a librarian a question. Inline answer is just talking: you ask, they answer. Tool-calling is more like writing letters. First letter: you send the question plus the rulebook of tools the librarian can use, and the librarian writes back saying 'I need to use the weather tool, give me Tokyo.' Then someone fetches the answer. Second letter: you send the original question AND the rulebook AGAIN AND the librarian's request AND the tool's reply, so the librarian has the full story to write the final answer. You paid postage for the rulebook twice and for the conversation history once. That is why a simple question with one tool can cost eight times what a direct answer would.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: walk both paths token by token, compute the totals (576 input, 70 output, 646 total), state the 7.8x ratio, and name prompt caching plus schema minimization as the two production mitigations.
| Token component | Path A (inline) | Path B (tool) | Notes |
|---|---|---|---|
| System prompt | 50 in | 50 + 50 = 100 in | Paid every turn |
| Tool schema | 0 | 180 + 180 = 360 in | Paid every turn (cacheable) |
| User message | 8 in | 8 + 8 = 16 in | Paid every turn |
| Tool_call message | 0 | 40 out + 40 in (Turn 2) | Output Turn 1, input Turn 2 |
| Tool_result message | 0 | 60 in | Input on Turn 2 |
| Final answer | 25 out | 30 out | Tokens the user actually sees |
| Totals | 58 in + 25 out = 83 | 576 in + 70 out = 646 | 7.8x ratio |
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Counting only the final answer tokens. The big cost is input: the schema is in the prompt twice (once per turn), and the tool_call and tool_result are re-fed as input on Turn 2. Plan capacity in tokens, not requests.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.