Same topic, related formats. Practice these next.
Anything the model emits bills as completion_tokens; anything you send (schemas, prior messages, tool results) bills as prompt_tokens on the turn it arrives in the request body.
Picture a restaurant where you pay separately for the menu pages you bring in and for the dishes the chef cooks. Every time you walk back in, the whole stack of menus you carry counts as input, even if some of those pages are notes from the last visit. The chef's plates are output. Now imagine the chef sometimes hands you a sticky note saying 'go grab this ingredient from the pantry' instead of cooking. That sticky note is still something the chef made, so it bills like a dish. When you return with the ingredient, the ingredient and all the prior menus and sticky notes count as input again. Each visit gets its own bill, and the day's total is the sum of all visits, not just the last one.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: per-turn input vs output rule + how schemas, tool_calls, and tool results map onto it + cumulative cost across a loop + prompt caching as the natural lever to dampen the quadratic growth.
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming the tool_calls JSON the model emits is free or input-priced, or treating only the final assistant text as the billed output. Both halves of the round-trip bill at the standard input and output rates.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.