Fill in the order of magnitude cost formula for a supervisor and workers run

Fill in blank·Medium·4.0 · 0·~1 min·Asked atBanana Dev Ey Perplexity

Attempt it

For a supervisor and workers workflow, total tokens approximately equal ( multiplied by supervisor_context_per_hop) plus the sum of per hop. The single biggest cost lever is moving the to a cheaper model than the workers, since its job is closer to classification than to open ended reasoning.

**The formula.** For a supervisor and workers run: $$\text{total tokens} \approx N_{\text{hops}} \times C_{\text{supervisor}} + \sum_{i=1}^{N_{\text{hops}}} C_{\text{worker}_i}$$ The supervisor runs once per hop to decide who works next; each worker runs once per hop to actually do the work. **Why the supervisor's job is cheaper-shaped.** The supervisor reads the latest message plus the agent roster and outputs a short routing decision (which worker, what payload). That is closer to classification than to open-ended reasoning. Classification-shaped problems can be solved well by smaller, faster, cheaper models. **The cost lever.** Moving the supervisor from a flagship model (Claude Opus 4.7, GPT-5.5) to a fast small model (Claude Haiku 4.5, GPT-5.5-mini, Gemini 3.1 Flash) typically cuts supervisor cost by 5 to 10 times with negligible quality regression. Since the supervisor accounts for 30 to 60 percent of total spend in chatty workflows, downsizing it is the highest-impact single change you can make. The workers stay on the flagship model because their job (open-ended reasoning, tool calls) actually benefits from the capability.

**The formula** is simple: $$\text{total tokens} \approx N_{\text{hops}} \times C_{\text{supervisor}} + \sum_{i=1}^{N_{\text{hops}}} C_{\text{worker}_i}$$ Every hop the supervisor runs to route, then a worker runs to do the work. Both contributions scale with hop count, but they scale at different per-call costs. **Why the supervisor cost is the lever.** The supervisor's job is closer to classification than to open-ended reasoning. It reads the latest message, sees the available worker roster, and picks. That is exactly the workload small models handle well. Workers, by contrast, do the actual cognitive work (research, code, multi-step reasoning) and benefit from the flagship capability. **Concrete numbers.** A 6-hop run with 1 supervisor and 4 workers on the same flagship model (3K supervisor context per hop, 4K worker context per hop) burns about 51K tokens, with the supervisor responsible for roughly 40 percent of the bill even though it never does real work. Move the supervisor to a fast small model and that 40 percent drops 5 to 10 times in cost terms. **The second-biggest lever** is prompt caching: the supervisor's stable prefix (system prompt, agent roster) is identical every hop, exactly the cached prefix plus dynamic suffix pattern caching is built for. Combining both levers (downsize plus cache) routinely cuts supervisor cost by 15 to 30 times.

TL;DR

Total tokens equals hop_count times supervisor cost per hop plus the sum of worker costs per hop; downsizing the supervisor model is the single biggest cost lever.

Memory aid

Easy to grasp

Imagine a restaurant where every order goes through a manager who reads it, picks a chef, hands it off, and reviews the plate before it goes out. Total kitchen cost is manager time plus chef time, multiplied across every order. The manager is doing simple work (which chef is free, who handles dessert) compared to the chef who actually cooks. If the manager were also a head chef on a head-chef salary, the bill would be silly. The fix is to put a fast, cheap dispatcher in the manager slot and reserve the expensive chefs for actual cooking. In multi-agent terms, the supervisor routes and the workers reason; sizing them the same way is the most common money-waster.

Key concepts

Multi-Agent Systems

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Cost in a supervisor and workers system is almost always under-attributed. Teams look at the monthly bill, divide by request count, and call that the cost per task. What they miss is that the bill has two structurally different components: routing decisions made by the supervisor and actual work done by the workers. Sized correctly, the supervisor should be a small share of total spend; sized badly, it can be the majority.

This walkthrough derives the two-term cost formula, shows where the supervisor share typically lands in chatty workflows, names the three cost levers in priority order, and explains why per-role span attribution is the diagnostic discipline that makes any of it actionable.

Mental model: the supervisor classifies, the workers reason. Pay classification prices for classification work and reasoning prices for reasoning work. Doing it backwards is the single most common money-waster in multi-agent systems.

The two-term cost formula

The formula

\text{total tokens} \approx N_{\text{hops}} \times C_{\text{supervisor}} + \sum_{i=1}^{N_{\text{hops}}} C_{\text{worker}_i}

Where:

$N_{\text{hops}}$ is the number of supervisor-routing decisions in one task.
$C_{\text{supervisor}}$ is the per-hop token cost of one supervisor call (context plus output).
$C_{\text{worker}_i}$ is the per-hop token cost of the worker chosen at hop $i$ .

Why two terms, not one

The supervisor runs every hop; the worker that runs each hop varies but each one runs exactly once per hop in the simple shape. If you collapse them into one number, you lose the ability to optimize either independently.

Walked example

6 hops, 1 supervisor, 4 workers. Supervisor uses 3K context plus 0.5K output per call. Workers use 4K context plus 1K output per call.

Supervisor: $6 \times 3.5\text{K} = 21\text{K}$ tokens.
Workers: $6 \times 5\text{K} = 30\text{K}$ tokens.
Total: 51K tokens, of which 21K (41 percent) is supervisor.

That 41 percent is the canonical 'chatty supervisor' shape: 30 to 60 percent of total spend on routing decisions, in a system where every routing decision is structurally classification.

Why the supervisor's job is classification-shaped

Three cost levers in priority order

Per-role attribution: the diagnostic discipline

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

LangGraph's `create_supervisor` prebuilt explicitly supports passing a different (smaller) model to the supervisor than to the workers; the docs call out the cost reason.
Claude Haiku 4.5 and GPT-5.5-mini are the canonical 2026 supervisor-tier models behind flagship-tier workers like Claude Opus 4.7 or GPT-5.5.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you measure that downsizing the supervisor did not regress routing quality?

Build an eval set of (input, correct worker choice) pairs. Run the flagship supervisor and the small supervisor on it. Compare routing accuracy. If small-model accuracy is within 1 to 2 percent of flagship, downsize. If it drops noticeably, the routing logic is harder than classification and the supervisor is doing real reasoning.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Running the supervisor on the same flagship model as the workers, so 30 to 60 percent of the bill goes to routing decisions a small model could make for a tenth the cost.

60 second bullets to scan on the way to the call.

The two-term cost formula for supervisor and workers
Why the supervisor's job is classification-shaped, not reasoning-shaped

Primary sources. Browse if you want the original framing.