Select every lever that reduces the cost of a chatty multi-agent workflow

Multi-select·Medium·4.0 · 0·~1 min·Asked atAi21 Amd Anduril

Attempt it

**Multi-agent cost** scales with two things: how many hops the system takes, and how much context each hop carries. The four real levers attack those two terms directly. **Cheaper supervisor model.** The supervisor's job is routing, not deep reasoning. Most routing decisions are short and fit a smaller model fine. Keep the frontier model for workers that do real work. **Trim per-agent context.** Every agent that re-reads the full transcript pays for the full transcript. Show each agent only what it needs. The supervisor often needs less than you think. **Structured handoffs.** Pass typed payloads (a Pydantic object, a JSON schema) instead of forwarding the whole chat history. Cuts input tokens dramatically and removes the role-bleed failure mode for free. **Hard caps on turns and handoffs.** When something goes wrong, you want it to fail fast, not burn the budget looking for a fix. The two wrong answers are common traps. Adding more supervisor hierarchy multiplies routing-call cost. Raising temperature does not speed anything up reliably and can make outputs less consistent, which lengthens runs.

**The cost mental model is roughly `(hops * supervisor_cost) + sum(worker_cost_per_hop)`.** Every real lever attacks one of those terms. **Cheaper supervisor.** The supervisor mostly routes and summarises, both of which a smaller model handles well. Move it from a frontier tier to a mid tier (or even a cheap small model for trivial routing) and the per-hop overhead drops by a large multiple. This is the single highest-impact change on most chatty workflows. **Trim per-agent context.** Each worker should see only its sub-task and the inputs that matter, not the entire running transcript. Strip stale messages, scope tool-result history, and avoid re-broadcasting prior worker outputs that the current worker does not need. **Structured handoffs.** Hand off a Pydantic object or typed JSON, never raw chat. The supervisor consumes a `WorkerResult(success, summary, artefacts)` instead of parsing prose. Input tokens drop, role-bleed disappears, and traces become readable. **Hard caps.** `max_turns`, `max_handoffs`, and `max_cost` on the run. When the system loops or misroutes, you want it to stop, not to spend twenty more hops trying to recover. **Why the other two are wrong.** Extra hierarchy means every routing decision pays multiple supervisor calls, so cost multiplies, not divides. Temperature is not a speed control; high temperature can lengthen runs by making outputs less consistent so workers backtrack more.

TL;DR

Cheaper supervisor, trimmed context, structured handoffs, and hard caps are the four reliable cost levers; adding hierarchy and tweaking temperature are not.

Memory aid

Easy to grasp

Think of a noisy office meeting that bills by the minute. You can save money four ways. Hire a cheaper chair to run the meeting. Stop printing the entire project history for everyone every round. Pass clean one-page memos instead of forwarding the full email thread. Set a hard cap on how long the meeting can run. What does not save money: adding two more chairs above the current chair (now three managers bill by the minute), or telling everyone to talk faster by being more random (they argue more and the meeting runs longer). Same logic for multi-agent LLM systems.

Key concepts

Multi-Agent Systems

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Cost is the single most common reason teams roll back from multi-agent to single-agent in 2026. The fix is rarely 'less ambition'; it is usually 'the same ambition, executed with the right levers'. This question separates engineers who have actually run multi-agent in production from those who have only read the marketing pages of three frameworks.

The correct answers all map to specific terms in a simple cost equation. The trap answers both sound plausible at first and both fail under any quantitative analysis.

One-line summary: cost in a multi-agent system is hops * supervisor + sum(workers). The four real levers attack those terms. Everything else is folklore.

The cost equation, in honest detail

The shape

For a supervisor and workers run that takes N hops, total tokens are approximately

T_{total} \approx N \cdot (C_{sup} + O_{sup}) + \sum_{h=1}^{N} (C_{w,h} + O_{w,h})

where C_sup is the supervisor's input context per hop, O_sup is its output, and C_w,h, O_w,h are the corresponding worker terms. Dollar cost is T_total multiplied by the appropriate per-token price, which differs across the supervisor and workers if you route to different models.

Where the dominant terms live

In practice, C_sup grows quickly because the supervisor accumulates message history. C_w,h for any given worker is bounded by how much you choose to pass. O_sup is small (a routing decision). O_w,h is usually the worker's main output and is task-bounded.

What this implies for levers

Any lever that does not reduce one of these terms (or the multiplier in front of them) is not a cost lever. That is the framework for the rest of the analysis.

The four real levers and which term they attack

Why the two trap answers are wrong

Measurement, defaults, and when to stop optimising

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Lever	Effect on cost equation	Real or trap
Cheaper supervisor model	Multiplier on largest hop-level term	Real
Trim per-agent context	Cuts *_context across all hops	Real
Structured (typed) handoffs	Cuts supervisor context, removes recovery hops	Real
max_turns and max_handoffs caps	Bounds worst-case hop count	Real
Add more supervisor layers	Multiplies supervisor cost per hop	Trap
Raise temperature	No causal effect on hop count; may lengthen	Trap

Real products, models, and research that use this idea.

Anthropic's published agent-design guidance recommends mixing Haiku for routing or summarisation with Sonnet or Opus for the hard reasoning steps.
LangGraph's create_supervisor lets you pass a different `llm` to the supervisor than to each worker, exactly to support heterogeneous routing.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you actually measure whether your cost optimisations worked?

Track per-run tokens by agent (supervisor vs each worker), per-run cost in dollars, and hop count. Plot the distribution, not just the mean. Cheaper supervisor should drop supervisor tokens by a multiple; typed handoffs should drop supervisor input tokens; caps should clip the tail of the hop-count distribution.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Reaching for more hierarchy to fix routing quality. Every layer adds an LLM call per hop, so you are multiplying the cost you are trying to control.

60 second bullets to scan on the way to the call.

The rough cost equation for supervisor plus workers (hops times supervisor cost plus worker costs)
Why heterogeneous routing (cheap supervisor, strong workers) is usually the biggest single win

Primary sources. Browse if you want the original framing.