Why CrewAI's manager_llm parameter is usually set to a cheaper model than the workers'

Flashcard·Medium·4.0 · 0·~30s·Asked atCanva Capgemini Krutrim

Attempt it

The manager's job is routing: read the conversation and the agent roster, decide who handles the next task, optionally rewrite the task description. That is a classification-shaped problem and a smaller model (a fast 4-bit-class chat model or a mini-tier API) does it well at a fraction of the cost. The worker agents do the actual reasoning, retrieval, and synthesis where capability matters.

The upside-down version (frontier model as manager, mini models as workers) is the wrong split: the expensive model spends its capability on a routing decision a smaller model would have nailed, and the workers underperform on the substantive work. The exception is when routing itself is genuinely hard (highly ambiguous tasks, many overlapping agent capabilities); then the manager gets bumped up. The default to start with is mini-manager plus capable workers.

Your answer

CrewAI hierarchical crews have one manager agent and several worker agents. The manager_llm parameter sets the model the manager uses; each worker has its own llm. They do not have to be the same model. The natural assumption is to use the best model everywhere. That is wrong, and the reason is the shape of each role's job. The manager reads the latest conversation, looks at the list of workers and what each is good at, and outputs a routing decision: who handles the next task, and what is the task description. That is a classification problem with a structured-output requirement. Small models do this kind of work very well. A mini-tier API or a fast 4-bit-quantized local model nails it for a fraction of the cost of a frontier model. The workers do the substantive work: research, write, code, analyse. That is where reasoning capability actually shows up in the output. The user reads the worker's output; they never see the manager's routing decision directly. The right default is mini-manager and capable-workers. The exception is when routing itself is genuinely ambiguous (lots of overlap between worker capabilities, vague task descriptions); then the manager benefits from being bumped up. Start with the cheap default and only escalate if eval shows routing errors.

**Routing** is classification-shaped; the **workers** do generation. Splitting model tiers along that line is the highest-leverage cost decision in a CrewAI hierarchical crew. The manager_llm runs once per hop, every hop. Its input is the conversation state plus the agent roster; its output is a structured choice: who handles the next task, what is the task description. A mini-tier model (gpt-4.1-mini, Claude Haiku 4, Gemini Flash) handles this at a tenth of the cost of a frontier model with no detectable quality loss on most routing decisions. The exception is when worker capabilities overlap and the task is genuinely ambiguous; in that case the routing error rate of a mini model becomes visible and the manager justifies a bump up. The workers do the work the user actually sees. Worker capability is the binding constraint on output quality. A frontier model writing the analysis, retrieving the evidence, generating the code: this is where the dollars buy something the user notices. **The upside-down split** (frontier manager, mini workers) is a common mistake and a bad one. The expensive model spends its capability on a decision that does not affect output quality (a mini-tier router would have picked the same worker), and the underpowered worker produces output that is visibly worse than the user expects. Net result: higher cost, worse quality, no win on any axis. The operational pattern in 2026: start with mini-manager plus capable-workers. Measure routing accuracy on a held-out eval. If routing errors are the bottleneck, escalate the manager; if worker quality is the bottleneck, escalate the workers. Almost never both: the two roles have orthogonal failure modes.

TL;DR

Routing is classification-shaped and runs every turn; spending frontier-tier dollars on it wastes capability. Default to a mini-tier manager and capable workers, and only invert when routing itself is the hard part.

Memory aid

Easy to grasp

Think of a restaurant. The host who walks people to tables does not need to be the chef. The host needs to know which table is free and which server handles that section. The chefs in the kitchen are the ones who actually need talent. If you put the chef at the door and a beginner at the stove, dinner is bad and you paid a lot for someone to greet you. CrewAI works the same way: the manager points tasks at the right specialist (easy, fast job), and the specialists do the cooking (the hard job that needs the smart model). The cheap-host, expensive-cooks layout is the default for the same reason every restaurant uses it.

Key concepts

Multi-agent systems LLM frameworks Inference optimization

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Hierarchical crews are CrewAI's answer to multi-agent orchestration: one manager agent picks which worker handles each task, and the workers do the substantive work. The manager_llm parameter is a separate handle on the manager's underlying model, distinct from the llm each worker is configured with. Most teams default to setting them all to the same frontier model. Most teams should not.

The model-tiering decision across roles in a hierarchical crew is the highest-leverage cost lever in the architecture. Getting it right cuts per-workflow cost by 50-80 percent without measurable quality loss. Getting it wrong (either way: frontier everywhere, or the inverted split with cheap workers) leaves money on the table and often degrades the output the user sees.

This section walks through the cognitive-shape argument for why routing and generation tier differently, the empirical pattern in 2026 production deployments, the conditions under which the default flips, and the operational discipline of measuring before escalating.

Routing is classification; generation is generation

The manager's job decomposes into three operations every hop: read the conversation state, scan the agent roster with each worker's role and capability description, output a structured choice (worker_name plus task_description).

This is classification-shaped. The input space is finite (one of N workers). The output is a discrete choice plus a short description. The required reasoning is shallow: 'this task mentions a paper, the researcher handles papers, route to researcher.' Smaller models with strong instruction-following nail this kind of work. Claude Haiku 4, gpt-4.1-mini, Gemini Flash all hit 90+ percent routing accuracy on typical crews at a fraction of the frontier-tier cost.

The worker's job is generation. The researcher reads sources and writes an analysis. The coder reads a spec and writes implementation. The analyst reads data and produces insights. These are open-ended generation tasks where output quality scales with model capability all the way to the frontier. The user reads the worker's output; the user is directly exposed to the difference between a Haiku-tier writer and a Sonnet-tier writer.

The decisive asymmetry: the user never reads the manager's routing reasoning. They consume the worker's output. So the manager's contribution to output quality is bounded by routing accuracy, which saturates below frontier capability for most crews. The worker's contribution to output quality is essentially unbounded by model capability for the substantive reasoning tasks workers handle.

Matching model tier to role shape produces a 5-10 percent vs 90-95 percent spend split that mirrors the actual quality contribution of each role.

Why the upside-down split fails

Measuring before escalating

Cross-framework parallels and the broader pattern

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

CrewAI production deployments in 2026 commonly pair Claude Haiku 4 manager_llm with Sonnet 4.5 workers, achieving 5-10 percent of spend on routing.
LangGraph supervisor patterns follow the same tiering: gpt-4.1-mini supervisor routing to gpt-5 worker nodes.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you measure routing accuracy on a hierarchical crew to decide if the manager needs a bump?

Hand-label 100 representative tasks with the correct worker target. Run the crew with the mini-tier manager and count first-hop accuracy. Below 85 percent indicates the manager is the bottleneck; above 95 percent confirms mini is sufficient. Between is judgement-call territory and depends on the cost of a routing miss.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Putting the frontier model on routing because the manager 'sounds important.' Routing is per-turn overhead; capability spent there is invisible to the user, and the workers underperform on the actual work.

60 second bullets to scan on the way to the call.

Why routing and generation have different cognitive shapes
How per-hop manager cost compounds over a long crew run

Primary sources. Browse if you want the original framing.