Pick the right CrewAI Process for a fan-out research pipeline and defend the choice
You have a pipeline: one query goes to four parallel researchers (web, arxiv, internal-docs, news), then one synthesizer combines their outputs into a single brief. Which CrewAI Process do you choose, and what concrete configuration makes the fan-out actually parallel rather than serial?
Sequential Process with async_execution=True on the four researcher Tasks and an explicit context=[...] fan-in on the synthesizer, hierarchical would add a manager LLM for a topology you already know.
Think of running a small newsroom. You know exactly what you want: four reporters file stories, one editor stitches them into a brief. You do not need to hire an editor in chief to decide who writes what, the assignment is obvious. You just need the four reporters to write at the same time instead of one after another, then have the editor wait for all four before starting. That is what async_execution does in CrewAI: it tells the four researcher Tasks to go in parallel, while the synthesizer Task stays patient and reads all four when they finish.
Detailed answer & concept explanation~5 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
6 to 8 min: Sequential vs Hierarchical decision rule + async_execution + sync synthesizer + context fan-in + rate-limit and timeout caveats.
Real products, models, and research that use this idea.
- Perplexity-style 'deep research' agents use exactly this pattern, fan out to several search backends, fan in to a synthesizer.
- Bloomberg-internal research bots fan out across earnings, news, and filings retrievers, then a writer Task synthesizes.
- CrewAI's own crew-examples repo ships a 'market research crew' with the four researchers plus writer shape.
- GPTResearcher (open source) implements the same fan-out / fan-in in raw asyncio without a framework, useful comparison for the no-framework alternative.
- LangGraph's parallel-branch + join nodes are the typed equivalent, same shape, different ergonomics.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf one researcher times out, how do you keep the synthesizer from blocking forever and still produce a useful brief?
QWhen would you graduate this pipeline from CrewAI Sequential to LangGraph?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Reaching for Process.hierarchical because the pipeline 'feels' multi-step. Hierarchical pays a manager-LLM call per routing decision, on a fixed 4-to-1 shape, that cost buys nothing.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.