When should you decompose a complex task into multiple LLM calls versus handling it in a single longer prompt?

Decomposing a complex task into multiple LLM calls means running a small pipeline instead of one big prompt. It helps in some cases and hurts in others. Decompose when the sub-tasks have distinct success criteria you can grade separately. That makes each stage testable. Decompose when sub-tasks need different output formats, like classify-then-generate. Decompose when an intermediate result needs to be validated or repaired before the next step runs. Keep it as one call when latency matters more than observability, when the sub-tasks are tightly coupled so separating them loses context, or when cost is tight and the orchestration overhead is not worth it. The practical rule is to look at the task's structure first. If you can name distinct sub-tasks with their own success criteria, decomposition is likely to pay. If the task is one continuous flow where each step needs the last, keep it as one prompt.

Decomposition is a structural choice, not a quality default. The right move depends on whether the task has properties that decomposition can exploit. Decompose when at least one of three signals holds. First, distinct success criteria per sub-task. If you can eval the classifier output and the generator output separately, you can isolate regressions, which is hard inside one monolithic call. Second, different output formats. A pipeline that classifies into JSON and then generates prose forces an awkward shape if you collapse them into one prompt. Third, intermediate results that need validation. A retrieval step that returns chunks you want to verify, or a planner step you want to inspect before execution, both benefit from being their own call. Keep it as a single call when coupling is tight and each sub-task needs the full state of the previous one. Also stay single-call when latency is tight and multiple round trips push past p95. Same when total cost is constrained and the orchestration overhead does not earn its keep. The pattern in production LLM apps is to start single-call, then decompose when one specific failure mode (observability gap, format conflict, validation need) becomes the dominant pain. Decomposing before you have evidence of that pain usually just adds latency and complexity.