Predict the result order of `chain.batch([...])` versus `await chain.abatch([...])` on a 10-input list
An LCEL chain is invoked two ways on the same list of 10 prompts: `results_sync = chain.batch(inputs, config={'max_concurrency': 5})` and `results_async = await chain.abatch(inputs, config={'max_concurrency': 5})`. The 10 prompts each take wildly different amounts of time at the model side. Predict what order the elements of `results_sync` and `results_async` come back in, and whether the calling code blocks during execution.Both .batch and .abatch return results in input-position order; the real difference is .batch blocks the thread while .abatch yields the event loop. Use .abatch_as_completed for completion-order results.
Picture handing ten parcels to a delivery service with a rule that says only five trucks can be on the road at once. The drivers come back in different orders depending on traffic. The service still gives you back ten signed receipts in the same order you handed over the parcels, not in the order the trucks returned. That is the position-preserving part. The other difference is whether you stand by the door waiting for every truck or whether you go answer the phone while you wait. Both eventually hand you the same stack of receipts; one just frees you up to do other work in the meantime.
Detailed answer & concept explanation~6 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
6 minutes: position-preserving behavior, the sync vs async axis, max_concurrency, the as_completed variants, and the nested-batch trap.
| Method | Result order | Concurrency model | Best fit |
|---|---|---|---|
| .batch | Input position | Thread pool, blocks caller | Sync scripts, eval runs, notebooks |
| .abatch | Input position | Coroutine, yields event loop | Async web handlers, websocket workers |
| .batch_as_completed | Completion order, with index | Thread pool | Progress streams in sync code |
| .abatch_as_completed | Completion order, with index | Async iterator | Progress streams in async services |
Real products, models, and research that use this idea.
- LangChain 0.3 documentation explicitly notes the order-preserving behavior and points to abatch_as_completed for the alternative.
- FastAPI services using LangChain almost always use .abatch to avoid pinning the event loop on the handler.
- Batch eval scripts for DSPy training sets typically use .batch in synchronous Python and tune max_concurrency to match the provider's per-minute quota.
- OpenAI and Anthropic both expose dedicated batch APIs with separate pricing; .batch over the synchronous endpoint is not the same thing and does not get the discount.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you implement a progress bar over a 10000-row eval that uses .abatch?
QWhat happens if one of the 10 calls in an .abatch raises an exception? How do you change that behavior?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming .batch returns results as each finishes and writing code that depends on completion-order, only to find the list arrives strictly in input order.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.