Predict the result order of `chain.batch([...])` versus `await chain.abatch([...])` on a 10 input list
An LCEL chain is invoked two ways on the same list of 10 prompts: `results_sync = chain.batch(inputs, config={'max_concurrency': 5})` and `results_async = await chain.abatch(inputs, config={'max_concurrency': 5})`. The 10 prompts each take wildly different amounts of time at the model side. Predict what order the elements of `results_sync` and `results_async` come back in, and whether the calling code blocks during execution.Both .batch and .abatch return results in input-position order; the real difference is .batch blocks the thread while .abatch yields the event loop. Use .abatch_as_completed for completion-order results.
Picture handing ten parcels to a delivery service with a rule that says only five trucks can be on the road at once. The drivers come back in different orders depending on traffic. The service still gives you back ten signed receipts in the same order you handed over the parcels, not in the order the trucks returned. That is the position-preserving part. The other difference is whether you stand by the door waiting for every truck or whether you go answer the phone while you wait. Both eventually hand you the same stack of receipts; one just frees you up to do other work in the meantime.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The .batch versus .abatch distinction is one of those small LangChain APIs that hides a sharp production trap. The trap is subtle because both methods do exactly what the documentation says they do; the trap is what happens when a developer assumes they do something else.
This deep dive walks through what each method actually returns, what max_concurrency actually controls, when to reach for the as_completed variants, and the most common mistake teams make when they adopt async LangChain in an existing sync codebase.
Why position-preserving is the right default
Both .batch and .abatch return results in input-position order. results[3] is the result for inputs[3], regardless of which call finished first.
This is the right default because the overwhelming majority of caller code expects it. A pattern like for inp, out in zip(inputs, results): only works if the orders line up. A pandas operation like df['answer'] = chain.batch(df['question'].tolist()) silently corrupts the DataFrame if the results come back in random order. Position-preservation makes the API safe to drop into existing code without surgery.
The internal mechanism
The runtime fans out the calls under a concurrency cap, stashes each completed result by its input index, and assembles the final list in order before returning. Slow calls do not delay fast calls from running; they delay the final return because the runtime has to wait for the slowest to come back before it can hand you a list with that slot filled.
Where this hurts
The position-preserving guarantee means the caller waits for the longest call before getting any result. If one of ten calls takes 30 seconds and the other nine take 1 second each, .batch returns after 30 seconds, not after 3. If you need to do anything with the fast results sooner, you need the as_completed variant instead.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Method | Result order | Concurrency model | Best fit |
|---|---|---|---|
| .batch | Input position | Thread pool, blocks caller | Sync scripts, eval runs, notebooks |
| .abatch | Input position | Coroutine, yields event loop | Async web handlers, websocket workers |
| .batch_as_completed | Completion order, with index | Thread pool | Progress streams in sync code |
| .abatch_as_completed | Completion order, with index | Async iterator | Progress streams in async services |
Real products, models, and research that use this idea.
- LangChain 0.3 documentation explicitly notes the order-preserving behavior and points to abatch_as_completed for the alternative.
- FastAPI services using LangChain almost always use .abatch to avoid pinning the event loop on the handler.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you implement a progress bar over a 10000-row eval that uses .abatch?
Switch to .abatch_as_completed for the iteration, update the bar on each yielded tuple, and assemble the final ordered list by sorting on the returned index at the end if you need order.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming .batch returns results as each finishes and writing code that depends on completion-order, only to find the list arrives strictly in input order.
60 second bullets to scan on the way to the call.
Whether .batch and .abatch preserve input order
What max_concurrency actually controls
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.