Multi-query versus decomposition: how do these query transforms raise recall on hard questions?

Short answer·Medium·4.0 · 0·~3 min·Asked atGlean Intel Mercor

Attempt it

Describe how multi-query retrieval and query decomposition each improve retrieval on complex questions, and explain when one fits better than the other.

Free · 2 AI evals / day

Both techniques exist because a single embedding of the user's question can miss the right chunks. If the phrasing is awkward or the question asks for several things, the one query vector lands in the wrong neighborhood and the relevant chunk falls just outside the top-k. Multi-query attacks phrasing risk. You ask an LLM to rewrite the question several different ways, retrieve for each rewrite, and combine all the hit lists into one pool. If any phrasing matches the right chunk, it makes it into the pool. This is for a single information need expressed in different words. Decomposition attacks multi-part questions. You split a question like "compare X and Y" into separate sub-questions, retrieve for each one independently, and gather the results. Each distinct fact gets its own targeted search instead of competing inside one blurred query. Use multi-query when the question is one need but the wording is risky. Use decomposition when the question genuinely bundles several needs. Both cost extra retrieval calls and latency, and both produce overlapping results, so you have to deduplicate before the chunks reach the LLM.

Both transforms exist to fix the same weakness: a single query embedding is brittle. Phrasing, vocabulary mismatch, or a many-part question can push the relevant chunks just outside the top-k. The shared fix is to issue more than one retrieval and merge. **Multi-query** generates several paraphrases of one question, retrieves for each, and unions or fuses the hit lists. A chunk that matches any phrasing makes the pool, so it widens recall for a single information need expressed in different words. It does nothing to separate distinct facts — every rewrite is still asking the same thing. Decomposition is the opposite move. It splits a genuinely multi-part question into sub-questions and retrieves for each separately, so each distinct fact gets its own targeted search rather than competing inside one blurred query vector. A comparison or a multi-hop chain is the canonical case. The selection rule follows from the failure they address. Use multi-query when the question is one need but vocabulary or phrasing is the risk. Use decomposition when the question bundles several needs that one query cannot cover. Both raise recall at the cost of extra retrieval calls and latency, and both produce overlapping hit lists, so you must deduplicate before the chunks reach the LLM. Reciprocal rank fusion is the usual way to merge multiple ranked lists into one.

TL;DR

Multi-query fuses hits from several paraphrases of one question to beat phrasing brittleness; decomposition splits a multi-part question into separately-retrieved sub-questions. Both raise recall, both need dedup.

Memory aid

Easy to grasp

Imagine looking for a book in a huge library with one librarian. Sometimes the problem is wording: you ask for "the heart-attack book" but the index says "cardiac arrest," so you come up empty. Multi-query fixes that by sending several friends to ask the same question in different words, then pooling whatever any of them found. Other times the problem is that you asked for two things at once: "compare the 2024 and 2025 budgets." One trip blurs both years together. Decomposition fixes that by splitting the request into separate errands, one for each year, so each comes back with the right shelf. Both tricks send out more than one search and then combine the results, which means you also have to throw away duplicates before handing the pile to the person who writes the final answer.

Key concepts

Retrieval-Augmented Generation Embeddings

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

These two transforms get mentioned in the same breath, which is exactly why interviews ask candidates to separate them. They share a surface gesture — issue several retrievals, merge the results — so it is easy to recite both and never explain what makes them different tools for different failures.

The real distinction is what each one varies. Multi-query holds the information need fixed and varies the wording. Decomposition holds the wording style fixed and varies the need, splitting one question into several. Both raise recall, but they raise it against different root causes, and applying the wrong one wastes calls or actively adds noise. The strongest answers name the specific failure each transform prevents and give a clean rule for choosing between them, then close the loop on the cost that both incur: more candidates, more latency, and a mandatory merge and dedup step.

Why one query embedding is brittle

A dense retriever embeds the user's question into a single vector and finds the nearest document vectors by similarity. The retrieval score is a dot product between the query vector and each candidate:

s(q, d) = q \cdot d

That single point has to land near the right chunks in embedding space. Two things knock it off target. First, vocabulary: if the user writes "heart attack" and the document says "myocardial infarction," the two phrasings may not embed close enough, and the right chunk sits just past the top-k cutoff. Second, multiplicity: if the question asks for two distinct things, the embedding averages toward a centroid between both topics and may sit near neither.

Both failures share a shape — the truth is in the corpus but the single query vector does not reach it. Top-k retrieval is unforgiving here. A chunk ranked k+1 is invisible to the generator, no matter how relevant. The two transforms are both ways to give the right chunk more than one chance to enter the pool, but they target the two different reasons it was missed.

Multi-query: variance over phrasing

Decomposition: partition over sub-needs

Choosing, merging, and controlling the precision cost

Why rank fusion beats a naive union

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Aspect	Multi-query	Decomposition
What it varies	Wording of one fixed need	Splits one question into distinct sub-needs
Best for	Vocabulary or phrasing mismatch	Comparisons and multi-hop chains
Retrieval calls	Parallel, one per paraphrase	Often sequential when a sub-answer feeds the next
Main failure mode	Cannot cover a genuinely two-part question	A wrong split drops a needed sub-need entirely
Merge step	Rank fusion plus dedup	Rank fusion plus dedup

Real products, models, and research that use this idea.

LlamaIndex query engines ship a MultiStepQueryEngine and sub-question query engine that decompose a complex question and retrieve per sub-question.
LangChain's MultiQueryRetriever generates several LLM paraphrases of a query and unions the retrieved documents.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you merge the ranked lists from several paraphrases or sub-questions?

Reciprocal rank fusion sums 1/(rank + c) across the lists a chunk appears in, rewarding consistent high ranks without trusting raw, uncalibrated similarity scores.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating multi-query and decomposition as the same thing — one varies the wording of a single need, the other splits genuinely distinct sub-needs.

60 second bullets to scan on the way to the call.

What single-embedding brittleness means and why it limits recall
How multi-query generates paraphrases and fuses their hits

Primary sources. Browse if you want the original framing.