When does query expansion help context assembly, and when does it just inflate noise?

Attempt it

Short user queries (think two or three words) under-specify the embedding lookup and miss documents that use different vocabulary. LLM-driven expansion produces a small fan of paraphrases, retrieves for each, and unions the results so synonym-only matches reach the assembly stage. The downside is also predictable: expansion broadens the retrieval cone, which pulls in more loosely relevant chunks, and the final context block bloats unless you rerank and trim hard after the union. Use it when recall is the bottleneck (the right chunk is in the KB but plain vector search misses it); skip it when precision is the bottleneck (top-k already returns the right chunks).

Your answer

Quick reference

TL;DR

Expansion paraphrases the query into a small fan and unions the results to lift recall on short under-specified queries; the cost is broader noise unless reranking and trimming clean up after the union.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine you ask a librarian for help with one search term and they only check that one term. If the book uses different words, they miss it. Query expansion is asking the librarian to also try two or three closely related phrases just in case. You will find more relevant books, but you will also find some only loosely related ones. The fix is to look at the whole pile and let a better filter pick the actual best ones. Expansion alone is half a solution. Expansion plus a strong second-pass filter is the full move.

Key concepts

Context engineering Retrieval-augmented generation Embeddings

Quick scan · 2–4 min

Everything important, quickly.

4 minutes: define query expansion, state the recall vs precision tradeoff, walk through the union rerank trim pipeline, mention HyDE as a sibling variant, name two workloads where expansion is the wrong default.

Code example

python

def expand_query(original: str, n: int = 3) -> list[str]:
    prompt = (
        f"Generate {n} different ways to phrase this query, each a single line, "
        f"covering different vocabulary the corpus might use. Original: {original}"
    )
    paraphrases = llm.complete(prompt).strip().split("\n")
    return [original] + paraphrases[:n]

def retrieve_with_expansion(user_query: str, k1: int = 20) -> list[Chunk]:
    queries = expand_query(user_query, n=3)
    # union and dedupe across all retrievals
    seen: dict[str, Chunk] = {}
    for q in queries:
        for c in vector_db.search(q, top_k=k1):
            seen.setdefault(c.id, c)
    # rerank against the ORIGINAL query, not the paraphrases
    candidates = list(seen.values())
    return reranker.rerank(user_query, candidates, top_n=5)

Real-world examples

Real products, models, and research that use this idea.

LangChain's MultiQueryRetriever (2026) wraps the expansion and union pattern as a single retriever class with configurable fan-out.
LlamaIndex's MultiStepQueryEngine and HyDEQueryTransform expose expansion variants as composable pipeline stages.
Cohere's RAG cookbooks recommend 2-3 paraphrases plus Cohere Rerank 3.5 against the original query as the standard expansion pipeline.
Anthropic's Claude 4.7 RAG reference for short conversational queries uses LLM expansion via a tool call before retrieval against the corpus.
Pinecone and Weaviate both ship multi-query convenience APIs that union results from a fan-out of embeddings.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow is HyDE different from query expansion?

Sign in or go Pro to see the approach for this follow-up.

QHow would you tell, from a labelled eval, whether expansion will help?

Sign in or go Pro to see the approach for this follow-up.

QWhat happens when you generate too many paraphrases?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.