Why does MMR appear specifically at the context-assembly stage of a RAG pipeline?

Flashcard·Medium·4.0 · 0·~30s·Asked atCohere Databricks OpenAI

Attempt it

A vector store happily returns five chunks that all say nearly the same thing because they all have high cosine similarity to the query. Plain top-k is a popularity contest; it has no objection to redundancy. When you concatenate those five chunks into the context, you have spent five chunk-slots on one fact and zero on neighbouring facts the model also needs. MMR rebalances at the assembly stage: each next pick maximises relevance to the query minus a penalty for similarity to chunks already selected. The result is a context block that is still topical but spreads its budget across distinct facts.

Your answer

MMR, Maximal Marginal Relevance, is a way to pick chunks for the context block that balances two things at once: how relevant each chunk is to the query, and how different it is from chunks already picked. The problem MMR solves is concrete. Plain top-k retrieval ranks chunks by similarity to the query and nothing else. If five chunks in the corpus all say nearly the same thing because the same fact is repeated across documents, all five will score similarly and all five will end up in the top-5. The context block is now five copies of one fact. MMR rebalances at the assembly stage. It still starts with the candidate set from retrieval, but instead of taking the top-k by relevance alone, it picks chunks one at a time using a scoring rule that subtracts a penalty for similarity to chunks already selected. The formula in plain terms is: next chunk = max over candidates of (relevance to query) minus lambda times (max similarity to chunks already picked). A lambda of 0 is pure relevance, same as top-k. A lambda of 1 is pure diversity, ignores the query. Lambda is usually 0.5 to 0.7, which keeps relevance dominant and uses diversity as a tiebreaker. The result is a context block where the chunks are still on-topic but they spread the budget across distinct facets of the topic instead of stacking redundant copies of one.

MMR is a context-engineering tool, not a retrieval tool, because it operates on the candidate set after retrieval and decides which subset ends up in the assembled context block. The job it does, trade a little relevance for a little diversity, only makes sense once you are deciding what goes into a context budget. The failure mode MMR prevents is **redundant context**. Plain top-k similarity has no objective term for diversity. If a knowledge base contains the same fact phrased five different ways, all five chunks will score similarly to a query about that fact and all five will land in the top-5. The context block ends up spending its entire chunk budget communicating one fact, leaving zero slots for neighbouring facts the model also needs. MMR's scoring rule, picking chunks greedily one at a time: $$\text{MMR}(d) = \lambda \cdot \text{sim}(q, d) - (1 - \lambda) \cdot \max_{d' \in S} \text{sim}(d, d')$$ where q is the query, `d` is a candidate, `S` is the set already selected, and `lambda` is the relevance versus diversity tradeoff. Lambda close to 1 is mostly relevance (similar to plain top-k); lambda close to 0 is mostly diversity (ignores the query). Production defaults sit around 0.5 to 0.7. Where MMR lives in the pipeline matters. It runs after retrieval (you need a candidate set) and after reranking (you want the reranker's quality signal as the relevance score), but before the final assembly. The output is the ordered chunk list that actually gets concatenated into the context block.

Quick reference

TL;DR

Plain top-k has no penalty for redundancy, so it can spend five slots on one fact; MMR runs at assembly and trades a little relevance for diversity so the slots cover distinct facts.

Memory aid

Easy to grasp

Picture packing a lunchbox. You like apples, so you fill all five slots with apples. You technically maximized apple-love, but now you have no sandwich, no juice, no chips, no snack. A smarter packer says yes apples are great, but pick one apple and use the other four slots for different foods that still go with the meal. That is what MMR does for a context block. Plain top-k stuffs the box with apples because apples score highest. MMR keeps the best apple and spends the remaining slots on neighbouring things the model also needs.

Key concepts

Context engineering Retrieval-augmented generation Embeddings

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 minutes: define MMR, state the score formula in words, explain why top-k cannot do this job, place MMR in the retrieve dedupe rerank-MMR pipeline, mention lambda defaults around 0.7.

Code example

python

import numpy as np

def mmr_select(
    query_emb: np.ndarray,
    cand_embs: np.ndarray,        # shape (n, d)
    rel_scores: np.ndarray,        # reranker scores, shape (n,)
    k: int = 5,
    lam: float = 0.7,
) -> list[int]:
    selected: list[int] = []
    remaining = list(range(len(cand_embs)))
    sim_matrix = cand_embs @ cand_embs.T  # (n, n)
    while len(selected) < k and remaining:
        if not selected:
            best = max(remaining, key=lambda i: rel_scores[i])
        else:
            def score(i: int) -> float:
                max_sim = max(sim_matrix[i, j] for j in selected)
                return lam * rel_scores[i] - (1 - lam) * max_sim
            best = max(remaining, key=score)
        selected.append(best)
        remaining.remove(best)
    return selected

Real-world examples

Real products, models, and research that use this idea.

LlamaIndex 2026 ships an MMR post-processor that runs after retrieval and reranking, with default lambda 0.7.
LangChain's MMR retriever exposes lambda as a direct config parameter and integrates with Chroma, FAISS, Pinecone, Qdrant.
Cohere's RAG cookbooks recommend MMR after Cohere Rerank 3.5 when the corpus is known to contain near-duplicates.
Vespa supports diversity-aware retrieval natively as a query operator, used in production at major search-quality teams.
Anthropic's Claude 4.7 RAG reference architecture for projects mode applies MMR-style diversification before final assembly to keep context-block chunks distinct.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you choose lambda for a new workload?

QWhy does MMR sit after reranking rather than before?

QWhat is the cheapest alternative to MMR if it adds too much complexity?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Setting MMR's lambda too low (heavy diversity penalty) so the assembly drifts to off-topic chunks. Lambda should bias toward relevance and use diversity as a tiebreaker.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.