Why does MMR appear specifically at the context assembly stage of a RAG pipeline?

Flashcard·Medium·4.0 · 0·~30s·Asked atAccenture Freshworks Gong

Attempt it

A vector store happily returns five chunks that all say nearly the same thing because they all have high cosine similarity to the query. Plain top-k is a popularity contest; it has no objection to redundancy. When you concatenate those five chunks into the context, you have spent five chunk-slots on one fact and zero on neighbouring facts the model also needs. MMR rebalances at the assembly stage: each next pick maximises relevance to the query minus a penalty for similarity to chunks already selected. The result is a context block that is still topical but spreads its budget across distinct facts.

Your answer

MMR, Maximal Marginal Relevance, is a way to pick chunks for the context block that balances two things at once: how relevant each chunk is to the query, and how different it is from chunks already picked. The problem MMR solves is concrete. Plain top-k retrieval ranks chunks by similarity to the query and nothing else. If five chunks in the corpus all say nearly the same thing because the same fact is repeated across documents, all five will score similarly and all five will end up in the top-5. The context block is now five copies of one fact. MMR rebalances at the assembly stage. It still starts with the candidate set from retrieval, but instead of taking the top-k by relevance alone, it picks chunks one at a time using a scoring rule that subtracts a penalty for similarity to chunks already selected. The formula in plain terms is: next chunk = max over candidates of (relevance to query) minus lambda times (max similarity to chunks already picked). A lambda of 0 is pure relevance, same as top-k. A lambda of 1 is pure diversity, ignores the query. Lambda is usually 0.5 to 0.7, which keeps relevance dominant and uses diversity as a tiebreaker. The result is a context block where the chunks are still on-topic but they spread the budget across distinct facets of the topic instead of stacking redundant copies of one.

MMR is a context-engineering tool, not a retrieval tool, because it operates on the candidate set after retrieval and decides which subset ends up in the assembled context block. The job it does, trade a little relevance for a little diversity, only makes sense once you are deciding what goes into a context budget. The failure mode MMR prevents is **redundant context**. Plain top-k similarity has no objective term for diversity. If a knowledge base contains the same fact phrased five different ways, all five chunks will score similarly to a query about that fact and all five will land in the top-5. The context block ends up spending its entire chunk budget communicating one fact, leaving zero slots for neighbouring facts the model also needs. MMR's scoring rule, picking chunks greedily one at a time: $$\text{MMR}(d) = \lambda \cdot \text{sim}(q, d) - (1 - \lambda) \cdot \max_{d' \in S} \text{sim}(d, d')$$ where q is the query, `d` is a candidate, `S` is the set already selected, and `lambda` is the relevance versus diversity tradeoff. Lambda close to 1 is mostly relevance (similar to plain top-k); lambda close to 0 is mostly diversity (ignores the query). Production defaults sit around 0.5 to 0.7. Where MMR lives in the pipeline matters. It runs after retrieval (you need a candidate set) and after reranking (you want the reranker's quality signal as the relevance score), but before the final assembly. The output is the ordered chunk list that actually gets concatenated into the context block.

TL;DR

Plain top-k has no penalty for redundancy, so it can spend five slots on one fact; MMR runs at assembly and trades a little relevance for diversity so the slots cover distinct facts.

Memory aid

Easy to grasp

Picture packing a lunchbox. You like apples, so you fill all five slots with apples. You technically maximized apple-love, but now you have no sandwich, no juice, no chips, no snack. A smarter packer says yes apples are great, but pick one apple and use the other four slots for different foods that still go with the meal. That is what MMR does for a context block. Plain top-k stuffs the box with apples because apples score highest. MMR keeps the best apple and spends the remaining slots on neighbouring things the model also needs.

Key concepts

Context engineering Retrieval-augmented generation Embeddings

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Plain top-k retrieval ranks each candidate against the query independently. It has no mechanism for noticing that two candidates are similar to each other, which means it cannot stop itself from filling a context block with five copies of the same fact. Maximal Marginal Relevance is the principled fix, a selection rule that trades a little relevance for a little diversity at the assembly stage.

This card walks through the score formula, where MMR lives in the pipeline, and how it relates to dedupe and reranking.

The failure mode and why retrieval cannot fix it

Retrieval is a unary scoring problem. For each candidate chunk, the retriever computes a score against the query, cosine similarity, BM25, or a reranker score, and returns the top-k by that score. There is no pairwise term that says 'this candidate is too similar to that other one'.

That shape is fine when the corpus has no near-duplicates. It breaks when the same fact appears in multiple chunks. A query about that fact will retrieve all the near-duplicate chunks because they all score similarly against the query. The context block becomes five copies of one fact.

Why this has to be solved at assembly

The retriever does not know what context block is being assembled or what the budget is. Two near-duplicates might both be useful, one might be in a different version of the document, one might be from a more authoritative source, and a hard-coded retrieval-time dedupe would lose that information.

The assembly stage knows the budget and the already-selected chunks. That is the natural place to apply a set-aware rule.

The MMR scoring rule

Where MMR fits in the pipeline

When MMR is overkill and how to tune it

Situations where this technique stops working.

2–4 min · Everything important, quickly.

python

import numpy as np

def mmr_select(
    query_emb: np.ndarray,
    cand_embs: np.ndarray,        # shape (n, d)
    rel_scores: np.ndarray,        # reranker scores, shape (n,)
    k: int = 5,
    lam: float = 0.7,
) -> list[int]:
    selected: list[int] = []
    remaining = list(range(len(cand_embs)))
    sim_matrix = cand_embs @ cand_embs.T  # (n, n)
    while len(selected) < k and remaining:
        if not selected:
            best = max(remaining, key=lambda i: rel_scores[i])
        else:
            def score(i: int) -> float:
                max_sim = max(sim_matrix[i, j] for j in selected)
                return lam * rel_scores[i] - (1 - lam) * max_sim
            best = max(remaining, key=score)
        selected.append(best)
        remaining.remove(best)
    return selected

Real products, models, and research that use this idea.

LlamaIndex 2026 ships an MMR post-processor that runs after retrieval and reranking, with default lambda 0.7.
LangChain's MMR retriever exposes lambda as a direct config parameter and integrates with Chroma, FAISS, Pinecone, Qdrant.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you choose lambda for a new workload?

Start at 0.7. A/B against 0.5 and 0.85 on a labelled eval. Higher lambda for narrow precision queries; lower for browse or exploratory.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Setting MMR's lambda too low (heavy diversity penalty) so the assembly drifts to off-topic chunks. Lambda should bias toward relevance and use diversity as a tiebreaker.

60 second bullets to scan on the way to the call.

State the MMR scoring rule in plain terms
Explain why plain top-k cannot penalize candidate redundancy

Primary sources. Browse if you want the original framing.