Pick the failure pattern that points to chunk size being too large rather than too small

MCQ·Medium·4.0 · 0·~1 min·Asked atAmd Deepseek Patronus

Attempt it

Chunk size is one of the most important knobs in any RAG pipeline, and the failure shapes tell you which direction to tune. **Too-large chunks** bring extra material along for the ride. The right paragraph is in there, but so are a few unrelated paragraphs that happened to share the same page or section. The signal to noise ratio per chunk drops. The model has to ignore more to find the answer, and the context block bloats unnecessarily. **Too-small chunks** split coherent facts. A fact that spans three sentences across the chunk boundary gets cut in half. The retrieval system finds one half (the one that matched the query) and misses the other half. The model receives partial evidence. Walk through the options: - **0. Missing specific facts that appear together in the source.** This is the too-small signature. The fact spans more than one chunk and only part of it gets retrieved. - **1. Right topic plus unrelated paragraph.** This is the too-large signature. Dilution, relevant content arrives bundled with irrelevant content. - **2. Near-duplicate chunks in top-k.** This is an embedding redundancy or no-MMR problem, not a chunk-size problem. The chunks themselves can be the right size; the retrieval just keeps fetching very similar ones. - **3. Reranker cannot tell two chunks apart.** This is a reranker capacity issue. A stronger reranker (Cohere Rerank v3, Voyage Rerank-2) handles this; chunk size is unrelated. The correct answer is option 1.

**The dilution signature is the giveaway.** Each retrieved chunk being on-topic but accompanied by unrelated material is what oversized chunks produce. ### Why oversized chunks dilute A RAG pipeline fetches the top-k chunks by embedding similarity to the query. The embedding model's view of a chunk is a single vector representing the chunk's average meaning. When the chunk is 1,200 tokens long and contains three different topics, the embedding represents the weighted average of those topics. A query about topic A retrieves the chunk because topic A is in there, but the chunk also delivers topics B and C as collateral. The model now has to do extra work: read the whole chunk, find the relevant paragraph, ignore the irrelevant ones. Two production consequences: - **Context bloat**: the top-k now consumes more tokens per chunk, so fewer chunks fit in the budget. You traded depth for width and lost the depth. - **Lost-in-the-middle**: oversized chunks push relevant content into the middle of the context, where attention is weakest. ### Why each other option is a different failure **Option 0, missing co-located facts.** This is the too-small signature. A fact that spans multiple sentences ('The XYZ standard, defined in section 4.3, requires a minimum of 8 bytes...') gets cut in the middle by aggressive chunking. The retriever finds the chunk with 'XYZ standard, section 4.3' but misses the chunk with 'minimum of 8 bytes.' Fix: enlarge chunks or add semantic boundary respecting chunking. **Option 2, near-duplicate chunks in top-k.** This is an embedding redundancy problem. Either the corpus has many similar documents (e.g., versioned docs, FAQ duplicates), or the retrieval lacks diversity (no MMR, no max marginal relevance). Fix: enable MMR with a lambda around 0.5-0.7; deduplicate the corpus at index time. **Option 3, reranker cannot separate two chunks.** This is a reranker capacity issue.

TL;DR

Right topic plus unrelated paragraph is the dilution signature of oversized chunks. Missing co-located facts is the opposite failure.

Memory aid

Easy to grasp

Imagine you ask a librarian for a book chapter on cats. Three things can go wrong. First, the librarian hands you a single page about cats but the next sentence is about somebody's grandmother, because the page got cut in the middle of the cat chapter. That is chunks too small. Second, the librarian hands you the cat chapter plus the dog chapter plus a recipe for soup, because all three were stapled together. The cat content is in there but you have to read past a lot of noise to find it. That is chunks too large. Third, the librarian gives you five copies of the same page. That is a problem with the librarian's index, not with the page size. The question asks for the second one.

Key concepts

Context engineering

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Chunk size is the most-tuned knob in RAG pipelines because it sits at the boundary between two failure modes that look superficially similar but have opposite fixes. Too-large chunks dilute. Too-small chunks split. Both produce 'retrieval is broken' complaints from end users, but the right intervention is different.

This MCQ tests whether the candidate can read failure shapes and distinguish them. The dilution shape, right topic plus unrelated material in the same chunk, is the signature of oversized chunks. The split shape, co-located facts missing because they straddle a chunk boundary, is the signature of undersized chunks. The other two options in the question (near-duplicates, reranker confusion) are different failure modes entirely; they live on the diversity and reranker axes, not on the chunk-size axis.

The production lesson buried in the question is that diagnosis comes from reading chunk contents, not from staring at retrieval scores. A 30-minute pass through a failing eval set's top-3 chunks per query usually surfaces the dominant failure mode, and the right fix follows. Teams that tune chunk size based on intuition without inspecting chunks tend to overshoot in one direction and then overshoot back.

The two chunk-size failures and how they look

Too-large chunks produce dilution. Each retrieved chunk contains the right paragraph plus extra material that happened to be nearby in the source document. The embedding model represents the chunk as a single vector that averages all the topics in it; a query about topic A retrieves the chunk because topic A is in there, but the chunk also delivers topics B and C.

The production effect is a context block that holds too much noise per slot. If your budget allows for 5 retrieved chunks and each chunk is 1,200 tokens with only 400 useful tokens, the model is reading 4,000 useful tokens disguised as 6,000 tokens of input. The downstream consequence is lost-in-the-middle: the useful content is buried in the middle of long chunks, exactly where attention is weakest.

Too-small chunks produce split facts. A multi-sentence fact,'the XYZ standard, defined in RFC 8259 section 4.3, requires keys to be unique within a single object', gets cut by aggressive chunking. The chunk with 'XYZ standard, defined in RFC 8259' matches the query, the chunk with 'requires keys to be unique within a single object' does not match (because it lacks the keywords from the query), and the retrieved evidence is incomplete.

The production effect is the model 'almost' having the answer. Users see responses that are confidently wrong or that hedge with 'I don't have specific information about X' when the source document literally contains X, just split across two chunks the retriever did not both fetch.

The two failures look different in retrieved-chunk inspection. Dilution: 'the chunks I retrieved have the right topic but they are long and mostly noise.' Split: 'I keep getting one half of an answer.' Read the chunks; the diagnosis is usually obvious.

Why options 2 and 3 are not chunk-size problems

2026 starting points and per-corpus tuning

The diagnostic playbook

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Symptom in retrieved chunks	Likely cause	Fix
On-topic but with unrelated material attached	Chunks too large	Reduce chunk size; semantic-boundary chunking
Co-located facts missing across chunks	Chunks too small	Increase chunk size; add overlap
Near-duplicate chunks in top-k	Diversity gap; corpus redundancy	Enable MMR; deduplicate at index time
Reranker cannot separate two chunks	Reranker capacity	Upgrade reranker (Cohere v3, Voyage 2)
Answer absent from retrieved chunks entirely	Retrieval/embedding mismatch	Try hybrid search; better embeddings

Real products, models, and research that use this idea.

LlamaIndex's SemanticSplitterNodeParser chunks at semantic boundaries rather than fixed token counts, reducing the too-small failure mode in 2026 RAG stacks.
LangChain's RecursiveCharacterTextSplitter is the default chunker for most LangChain-based agents; teams typically tune chunk size 300-500 with 50-100 overlap.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does chunk overlap interact with rerank?

Overlap creates near-duplicate candidates that rerank should consolidate. A strong reranker handles this; a weak one ranks both halves of an overlap pair high and wastes a slot. If overlap is creating duplicate-looking top-k, the reranker is the lever, not chunk size.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Picking option 0 (missing co-located facts) because it 'sounds like a retrieval problem.' It is the opposite failure mode, chunks too small to span the fact. Read the diagnostic shape, not the vibe.

60 second bullets to scan on the way to the call.

The dilution signature that points to chunks being too large
The split-fact signature that points to chunks being too small

Primary sources. Browse if you want the original framing.