Proposition based indexing and the granularity problem it solves

Short answer·Medium·4.0 · 0·~3 min·Asked atInflection Ai Jane Street Runway

Attempt it

Explain proposition based indexing. How does it differ from chunking a document into fixed spans, and what retrieval problem does it solve?

Free · 2 AI evals / day

Most RAG systems chop documents into fixed-size chunks — windows of a few hundred tokens cut wherever the length math lands. The problem is that one chunk usually carries several different ideas plus pronouns and references that only make sense from earlier text. When you embed that chunk, its vector is a blurry average of everything inside it. That blur hurts retrieval. A query matches the chunk partly on the one relevant sentence and partly on the unrelated text bundled with it. The relevant fact still gets retrieved, but it arrives surrounded by noise the generator has to wade through. **Proposition-based indexing** changes the unit of retrieval. Instead of arbitrary spans, you first decompose each document into atomic, self-contained factual statements — each proposition expresses one fact, with all references resolved so it stands alone. You embed each proposition individually. Because every indexed item is now a single clean claim, similarity matching gets much sharper. A query retrieves exactly the facts it needs, with far less distractor noise. The cost is an extra LLM pass at ingestion. A single fact can also lose the context a fuller passage carries, so you usually keep a link back to the source passage.

**Proposition-based indexing** attacks the granularity problem at the root: it makes the unit of retrieval a single atomic fact rather than an arbitrary span of text. Fixed-span chunking cuts documents into windows decided by length, not meaning. A typical chunk therefore mixes several ideas and contains pronouns and references — 'it', 'this approach', 'the company' — that only resolve from text outside the chunk. The chunk's embedding is a muddy average of everything in it, so a query matches partly on the one relevant sentence and partly on the unrelated material bundled alongside. The right fact is retrieved buried in distractor noise. Proposition indexing instead runs a decomposition step that rewrites each document into a list of atomic, self-contained statements. Each proposition expresses exactly one fact, with references resolved so it reads correctly on its own, and each is embedded as its own retrievable unit. Because the retrieval unit is now one clean claim, similarity matching is far sharper. The query pulls exactly the facts it needs, and the generator's context carries far less irrelevant text. The tradeoffs are real and worth stating in an interview. Generating propositions is an extra LLM preprocessing pass over the whole corpus — real cost and latency at ingestion. Atomic facts can also lose the connective context a fuller passage provides. So you typically pair each proposition with a pointer back to its source passage, fetched when more context is needed.

TL;DR

Proposition indexing replaces arbitrary fixed spans with atomic, self-contained facts as the retrieval unit, sharpening precision — at the cost of an LLM preprocessing pass and lost connective context.

Memory aid

Easy to grasp

Imagine you have a thick textbook and you want to answer quick questions from it. One way is to photocopy random half-page chunks and toss them in a box; when you search, you grab a half page that mentions your topic but also a lot of unrelated stuff. The other way is to first read the book and write out every single fact on its own index card, rewritten so each card makes sense by itself — no 'it' or 'this' that points back to a missing sentence. Now when you search, you pull exactly the cards that answer the question, with no clutter. Writing all those cards is slow work up front, and a single card can lose the story around it, but each card is razor-sharp.

Key concepts

Retrieval-Augmented Generation Embeddings Vector Databases

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Every RAG system makes one decision that quietly governs its ceiling: what is the unit of retrieval? That unit is what gets embedded, what gets matched against a query, and what lands in the generator's context. Fixed-span chunking is the default answer, and it is a compromise — every chunk has to be simultaneously big enough to make sense alone and small enough to be precise, two goals that pull in opposite directions.

Proposition-based indexing refuses the compromise by changing the unit entirely. Instead of arbitrary windows of text, it indexes atomic facts. This walkthrough explains why fixed spans produce noisy retrieval, what a proposition actually is, how it sharpens similarity matching, and the costs that mean it almost always ships as part of a small to big architecture rather than on its own.

The granularity problem with fixed spans

A fixed-span chunker cuts a document every N tokens. The resulting chunk is a window of text chosen by counting, so it routinely contains several distinct ideas — a definition, an example, a caveat, an unrelated aside — all bundled together. It also contains references that point outside itself: pronouns like 'it' and 'they', and phrases like 'this method' or 'the company' that only resolve from text the chunk does not include.

When you embed that chunk into a vector, the embedding is a blend of everything in it. A vector that has to represent four ideas at once sits in vector space at a kind of average position, close to none of the four cleanly. This is the granularity problem: the matching unit is too coarse to represent any single fact precisely.

The consequence at query time is imprecise retrieval. The chunk gets pulled because one sentence in it is relevant, but its similarity score is dragged around by the unrelated material. You retrieve the right fact wrapped in distractors, and those distractors then sit in the generator's context where they invite lost-in-the-middle errors and dilute the signal the model should be focusing on.

What a proposition is

Why precision improves

Costs, limits, and the small to big pairing

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

The 'dense X retrieval' line of work showed that indexing propositions instead of passages improves retrieval precision on open-domain QA benchmarks.
LlamaIndex and LangChain expose proposition / atomic-statement extraction nodes that run an LLM decomposition pass during ingestion.

What an interviewer would ask next. Try answering before peeking at the approach.

QThe decomposition LLM can drop or hallucinate facts. How do you stop a bad preprocessing pass from silently corrupting the index?

Discuss evaluating the decomposer (coverage and faithfulness of propositions against the source), grounding each proposition to a source span, and spot-checking or LLM-judging a sample before bulk ingestion.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Indexing propositions but never keeping a link back to their source passage — so when a query needs the surrounding context or a citation, the clean fact has nowhere to expand to.

60 second bullets to scan on the way to the call.

Explain why a fixed-span chunk embedding is a noisy average of mixed ideas
Define a proposition with the atomic and self-contained properties

Primary sources. Browse if you want the original framing.