Define ColBERT's late-interaction approach and its tradeoff against pooled embeddings

A standard bi-encoder produces one pooled vector per document, typically by averaging the token embeddings or taking the `[CLS]` token. **ColBERT** skips the pooling. Each document is stored as a set of vectors, one per token, and the relevance score is computed over the whole set at query time. The scoring function is **MaxSim**. The query is also tokenized and embedded into one vector per token. For each query token, MaxSim finds the document token with the highest cosine similarity, then sums those per query token maxima to get the final score. The phrase *late interaction* describes when the query and document tokens interact. In a bi-encoder they never interact: each side is pooled before comparison. In a cross-encoder they interact early: they enter the transformer together. In ColBERT they interact late, after both sides are encoded independently but before the final score is computed. The benefit is fine-grained matching. A relevant token in the document cannot be averaged away by an irrelevant majority. The cost is storage: each document stores one vector per token instead of one per document, which is 10-50× more data. Quality improvements over bi-encoders are typically 5-15 points on standard benchmarks.

**Late interaction** is the architectural choice to keep query and document tokens as independent vectors until score time, and to compare them token by token at query time. Standard bi-encoders pool token embeddings into one vector per document; ColBERT skips pooling and stores one vector per token. The relevance score is MaxSim: $$\text{MaxSim}(q, d) = \sum_{i \in q} \max_{j \in d} \cos(\mathbf{q}_i, \mathbf{d}_j)$$ For each query token, find the best-matching document token; sum across query tokens. The name *late interaction* refers to when the per-token comparison happens: late in the pipeline, at query time, after both sides have been encoded independently. **What it buys you.** Quality: fine-grained term-level evidence is preserved because nothing is averaged away by pooling. On BEIR-style benchmarks, ColBERTv2 typically lifts NDCG@10 by 5-15 points over equivalent bi-encoders. The lift is largest on datasets where relevance depends on a few specific tokens (named entities, technical identifiers, numeric constraints). **What it costs.** Storage is 10-50× a pooled bi-encoder because you store one vector per token instead of one per document. Query latency is heavier than a single ANN lookup because MaxSim has to compare every query token against many document tokens. **Mitigations.** ColBERTv2 quantizes vectors and stores centroid + residual representations, reducing storage to roughly 2-5× pooled. PLAID adds a clustered index for fast approximate MaxSim. In 2026, most production stacks still default to bi-encoder + cross-encoder rerank because the operational complexity is lower; ColBERT-style retrieval is the right call when one-stage quality matters and the corpus is mid-sized.