Chunking
Also known as: Document chunking, Text splitting
Cutting documents into retrievable pieces before embedding them.
The process of splitting documents into smaller segments for embedding and retrieval in a RAG pipeline. Chunk size, overlap, and boundary strategy (sentence, paragraph, semantic) critically affect retrieval quality.
In practice
Chunk size and overlap directly determine recall and answer quality, and they're the most common knob interviewees forget to tune.
Related topics
Practice questions
Questions that mention this term
Related terms
Retrieval-Augmented Generation (RAG)
Fetch relevant docs at inference time and stuff them into the prompt so the model can answer from real data.
Context Window
The max number of tokens a model can attend to at once.
Hallucination
When a model confidently makes up something that isn't true.
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
HNSW (Hierarchical Navigable Small World)
A graph-based ANN index with log-time approximate search, excellent recall/latency trade-off.
Lost in the Middle
Models attend best to the start and end of long contexts and lose info in the middle.