Chunking
Also known as: Document chunking, Text splitting
Cutting documents into retrievable pieces before embedding them.
The process of splitting documents into smaller segments for embedding and retrieval in a RAG pipeline. Chunk size, overlap, and boundary strategy (sentence, paragraph, semantic) critically affect retrieval quality.
In practice
Chunk size and overlap directly determine recall and answer quality, and they're the most common knob interviewees forget to tune.
Related topics
Practice questions
Questions that mention this term
- Predict how cl100k_base tokenizes…
- When does binary quantization…
- Why does a cross-encoder reranker pay off even though it is slower than a bi-encoder retriever?
- How does Sarathi-Serve's chunked…
- Which of the following content types belong in the SYSTEM message (not the user message) in a production chat style prompt?
- Order the slots to…
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Context Window
The max number of tokens a model can attend to at once.
Fixed-size Chunking
Slice documents every N tokens with a small overlap. Fast and format-agnostic, but cuts through headings and mid-topic.