Semantic Chunking
Also known as: Topic-based chunking, Embedding-boundary chunking
Split at topic boundaries by detecting big embedding drops between adjacent sentences.
A chunking strategy that embeds each sentence and inserts a break whenever the embedding distance between adjacent sentences exceeds a threshold. Chunks correspond to topics or ideas, improving embedding quality on long-form prose. Costs an extra embedding pass at ingest and produces variable-length chunks.
In practice
The natural next step after fixed-size when recall drops on articles or transcripts. Interviewers ask about the trade-off between coherent chunks and predictable token cost.
How it compares
Fixed-size ignores meaning; semantic uses embedding-similarity gaps between adjacent sentences to place breaks.
Semantic follows meaning boundaries; recursive follows the document's structural boundaries (sections, paragraphs, sentences).
Comparisons that include Semantic Chunking
Related topics
Questions that mention this term
Related terms
Retrieval-Augmented Generation (RAG)
Fetch relevant docs at inference time and stuff them into the prompt so the model can answer from real data.
Embeddings
Dense numeric vectors that capture meaning; close vectors = similar text.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Window
The max number of tokens a model can attend to at once.
Hallucination
When a model confidently makes up something that isn't true.
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.