Semantic Chunking
Also known as: Topic-based chunking, Embedding-boundary chunking
Split at topic boundaries by detecting big embedding drops between adjacent sentences.
A chunking strategy that embeds each sentence and inserts a break whenever the embedding distance between adjacent sentences exceeds a threshold. Chunks correspond to topics or ideas, improving embedding quality on long-form prose. Costs an extra embedding pass at ingest and produces variable-length chunks.
In practice
The natural next step after fixed-size when recall drops on articles or transcripts. Interviewers ask about the trade-off between coherent chunks and predictable token cost.
How it compares
Fixed-size ignores meaning; semantic uses embedding-similarity gaps between adjacent sentences to place breaks.
Semantic follows meaning boundaries; recursive follows the document's structural boundaries (sections, paragraphs, sentences).
Comparisons that include Semantic Chunking
Related topics
Questions that mention this term
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Context Window
The max number of tokens a model can attend to at once.