What is semantic chunking and when does it beat fixed size chunking?

**Semantic chunking** is a way of splitting text into pieces by meaning rather than by token count. The algorithm walks through the document, embeds each sentence or short window, computes the embedding similarity between adjacent windows, and cuts wherever the similarity drops sharply. Drops mark places where the topic shifts. Fixed-size chunking is the simpler alternative. It just counts characters or tokens and cuts at a fixed length, ignoring what the text is actually about. This is cheap and predictable, but it sometimes cuts mid-sentence or mid-paragraph, producing chunks that mix two unrelated ideas. Semantic chunking wins when the documents are long and heterogeneous. Think of a research paper that includes background, method, results, and discussion in one file, or a customer support article that bundles unrelated topics. Fixed-size splits across these boundaries produce chunks whose embeddings are a confused average of two topics, which hurts retrieval. Fixed-size chunking is fine, and sometimes preferable, when documents are short, uniform, or already structured by clear headings. The extra cost of computing per-sentence embeddings to find boundaries does not pay off when there are no interesting boundaries to find.

**Semantic chunking** splits documents at points where adjacent-sentence embedding similarity drops below a threshold, which corresponds to topic shifts. Fixed-size chunking ignores content and cuts at a fixed token count. The algorithmic difference is small but consequential. Semantic chunking embeds each sentence, computes pairwise similarities along the sequence, and inserts a break wherever the rolling similarity drops below a tuned threshold. Fixed-size simply increments a token counter. Semantic chunking wins on long, heterogeneous documents where fixed splits would slice mid-thought. Examples include long-form articles, multi-topic reports, customer-support documentation with mixed subjects, and book-length material. The improvement shows up in top-k retrieval precision because each chunk's embedding describes a coherent thought. Fixed-size wins on short, uniform documents and on ingestion paths with tight latency budgets. Computing per-sentence embeddings at indexing time multiplies the embedding cost, sometimes by an order of magnitude, and that cost is wasted when documents lack interesting topic boundaries. The practical recommendation is to evaluate on a labelled retrieval benchmark for the actual corpus, since the answer depends heavily on what the documents look like rather than on theory.