A source document is edited: sequence the incremental index update steps

Order steps·Medium·4.0 · 0·~1 min·Asked atBaseten Canva Cerebras

Attempt it

1Detect the change by document id or content hash
2Re-chunk and re-embed the new version of the document
3Delete the old chunks and vectors for that document id
4Refresh any derived caches (semantic cache, summaries)
5Upsert the new vectors into the index

Incremental indexing means updating only what changed instead of rebuilding the whole index. When a source document is edited, the pipeline runs five steps in a specific order. First, **detect the change** — usually by comparing a content hash or a last-modified timestamp against the document id you already stored. Nothing should happen until you know a document actually changed. Next, delete the old chunks and vectors tied to that document id. This is the step people get wrong: you remove stale data *before* adding the new data, so the index never holds two versions of the same document at once. Then re-chunk and re-embed the new version, and upsert those fresh vectors into the index. Only now is the document searchable in its current form. Finally, refresh any derived caches — a semantic cache of past answers, document summaries, or precomputed retrieval results. Those depend on the underlying vectors, so they can only be rebuilt once the new vectors are live. Refresh them last, or they will keep serving answers grounded in the old text.

The correct sequence is detect, delete, re-embed, upsert, refresh caches. Two constraints drive the ordering. First, don't do work you don't need to. Second, never let stale and current copies of the same document coexist in the searchable index. **Detect first**. Compare a content hash or modified timestamp keyed by document id. This gates the whole pipeline — re-embedding is the expensive step, so you only pay it when content genuinely changed, not on every sync. **Delete before write**. Remove the old chunks and vectors for that document id before upserting the new ones. If you upsert first, there is a window where retrieval can return both the old and new chunks for the same source. The model may then ground its answer in outdated text or surface duplicate, contradictory hits. Deleting first closes that window. Re-chunking can also move chunk boundaries and counts, so old chunk ids may not line up with new ones. A clean delete avoids orphaned stale vectors. **Re-chunk, re-embed, upsert**. Produce the new chunks, embed them with the same model and settings as the rest of the index, and write them in. **Refresh derived caches last**. A semantic cache, summaries, or precomputed results all depend on what retrieval returns. They can only be correctly rebuilt once the new vectors are in place; refresh them earlier and they capture the old state. In strict-consistency systems you instead invalidate these caches at delete time and let them lazily repopulate.

TL;DR

Detect the change, delete the old chunks for that doc id, re-chunk and re-embed, upsert the new vectors, then refresh derived caches — delete before write so stale and fresh copies never coexist.

Memory aid

Easy to grasp

Imagine updating one recipe card in a recipe box. First you notice the card changed. Then you pull out the old card so nobody grabs the outdated version by mistake. You write the new card, slot it back in, and finally tear up any sticky notes that summarized the old recipe. If you slipped the new card in before pulling the old one, the box would hold two versions of the same recipe and a hungry cook might grab the wrong one. That is why the order matters: remove the stale copy first, add the fresh copy, and clean up the notes last.

Key concepts

RAG Vector Databases LLM System Design

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Most RAG demos load a corpus once and never touch it again. Production indexes are the opposite: documents get edited, deleted, and added continuously, and rebuilding the whole index for every change is wasteful and slow. Incremental indexing is the discipline of updating only what changed — and the order of operations is where correctness lives or dies.

The trap is that a vector index is a shared, concurrently-read data structure. While you are halfway through swapping a document, real queries are hitting the index. If your update order ever lets the old and new versions of a document coexist, retrieval can ground an answer in text the user already corrected. If it ever lets the document vanish, retrieval silently loses a source.

This deep dive walks the five steps in order, explains the constraint behind each transition, and then pushes into the consistency and deletion edges — the parts that separate someone who has shipped an ingestion pipeline from someone who has only read about one.

Why detection gates everything

The pipeline starts by detecting that a document actually changed, and it starts there for a cost reason. Embedding is the expensive step — every chunk is a model call, and a large document can be dozens of chunks. If your sync job re-embeds every document it sees, you pay that cost on every cycle even when nothing changed, which at scale is the difference between a cheap nightly job and a runaway bill.

The standard mechanism is a content hash stored per document id. On each sync you recompute the hash of the incoming document and compare it to the stored one. Equal hash, skip the document entirely. Different hash, proceed. This also makes the pipeline idempotent: replaying the same unchanged input is a guaranteed no-op, which matters when an upstream connector redelivers records.

A timestamp is a weaker gate. Files get re-saved, metadata gets touched, and clocks drift across sources, all of which can bump a modified time without changing a single byte that gets chunked. A hash keyed on the chunkable content fires only on real change. The practical rule: detect on what you embed, not on when the file was written.

Delete before write: the load-bearing ordering

Re-chunk, re-embed, upsert: producing the new state

Caches are derived state, so they go last

Consistency under concurrent reads

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Step	Why it is here	What breaks if skipped or moved
Detect change (hash/id)	Gate the costly re-embed	Re-embed everything every sync; wasted cost
Delete old chunks	Clean replace before write	Stale and fresh copies coexist in search
Re-chunk and re-embed	Produce current vectors	Index never reflects the new content
Upsert new vectors	Make new version searchable	Document missing from results
Refresh derived caches	Rebuild against fresh vectors	Cache keeps serving stale answers

Real products, models, and research that use this idea.

LlamaIndex's IngestionPipeline with a docstore uses upserts keyed on a document hash, skipping unchanged docs and replacing changed ones — the detect then replace pattern in code.
Pinecone and Weaviate expose delete by metadata filter (delete all vectors with a given document id) precisely so an edited source can be cleanly removed before re-upsert.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow do you avoid a window where the edited document is missing from search under concurrent reads?

Either wrap the delete and upsert in a per-document transaction if the store supports it, or write the new chunks under a new version tag and atomically flip a pointer, deleting the old version afterward. The goal is that any reader sees exactly one version — never zero, never two. Discuss what your specific vector store guarantees.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Upserting new chunks before deleting the old ones. For a brief window the index holds both versions, so retrieval can surface stale and current text for the same document and the model may answer from the outdated copy.

60 second bullets to scan on the way to the call.

State the five steps of an incremental update in order from memory.
Explain why change detection comes before any re-embedding work.

Primary sources. Browse if you want the original framing.