2Re-chunk and re-embed the new version of the document
3Delete the old chunks and vectors for that document id
4Refresh any derived caches (semantic cache, summaries)
5Upsert the new vectors into the index
Incremental indexing means updating only what changed instead of rebuilding the whole index. When a source document is edited, the pipeline runs five steps in a specific order.
First, **detect the change** — usually by comparing a content hash or a last-modified timestamp against the document id you already stored. Nothing should happen until you know a document actually changed.
Next, delete the old chunks and vectors tied to that document id. This is the step people get wrong: you remove stale data *before* adding the new data, so the index never holds two versions of the same document at once.
Then re-chunk and re-embed the new version, and upsert those fresh vectors into the index. Only now is the document searchable in its current form.
Finally, refresh any derived caches — a semantic cache of past answers, document summaries, or precomputed retrieval results. Those depend on the underlying vectors, so they can only be rebuilt once the new vectors are live. Refresh them last, or they will keep serving answers grounded in the old text.
The correct sequence is detect, delete, re-embed, upsert, refresh caches. Two constraints drive the ordering. First, don't do work you don't need to. Second, never let stale and current copies of the same document coexist in the searchable index.
**Detect first**. Compare a content hash or modified timestamp keyed by document id. This gates the whole pipeline — re-embedding is the expensive step, so you only pay it when content genuinely changed, not on every sync.
**Delete before write**. Remove the old chunks and vectors for that document id before upserting the new ones. If you upsert first, there is a window where retrieval can return both the old and new chunks for the same source. The model may then ground its answer in outdated text or surface duplicate, contradictory hits. Deleting first closes that window. Re-chunking can also move chunk boundaries and counts, so old chunk ids may not line up with new ones. A clean delete avoids orphaned stale vectors.
**Re-chunk, re-embed, upsert**. Produce the new chunks, embed them with the same model and settings as the rest of the index, and write them in.
**Refresh derived caches last**. A semantic cache, summaries, or precomputed results all depend on what retrieval returns. They can only be correctly rebuilt once the new vectors are in place; refresh them earlier and they capture the old state. In strict-consistency systems you instead invalidate these caches at delete time and let them lazily repopulate.
Quick reference
TL;DR
Detect the change, delete the old chunks for that doc id, re-chunk and re-embed, upsert the new vectors, then refresh derived caches — delete before write so stale and fresh copies never coexist.
Imagine updating one recipe card in a recipe box. First you notice the card changed. Then you pull out the old card so nobody grabs the outdated version by mistake. You write the new card, slot it back in, and finally tear up any sticky notes that summarized the old recipe.
If you slipped the new card in before pulling the old one, the box would hold two versions of the same recipe and a hungry cook might grab the wrong one. That is why the order matters: remove the stale copy first, add the fresh copy, and clean up the notes last.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Tiny static corpora that change a few times a year. A full rebuild on each change is simpler and less error-prone than maintaining incremental delete and upsert logic.
Sources with no stable document id or content hash. Change detection and a clean delete by id both depend on a stable key; without one you cannot target the right chunks to remove.
Strict read-consistency requirements during the update window. A plain delete then upsert leaves a moment where the doc is absent; you need a transactional swap or versioned pointers instead.
High-churn streams where most documents change every cycle. The detect gate saves nothing when almost everything changed; a batched rebuild can amortize embedding cost better.
Quick scan · 2–4 min
Everything important, quickly.
45s: give the order — detect, delete, re-embed, upsert, refresh caches; stress deleting before the upsert to avoid coexisting versions; note re-chunking shifts ids; mention the missing-document window and the versioned-pointer fix; close on caches being derived state refreshed last.
Side-by-side
Step
Why it is here
What breaks if skipped or moved
Detect change (hash/id)
Gate the costly re-embed
Re-embed everything every sync; wasted cost
Delete old chunks
Clean replace before write
Stale and fresh copies coexist in search
Re-chunk and re-embed
Produce current vectors
Index never reflects the new content
Upsert new vectors
Make new version searchable
Document missing from results
Refresh derived caches
Rebuild against fresh vectors
Cache keeps serving stale answers
Real-world examples
Real products, models, and research that use this idea.
1LlamaIndex's IngestionPipeline with a docstore uses upserts keyed on a document hash, skipping unchanged docs and replacing changed ones — the detect then replace pattern in code.
2Pinecone and Weaviate expose delete by metadata filter (delete all vectors with a given document id) precisely so an edited source can be cleanly removed before re-upsert.
3A GDPR deletion request is the same flow without the re-embed step: detect the subject's document id, delete its vectors, then invalidate caches that may have memorized the content.
4Airbyte- or Fivetran-style CDC connectors emit content hashes per record so a RAG ingestion job only re-embeds rows that actually changed.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you avoid a window where the edited document is missing from search under concurrent reads?
Sign in or go Pro to see the approach for this follow-up.
QWhy is keying change detection on a content hash better than a modified timestamp?
Sign in or go Pro to see the approach for this follow-up.
QHow does this flow change for a GDPR right to be forgotten request?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Upserting new chunks before deleting the old ones. For a brief window the index holds both versions, so retrieval can surface stale and current text for the same document and the model may answer from the outdated copy.