The documents behind a RAG system are constantly created, edited, and deleted. Explain how to keep the index fresh incrementally rather than rebuilding it, and what you must handle for deletions.
Free · 2 AI evals / day
A RAG index is built from document chunks turned into vectors. In the real world those documents change all the time — new ones appear, existing ones get edited, and some get deleted. The naive approach is to rebuild the whole index whenever anything changes, but at any real scale that is far too slow and expensive to do on every edit.
Instead you sync incrementally: only touch what changed. The trick is to give every chunk a stable document id linking it back to its source, and to track a version or content hash per document. On each sync you compare the current state of the source against what you indexed and find the new, edited, and removed documents.
For new or edited documents you upsert — re-chunk, re-embed, and write the fresh vectors, replacing the old ones for that id. For deletions you must actually remove the vectors from the store, not just mark them hidden somewhere. A vector that still lives in the index can still be retrieved and shown to a user, even if your application thinks the document is gone. So real deletion is the part people get wrong.
**Incremental indexing** keeps the vector store in sync with a changing corpus without paying for a full rebuild on every change. The foundation is identity: every chunk carries a stable document id, and you track a version or content hash per source document.
A sync run does change detection. You discover what is new, edited, or removed using a content hash comparison, an `updated_at` timestamp, or change events streamed from the source system. For new and edited documents you upsert: re-chunk, re-embed, and write the new vectors. Crucially an edit is a delete then insert keyed by the document id. Re-chunking changes how many chunks the document produces and where their boundaries fall, so the old chunk set will not line up with the new one. If you skip the delete, orphaned old chunks survive alongside the new ones and keep getting retrieved.
Deletions are the sharp edge. A removed document must be hard-deleted from the vector store. Soft-deleting it — setting a hidden flag while the vectors stay searchable — leaves the content retrievable. That is a correctness bug and, for revoked-access or right to be forgotten data, a compliance violation. So the index must track which chunks belong to which document and version, so stale or deleted content can never be returned to a query.
Quick reference
TL;DR
Key chunks to a stable document id, detect changes by hash or timestamp, upsert edits as delete then insert, and hard-delete removals so stale content can never be retrieved.
Picture a library where the shelves are your search index and the catalogue cards point readers to the right shelf. Every day some books get added, some get a new edition, and some get pulled. You would not burn down the library and rebuild it from scratch each morning. Instead you give every book a fixed ID, check which ones changed since yesterday, and only touch those. A new edition means you take the old copy off the shelf and put the new one up under the same ID. A pulled book means you actually remove it, not just slip a 'do not lend' note inside while leaving it on the shelf — because if it is still on the shelf, a reader can still grab it. A RAG index works the same way: update only what moved, and make deletions real.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The corpus is small and rebuilds finish in seconds A periodic full rebuild is simpler and the incremental machinery adds complexity with no payoff at that scale
The underlying embedding model is being changed A model swap invalidates every existing vector, so you re-embed the whole corpus anyway — that is a full re-index, not an incremental sync
Documents are append-only and never edited or deleted If sources only ever get added, you only need inserts and can skip the delete then insert and deletion-handling logic
Quick scan · 2–4 min
Everything important, quickly.
8 min: reject full rebuilds, set up chunk identity, walk the three change-detection strategies, explain upsert as delete then insert with orphan risk, then make the case for hard deletes and a reconciliation pass.
Real-world examples
Real products, models, and research that use this idea.
1LlamaIndex's IngestionPipeline ships a docstore-backed upsert that hashes each document and skips, updates, or deletes nodes so re-running the pipeline only touches changed files.
2Pinecone and Weaviate expose upsert by id and delete by id so a sync job can replace or remove exactly the chunks for one document without rebuilding the collection.
3GDPR and CCPA right to be forgotten requests force teams to prove a deleted user's documents are truly gone from every index, not just flagged hidden.
4Enterprise search over a permissioned document store revokes access by hard-deleting an employee's now-restricted files from the shared index the moment their access changes.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you keep deletions from leaking when access to a document is revoked rather than the document being removed?
Sign in or go Pro to see the approach for this follow-up.
QYour change events occasionally get dropped — how do you stop the index from silently drifting from the source?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Soft-deleting a removed document by flagging it instead of deleting its vectors — the chunks stay searchable and get retrieved, which is a correctness bug and, for revoked-access or right to be forgotten content, a compliance violation.