Write acknowledgement and search visibility are separate steps; the upsert is durable but waits in a buffer or fresh segment until the next flush or refresh makes it queryable.
Imagine handing a note to a librarian who immediately puts it in a tray on the counter and says, got it. The note is safe and is going into the catalogue, but the catalogue card has not been typed and filed yet. If you ask the catalogue ten seconds later whether the note exists, the catalogue says no, because the typing happens in batches every few minutes. The note is not lost. The catalogue simply has a refresh schedule, and your query landed between two refreshes. Most vector databases work the same way: a write is accepted and durable long before it is searchable.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Freshness lag is one of those production behaviours that looks like a bug the first time a developer hits it and is documented as a feature in every engine that has it. The mental model that confuses people is borrowed from a relational database, where a successful insert is immediately visible to the next select inside the same transaction. Vector databases do not work that way.
The rest of this section walks through why the separation exists, how the major engines expose it, and what application patterns make it invisible to users. The key insight is that vector indexes are optimized for query throughput and recall, not for write after read consistency, and the freshness lag is the price of that optimization.
Why durability and queryability are separate
A vector database has two jobs: store vectors safely, and answer nearest neighbour queries fast. The structures that do those two jobs are different.
Durability lives in a write ahead log and replicated segment files on disk. Adding a record is cheap: append to the log, flush, return success. The cost is dominated by the disk fsync, which is fast on modern SSDs.
Queryability lives in an HNSW graph or an IVF posting structure. Inserting into an HNSW graph means walking the upper layers to find the right insertion point, choosing M neighbours, and updating bidirectional links on each of them. This is much more expensive than appending to a log; doing it synchronously on every write would cap ingest throughput at a small fraction of what the log can absorb.
The fix is to batch. Writes accumulate in a buffer (a memtable, a fresh segment, or a delta layer), and the engine merges the buffer into the main index periodically. The acknowledgement is fast because it requires only the log write; the index update happens in the background.
The consequence is that for some interval after acknowledgement, the vector is durable but not findable. This interval is the freshness window. Different engines expose different windows because they batch differently.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Pinecone serverless documents a freshness layer with second scale staleness for the most recent writes before they merge into the main index
- Elasticsearch and OpenSearch ship with a one second default refresh interval; per request refresh exists for tests but is the wrong default for production
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you give the application read your own writes without forcing strong consistency on every query?
Keep the source of truth in a transactional database. For the just written document, return it directly to the user from the source. For semantic neighbours of older documents, use the vector index. Merge in the application layer.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming write acknowledgement equals search visibility, then chasing phantom bugs in upsert payloads when the real cause is a documented freshness window.
60 second bullets to scan on the way to the call.
Why durability and queryability are separate steps in a vector database
What a refresh interval is and why per write refresh is an anti-pattern
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.