Describe the dual write pattern teams use during an embedding model swap.

Flashcard·Easy·4.0 · 0·~30s·Asked atCitadel Meesho Whylabs

Attempt it

From the moment the migration starts, every new or updated document is embedded with BOTH the old and the new model, and each vector is written to its own collection (the spaces are incompatible, often with different dimensions). Reads keep hitting the old collection while a backfill re-embeds the historical corpus into the new one. Once the new collection is complete and quality-checked, reads flip over, and dual writes continue for a soak period so a rollback is one flag away. The key property: at no point does either collection have a gap, because dual writes started before the backfill.

Your answer

The goal is to swap embedding models while search stays online and nothing is lost. The dual-write pattern is the standard way to do this. **Step 1.** Stand up a second collection that is shaped for the new model (often a different dimension and definitely a different vector space). **Step 2.** From the moment migration begins, every write (insert, update, delete) is embedded with both the old and the new model. Old-model vectors go to the old collection; new-model vectors go to the new collection. Reads still hit the old collection. This is the dual-write step, and turning it on first is what guarantees no document arrives during migration without landing in the new collection. **Step 3.** A background backfill job walks the historical corpus, re-embeds each document with the new model, and writes into the new collection. This is rate-limited so it does not knock over the embedding provider or the index. **Step 4.** Once the new collection is complete and quality has been compared (recall, latency, business metric), reads flip over. Dual writes continue for a soak period so a rollback is a single flag away. The pattern works because dual writes start before the backfill, which means no document written during migration is missed in either collection.

**Dual-write** is the canonical zero-downtime embedding migration pattern. The sequence has three properties that must hold together: - **Two collections, two models.** Old-model vectors and new-model vectors live in physically separate collections (or namespaces). They have different dimensions and inhabit different vector spaces, so distances between them are meaningless. Each vector is tagged with the model version that produced it. - **Dual writes turn on before the backfill starts.** Every write going forward is embedded with both models and routed to both collections. This is what closes the migration gap: any document touched after t=0 lands in the new collection without help from the backfill. - **The backfill walks history.** A rate-limited job re-embeds every pre-existing document with the new model and writes to the new collection. It runs alongside live dual writes, never instead of them. Reads stay on the old collection throughout the backfill. Once the new collection is complete, you compare quality (recall on a probe set, latency, the actual business KPI, click-through, answer-correctness) before flipping reads. After the cutover, dual writes continue for a soak window (days to weeks) so rollback is a single flag flip rather than a re-migration. The pattern is symmetric to dual-write database migrations and to blue-green deploys. The single rule that makes it correct: writes are dual before the backfill begins and stay dual through the soak.

TL;DR

Embed every new write with both models into two separate collections, backfill the old corpus into the new collection, then cut reads over once parity holds.

Memory aid

Easy to grasp

Imagine swapping the filing system in a busy library without closing the doors. From the day you start, every new book gets two index cards filed in two completely separate cabinets, the old one and the new one. While the staff is silently re-cataloguing every old book into the new cabinet, visitors keep using the old cabinet so nothing they look up disappears. Only after every book sits in the new cabinet do you move the visitors over, and you keep filing two cards for a while longer in case you need to send everyone back. Nothing is ever missing because two cards started flowing before any reorganization began.

Key concepts

Vector Databases

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Embedding model swaps are one of the highest-stakes operations a vector-database-backed system runs. The model defines the geometry of the entire index; changing it changes what neighbours mean. The dual-write pattern is the standard way to perform the change without taking search offline and without losing any documents.

The pattern looks simple from a distance: write to two places, backfill, cut over. The interesting part is the ordering and the boundary conditions. This section walks through why each step exists, what failure mode it prevents, and how the pattern is operationalized in production at companies running corpora from millions to billions of vectors.

Why two collections, not one

Old-model vectors and new-model vectors live in different vector spaces. Even if the dimensions happen to match, the geometry is unrelated: the angle between an old-model vector and a new-model vector is essentially random. Distance comparisons across the two are meaningless.

Mixing both kinds in a single collection is the most common variant of this mistake, because it looks like a small simplification. Search still works; results still come back; nothing throws an error. But the rankings are broken in a particular way: queries returned by the new embedder are ranked correctly against new-model documents and randomly against old-model documents, and you cannot tell which is which from the score alone.

The fix is structural. Use two collections (or namespaces, or aliases pointing at versioned underlying collections). Tag every vector with embedding_model_version in the payload. The read path picks a collection based on the deployed query embedder, never both. Engines like Qdrant, Weaviate, and Pinecone all support this directly; the API for switching collections atomically (Qdrant aliases, Pinecone namespaces, Weaviate classes) exists precisely to enable this pattern.

Why dual writes start before the backfill

Backfill mechanics and the embedding provider as the bottleneck

Cutover, soak, and rollback

Why dual writes drift, and what the 2026 pattern looks like

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Pinecone serverless documents the dual-collection migration pattern for embedding model upgrades, including version tags on each vector
Weaviate guides for changing the vectorizer module use a new class (collection) plus a backfill job rather than in-place rewrites

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you validate that the new collection is actually ready before flipping reads?

Run a probe set with ground-truth labels through both collections, compare recall and the downstream business KPI on a sampled traffic slice, and require the comparison to hold for a fixed window before flipping.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Writing both vector types into the same collection, or starting the backfill before dual writes turn on, which leaves a gap of documents that arrive mid-migration.

60 second bullets to scan on the way to the call.

Why old-model and new-model vectors must never share a collection
Why dual writes start before, not after, the backfill

Primary sources. Browse if you want the original framing.