Click any words you think contain an error. Click again to unmark.
Re-embedding only the new docs mixes v2 and v3 vectors in one index. v3 queries score old v2 vectors as near-random, so retrieval silently collapses — you must re-embed the whole corpus.
Imagine a library where every book's location is decided by a secret code. You upgrade to a new, better code — but you only re-file the books that arrived this week. The old books still sit where the old code put them. Now you search using the new code: it finds this week's books fine, but for everything older it sends you to random shelves, because those books were filed by a code that no longer matches. The catch is nobody gets an error — the library just quietly stops finding old books. Swapping the filing code is the same: once you change it, you have to re-file every book, not just the new ones.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Embedding-model upgrades are one of the most reliably catastrophic operations in a RAG system, precisely because the catastrophe is invisible. New embedding models ship constantly — better quality, lower cost, more dimensions — and the temptation is to treat the swap like any other index maintenance: embed the new documents, move on. The code in this question does exactly that, and it quietly destroys retrieval for most of the corpus.
This is a staff-level question because the bug isn't a typo — it's a conceptual error about what an embedding vector is. The skill being tested is whether you understand that vectors are only meaningful within a single model's learned space, that mixing spaces fails silently, and that an embedding upgrade is therefore a migration with its own runbook, not an upsert. We'll work through why the spaces are incompatible, why nothing errors, and how a senior actually executes the upgrade.
Why two models' vectors live in different worlds
An embedding model is a learned function from text to a point in a high-dimensional space. The crucial fact is that the space itself is learned. During training, the model arranges its axes so that semantically similar texts land near each other — but the specific orientation, scale, and meaning of each axis is an artifact of that particular training run.
Train a second model independently — text-embedding-v3 instead of v2 — and you get a different arrangement. The same sentence maps to a different point, and there is no shared frame of reference between the two. They are two maps of the same territory drawn with unrelated coordinate grids.
This is why a v2 document vector and a v3 query vector cannot be compared. Cosine similarity measures the angle between two vectors in the same space:
Feed it a v3 query and a v2 document and the dot product is over two unrelated bases. The number it returns has no relationship to semantic relevance — it's noise. There's no rotation you can assume aligns them; by default the spaces are simply incomparable.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
# Correct: full re-embed into a NEW index, then atomic cutover.
new_index = create_index(name="corpus_v3")
for doc in all_docs: # entire corpus, not just new_docs
new_index.upsert(doc.id, embed_v3(doc.text))
validate_recall(new_index, eval_set) # gate cutover on retrieval eval
# Flip query encoder -> v3 and route reads to new_index together.
activate(query_encoder=embed_v3, index=new_index)Real products, models, and research that use this idea.
- Pinecone and Qdrant docs both warn that switching embedding models requires reindexing the full collection, not partial upserts
- OpenAI's 2024 migration from its older first-generation embedding model to the text-embedding-3 family forced teams to re-embed entire corpora before querying with the new model — a historical example of this exact migration
What an interviewer would ask next. Try answering before peeking at the approach.
QThe new and old models output the same vector dimension — does that make the vectors compatible?
No. Dimensionality is just the length of the array; compatibility requires the same learned axes and geometry. Two models with 1536-dim outputs still place text at unrelated coordinates, so cosine across them is noise. Matching dimensions is what makes the bug silent — the upsert succeeds — not what makes it correct.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating an embedding-model upgrade like a normal incremental index update — re-embedding only new documents — when it actually invalidates every existing vector and forces a full re-embed.
60 second bullets to scan on the way to the call.
State why a v2 document vector and a v3 query vector are not comparable
Explain what an index ends up containing after a partial re-embed
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.