Design the migration plan when upgrading a 500M-doc index from text-embedding-3-small to 3-large

The migration plan has four stages and the first thing to understand is that you cannot do it incrementally. You cannot leave old documents embedded with text-embedding-3-small and embed new documents with text-embedding-3-large. The two models live in different coordinate systems, so mixing them in one index gives meaningless cosine results. Stage one: figure out the cost. 500M documents at roughly 500 tokens each is 250 billion tokens. The text-embedding-3-large via OpenAI's batch API costs about $0.065 per million tokens (the sync price is $0.13 with a 50% batch discount). So 250,000 million tokens times $0.065 is about $16,250. Storage doubles because the new model's vectors are 3072 dim vs 1536 dim. The index storage grows from about 3 TB to about 6 TB. Stage two: build a parallel index. Stand up a fresh index in the vector database. Run batch embedding jobs over the corpus, writing the new vectors to the new index. The old index keeps serving production queries the entire time. Stage three: dual-write and shadow-test. Once the new index has caught up, start writing every new document to both indexes. Fire production queries against both indexes in shadow, compare the top-K, and compute recall@K on a labeled eval set. Stage four: atomic cutover. When the new index meets or beats the old on labeled-set recall, flip query traffic. Keep the old index hot for a week as rollback, then decommission.

Embedding-model migrations have to be parallel-index migrations. Incremental upgrade is impossible because mixed-model indexes silently break retrieval. Stage one is sizing. 500M docs at 500 tokens each is 250B tokens. The text-embedding-3-large via batch API runs around $0.065 per million tokens (50% discount on the $0.13 sync price), giving about $16K for the re-embedding job. Sync API would roughly double that to $32K. Storage grows from about 3 TB at 1536 dim to about 6 TB at 3072 dim during the cutover window. Stage two is build. Stand up a fresh vector index. Submit batch embedding jobs over the corpus; the OpenAI batch API typically completes within 24 hours per submission. Write the new vectors to the new index. The old index keeps serving live traffic. Stage three is dual-write and shadow-test. Once the new index has caught up, start writing every incoming document to both indexes. Fire production queries against both, compare top-K overlap, and compute recall@K on a labeled set of (query, doc, relevant?) tuples that reflects your corpus. The shadow window typically runs for a week or two depending on traffic volume needed to validate. Stage four is atomic cutover. When recall@K on the new index meets or beats the old, flip query traffic. The cutover is a config change. Keep the old index hot for a week as a rollback option, then decommission. The risk to flag: text-embedding-3-large outperforms text-embedding-3-small on MTEB, but MTEB is benchmark performance. Your corpus may not benefit equally. Validate on YOUR labeled queries before cutover; do not assume the benchmark gain transfers.