Identify the symptoms that indicate stale or version mixed embeddings in production

Stale or version-mixed embeddings break retrieval quality without breaking the infrastructure. That is the key idea. The vector database keeps storing, querying, and responding normally. Latency stays roughly the same. Errors do not fire. The only thing that drifts is the quality of the results. Four symptoms are diagnostic. First, recall@K on a fixed labeled eval set declines over time. This is the gold-standard detector because the labeled set is your ground truth. Second, the typical cosine scores on a fixed set of (query, relevant doc) pairs shift downward or upward; the absolute geometry of the space has moved. Third, documents indexed before a known model change consistently rank lower than newer ones for the same query; the age correlation gives the bug away. Fourth, the embedding API starts returning vectors of a different dimension count, indicating the vendor pinned a new model under the same alias. The two non-symptoms are ops issues. Upsert 500 errors are an infrastructure problem (vector DB outage, network), not an embedding problem. Doubled ANN search latency at the same QPS is an index-tuning or capacity issue, not an embedding-quality issue.

Stale and mixed embeddings produce silent quality degradation. The four diagnostic signals are. Recall@K drift on a fixed labeled set, distribution shift in cosine scores on known relevant pairs, age-correlated ranking where older documents systematically lose to newer ones. An unexpected dimension count from the vendor's API. Recall@K drift is the strongest detector. If your labeled set has been stable for months and your indexing and query code have not changed. Recall@10 dropped from 0.84 to 0.62, something on the embedding layer has moved. It might be a vendor snapshot bump or a mixed-model state from a botched migration. The distribution shift in cosine scores on known pairs is the same signal at a different granularity. Track the median cosine on a known (query, relevant doc) labeled set as a daily metric. A step change tells you the underlying space has rotated. Age-correlated ranking is the smoking gun for a mixed-model index. If documents indexed before a particular date consistently underperform on equivalent queries, the cohort boundary suggests a model change at that date. Dimension mismatches are rare but catastrophic. They only happen when a vendor pins a new model under the same alias and the dimension changes (e.g., text-embedding-ada-002 to text-embedding-3-large would jump from 1536 to 3072). The two trap options are ops issues, not embedding issues, and would show up on different dashboards.