Why is 'cosine ≥ 0.8' not a model agnostic relevance test?
Cosine values live inside one model's coordinate system; 0.8 means whatever that model's distribution makes it mean, and the value does not transfer to any other model.
Imagine two different teachers grading essays on a zero to one scale. One teacher's 0.8 means 'pretty good' because she grades generously. The other teacher's 0.8 means 'almost perfect' because he is strict. The number looks identical, but the meaning is set by each teacher's habits. Search tools are the same. Each tool spreads its scores differently, so a similarity of 0.8 from one and 0.8 from another are not the same evidence of a match. You have to re-check what 0.8 means on the new teacher's curve, or just rank essays from best to worst and pick the top three, which works no matter who grades.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A surprising number of production RAG outages trace back to one assumption: that a cosine similarity number means the same thing on any embedding model. It does not. Cosine is a geometric measurement inside one model's learned vector space, and that space is reshaped every time the model is retrained or replaced. The number is portable in look only; the meaning is not.
This question matters because the threshold debate keeps coming up. A team tunes 0.78 on text-embedding-3-small, ships it, then someone runs an A/B with Voyage-3 for the quality gain. Recall drops without warning. The cosine number is still 0.78, but it now sits at a different point on the relevance curve. Understanding why is what separates an engineer who can debug this from one who cannot.
Why each model's score distribution is different
Embedding models are trained with a contrastive objective: pull paired (query, positive) close in cosine and push (query, negative) far apart. The training data, the loss function (InfoNCE, MNRL, triplet), the temperature, the negative mining strategy, and any instruction-prefix conditioning all shape how spread out the final space is.
One model might produce a tight space where random pairs sit around cosine 0.7 and relevant pairs sit around 0.85. Another might produce a spread-out space where random pairs sit near 0.3 and relevant pairs sit near 0.7. Both spaces are valid; the geometry just differs. A threshold of 0.8 means 'very confident' in one and 'definitely relevant' in the other.
This is also why instruction-prefixed models (asymmetric query and document encoders) shift the distribution again. Adding 'query: ' to the input pushes the cosine baseline; forgetting to add it on one side breaks the calibration silently.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Production RAG stacks at Perplexity and Notion pair embedding retrieval (top-K, no threshold) with a cross-encoder reranker like Cohere Rerank v3 to sidestep per-model calibration drift.
- OpenAI's text-embedding-3 docs in 2026 explicitly warn that cosine values are not comparable across models and recommend re-deriving any threshold after a model change.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you re-derive the cutoff for a new model?
Build a labeled (query, doc, relevant?) set, embed both sides with the new model, sweep the cosine cutoff and plot precision vs recall, then pick the operating point that matches the SLO.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Carrying over a tuned threshold like 0.78 from one embedding model to a new one without re-deriving it on the new model's score distribution.
60 second bullets to scan on the way to the call.
Cosine as geometry, not as probability
Why each embedding model induces its own score distribution
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.