Zenaique

Why is 'cosine ≥ 0.8' not a model agnostic relevance test?

Flashcard·Medium·4.0 · 0·~30s·Asked atAi21Lightning AiSpotify
Attempt it
TL;DR

Cosine values live inside one model's coordinate system; 0.8 means whatever that model's distribution makes it mean, and the value does not transfer to any other model.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Imagine two different teachers grading essays on a zero to one scale. One teacher's 0.8 means 'pretty good' because she grades generously. The other teacher's 0.8 means 'almost perfect' because he is strict. The number looks identical, but the meaning is set by each teacher's habits. Search tools are the same. Each tool spreads its scores differently, so a similarity of 0.8 from one and 0.8 from another are not the same evidence of a match. You have to re-check what 0.8 means on the new teacher's curve, or just rank essays from best to worst and pick the top three, which works no matter who grades.

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

A surprising number of production RAG outages trace back to one assumption: that a cosine similarity number means the same thing on any embedding model. It does not. Cosine is a geometric measurement inside one model's learned vector space, and that space is reshaped every time the model is retrained or replaced. The number is portable in look only; the meaning is not.

This question matters because the threshold debate keeps coming up. A team tunes 0.78 on text-embedding-3-small, ships it, then someone runs an A/B with Voyage-3 for the quality gain. Recall drops without warning. The cosine number is still 0.78, but it now sits at a different point on the relevance curve. Understanding why is what separates an engineer who can debug this from one who cannot.

Why each model's score distribution is different

Embedding models are trained with a contrastive objective: pull paired (query, positive) close in cosine and push (query, negative) far apart. The training data, the loss function (InfoNCE, MNRL, triplet), the temperature, the negative mining strategy, and any instruction-prefix conditioning all shape how spread out the final space is.

One model might produce a tight space where random pairs sit around cosine 0.7 and relevant pairs sit around 0.85. Another might produce a spread-out space where random pairs sit near 0.3 and relevant pairs sit near 0.7. Both spaces are valid; the geometry just differs. A threshold of 0.8 means 'very confident' in one and 'definitely relevant' in the other.

This is also why instruction-prefixed models (asymmetric query and document encoders) shift the distribution again. Adding 'query: ' to the input pushes the cosine baseline; forgetting to add it on one side breaks the calibration silently.

Re-deriving the threshold against a labeled set
Rank-based filtering and per-model calibration
Production patterns and detection
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • Production RAG stacks at Perplexity and Notion pair embedding retrieval (top-K, no threshold) with a cross-encoder reranker like Cohere Rerank v3 to sidestep per-model calibration drift.
  • OpenAI's text-embedding-3 docs in 2026 explicitly warn that cosine values are not comparable across models and recommend re-deriving any threshold after a model change.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you re-derive the cutoff for a new model?
A

Build a labeled (query, doc, relevant?) set, embed both sides with the new model, sweep the cosine cutoff and plot precision vs recall, then pick the operating point that matches the SLO.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Carrying over a tuned threshold like 0.78 from one embedding model to a new one without re-deriving it on the new model's score distribution.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • Cosine as geometry, not as probability

  • Why each embedding model induces its own score distribution

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
Why is cosine similarity preferred over Euclidean distance for text embeddings?
Flashcard·Easy