Why is 'cosine ≥ 0.8' not a model-agnostic relevance test?

Attempt it

Your answer

Quick reference

TL;DR

Cosine values live inside one model's coordinate system; 0.8 means whatever that model's distribution makes it mean, and the value does not transfer to any other model.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine two different teachers grading essays on a zero to one scale. One teacher's 0.8 means 'pretty good' because she grades generously. The other teacher's 0.8 means 'almost perfect' because he is strict. The number looks identical, but the meaning is set by each teacher's habits. Search tools are the same. Each tool spreads its scores differently, so a similarity of 0.8 from one and 0.8 from another are not the same evidence of a match. You have to re-check what 0.8 means on the new teacher's curve, or just rank essays from best to worst and pick the top three, which works no matter who grades.

Key concepts

Embeddings Retrieval-augmented generation

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

2 min: cosine as geometry not probability + per-model score distributions + re-derivation recipe + rank-based filtering escape hatch + calibration via isotonic regression.

Real-world examples

Real products, models, and research that use this idea.

Production RAG stacks at Perplexity and Notion pair embedding retrieval (top-K, no threshold) with a cross-encoder reranker like Cohere Rerank v3 to sidestep per-model calibration drift.
OpenAI's text-embedding-3 docs in 2026 explicitly warn that cosine values are not comparable across models and recommend re-deriving any threshold after a model change.
Voyage AI's voyage-3 release notes recommend rank-based filtering over fixed-cutoff filtering precisely because of this calibration gap.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you re-derive the cutoff for a new model?

Sign in or go Pro to see the approach for this follow-up.

QWhen would you skip calibration entirely?

Sign in or go Pro to see the approach for this follow-up.

QHow do you detect that a vendor silently bumped their model?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Carrying over a tuned threshold like 0.78 from one embedding model to a new one without re-deriving it on the new model's score distribution.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.