Drag each answer to line up with its matching prompt
InfoNCE
Treats other batch examples as in batch negatives; no explicit negative mining
Triplet loss
Hinge loss with margin: pull anchor positive close, push anchor negative beyond margin
MNRL (Multiple Negatives Ranking Loss)
Direct supervision: target cosine = +1 for similar, -1 for dissimilar pairs
Cosine embedding loss
Cross-entropy over softmax of similarities; 1 positive vs N-1 negatives
Four contrastive losses, four mechanisms: InfoNCE uses softmax, triplet uses a margin, MNRL piggybacks on the batch, cosine embedding regresses to a target similarity.
Think of four different sports coaches teaching a team. The first coach lines up a whole group of players and rewards the one who looks most like the right pick. The second coach picks three players at a time and yells if the right one is not a fixed distance ahead of the wrong one. The third coach is lazy. Whoever happens to be in the gym counts as the competition. The fourth coach hands every pair of players a target friendliness score and grades them on hitting that exact score. Same sport, same goal of getting the team in shape. Just four different drills, with four different costs, and four different things they actually improve.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The contrastive-loss family has four members worth knowing well, and they're often confused because all four share the same broad goal: pull related inputs together, push unrelated ones apart. The mechanisms, scaling behaviour, and production niches are different, and an interviewer probing depth on embedding training will ask you to separate them.
This deep dive walks through each loss in detail, identifies the design trade-offs, and explains how production embedding teams combine them in multi-task training pipelines.
InfoNCE: cross-entropy over softmax of similarities
InfoNCE is the dominant contrastive loss in modern embedding training. The formula:
Reads as: "the probability the model assigns to the positive among N candidates, scored with softmax over similarities." Cross-entropy then penalises the model for assigning low probability to the positive.
The key property is scalability with negatives. Oord et al. proved the mutual-information lower bound I(q; k+) ≥ log(N) - L_InfoNCE. More negatives = tighter bound = stronger gradient signal toward maximising MI. This is why frontier embedding models train with batch sizes of 4k-32k.
Temperature τ controls softmax sharpness. Production values are 0.05-0.07 for L2-normalised vectors. Too low and gradients collapse to nearest neighbour only; too high and the loss flattens.
Production users: OpenAI text-embedding-3, Cohere embed-v3, Voyage v3, BGE-M3, E5, Jina v3: essentially all frontier text embedding models in 2026.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI text-embedding-3 series uses InfoNCE with curated and mined hard negatives.
- FaceNet (Google) introduced triplet loss for face verification, still used in face-recognition pipelines.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy did InfoNCE supersede triplet loss at production scale?
InfoNCE benefits from more negatives via the mutual-information lower bound. Triplet loss only has one negative per anchor, so its gradient signal is capped regardless of batch size. InfoNCE also avoids margin tuning, which was a per-dataset hyperparameter in triplet training.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Lumping all four losses into one bucket because they're all "contrastive." The mechanisms are different and the trade-offs aren't interchangeable.
60 second bullets to scan on the way to the call.
Write the InfoNCE loss formula.
Write the triplet loss formula with margin.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.