Zenaique
Part ofRAG Engineer·Week 1: Embeddings & Vector SearchView roadmap →

Match each contrastive loss to its defining mechanism

Match pairs·Medium·4.0 · 0·~2 min·Asked atCohereHaptikMu Sigma
Attempt it

Drag each answer to line up with its matching prompt

InfoNCE

Treats other batch examples as in batch negatives; no explicit negative mining

Triplet loss

Hinge loss with margin: pull anchor positive close, push anchor negative beyond margin

MNRL (Multiple Negatives Ranking Loss)

Direct supervision: target cosine = +1 for similar, -1 for dissimilar pairs

Cosine embedding loss

Cross-entropy over softmax of similarities; 1 positive vs N-1 negatives

TL;DR

Four contrastive losses, four mechanisms: InfoNCE uses softmax, triplet uses a margin, MNRL piggybacks on the batch, cosine embedding regresses to a target similarity.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Think of four different sports coaches teaching a team. The first coach lines up a whole group of players and rewards the one who looks most like the right pick. The second coach picks three players at a time and yells if the right one is not a fixed distance ahead of the wrong one. The third coach is lazy. Whoever happens to be in the gym counts as the competition. The fourth coach hands every pair of players a target friendliness score and grades them on hitting that exact score. Same sport, same goal of getting the team in shape. Just four different drills, with four different costs, and four different things they actually improve.

Key concepts

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

The contrastive-loss family has four members worth knowing well, and they're often confused because all four share the same broad goal: pull related inputs together, push unrelated ones apart. The mechanisms, scaling behaviour, and production niches are different, and an interviewer probing depth on embedding training will ask you to separate them.

This deep dive walks through each loss in detail, identifies the design trade-offs, and explains how production embedding teams combine them in multi-task training pipelines.

InfoNCE: cross-entropy over softmax of similarities

InfoNCE is the dominant contrastive loss in modern embedding training. The formula:

LInfoNCE=logexp(sim(q,k+)/τ)i=1Nexp(sim(q,ki)/τ)\mathcal{L}_{\text{InfoNCE}} = -\log \frac{\exp(\text{sim}(q, k^+) / \tau)}{\sum_{i=1}^{N} \exp(\text{sim}(q, k_i) / \tau)}

Reads as: "the probability the model assigns to the positive among N candidates, scored with softmax over similarities." Cross-entropy then penalises the model for assigning low probability to the positive.

The key property is scalability with negatives. Oord et al. proved the mutual-information lower bound I(q; k+) ≥ log(N) - L_InfoNCE. More negatives = tighter bound = stronger gradient signal toward maximising MI. This is why frontier embedding models train with batch sizes of 4k-32k.

Temperature τ controls softmax sharpness. Production values are 0.05-0.07 for L2-normalised vectors. Too low and gradients collapse to nearest neighbour only; too high and the loss flattens.

Production users: OpenAI text-embedding-3, Cohere embed-v3, Voyage v3, BGE-M3, E5, Jina v3: essentially all frontier text embedding models in 2026.

Triplet loss: margin-based metric learning
MNRL: InfoNCE with in-batch negatives
Cosine embedding loss: supervised similarity regression
The multi-task production recipe
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • OpenAI text-embedding-3 series uses InfoNCE with curated and mined hard negatives.
  • FaceNet (Google) introduced triplet loss for face verification, still used in face-recognition pipelines.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy did InfoNCE supersede triplet loss at production scale?
A

InfoNCE benefits from more negatives via the mutual-information lower bound. Triplet loss only has one negative per anchor, so its gradient signal is capped regardless of batch size. InfoNCE also avoids margin tuning, which was a per-dataset hyperparameter in triplet training.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Lumping all four losses into one bucket because they're all "contrastive." The mechanisms are different and the trade-offs aren't interchangeable.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • Write the InfoNCE loss formula.

  • Write the triplet loss formula with margin.

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
Why is cosine similarity preferred over Euclidean distance for text embeddings?
Flashcard·Easy