Why does contrastive training produce useful embedding geometry?

A **contrastive loss** is a training objective designed around pairs of inputs. For each batch, the model sees positive pairs (semantically related. E.g. a question and its correct answer) and negative pairs (unrelated. E.g. the same question with a random document). The loss pulls positives together in vector space and pushes negatives apart. This is the trick that installs the geometry every embedding application depends on. Before training, the embedding space is random. After enough contrastive updates, similar inputs land near each other and dissimilar inputs land far apart, automatically, without any human annotating dimensions. Three variants dominate in 2026 embedding training. **InfoNCE** is cross-entropy over a softmax of pairwise similarities. The positive pair is the "correct class" among N candidates. **Triplet loss** uses a hard margin: anchor-positive distance must be smaller than anchor-negative distance by some margin. MNRL (Multiple Negatives Ranking Loss) is a practical variant of InfoNCE that treats every other example in the batch as a negative: no explicit negative mining required. The family won because it's data-efficient (no per-token labels needed, just pairs), the geometry it produces matches the similarity-search use case, and it scales to billions of pairs.

**Contrastive training optimises a pulling-pushing objective: positive pairs (semantically related inputs) are pulled closer in vector space. Negative pairs (unrelated inputs) are pushed apart.** The cumulative effect across millions of pairs is a geometry where cosine similarity tracks semantic similarity. The dominant formulation is **InfoNCE**: cross-entropy over a softmax of similarities, treating the positive pair as the correct class among N candidates. The loss is: $$\mathcal{L}_{\text{InfoNCE}} = -\log \frac{\exp(\text{sim}(q, k^+) / \tau)}{\sum_i \exp(\text{sim}(q, k_i) / \tau)}$$ where `sim` is typically the cosine similarity, `k^+` is the positive, `k_i` ranges over positive plus negatives, and `τ` is a temperature. Two cousin formulations show up regularly: **triplet loss** uses a margin-based hinge (anchor-positive closer than anchor-negative by margin `m`). MNRL treats every other batch example as an in-batch negative: fast and practical, no explicit mining. What the loss buys is twofold. The geometry is **angular** because the loss operates on normalised dot products: magnitudes don't matter, only directions. And the geometry is task-aligned: the model arranges the space to make the contrastive signal as low as possible, which directly produces the property downstream applications need.