Why does hard negative mining outperform 'just use a bigger batch' for embedding quality?

In early contrastive training, the model knows nothing. A random negative from the corpus has just as much chance of being near the query as the positive does, so the loss is high and the gradient is strong. The model learns coarse semantic structure quickly. After a few thousand steps, the geometry has settled. The model now places semantically related items near each other and unrelated items far apart. **The problem**: a random negative is now reliably far from the query in cosine space. The softmax in InfoNCE assigns it near-zero probability, the cross-entropy loss for that pair is near-zero, and the gradient contribution is essentially nothing. The model has run out of things to learn from random negatives. Scaling batch size adds more random negatives. With diminishing returns. Going from 1k to 4k batch size produces marginal improvement; going from 4k to 16k produces even less. The remaining negatives are all in the "already trivial" pile. **Hard-negative mining** is the fix. Instead of sampling negatives randomly, you select negatives that are semantically similar to the query but not the correct positive. These sit near the model's current decision boundary, where it's still uncertain. The loss is non-trivial, the gradient is informative, and the model keeps learning. This is why models like E5, BGE, and Voyage v3 all rely heavily on mined hard negatives. Ablations consistently show 5-15% absolute MTEB gains over random-negative baselines.

Contrastive training saturates in a way batch size alone can't fix. The mechanics are gradient-driven. In InfoNCE, a negative's gradient contribution scales with its softmax probability: $$\nabla_q \mathcal{L} \propto p_i k_i - k^+, \quad p_i = \frac{\exp(q \cdot k_i / \tau)}{\sum_j \exp(q \cdot k_j / \tau)}$$ After early training, a random negative has cosine near zero with the query. With `τ ≈ 0.05`, its logit is tiny while the positive's logit is large. The softmax `p_i` for that random negative is near-zero. Gradient contribution: essentially nothing. Adding more random negatives doesn't help. The sum in the denominator is dominated by the positive's term; additional zero-similarity negatives barely move the loss. The MI bound `I(q; k+) ≥ log(N) - L` needs negatives with non-trivial competition to actually tighten. **Hard-negative mining** fixes this by selecting negatives with non-trivial similarity to the query. Three strategies. - **BM25-based**: retrieve top-K BM25 results, exclude the positive, use the rest. Lexically similar, genuinely hard. - **Prior-model top-K**: train an initial model, find top-K nearest neighbours of each query (excluding positives), use as hard negatives next stage. A curriculum. - Cross-encoder distillation: use a cross-encoder to label borderline candidates, train the bi-encoder to match. Production impact: E5, BGE-M3, Voyage v3 all rely on hard-negative mining. Ablations show 5-15% absolute MTEB gains over random baselines, far larger than comparable batch-size increases. Caveat: mining introduces **false negatives** (relevant but unlabelled). Cross-encoder filtering or threshold-based denoising is necessary at scale.