Misconception: 'higher LoRA rank always wins on benchmarks'

**LoRA** adds two small trainable matrices next to a frozen weight. Their shared inner dimension is the rank, written r. Rank controls how much the adapter can change. It is tempting to think more rank means more capacity, so more rank means better quality. That holds at the very low end. Going from r=2 to r=8 usually helps because the task needs a bit more room to learn. But the curve flattens fast. Once the rank is large enough to capture your task, adding more does almost nothing. The adapter has all the room it needs, so extra rank is wasted parameters and wasted memory. Worse, on small or medium datasets, a big rank starts to memorise quirks of your examples. That hurts held-out performance, which is overfitting. So r=256 often scores below r=32 on the very same data. The practical rule: start around r=8 to r=16, raise it only if evaluation says the task needs more. Most instruction tuning saturates between r=8 and r=32.

The claim is false. Higher **LoRA rank** is not reliably better. Rank sets the dimension of the low-rank update. At tiny ranks, more capacity does help, since the adapter needs room to fit the task. But quality saturates once rank matches the task's intrinsic complexity. Past that point, extra rank adds parameters and memory without lifting held-out scores. Two forces push back on bigger. First, overfitting: a high-rank adapter can memorise idiosyncrasies of a small dataset, so generalization drops. Second, the alpha interaction. The effective scale is alpha divided by r. If you raise r but hold alpha fixed, the update shrinks, so naive scaling can hurt directly. Empirically, most instruction tuning saturates at r=8 to r=32. Jumping to r=256 routinely underperforms r=32 on the same data, while costing far more memory. The right move is to size rank to the task and validate. Start near r=16, scale alpha with r, and only raise rank if a held-out set says the task genuinely needs it.