If you change LoRA's rank from 16 to 64 with α fixed, what actually happens?

**LoRA** freezes the base model and learns two small matrices, B and A, whose product is added to a weight. The rank r is the shared inner dimension of those two matrices. When you grow r from 16 to 64, both matrices get taller or wider in their r dimension. Their parameter count is proportional to r, so quadrupling r roughly quadruples how many numbers you are training. There is a second, sneakier effect. LoRA does not add the raw product BA. It multiplies it by a scalar equal to alpha divided by r. If you raise r but leave alpha alone, that scalar shrinks by the same factor, so every update is scaled down to a quarter of its old size. So two things move at once: more parameters, and smaller per-step updates. Training can look like it slowed down even though you added capacity. The fix is the convention of tying alpha to r, for example alpha equal to twice r. Then the scaling factor stays roughly constant and only the capacity changes.

Raising rank touches two independent dials, and the question rewards keeping them apart. The first dial is **parameter count**. The matrix B has shape d by r and A has shape r by k, so their combined size grows linearly with r. Going from 16 to 64 roughly quadruples the trainable parameters. The second dial is the update magnitude. LoRA scales the learned product by the factor alpha over r. The forward path is: $$h = W_0 x + \tfrac{\alpha}{r}\, BA\, x$$ That scalar lives in the forward pass, not the parameter list. So when r quadruples with alpha fixed, the factor shrinks fourfold. Every effective update is quartered, and training appears to slow. The two effects combine, which is the whole point. You get more capacity but weaker steps. That is why practitioners tie alpha to r, commonly alpha equal to twice r. Holding the ratio constant means rank changes capacity without secretly rescaling the learning signal.