LoRA freezes the base weights: does that mean activation checkpointing on the base is unnecessary?

The correct answer is **A**. Freezing the base weights does not remove activations from the picture, and **gradient checkpointing** on those base modules still pays off. Here is the core distinction. Freezing a weight means the optimizer never updates it, so no gradient needs to be computed for that weight specifically. That saves the gradient buffer on those weights. But the chain rule still requires the activations of each layer to compute gradients for anything downstream that does need them, and that includes the LoRA adapters sitting on top. When the loss gradient flows backward, it passes through every base-layer operation that produced the activation feeding the LoRA adapter. Each of those operations has a local Jacobian that depends on its input. Without the cached input activation, the Jacobian cannot be evaluated, so the gradient cannot continue backward into the adapter. The other options are wrong for specific reasons. B confuses no gradient on weight with no activations needed. C invents a pre-norm versus post-norm distinction that the chain rule does not respect. D invents an optimizer-specific rule, but activation memory is decided by the autograd graph, not by which optimizer reads the gradients afterward. The practical takeaway: LoRA runs on long contexts still hit activation memory walls, and checkpointing on the base modules is one of the most useful levers for fitting them.

**The answer is A**, and the right framing is to separate two distinct concepts that the trap option B fuses incorrectly. Freezing a weight removes gradient computation for that weight. It does not remove the requirement to cache the activations needed by the chain rule to compute gradients for anything downstream. In a LoRA setup, the adapter A and B sit alongside or after a frozen base projection. During backprop the loss gradient enters the adapter's output and flows backward through both the adapter math and the base layer's contribution. To complete the chain, autograd needs the input activation that fed the base module. Without it, the local Jacobian cannot be evaluated and the gradient cannot continue backward into A and B at all. So activation memory in a LoRA run scales the same way it does in a full fine-tune. It scales with batch size, sequence length, layer count, and hidden dimension, and it is independent of whether the base weights are trainable. **Gradient checkpointing** on the base modules remains one of the highest-leverage memory levers for LoRA, especially for long-context fine-tunes where activations dominate the memory budget. The distractors are each wrong in instructive ways. B is the seductive misreading. C invents an architectural distinction that does not exist; the chain rule does not care about pre-norm versus post-norm placement of the adapter. D ties activation memory to the optimizer, but the optimizer reads gradients after they are computed and has no influence on which intermediate tensors the autograd graph must keep. The 'LoRA is small' intuition refers to weight memory and optimizer state, both of which scale with the trainable parameter count. Activation memory scales with the forward pass and remains the dominant line in long-context regimes regardless of which weights are frozen.