Lora

Walk through what happens during backprop for a LoRA wrapped linear layer

Walk through the triage order when a customer support LoRA tanks MMLU by 4 points post-deploy

LoRA freezes the base weights: does that mean activation checkpointing on the base is unnecessary?

Merging a LoRA back into an NF4 quantized base hits one subtle problem: name it and explain why

Adam stores two fp32 buffers per trainable parameter: what does that imply for LoRA vs full FT on a 7B?

Why might a LoRA that passes every eval check then garble one in five prod requests?

Estimate the trainable param count for a Llama-3 8B LoRA at r=16 targeting all linear modules.

Only 200 examples for a niche format task: which PEFT method gives the best shot?

Match each PEFT method to the standout advantage that defines it.

target_modules in a PEFT config: what does this list control?

Rank r in LoRA: what does the number actually count?

PEFT: unpack the acronym and the core problem it tackles.

When does a PEFT config actually need `modules_to_save` set, and what goes in it?

After calling `merge_and_unload()` on a PEFT model, what does the returned object look like?

In LoRA, where in the forward pass does `lora_dropout` apply its mask?

lora_alpha in plain English: what role does that scalar play?

Loss spikes the moment warmup ends: which hyperparameter is the prime suspect?

Pick the three first things to check when LoRA training loss sits flat from step 0

Compare LoRA r=8 vs r=64 on a 13B model on the quality vs cost curve

PremiumDefend the claim that…

Compute the LoRA trainable param count for a 4096x4096 projection at rank r=8

PremiumWhen IS fine-tuning the…

How does QLoRA fit a 65B fine-tune onto a single 48GB GPU?

Which statements about QLoRA are true?

PremiumWhy has prefix/prompt tuning…