Lora

Prompt-tuning vs LoRA: pick the architectural reason LoRA tends to win

Match PEFT method to the scenario it fits best

Misconception: 'higher LoRA rank always wins on benchmarks'

Misconception: 'LoRA is just a quantization technique'

Merge LoRA into the base vs keep it as a swappable adapter: when do you pick which?

Multi-tenant SaaS with 500 customer-specific fine-tunes. Merge or swap?

Match training regime to sensible peak LR range

Fill in typical peak LRs for full FT, LoRA, and DPO

Which of these are linear layers that 'all-linear' LoRA targets in a SwiGLU transformer?

When does targeting all-linear with LoRA pay off vs. just W_q+W_v?

Pick a LoRA rank for a domain task and justify why r=512 isn't a free upgrade

Match LoRA rank to task type

LoRA+: why does B want a higher learning rate than A?

Why is LoRA's B initialised to zero and A to Gaussian. What breaks if both are Gaussian?

LoRA init: why B = 0 and A ~ Gaussian (not both Gaussian)?

Explain why a rank-r B·A update captures most of full fine-tuning's signal

Can you stack two LoRA adapters? What's the failure mode?

Stacking two unrelated LoRA adapters: most likely outcome

What does LoRA's α/r ratio control, and why is α = 2r a common default?

If you change LoRA's rank from 16 to 64 with α fixed, what actually happens?

IA3 vs LoRA: what does IA3 parameterise, and what's the param-count win?

Estimate peak VRAM for a 7B full-FT run in bf16 + Adam

Match fine-tuning framework to its strongest fit

What does DoRA decompose, and why does it beat LoRA at the same rank?

DoRA: which component does LoRA actually adapt?