Peft

Walk through what happens during backprop for a LoRA wrapped linear layer

LoRA freezes the base weights: does that mean activation checkpointing on the base is unnecessary?

Adam stores two fp32 buffers per trainable parameter: what does that imply for LoRA vs full FT on a 7B?

Estimate the trainable param count for a Llama-3 8B LoRA at r=16 targeting all linear modules.

Only 200 examples for a niche format task: which PEFT method gives the best shot?

Match each PEFT method to the standout advantage that defines it.

target_modules in a PEFT config: what does this list control?

Rank r in LoRA: what does the number actually count?

PEFT: unpack the acronym and the core problem it tackles.

When does a PEFT config actually need `modules_to_save` set, and what goes in it?

After calling `merge_and_unload()` on a PEFT model, what does the returned object look like?

In LoRA, where in the forward pass does `lora_dropout` apply its mask?

lora_alpha in plain English: what role does that scalar play?

Compute the LoRA trainable param count for a 4096x4096 projection at rank r=8

How does QLoRA fit a 65B fine-tune onto a single 48GB GPU?

Which statements about QLoRA are true?

PremiumWhy has prefix/prompt tuning…

Prompt tuning vs LoRA: pick the architectural reason LoRA tends to win

Match PEFT method to the scenario it fits best

PremiumMisconception: 'LoRA is just…

Best architecture for a chatbot over a frequently updated company knowledge base?

Which of these are linear layers that 'all linear' LoRA targets in a SwiGLU transformer?

PremiumWhen does targeting all…

PremiumPick a LoRA rank…

Match LoRA rank to task type