Match PEFT method to the scenario it fits best

Attempt it

LoRA

QLoRA

IA3

Prefix-tuning

Full fine-tuning

Quick reference

TL;DR

Each PEFT method has one defining trade-off. Match the scenario's scarcest resource, VRAM, data size, or per-task storage, to the method whose mechanism solves exactly that.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Think of customising a rental car you must hand back unchanged. LoRA is a clip-on steering cover that subtly changes how it drives. QLoRA is the same cover, but you first deflate the seats so the whole car fits in a tiny garage. IA3 is just turning a few knobs up or down, perfect when you only get a short test drive and don't want to overdo it. Prefix tuning slips a sticky note onto the dashboard the car reads first, cheap to swap per renter. Full fine-tuning means rebuilding the engine, powerful, but only worth it with a big workshop and lots of time. You pick by what is scarce: space, data, or per-renter storage.

Key concepts

Parameter-efficient fine-tuning LoRA QLoRA IA3 activation scaling

Quick scan · 2–4 min

Everything important, quickly.

5 min: state each method's mechanism, then map all five scenarios by their binding constraint, then defend QLoRA as a VRAM call and full FT as the rank-ceiling escape.

Side-by-side

Method	Mechanism	Picks it when
LoRA	Low-rank delta beside frozen weights	Base fits in bf16; you want the quality default
QLoRA	LoRA delta over a 4-bit frozen base	VRAM is the bottleneck
IA3	Per-channel activation scaling vectors	Tiny dataset; overfitting is the risk
Prefix tuning	Learned soft tokens per layer	Many tiny tasks; per-task storage must be small
Full fine-tuning	Update all weights	Aggressive domain shift with large data

Real-world examples

Real products, models, and research that use this idea.

Hugging Face PEFT ships LoRA, QLoRA, IA3, and prefix tuning behind one config, so teams swap methods by changing a few lines.
QLoRA's 4-bit recipe made fine-tuning a 65B model on a single 48 GB GPU routine, and it is the default for community Llama 4 fine-tunes.
Predibase and Together.ai serve thousands of per-customer LoRA adapters over one frozen base, the multi-tenant pattern prefix tuning also targets.
Unsloth optimises LoRA and QLoRA training kernels, cutting consumer-GPU fine-tune time for open-weight bases like Llama 4 and DeepSeek V4.
Researchers fine-tuning a domain model on a few hundred curated examples often reach for IA3 to avoid the overfitting a full LoRA would invite.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does QLoRA recover most of LoRA's quality despite a 4-bit frozen base?

Sign in or go Pro to see the approach for this follow-up.

QHow does IA3 differ mechanically from LoRA, and why does that change its overfitting profile?

Sign in or go Pro to see the approach for this follow-up.

QWhen does serving many LoRA adapters beat prefix tuning for multi-tenant deployment?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.