PEFT: unpack the acronym and the core problem it tackles.

Attempt it

Parameter-Efficient Fine-Tuning. Instead of updating every weight in a multi-billion parameter base model, PEFT freezes the base and trains a tiny extra set of parameters (often under 1% of the total): adapter layers, LoRA matrices, prompt tokens, or a few bias terms. The win is storage and serving: one frozen base plus many small task-specific deltas (a few MB each) replaces shipping a full fine-tuned checkpoint per task, and training fits on far smaller hardware.

Your answer

Quick reference

TL;DR

PEFT freezes the base model and trains a tiny extra parameter set (often under 1% of weights) so one base can host many cheap task adapters.

Memory aid

Easy to grasp

Imagine a giant cookbook the chef has already memorised. Teaching that chef a brand-new cuisine the old way means rewriting every page, expensive, slow, and the cookbook gets enormous. PEFT does something cleverer. It leaves the cookbook untouched and clips a slim recipe card to a few key pages. The card is small, fast to write, and easy to swap. One chef can carry dozens of cards, one per cuisine, and pull the right one when needed. The cookbook (the frozen base model) never changes; the recipe cards (the tiny trainable adapters) carry the new behaviour. You get a per-task speciality without copying the whole book each time.

Key concepts

Parameter-Efficient Fine-Tuning LoRA Adapter layers

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 min: PEFT acronym + the full fine-tune pain it solves + the three method families + LoRA mechanics + multi-adapter serving + when full fine-tuning still wins.

Side-by-side

Aspect	Full fine-tuning	PEFT (LoRA-style)
Trainable parameters	100% of weights	Often under 1%
Optimizer memory	Roughly 3x model size	Roughly 3x the tiny delta
Checkpoint size per task	Full model copy	A few megabytes
Multi-task serving	One model per task in memory	One base hosts many adapters
Quality ceiling	Highest	Slightly lower, often within 1-2 points

Real-world examples

Real products, models, and research that use this idea.

Hugging Face PEFT library ships LoRA, AdaLoRA, IA3, prefix tuning, and prompt tuning under one API used across the open-weight ecosystem.
QLoRA fine-tunes of Llama 4 Maverick and DeepSeek V4 routinely run on a single 80 GB GPU thanks to 4-bit base quantization plus low-rank adapters.
vLLM and Text Generation Inference both support multi-LoRA serving, hosting hundreds of small adapters against one resident base model.
Anthropic and OpenAI fine-tuning endpoints expose LoRA-style PEFT under the hood so customer checkpoints stay small and cheap to host.
Mistral and Qwen 3.5 publish official LoRA recipes for domain adaptation, treating PEFT as the default rather than a fallback.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does PEFT typically not teach new factual knowledge well?

QHow does multi-LoRA serving actually work in vLLM?

QWhen would you pick adapters over LoRA?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating PEFT as a single algorithm. PEFT is an umbrella covering LoRA, adapters, prefix tuning, prompt tuning, and BitFit, each with different placement, memory, and quality trade-offs.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.