PEFT: unpack the acronym and the core problem it tackles.

**PEFT** stands for Parameter-Efficient Fine-Tuning. It is a family of techniques that adapt a large pretrained model to a new task without rewriting all of its weights. Full fine-tuning updates every parameter in the base model. For a 70B model that means roughly 280 GB of optimizer state on top of the weights themselves, plus a fresh 140 GB checkpoint per task. PEFT skips that cost. It freezes the base entirely and trains a small extra set of parameters that sits alongside or inside the existing layers. Popular PEFT methods include LoRA, which factorises weight updates into two small low-rank matrices, adapter layers that insert tiny bottleneck blocks between transformer layers, prefix and prompt tuning, which prepend learnable vectors, and BitFit, which trains only bias terms. The payoff is storage and hardware. You can host one shared base model and serve dozens of task-specific adapters, each just a few megabytes. Training fits on far smaller GPUs, often a single consumer card for models that would otherwise need a multi-node cluster.

**PEFT** is Parameter-Efficient Fine-Tuning. The name captures the design goal: adapt a large base model to a new task while training only a tiny fraction of its weights, typically well under one percent. The motivation is straightforward. Full fine-tuning of a 70B model costs hundreds of gigabytes of optimizer state and produces a fresh 140 GB checkpoint per task. Storing one such checkpoint per customer, language, or domain stops being practical fast. PEFT replaces those bloated artifacts with small task deltas (a few megabytes each) that all share one frozen base. The family includes several recipes. LoRA factorises the weight update into two low-rank matrices and adds the product to existing projections. Adapter layers insert small bottleneck modules between transformer blocks. Prefix and prompt tuning prepend learnable virtual tokens to every layer or just the input. BitFit trains only the bias terms. The practical wins are three. Training fits on much smaller hardware, often a single consumer GPU for a 7B model. Serving stays cheap because one base model in memory can host many adapters, swapped per request. And the small deltas are easy to version, ship, and roll back, which matters more than people expect when you run dozens of fine-tunes in parallel.