Prompt tuning vs LoRA: pick the architectural reason LoRA tends to win
Prefix and prompt tuning only steer the model through input vectors; LoRA edits the weight matrices directly, so it adapts harder per parameter and merges to zero inference cost.
Imagine a huge orchestra playing a fixed score. Prompt tuning is like whispering instructions to the conductor before the piece starts. The musicians never change, you just nudge what they hear at the very beginning, and that nudge has to ripple all the way through. LoRA is like handing every section a small set of edits to their own sheet music. The cellos, the brass, the percussion each get tweaked directly, so the whole sound shifts with far less effort. Whispering to the conductor can only do so much. Editing the parts themselves gives you real control. And once LoRA's edits are pencilled into the score, the performance costs nothing extra to play.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Prefix tuning, prompt tuning, and LoRA are all parameter-efficient fine-tuning methods. They share a goal: adapt a large frozen base model to a new task while training only a tiny fraction of its parameters. The interview trap is to compare them on surface features like memory or latency. The real distinction, and the reason LoRA usually wins on quality, is architectural. Each method is allowed to edit a different part of the network.
Soft-prompt methods, the family that includes prompt tuning and prefix tuning, never touch a single weight. They learn continuous vectors and inject them at the input embeddings or as a prefix to the attention keys and values. The frozen model then has to interpret those vectors and let their influence propagate through attention. That is an indirect, input-space form of control.
LoRA takes the opposite stance. It leaves the original weights frozen too, but adds a small low-rank delta directly to chosen weight matrices. The adaptation lives inside the transform the model computes, not in the text it reads. This deep dive walks through what each family changes, why the input-space route has a ceiling, the cost story at serving time, and when soft prompts still earn a place in the toolbox.
Soft prompts steer through the input
Prompt tuning learns a handful of continuous vectors that are prepended to the input embedding sequence. Prefix tuning goes a little deeper, prepending learned vectors to the keys and values at every attention layer. P-tuning is a close cousin that reparameterises the learned prompt through a small encoder. In all three the base weight matrices stay completely frozen.
It helps to be precise about the distinction inside this family. Prompt tuning touches only layer zero, the embedding input. Prefix tuning injects fresh learned key-value vectors at each layer, which gives it slightly more reach than pure prompt tuning. Even so, both remain soft-prompt methods, because the injected vectors are read through attention rather than baked into any projection.
The shared property is that adaptation is mediated entirely by attention. The model attends to these learned vectors as if they were extra context, and whatever behavior change you want has to be expressible as something the frozen network can read and react to. Nothing about the underlying computation changes.
This is elegant and extremely cheap. A prompt-tuning checkpoint can be a few kilobytes. You can keep thousands of them and switch tasks by swapping a tiny vector. The cost is paid at quality, because steering from the input is a narrow channel into a fixed function.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Concern | Prefix / prompt tuning | LoRA |
|---|---|---|
| Where it edits | Input or key-value vectors only | Attention and MLP weight matrices |
| Weights changed | None; base stays frozen | Low-rank delta on chosen matrices |
| Representational ceiling | Capped by input-space bottleneck | Much higher; per-layer leverage |
| Inference cost | Extra key-value tokens every request | Zero after merging into base |
| Typical use | Light task switching, many tiny adapters | Serious behavior and reasoning change |
Real products, models, and research that use this idea.
- Hugging Face PEFT ships LoRA, prefix tuning, prompt tuning, and P-tuning side by side, and LoRA is by far the most downloaded adapter type for Llama 4 fine-tunes.
- QLoRA fine-tunes of Llama 4 and DeepSeek V4 on a single consumer GPU are the default community recipe, with prefix tuning rarely chosen for serious behavior change.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does adding more soft-prompt tokens eventually stop improving quality?
Frame it as an architectural bottleneck, not a capacity budget. The influence is still mediated through attention from the input side, so extra tokens add diminishing steering signal rather than new computational leverage.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Picking the latency answer. Merging to zero overhead is a real LoRA perk, but it is a side benefit. The quality gap comes from where each method gets to edit the network.
60 second bullets to scan on the way to the call.
Where each method injects its learned parameters
Why input-space steering hits a representational ceiling
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.