With 500 distinct customer fine-tunes, keep LoRA adapters swappable on one shared base via S-LoRA or vLLM multi-LoRA. Merging only fits a single static task.
Picture a library with one giant reference book that everyone shares. Each customer also has a thin sticky-note pack that tweaks how the book answers them. Keeping the packs separate means one heavy book sits on the desk, and the librarian just clips on whichever customer's notes arrived with the question. That is swappable adapters: one base model in memory, 500 tiny adapters clipped in per request. Merging would be like photocopying the entire reference book 500 times, once per customer, so each gets their notes baked in. That fills the whole building and you cannot keep them all on the desk. Merge only when one customer needs one frozen book forever.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This question looks like a fine-tuning question but is really a serving-architecture question. You have already trained 500 customer-specific LoRA adapters. The open decision is how to put them into production, and the two real options are merging each adapter into base weights or keeping the adapters swappable on a shared base. The other two choices on the table, averaging the adapters and training one giant adapter, are distractors that quietly destroy the specialisation you built.
The trap is that merging is what almost every tutorial demonstrates. You fine-tune, you call a merge utility, you get one clean model, and you deploy it. That workflow is correct for the demo case of one adapter and one task. It quietly assumes a single behaviour per served model, and that assumption shatters the moment you have hundreds of tenants who each expect their own specialisation. The merge habit is so ingrained that many engineers never notice they are making an architectural choice at all.
The right framing is an economics one. Merging trades storage and flexibility for zero serving overhead. Swappable serving trades a small per-request cost for the ability to share one expensive base across every tenant. The base model is the dominant cost in both compute and memory, so the question becomes whether you replicate that expensive base per tenant or share it across all of them. At 500 tenants the storage and isolation math makes the swappable design the only viable one, while the merge design remains the right call for a single static deployment.
This deep dive walks what merging actually does to the weights, what swappable serving costs and saves, why the two averaging and pooling distractors fail, and where merging legitimately wins.
What merging a LoRA actually does
A LoRA adapter learns two low-rank matrices whose product is added to a frozen base weight matrix. During training the base stays fixed and only these small matrices update. The effective weight is the base plus a scaled low-rank update. The two matrices are tiny relative to the base, which is why adapters are only a few hundred megabytes while the base is many gigabytes.
Merging computes that update and folds it permanently into the base matrix. The formula is direct:
Here the base matrix is W0, the two learned matrices are B and A, and the scalar alpha over r is the LoRA scaling factor fixed at training time. Once you compute this sum, the result is just an ordinary weight matrix of the same shape as the base.
After merging, there is no separate adapter and no extra runtime math. The model runs exactly like any plain checkpoint at full base speed. That zero-overhead property is the entire appeal of merging, and it is genuinely valuable when you have one behaviour to serve.
The cost is rigidity. The behaviour is now baked into the weights, so one merged checkpoint encodes exactly one customer's adaptation. To serve a second customer you need a second full copy of the base, not just a second small adapter. The lightweight, composable nature of the adapter is gone the instant you merge it.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Concern | Merge into base | Keep adapters swappable |
|---|---|---|
| Best for | Single static task, one behaviour | Multi-tenant, many concurrent fine-tunes |
| Serving overhead | Zero (runs as plain base) | Small per-request adapter apply, amortised by batching |
| Storage at 500 tenants | 500 full base copies (terabytes) | One base plus 500 small adapter files |
| Hot-swap per request | No (frozen checkpoint) | Yes (page adapter in per request) |
| Data isolation | Per copy, but copies explode | Each adapter stays its own file |
Real products, models, and research that use this idea.
- Predibase and LoRAX serve hundreds of customer LoRA adapters over a shared base, paging adapters in per request rather than merging.
- vLLM multi-LoRA lets a single Llama 4 base in GPU memory batch requests that each carry a different tenant adapter.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does S-LoRA batch requests that each carry a different adapter in one forward pass?
Think about a unified paging system for adapter weights plus a custom kernel that gathers the right adapter rows per request, so the base GEMM stays shared and only the low-rank residual differs per sequence.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Merging by default. Merging suits one static single-task deploy, but 500 tenants need swappable adapters on a shared base, or storage and isolation both collapse.
60 second bullets to scan on the way to the call.
What merging a LoRA into the base actually does to the weights
Why merge gives zero serving overhead but one frozen behaviour
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.