Multi-tenant SaaS with 500 customer-specific fine-tunes. Merge or swap?
With 500 distinct customer fine-tunes, keep LoRA adapters swappable on one shared base via S-LoRA or vLLM multi-LoRA. Merging only fits a single static task.
Picture a library with one giant reference book that everyone shares. Each customer also has a thin sticky-note pack that tweaks how the book answers them. Keeping the packs separate means one heavy book sits on the desk, and the librarian just clips on whichever customer's notes arrived with the question. That is swappable adapters: one base model in memory, 500 tiny adapters clipped in per request. Merging would be like photocopying the entire reference book 500 times, once per customer, so each gets their notes baked in. That fills the whole building and you cannot keep them all on the desk. Merge only when one customer needs one frozen book forever.
Detailed answer & concept explanation~7 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: merge vs swap mechanics + serving-overhead trade + 500-tenant storage math + why averaging and pooling fail + the single-tenant merge case + which systems serve multi-LoRA.
| Concern | Merge into base | Keep adapters swappable |
|---|---|---|
| Best for | Single static task, one behaviour | Multi-tenant, many concurrent fine-tunes |
| Serving overhead | Zero (runs as plain base) | Small per-request adapter apply, amortised by batching |
| Storage at 500 tenants | 500 full base copies (terabytes) | One base plus 500 small adapter files |
| Hot-swap per request | No (frozen checkpoint) | Yes (page adapter in per request) |
| Data isolation | Per copy, but copies explode | Each adapter stays its own file |
Real products, models, and research that use this idea.
- Predibase and LoRAX serve hundreds of customer LoRA adapters over a shared base, paging adapters in per request rather than merging.
- vLLM multi-LoRA lets a single Llama 4 base in GPU memory batch requests that each carry a different tenant adapter.
- The S-LoRA system from UC Berkeley demonstrated serving thousands of concurrent LoRA adapters on one base with unified paging.
- OpenAI's hosted fine-tuning serves many customer fine-tunes behind shared base infrastructure rather than dedicating a model per customer.
- Hugging Face PEFT exposes merge_and_unload for the single-task merge case and adapter hot-swapping for the multi-tenant case.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does S-LoRA batch requests that each carry a different adapter in one forward pass?
QWhen could merging a handful of related adapters via task arithmetic actually work?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Merging by default. Merging suits one static single-task deploy, but 500 tenants need swappable adapters on a shared base, or storage and isolation both collapse.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.