Multi-tenant SaaS with 500 customer-specific fine-tunes. Merge or swap?

Attempt it

Quick reference

TL;DR

With 500 distinct customer fine-tunes, keep LoRA adapters swappable on one shared base via S-LoRA or vLLM multi-LoRA. Merging only fits a single static task.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Picture a library with one giant reference book that everyone shares. Each customer also has a thin sticky-note pack that tweaks how the book answers them. Keeping the packs separate means one heavy book sits on the desk, and the librarian just clips on whichever customer's notes arrived with the question. That is swappable adapters: one base model in memory, 500 tiny adapters clipped in per request. Merging would be like photocopying the entire reference book 500 times, once per customer, so each gets their notes baked in. That fills the whole building and you cannot keep them all on the desk. Merge only when one customer needs one frozen book forever.

Key concepts

LoRA adapters Multi-tenant serving S-LoRA / multi-LoRA Adapter merging

Quick scan · 2–4 min

Everything important, quickly.

4 min: merge vs swap mechanics + serving-overhead trade + 500-tenant storage math + why averaging and pooling fail + the single-tenant merge case + which systems serve multi-LoRA.

Side-by-side

Concern	Merge into base	Keep adapters swappable
Best for	Single static task, one behaviour	Multi-tenant, many concurrent fine-tunes
Serving overhead	Zero (runs as plain base)	Small per-request adapter apply, amortised by batching
Storage at 500 tenants	500 full base copies (terabytes)	One base plus 500 small adapter files
Hot-swap per request	No (frozen checkpoint)	Yes (page adapter in per request)
Data isolation	Per copy, but copies explode	Each adapter stays its own file

Real-world examples

Real products, models, and research that use this idea.

Predibase and LoRAX serve hundreds of customer LoRA adapters over a shared base, paging adapters in per request rather than merging.
vLLM multi-LoRA lets a single Llama 4 base in GPU memory batch requests that each carry a different tenant adapter.
The S-LoRA system from UC Berkeley demonstrated serving thousands of concurrent LoRA adapters on one base with unified paging.
OpenAI's hosted fine-tuning serves many customer fine-tunes behind shared base infrastructure rather than dedicating a model per customer.
Hugging Face PEFT exposes merge_and_unload for the single-task merge case and adapter hot-swapping for the multi-tenant case.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does S-LoRA batch requests that each carry a different adapter in one forward pass?

Sign in or go Pro to see the approach for this follow-up.

QWhen could merging a handful of related adapters via task arithmetic actually work?

Sign in or go Pro to see the approach for this follow-up.

QWhat changes if adapters have different ranks or target different layers across tenants?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.