A B2B RAG product has 5000 customer tenants ranging from 10 documents to 500k documents each. Compare the two main multi-tenancy patterns for the vector layer (namespace-per-tenant vs single shared index with a tenant_id filter). Cover correctness, cost, noisy neighbor risk, and the small-tenant cold start problem.
Free · 2 AI evals / day
**The two patterns.**
*Namespace-per-tenant.* Every tenant gets a dedicated index. Pinecone calls these namespaces; Weaviate uses classes; Qdrant uses collections. Strengths: bulletproof isolation, zero risk of cross-tenant data leakage, dropping a tenant is one DELETE. Weaknesses: per-index fixed costs (entry points, metadata, replication tier) multiplied by 5000 tenants are operationally heavy; a tenant with 10 documents pays the same overhead as a tenant with 500k.
*Shared index with `tenant_id` filter.* One big HNSW or IVF index holds everyone's vectors; every vector carries a `tenant_id` metadata field; queries are filtered on `tenant_id = X`. Strengths: the per index overhead is paid once across everyone; small tenants don't pay disproportionately; cold start is solved. Weaknesses: HNSW with a selective metadata filter is the classic 'filter broken recall' trap. If the filter is highly selective (your 10-document tenant), the graph walk reaches a node, finds none of its neighbors pass the filter, and either stalls (returns fewer than K) or has to do a much wider search.
**The 5000-tenant reality.** With a 10-to-500k document range, the distribution is heavy tailed. The top 1% of tenants probably hold 50%+ of the volume. The right pattern is a hybrid: dedicated namespaces for the heavy tenants (they would dominate any shared index anyway), shared index for the long tail of small tenants (per tenant overhead would crush them).
**Vendor support.** Pinecone explicitly recommends this hybrid in their 2025+ multi-tenancy guide. Qdrant supports tenant partitioned payload indexing for the shared-collection variant. Weaviate has multi-tenancy as a first-class feature on classes.
**The two patterns and their tradeoffs.**
**Namespace-per-tenant.** Each tenant gets its own index instance (Pinecone namespace, Weaviate class, Qdrant collection, pgvector schema or table). What you gain: complete isolation (no cross-tenant query bleed, no shared graph), easy deletion (drop one collection), bounded noisy neighbor blast radius (a hot tenant's write traffic does not impact others' query latency unless they share compute pool), per tenant tuning (each tenant can have its own M, ef_search, even its own embedding model). What you lose: 5000 indexes carry 5000 sets of per index overhead. Each HNSW index has an entry point, a graph metadata structure, a replication tier; cold start for small tenants is poor because the graph has too few nodes to be a good search structure. Cost-per-tenant is roughly constant, so a tenant with 10 documents pays nearly as much as one with 500k in fixed overhead.
**Shared index with tenant_id filter.** One large HNSW or IVF index spans all tenants; every vector has a `tenant_id` metadata field; queries are `vector_search(query) WHERE tenant_id = X`. What you gain: per index overhead is paid once for everyone; small tenants pay only their proportional share of storage; cold start is a non-issue; index build cost is amortized. What you lose: this is the classic filter broken recall configuration for HNSW. The graph walk can hit a node whose neighbors are all of tenant Y, find zero passing the `tenant_id = X` filter, and stall short. Many systems work around this by post-filtering (search top 100, filter, return top 10) or by pre-filtering (only consider nodes matching the filter); both have failure modes on highly selective filters. Tenant deletion is also expensive (scan and tombstone every matching vector). Noisy-neighbor risk is real: one tenant pushing high write QPS can affect everyone's query latency.
**The hybrid (what 5000-tenant B2B products actually deploy).**
- **Tier 1 (dedicated):** The top 10 to 20
Quick reference
TL;DR
Namespace-per-tenant gives strong isolation but pays per tenant overhead, hurting small tenants. Shared-index-with-filter amortizes cost but hits the filter broken recall HNSW pitfall and noisy neighbor risk.
Imagine you run a self-storage facility, and 5000 small businesses are renting space in your search system. **Option A: a separate room per tenant.** Their stuff is totally separated; nobody can peek; you can clear out a tenant by emptying their room. But you pay rent on 5000 rooms, and many of them are nearly empty. **Option B: one giant warehouse with everyone's stuff mixed and tagged with stickers.** Cheaper rent overall, but when a small tenant asks for their stuff, you have to search through the whole warehouse looking for their sticker, and sometimes the warehouse layout makes you miss their items. **The answer in real life:** put your three biggest tenants (who fill the warehouse) in their own rooms anyway, and pool the other 4997 into the shared warehouse.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Tenant distribution is uniform (every tenant has roughly the same size) The hybrid's value comes from heavy-tail dynamics; in a uniform world a single pattern usually suffices.
Compliance requires physical-level data separation per tenant Shared indexes cannot satisfy 'data must not co-mingle in the same file or process'; you must go namespace-per-tenant or further.
Sub-100 tenants and each has at least 10k documents Namespace-per-tenant is simpler and the long tail cost problem doesn't bite at this scale.
Quick scan · 2–4 min
Everything important, quickly.
5 min: walk through both pure patterns + filter broken recall pitfall + heavy-tail distribution argument + layered hybrid recommendation + 2026 vendor primitives.
Side-by-side
Concern
Namespace per tenant
Shared index + filter
Hybrid
Isolation
Strong
Filter-only (not a security boundary)
Strong for whales, filter for tail
Per tenant cost
Constant overhead (bad for small tenants)
Amortized
Right-sized per tier
Cold-start
Bad for tiny tenants
Non-issue
Solved per tier
Noisy-neighbor
Bounded per index
Cross-tenant interference
Bounded for whales, pooled for tail
GDPR delete
Drop one collection
Scan + tombstone (expensive)
Drop for whales, scan-filtered for tail
HNSW recall safety
Always good
Filter-broken-recall risk
Good for whales; mitigated by filterable HNSW for tail
Real-world examples
Real products, models, and research that use this idea.
1Pinecone's multi-tenancy guide explicitly recommends the hybrid: namespaces for high volume tenants, shared index with tenant_id filter for the long tail.
2Qdrant added tenant partitioned payload indexing to mitigate filter broken recall in shared collections.
3Weaviate has multi-tenancy as a first-class feature on classes with active tenant memory residency.
4Vespa's filterable HNSW implements graph level filter awareness so selective filters don't strand the walk.
5Notion, Glean, and similar B2B RAG products publicly describe layered tenant tiers (whales / mid tail / long tail) in engineering talks.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does Qdrant payload indexed search mitigate filter broken recall?
Sign in or go Pro to see the approach for this follow-up.
QWhat is the cost of GDPR 'forget this tenant' in each pattern?
Sign in or go Pro to see the approach for this follow-up.
QIf you must use pgvector, what's the right multi-tenancy pattern?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Picking 'shared index with tenant_id filter' as universally right because it sounds efficient, without noticing that HNSW with a selective metadata filter is exactly the configuration that returns fewer than K results.