After a user reports the assistant quoting another company's internal doc, run the postmortem

Short answer·Hard·4.0 · 0·~3 min·Asked atMercor Persistent Promptlayer

Attempt it

A customer reports that your multi-tenant RAG assistant answered their question using content from a different company's internal document. Run the postmortem: what most likely went wrong at retrieval time, and what immediate and structural fixes do you put in place?

Free · 2 AI evals / day

This is a **cross-tenant data leak**, one of the most serious bugs a multi-tenant RAG system can have. The diagnosis starts from the symptom: a chunk belonging to a different company ended up in the prompt, and the model quoted it. So how did a foreign chunk get into retrieval at all? The usual cause is that the search was not scoped to the asking tenant. Common shapes: everyone's documents live in one shared index with no per-tenant filter, so a semantically similar foreign chunk ranked into the top-k. Or a filter ran after generation or only hid results in the UI, so the chunk still reached the model. Or the document's tenant tag was missing or wrong. The critical idea is that **the prompt is not a security boundary**. Once a foreign chunk is in the context window, the model can summarize or quote it, no matter what the system prompt says. Immediately, treat this as a security incident: use audit logs to find what leaked and to whom, notify per your obligations, and contain it. Structurally, enforce a tenant filter on the search itself, tied to the user's authenticated identity, and ideally give each tenant a separate index.

The symptom is unambiguous: a multi-tenant assistant surfaced another tenant's content, a cross-tenant data leak. The root cause is almost always the same class of bug: **retrieval was not scoped to the requesting tenant's access boundary**. The postmortem should name which shape it took. There are three common shapes. A single shared index with no tenant pre-filter, where a similar foreign chunk ranked into the top-k. A filter that ran in the wrong place — after generation, or only in the UI — so the forbidden chunk still reached the prompt. Or a metadata problem: a missing, stale, or wrong tenant tag, so even a correct filter let it through. The unifying lesson is that once a chunk is in context, the model can quote it; the prompt is not a security boundary. The immediate response is incident handling, not a quality patch. Determine what was exposed and to whom, audit retrieval logs to scope the blast radius, notify per your obligations, and contain via an emergency filter. The structural fixes are the real answer. Enforce a **mandatory tenant pre-filter at retrieval** so the search only considers the requesting tenant's chunks, with the tenant ID from authenticated identity, never anything the user can set. Prefer hard isolation — a per-tenant index or namespace — over a shared index trusting a single filter. Make missing or stale ACL metadata deny by default, and add continuous testing plus audit logging that asserts no result crosses a tenant boundary.

TL;DR

A cross-tenant leak means retrieval wasn't scoped to the tenant's ACL. Fix it with a mandatory pre-filter at search time from authenticated identity, ideally per-tenant isolation. The prompt is never the boundary.

Memory aid

Easy to grasp

Imagine a shared filing cabinet for many companies, where a clerk fetches papers for whoever asks. The right design is: before the clerk even opens a drawer, the building checks your badge and only unlocks your company's drawer. The leak happened because the clerk was allowed to pull papers from every drawer and was merely asked, politely, to ignore the ones that weren't yours — and one slipped through. Telling the clerk 'please don't read other companies' files' is not security; it's a suggestion. The fix is to lock the other drawers at the door, based on who you actually are (your badge, not a name you typed in). The strongest version gives each company its own locked cabinet entirely, so there is nothing to mix up.

Key concepts

Retrieval-Augmented Generation LLM System Design

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

A cross-tenant leak in a multi-tenant RAG product is the kind of incident that ends up in a breach notification and a customer's legal team's inbox. It is also a near-perfect interview question, because the wrong answers are revealing. A candidate who suggests fixing it with a better system prompt has no security model; a candidate who reaches first for retrieval scoping and authenticated identity has built real multi-tenant systems.

The scenario is deliberately framed as a postmortem because that is how these surface in the real world: a customer notices the assistant quoting something it should never have had access to, and now you have to reason backwards from a leaked chunk to the failure that let it in. The discipline the question rewards is treating the context window as a trust boundary — anything that reaches it is, for security purposes, already exposed.

This deep dive establishes the governing principle, enumerates the three root-cause shapes you should be able to name on sight, explains precisely why the prompt cannot be the control, and then lays out the immediate incident response and the structural fixes that make the leak impossible rather than merely unlikely.

The governing principle: the boundary is at retrieval

Start every multi-tenant RAG security discussion from one rule: a chunk that reaches the context window is, for security purposes, already disclosed. The model can quote it, paraphrase it, or summarize it, and no downstream control can reliably claw that back.

That rule forces the boundary upstream. The only safe place to enforce tenant isolation is at retrieval — the search must be constructed so that another tenant's chunks are never even candidates. If a foreign chunk can be a candidate and you are relying on something later to filter it out, you have already lost, because "later" includes the model itself, which is not a filter you control.

This is why the leak is a retrieval problem and not a generation problem. The model did exactly what models do: it used the context it was given. The defect is that the context contained data the caller had no right to. Every correct fix in this postmortem is about making that impossible at the search step.

Framing it this way also clarifies the incident severity. The moment a foreign chunk entered context, the leak occurred — whether or not the model chose to quote it in this particular response. You scope the incident by what could have been exposed across all affected queries, not just the one the customer happened to notice.

The three root-cause shapes

Why the prompt can never be the control

Immediate incident response

Structural fixes that make it impossible

Hard isolation, deny by default, and continuous testing

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Pinecone and Qdrant support per-tenant namespaces precisely so cross-tenant retrieval is physically impossible rather than filter-dependent
SaaS RAG products derive the tenant ID from the authenticated JWT or session, never from a request body field the client controls

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy is deriving the tenant ID from a user-supplied request field a vulnerability even with a correct pre-filter?

If the filter keys off a value the client controls, an attacker can change it to another tenant's ID and the pre-filter dutifully returns that tenant's chunks. The filter is only as trustworthy as its input. The tenant ID must come from the authenticated session or token the server validates, so the user cannot influence which tenant's data is in scope.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Trying to fix a cross-tenant leak with a prompt instruction telling the model to ignore other tenants' data — the prompt is not a security boundary, and once a foreign chunk is in context the model can quote it.

60 second bullets to scan on the way to the call.

Why a foreign chunk in the context window is already a leak
The three common root-cause shapes of a cross-tenant retrieval leak

Primary sources. Browse if you want the original framing.