Design access control for a multi-tenant RAG system where users may only see documents they are authorized for. Where in the pipeline must permissions be enforced, and why is enforcing them in the prompt unsafe?
Free · 2 AI evals / day
Access control in a multi-tenant RAG system must be enforced at **retrieval time**, not at generation time. The principle is simple: an unauthorized chunk should never enter the context window in the first place.
The way to do that is to attach access metadata to every document or chunk — owner, tenant, group membership, an access control list — and then constrain the retrieval query by the requesting user's identity. The vector search only ever considers chunks the user is allowed to see. This is called a pre-filter: you filter on permissions before or during ranking, so forbidden content is excluded from the candidate set entirely.
The unsafe design is to retrieve documents broadly and then ask the LLM to ignore the ones the user should not see. The problem is that once a forbidden chunk is sitting in the prompt, you have lost the guarantee. The model might quote it, summarize it, or be tricked past the instruction by a prompt injection hidden in the text.
Access control is an authorization decision. It belongs to the system you trust — the retrieval layer — not to a probabilistic model that only follows instructions most of the time.
Permissions must be enforced at **retrieval time** as a hard filter, never delegated to the generation prompt. The design goal is that an unauthorized chunk is never even a candidate, so it cannot reach the context window.
Concretely: every chunk carries access metadata — tenant ID, owner, group or role membership, an ACL. At query time you resolve the requesting user's identity into their permission set and join it into the retrieval filter. The vector search is constrained so it only considers chunks that user is authorized for. This is a pre-filter. The ACL constraint is applied before or during ranking, so forbidden chunks are excluded from the candidate pool, not removed after the fact.
Enforcing in the prompt is unsafe because it treats a probabilistic model as a security boundary. Once a forbidden chunk is in context, the guarantee is gone. The model can quote it, paraphrase it, leak it through a summary, or be steered past the 'ignore this' instruction by an indirect injection in the retrieved text. Authorization is a deterministic decision the retrieval system can make with certainty. The LLM cannot.
The production checklist has three parts. Partition or tag the index by tenant so cross-tenant leakage is structurally impossible. Fold the user's permission set into every metadata filter, and treat a missing, malformed, or stale ACL as deny by default. Post-filtering after ranking can starve k, so prefer pre-filtering and never put the decision in the prompt.
Quick reference
TL;DR
Enforce permissions at retrieval time as an identity-scoped pre-filter so unauthorized chunks never enter the prompt. A probabilistic model told to ignore forbidden chunks is not an access boundary.
Imagine a library where some shelves are off-limits to you. The safe rule is that the librarian only ever hands you books you're allowed to read — the forbidden ones never leave the locked room. The unsafe rule is the librarian piling every book on your desk, including the secret ones, with a sticky note saying "please don't read the red ones." Maybe you obey, maybe you peek, maybe someone tricks you into reading one aloud. In a retrieval system, the locked room is the search filter tied to who you are. Once a secret document is on the desk — in the model's context — there's no real guarantee it stays unread.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single-tenant systems where every user can see every indexed document. There is no authorization boundary to enforce, so per-user retrieval filtering adds query complexity and latency for no security gain.
Vector stores or ANN backends with no efficient filtered-search support, under tight latency. Pre-filtering may force naive post-filtering that starves recall; consider per-tenant index partitioning instead of metadata filters.
Highly dynamic, fine-grained permissions that change many times per minute per document. Keeping index-side ACL metadata in sync is impractical; a live permission-service join at query time fits better than cached index tags.
Quick scan · 2–4 min
Everything important, quickly.
Lead with the principle: authorization is enforced at retrieval time by a trusted system, never by the prompt. Describe identity-scoped pre-filtering — per-chunk ACL metadata joined with the user's resolved permission set so forbidden chunks are never candidates. Explain precisely why prompt enforcement fails: a probabilistic model is not a boundary, and indirect injection can defeat any 'ignore this' instruction once the chunk is in context. Then cover the hard parts: pre-filter versus post-filter and ANN starvation, tenant isolation via partitioning, deny by default on missing or stale ACLs, revocation propagation, and defense in depth with sanitization and audit logging.
Real-world examples
Real products, models, and research that use this idea.
1Pinecone and Qdrant support per-vector metadata filters, so a tenant ID and ACL filter can constrain the search to authorized chunks.
2Azure AI Search exposes security trimming where the query carries the user's group memberships to filter documents at search time.
3Glean and enterprise RAG platforms mirror each source system's permissions into the index so retrieval respects the original ACLs.
4Multi-tenant SaaS RAG often uses per-tenant namespaces or separate indexes so cross-tenant leakage is structurally impossible.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QNaive post-filtering after a top-k ANN search can return almost nothing for a user with sparse permissions. How do you keep recall while staying secure?
Sign in or go Pro to see the approach for this follow-up.
QA user's access to a document is revoked. What has to happen for retrieval to respect that immediately, and where do systems get this wrong?
Sign in or go Pro to see the approach for this follow-up.
QEven with correct retrieval-time filtering, why might you still sanitize retrieved documents, and what threat does that address?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Retrieving broadly and instructing the LLM to ignore unauthorized chunks — once a forbidden chunk is in the context window, no prompt instruction is a real access boundary.