Enforcing per user document permissions in a multi-tenant RAG system: where does access control belong?

Short answer·Hard·4.0 · 0·~3 min·Asked atIBM Jump Trading Lakera

Attempt it

Design access control for a multi-tenant RAG system where users may only see documents they are authorized for. Where in the pipeline must permissions be enforced, and why is enforcing them in the prompt unsafe?

Free · 2 AI evals / day

Access control in a multi-tenant RAG system must be enforced at **retrieval time**, not at generation time. The principle is simple: an unauthorized chunk should never enter the context window in the first place. The way to do that is to attach access metadata to every document or chunk — owner, tenant, group membership, an access control list — and then constrain the retrieval query by the requesting user's identity. The vector search only ever considers chunks the user is allowed to see. This is called a pre-filter: you filter on permissions before or during ranking, so forbidden content is excluded from the candidate set entirely. The unsafe design is to retrieve documents broadly and then ask the LLM to ignore the ones the user should not see. The problem is that once a forbidden chunk is sitting in the prompt, you have lost the guarantee. The model might quote it, summarize it, or be tricked past the instruction by a prompt injection hidden in the text. Access control is an authorization decision. It belongs to the system you trust — the retrieval layer — not to a probabilistic model that only follows instructions most of the time.

Permissions must be enforced at **retrieval time** as a hard filter, never delegated to the generation prompt. The design goal is that an unauthorized chunk is never even a candidate, so it cannot reach the context window. Concretely: every chunk carries access metadata — tenant ID, owner, group or role membership, an ACL. At query time you resolve the requesting user's identity into their permission set and join it into the retrieval filter. The vector search is constrained so it only considers chunks that user is authorized for. This is a pre-filter. The ACL constraint is applied before or during ranking, so forbidden chunks are excluded from the candidate pool, not removed after the fact. Enforcing in the prompt is unsafe because it treats a probabilistic model as a security boundary. Once a forbidden chunk is in context, the guarantee is gone. The model can quote it, paraphrase it, leak it through a summary, or be steered past the 'ignore this' instruction by an indirect injection in the retrieved text. Authorization is a deterministic decision the retrieval system can make with certainty. The LLM cannot. The production checklist has three parts. Partition or tag the index by tenant so cross-tenant leakage is structurally impossible. Fold the user's permission set into every metadata filter, and treat a missing, malformed, or stale ACL as deny by default. Post-filtering after ranking can starve k, so prefer pre-filtering and never put the decision in the prompt.

TL;DR

Enforce permissions at retrieval time as an identity-scoped pre-filter so unauthorized chunks never enter the prompt. A probabilistic model told to ignore forbidden chunks is not an access boundary.

Memory aid

Easy to grasp

Imagine a library where some shelves are off-limits to you. The safe rule is that the librarian only ever hands you books you're allowed to read — the forbidden ones never leave the locked room. The unsafe rule is the librarian piling every book on your desk, including the secret ones, with a sticky note saying "please don't read the red ones." Maybe you obey, maybe you peek, maybe someone tricks you into reading one aloud. In a retrieval system, the locked room is the search filter tied to who you are. Once a secret document is on the desk — in the model's context — there's no real guarantee it stays unread.

Key concepts

RAG LLM System Design

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Multi-tenant RAG is where security and machine learning collide, and the collision exposes a tempting but dangerous shortcut. Because the LLM follows instructions, an engineer is tempted to handle permissions the way they handle everything else with an LLM: write an instruction. "Here are the retrieved documents; only use the ones this user is allowed to see." It reads as reasonable. It is a data breach waiting to happen.

The question is testing whether a candidate understands a boundary that does not move: a probabilistic model is never a security control. This deep dive establishes where authorization actually belongs, walks through exactly how prompt-side enforcement fails under adversarial pressure, and then covers the parts that make production multi-tenant retrieval genuinely hard — filtered ANN and recall starvation, tenant isolation, failure semantics, and revocation.

The boundary that does not move: authorize in the query

Access control answers a deterministic question: is this identity allowed to see this resource? The answer is yes or no, and a system you trust — your retrieval and authorization layer — can compute it with certainty. The entire design follows from refusing to delegate that question to anything that answers probabilistically.

The mechanism is identity-scoped retrieval. At index time, every chunk is stored with access metadata: tenant ID, owner, the groups or roles permitted, or a reference to an ACL. At query time, the user's authenticated identity is resolved into a concrete permission set, and that set is folded into the retrieval filter. The search is constrained so it only ever considers chunks the user is authorized for.

The crucial property is that this is a pre-filter. The ACL constraint applies before or during ranking, which means an unauthorized chunk is never a candidate. It does not compete for a top-k slot, it is not retrieved, and it never reaches the prompt. Contrast that with the alternative of retrieving everything and trimming later: the difference is whether forbidden content was ever in the system's hands at all. A pre-filter makes the unsafe state — a forbidden chunk in context — structurally unreachable, which is exactly what a security boundary should do.

Why a prompt instruction is not a control

Pre-filter versus post-filter and the ANN starvation trap

Failure semantics, revocation, and defense in depth

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Pinecone and Qdrant support per-vector metadata filters, so a tenant ID and ACL filter can constrain the search to authorized chunks.
Azure AI Search exposes security trimming where the query carries the user's group memberships to filter documents at search time.

What an interviewer would ask next. Try answering before peeking at the approach.

QNaive post-filtering after a top-k ANN search can return almost nothing for a user with sparse permissions. How do you keep recall while staying secure?

Use filtered ANN where the index supports it, so the permission constraint is applied during graph traversal rather than after, keeping the returned k authorized and full. Alternatively over-fetch a larger candidate set before filtering, or partition the index by tenant so the search space is already scoped. The goal is to apply the ACL inside the search, not as a lossy post-step.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Retrieving broadly and instructing the LLM to ignore unauthorized chunks — once a forbidden chunk is in the context window, no prompt instruction is a real access boundary.

60 second bullets to scan on the way to the call.

State that authorization is enforced at retrieval time, never in the prompt
Describe identity-scoped pre-filtering with per-chunk ACL metadata

Primary sources. Browse if you want the original framing.