Design the retrieval architecture when 95% of queries carry five or more metadata predicates of wildly varying selectivity.

Short answer·Hard·4.0 · 0·~3 min·Asked atAmazon Dify Tata Digital

Attempt it

An enterprise document platform runs vector search where 95% of queries combine semantic similarity with five or more metadata predicates (department, ACL groups, date ranges, doc type, region), and predicate selectivity swings from 0.01% to 80% depending on the query. Design the architecture: how metadata is indexed, how the engine chooses a filtering strategy per query, and what you partition versus filter.

Free · 2 AI evals / day

A **vector database** does two things at once on every query: it finds vectors close to the query vector, and it keeps only those whose metadata satisfies the request. The first part is fast on its own; mixing in metadata is where designs break. The key idea is that there is no single best way to combine the two. If the filter is very tight (say only 50 documents survive), the smartest move is to grab those 50 first and brute-force the similarity, because brute force on 50 items is trivial and recall is perfect. If the filter is very loose (most documents pass), you run the normal vector search and drop the few non-matches at the end. In between, you walk the vector graph but skip past nodes that do not match the filter. The other half of the design is what becomes a partition (a separate physical bucket searches never cross) versus a runtime filter. Tenant and ACL groups become partitions because every query has them and they enforce security. Volatile things like document type or tags stay as filters because partitioning by every combination would create thousands of tiny, mostly-empty indexes.

**Filtered ANN with five-plus predicates is a query-planning problem, not an index choice.** Three pieces matter. First, **index the metadata properly**. Every filterable field needs a real secondary index colocated with the vectors (payload indexes in Qdrant, B-tree or GIN in pgvector). Without indexes, predicates cannot be evaluated quickly and cardinality cannot be estimated at all. Second, **pick a strategy per query based on estimated selectivity of the conjunction**. The planner needs three modes: - **Tight filters** (under roughly 100k survivors): pre-filter, then exact search. Recall is 1.0 and the graph is bypassed. - **Mid selectivity**: filter-aware ANN. Traverse the HNSW graph but skip non-matching nodes (ACORN-style), or restrict IVF probes to matching entries. - **Broad filters** (most candidates pass): ANN-first with post-filter and modest oversampling. The graph runs unconstrained and the filter is a cheap final pass. Third, **partition what is always present and ownership-shaped; filter what is volatile.** Tenant and ACL become partitions or namespaces because security demands structural isolation and they appear on every query. Date, doc type, and tags stay as filters because partitioning their combinations would explode into thousands of micro-indexes. Date ranges deserve time-based segments so cold data can be tiered or pruned. Operationally, the planner needs cardinality statistics, per-strategy recall probes, and ACL enforced at the API layer so a malformed query can never drop it.

TL;DR

Treat filtered ANN as query planning: index metadata for cardinality, pick pre, during, or post-filter per selectivity, partition stable ownership, filter volatile predicates.

Memory aid

Easy to grasp

Imagine a giant library where every book has a coordinate (the vector) and a stack of sticky notes (metadata). Someone asks for books near a topic, written by the legal department, in French, from last quarter, that the marketing team can read. If only twelve books match those notes, you just hand them over and forget the coordinate. If almost every book matches, you do the coordinate search normally and toss the few that fail the notes. In the messy middle, you walk the library map but skip past any shelf whose notes do not fit. The librarian who picks the right approach per request, based on how many books each filter narrows down to, is the query planner. The architecture is really about teaching the planner to count quickly.

Key concepts

Vector Databases

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Filtered vector search at five-plus predicates is where many production systems quietly degrade. A team that picked one strategy at design time eventually meets a query whose selectivity sits on the opposite side of the bell curve, and either latency triples or recall craters. The lesson learned across Qdrant, Weaviate, Milvus, Pinecone, and pgvector deployments is the same: at this complexity, filtering stops being an index property and becomes a planning problem.

This deep-dive walks through the architecture in four passes. First, the planner: what data structures it needs, how it estimates selectivity, and how it dispatches. Second, the three executor modes and the selectivity bands where each is optimal. Third, the partition-versus-filter rule and why combinatorial partitioning is a trap. Fourth, the operational pieces (ACL enforcement, observability) that keep the design honest as traffic shape drifts.

Mental model: pretend you are designing a relational query planner. The vector similarity is one more operator, ranking-only, that the planner can sequence before, during, or after the WHERE clause.

The planner: cardinality first, strategy second

What the planner needs

The planner takes a conjunction of predicates plus a query vector and produces an execution plan. It cannot do that without two ingredients. The first is real secondary indexes on every filterable field, colocated with the vectors so the engine can intersect predicate posting lists in one pass. The second is cardinality statistics: per-field histograms or sketches, plus enough correlation data to estimate selectivity(p1 AND p2 AND ... AND p5) without scanning the whole index.

A classic independence-assumption estimate multiplies per-predicate selectivities. That works when predicates are uncorrelated and breaks badly when they are not (department and ACL group are highly correlated; date and doc type usually are not). Production planners patch the gap with a correlation table for known-correlated pairs and a small safety multiplier on the rest.

What the planner emits

The planner emits one of three executor choices plus a few parameters: oversample_factor for post-filter, pruning_aggressiveness for filter-aware traversal, and a recall budget tag for observability. The output is intentionally small; the cleverness lives in the estimator, not the executor.

Updating the statistics

Statistics drift. Date ranges shift, new tenants land, ACL groups change shape. A robust planner has a background refresh on histograms plus an online learning loop that compares the planner's predicted selectivity to the measured pass rate after execution. When the prediction error exceeds a band for a query class, alarm and recompute.

The three executor modes and where each wins

Partition versus filter: the cost of getting this wrong

ACL as a structural contract and the observability that keeps the design honest

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Qdrant ships payload indexes plus ACORN-style filtered HNSW so the engine can pre, post, or filter-traverse per query without app-level glue.
pgvector with HNSW relies on Postgres B-tree and GIN indexes for predicates; the query planner uses statistics to choose between pre-filter and post-filter paths.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the planner estimate selectivity without scanning every predicate?

Maintain histograms or sketches per field plus a correlation table for common predicate pairs. For unknown conjunctions, fall back to independence assumption plus a safety multiplier; recalibrate on observed selectivity post-execution.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Hardcoding one filter strategy (always pre or always post) and watching recall or latency collapse on the half of traffic whose selectivity does not match the chosen mode.

60 second bullets to scan on the way to the call.

Three filter modes and the selectivity bands where each wins
Why payload or secondary indexes are required for cardinality estimation

Primary sources. Browse if you want the original framing.