Aggressive metadata filtering can silently destroy recall on an HNSW index even when matches exist. Why?

HNSW works by building a multi layer proximity graph at index time. Each vector becomes a node; edges connect each node to its approximate nearest neighbors. At query time, the search does a **greedy graph walk**: start at an entry node, look at all its neighbors, jump to the closest one to the query, repeat. The walk converges when no neighbor is closer than the current node. This works beautifully when there are no filters. The problem is that the graph edges were chosen at build time based on vector similarity alone; the index knows nothing about your metadata predicates. **What happens when you add a tight filter.** Suppose your query is 'find similar vectors WHERE user_id = X AND created_at > Y'. The search arrives at a node and looks at its neighbors; the implementation filters those neighbors against the predicate before adding them to the candidate list. If none of the neighbors pass the predicate, the candidate list empties and the search terminates: even though matching vectors might exist elsewhere in the corpus, the graph has no edges that lead there from the current region. **The result.** Recall@K collapses to whatever fraction of matching vectors happened to be reachable from the entry point through filter passing nodes. You requested K=10, the DB returns 3. The failure is silent: the DB has no way to know it missed seven other matches that exist in the data. **Modern fixes.** Qdrant payload indexing builds secondary indexes on filterable attributes and integrates them into the graph traversal. Weaviate offers dynamic filter strategies that switch between pre-filter, post-filter, and graph-aware paths based on filter selectivity. Pinecone routes high-selectivity filters to a brute force scan instead of the ANN index. The pattern is the same: do not pretend a filter is just a post-processing step on top of a filter blind graph.

**HNSW is a greedy graph walk.** At query time, the search starts at an entry node, looks at its graph neighbors, jumps to whichever is closest to the query vector, and repeats until no neighbor is closer. The graph itself was constructed at build time using vector similarity edges; the index has no knowledge of any future metadata predicate. **The stranding failure.** When a query attaches a tight metadata filter (e.g. `user_id = X AND created_at > Y`), the naive implementation evaluates the predicate as nodes are visited; non-matching nodes are silently skipped. If a region of the graph has no filter passing neighbors, the candidate list empties and the search terminates. Matching vectors elsewhere in the corpus exist but the graph has no edges leading to them from the current region. Recall@K collapses to whatever fraction of matching vectors was reachable from the entry point through filter passing nodes. For a tight filter (say 0.1% selectivity over a 10M corpus, leaving 10,000 matches), the fraction reached might be 5-20%: you asked for K=10 and got 2, even though the data has 10,000 valid matches. **The failure is silent.** The DB has no way to know that matching vectors exist that it failed to reach. It returns the result set it found, reports success, and the application has no signal that recall just collapsed. Standard query-latency monitoring will not catch this; only **result count distribution monitoring** will. **Modern fixes, three families.** *Filterable HNSW.* Qdrant payload indexing, Weaviate dynamic filter strategies, and Pinecone's metadata indexing build secondary indexes on filterable attributes and weave them into the graph traversal. The graph effectively becomes filter aware: the search can route around pruned regions instead of dying in them.