What is filtered vector search?
Filtered vector search is a nearest neighbour query restricted by metadata conditions, such as tenant, date or access rights, so results are both semantically close to the query and allowed by the filter.
Checked
Almost no production vector query is unfiltered. A support agent searches one customer's tickets, a legal assistant searches documents the user may see, a sales agent searches this quarter's notes. The filter is not optional, and getting it wrong is either a quality bug or a data leak.
The difficulty is that approximate indexes such as HNSW are built for similarity alone. Combining them with an arbitrary filter without losing recall or speed has been an active research problem, and in 2026 it is one of the clearer differences between vector stores.
Three ways to apply a filter
Post-filtering
Run the vector search, then drop results that fail the filter. Simple, but a selective filter can remove almost everything, leaving fewer results than asked for.
Pre-filtering
Find the rows that pass the filter, then search only those. Exact and safe, but it falls back to brute-force comparison when the index cannot be restricted.
Filter-aware indexes
Apply the filter during the graph traversal itself, so the search keeps walking until it finds enough matching neighbours. This is where most 2024 to 2026 work has gone.
How the main stores handle it
pgvector
Version 0.8.0 (announced 2024-11-11) added iterative index scans to prevent over-filtering: if the first scan does not satisfy the query, the index keeps searching up to a limit such as hnsw.max_scan_tuples, 20,000 by default. The README also suggests partial indexes for a few distinct filter values and partitioning for many.
ACORN
Liana Patel, Peter Kraft, Carlos Guestrin and Matei Zaharia described ACORN in March 2024. It extends HNSW with predicate subgraph traversal so any filter can be applied during search, and the authors report 2 to 1,000 times higher throughput than prior methods at fixed recall.
Weaviate
Weaviate 1.40 (2026-10-14) offers three HNSW filter strategies: acorn, sweeping and a new PathSeer. In Weaviate's own tests PathSeer was fastest or close on most filters, while ACORN stayed ahead on strongly negatively correlated filters.
Qdrant
Payload indexes feed filter-aware edges into HNSW and help the planner estimate filter cardinality. Qdrant's docs say payload indexes should be created before ingesting data, or the HNSW graph must be rebuilt to benefit.
Where filtered search fails
Correlated filters
When the filter selects items far from the query in vector space, a graph search wanders through many non-matching neighbours before finding valid ones. Recall drops quietly.
Hard limits
Some stores cap filters. Amazon S3 Vectors allows 50 metadata keys per vector and a top-k of at most 100.
Permissions as a prompt
Access control belongs in the filter, enforced by the store, not in an instruction asking the model to ignore documents the user should not see.
Confused with hybrid search
Some papers call vector-plus-structured queries "hybrid search". In RAG, hybrid search usually means keyword plus vector ranking, which is a different problem.
How Motiontic's engineers apply it
Motiontic's engineers build eval queries with the client's real filters attached, including the most selective tenants and date ranges, and measure recall@k with and without the filter. Permission filters are enforced in the store and covered by tests, so a retrieval change cannot widen what an agent is allowed to read.
Post-filtering, pre-filtering and filter-aware indexes
| Post-filter | Pre-filter | Filter-aware index | |
|---|---|---|---|
| When the filter runs | After the vector search | Before the vector search | During graph traversal |
| Risk | Too few results | Slow on broad filters | Weaker on correlated filters |
| Recall under selective filters | Drops | Exact | Holds, if implemented well |
| Examples | Plain WHERE after an ANN scan | Partial indexes, partitions | pgvector iterative scans, ACORN, PathSeer |
Common questions
Why does a filtered vector query return fewer results than requested?
The index returned its nearest candidates first and the filter removed most of them. pgvector calls this over-filtering and added iterative index scans in version 0.8.0 to keep searching until enough rows pass.
What is the difference between pre-filtering and post-filtering?
Pre-filtering narrows the candidate set before the similarity search. Post-filtering searches first and discards non-matching results afterwards, which is faster to build but can lose results.
What is ACORN in vector search?
ACORN is a 2024 method by Patel, Kraft, Guestrin and Zaharia that makes HNSW search filter-aware, so arbitrary predicates are applied during graph traversal. Weaviate offers it as one of its filter strategies.
Can metadata filters enforce document permissions in RAG?
Yes, and they should. Filtering by tenant or access group in the store keeps forbidden documents out of the model's context entirely, which prompt instructions cannot guarantee.
Related
HNSW
The layered graph index behind most vector search.
Vector database
Storage and nearest-neighbour search for embeddings, now in many shapes.
pgvector vs Qdrant
Vectors inside Postgres compared with an open-source Rust vector engine.
Recall@k
Did the right passages make the shortlist? The first number to check in RAG.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Sources
- PostgreSQL news, pgvector 0.8.0 released (2024-11-11)
- pgvector README on GitHub (version 0.8.7)
- Patel, Kraft, Guestrin and Zaharia, ACORN (arXiv, 2024-03-07)
- Weaviate, 1.40 release (2026-10-14)
- Qdrant docs, Indexing (payload indexes and filterable HNSW)
- AWS, Amazon S3 Vectors generally available (2025-12-02)
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.