85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is DiskANN?

DiskANN is an approximate nearest neighbour index from Microsoft Research that stores a graph of vectors on SSD instead of in memory, so one machine can search a billion vectors with high recall and low latency.

Checked

HNSW is fast because its graph and vectors sit in memory, and memory is what makes large vector indexes expensive. DiskANN moves the graph to SSD and designs the search to need few disk reads.

It was described by Suhas Jayaram Subramanya, Devvrit, Rohan Kadekodi, Ravishankar Krishnaswamy and Harsha Simhadri at NeurIPS 2019. On the SIFT1B dataset, a 16-core workstation with 64 GB of RAM and an inexpensive SSD served more than 5,000 queries per second at under 3 ms mean latency and over 95% 1-recall@1, by the authors' measurements.

How DiskANN works

The paper introduced Vamana, a graph construction with a smaller diameter than NSG or HNSW graphs, so a search needs fewer sequential hops. Fewer hops matter more on SSD, because each hop is a disk read rather than a memory lookup. Compressed vectors cached in memory guide the walk, and the full-precision vectors stored on SSD with the graph refine the final ranking. Azure Cosmos DB's diskANN index, for example, uses product quantization by default.

The authors report that in the high-recall regime DiskANN can index and serve 5 to 10 times more points per node than in-memory graph methods such as HNSW and NSG.

Where DiskANN ships

  • Microsoft DiskANN

    The MIT-licensed library, now described as bringing fast, fresh and filtered search to databases. Release v0.61.0 shipped on 2026-10-26.

  • Azure Cosmos DB

    A diskANN vector index type for up to 4,096 dimensions that needs at least 1,000 vectors. Microsoft recommends it when a query is scoped to more than 50,000 vectors, with recall tunable per query through searchListSizeMultiplier.

  • Postgres via pgvectorscale

    Tiger Data's StreamingDiskANN index, inspired by DiskANN, adds statistical binary quantization and label-based filtering. Release 0.9.1 shipped 2026-09-04, but Tiger Data said on 2026-10-13 that pgvectorscale will enter maintenance mode as work moves to pg_vectorsearch, which is still alpha.

Where DiskANN fails, and what it is confused with

  • Disk latency

    Every hop that misses the cache is an SSD read. Fast local NVMe matters, and network-attached disks erase much of the advantage.

  • Non-deterministic results

    Like any approximate index, results can vary slightly between runs. Cosmos DB's docs note that each replica builds its index independently, so the same query can return a different ordering.

  • DiskANN versus HNSW

    Both are graph indexes. HNSW layers its graph and keeps it in memory; DiskANN uses a flatter Vamana graph tuned for disk reads.

  • DiskANN versus other disk indexes

    Not every disk-based vector index is DiskANN. Cluster-based designs such as IVF are another route to low memory, and object-storage engines tend to favour them.

How Motiontic's engineers apply it

Motiontic's engineers consider a DiskANN-style index when HNSW memory becomes the cost driver and the corpus is still growing. It is benchmarked on the client's data and hardware against HNSW with quantization, since both attack the same memory bill, and the maintenance status of the chosen extension is checked before it goes into a long-lived system.

DiskANN compared with HNSW and IVF

DiskANN compared with HNSW and IVF
HNSWDiskANNIVF
StructureLayered graphSingle Vamana graphClusters around centroids
Where the index livesMemorySSD, with compressed vectors cachedMemory or disk
Memory per vectorHighLowLow to medium
Scale per nodeBound by RAMBillions on one workstation (paper)Large, with recall trade-offs
Examplespgvector, Elasticsearch, RedisCosmos DB, pgvectorscalepgvector IVFFlat, Faiss

Common questions

What is the difference between DiskANN and HNSW?

Both are graph-based approximate nearest neighbour indexes. HNSW keeps a layered graph in memory, while DiskANN keeps its graph on SSD and is built to reach answers in few disk reads, which cuts memory cost.

Is DiskANN open source?

Yes. Microsoft publishes DiskANN on GitHub under the MIT licence, with release v0.61.0 on 2026-10-26.

Can Postgres use DiskANN?

Through pgvectorscale's StreamingDiskANN index. Tiger Data announced on 2026-10-13 that pgvectorscale will move to maintenance mode as development shifts to pg_vectorsearch, so new projects should weigh that.

Is DiskANN exact?

No, it is approximate. It favours high recall, and implementations expose a search list size that trades latency for recall per query.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.