85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is an object-storage vector database?

An object-storage vector database keeps its vectors and indexes in cloud object storage such as Amazon S3, with memory and SSD caches in front, trading slower cold queries for much lower storage cost and simpler scaling.

Checked

The first generation of vector databases kept indexes in memory on provisioned nodes. That is fast and expensive, and it scales badly when a product has thousands of tenants whose data is mostly idle. Object storage flips the cost model: durable storage is cheap, and only the data being queried needs to sit in a cache.

Between late 2025 and mid 2026 this became the default design for new vector engines, and AWS built vector indexes into S3 itself.

How the architecture works

turbopuffer's architecture docs describe the pattern clearly. Each namespace writes a log of changes to object storage, and a write is acknowledged only once it is durable there, which turbopuffer gives as a p50 of 165 ms for a 500 kB write. Queries are served from memory and NVMe SSD caches. A warm query on 1M documents has a p50 of 14 ms; a cold query that reads object storage directly takes several hundred milliseconds.

Because each object storage round trip is slow, these engines favour cluster-based indexes such as IVF over graph indexes such as HNSW, which need many small reads.

Who ships it

  • turbopuffer

    Managed vector and full-text search on object storage, with strong consistency by default. Covered in depth on its own page.

  • Amazon S3 Vectors

    Generally available since 2025-12-02: up to 2 billion vectors per index and 10,000 indexes per bucket, with frequent queries around 100 ms and infrequent ones under a second. Limits include a top-k of 100, 50 metadata keys per vector and no BM25.

  • Supabase Vector Buckets

    Announced 2025-12-01 as a public alpha, built on S3-backed storage with up to 50M vectors per index. Supabase positions pgvector for hot vectors and Vector Buckets for warm and cold ones.

  • Milvus 3.0

    Announced 2026-07-16 as lake-native, with External Collections that query vectors stored in Lance, Iceberg, Parquet and Vortex files.

Where it fails, and what it is confused with

  • Cold queries

    The first query against an uncached namespace reads object storage and is an order of magnitude slower than a warm one. Products with spiky per-tenant traffic feel this directly.

  • Write latency

    Durable writes to object storage take longer than writes to a local disk, so read-after-write flows need checking.

  • Thin query features

    Some services trade features for price: S3 Vectors has no keyword search and a low top-k cap, so hybrid search and reranking happen elsewhere.

  • Not the same as serverless

    Serverless describes billing and operations. Object storage describes where the data lives. A serverless product can still be memory-resident underneath.

How Motiontic's engineers apply it

Motiontic's engineers look at object-storage stores when the corpus is large, split across many tenants, or queried unevenly, which describes most agent memory and per-customer RAG. Cold-start latency is measured on the client's real access pattern before the choice is made, and hot tenants can be kept warm or served from a memory-resident tier.

Memory-resident and object-storage vector stores compared

Memory-resident and object-storage vector stores compared
Memory-residentObject-storage first
Where data livesRAM on provisioned nodesS3-style storage, cached on SSD and RAM
Storage costHighLow
Warm query latencyLowLow once cached
Cold query latencyNot applicableHundreds of milliseconds or more
Favoured indexHNSWIVF and other cluster indexes
SuitsSteady, latency-critical trafficMany tenants, large or idle corpora
ExamplesRedis, in-memory HNSW deploymentsturbopuffer, S3 Vectors, Milvus 3.0 external collections

Common questions

Is Amazon S3 Vectors a vector database?

AWS calls it vector buckets and indexes built into S3. It stores and queries vectors cheaply but lacks database features such as keyword search, and caps top-k at 100.

Why are cold queries slower in an object-storage vector database?

A cold query has to read index files from object storage, where each round trip takes around 100 ms. Once the namespace is cached on SSD or in memory, queries drop to milliseconds.

When is an object-storage vector database a poor fit?

When every query must be fast from the first request and traffic is spread thinly across many rarely used namespaces, or when the workload needs features such as BM25 that the service lacks.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.