85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is vector quantization?

Vector quantization is the compression of embedding vectors into fewer bits per dimension, such as 8-bit, 4-bit or 1-bit codes, to cut the memory and storage a vector index needs, usually with a rescoring step to recover accuracy.

Checked

Embeddings are usually stored as 32-bit floats, and a vector index wants them in memory to search fast. At tens of millions of vectors that memory bill dominates the cost of running retrieval.

Quantization stores a compact code instead of the full vector. Search runs on the codes, and the top candidates are often rescored against the full-precision vectors kept on disk, which recovers most of the accuracy the compression cost.

The main kinds

  • Scalar quantization

    Each 32-bit float becomes a smaller integer, commonly 8 bits. Simple, with modest accuracy loss.

  • Binary quantization

    Each dimension becomes a single bit. The largest saving, so it is usually paired with oversampling and rescoring.

  • Product quantization

    The vector is split into sub-vectors, and each is replaced by the ID of its nearest entry in a learned codebook.

  • Rotational quantization

    The vector is rotated before quantizing so information spreads evenly across dimensions, which lets very low bit widths keep more accuracy. RaBitQ is a known example.

How engines ship it in 2026

  • Weaviate

    1.40 (2026-10-14) made 4-bit rotational quantization generally available.

  • Qdrant

    1.20 (2026-10-22) added TurboQuant 16-bit and 8-bit types. Qdrant also reports 21x compression on late-interaction page embeddings while keeping 96.5% of full-precision quality, in its own test.

  • Elasticsearch

    Better Binary Quantization (BBQ), plus the disk-based DiskBBQ format. Elastic's docs note the recall trade-offs per setting.

  • Postgres

    pgvectorscale adds statistical binary quantization. Pinecone's pg_pinecone runs an IVF index with RaBitQ, and Pinecone's own benchmark on 1M 768-dimension vectors reports a 136 MB index against 2.0 GB for a half-precision HNSW index.

How Motiontic's engineers apply it

Motiontic's engineers measure recall at the cut the application actually uses, such as top 10, on the client's own queries before and after quantization. Rescoring depth and oversampling are tuned against that number and the latency budget, because a vendor's headline compression ratio says little about a specific corpus.

Bits per dimension and compression compared with 32-bit floats

Bits per dimension and compression compared with 32-bit floats
FormatBits per dimensionCompression vs float32
Float32 (baseline)321x
Half precision162x
Scalar int884x
4-bit48x
Binary132x

Common questions

Does vector quantization reduce accuracy?

Some, and the loss grows as bits shrink. Most engines recover much of it by searching on compressed codes and then rescoring the top candidates with the full-precision vectors.

What is binary quantization?

Binary quantization stores one bit per dimension, a 32x reduction from 32-bit floats. It is fast and small, and is usually combined with oversampling and rescoring to keep recall acceptable.

Is quantization the same as using a smaller embedding model?

No. A smaller model produces different, usually shorter vectors. Quantization keeps the same model and stores each vector more compactly.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.