85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What are Matryoshka embeddings?

Matryoshka embeddings are vectors trained so that their leading dimensions carry the most information, letting a system truncate a long embedding to a fraction of its dimensions and still get useful similarity search.

Checked

Index memory, storage and query time all grow with vector length. A 3,072-dimension embedding costs four times what a 768-dimension one does. Matryoshka training lets one model serve several sizes: the first 768 numbers of the long vector are themselves a usable embedding.

The method comes from Matryoshka Representation Learning by Aditya Kusupati and colleagues, first posted to arXiv on 2022-05-26. The authors report up to 14 times smaller embeddings at the same accuracy for ImageNet-1K classification and up to 14 times real-world speed-ups for large-scale retrieval.

How Matryoshka training works

During training, the loss is computed not only on the full vector but also on several nested prefixes of it, for example the first 64, 128, 256 and so on dimensions. The model learns to pack the most useful information first. Truncating then removes detail rather than meaning.

Models that support it in 2026

  • OpenAI text-embedding-3

    Defaults to 1,536 dimensions for text-embedding-3-small and 3,072 for text-embedding-3-large, with a dimensions parameter to request shorter vectors. OpenAI's guide notes that vectors shortened by hand must be renormalised.

  • Google Gemini Embedding 2

    Released in preview on 2026-03-10 with a 3,072-dimension default and Matryoshka training. Google recommends 1,536 and 768 as smaller sizes.

  • Voyage 4 family

    Released 2026-01-15. Every model supports 2,048, 1,024, 512 and 256 dimensions, plus int8 and binary precision.

Why it matters in practice

Shorter vectors fit more of an index in memory and make every comparison cheaper. There is also a hard limit to clear: pgvector indexes the vector type up to 2,000 dimensions and halfvec up to 4,000, so a 3,072-dimension embedding needs halfvec or truncation before it can be indexed.

A common pattern is two-stage search: shortlist with short prefixes, then rescore the shortlist with the full vectors stored alongside.

Where it fails, and what it is confused with

  • Only for models trained this way

    Truncating an embedding from a model without Matryoshka training discards information unevenly and can damage retrieval badly.

  • Quality still drops

    Each cut loses some accuracy, and how much depends on the corpus. The loss has to be measured on real queries, not assumed from a vendor chart.

  • Versus quantization

    Quantization stores each dimension in fewer bits; Matryoshka truncation keeps fewer dimensions. They stack, as Voyage 4's binary and short options show.

  • Versus PCA

    PCA reduces dimensions after the fact with a separate projection. Matryoshka models are trained so a simple cut works without one.

How Motiontic's engineers apply it

Motiontic's engineers run the client's eval set at two or three dimension sizes before fixing a schema, and pick the smallest size that holds recall. Full-length vectors are kept where rescoring is planned, and the choice is recorded with the model version so a later re-embedding does not mix sizes in one index.

Ways to shrink embeddings compared

Ways to shrink embeddings compared
Matryoshka truncationQuantizationPCA
What shrinksNumber of dimensionsBits per dimensionNumber of dimensions
Needs special trainingYesNoNo, but needs a fitted projection
AppliedAt request or by slicingAt index timeAfter embedding
Combines with othersYes, with quantizationYes, with truncationRarely needed with Matryoshka models

Common questions

Can any embedding be truncated to fewer dimensions?

Not safely. Only models trained with Matryoshka-style objectives keep their quality when cut. Others lose information unevenly.

Do OpenAI embeddings support Matryoshka truncation?

Yes. The text-embedding-3 models accept a dimensions parameter, and OpenAI's guide recommends it over manual slicing, which also requires renormalising the vector.

What is the difference between Matryoshka embeddings and quantization?

Matryoshka embeddings drop dimensions; quantization keeps every dimension but stores it with fewer bits. Both cut storage and can be used together.

Which dimension size should a RAG system use?

The smallest one that holds recall on the system's own eval set. Google, for example, recommends 3,072, 1,536 or 768 for Gemini Embedding 2, but the corpus decides.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.