85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What are sparse vectors and SPLADE?

Sparse vectors are embeddings with one dimension per vocabulary term, almost all zero, whose non-zero weights mark which terms matter; SPLADE is a neural model that learns those weights and adds related terms.

Checked

Dense embeddings pack meaning into a few hundred or thousand numbers, none of which maps to a word. Sparse vectors keep one dimension per term in a vocabulary of tens of thousands, and only the terms that matter for a text get a weight. They can be searched with an inverted index, like keywords, but the weights can come from a model.

SPLADE, by Thibault Formal, Benjamin Piwowarski and Stéphane Clinchant, was posted to arXiv on 2021-07-12 as a SIGIR 2021 short paper. It uses a language model to weight terms, adds expansion terms that do not appear in the text, and applies sparsity regularisation so the vectors stay small enough to search efficiently.

How learned sparse retrieval works

The model reads a passage and outputs a weight for every vocabulary term. Terms in the passage get weights by importance, and related terms that are absent can get weights too, which is how a passage about "cancel the plan" can match a query about "ending a subscription". Queries are encoded the same way, and the score is the dot product over shared terms.

Elastic's ELSER (Elastic Learned Sparse EncodeR) works on this principle. Elastic's docs stress that the expansion terms are learned associations, not synonyms.

Where sparse vectors ship

  • Elasticsearch

    ELSER stores expanded terms and weights in a sparse_vector field. It is recommended for English and encodes only the first 512 tokens of a field.

  • Pinecone

    Sparse indexes with a hosted pinecone-sparse-english-v0 model. Records with a zero dot-product score are dropped, so a query can return fewer than top_k results.

  • pgvector

    A sparsevec type, indexable up to 1,000 non-zero elements.

  • Qdrant

    Sparse vectors remain supported, but Qdrant 1.20 (2026-10-22) added BM25 directly on full-text indexes, so keyword ranking no longer needs a sparse-vector pipeline there.

Where sparse vectors fail, and what they are confused with

  • Language coverage

    Learned sparse models are often English-first. Elastic points non-English semantic search to a dense model instead of ELSER.

  • Input limits

    Long fields must be chunked to the model's limit, or text past it is never indexed.

  • Opaque expansions

    Expansion terms are learned, so a surprising match can be hard to explain, unlike BM25's literal matches.

  • Versus BM25

    BM25 can be written as a sparse vector too, but its weights are corpus statistics, not model output, and it never adds absent terms.

  • Versus dense vectors

    Dense vectors capture meaning more broadly but cannot be read term by term and need an ANN index rather than an inverted one.

How Motiontic's engineers apply it

Motiontic's engineers test learned sparse retrieval as the keyword side of hybrid search when plain BM25 misses vocabulary mismatches and dense search misses exact terms. Language coverage and input limits are checked against the client's documents first, because a model that only reads the first 512 tokens changes the chunking plan.

BM25, learned sparse vectors and dense vectors compared

BM25, learned sparse vectors and dense vectors compared
BM25Learned sparse (SPLADE, ELSER)Dense embeddings
DimensionsOne per termOne per termHundreds to thousands, no term mapping
Weights fromCorpus statisticsA trained modelA trained model
Adds missing termsNoYes, by expansionNot applicable
IndexInverted indexInverted indexANN index such as HNSW
ExplainableFullyPartlyRarely

Common questions

What is the difference between sparse and dense vectors?

Sparse vectors have one dimension per vocabulary term and are mostly zero, so they can use an inverted index. Dense vectors have far fewer dimensions, all non-zero, and need an approximate nearest neighbour index.

Is SPLADE better than BM25?

It addresses BM25's vocabulary mismatch by learning term expansions, at the cost of running a model at index and query time. Which one wins depends on the corpus and should be measured on real queries.

Does pgvector support sparse vectors?

Yes. pgvector has a sparsevec type, and its indexes support sparse vectors with up to 1,000 non-zero elements.

What is ELSER?

ELSER is Elastic Learned Sparse EncodeR, Elastic's learned sparse retrieval model. It expands text into weighted terms stored in a sparse_vector field and is recommended for English.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.