What are sparse vectors and SPLADE?
Sparse vectors are embeddings with one dimension per vocabulary term, almost all zero, whose non-zero weights mark which terms matter; SPLADE is a neural model that learns those weights and adds related terms.
Checked
Dense embeddings pack meaning into a few hundred or thousand numbers, none of which maps to a word. Sparse vectors keep one dimension per term in a vocabulary of tens of thousands, and only the terms that matter for a text get a weight. They can be searched with an inverted index, like keywords, but the weights can come from a model.
SPLADE, by Thibault Formal, Benjamin Piwowarski and Stéphane Clinchant, was posted to arXiv on 2021-07-12 as a SIGIR 2021 short paper. It uses a language model to weight terms, adds expansion terms that do not appear in the text, and applies sparsity regularisation so the vectors stay small enough to search efficiently.
How learned sparse retrieval works
The model reads a passage and outputs a weight for every vocabulary term. Terms in the passage get weights by importance, and related terms that are absent can get weights too, which is how a passage about "cancel the plan" can match a query about "ending a subscription". Queries are encoded the same way, and the score is the dot product over shared terms.
Elastic's ELSER (Elastic Learned Sparse EncodeR) works on this principle. Elastic's docs stress that the expansion terms are learned associations, not synonyms.
Where sparse vectors ship
Elasticsearch
ELSER stores expanded terms and weights in a sparse_vector field. It is recommended for English and encodes only the first 512 tokens of a field.
Pinecone
Sparse indexes with a hosted pinecone-sparse-english-v0 model. Records with a zero dot-product score are dropped, so a query can return fewer than top_k results.
pgvector
A sparsevec type, indexable up to 1,000 non-zero elements.
Qdrant
Sparse vectors remain supported, but Qdrant 1.20 (2026-10-22) added BM25 directly on full-text indexes, so keyword ranking no longer needs a sparse-vector pipeline there.
Where sparse vectors fail, and what they are confused with
Language coverage
Learned sparse models are often English-first. Elastic points non-English semantic search to a dense model instead of ELSER.
Input limits
Long fields must be chunked to the model's limit, or text past it is never indexed.
Opaque expansions
Expansion terms are learned, so a surprising match can be hard to explain, unlike BM25's literal matches.
Versus BM25
BM25 can be written as a sparse vector too, but its weights are corpus statistics, not model output, and it never adds absent terms.
Versus dense vectors
Dense vectors capture meaning more broadly but cannot be read term by term and need an ANN index rather than an inverted one.
How Motiontic's engineers apply it
Motiontic's engineers test learned sparse retrieval as the keyword side of hybrid search when plain BM25 misses vocabulary mismatches and dense search misses exact terms. Language coverage and input limits are checked against the client's documents first, because a model that only reads the first 512 tokens changes the chunking plan.
BM25, learned sparse vectors and dense vectors compared
| BM25 | Learned sparse (SPLADE, ELSER) | Dense embeddings | |
|---|---|---|---|
| Dimensions | One per term | One per term | Hundreds to thousands, no term mapping |
| Weights from | Corpus statistics | A trained model | A trained model |
| Adds missing terms | No | Yes, by expansion | Not applicable |
| Index | Inverted index | Inverted index | ANN index such as HNSW |
| Explainable | Fully | Partly | Rarely |
Common questions
What is the difference between sparse and dense vectors?
Sparse vectors have one dimension per vocabulary term and are mostly zero, so they can use an inverted index. Dense vectors have far fewer dimensions, all non-zero, and need an approximate nearest neighbour index.
Is SPLADE better than BM25?
It addresses BM25's vocabulary mismatch by learning term expansions, at the cost of running a model at index and query time. Which one wins depends on the corpus and should be measured on real queries.
Does pgvector support sparse vectors?
Yes. pgvector has a sparsevec type, and its indexes support sparse vectors with up to 1,000 non-zero elements.
What is ELSER?
ELSER is Elastic Learned Sparse EncodeR, Elastic's learned sparse retrieval model. It expands text into weighted terms stored in a sparse_vector field and is recommended for English.
Related
BM25
The keyword ranking function behind most full-text search, and half of hybrid search.
Hybrid search
Keyword ranking and vector similarity, fused into one result list.
Embeddings
Meaning turned into vectors, so similar things can be found by distance.
Elasticsearch vs OpenSearch
The original engine and its Apache-licensed fork, compared on licence, governance and vector search.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.