What is late interaction (ColBERT)?
Late interaction is a retrieval method, introduced by the ColBERT model in 2020, that stores one vector per token of each document and scores a query by matching each query token against its closest document token.
Checked
A standard embedding squeezes a whole passage into one vector. That is compact, but fine detail gets averaged away. Late interaction keeps a vector for every token, so a query about one specific clause can match the tokens that carry it.
The idea comes from ColBERT, published by Omar Khattab and Matei Zaharia in April 2020. Queries and documents are encoded separately, so document vectors can still be computed ahead of time, and the interaction between them happens late, at scoring time.
How scoring works
For each query token, the system finds the most similar token vector in the document and keeps that maximum similarity. The document's score is the sum of those maxima across all query tokens, an operation usually called MaxSim.
The same approach extends to images. ColPali-style models treat patches of a page image as tokens, which lets retrieval run over scanned pages and slides without a text extraction step.
The storage trade-off
One vector per token means many vectors per passage, so storage and index size grow sharply. The industry answer has been compression and encoding. Qdrant reports 21x storage compression for Perplexity's late-interaction page embeddings while keeping 96.5% of full-precision quality without rescoring, a figure from Qdrant's own test.
Native support in 2026
Milvus 3.0
Announced on 2026-07-16 with a StructList type for ColBERT and ColPali multi-vectors.
Weaviate 1.40
Released on 2026-10-14 with multi-vector support on its disk-based HFresh index, encoded with MUVERA.
turbopuffer
Its pricing page bills a late-interaction ANN query as two queries, a sign of the extra work involved.
How Motiontic's engineers apply it
Motiontic's engineers usually test late interaction first as a reranking stage over a shortlist from hybrid search, where storage cost stays small. It moves into first-stage retrieval only when evals on the client's documents, often dense technical or visual pages, show a gain that justifies the index size.
Single-vector embeddings compared with late interaction
| Single-vector | Late interaction | |
|---|---|---|
| Vectors per passage | One | One per token or image patch |
| Scoring | One similarity per passage | Sum of per-token maximum similarities |
| Fine detail | Averaged away | Preserved |
| Storage | Small | Much larger, reduced with quantization |
| Common role | First-stage retrieval | Reranking, or first stage for hard corpora |
Common questions
What does late interaction mean in ColBERT?
It means the query and document are encoded separately and only compared at scoring time, token by token. That keeps document encoding offline while allowing finer matching than a single vector per passage.
Is late interaction better than a cross-encoder reranker?
They sit at different points. A cross-encoder reads query and passage together and is more precise but cannot precompute anything. Late interaction precomputes document vectors, so it can search larger sets, at a higher storage cost.
Which vector databases support multi-vector retrieval?
In 2026 Milvus 3.0, Weaviate 1.40 and Qdrant document native multi-vector support, and turbopuffer supports late-interaction queries at the price of two queries each.
Related
Reranker
A second pass that rereads the shortlist and puts the right passages first.
Embeddings
Meaning turned into vectors, so similar things can be found by distance.
Vector quantization
Fewer bits per dimension: smaller, cheaper indexes at a small cost in recall.
Vector database
Storage and nearest-neighbour search for embeddings, now in many shapes.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.