Token Pooling-Aware Training for Multi-Vector Embedding Models
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Pooling-aware training raises compressed multi-vector retrieval quality from 94.6% to 97.8% of unpooled NanoBEIR nDCG@10 at the same 4x document-vector reduction. For production ColBERT-style systems, this means token pooling can cut index size and serving cost materially without the usual relevance hit, but the encoder should be trained with pooling in the loop rather than compressed only after training.
Multi-vector embedding models can now be trained to retain 97.8% of their original retrieval quality at a 4x reduction in storage, thanks to a new "token pooling-aware training" method that teaches the encoder to preserve information when vectors are merged, enabling more efficient search without sacrificing performance. This directly impacts the storage and compute costs for running such models in production. Shipping with this capability can reduce storage costs by 4x for the same retrieval quality.