[HN Gopher] Muvera: Making multi-vector retrieval as fast as sin...
       ___________________________________________________________________
        
       Muvera: Making multi-vector retrieval as fast as single-vector
       search
        
       Author : georgehill
       Score  : 83 points
       Date   : 2025-06-26 10:29 UTC (12 hours ago)
        
 (HTM) web link (research.google)
 (TXT) w3m dump (research.google)
        
       | trengrj wrote:
       | We added Muvera to Weaviate recently
       | https://weaviate.io/blog/muvera and also have a nice podcast on
       | it https://www.youtube.com/watch?v=nSW5g1H4zoU.
       | 
       | When looking at multi-vector / ColBERT style approaches, the
       | embedding per token approach can massively increase costs. You
       | might go from a single 768 dimension vector to 128 x 130 = 16,640
       | dimensions. Even with better results from a multi-vector model
       | this can make it unfeasible for many use-cases.
       | 
       | Muvera, converts the multiple vectors into a single fixed
       | dimension (usually net smaller) vector that can be used by any
       | ANN index. As you now have a single vector you can use all your
       | existing ANN algorithms and stack other quantization techniques
       | for memory savings. In my opinion it is a much better approach
       | than PLAID because it doesn't require specific index structures
       | or clustering assumptions and can achieve lower latency.
        
       | dinobones wrote:
       | So this is basically an "embedding of embeddings", an
       | approximation of multiple embeddings compressed into one, to
       | reduce dimensionality/increase performance.
       | 
       | All this tells me is that: the "multiple embeddings" are probably
       | mostly overlapping and the marginal value of each additional one
       | is probably low, if you can represent them with a single
       | embedding.
       | 
       | I don't otherwise see how you can keep comparable performance
       | without breaking information theory.
        
         | kevmo314 wrote:
         | > marginal value of each additional one is probably low
         | 
         | This is the point of the paper. Specifically, that single
         | embedding vectors are sparse enough that you can compact more
         | data from additional vectors together to improve retrieval
         | performance.
        
       | bobosha wrote:
       | how is this different from generating a feature hash of the
       | embeddings i.e reduce from many to one embedding reduction? Could
       | a UMAP or such technique be helpful in reducing to a single
       | vector?
        
         | dinkdonkbell wrote:
         | UMAP doesn't project values into the same coordinate space.
         | While the abstract properties are the same between projections,
         | where it projects it to in coordinate space won't be the same.
        
       | nighthawk454 wrote:
       | Seems to be a trend away from mean-pooling into a single
       | embedding. But instead of dealing with an embedding per token
       | (lots) you still want to reduce it some. This method seems to
       | cluster token embeddings by random partitioning, mean pool for
       | each partition, and concatenate the resulting into a fixed-length
       | final embedding.
       | 
       | Essentially, full multi vector comparison is challenging
       | performance wise. Tools and performance for single vectors are
       | much better. To compromise, cluster into k chunks and
       | concatenate. Then you can do k-vector comparison at once with
       | single-vector tooling and performance.
       | 
       | Ultimately the fixed length vector comes from having a fixed
       | number of partitions, so this is kind of just k-means style
       | clustering of the token level embeddings.
       | 
       | Presumably a dynamic clustering of the tokens could be even
       | better, though that would leave you with a variable number of
       | embeddings per document.
        
       | lawlessone wrote:
       | I'm only vaguely familiar with this. So I apologize how I phrase
       | this.
       | 
       | If make a basic sequel query to return all the first names in
       | table, then i can generally expect it to return them all.
       | 
       | If I do a similar query with these neural embeddings could i
       | expect the same or is it more fuzzy?
        
       ___________________________________________________________________
       (page generated 2025-06-26 23:00 UTC)