[HN Gopher] An early look at HNSW performance with pgvector
       ___________________________________________________________________
        
       An early look at HNSW performance with pgvector
        
       Author : kiwicopple
       Score  : 17 points
       Date   : 2023-08-10 19:43 UTC (3 hours ago)
        
 (HTM) web link (jkatz05.com)
 (TXT) w3m dump (jkatz05.com)
        
       | kiwicopple wrote:
       | The results are very promising for the Postgres ecosystem, as the
       | HNSW index shows a significant performance/recall improvement
       | over the current ivfflat index in pgvector.
       | 
       | HNSW will be merged in v0.5.0. I can't speak for Andrew (the
       | creator) but it seems that this release is imminent[0], pending
       | some benchmarking and minor improvements. This is a first look at
       | the performance of pgvector's HNSW implementation at a specific
       | commit[1].
       | 
       | [0]
       | https://github.com/pgvector/pgvector/commit/51d292c93dff82f6...
       | 
       | [1] https://github.com/pgvector/pgvector/commit/600ca5a7
        
       | ilaksh wrote:
       | This may be a dumb question but with OpenAI embeddings do we need
       | to use cosine similarity or is the simple distance equivalent? I
       | used cosine similarity before but not sure.
        
         | jkatz05 wrote:
         | Blog author. You can choose to use any distance metrics. One
         | reason cosine similarity is popular (and used) is that for many
         | of these higher dimensional datasets, it gives a better
         | representation of "nearness" across all the data basd on the
         | nature of "angular" distance. But depending on how your data is
         | distributed, something like L2 distance (Euclidean) could make
         | more sense.
        
       | pashkinelfe wrote:
       | I like these measurements with ANN-benckmark! They allow to
       | compare performance of different index implementations apples-to-
       | apples i.e. at the same build parameters set rather than using
       | some fixed settings (or, even worse, default settings that are
       | different).
       | 
       | The blogpost is very thorough, lots of measurements of different
       | datasets, including great 1536-dimensional 1M rows dbpedia-openai
       | dataset. Furthermore, a very strong point is that all parameters
       | and method is described and transparent.
        
         | jkatz05 wrote:
         | Blog author. Thanks for the analysis -- I agree that the ANN
         | Benchmark does provide a nice framework for helping with
         | apples-to-apples comparisons. In this case, being able to use
         | the "--local" flag made it easier to run using the native
         | environment, vs. putting it into a container. I'm looking for
         | to ANN Benchmark having more datasets!
        
       ___________________________________________________________________
       (page generated 2023-08-10 23:01 UTC)