[HN Gopher] On-disk HNSW index for Postgres with pg_embedding
___________________________________________________________________
On-disk HNSW index for Postgres with pg_embedding
Author : nikita
Score : 41 points
Date : 2023-08-03 18:08 UTC (4 hours ago)
(HTM) web link (neon.tech)
(TXT) w3m dump (neon.tech)
| random_moonwalk wrote:
| This looks very cool.
|
| I'm interested in how many vectors are indexed/how large is the
| index corresponding to the latency chart? If we have an in-memory
| HNSW index of 10M vectors at ~20GB (512 dim), say, what are the
| RAM requirements when using the disk-based version?
| nikita wrote:
| CEO of Neon here. After we built an in memory HNSW index for
| Postgres that allowed us to establish a baseline in performance
| and prove that it's the right approach to support vector search
| we now built it "the right way" and now it support restarts of
| Postgres, replication and the rest of the Postgres machinery.
| myth17 wrote:
| Can you explain: What does it mean to be constructed on disk?
| nikita wrote:
| This means that the index is persisted in Postgres pages
| which are durable against restarting Postgres. Our previous
| version what fully in-memory and wasn't "durable".
| nh2 wrote:
| Can you clarify:
|
| Does the the initial _construction_ (creation) of the index
| need to fit into RAM?
|
| Also, should one expect the reading via Postgres's buffer cache
| to have better access patterns / need less memory / have other
| benefit vs having a non-Postgres HNSW index that's mmapped from
| disk?
| nh2 wrote:
| I would appreciate a rough comparison with usearch:
|
| https://unum-cloud.github.io/usearch/
|
| Which was also recently on HN:
| https://news.ycombinator.com/item?id=36942993
| nikita wrote:
| I think it's good technology. Our team flagged it, we haven't
| tested it yet.
| thewataccount wrote:
| Forgive me I'm not super familiar with the vector indexes outside
| of the basic tsvector for text search.
|
| What's the difference between pg_embedding, pg_vector, and
| tsvector? Are they compariable/interchangable? And how do you
| know which one to pick?
|
| My understanding is pg_vector has poorer performance compared to
| some dedicated vector databases, does pg_embedding perform
| better?
|
| Sorry if these are silly questions.
| HammadB wrote:
| pg_embedding is based on a different approximate nearest
| neighbors algorithm - HNSW, which is generally considered - and
| studied - to be faster as well as more accurate than pgvectors
| IVF (ignoring a lot of nuance).
|
| However pg_embedding serves the index out of disk, whereas most
| vector databases opt to serve the index out of memory, or
| delegate to mmap'ed files. HNSW is a graph-based algorithm,
| thus access patterns are random and this does not lend itself
| to disk based access. I'd expect pg_embedding to be slower than
| memory-resident indices due to this fact. Also in general with
| a postgres index, my concern would be scalability and resource
| isolation. It's convenient to colocate these things but ANN
| indices have very different CPU/Memory/Disk usage patterns than
| what you may need for just your relational data.
|
| For example
|
| Chroma -> Serves HNSW out of memory, persists to disk with a
| WAL.
|
| Weviate -> Serves HNSW out of memory, writes HNSW graph search
| to WAL and uses that for durability.
|
| Milvus -> Serves HNSW out of memory, supports partial mmap,
| also supports another algorithm called DiskANN which is
| optimized for SSD. It uses cloud storage with a shared-
| everything architecture for durability (a lot of nuance here.)
|
| QDrant -> Has a WAL, supports memory and mmap'ed indices.
| nikita wrote:
| Well it doesn't serve it from disk. It's persisted to disk
| and Postgres buffer cache keeps the working set in memory.
| HammadB wrote:
| Maybe I am misunderstanding but the postgres buffer cache
| is LRU and will evict pages right? At times data may go to
| disk and then during serving it will have to be loaded into
| memory? So it is quite dependent on the size of your buffer
| cache as well as contention for that buffer cache.
|
| Also the cache access patterns will vary between these
| implementations and is worth considering.
| nikita wrote:
| For sure.
| roseway4 wrote:
| As I mentioned elsewhere, there's more to vector database
| selection than raw performance: Developers leveraging their
| existing experience with Postgres, existing infrastructure
| investment (often in managed Postgres on AWS/GCP etc), and a
| single API into the vector store shared with other parts of
| their app (their ORM / DB layer).
|
| Many teams can also get away with _good_ performance vs the
| _fastest_ performance, given smaller index sizes and the
| other tradeoffs I mentioned.
|
| That said, I can imagine the pgvector folks precaching the
| new HNSW index support they're working on, as they do with
| their IVFFLAT index.
|
| * edited for the grammar gremlins
| HammadB wrote:
| Sure, I don't disagree that there is more to vector
| database selection than raw performance. Any technical
| decision is filled with many considerations.
|
| The commenter - thewataccount - asked about performance and
| I shared my intuition.
| nikita wrote:
| pgvector is working HNSW too. We have a blog post about it:
| https://neon.tech/blog/pgvector-meets-hnsw-index
| lukev wrote:
| Different vector indexing algorithms under the hood, with
| different tradeoffs.
|
| In this case "hnsw" is the name of the algorithm. If you want
| to know more, just search "hnsw" and "ivfflat" to understand
| the differences.
|
| tsvector is something else; it's not a numeric vector index,
| it's a different kind of data structure that stores the actual
| lexemes (keywords) with positions and weights to facilitate
| full text search (rather than using a text embedding model.)
| fzliu wrote:
| Thanks for sharing. How does this compare with DiskANN
| (https://zilliz.com/blog/diskann-a-disk-based-anns-solution-w...)
| or HNSW-IF (https://blog.vespa.ai/vespa-hybrid-billion-scale-
| vector-sear...)?
| roseway4 wrote:
| This is comparing apples and oranges. You've pointed to some
| technologies from vector database vendors. Neon isn't a vector
| database vendor. They're a Postgres vendor. The objective of
| pg_embedding, pg_vector, and others is to offer teams who
| deploy Postgres the opportunity to use their existing
| infrastructure for vector search. The added and important
| benefit here is hybrid search: using existing DB data to filter
| semantic search results.
|
| What the work done by Neon, the pgvector team, Supabase and
| others points to is that "speed" isn't the only factor in
| vector database selection. Developer experience and existing
| infrastructure investment are too.
| HammadB wrote:
| No one is discrediting (or even discussing really) the
| validity of the point that if you use postgres pgvector or
| pgembedding may makes sense for the reasons you mention. The
| question was about how the algorithms compare, I compared the
| algorithms. Comparing ANN algorithms is relatively apples-to-
| apples, but is still complicated and filled with nuance.
| HammadB wrote:
| DiskANN construction is much much slower than HNSW, and the
| support around updates has been discussed in literature but has
| not been open sourced yet (They call this FreshDiskANN in the
| paper - DiskANN is from microsoft research btw).
|
| HNSW-IF is an excellent extension to HNSW (that the vespa team
| has made easy to implement) that takes advantage of the
| speed/recall of HNSW in combination with the disk scalability
| of inverted indices - it is a hybrid approach.
| nikita wrote:
| DiskANN has some great ideas that will likely make it into
| pg_embedding. The most important implication is how many
| vectors it can support. DiskANN claims it can manage 1Bln
| vectors and it's a huge deal. Index construction does take a
| long time, but we are researching how we can scale it out.
___________________________________________________________________
(page generated 2023-08-03 23:01 UTC)