[HN Gopher] Sqlite-vec: Work-in-progress vector search SQLite ex...
___________________________________________________________________
Sqlite-vec: Work-in-progress vector search SQLite extension that
runs anywhere
Author : brylie
Score : 218 points
Date : 2024-08-02 10:41 UTC (12 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mic47 wrote:
| Nice. Been waiting for this release to try it out.
| bodantogat wrote:
| This sounds useful (I do a lot of throw-away text analysis on my
| laptop)
| marvel_boy wrote:
| Could anybody explain me a simple example how to do text
| analysis via this vector search. It just searches for the
| closer vector?
| brylie wrote:
| One use case is for retrieving similar documents, such as
| when recommending related content. Another use case is
| retrieving document segments that are similar to a user query
| and passing them along with the user query to a large
| language model for improvement in the generated response.
| Vector search is also better in some ways than keyword search
| since it can find documents that are semantically similar
| even when the user may not have used the exact keyword, or
| even partial keywords like "Postgres" instead of
| "PostgreSQL."
| alexgarcia-xyz wrote:
| You can generate "text embeddings" (ie vectors) from your
| text with an embedding model. Once you have your text
| represented as vectors, then "closest vector" means "more
| semantically similar", as defined by the embedding model you
| use. This can allow you to build semantic search engines,
| recommendations, and classifiers on top of your text and
| embeddings. It's kindof like a fancier and fuzzier keyword
| search.
|
| I wouldn't completely replace keyword search with vector
| search, but it's a great addition that essentially lets you
| perform calculations on text
| yAak wrote:
| Thank you! This was an extremely helpful explanation to me,
| as someone not familiar on the topic. =)
| bodantogat wrote:
| Nice explanation. One use case where keywords haven't
| worked well for me , and (at least at first glance) vectors
| are doing better are longer passages -- finding sentences
| that are similar rather than just words.
| simonw wrote:
| I put together an extensive guide to understanding embeddings
| and vector search last year:
| https://simonwillison.net/2023/Oct/23/embeddings/
| simonw wrote:
| Lots more details in Alex's blog post here:
| https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
| rsingel wrote:
| Would this play nicely with datasette?
| simonw wrote:
| Yes! There's a bug in the datasette-sqlite-vec plugin right
| now but expect a fix shortly.
| simonw wrote:
| In the meantime this workaround works:
| pip install sqlite-vec datasette --load-extension
| "$(python -c
| 'print(__import__("sqlite_vec").loadable_path())')"
| pjot wrote:
| I've done something similar, but using duckDB as the backend.
|
| https://github.com/patricktrainer/duckdb-embedding-search
| youngbum wrote:
| Duckdb is an excellent choice for this task, and it's
| incredibly fast!
|
| We've also added vector search to our product, which is really
| useful.
|
| OpenAI's official examples of embedding search use cosine
| similarity. But here's the cool part: since OpenAI embeddings
| are unit vectors, you can just run the dot product instead!
|
| DuckDB has a super fast dot product function that you can use
| with SQL.
|
| In our product, we use duckdb-wasm to do vector searches on the
| client side.
| bhl wrote:
| What library do you use to compute embedding right now? I'm
| wondering if it's possible to do both embedding and vector
| search client-side for a local-first app.
| rkagerer wrote:
| _> > In our product, we use duckdb-wasm to do vector searches
| on the client side._
|
| Curious, what is your product?
|
| Edit: Nevermind, your recent post explained it quite well:
| https://news.ycombinator.com/item?id=40520073
| pietz wrote:
| Is this also what turso uses in their "AI feature"?
| alexgarcia-xyz wrote:
| No, libsql added custom vector search directly into their
| library, while sqlite-vec is a separate SQLite extension.
|
| The libsql vector feature only works in libsql, sqlite-vec
| works in all SQLite versions. The libsql vector feature works
| kindof like pgvector, while sqlite-vec works more like the FTS5
| full text SQLite extension.
|
| I'd say try both and see which one you like more. sqlite-vec
| will soon be a part of Turso's and SQLite Cloud's products.
|
| Turso's version: https://turso.tech/vector
| alexgarcia-xyz wrote:
| Author here, happy to answer any questions! Been working on this
| for a while, so I'm very happy to get this v0.1.0 "stable"
| release out.
|
| sqlite-vec works on MacOS, Linux, Windows, Raspberry Pis, in the
| browser with WASM, and (theoretically) on mobile devices. I
| focused a lot on making it as portable as possible. It's also
| pretty fast - benchmarks are hard to do accurately, but I'd
| comfortable saying that it's a very very fast brute-force vector
| search solution.
|
| One experimental feature I'm working on: You can directly query
| vectors that are in-memory as a contiguous block of memory (ie
| NumPy), without any copying or cloning. You can see the
| benchmarks for that feature here under "sqlite-vec static", and
| it's competitive with faiss/usearch/duckdb
| https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
| Cieric wrote:
| I feel like I've touched a lot of things where something like
| this is useful (hobby projects). In my case I've done a
| recommendation engine, music matching (I specifically use it for
| matching anime to their data), and perceptual hash matching.
| alexgarcia-xyz wrote:
| Really curious to hear about what kind of music embedding
| models/tools you used! I've tried finding some good models
| before but they were all pretty difficult to use
| Cieric wrote:
| In the past I mainly screwed around with [2] musig as it was
| fairly easy to modify, right off hand I don't know why I
| stopped using it. [1] seek-tune (not-shazam) was just post
| here on hn yesterday, I had to modify it quite a bit to
| actually get something working in a way I was happy with.
| Overall it was a little slow, but that's not something I'm
| gonna fault the author for as the project is still quite new.
| While browsing around yesterday after messing with [2], I
| found [3] cpuimage/shazam. It's a little easier for me to
| screw around with as it's in c, and would probably be the one
| I try and modify to use this project.
|
| Everything is based on the shazam algorithm or something
| similar, so chunking the audio stream, and fingerprinting it.
| I was focused on that because it allows me to figure out
| where OP and ED are in some anime and allows me to add a skip
| button. Also since anime in the same season typically have
| the same OP/ED I can use that to try and auto categorize them
| after the first ones been identified.
|
| TL;DR: It's all shazam look alikes, everything else was to
| annoying to use.
|
| [1] https://github.com/cgzirim/seek-tune [2]
| https://github.com/sfluor/musig [3]
| https://github.com/cpuimage/shazam
| yard2010 wrote:
| Can you elaborate about your projects please? What tools are
| you using?
| haolez wrote:
| What's the maximum vector size?
| alexgarcia-xyz wrote:
| vec0 virtual tables have a hard-coded max of 8192 dimensions,
| but I can raise that very easily (I wanted to reduce resource
| exhaustion attacks). But if you're comparing vectors manually,
| then the `vec_distance_ls()` and related functions have no
| limits (besides SQLite's 1GB blob limit)
| dang wrote:
| Related:
|
| _I'm writing a new vector search SQLite Extension_ -
| https://news.ycombinator.com/item?id=40243168 - May 2024 (85
| comments)
___________________________________________________________________
(page generated 2024-08-02 23:00 UTC)