[HN Gopher] Sqlite-vec: Work-in-progress vector search SQLite ex...
___________________________________________________________________
Sqlite-vec: Work-in-progress vector search SQLite extension that
runs anywhere
Author : brylie
Score : 293 points
Date : 2024-08-02 10:41 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mic47 wrote:
| Nice. Been waiting for this release to try it out.
| bodantogat wrote:
| This sounds useful (I do a lot of throw-away text analysis on my
| laptop)
| marvel_boy wrote:
| Could anybody explain me a simple example how to do text
| analysis via this vector search. It just searches for the
| closer vector?
| brylie wrote:
| One use case is for retrieving similar documents, such as
| when recommending related content. Another use case is
| retrieving document segments that are similar to a user query
| and passing them along with the user query to a large
| language model for improvement in the generated response.
| Vector search is also better in some ways than keyword search
| since it can find documents that are semantically similar
| even when the user may not have used the exact keyword, or
| even partial keywords like "Postgres" instead of
| "PostgreSQL."
| alexgarcia-xyz wrote:
| You can generate "text embeddings" (ie vectors) from your
| text with an embedding model. Once you have your text
| represented as vectors, then "closest vector" means "more
| semantically similar", as defined by the embedding model you
| use. This can allow you to build semantic search engines,
| recommendations, and classifiers on top of your text and
| embeddings. It's kindof like a fancier and fuzzier keyword
| search.
|
| I wouldn't completely replace keyword search with vector
| search, but it's a great addition that essentially lets you
| perform calculations on text
| yAak wrote:
| Thank you! This was an extremely helpful explanation to me,
| as someone not familiar on the topic. =)
| bodantogat wrote:
| Nice explanation. One use case where keywords haven't
| worked well for me , and (at least at first glance) vectors
| are doing better are longer passages -- finding sentences
| that are similar rather than just words.
| simonw wrote:
| I put together an extensive guide to understanding embeddings
| and vector search last year:
| https://simonwillison.net/2023/Oct/23/embeddings/
| bambax wrote:
| Yes this is an excellent article, thank you.
|
| Do embeddings work across languages? It must depend on the
| model, but are there models where similar concepts in
| different languages occupy the same location in vector
| space?
|
| If yes it would allow to search a corpus in one language
| using another, without translating anything up front
| (neither the corpus nor the query).
| simonw wrote:
| Lots more details in Alex's blog post here:
| https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
| rsingel wrote:
| Would this play nicely with datasette?
| simonw wrote:
| Yes! There's a bug in the datasette-sqlite-vec plugin right
| now but expect a fix shortly.
| simonw wrote:
| In the meantime this workaround works:
| pip install sqlite-vec datasette --load-extension
| "$(python -c
| 'print(__import__("sqlite_vec").loadable_path())')"
| pjot wrote:
| I've done something similar, but using duckDB as the backend.
|
| https://github.com/patricktrainer/duckdb-embedding-search
| youngbum wrote:
| Duckdb is an excellent choice for this task, and it's
| incredibly fast!
|
| We've also added vector search to our product, which is really
| useful.
|
| OpenAI's official examples of embedding search use cosine
| similarity. But here's the cool part: since OpenAI embeddings
| are unit vectors, you can just run the dot product instead!
|
| DuckDB has a super fast dot product function that you can use
| with SQL.
|
| In our product, we use duckdb-wasm to do vector searches on the
| client side.
| bhl wrote:
| What library do you use to compute embedding right now? I'm
| wondering if it's possible to do both embedding and vector
| search client-side for a local-first app.
| cxcorp wrote:
| I think Transformers.js can do that client-side, e.g. see
| the documentation for calculating embeddings with Sentence
| Transformers models: https://huggingface.co/docs/transforme
| rs.js/api/pipelines#mo...
| rkagerer wrote:
| _> > In our product, we use duckdb-wasm to do vector searches
| on the client side._
|
| Curious, what is your product?
|
| Edit: Nevermind, your recent post explained it quite well:
| https://news.ycombinator.com/item?id=40520073
| pietz wrote:
| Is this also what turso uses in their "AI feature"?
| alexgarcia-xyz wrote:
| No, libsql added custom vector search directly into their
| library, while sqlite-vec is a separate SQLite extension.
|
| The libsql vector feature only works in libsql, sqlite-vec
| works in all SQLite versions. The libsql vector feature works
| kindof like pgvector, while sqlite-vec works more like the FTS5
| full text SQLite extension.
|
| I'd say try both and see which one you like more. sqlite-vec
| will soon be a part of Turso's and SQLite Cloud's products.
|
| Turso's version: https://turso.tech/vector
| alexgarcia-xyz wrote:
| Author here, happy to answer any questions! Been working on this
| for a while, so I'm very happy to get this v0.1.0 "stable"
| release out.
|
| sqlite-vec works on MacOS, Linux, Windows, Raspberry Pis, in the
| browser with WASM, and (theoretically) on mobile devices. I
| focused a lot on making it as portable as possible. It's also
| pretty fast - benchmarks are hard to do accurately, but I'd
| comfortable saying that it's a very very fast brute-force vector
| search solution.
|
| One experimental feature I'm working on: You can directly query
| vectors that are in-memory as a contiguous block of memory (ie
| NumPy), without any copying or cloning. You can see the
| benchmarks for that feature here under "sqlite-vec static", and
| it's competitive with faiss/usearch/duckdb
| https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/i...
| rcarmo wrote:
| Great to see this. Seems simple enough, but I can't wait until
| ORMs like peewee incorporate support alongside things like FTS,
| etc. just for the sake of case of use.
| cyanydeez wrote:
| Which wasm sqlite project would it be compatible with?
| ncruces wrote:
| It can be statically compiled into the official SQLite Wasm
| builds:
|
| https://alexgarcia.xyz/sqlite-vec/wasm.html
|
| It can also be statically compiled into the Wasm used by my
| Go bindings.
| bambax wrote:
| Thank you for this, it's really super exciting!
|
| The link on _See Installing sqlite-vec for more details._
| https://alexgarcia.xyz/sqlite-vec/installing.html is a a 404
| (the correct link is https://alexgarcia.xyz/sqlite-
| vec/installation.html presumably).
|
| The datasette link https://datasette.io/plugins/datasette-
| sqlite-vec is an error 500.
|
| On the releases page https://github.com/asg017/sqlite-
| vec/releases/tag/v0.1.0 can you explain what is vec0.dll vs
| sqlite-vec-0.1.0-loadable-windows-x86_64.tar.gz, which also
| contains a similarly named vec0.dll but of a different size?
| alexgarcia-xyz wrote:
| will fix up those links this weekend, thanks for pointing it
| out!
|
| the cvec0.all file can be ignored. i was debugging a issue
| and uploaded a mscv compiled version as that, but its not
| part of the official release. prefer the tar.gz version and
| let me know if it doesn't work
| bambax wrote:
| Thanks.
|
| Can't seem to make the loadable version work on Windows
| client (using .load and the latest x64 version of sqlite).
|
| On Windows 10 it says "Error: the specified module could
| not be found."
|
| On Windows 7 the error is more detailed and complains about
| missing libgcc_s_seh-1.dll which, according to a quick
| Google search, may or may not result from a compilation
| problem with MinGW...?
| Cieric wrote:
| I feel like I've touched a lot of things where something like
| this is useful (hobby projects). In my case I've done a
| recommendation engine, music matching (I specifically use it for
| matching anime to their data), and perceptual hash matching.
| alexgarcia-xyz wrote:
| Really curious to hear about what kind of music embedding
| models/tools you used! I've tried finding some good models
| before but they were all pretty difficult to use
| Cieric wrote:
| In the past I mainly screwed around with [2] musig as it was
| fairly easy to modify, right off hand I don't know why I
| stopped using it. [1] seek-tune (not-shazam) was just post
| here on hn yesterday, I had to modify it quite a bit to
| actually get something working in a way I was happy with.
| Overall it was a little slow, but that's not something I'm
| gonna fault the author for as the project is still quite new.
| While browsing around yesterday after messing with [2], I
| found [3] cpuimage/shazam. It's a little easier for me to
| screw around with as it's in c, and would probably be the one
| I try and modify to use this project.
|
| Everything is based on the shazam algorithm or something
| similar, so chunking the audio stream, and fingerprinting it.
| I was focused on that because it allows me to figure out
| where OP and ED are in some anime and allows me to add a skip
| button. Also since anime in the same season typically have
| the same OP/ED I can use that to try and auto categorize them
| after the first ones been identified.
|
| TL;DR: It's all shazam look alikes, everything else was to
| annoying to use.
|
| [1] https://github.com/cgzirim/seek-tune [2]
| https://github.com/sfluor/musig [3]
| https://github.com/cpuimage/shazam
| yard2010 wrote:
| Can you elaborate about your projects please? What tools are
| you using?
| Cieric wrote:
| Not really sure what I can give you. The tools were basically
| just golang, gcc and docker. Everything else inspection wise
| just came while I was writing the code. The viewer I use is
| jellyfin, but I didn't get far enough in the project to have
| a functioning plugin for it. If you want more info feel free
| to email me or shoot me a friend request on discord (both
| visible on my profile).
| haolez wrote:
| What's the maximum vector size?
| alexgarcia-xyz wrote:
| vec0 virtual tables have a hard-coded max of 8192 dimensions,
| but I can raise that very easily (I wanted to reduce resource
| exhaustion attacks). But if you're comparing vectors manually,
| then the `vec_distance_ls()` and related functions have no
| limits (besides SQLite's 1GB blob limit)
| dang wrote:
| Related:
|
| _I'm writing a new vector search SQLite Extension_ -
| https://news.ycombinator.com/item?id=40243168 - May 2024 (85
| comments)
| deepsquirrelnet wrote:
| I love this. I know how much work addressing the dependencies
| must be, but you're really attacking the right problems. Looking
| forward to trying this out with my project.
| nattaylor wrote:
| I have a use case for this that I'm excited to try. I'm glad
| AlexG has put so much effort into this. Even the docs are pretty
| good!
|
| My pyenv python3.12.2's sqlite won't load extensions even after
| installing with what I think are the correct command line flags.
| Argh!
|
| My brew installed python3.12's sqlite will load extensions
| though, so I can proceed.
| fsndz wrote:
| I love this. I am currently doing this tutorial of RAG where the
| vector DB is simply postgreSQL and pgvector. I guess I can try to
| reproduce that with SQLite and sqlite-vec now ! Awesome:
| https://www.lycee.ai/courses/91b8b189-729a-471a-8ae1-717033c...
| remram wrote:
| Be less obvious about your spam for lycee.ai? You only comment
| links to your own damn site.
| huevosabio wrote:
| Been using this for video games and it's absolutely awesome.
| Alex, the author, is also great and very approachable.
|
| I've been looking for something like this for a while.
| bcjordan wrote:
| Curious what applications for vector search you've found
| interesting
___________________________________________________________________
(page generated 2024-08-03 23:01 UTC)