Posts by alexgarciaxyz@data-folks.masto.host
(DIR) Post #AUQHwrDMZBKnBGvpPE by alexgarciaxyz@data-folks.masto.host
2023-04-07T22:49:23Z
0 likes, 0 repeats
Took some time to organize all my SQLite extensions into one repo!The scale is quite something:- 11 SQLite extensions, in Rust/C/C++/Go- 10 pip packages- 10 npm packages- 10 deno modules- 9 Datasette plugins- 35 individual compiled assetshttps://github.com/asg017/sqlite-ecosystemAnd that's just the start! Check out the "roadmap" section of new extensions/tools I plan to release in the near future
(DIR) Post #AURjRej6h5F3RLb93Q by alexgarciaxyz@data-folks.masto.host
2023-04-08T15:32:13Z
0 likes, 0 repeats
@implgeo that would be awesome!Here's the repo. I still have some in-progress work that I'll try to push this week, along with a contributing guide. Any help (including issues to get some discussion started) would be greatly appreciated!https://github.com/asg017/sqlite-geo
(DIR) Post #AUWae74VCoMX1qGpYO by alexgarciaxyz@data-folks.masto.host
2023-04-10T23:47:10Z
0 likes, 0 repeats
v0.0.2 of sqlite-vss is out!You can now install sqlite-vss with pip for Python, npm for Node.js, denoland/x for Deno, and as a Datasette plugin!As far as I know, this is the only *local* vector search tool with Python+Node.js+Deno bindingshttps://github.com/asg017/sqlite-vss
(DIR) Post #AWUM9Nu1u64cpY4Ky8 by alexgarciaxyz@data-folks.masto.host
2023-06-08T17:39:31Z
0 likes, 0 repeats
SQLite extensions that are pip install'able, npm install'able, and now gem install'able for Ruby developers!Details, context, and how it all works: https://observablehq.com/@asg017/making-sqlite-extension-gem-installable
(DIR) Post #AiBkcddFaR7mUQ2i00 by alexgarciaxyz@data-folks.masto.host
2024-05-23T16:39:25Z
0 likes, 0 repeats
A new blog post from me: Vector search, using SQL, in multiple programming languages and runtimes, all with the new upcoming sqlite-vec project. - Python- Ruby- Node.js (and Bun/Deno)- Javascript + WASM- Go- Rust- CCheck it out! https://alexgarcia.xyz/blog/2024/sql-vector-search-languages/index.html
(DIR) Post #AivlFsHPEtTZfr6E2C by alexgarciaxyz@data-folks.masto.host
2024-06-14T21:23:15Z
0 likes, 0 repeats
I'm thrilled to announce that sqlite-vec has found a main sponsor: Mozilla through their new Mozilla Builders project!fly.io, Turso, and SQLite Cloud have joined as additional sponsors. I'm hard at work for getting a v0.1.0 release out soon - stayed tuned for some sneak peeks + blog posts!https://future.mozilla.org/news/announcing-sqlite-vec-mozilla-builders-project/
(DIR) Post #AkIcr6Vip3dZOVlF8S by alexgarciaxyz@data-folks.masto.host
2024-07-25T19:59:36Z
0 likes, 0 repeats
Hello hello, a few new SQLite extensions for you all: First off: sqlite-lembed, for generating text embeddings from local GGUF embedding models using llama.cpp. https://alexgarcia.xyz/blog/2024/sqlite-lembed-init/index.htmlNext: sqlite-rembed, for generating text embeddings from remote sources like OpenAI, Nomic, Cohere, Ollama, llamafile, and more!https://alexgarcia.xyz/blog/2024/sqlite-rembed-init/index.htmlAnd finally: sqlite-vec v0.1.0 is coming next week!
(DIR) Post #AkWo81yojwCRyT5DBw by alexgarciaxyz@data-folks.masto.host
2024-08-01T16:11:52Z
0 likes, 0 repeats
sqlite-vec, a vector search SQLite extension that runs everywhere, now has its first stable v0.1.0 release!You can install it with:- Python (`pip install sqlite-vec`)- Node.js (`npm install sqlite-vec`)- Ruby (`gem install sqlite-vec`)- Rust (`cargo add sqlite-vec`)- Go (CGO) `go get github.com/asg017/sqlite-vec-go-bindings/cgo`- Go (WASM) `go get github.com/asg017/sqlite-vec-go-bindings/ncruces`- yolo `curl -L https://github.com/asg017/sqlite-vec/releases/download/v0.1.0/install.sh | sh`Try it out!https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html
(DIR) Post #Akfo4NeDzCNyRMLjhA by alexgarciaxyz@data-folks.masto.host
2024-08-06T00:23:36Z
0 likes, 0 repeats
sqlite-vec v0.1.1 has been released with a few bug fixes:- Datasette and sqlite-utils bindings are fixed- You can now statically compile sqlite-vec with at least SQLite version 3.31.1 (2020)- Expose serialize_float32() in Python APIhttps://github.com/asg017/sqlite-vec/releases/tag/v0.1.1
(DIR) Post #AmbOooKARfg4O8u9wG by alexgarciaxyz@data-folks.masto.host
2024-10-02T17:06:00Z
0 likes, 0 repeats
New blog post: How to do "Hybrid search" with SQLite FTS5 and sqlite-vecReciprocal rank fusion, FTS-first, re-order by semantics, and more!Featuring the great snowflake-arctic-embed-m-v1.5 model and sqlite-lembed, using pure SQL examples.https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/index.html
(DIR) Post #AmbZ0vv71qpuxmbY6C by alexgarciaxyz@data-folks.masto.host
2024-10-02T19:00:16Z
0 likes, 0 repeats
@markus Tiny models are super useful! I personally can't tell much difference btwn openai and OS models like Snowflake Artic/Nomic/MixedBread.Also, fine-tuning scared me at first, but much easier/cheaper to do that than relying on 3rd parties, if you really wanna squeeze perf. Most of these models claim better benchmarks than OpenAI anyway. I personally never use OpenAI embeddings, but I'm sure they have their place. Ex maybe it's better at multi-lingual/code, but haven't seen that yet
(DIR) Post #Ambdz0HsJvvjhQ9QK8 by alexgarciaxyz@data-folks.masto.host
2024-10-02T19:55:56Z
0 likes, 0 repeats
@markus Thanks! llamafile does support embeddings models btw, here's a guide: https://github.com/Mozilla-Ocho/llamafile/blob/main/llamafile/server/doc/getting_started.mdThe `llamafiler` binary is available is the .zip release: https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.8.13
(DIR) Post #Ambf05BjgTrp8wMFEm by alexgarciaxyz@data-folks.masto.host
2024-10-02T20:07:21Z
0 likes, 0 repeats
@leafmuncher thank you!
(DIR) Post #Ao4qKJ4wNH5MahsxN2 by alexgarciaxyz@data-folks.masto.host
2024-11-15T19:52:58Z
0 likes, 0 repeats
Next week I'll be hosting a sqlite-vec online event about metadata filtering, which is coming VERY soon — come through! Wednesday November 20th at 10am PST, in the @MozillaAI Discord(will be recorded)https://discord.gg/4ZMAgmrwqf?event=1300483739872399411
(DIR) Post #AoEGrztyWgTsBx7aUK by alexgarciaxyz@data-folks.masto.host
2024-11-20T09:02:48Z
0 likes, 0 repeats
👀
(DIR) Post #AoEvgc0z5q8L2j5cJc by alexgarciaxyz@data-folks.masto.host
2024-11-20T16:40:09Z
0 likes, 0 repeats
@markus yessir!
(DIR) Post #AoEw29VdkytB0OhRwW by alexgarciaxyz@data-folks.masto.host
2024-11-20T16:44:03Z
0 likes, 0 repeats
sqlite-vec v0.1.6 is now out, with metadata support!SQLite vector search w/ metadata filters 👀- Perform extra filtering w/ WHERE clause in KNN queries- Internally shard vector indexes with partition keys- Aux columns for easy lookupsread more: https://alexgarcia.xyz/blog/2024/sqlite-vec-metadata-release/index.html
(DIR) Post #AoTsC4YspwctusqV4C by alexgarciaxyz@data-folks.masto.host
2024-11-27T21:41:41Z
0 likes, 0 repeats
Please merge my +321,000 line PR, it's only a few small changes 🥺
(DIR) Post #AoTsH9bxURFn1dhu5o by alexgarciaxyz@data-folks.masto.host
2024-11-27T21:42:36Z
0 likes, 0 repeats
PR here, it vendors in sqlite3.c and a bunch of other large dependencies, I'm not a mad man https://github.com/Mozilla-Ocho/llamafile/pull/644
(DIR) Post #AoXsrNHutX42V5RZYW by alexgarciaxyz@data-folks.masto.host
2024-11-29T20:07:59Z
0 likes, 0 repeats
@markus lol i saw that! should be possible with appendvfs, which lets you append a database to an executable