[HN Gopher] Show HN: Lantern - a PostgreSQL vector database for ...
___________________________________________________________________
Show HN: Lantern - a PostgreSQL vector database for building AI
applications
We are excited to share Lantern! Lantern is a PostgreSQL vector
database extension for building AI applications. Install and use
our extension here: https://github.com/lanterndata/lantern We have
the most complete feature set of all the PostgreSQL vector database
extensions. Our database is built on top of usearch -- a state of
the art implementation of HNSW, the most scalable and performant
algorithm for handling vector search. There's three key metrics we
track. CREATE INDEX time, SELECT throughput, and SELECT latency. We
match or outperform pgvector and pg_embedding (Neon) on all of
these metrics. ** Here's what we support today ** - Creating an
AI application end to end without leaving your database (example:
https://github.com/ezra-varady/lanterndb-semantic-image-sear...) -
Embedding generation for popular use cases (CLIP model, Hugging
Face models, custom model) - Interoperability with pgvector's data
type, so anyone using pgvector can switch to Lantern - Parallel
index creation capabilities -- Support for creating the index
outside of the database and inside another instance allows you to
create an index without interrupting database workflows. ** Here's
what's coming soon ** - Cloud-hosted version of Lantern -
Templates and guides for building applications for different
industries - Tools for generating embeddings (support for third
party model API's, more local models) - Support for version
control and A/B test embeddings - Autotuned index type that will
choose appropriate index creation parameters - 1 byte and 2 byte
vector elements, and up to 8000 dimensional vectors support ** Why
we started Lantern today ** There's dozens of vector databases on
the market, but no enterprise option built on top of PostgreSQL. We
think it's super important to build on top of PostgreSQL -
Developers know how to use PostgreSQL. - Companies already store
their data on PostgreSQL. - Standalone vector databases have to
rebuild all of what PostgreSQL has built for the past 30-years,
including all of the optimizations on how to best store and access
data. We are open source and excited to have community
contributors! Looking forward to hearing your feedback!
Author : ngalstyan4
Score : 103 points
Date : 2023-09-13 17:41 UTC (5 hours ago)
(HTM) web link (docs.lantern.dev)
(TXT) w3m dump (docs.lantern.dev)
| justanotheratom wrote:
| can I use this in Supabase?
| diqi wrote:
| Unless Supabase decides to integrate Lantern (currently they
| integrate pgvector) you unfortunately cannot use Lantern with
| Supabase.
|
| That said, we will offer Lantern Cloud, our own hosted postgres
| offering (very soon. Happy to keep you in the loop. If you're
| interested, please feel free to join the waitlist here:
| https://forms.gle/PouJxAWiSa63udJW8
| radus wrote:
| Not a Supabase user, but to my knowledge -- no. Except for a
| set of blessed extentions, Supabase only supports extensions
| that are written in "trusted languages" (ie. supported by
| pg_tle), and Lantern is written in C, which is currently not
| supported.
| diqi wrote:
| pgvector is written in C and is supported by Supabase.
| There's nothing inherent preventing Supabase from supporting
| Lantern.
| bryan0 wrote:
| > Switch from pgvector, get FREE AirPods Pro.
|
| > Book some time here, and we will help switch you over for FREE
| and get you a pair of FREE AirPods Pro
|
| This just comes off as sketchy to me. If the tech is good it will
| stand on its own.
| consoomer wrote:
| Not going to lie.. the more I use Postgres the more I realize my
| entire application is Postgres. Soon you'll be doing entire CRUD
| endpoints and sending emails from Postgres...
|
| Wait, PostgREST already does...
|
| _builds entire SaaS with Postgres_
| codetrotter wrote:
| PGaaS :D
| [deleted]
| _boffin_ wrote:
| Postgres is love. Postgres is life.
| ezra-varady wrote:
| Hey everyone for those interested I built an updated version of
| the lanterndb semantic search application that should be a bit
| nicer. An instance is running at
|
| http://170.187.170.169/
|
| And code can be found at
|
| https://github.com/ezra-varady/react-semantic-search
| simonw wrote:
| "There's three key metrics we track. CREATE INDEX time, SELECT
| throughput, and SELECT latency."
|
| There's a fourth metric that I'm really interested in: assuming
| it's possible, how long does it take to update the index with
| just one or two updated or inserted vectors?
|
| Is the expectation with this (and the other) tools that I'll do a
| full index rebuild every X minutes/hours, or do some of them
| support ongoing partial updates as data is inserted and updated?
|
| Just had a thought: maybe I could handle this case by maintaining
| an index of every existing vector, then tracking rows that have
| been created since that index itself.
|
| Then I could run an indexed search that returns the top X results
| + their distance scores, then separately do a brute-force
| calculation of scores just for the small number of rows that I
| know aren't in the index - and then combine those together.
|
| Would that work OK?
|
| Even if the index doesn't return the scores directly, if it gives
| me the top 20 I could re-calculate distance scores against those
| 20 plus the X records that have been inserted since the index was
| creation and return my own results based on that.
| diqi wrote:
| > There's a fourth metric that I'm really interested in:
| assuming it's possible, how long does it take to update the
| index with just one or two updated or inserted vectors?
|
| Here's a chart for INSERT latency (sorry about the formatting):
| https://docs.lantern.dev/graphs/insert.png
|
| At the moment, we underperform Neon wrt this metric, but a
| better implementation is coming soon that will address this.
|
| > Is the expectation with this (and the other) tools that I'll
| do a full index rebuild every X minutes/hours, or do some of
| them support ongoing partial updates as data is inserted and
| updated?
|
| The HNSW algorithm updates the index after every insert. So all
| existing HNSW options (Lantern, pgvector, Neon, ...) already
| support this.
|
| With pgvector IVFFlat, you expect the performance to degrade
| over time, and you will need to re-index. This is because
| IVFflat's index quality heavily depends on the centroids chosen
| at index creation time. HNSW does not have this limitation.
|
| In both cases, you might want to do a full-index build to tune
| your hyperparameters.
|
| We're working on this in a few ways. One is automatic
| hyperparameter tuning. Another is supporting external index
| creation that would offload this to another server. Does this
| answer your question?
| simonw wrote:
| That's really useful, thanks.
| raoufchebri wrote:
| How do you handle conflict with pgvector's hnsw if you want to
| install both extensions ?
|
| CREATE INDEX semantic_image ON image_table USING hnsw (v
| dist_cos_ops) WITH (M=5, ef=30, ef_construction=30, dims=512);
| ngalstyan4 wrote:
| Our index access method will be called lantern_hnsw if pgvector
| or any other provider has already taken the hnsw access method
| name.
|
| btw, we did not create our own vector type and just use size-
| enforced real[] arrays to represent embeddings. However, you
| can use our index with pgvector's vector type. So, if you
| already have a table with pgvector's vector column type, you
| can start using Lantern by just creating an index on the same
| column.
| swalsh wrote:
| I like how easy postgresql vector is to use, but scaling up seems
| to get pretty expensive when comapred to something like qdrant.
| jeungsp wrote:
| Nothing fundamentally stops a postgres implementation being
| equally performant as something like Qdrant.
|
| Fundamentally, an index's performance is based on the hardware
| and the algorithm and the quality of implementation. Any
| optimizations Qdrant can make, we can also make.
|
| We will be benchmarking ourselves against all of the other
| standalone database options as well and we'll be working to try
| and outperform them. Excited to share those once we have them.
|
| We are curious - what are you most concerned about? SELECT
| time? INDEX size? Latency? Throughput?
| howon92 wrote:
| Congrats on the launch!!! I suggest you highlight "why lantern is
| better than pgvector" at the top of your page. The first thing
| that came to my mind after reading this was "why should I use
| this instead of pgvector?"
| jeungsp wrote:
| This is a great idea. We will have lots of content in the days
| ahead to talk about why people should make the switch.
|
| Do you use pgvector now? Would love to switch you over.
| fakedang wrote:
| This might be a noob question but what does Lantern have that a
| normal Postgres dB with pgvector does not? I think Supabase
| already has a Postgres as a service product with the pgvector
| extension too.
|
| Second:
|
| >Creating an AI application end to end without leaving your
| database (example: https://github.com/ezra-varady/lanterndb-
| semantic-image-sear...)
|
| What does "without leaving your database" mean in this context?
| simonw wrote:
| I assume that question - Lantern v.s. pgvector - is meant to be
| answered by those performance graphs: Lantern is a bit faster.
|
| (I'd find those graphs easily to quickly understand if they had
| a "lower is better"/"higher is better" note on each one.)
| jeungsp wrote:
| Hi, Jeung here, one of the co-founders of Lantern.
|
| You're right. Our performance is just a bit faster today.
|
| We expect everyone's performance to continue to improve, but
| we have lots of improvements coming soon, and we plan to
| continue to widen the lead.
|
| But there are more important reasons to use Lantern besides
| performance. If you look at our features list and what's
| coming soon, we are looking to create tools that make for
| better applications and better developer experience.
| ngalstyan4 wrote:
| Pgvector builds a vector index.
|
| Our extension, similarly, builds an index but also extends SQL
| in more ways.
|
| For example,
|
| - Generating embeddings to augment plain relational data
|
| - Using data from local proprietary embedding models or third-
| party model APIs in queries.
|
| We have more things planned like vector versioning, data
| retention policies and recall regression tracking.
|
| > What does "without leaving your database" mean in this
| context?
|
| You can work with embeddings with just SQL. For instance, a
| table of academic papers can be augmented with CLIP model
| embeddings produced locally. This entire process - creating,
| storing, and querying - happens using just SQL."
| SELECT abstract, introduction, figure1,
| clip_text(abstract) AS abstract_ai,
| clip_text(introduction) AS introduction_ai,
| clip_image(figure1) AS figure1_ai INTO papers_augmented
| FROM papers; SELECT abstract,
| introduction FROM papers_augmented ORDER BY
| abstract_ai <-> clip_text("The Foundation of the General Theory
| of Relativity") LIMIT 10;
| saurik wrote:
| When you say "produced locally", do you mean on the client?
| If so, does this mean you require me to use some alternate
| PostgreSQL driver locally, parsing the SQL to add your one
| feature?
|
| (If it is, this really feels like it should be a separate
| general purpose local extension mechanism into which random
| functions can be added, instead of something tied to this use
| case... maybe I want to add some locally-executed string
| parsing function, for example...)
|
| (...but, the entire concept of having some functions be
| "locally" executed also feels really awkward/limited and will
| involve a ridiculous amount of work to make, at the end of
| the day, it only sort of work in some places in the query, so
| I bet you don't mean what I do when I say "locally", right?)
|
| (But, like... doing it remotely--on the database server as
| part of the query plan--frankly seems kind of crazy to me, as
| it is going to be _so slow_ and add a massive CPU load to
| what should be an I /O workload. Makes for good demos I bet,
| but otherwise unusable in a database context.)
|
| (Regardless, the premise of seeing this as a feature kind of
| squicks me... like, it honestly gives me strong apprehensions
| about using your extension at all, as I can see--very clearly
| --the mission creep it is going cause as you deal with
| demands to drag more and more popular embedding models with
| lots of execution dependencies as part of the extension that
| has to be loaded into the server, as well as fielding
| distracting discussions about the performance of the
| embedding helpers...)
|
| (...this frankly shouldn't be part of the same extension: it
| should be another extension that happens to return this
| extensions data type--or even potentially returns some more
| generic one, like an array of float, making it drop-in
| compatible with other extensions for vector indexing--and
| there should then almost certainly be separate such
| extensions for each major model you want to support.)
| ngalstyan4 wrote:
| >When you say "produced locally", do you mean on the
| client?
|
| Sorry for the confusion. By "produced locally" I meant
| "produced on your DB server" as opposed to being an API
| call to a third party service such as OpenAI or
| HuggingFace.
|
| (But, like... doing it remotely--on the database server as
| part of the query plan--frankly seems kind of crazy to me,
| as it is going to be so slow and add a massive CPU load to
| what should be an I/O workload. Makes for good demos I bet,
| but otherwise unusable in a database context.)
|
| It seems like you're worried about these workflows being on
| the Postgres server, which may lead to performance issues.
|
| However, if performance becomes an issue, the functions can
| be executed on another server. In this approach, whether or
| not the functions run on the Postgres server, the end user
| gets access to a better developing experience as all the
| functions they need are available within SQL.
|
| >...this frankly shouldn't be part of the same extension We
| agree. These functions are already in another repository,
| and not part of the same extension. The repository is here:
| https://github.com/lanterndata/lantern_extras
| ashvardanian wrote:
| Epic result, and thank you for mentioning USearch! Would be happy
| to further optimize it for your needs!
|
| I also love your specialized CI! Pgvector probably doesn't report
| performance changes between releases, or does it? Was it easy to
| implement? Do you run the whole eval on GitHub?
| diqi wrote:
| Thanks!
|
| I don't believe pgvector reports performance changes between
| releases.
|
| At the moment, we run the benchmarking on Github CI, but we
| plan to move this to an external machine, since the results are
| unstable on Github machines. We're planning to extend
| benchmarking across other repos and versions.
| mattashii wrote:
| How do you do cleanup of the index during VACUUM?
|
| And, do you have recall-vs-qps graphs like those on https://ann-
| benchmarks.com/ ? Those are generally more apples-to-apples, as
| 100k rows isn't exactly a reputable benchmark.
| diqi wrote:
| We don't do cleanup of the index during VACUUM yet. That said,
| it's coming very soon. We're built on top of Usearch, which
| supports deletes. We plan to work with the Usearch team to port
| the post performant deletes to Lantern, and thereby support
| VACUUM.
|
| With respect to recall vs QPS, we went ahead and generated this
| plot, hope this is helpful?
| http://docs.lantern.dev/graphs/recall-tps.png
|
| You're right, 100k rows isn't a reputable benchmark. We wanted
| to launch very quickly, and have benchmarking for larger
| datasets coming soon. Benchmarking is baked into our CI/CD, we
| take it very seriously!
| carlossouza wrote:
| I'm using pgvector in production, mainly in a table with 500k-1M
| rows.
|
| My main use case is to return search results _with pagination_ :
| page 1 from 1-50, page 2 from 51-100, page 3 from 101-150, etc.
| (Think LIMIT and OFFSET).
|
| After a lot of experimentation and help from pgvector's team, we
| discovered that, for this specific use case, IVFFLAT index is
| much faster than HNSW.
|
| I looked at your documentation and only saw HNSW, no IVFFLAT.
|
| What would be Lantern's performance for this specific use case?
|
| Thx!
| ngalstyan4 wrote:
| This sounds like a very useful feature, and we will prioritize
| this.
|
| You're correct that IVFFLAT would be faster for your use case.
| However, IVFFLAT's shortcoming is bad recall, which means less
| relevant results for your application. We believe that our HNSW
| implementation (or other indexes) can handle use cases like
| yours.
|
| We currently handle a similar use-case by rerunning our index
| searches with exponentially increasing LIMITs and dropping the
| results which are not needed. Could you send us an email at
| support@lantern.dev? We can generate the numbers by this
| weekend, and get back to you with concrete results.
|
| By the way - not sure if you saw in our blog post, if you're
| using pgvector in production and switch to Lantern, we'll help
| you every single step of the way. It's very quick, and we'll
| also send you some free AirPods Pro at the end of it!
| dalberto wrote:
| Any plans to support sparse vectors?
| jeungsp wrote:
| We're built on top of Usearch, which will very soon support
| sparse vectors. We're working with them to make sure it also
| works in Lantern.
|
| Can you tell me more about your use case?
| dalberto wrote:
| For hybrid search
___________________________________________________________________
(page generated 2023-09-13 23:00 UTC)