[HN Gopher] LlamaIndex: Unleash the power of LLMs over your data
___________________________________________________________________
LlamaIndex: Unleash the power of LLMs over your data
Author : danboarder
Score : 189 points
Date : 2023-07-08 04:39 UTC (18 hours ago)
(HTM) web link (www.llamaindex.ai)
(TXT) w3m dump (www.llamaindex.ai)
| mritchie712 wrote:
| If you're interested in the SQL component of this, we're building
| a product strictly focused on that at https://www.definite.app/.
| We let non-technical users ask questions of their SQL database.
| We do this by:
|
| 1. Pulling in your schema information and structuring it in a way
| LLM's can reason about it
|
| 2. Pulling in your prior query history against the database to
| understand how you actually use your data (e.g. what JOIN's are
| common, what tables are used most frequently, etc.)
|
| 3. Adding context from other tools you may be using (e.g. we can
| pull in metadata and tests from your dbt project)
|
| We also have a Slackbot you can add to your #urgent-data-requests
| channel. If you @Definite in a thread, it'll parse out messages
| that can be converted to SQL tasks and return the answer from
| your database.
|
| You could certainly build this yourself with (or without)
| LlamaIndex, but it's still quite a bit of work to set up.
| kulikalov wrote:
| I've built a similar solution around BigQuery. I used temporary
| tables to let user iterate over the data instead of writing
| huge analytics queries.
|
| I would open source it if I had decent experience with OSS.
|
| If anyone with OSS maintainer experience wants to help with
| that - ping me.
| sheepscreek wrote:
| That's really neat. I'll take you up on that. Will reach out
| to your HN profile email.
| totalhack wrote:
| Just to throw it out there, an alternative approach is to have
| the AI talk to a semantic layer. There are pros and cons to
| this and in practice you'd probably want an agent that can
| intelligently decide whether to call your semantic layer or go
| through some other process that may involve multiple queries,
| function calls, etc.
|
| IMO a semantic layer makes it so easy to query that natural
| language querying is often not beneficial, even for business
| users. I take a hybrid approach of the semantic layer solving
| >95% of use cases and custom SQL as needed in backend
| processing.
|
| https://github.com/totalhack/zillion
| mritchie712 wrote:
| I like the concept of a semantic layer, but I haven't seen
| strong adoption of any of them (outside of LookML within
| Looker). Most of the companies I've talked to that are heavy
| dbt users don't use their semantic layer, but maybe that will
| change as the products mature!
| lmeyerov wrote:
| We are seeing similar for Louie.ai . Data catalog yes,
| semantic no. We are also doing a lot of non-SQL like logs
| DBs (SIEMs) and graph DBs, which adds even more fun to that
| kind of question. My hypothesis here is that "semantic
| layer is to semantic web as LLM + vector search over
| activity is to Google/pagerank": easier to let the machines
| figure it out. We use knowledge graphs in some of our work,
| so definitely not disputing the value when people do put in
| the effort.
|
| Wrt llmindex, the new tools like vector dbs & llmindex have
| been interesting. Fine-ish for our self-hosted, but most
| start getting tricky when a multi-tenant SaaS where we
| lower costs for teams and users by sharing infra yet need
| sharing boundaries.
| wanderingmind wrote:
| Do you have your source and model open? Its hard to use a AI
| service to give any type of access to prod DB without
| understanding the underlying code and models
| mritchie712 wrote:
| Totally understand the concern. We're not open source yet,
| but working towards that as the open source LLMs improve.
|
| We're SOC2 compliant which has been enough for many companies
| to get comfortable with security / data handling practices.
| greatpostman wrote:
| I built this in like one day for a pretty complex project,
| there's no moat on this idea
| chaxor wrote:
| Same is true for slack, since suc apparently does it in 5
| lines.
| kulikalov wrote:
| Technically, one can say the same about any devops services
| out there, for instance. Makes more sense to opensource it.
|
| Still, would you mind sharing the use case and the outcome? I
| have a couple of clients interested in similar solutions, but
| so far the potential outcome of this approach doesn't look
| promising.
| Art9681 wrote:
| You should try building a bridge. Moats are a pitfall,
| literally.
| splatzone wrote:
| With respect, it's all in the quality of the execution. Care
| to share your version?
| mritchie712 wrote:
| thanks, giving up now.
| alecco wrote:
| What's the difference vs embeddings on a vector database combined
| with GPT?
| lmeyerov wrote:
| Not everything should persist, such as in a classic web
| architecture, analogous to redis vs postgres
|
| Ex: For some of our use cases for more-text data sources and
| APIs we are helping folks index for Louie.ai, we do ingest-time
| embedding into persistent vector DBs for the raw, and are
| looking at llmindex and others for more ephemeral run-time
| caching of post-proccessed results. Ex: Pre-embed 1GB of text
| for fast search, but then only cache ad-hoc genAI task results
| on those, like classifications. We may still want to persist
| the embedded task results for other reasons, so not obvious.
|
| If figuring out that kind of thing for use cases like analysts
| talking to DBs for cyber investigations, supply chain,
| emergencies, etc, we are actively hiring in backend engineering
| (remote): https://www.graphistry.com/careers, who build
| louie.ai
| Ozzie_osman wrote:
| This is a layer of abstraction above that. So it handles
| ingestion of the data (possibly as embeddings), storing it
| (possibly in a vector database), and then querying of that data
| for LLM queries. You can swap in different embedding models,
| different storage databases, different retrieval methods, etc.
| ramoz wrote:
| If you're doing legitimate retrieval rerank in the commercial
| enterprise setting, then I doubt this is a library that can
| support you beyond prototyping.
|
| Retrieval involves complex integration (not just data connectors
| and open API wrappers), and meaningful rerank requires
| domain/context-specific trained models (that you can deploy
| performantly and cost effectively). If you're doing these things,
| you're well beyond the capability at platform scale vs what a
| python library provides
| byteknight wrote:
| Is this a gripe with the verbiage used or is this a knock to
| the library itself?
|
| I have been playing with langchain and llamaindex a bit.
|
| I like the data loading abstraction and am very curious why you
| say it doesn't work? It uses ChatGPT for the reranking.
| ramoz wrote:
| It's just interesting to see the VC money pouring into these
| tools. My argument is serious integration/scale doesn't
| involve a library like these (honestly prototyping doesn't
| really need to either).
|
| Id be more bullish on paradigm (platform/cloud level) shifts
| vs connectors, wrappers, and utility functions
|
| Ymmv and to be fair I haven't tried to scale these tools. I
| have worked on scaled platforms around embedding retrieval
| and rerank (including LLMs) so it's just my take.
| byteknight wrote:
| I would argue the level of abstraction it provides lowers
| the barrier to entry for most average programmers, myself
| included. LllamaIndex was my entrance to programatically
| utilizing LLMs. I have since moved to LangChain, with some
| documents loaded via LlamaIndex, but it has been a blast.
| ramoz wrote:
| Right on-- I should've and do recognize the utility of
| open source abstractions; esp with AI/ML.
| ibains wrote:
| Yes but this is just ETL - LlamaIndex and LangChain are
| re-inventing it - why use them when you have robust
| technology already?
|
| 1. You ETL your documents into a vector database - you
| run this pipeline everyday to keep it up to date. You can
| run scalable, robust pipelines on Spark for this.
|
| 2. You have a streaming inference pipeline that has
| components that make API calls (agents) and between them
| transform data. This is Spark streaming.
|
| Prophecy is working with large enterprises to implement
| generative AI use cases, but they don't talk so much on
| HN. Here's our talk from Data+AI Summit:
|
| Build a Generative AI App on Enterprise Data in 13
| Minutes
|
| https://www.youtube.com/watch?v=1exLfT-b-GM
|
| Here's a blog/demo
|
| https://www.prophecy.io/blog/prophecy-generative-ai-
| platform...
| bread90 wrote:
| Cool! Lets say I have thousands of documents that I want
| questions and answers for. Would your solution work for
| this? I wouldn't know which documents to send with the
| prompts though as I want info on the aggregate (like
| trends and most mentioned phrases or words).
| bandyaboot wrote:
| This sounds really cool. I predict it will consume a good chunk
| of my free time in the next week or so.
| freezed88 wrote:
| Hey all! Jerry here (from LlamaIndex).
|
| We love the feedback, and one main point especially seems to be
| around making the docs better: - Improve the organization to
| better expose both our basic and our advanced capabilities -
| Improve the documentation around customization (from LLM's to
| retrievers etc.) - Improve the clarity of our examples/notebooks.
|
| Will have an update in a day or two :)
| rollinDyno wrote:
| I gave this a shot a while back and found plenty of examples but
| little documentation.
|
| For instance, there is a tree structure for storing the
| embeddings and the library is able to construct it with a single
| line. However, I couldn't find an clear explanation of how that
| tree is constructed and how to take advantage of it.
| luckyt wrote:
| Yea, this was my experience too when I tried it out last week
| for my side project. It's easy to get started, but it's quite
| complex and disorganized and poorly documented. There are
| usually several ways to do things (which is by design, since
| it's meant to give you flexibility of either going with the
| default or customizing).
|
| The main problem is the documentation is too disorganized, it's
| hard to figure out what even is the default and what are the
| configuration options, documentation is spread over a bunch of
| tutorials, reference pages, and blog posts by the founder.
| Sometimes the example code doesn't quite work because the
| library is changing so quickly.
|
| We'll see if the community can figure out the best set of
| useful abstractions for this domain -- right now LlamaIndex is
| a mess and makes building things harder instead of easier and
| it's probably simpler to roll your own solution from scratch.
| However, the founders seem pretty smart, so hopefully with some
| time, they'll improve it and make it more usable.
| mark_l_watson wrote:
| I wrote a short LangChain+LlamaIndex book that you can read
| free online: https://leanpub.com/langchain/read
|
| Also, check out LlamaHub - lots of examples.
| qwertox wrote:
| There once existed Google Desktop which was really useful.
|
| Is this something similar, but with the added feature of being
| able to query the data with the help of a LLM?
|
| Like: Find me all the text files which I've modified last month,
| there should be one containing a log snippet with a TODO I added
| to it.
| patrakov wrote:
| The name is misleading: this project is not based on LLaMA as
| released by Meta. It sends data to OpenAI by default.
| kernelsanderz wrote:
| It's not really. It was unfortunate timing. This used to be
| called GPT-Index, and then they changed names before Meta
| released their LLM. So their use predated it.
|
| I feel sorry for the amazing team behind this great library.
| Changing names is hard.
| homarp wrote:
| 17 feb 2023,
| https://twitter.com/llama_index/status/1626387226639888385
|
| 'Today, we're kicking off a rebrand of @gpt_index to :
| LlamaIndex '
|
| Meta announcing LLaMa 24 feb 2023
| https://ai.facebook.com/blog/large-language-model-llama-
| meta...
| gtirloni wrote:
| Was Llama a top-secret name at Meta? Could it have leaked?
| lolinder wrote:
| It's a pretty easy pun to arrive at from the letters LLM,
| so I don't think we need a leak to explain the coincidence.
| ShamelessC wrote:
| Wow, that is some really bad luck.
| peterisza wrote:
| Why is it hard? They should change it back.
|
| s/Llama/GPT-/g
| lolinder wrote:
| OpenAI has started sending out threatening letters to
| people who use GPT in their project name. If they don't
| want to be the test case for that (pending) trademark, I
| don't blame them.
| peterisza wrote:
| Holy sh.t! I didn't know that.
| lolinder wrote:
| Discussed here a few months back:
|
| https://news.ycombinator.com/item?id=35973645
| peterisza wrote:
| Thanks
| swader999 wrote:
| Imma gonna go and grab gippity.com asap.
| behnamoh wrote:
| Still, they could have used a more general term other than
| "llama". They had it coming imo.
|
| Sometimes developers come up with creative and interesting
| names for their products, but other times they fail miserably.
| lolinder wrote:
| They being Meta or they being the developers of LlamaIndex?
| As noted in your sibling comment, LlamaIndex was public
| first.
| behnamoh wrote:
| the latter. Meta's naming made sense cuz it was an acronym.
| LlamaIndex sounds like a joke name and not well thought out
| for such an ambitious product.
| lolinder wrote:
| Meta's naming is clearly also a joke. It's a backronym,
| not an acronym-- it's pretty obvious they started with
| Llama and came up with a cute acronym to justify it.
|
| Large LAnguage Model Meta AI
|
| Not saying they're not entitled to use the name too, but
| laying the blame on the developers of LlamaIndex when
| they had no idea that LLaMA was coming isn't fair to
| them.
| poxrud wrote:
| Is this an alternative/competitor to langchain? If so which one
| is easier to use?
| minimaxir wrote:
| That is what I am specifically going for with simpleaichat:
| https://news.ycombinator.com/item?id=36393782
|
| It still needs a lot of work, but the goal is to decouple tools
| from LLM business logic (e.g. you would bring your on vector
| retrieval logic)
| lmeyerov wrote:
| Complement
|
| This stuff is like redis. Great for the infra folks
|
| Langchain is like ruby on rails. Great for the AI app devs
|
| There is overlap, where you might start and stay on langchain
| as a kitchen sink... But as you do full app dev, esp SaaS, the
| bottom starts falling out for more data infra centric tooling.
| Der_Einzige wrote:
| I just switched back to langchain when I realized that
| llamaindex doesn't support any kind of memory modules, at least
| not in a documented way...
| mabcat wrote:
| It's an alternative, does a similar job, depends on/abstracts
| over langchain for some things. It's easier to use than
| langchain and you'll probably get moving much faster.
|
| They've aimed to make a framework that starts concise and
| simple, has useful defaults, then lets you adjust or replace
| specific parts of the overall "answer questions based on a
| vectorized document collection" workflow as needed.
|
| This works well overall, but some bits have kept me scratching
| my head for hours. Partly due to huge holes in the
| documentation when it comes to specifics ("plenty of examples
| but little documentation" another commenter wrote and I agree).
| Partly due to the frenetic release schedule, this project is
| highly active even by frothy LLM craze standards and interfaces
| change rapidly.
|
| Overall recommend, LlamaIndex has helped me make good progress
| on my project
___________________________________________________________________
(page generated 2023-07-08 23:02 UTC)