[HN Gopher] Langchain Is Pointless
___________________________________________________________________
Langchain Is Pointless
Author : behnamoh
Score : 220 points
Date : 2023-07-08 15:56 UTC (7 hours ago)
(HTM) web link (old.reddit.com)
(TXT) w3m dump (old.reddit.com)
| lmeyerov wrote:
| I've definitely got heartache here, and they merit criticism, but
| it is real
|
| We need a lot of pluggability to support diff vendor LLMs and BYO
| LLMs in Louie.ai, so having langchain has been nice for helping
| code to interfaces vs vendor lockin. It definitely has growing
| pains - ex: sync & multithreading is important for us so we are
| generally coding around langchain while that smooths out.
| Likewise, we ended up building much of our conversational and
| multitool capabilities as custom libraries vs using theirs for
| similar quality reasons. We can't use any of the codegen
| capabilities because they are massive security holes so doing our
| own work there too.
|
| If anyone is into that kind of work (backend, AI & web infra,
| ...), definitely hiring for core platform & cool customer
| projects here: louie.ai / Graphistry.com/careers
| lmeyerov wrote:
| Also, the thread title is unintentionally funny: I'd like the
| interface to be more functional so we can write truly 'point-
| free' pipelines, especially around areas like memory. Ex: When
| dealing with multithreading, that makes it a lot safer. There
| are projects exploring that, but langchain is winning as a
| pluggable interface for many new LLM providers.
|
| Edit: link - https://en.wikipedia.org/wiki/Tacit_programming
| tedtimbrell wrote:
| For me the biggest benefit is just following the project and
| seeing what the community is doing with llms (It's also not bad
| for quick proofs of concept).
|
| That said especially in python it's not that hard to reimplement
| things yourself in a cleaner way. Output parsing for agents was
| nice but with the function update from OpenAI it's not really
| necessary (if you're just using their API)
| codeptualize wrote:
| Coming from a frontend background this reminds me a lot of the
| frontend situation some years ago when it was super common to npm
| install the stupidest pointless packages.
|
| Of course some people still do, but hard lessons were learned and
| all experienced people I know are a lot more mindful and cautious
| what dependencies they add.
|
| It seems to me that this space, and maybe data science more
| broadly, is currently in that situation. Maybe it's the lack of
| coding skills, maybe it's the transition from one-off research
| notebooks to production applications, or maybe it's just the norm
| to have big do it all libraries (like pandas, scikit etc), idk,
| but I expect the same lessons will be learned.
|
| For Langchain I wonder how they will keep up with changes in all
| the things they have their abstractions on. I guess having a huge
| community helps, but that doesn't help you with compatibility. It
| would not surprise me if that will get really ugly.
| ShamelessC wrote:
| The problem is that coders are used to dealing with code. GPT-4
| is a robust processor for semantics as conveyed by strings of
| characters. The whole point is that you don't need code. You
| just ask it what you want.
|
| But programmers love to think they can still make improvements
| to such a system using code. In reality, any improvements that
| can be made are the responsibility of f-strings and/or a
| templating mechanism and even that may be overkill in many
| cases.
| aiunboxed wrote:
| Thank god, there is someone else who feels the same way. There
| are tons of modules and none of them is production ready.
|
| It feels good for hobby / college projects, but would not use it
| for production.
|
| The pace at which they release features is also scary.
| jmugan wrote:
| I found that you need to dig into the actual code and debug it
| while it is running to see what is going on to actually use the
| library (both with Langchain and LlamaIndex). That's unfortunate,
| but it does show a path to get you to where you want to go and is
| probably faster in the long run than writing your own code
| because you can swap components (e.g., which LLM or which vector
| DB) in and out.
| jangletown wrote:
| I agree, I really don't like LangChain abstractions, the chains
| they say are "composable" are not really, you spend more time
| trying to figure out langchain than actually building things with
| it, and it seems it's not just me after talking to many people
|
| Their code seems all rushed, and seems it worked out for initial
| popularity, but with their current abstractions I personally
| don't think it's a good long term framework to learn and adopt
|
| That's why I built my own alternative to it, I call it LiteChain,
| where the chains are actual composable monads, the code is async
| streamed by default not ducktaped, it's very bare bones yet but
| I'm really putting effort on building a solid foundation first,
| and having a final simple abstractions for users that don't get
| in the way, check it out:
|
| https://github.com/rogeriochaves/litechain
| ibains wrote:
| Why is this just not ETL, why do you need anything here? There
| is no new category or product needed here.
| jangletown wrote:
| Just saw the video you shared on the other comment using
| prophecy, very cool
|
| Generally I don't care much about the embedding and retrieval
| and connectors etc for playing with the LLMs, I imagined much
| more robust tools were available already indeed, my focus was
| more on the prompt development actually, connecting many
| prompts together for a better chain of thought kinda of
| thing, working out the memory and stateful parts of it and so
| on, and I think there might be a case for an "LLM framework"
| for that, and also a case for a small lib to solve it instead
| of an ETL cannon
|
| However, I am indeed not experienced with ETLs, have to play
| more with the available tools to see if and how can I do the
| things I was building using them
| teaearlgraycold wrote:
| I have done a bit of research trying to figure out why anyone
| would use langchain. The main two reasons I've found are these:
|
| 1. Newbies that want to play with LLMs don't know where to start
| or what the major building blocks even are. Despite the
| complaints here about documentation their getting started docs
| will walk you through the concepts. Going from total ignorance
| and confusion to now having a rough understanding of loading a
| prompt with chat history, using an embeddings database, calling a
| completions endpoint, etc. will make people feel accomplished.
| And then lang chain has earned some loyalty just because they
| were there for you first.
|
| 2. In the case that you don't know which embedding db, AI host,
| or model you want to use you can quickly swap those in and out
| and measure the results. That means there's little reason to
| complicate your back end code with lang chain (I've always just
| written my own abstraction layer to make this possible with very
| few lines of code). But for a python notebook it can make sense.
| solomonb wrote:
| LangChain is a perfect example of unnecessary abstraction. You
| could build a much simpler and more composable library for
| working with LLMs simply using functions and function
| composition. This isn't rocket science.
| m3kw9 wrote:
| Langchain is as pointless as one of those kitchen tools that
| specifically cuts egg into slices, but you could have done that
| with a knife.
| earleybird wrote:
| I like that analogy. I have one of those slicers and every time
| I use it I have two thoughts. First, this slices so quick and
| evenly. Second, there's a whole lot more I have to clean up
| compared to the knife.
| hamhamed wrote:
| I tried using langchain.js after a bit of hype and overwhelming
| dev support and incremental releases, but it became added
| complexity for no reason for our case.
|
| OpenAI's NPM is much easier to use. A lot of what langchain
| promised out of the box didn't work like caching. Anyways it's
| still beta i believe
| rgrieselhuber wrote:
| I've built a few LLM-based projects now and quickly discovered
| that Langchain was overkill (and not even very good overkill) for
| my use cases. I thought it was just me, glad to hear it's not.
| codeptualize wrote:
| I have looked for the value and never really found it.
|
| It seems to mostly be (bad) abstractions around things you could
| easily do without langchain.
|
| Take one of the most important llm things: prompt templates. What
| does langchain add over a simple function and an f string? Maybe
| I'm missing the point, but I can't find anything.
|
| Anyway, it seems people like it so who am I to judge, but I don't
| like making our codebase dependent on a huge new library with
| unnecessary abstractions and little or no value add.
| TeMPOraL wrote:
| > _Take one of the most important llm things: prompt templates.
| What does langchain add over a simple function and an f string?
| Maybe I 'm missing the point, but I can't find anything._
|
| Seconding.
|
| Ever since learning about it, then seeing a co-worker use it
| for some simple embedding job (and being impressed in how few
| lines of code it took, but that's actually _not_ thanks to
| LangChain), then reading its docs end-to-end, about once a
| week, I find myself going through the following sequence of
| thoughts:
|
| 1. Alright, let's set up LangChain and implement my ${most
| recent harebrained idea};
|
| 2. Oh, but it's in Python. I don't like Python, I don't know
| Python, I hate dealing with its dependency issues even more
| than with NPM ones. Could I do things I need from it directly
| in ${my preferred environment, which half the time is just
| Emacs}?
|
| 3. _Wait a minute_. Chaining "DAGs" the way it does is
| basically equivalent to a sequence of function calls in a while
| loop, occasionally mixed with some if/else or goto. Generating
| prompts is... string interpolation that can be wrapped in a
| helper function. LMAO.
|
| 4. No, really. Why bother? The only useful thing here seems to
| be _discoverability_ - i.e. a list of toolkits it supports, and
| said support working as intro 101 tutorial. Given the surface
| areas of those plugins are so small, I can literally wrap what
| I need in a bunch of functions, and then do the "chain" part
| as... plain old sequential code.
|
| So yeah, right now, I think about the only value this project
| has is in being a convenient list of AI tools with examples of
| using their APIs. Everything else seems better done either by
| coding it directly, or (for certain needs) by building up a
| more complex dataflow framework.
| rdli wrote:
| I found it helpful for prototyping and learning some basics,
| but I quickly found the abstractions were not useful and had to
| implement my own.
| davesque wrote:
| Honestly, it's hard to even tell if people really like it or if
| the LangChain team have just done a really good job of
| evangelizing for it. I saw that they did some kind of interview
| with Andrew Ng the other day. I feel like that sort of thing
| doesn't just happen by accident, but because someone actively
| reached out to him, especially considering that LangChain has
| only been on the scene for a couple of months.
| minimaxir wrote:
| LangChain has been active since last year (after the ReAct
| paper was released), but gained most of its popularity after
| the ChatGPT hype.
| codeptualize wrote:
| True, you gotta respect the marketing, they definitely got
| that covered and actually created some real value there, a
| $10M seed round to be precise.
|
| I think they got the timing right and the idea is kinda
| alluring as well: have everything standardized, pluggable and
| swappable sounds neat. Just doesn't really work.
|
| They are also doing a great job at maintaining the mirage of
| adding value. Templates for example will work totally fine,
| as they are just f strings. It seems there are plenty of
| people who don't really think twice about it, and they have
| captured that audience very well.
|
| There is also a low barrier to contribute as everything is
| kinda basic, that must help a lot as well.
| trolan wrote:
| I actually found a lot of value when I left the LangChain
| ecosystem and started using jinja templates. The syntax from
| home assistant moved over and I can just pass dictionaries to
| render a prompt now.
| saulpw wrote:
| I agree, and that's why I've been working on AIPL[0]. Our first
| v0.1 release should be in the next few days.
| https://github.com/saulpw/aipl
|
| It's basically just a simple scripting language with array
| semantics and inline prompt construction, and you can drop into
| Python any time you like.
| codeptualize wrote:
| With peace and love, why would I not just write some code?
|
| LLM pipelines are not very complex: it's string manipulation,
| api calls, and storage, there is not much more too it. All
| are quite easy to do, often needing nothing else but the
| standard library.
|
| For more complex cases or bigger scale you have a plethora of
| battle tested solutions to manage things like queues, back
| off and retries, concurrency, etc.
|
| Maybe it's me missing the point (again), but I wonder what
| the added value is of learning a new language to do things
| that are super easy to do in Python/JS/whatever my language
| of choice is? Or maybe I'm just not be your target audience,
| very possible.
| Reticularas wrote:
| I get this feeling too "maybe I'm not the target audience"
| This feeling is followed closely by "Who is the target
| audience?". Some abstract concept of an audience doing
| complex LLM work to accomplish... something?
|
| The value in statistics analysis for LLMs is clear, the
| value in chaining responses is very unclear.
| LASR wrote:
| The reason why Langchain is pointless is that it's trying to
| solve problems on top of technical foundations that just cannot
| support it.
|
| The #1 learning is that there is no reusability with the current
| generation of LLMs. We're using GPT-4 and 3.5T exclusively.
|
| Over the last several months, my team has been building several
| features using highly sophisticated LLM chains that do all manner
| of reasoning. The ultimate outputs are very human-like to the
| point where there is some private excitement that we've built an
| AGI.
|
| Each feature requires very custom handwritten prompts. Each step
| in the chain requires handwritten prompts. The input data has to
| be formatted a very specific way to generate good outputs for
| that feature/chain step. The part around setting up a DAG
| orchestration to run these chains is like 5% of the work. 95% is
| really just in the prompt tuning and data serialization formats.
|
| None of this stuff is reusable. Langchain is attempting to set up
| abstractions to reuse everything. But what we end up with a
| mediocre DAG framework where all the instructions/data passing
| through is just garbage. The longer the chain, the more garbage
| you find at the output.
|
| We briefly made our own internal Langchain. We tore it down now.
| Again not that our library or Langchain was bad engineering. It's
| just not feasible on top of the foundation models we have right
| now.
| applgo443 wrote:
| I saw your comment, got curious, and looked at a lot of your
| old comments. Lots of interesting insights - Thanks for sharing
| them.
|
| If you don't mind me asking, what do you do? I'm a researcher
| at FAANG working on language models and starting a new company
| in the space. Would love to connect. Feel free to email me -
| idyllic.bilges0p@icloud.com
| Der_Einzige wrote:
| Much of why this stuff is not reusable is that eventually
| someone in the NLP world is going to properly migrate the
| features for promopt engineering that the coomers over in
| stable-diffusion/automatic1111 land have "pioneered", such as
| token weighting, negative prompts, token averaging, or etc.
| Literally all of these techniques work with regular LLMs (if
| you don't believe me, see here: https://gist.github.com/Helliso
| therpeople/45c619ee22aac6865c...). NLP folks just haven't built
| the right tooling for it. Particularly sad since there's
| supposed to be an "Automatic1111 for LLMs" project called
| "Oogabooga" but it doesn't have any of the good features.
|
| The future of LLM prompting will involve highly specialized and
| engineered prompts, much as is the case with most images seen
| on civit.ai
|
| We are all likely to eventually throw away a lot of our current
| prompts
| 8f2ab37a-ed6c wrote:
| Any chance you might have shared some of these hard-earned
| lessons, so that the rest of us could learn from them as well?
| space_fountain wrote:
| How do you deal with the prompt iteration phase and how coupled
| is that to the DAG phase? I've only worked on a few proofs of
| concept in this phase, but a thing I struggled with was a
| strong desire to allow non technical colleagues to mess with
| the prompts. It wasn't clear to me how much the prompts need to
| evolve in tandem with the the DAG and how much they can exist
| separately
| LASR wrote:
| There are a few increasingly harder things when it comes to
| prompt customization:
|
| 1. Prompts ask LLM to generate input for the next step
|
| 2. Prompts ask LLM to generate instructions for the next step
|
| 3. Prompts ask LLM to generate the next step
|
| Doing #3 across multiple steps is the promise of Langchain,
| AutoGPT et al. Pretty much impossible to do with useful
| quality. Attempting to do #3 very often either ends up
| completing the chain too early, or just spinning in a loop.
| Not the kind of thing you can optimize iteratively to good
| enough quality at production scale. "Retry" as a user-facing
| operation is just stupid IMO. Either it works well, or we
| don't offer it as a feature.
|
| So we stopped doing 3 completely. The features now have a
| narrow usecase and a fully-defined DAG shape upfront. We feed
| some context on what all the steps are to every step, so it
| can understand the overall purpose.
|
| #2, we tune these prompts internally within the team. It's
| very sensitive to specific words. Even things like newlines
| affects quality too much.
|
| #1 - we've found it's doable for non-tech folks. In some of
| the features, we expose this to the user somewhat as
| additional context and mix that in with the pre-built
| instructions.
|
| So #2 is where it's both hard to get right and still
| solvable. Every prompt change has to be tested with a huge
| number of full-chain invocations on real input data before it
| can be accepted and stabilized. The evaluation of quality is
| all human, manual work. We tried some other semi-automated
| approaches, but just not feasible.
|
| All of this is why there is no way Langchain or anything like
| it is currently useful to built actually valuable user-facing
| features at production scale.
| teaearlgraycold wrote:
| 100% agreed. I've used GPT professionally and we would try out
| different hosts, AI21, etc. and it there were always clear
| quality issues with just re-using your prompt and
| hyperparameters. Some of that was down to other models being
| lesser quality, but we'd also need to re-tune prompts when
| upgrading to new OpenAI models for the best effect. It turns
| out that LLMs aren't quite a commodity.
| moneywoes wrote:
| In that case what pattern do you use for integrations?
| DrDroop wrote:
| const THE_LEFT_HANDED_IDEAL_FOR_THE_EVEN_NUMBERS_OF_THE_NATURAL_N
| UMBERS_UNDER_MULTIPLICATION = 2
| resiros wrote:
| I believe the abstractions in Langchain are inherently flawed.
| The core problem resides in the composability of chains. While it
| offers a handy way to create prototypes, it becomes restricting
| when you desire to modify a specific element within the chain.
| The hierarchical design of chains in Langchain conceals the
| component you wish to alter and obscures the parts developers
| might want to adjust, making the process of experimenting and
| refining the pipeline difficult.
|
| The optimal abstraction for LLM apps, in my view, should resemble
| a DAG or a state machine. This alternative exposes the distinct
| stages in the pipeline rather than masking them in a hierarchy.
| Yes, adopting this new abstraction might lead to more code but it
| offers superior control. It's hardly surprising that many users
| start prototyping with Langchain, but then, when ready, they
| clone the prompts and construct their own systems.
|
| Fixing this fundamental issue would be very difficult. It would
| necessitate reworking the library from the ground up.
| joshka wrote:
| Take a look at https://promptfile.org/ for an alternative
| approach to the prototyping -> app flow.
| mpaepper wrote:
| I agree it's not so easy to work with Langchain.
|
| That's wy I built the simple LLM agents repo which you can
| understand in 5 minutes:
|
| https://github.com/mpaepper/llm_agents
| krychu wrote:
| Using an LLM framework at this moment doesn't make sense and can
| be damaging, in my humble opinion. Ways to extract value from
| LLMs are in early exploration stage. Look at research in
| prompting: chain of thought, react, reflection, tree of thoughts,
| zero vs few hot etc. Then completion vs conversational
| interfacing. Then memory management via vector databases and
| prompt expansion vs compression vs progressive summarization etc.
| All these are fairly recent developments. They are not
| abstractions worth cementing, this is _search_ and _creative_
| phase. LLMs threw everything in the air, but the dust is far from
| settling. I think it's important to recognize the phase we're in
| and pick your weapon accordingly. You have to stay nimble and
| light, ready to experiment with a new idea that will come out
| next week. You should be hacking these things together by
| yourself. If you pick a framework at this stage know that the
| framework will have to pay the price of trying to cement things
| in the times of storm. And you'll be a few steps behind. Of
| course this is my personal take.
| Der_Einzige wrote:
| Gosh you are so fking wrong it's sad.
|
| Microsoft Guidance or LMQL prove you wrong a million times
| over.
|
| LLM frameworks are awesome.
| simonw wrote:
| Can you back that up?
|
| I've looked at the documentation for both Microsoft Guidance
| and LMQL and they look like LangChain to me: frameworks where
| I'd have to spend a whole lot more time learning how to use
| them than if I just imported the OpenAI Python library
| directly and started running prompts through my own thin set
| of custom functions.
| sandkoan wrote:
| Wholeheartedly agree--it adds layers of bloat and abstraction
| between you and the actual ReAct pattern, which can be trivially
| implemented in maybe 50 sloc.
| namuol wrote:
| I tried to improve the type coverage of the JS package and I came
| to a lot of the same conclusions. It feels like a lot of
| unnecessary and poorly conceived indirection and abstraction.
| There's basically just a lot more to learn especially if you're
| just getting started.
|
| It's maybe useful as a repository of prompting techniques, but I
| found myself constantly monitoring the actual prompt text being
| generated rather than reading the code that produces it...
| jdprice wrote:
| Completely agree. I had this exact thought last week. It's been
| helpful to go through their repo to see how they've done certain
| things, but it's a textbook leaky abstraction.
| cloudking wrote:
| What business problems are you solving with Langchain and LLMs?
| hcks wrote:
| It's LLM wrappers all the way down
| wahnfrieden wrote:
| Reselling access to Langchain and LLMs
| shon wrote:
| Lol
| cloudking wrote:
| Perfect.. I've seen so many demos showing people feeding data
| to LLMs so they can "ask questions about their data" but
| still not seen any real business use cases. Is anyone using
| these tools in production for a real problem?
| ramoz wrote:
| The enterprise search domain has evolved quite a bit,
| utilizing & integrating retrieval-rerank architectures for
| a few years now. That includes embedding/vector storage, as
| well as LLM usage (eg using Google T5 for query
| understanding and model rerank)... there's some generative
| modeling but not really like GPT... more like structuring
| an abstract tree from a complex Boolean query, and sending
| that tree into an custom retrieval approach over ANN &
| rerank ensemble.
|
| If you're doing that type of work in a serious manner you
| don't use libraries like langchain or lamaindex. They are a
| bit late/irrelevant to the engineering that exists in that
| type of environment.
| sandGorgon wrote:
| I think that Generative AI applications is a config management
| problem. Think Prompts X Chains X LLMs. Your prompts wont work
| across everything and everything will break on model change.
| Coding this into ur classes is what everyone does.
|
| I think the better answer is to _declaratively_ pull out the
| prompts X chains as jsonnet code. Call it trauma & learnings
| from the K8s/Borg world. We have formats that have evolved as a
| result of millions of lines of code wrangling
| clusters/terraform/etc - so we decided to build a SDK over it.
|
| that is what we did here -
| https://github.com/arakoodev/EdgeChains/releases/tag/0.2.0
|
| EdgeChains is basically Generative AI prompt engineering modeled
| as config management.
| Der_Einzige wrote:
| This guy's analysis is terrible. I literally built a retrieval
| augmented generation with memory in langchain last night
| (comments claim you can't do this)
|
| Sure, langchain has poor documentation and useless helper
| functions. That doesn't mean that it's pointless.
| risyachka wrote:
| The point is in 99% of use cases no one will ever swap OpenAI
| or vector storage db for anything else. All Langchain does in
| these cases is introduces useless abstraction that takes longer
| to implement and makes things less transparent.
|
| And when you need some customization instead of taking you 10
| minutes it takes an hour to work around this abstraction.
| chpmrc wrote:
| Could you please share the code? I tried doing the same and,
| like the people in that thread, could not. TIA!
| tehsauce wrote:
| Just fyi, if you use a standard knn library like faiss and
| pretty much any embedding + language model raw from huggingface
| or an API, it will require ~15 lines of code to do what you
| describe. I'm not sure how much shorter langchain made your
| implementation but I can't imagine it saving too much.
| CGamesPlay wrote:
| Langchain was useful to me personally for two reasons: using
| their prompt templates as a starting point for my own, and seeing
| how their "tools" were built to learn about good Python libraries
| to build my own tools with.
|
| Viewed through this lens, LangChain was more a "sample codebase"
| than a library for me, and it was reasonably good for that.
| hospitalJail wrote:
| Does it introduce more lines of code? Yes
|
| Does it introduce features that you don't need to implement
| yourself? Yes
|
| Does it make it easy to drop OpenAI? Yes
|
| What's the problem?
| teaearlgraycold wrote:
| You'll still need to re-tune your prompt and the
| hyperparameters when switching models. So the actual effort of
| switching models is not improved much if at all.
| redox99 wrote:
| Hopefully as LLMs get smarter, you'll need less prompt
| engineering and stuff will "just work" across models.
| celestialcheese wrote:
| This. I'll still use langchain for token/cost counting and
| some nice abstractions on top of the LLM, and the document
| loader system is semi-useful, but all of the retriever/chain
| stuff abstracts away the most important part - the prompt.
| zoba wrote:
| I want an off the shelf "talk to PDFs" solution and was hopeful
| langchain was it.
| ufo wrote:
| What is langchain?
| noman-land wrote:
| I pasted your question directly into DDG and the entire first
| page has the answer.
|
| https://duckduckgo.com/?q=What+is+langchain
| szopa wrote:
| At one point I got so frustrated with langchain that I wrote my
| own stuff in go: https://github.com/ryszard/agency
| ravenstine wrote:
| I had the same thought about Langchain, and it's essentially my
| same criticism of most wrapper libraries and SDKs. What are they
| actually doing that can't be just as easily done with straight up
| string manipulation and HTTP requests? Usually very little. ORMs
| might be one of the few exceptions in some cases.
|
| Even so, ORMs sell a similar false promise to Langchain, which is
| that you can "easily" swap out the underlying thing; a migration
| that is rare in practice and almost always not that simple.
|
| You might not need Langchain.
| refulgentis wrote:
| It's a really interesting question: the converse of this is its
| _really_ tricky nailing all this down, much harder than you'd
| think -- see the GPT4 GA thread earlier this week for people
| who swear up and down the OpenAI API acts in bizarre ways,
| you'll note that it almost seems fantastical that it could be
| that bizarre: it is fantastical. If you're truly used to it, it
| reads like "hey I have auto retries enabled in my HTTP client
| framework. p.s. whats 'context size'?"
|
| But Langchain is far, far, _far_ away from being truly helpful
| with that. It's glorious that we're 5 months into GPT-4 and
| most people either got bored or are building on rickety rushed
| structures in Python to rush out a proof of concept web app.
| version_five wrote:
| I don't know enough to agree that it's pointless, but I'd agree
| that when I looked at it I saw a lot of abstraction of already
| simple stuff (like the examples the post gives) and decided that
| for what I was doing it would be faster and easier to understand
| to just write my own python script. Though I can picture for very
| inexperienced developers the abstractions may be helpful short
| term?
| paulgb wrote:
| I had a similar experience; looked into using it for something,
| and felt that it would be easier to recreate the things I would
| use langchain for in Jinja2, than it would be to reshape my
| code to the interface that langchain wanted.
| crosen99 wrote:
| I keep going back to LangChain thinking it just hasn't found its
| legs yet, but every time I do I retreat exasperated. I don't find
| their abstractions useful or intuitive, and their documentation
| is woefully scattered and incomplete. Things are moving so
| quickly with LLMs that theirs is no easy task, but so far they
| haven't really cracked the nut of making LLM app development
| easier.
| senko wrote:
| Remember, this is the project that raised ~$30m from Benchmark
| and Sequoia.
|
| There was a controversial "quality doesn't matter for software
| products" post and discussion[0] here on HN a few days ago and
| this is a beautiful example.
|
| Product may matter eventually, but you can sure surf the hype for
| a long time before the reckoning comes (and if you're lucky, you
| may even be able to get someone else to hold the bag then).
|
| 0: https://news.ycombinator.com/item?id=36615286
| RC_ITR wrote:
| We've seen the top dog fall from grace in the VC world before
| (KPCB missing social and mobile after a truly epic run in the
| 90s)
|
| Is that what we are starting to see for Sequoia?
|
| Like KP, I doubt they'll fold, but it would be interesting to
| live in a world where sequoia is just top half instead of top
| 1.
| nextworddev wrote:
| Any ideas what valuation range that might imply?
| PhoenixReborn wrote:
| The rumored valuation was around $200M, and the product is
| pre-revenue right now. So that seems pretty ridiculous.
| ShamelessC wrote:
| I'm curious - who gets screwed over the most here? Is it
| investors who got tricked into over valuing? Or langchain
| who now can't meet their expected revenue targets and will
| be forced to pivot?
|
| Speaking in hypothetical terms of course. I'm assuming the
| langchain folks are probably paying themselves pretty well
| and not working super hard (at least not on engineering
| stuff)?
| robbywashere_ wrote:
| Sad to hear the news, I have 7 years of langchain experience on
| my resume, now what am I going to do!?
| [deleted]
| beepbooptheory wrote:
| You should probably not mention it by name on a resume anyway,
| but emphasize your actual skills, experience, and familiarity
| with the lower level technologies and APIs that underly it.
|
| This hype train isn't going to last forever, and it's probably
| better to advertise sustainable, evergreen skills rather than
| being an expert in the flavor of the week. With that much
| experience, you shouldn't have to feel tied to any one
| company's software.
| skwirl wrote:
| Whooosh
| ta988 wrote:
| It was a joke, langchain was released last year.
| beepbooptheory wrote:
| Ha! I guess I don't really get the joke, but relieved for
| gp's sake it is one, despite the egg on my face.
| pessimizer wrote:
| * _didn 't_ really get the joke,
|
| If you don't get it now, I'm worried about you:)
| beepbooptheory wrote:
| No no, that is the right tense! I thought initially it
| was like light sarcasm targeting the conceit of the
| reddit post, i.e., "how could it be pointless if I have
| amassed all this experience." Beyond that.. not sure (I'm
| not that smart).
|
| _Perhaps_ its just something like: "the scandal that
| this library is bullshit amounts to not a lot considering
| it's still pretty new thing." But the, erm, strong
| showing of downvotes of my original post makes me think
| its a better bit than that!
|
| Indulge me with the joke if you'd like, but don't worry
| about me! I'm doing pretty good, despite my slow mind.
| drdrey wrote:
| The context for the joke is that some job listings have
| inflated requirements, like N years of experience from
| framework/language X that realistically nobody has (or in
| this case, can have)
| moneywoes wrote:
| Didn't they raise at a $200 million valuation
| arpowers wrote:
| Proper term is that it's a "false abstraction"...
|
| It abstracts some work only to introduce introduce it's own API
| (which is ultimately more complicated, less documented,
| introduces limits and constraints, and comes with its own bugs
| and dev politics)...
|
| But nobody asked me, keep using it
| joshka wrote:
| To coin a portmanteau: that's the Fauxcade pattern ;)
|
| (though it does exist already)
| https://www.urbandictionary.com/define.php?term=fauxcade
| jpulec wrote:
| I've been using the JS version of langchain for a few months now,
| and despite there being a lot of valid criticism, (especially
| around the abstractions it provides) I'm still glad to be using
| it.
|
| We get the benefits of a well used library, which means making
| certain changes is easy. For example, swapping our vector
| database was a one line change, as was swapping our cache
| provider. When OpenAI released GPT-4, we were able to change one
| parameter, and everything still just worked.
|
| Sure, it's moving fast, and could use a lot better documentation.
| At this point, I've probably read the entire source code several
| times over. But when we start testing performance of different
| models, or decide that we need to add persistent replayability to
| chains of LLM calls, it should be pretty easy. These things
| matter to production applications.
| [deleted]
| mmq wrote:
| > When OpenAI released GPT-4, we were able to change one
| parameter, and everything still just worked.
|
| Wouldn't that be the same if you used the OAI js library
| directly? Basically swapping the model parameter?
| jpulec wrote:
| It's not. The API is different, since GPT-4 is a chat based
| model, and davinci isn't. It's not a huge difference, but
| these little sort of things add up.
| mmq wrote:
| I see, thought you were using GPT-3.5 and moved to GPT-4.
| minimaxir wrote:
| I have a full-on "The Problem With LangChain" blog post in the
| pipeline, and the reason I made a simple alternative
| (https://news.ycombinator.com/item?id=36393782) because I spent a
| month working with LangChain and coming to the conclusion that
| it's just easier to make my own Python package than it is to hack
| LangChain to fit my needs.
|
| A few bullet points:
|
| - LangChain encourages tool lock-in for little developer benefit,
| as noted in the OP. There is no inherent advantage into using
| them, and some have suboptimal implementations.
|
| - The current implementations of the ReAct workflow and prompt
| engineering are based on InstructGPT (text-davinci-003), and are
| extremely out of date compared to what you can do with
| ChatGPT/GPT-4.
|
| - Debugging a LangChain error is near impossible, even with
| verbose=True.
|
| - If you need _anything_ outside the workflows in the
| documentation, it 's extremely difficult to hack, even with
| Custom Agents.
|
| - The documentation is missing a lot of relevant detail (e.g. the
| difference between Agent types) that you have to go diving into
| the codebase for.
|
| - The extreme popularity of LangChain is warping the entire AI
| ecosystem around the workflows to the point of harming it. Recent
| releases by Hugging Face and OpenAI recontextualize themselves
| around LangChain's "it's just magic AI" to the point of hurting
| development and code clarity.
|
| Part of the reason I'm hesitant to release said blog post is
| because I don't want to be _that_ asshole who criticizes open
| source software that 's operating in good faith.
| ibains wrote:
| It is pointless - LlamaIndex and LangChain are re-inventing ETL -
| why use them when you have robust technology already?
|
| 1. You ETL your documents into a vector database - you run this
| pipeline everyday to keep it up to date. You can run scalable,
| robust pipelines on Spark for this.
|
| 2. You have a streaming inference pipeline that has components
| that make API calls (agents) and between them transform data.
| This is Spark streaming.
|
| Prophecy is working with large enterprises to implement
| generative AI use cases, but they don't talk so much on HN.
|
| Here's our talk from Data+AI Summit: Build a Generative AI App on
| Enterprise Data in 13 Minutes
| https://www.youtube.com/watch?v=1exLfT-b-GM
|
| Here's a blog/demo https://www.prophecy.io/blog/prophecy-
| generative-ai-platform...
| lmeyerov wrote:
| We also do platform & customer work there (cool pipelines to
| feed louie.ai or real-time headless versions), and agreed those
| pipelines have simple uses of LLM where langchain is mostly
| useful just for a vendor neutrality. Think BYO LLM as it is now
| a zoo. Basically apache nifi or spark streaming with simple LLM
| & vector DB call outs. Our harder work here is more at the data
| engineering level.
|
| But....a lot of our louie.ai work happens for less trivial
| scenarios where it isn't just the ETL NLP 2.0 tier . That logic
| is much more complicated, so structured programming
| abstractions matter a LOT more for AI-style business logic.
| Think talk to your data and generate on-the-fly analytics
| pushdown with an interactive data viz UI. That's.. a lot of
| code.
| techwizrd wrote:
| I agree that it's a little silly, but I mostly use it to
| abstract over BYO LLMs and extract information from
| documents. It's nice to be able to quickly prototype
| something and swap out the underlying language model than set
| up a whole pipeline with Apache Tika, ETL, etc. Once the idea
| is feasible, then sure.
|
| That said, langchain is really inefficient and I often find I
| can re-implement the pieces I need much faster than dealing
| with langchain's bugs and performance issues.
| ibains wrote:
| That's assuming you're not using low-code. There are
| inbuilt connectors to read data, transform data, read/write
| to pinecone, make api calls to LLMs. It is much faster to
| prototype with Prophecy.io
| [deleted]
| siva7 wrote:
| It's easy to hate on langchain. Sure, the architecture isn't the
| best but it's where the crowd is so it's a safe bet
| tikhonj wrote:
| That just makes it _easier_ to hate. Our collective tendency to
| build atop mediocre foundations just because they got popular
| first is _exhausting_.
| PaulHoule wrote:
| Every system I've seen for managing this kind of system has
| flaws, including the ones that I have written.
|
| For instance scikit-learn implements excellent algorithms for
| model selection that would apply, in principle, to a model based
| on huggingface transformers that might take 2 hours to train. skl
| is a fast machine if memory fits in RAM on a single computer, but
| it is not up to task for multiple computers or anything mortal to
| a single process such as the computer bring turned off.
|
| HF has model selection algorithms too, but not as nice. They
| don't take the same kind of datasets as all so it would be a
| hassle to import my ski models into HF.
|
| I have to be able to compare models generated with any kind of
| tools so I think I will build a universal model selection
| framework (builds and test models) but then you run into the
| problems langchain did where there is a lot of structure imposed
| and all sorts of quirks and performance losses because of that
| structure.
|
| For instance my current skl selector wastes a lot of resources
| computing stuff from scratch over and over again and if the code
| were properly organized it could get the job done 3 times faster
| but the same trick wouldn't work for every other experiment I
| might want to do.
|
| So we are all running into hurdles and finding ways to jump over
| them, making a lot of mistakes because we are in a rush and don't
| know better yet.
| revskill wrote:
| I don't trust them, because they never talk about their
| "tradeoffs". Every software has its tradeoff.
| politelemon wrote:
| I've rarely ever seen software talk about its tradeoffs. They
| want you to use the software.
|
| K8s and react are wildly popular, huge tradeoffs, but mention
| none of them.
| shermix011 wrote:
| DuckDB does on their page. https://duckdb.org/
| handonam wrote:
| the consistency and conventionality of the methods in the
| langchain library are very sporadic, imo
| dereg wrote:
| Since last year, before I heard about langchain, I've been
| building my own stack of tooling for my own LLM projects that
| probably now covers about 10-20% of Langchain's functionality. I
| heard about Langchain earlier this year and groaned, thinking
| that I did a lot of work for nothing..
|
| ..Then I actually _used_ langchain. I was shocked at how poorly
| performant the code is. Some operations took 10x longer than how
| I did it, and all the while producing worse results. As tempting
| as it is to just roll with langchain from day one, I 'd highly
| advise against it. Think deeply about what you're actually trying
| to accomplish and instead of just injecting langchain in the
| middle of everything as this messy, amorphous glue code thing.
| FemmeAndroid wrote:
| I had this exact same experience. I was happy to move to
| something good, but I couldn't find a lot of benefit. Maybe I'm
| missing something, but the added complexity is not worth it to
| me for what it provides.
| [deleted]
| stan_kirdey wrote:
| there is an alternative that is production-grade - deepset
| haystack https://haystack.deepset.ai/
|
| p.s. i am contributor so there could be bias
| [deleted]
___________________________________________________________________
(page generated 2023-07-08 23:01 UTC)