[HN Gopher] Zep: Fast, scalable building blocks for production L...
       ___________________________________________________________________
        
       Zep: Fast, scalable building blocks for production LLM apps
        
       Author : roseway4
       Score  : 86 points
       Date   : 2023-09-22 12:02 UTC (10 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | fuddle wrote:
       | The video in the github repo is very helpful. I think it needs an
       | example of what can be built with Zep though, not just the
       | features.
        
         | roseway4 wrote:
         | Thanks. I definitely want to make a longer video demonstrating
         | Zep end-to-end.
        
       | verdverm wrote:
       | Is this building blocks or a platform?
       | 
       | When someone tells me "building block" I would expect to be able
       | to use just one of the pieces in my application without a lot of
       | extra fuss.
       | 
       | This looks way more like a framework and platform, with multiple
       | deployments needed to just get started, and a cloud offering in
       | the works.
       | 
       | If this is a platform, what makes it different from all the other
       | platforms? (besides prefixing the library functions with Zep,
       | like ZepReader and ZepStorage)
        
       | jondwillis wrote:
       | Convince me to not just use Postgres for everything!
        
       | osener wrote:
       | Interesting project, thanks for submitting it!
       | 
       | Obviously not commenting on this project as I've yet to read up
       | on it, but what are your proven to be useful building blocks for
       | LLM apps? This space is super new and everyone is experimenting,
       | promoting their solutions and trying to build
       | communities/stargazers/customers around them. It is getting hard
       | to distinguish what's just fluff and what is an actual
       | improvement over raw OpenAI APIs.
       | 
       | Even the more popular projects such as LangChain and TypeChat are
       | a bit hard to rate regarding whether they're actually quality
       | higher level libraries worth adopting or if I should just read
       | their source as inspiration and build my API calls myself.
       | 
       | I would really appreciate suggestions!
        
         | roseway4 wrote:
         | Zep author here. LangChain is a great framework and has a very
         | broad and active ecosystem. Many of LangChain's core
         | components, such as chat history "memory," history
         | summarization, entity extraction, vector search, and more,
         | don't scale well in production. They operate in memory and
         | often synchronously within the chat loop, resulting in poor
         | user experiences and limiting deployment options.
         | 
         | You can use third-party ecosystem integrations with external
         | SQL databases, vector databases, etc to fix some of these
         | issues. This requires some know-how, infrastructure, and time.
         | Zep's building blocks are turn-key solutions to these
         | challenges and offered in a single service.
         | 
         | The Zep GitHub project, website, and demo video provide a good
         | overview of the project's functionality and how the service
         | solves these issues.
         | 
         | This blog post on the LangChain website offers some benchmark
         | data using Zep vs LangChain's core memory components:
         | https://blog.langchain.dev/zep-x-langchain-slow-chatbots/
        
           | esafak wrote:
           | Since you are targeting developers I would highlight the
           | differentiating _features_ : "database-backed", and
           | "asynchronous", or whatever they are. This is more
           | informative than "production-grade", "fast", or "scalable",
           | which anyone can claim. In fact, the competitors have a
           | better claim to being fast -- they're in memory! -- so I
           | would drop them unless I had benchmarks. Something I would
           | also encourage you to work on.
           | 
           | Finally, call out your competitors, and explain your value
           | add for each one. Maybe in a table. Why not just use a vector
           | database and hand roll the rest, for example?
           | 
           | Good luck!
        
             | roseway4 wrote:
             | Thanks for the advice! Some benchmarks here:
             | https://blog.langchain.dev/zep-x-langchain-slow-chatbots/
        
               | verdverm wrote:
               | That's not really benchmarks. To me, it just shows why
               | putting things like langchain in the mix makes your life
               | harder in the long run. Having to add more tools is not
               | an ideal solution. What people want are actual "building
               | blocks" that don't require adopting a framework
        
         | tinyhouse wrote:
         | Great question. IMHO it's better to write something yourself.
         | These things are not complicated and you can have better
         | control and understanding by writing it yourself. Once the
         | industry settles on a set of tools, you can start thinking of
         | migrating if needed.
        
           | roseway4 wrote:
           | Zep has Python and TypeScript packages for teams building
           | without LangChain: https://docs.getzep.com/sdk/
        
       ___________________________________________________________________
       (page generated 2023-09-22 23:01 UTC)