[HN Gopher] Open Source LLMOps Stack
       ___________________________________________________________________
        
       Open Source LLMOps Stack
        
       Some background: I work on Langfuse and we've been collaborating
       with LiteLLM.  (LiteLLM is a Python library and proxy/gateway that
       handles cost management, virtual keys, caching, and rate-limiting
       for OpenAI or other LLM APIs. Langfuse manages LLM tracing,
       evaluation, prompt management, and experiments.)  We've each been
       building our open-source projects since early 2023 and learned that
       many devs and especially platform teams use the two together, so we
       created an integrated "OSS LLMOps stack."  This is a fully self-
       hostable, technology-agnostic setup that lets you (1) Use LLMs via
       a standardized interface without adding complexity to the
       application; (2) Keep LLM Tracing, Evaluation, Prompt Management
       in-house for compliance; (3) Track cost and usage via a single
       interface, create virtual API keys for attribution of costs  It
       also enables direct transfer of LLM traces from the LiteLLM proxy
       to Langfuse. This simplifies the rollout of LLMOps practices
       (observability and evaluations) across multiple projects--you don't
       need to instrument all applications.  Additionally, the LiteLLM
       proxy can fetch and cache prompts from Langfuse's prompt management
       system, using them as templates for requests made through the
       proxy.  Both of these workflows can function without the
       integration, but are easier to manage with it!  We'd love your
       feedback!
        
       Author : marcklingen
       Score  : 11 points
       Date   : 2025-02-26 09:41 UTC (13 hours ago)
        
 (HTM) web link (oss-llmops-stack.com)
 (TXT) w3m dump (oss-llmops-stack.com)
        
       | [deleted]
        
       | mrbonner wrote:
       | It was just Ops, then DevOps, MLOps, AIOps and now LLMOps. I lost
       | count of how many promote cases I reviewed over the last 10 years
       | with those "Ops" as the sole reason to promote.
        
         | remram wrote:
         | Oh man, they went crazier than that with "secops", "itops",
         | "finops", "dataops". Every Foozler person is now "doing
         | FooOps", every dev is "devops".
         | 
         | Titles have never been about clarity or distinctiveness but how
         | cool that sounds, and apparently this is the fashion. Nevermind
         | that "ops" used to stand for something. Me wants shiny word.
        
       ___________________________________________________________________
       (page generated 2025-02-26 23:00 UTC)