[HN Gopher] Structured Output with LangChain and Llamafile
___________________________________________________________________
Structured Output with LangChain and Llamafile
Author : brakmic
Score : 41 points
Date : 2025-06-22 17:01 UTC (4 days ago)
(HTM) web link (blog.brakmic.com)
(TXT) w3m dump (blog.brakmic.com)
| dcreater wrote:
| People still use langchain?
| owebmaster wrote:
| No
| anshumankmr wrote:
| Its good for quickly developing something but for production, I
| do not think so.We used it for a RAG application I built last
| year with a client, ended up removing it piece by piece, and
| found our app responded faster.
|
| But orgs think its some sort of flagbearer of LLMs.As I am
| interviewing for other roles now, HRs from other companies
| still ask for how many years of exp I have with Langchain and
| Agentic AI.
| zingababba wrote:
| What should be used instead?
| Hugsun wrote:
| I gave up after it didn't let me see the prompt that went
| into the LLM, without using their proprietary service. I'd
| recommend just using the API directly. They're very simple.
| There might be some simpler wrapper library if you want all
| the providers and can't be bothered to implement the support
| for each. Vercel's ai-sdk seems decent for JS.
| halyconWays wrote:
| >I gave up after it didn't let me see the prompt that went
| into the LLM, without using their proprietary service.
|
| Haha, really?
| ebonnafoux wrote:
| httpx to make the call yourself, or if you really want a
| wrapper the openAI python https://github.com/openai/openai-
| python.
| Jimmc414 wrote:
| pydanticai, dspy or deal directly with the provider sdks
| codestank wrote:
| i do because i don't know any better since i'm new to the AI
| space.
| nilamo wrote:
| My experience, as someone who is also new and trying to
| figure things out, is that langchain works great as long as
| everything you want to do has an adapter. Try to step off the
| path, and things get really complex really fast. After
| hitting that several times, I've found it's easier to just do
| things directly instead of trying to figure out the langchain
| way of doing things.
|
| I've found dspy to work closer to how I think, which has made
| working with pipelines so much easier for me.
| screye wrote:
| It is useful if you keep swapping things out. Langchain's
| wrappers stay stable and up-to-date because of their
| popularity. In production, it's ideal startups that undergo a
| lot of flux.
|
| I would suggest against using their orchestration tooling, DSLs
| or default prompts. Those components are either underbaked or
| require deep adoption in a way that is harder to strip out
| later.
|
| We change models, providers and search tooling quite often.
| Having consistent interfaces helps speed things up and reduce
| legacy buildup. Their stream callbacks, function calling
| integration, RAG primitives and logging solutions are nice.
|
| One way of another, it is useful to have a langchain-like
| solution for these needs. Pydanticai + logfire seems like a
| better version of what I like about langchain. Haven't tried
| it, but I bet it's good.
| reedlaw wrote:
| The use case in the article is relatively simple. For more
| complex structures, BAML (https://www.boundaryml.com/) is a
| better option.
| pcwelder wrote:
| ```
|
| try: answer = chain.invoke(question)
| # print(answer) # raw JSON output
| display_answer(answer)
|
| except Exception as e: print(f"An error
| occurred: {e}") chain_no_parser = prompt | llm
| raw_output = chain_no_parser.invoke(question)
| print(f"Raw output:\n\n{raw_output}")
|
| ```
|
| Wait, are you calling LLM again if parsing fails just to get what
| LLM has sent to you already?
|
| The whole thing is not difficult to do if you directly call API
| without Lang chain, it'd also help you avoid such inefficiency.
| moribunda wrote:
| I don't get the langchain hate, but I agree that this "blog
| post" is bad.
|
| Langchain has a way to return raw output, aside "with
| structured output":
| https://python.langchain.com/docs/how_to/structured_output/#...
|
| It's pretty common to use a cheaper model to fix these errors
| to match the schema if it fails with a tool call.
| crystal_revenge wrote:
| > It's pretty common to use a cheaper model to fix these
| errors to match the schema if it fails with a tool call.
|
| This has not be true for a while.
|
| For open models there's 0 need for these kind of hacks with
| libraries like Xgrammar and Outlines (and several others)
| both existing as a solution on their own and being used by a
| wide range of open source tools to ensure structured
| generation happens at the logit levels. There's no-need to
| add multiples to your inference cost, when in some cases
| (xgrammar) they can _reduce_ inference cost.
|
| For proprietary models more and more providers are using
| proper structured generation (i.e. constrained decoding)
| under-the-hood. Most notably OpenAI's current version of
| structure outputs makes use of logit based methods to
| guarantee the structure of the output.
| Hugsun wrote:
| The version of llama.cpp that Llamafile uses supports structured
| outputs. Don't waste your time with bloat like langchain.
|
| Think about why langchain has dozens of adapters that are all
| targeting services that describe themselves as OAI compatible,
| Llamafile included.
|
| I'd bet you could point some of them at Llamafile and get
| structured outputs.
|
| Note that they can be made 100% reliable when done properly.
| They're not done properly in this article.
| halyconWays wrote:
| >Don't waste your time with bloat like langchain.
|
| Amen. See also: "Langchain is Pointless"
| https://news.ycombinator.com/item?id=36645575
| kristjansson wrote:
| It's right there. In the screenshot in the blog post. Grammar >
| 'JSON Schema + Convert'. That's what structured output is.
|
| ... it's going to be september forever, isn't it?
___________________________________________________________________
(page generated 2025-06-26 23:01 UTC)