[HN Gopher] LLM Hallucinations: why they occur and how to avoid ...
___________________________________________________________________
LLM Hallucinations: why they occur and how to avoid them
Author : ofermend
Score : 19 points
Date : 2023-05-02 20:50 UTC (2 hours ago)
(HTM) web link (vectara.com)
(TXT) w3m dump (vectara.com)
| yawnxyz wrote:
| That's a lot of words to say "we feed it up to date SERP results"
| mdp2021 wrote:
| > _When the language model's predictions contradict our
| expectations, experiences or prior knowledge, or when we find
| counter-factual evidence to that response (sequence of predicted
| tokens) - that's when we find hallucinations_
|
| No. Hallucination is any idea which was not assessed for truth.
| Whatever statement is not put over the "testing table" and
| analyzed foundationally counts as hallucination.
| valine wrote:
| The proposed solution to eliminate hallucination is to ground the
| model with external data. This is the approach taken with Bing
| Chat, and while it kinda works, it doesn't play to the strengths
| of LLMs. Every time Bing Chat searches something for me I can't
| help but feel like I could have written a better search query
| myself. It feels like a clumsy summarization wrapper around
| traditional search, not a revolutionary new way of parsing
| information.
|
| Conversing with an LLM on subjects that it's well trained on,
| however, absolutely does feel like a revolutionary new way of
| parsing information. In my opinion we should be researching ways
| to fix hallucinations in the base model, not papering over it by
| augmenting the context window.
| og_kalu wrote:
| Fixing hallucinations from the source will be a tough one. The
| root of the issue is that loss doesn't discriminate. A probable
| guess will lower loss much better than "I don't know" or
| whatever equivalent. Educated guessing becomes an essential
| skill the model learns during initial training.
|
| So the objective function encourages it. But also the dataset
| encourages it as well. There will be many many sentences that
| can't be completed accurate to source even with all the
| knowledge and understanding in the world. Many completions will
| have numerous sensible options. The dataset doesn't
| discriminate. Fiction, Fact, Opinion. All the same. All given
| equal weight.
| roywiggins wrote:
| Even Bing AI seems to sometimes get confused and fail to
| summarize results properly, as when seems to have parsed a
| debunking of a ChatGPT hallucination but still spit out the
| hallucination again:
|
| https://twitter.com/WillOremus/status/1643692259332743171
| electric_mayhem wrote:
| Tl;dr: I think that a left brain/right brain parallel is in the
| cards for LLMs
|
| Our own brains have multiple neural circuits. Parallel, serial,
| competing, cumulative...
|
| Including ones which error-check others' output.
|
| I guess the term in the ML niche is 'adversarial networks'.
|
| From the robotics side, there was subsumption architecture
| which used the real world as a basis for informing decisions.
|
| So, I respectfully disagree that fixing up creative but
| occasionally erroneous networks is papering over the problem.
|
| If our own brains use multiple neural circuits to check each
| other to try to optimize for output that agrees with reality,
| why would it be preferable or logical to try to create a single
| artificial neural network that has perfect output?
|
| My sense from playing with LLMs is they're knowledge without
| understanding. Weirdly akin to my own dreaming consciousness,
| to the extent that conscious me has seen it.
|
| It seems very natural to me that the next step would be
| integrating checks/validations in parallel and considering each
| half of a matched 'whole'.
|
| It could still be considered a single network. Just made of two
| distinct smaller specialized ones.
| valine wrote:
| Having a second adversarial network could absolutely be part
| of the solution, I wasn't arguing against that. My problem is
| with in context learning, ie the idea that including
| background information in the prompt can solve hallucination.
| electric_mayhem wrote:
| I see where you're coming from, but I've had a couple
| occasions over the years that have led me to expect and
| accept that at any given time my brain harbors an abundance
| of different ideas and not all of them are valid.
|
| Like the time I decided to sit on the ceiling to spite a
| friend who was being pushy.
|
| Made perfect sense, and I was about to. Until a different
| part kicked in and assured the whole of myself that it
| wasn't possible.
|
| Neural networks are going to spawn occasional nonsense.
| Aiming for perfection is noble, but nature shows a
| different path that already works.
|
| But fundamentally I think we're agreeing- just the twist is
| I'm saying LLMs should be an aspect of a whole rather than
| seen as a solitary solution to be perfected.
| cyanydeez wrote:
| The problem is you're proposing, basically to implement a
| "ASKJEEVES" model for Wikipedia, with a unknown concentration.
|
| That'd cost real money, requires real licensing, and a clear
| business plan.
|
| Absolutely it'll happen, but almost every sector will need a
| leader for X subject.
| valine wrote:
| Relying on subject matter experts isn't a scalable solution.
| I would speculate that LLMs don't hallucinate when a fact is
| included a handful of times in the training set. I'd also
| speculate that our current LLMs can say whether a piece of
| text looks like a fact.
|
| I imagine the long term solution to hallucination will look
| something like this: Loop over the entire training set with
| the latest GPT and build a list of "facts", then train a new
| LLM using this new dataset to predict both the probability of
| the next token, and the probability that the next token is
| factual.
| redskyluan wrote:
| I would vote for finetuning, prompt engineering, rather than only
| add domain specific knowledge. Others are playing detective
| ,digging into the ethical conundrums of AI-generated content.
| eskibars wrote:
| I know the article doesn't go into this particular element, but I
| do wonder how much opportunity is still in front of us for
| adversarial LLM systems that try to detect/control for
| hallucinations. I'm pretty excited by the research in LLM
| explainability and quantitative measures on how accurate
| generative LLMs are measured
|
| (Full disclosure: I work at Vectara, where this blog was
| published)
| cornhole34 wrote:
| Please do not anthropomorphize LLM.
| eskibars wrote:
| I wasn't, and I'm not sure how you got that out of what I
| said. I'm not claiming "understanding," "sentience," etc.
|
| I'm claiming there's a great deal of work in the realm of
| research that I'm excited by: research I expect to be done by
| humans. I do expect/am supposing that result may be LLMs of a
| different nature: to apply guard rails around generative LLM
| systems, but that's not to anthropomorphize them: just to
| suppose their purpose.
|
| The term "hallucination" does anthropomorphize LLMs, but I
| think that's now accepted nomenclature in the industry, at
| least for the time being, and it's helpful to have some
| standard nomenclature to describe some of the benefits and
| problems.
___________________________________________________________________
(page generated 2023-05-02 23:01 UTC)