(C) Alec Muffett's DropSafe blog.
Author Name: Alec Muffett
This story was originally published on allecmuffett.com. [1]
License: CC-BY-SA 3.0.[2]
 

LLM hallucinations predicted with new algorithm, open-sourced

2025-09 

‘LLM hallucinations aren't bugs, they're compression artefacts’, writes Leon Chlon, PhD. It’s really good to see this intelligent, considered response to so called AI ‘hallucinations’ - a word that is not particularly useful and arguably has helped prompt a moral panic about generative AI techniques. Chlon and colleagues offer a paper on how they can predict ‘hallucinations’ before they happen. The breakthrough is straightforward enough when you appreciate the maths. As they explain: ‘We proved hallucinations occur when information budgets fall below mathematical thresholds’. Hence you look for those thresholds. To explain more, they use the analogy of Zip compression: ‘When your LLM confidently states that "Napoleon won the Battle of Waterloo," it's not broken. It's doing exactly what it was trained to do: compress the entire internet into model weights, then decompress on demand. Sometimes, there isn't enough information to perfectly reconstruct rare facts, so it fills gaps with statistically plausible but wrong content. Think of it like a ZIP file corrupted during compression. The decompression algorithm still runs, but outputs garbage where data was lost.’ (Leon Chlon, PhD et al.) As an aside the reference to Zip compression can be contested with Ted Chiang’s The New Yorker article, ‘ChatGPT Is a Blurry JPEG of the Web’. I don’t necessarily agree with Chiang’s metaphor (cf. a book due out soon, Vector Media, provides a good critique). Nonetheless, it can be helpful to consider these various analogies to get us back to the actual data science of AI, not to be drawn into hype and mystification. #OpenSource: It’s great to see Leon Chlon, PhD et al. have not only taken on the subject of AI hallucinations with rigour, they have made their Hallucination Risk Calculator open source. ‘The era of "trust me, bro" AI is ending,’ they write, ‘Welcome to bounded, predictable AI reliability’. #AI #MachineLearning #ResponsibleAI #LLM #Innovation
[END]

[1] URL: https://www.linkedin.com/posts/leochlon_ai-machinelearning-responsibleai-activity-7368294551867858947-cUgm
[2] URL: https://creativecommons.org/licenses/by-sa/3.0/

DropSafe Blog via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/alecmuffett/