Post ASejU2S32OuvH95aWe by xian@xoxo.zone
(DIR) More posts by xian@xoxo.zone
(DIR) Post #ASedaqtpmnfBkZx4L2 by simon@fedi.simonwillison.net
2023-02-14T01:11:52Z
0 likes, 0 repeats
If you want to convince yourself that getting large langauge models to stick to the "truth" is a viciously difficult problem, ask one of them to write a fictional story for you (they're really good at this)Then ask yourself how something capable of doing that could be engineered to be able to distinguish fact from fiction
(DIR) Post #ASedmO3Sr7CtlUxM7E by simon@fedi.simonwillison.net
2023-02-14T01:13:14Z
0 likes, 0 repeats
If you did have a language model that could only output "truth" would it even be useful?Many of the tasks we ask of them - summarization, explaining concepts in different terms, brainstorming new ideas - would stop working if there was a hard "truth" requirement to everything they output
(DIR) Post #ASedyU1ySleTm41kv2 by xian@xoxo.zone
2023-02-14T01:16:00Z
0 likes, 0 repeats
@simon i don't really follow in that if they can follow instructions to write fiction it would suggest they could be told to avoid doing that.
(DIR) Post #ASeeAWnxyVIjqwDcIK by simon@fedi.simonwillison.net
2023-02-14T01:16:22Z
0 likes, 0 repeats
Something I'm struggling with right now is that I /really/ want a product that delivers on the promise of the new Bing search: an AI I can talk to which runs searches for me and summarizes the results to answer my questionsBut I'm not at all convinced the current generation of langauge models are capable of doing that in a useful way - despite the fact that they can do a VERY convincing imitation of it
(DIR) Post #ASeeMBD65aP4LxWshk by xian@xoxo.zone
2023-02-14T01:16:48Z
0 likes, 0 repeats
@simon does this just sort of mean it's actually orders of magnitude harder to do what the LLMs are pretending to do now? (give valuable answers to questions)
(DIR) Post #ASeeYeBkAtcjhbqDPk by zubakskees@mastodon.social
2023-02-14T01:17:00Z
0 likes, 0 repeats
@simon 🤔
(DIR) Post #ASeej0V64cZNnXpjiC by simon@fedi.simonwillison.net
2023-02-14T01:17:23Z
0 likes, 0 repeats
@xian how can they know the difference between fact and fiction?It's all just "how can I best complete this sentence to look statistically similar to my training set?"
(DIR) Post #ASeevJEUFne1MyvkFk by simon@fedi.simonwillison.net
2023-02-14T01:19:01Z
0 likes, 0 repeats
@xian I'm beginning to suspect so - but I'm really just thinking out loud hereDelivering on the promise feels like it will take a whole lot more than just a language model, no matter how big they getMaybe this needs full AGI, whatever that turns out to mean?
(DIR) Post #ASef6SwKVGIKf3l11c by danyork@mastodon.social
2023-02-14T01:20:10Z
0 likes, 0 repeats
@simon I want that, too! But yes, these current models are NOT there yet!
(DIR) Post #ASefHXddNIXkQ1IHlQ by ids1024@fosstodon.org
2023-02-14T01:21:56Z
0 likes, 0 repeats
@simon I don't know the specific technological innovations behind these large language models (vs something simple like an n-gram markov model, which I do understand). But I guess by definition a generative language model is system that produces text that "look like" the kind of text being modeled?Given this it's impressive how often they're accurate, but it's not clear if something producing only truth would be a better language model or something wholly different.
(DIR) Post #ASefHZYkERowNR1PQ8 by ids1024@fosstodon.org
2023-02-14T01:30:14Z
0 likes, 0 repeats
@simon In the earlier days of AI research it was assumed that a grandmaster level chess AI would require a high level of general intelligence. But that turned out to be wrong. It just beat chess a different way from the humans.LLMs look like they're close to replacing various human jobs, but it might turn out that this is an illusion, and getting from something that seems "close" to actually being acceptably accurate requires an entirely different approach that we are nowhere near developing.
(DIR) Post #ASefUMvWVkECximIWO by stevelord@bladerunner.social
2023-02-14T01:27:51Z
0 likes, 0 repeats
@simon isn't that the crux though? The big search people see this idea of an agi displacing them, and it scares them. It's a fear of being the next Yahoo! or AOL. So they must intermediate instead of add value. If they indexed results, then gave you the best results with decent summaries and additional info it'd work well, but that's not what they're geared up for.
(DIR) Post #ASefg8I4koQX2lw6Ns by zzzeek@fosstodon.org
2023-02-14T01:30:30Z
0 likes, 0 repeats
@simon AI that folds laundry. this is much more important to me. maybe the AI can put little ads in amongst my socks , that way tech companies would care about this use case?
(DIR) Post #ASefuj7OcscTJitqsq by J12t@social.coop
2023-02-14T01:34:56Z
0 likes, 0 repeats
@simon Those language models are missing context. Is something intended to be the well-established truth, a conjecture, straight fiction, disagreed between experts or perhaps something that was thought to be true for a long time and then it was proven to be wrong. Not sure how a "language model" could even capture this.
(DIR) Post #ASeg6V4SgC1dE18oeu by reconbot@toot.cafe
2023-02-14T01:35:12Z
0 likes, 0 repeats
@simon it's going to be rough but I'm for the growing pains as I don't use bing anyway 😂
(DIR) Post #ASegHn2YD8QO8ndArg by JigenD@mastodon.social
2023-02-14T01:35:43Z
0 likes, 0 repeats
@simon I seriously doubt it can actually be accomplished with transformer based LLMs for live searched information.Maybe someone should develop a 'fact' from 'fiction' unsupervised AI and plug that into the web search side of things.
(DIR) Post #ASeggjfJtY2gujexvs by nevali@troet.cafe
2023-02-14T01:39:30Z
0 likes, 0 repeats
@simon biological brains are… architected (have evolved) with "reasoning" and "language" and "memory" as distinct regions and it feels like GPT models are an attempt at creating language+memory simulators with no space for reasoning to exist at all
(DIR) Post #ASeggmoeBKHmgX2xO4 by nevali@troet.cafe
2023-02-14T01:42:44Z
0 likes, 0 repeats
@simon i do think something close to these models will be immensely powerful for human-computer interaction one day, and possibly even quite soon, but at the moment it's spaghetti sliding down a wall: without specially-designed training material (costs $$$ + time) and without a place to insert hard logic, it feels like a tech demo gone rogue
(DIR) Post #ASeggoMMRSqdTNpOZk by nevali@troet.cafe
2023-02-14T01:45:58Z
0 likes, 0 repeats
@simon i would LOVE, for example, it if we could get to a place where metadata bundled with software describes what it does (i.e., its commands) and an ML model that has been tuned for the user and speaks their language perfectly were able to provide a user interface, in a generic and open-ended/standard kinda way
(DIR) Post #ASegr8Zc8tj53ox15U by simon@fedi.simonwillison.net
2023-02-14T01:42:20Z
0 likes, 0 repeats
@nevali yeah my current hunch is that a useful version of what Bing is trying to do will have a LLM as a component of it, but another more important component will be some other form of "AI" that hasn't come together just yetBut the pace this field moves at means I wouldn't be surprised if whatever that is happens in the next few months... or takes another decade or more
(DIR) Post #ASeiqJqKxLsHdEVhw0 by dio@mastodon.online
2023-02-14T02:10:41Z
0 likes, 0 repeats
@simon Why do you think these things would stop working if there was a hard truth Requirement period from my understanding all that would change Would be The citations offered and the confidence it's given to you the statements at hand.
(DIR) Post #ASej4rwuwn5hSqWhQ8 by simon@fedi.simonwillison.net
2023-02-14T02:13:22Z
0 likes, 0 repeats
@dio I was hinting at the idea that if you insist on "truth" then no summary could ever be true, because it lies through omission of detailsLikewise, I often ask ChatGPT things like "explain the Lagrangian equation for the standard model using analogies to the Muppets" - but that's asking it to invent untrue scenarios involving the Muppets
(DIR) Post #ASejU2S32OuvH95aWe by xian@xoxo.zone
2023-02-14T02:17:41Z
0 likes, 0 repeats
@simon well, exactly — it doesn’t even have a model of truth as far as I can imagine
(DIR) Post #ASeji3EGwVF5k4KMCW by xian@xoxo.zone
2023-02-14T02:19:56Z
0 likes, 0 repeats
@simon a smart friend of mine (@Ted_Nadeau) pointed out that chatGPT while just parrots things about math right now, teaching it actual math would be one of the easier things to fix, but i suspect you run short of things like that quickly
(DIR) Post #ASekv2qdlrHuHL28wq by mikejackmin@liberdon.com
2023-02-14T02:33:50Z
0 likes, 0 repeats
@simon As best I can tell, the word truth means something like "is consistent with other things that we regard as truth".Get serious about that and you end up here:https://en.wikipedia.org/wiki/M%C3%BCnchhausen_trilemma
(DIR) Post #ASelTn4yOIcbVYJ0L2 by dio@mastodon.online
2023-02-14T02:40:20Z
0 likes, 0 repeats
@simon there might be a better way t9 engage with truth here. We can say a summary is truthful when it gives the correct amount of partial information suiting a context. You don't need to provide all details in a summary for it to be true. A summary is a summary because it omits.
(DIR) Post #ASem1KMZfB0SIBWuuW by simon@fedi.simonwillison.net
2023-02-14T02:46:20Z
0 likes, 0 repeats
@xian @Ted_Nadeau you can solve math to a certain extent by giving it the ability to call out to another engine that runs calculations for it - similar to how Bing can run searches and then perform further actions based on the results
(DIR) Post #ASemcyGEHskkkOVo3s by josephholsten@mstdn.social
2023-02-14T02:52:52Z
0 likes, 0 repeats
@simon hardly! We have RDF, prolog, dependency resolvers, every manager of graph analysis and coq to verify them, plenty of other systems which are perfectly suited to truth. All are classically called AIs. People don’t want truth. Truth is too hard for humans.
(DIR) Post #ASewcztjCSJlMlde4m by Andres@mastodon.hardcoredevs.com
2023-02-14T04:44:54Z
0 likes, 0 repeats
@simon I really see ChatGPT and Bing results failing at basic things like logic. For me it's an interesting way to describe and create text from ideas.I like much better the solution developed by perplexity.ai where it shows you a resume of their findings with references to the websites.
(DIR) Post #ASex4LQzEh3pylEdk0 by simon@fedi.simonwillison.net
2023-02-14T04:50:06Z
0 likes, 0 repeats
@Andres I didn't think Bing and perplexity were very different from each other - they both run searches and use the search result summaries to try and answer questions, with inline citations
(DIR) Post #ASexKqfuqpqrVJ5VXU by Andres@mastodon.hardcoredevs.com
2023-02-14T04:52:51Z
0 likes, 0 repeats
@simon So the images that I saw were fake news.
(DIR) Post #ASexbPrRnYIjGTNib2 by simon@fedi.simonwillison.net
2023-02-14T04:55:59Z
0 likes, 0 repeats
@Andres a bunch of Bing screenshots having been doing the rounds today that are actually "old Bing" - not the new waitlist-protected thing that uses the new chatbot thingHere's what the new thing looks like: https://www.theverge.com/2023/2/7/23587454/microsoft-bing-edge-chatgpt-ai
(DIR) Post #ASexp9p6DufNZupsDw by simon@fedi.simonwillison.net
2023-02-14T04:57:38Z
0 likes, 0 repeats
This post here is an example of a screenshot of old Bing https://hachyderm.io/@tehstu/109861147970437158
(DIR) Post #ASey0sjvZ5V44P0Vay by Andres@mastodon.hardcoredevs.com
2023-02-14T04:58:17Z
0 likes, 0 repeats
@simon Nice! thanks for the link.
(DIR) Post #ASf0S6sHJvEddR1ZKK by samueljohn@mastodon.world
2023-02-14T05:27:57Z
0 likes, 0 repeats
Good point, @simon. We used to think we need to start from logical rules as first principle to reach intelligent truth following AI. The idea that computers don't make mistakes. Like Data in #startrek. The perfect thinking machine not a able to tell a joke. We wanted AI to overcome the limitation that our brains are neither understanding logic nor stochastic very well are prone to false memory and all kind of illusions. And we have to educate our self to follow logic and rational thinking.
(DIR) Post #ASfGOmdrgbu9Hbib9k by codepope@mastodon.org.uk
2023-02-14T08:26:29Z
0 likes, 0 repeats
@simon i suspect we want it to be able to establish what a fact is and to resolve questions about facts with answers based on that establishment. These current models though are models about language, not models about knowledge.