Post ArQuXjCmD58VXLgFfs by marshray@infosec.exchange
(DIR) More posts by marshray@infosec.exchange
(DIR) Post #ArQZDdMq6g2ubPxr4S by futurebird@sauropods.win
2025-02-24T01:01:11Z
0 likes, 0 repeats
"Chat GPT told me that it *can't* alter its data set but it did say it could simulate what it would be like if it altered it's data set"NO. It has no idea if it's telling the truth or not and when it says "I can simulate what this would be like" This guy is pretty sharp about philosophy but people really really really do not *get* how this works."Chat GPT told me this is what it did"No! It told you what you thought it should say if your asked it what it did!https://www.youtube.com/watch?v=160F8F8mXlo
(DIR) Post #ArQZIWqllV1ZpzIXI0 by futurebird@sauropods.win
2025-02-24T01:02:03Z
0 likes, 0 repeats
Half a million views and from a day ago. Good lord.
(DIR) Post #ArQZbwXaTL4n19aDY0 by Htaggert@mstdn.social
2025-02-24T01:05:29Z
0 likes, 0 repeats
@futurebird it seriously just makes up stuff.
(DIR) Post #ArQZfuEH0ZnP0URGb2 by futurebird@sauropods.win
2025-02-24T01:06:18Z
0 likes, 1 repeats
Chat GPT will say things if you ask it what it did, the answers will be similar to texts it has processed about describing how things are done, some in the context of describing how a computer program might do something. It might even give you a good run down of how LLMs work mixed in there, it might not. But it's not able to ... interrogate it's own process this simply isn't possible. It's not part of how it's designed.
(DIR) Post #ArQZtDLvDf7A57MWYa by futurebird@sauropods.win
2025-02-24T01:08:37Z
0 likes, 1 repeats
@FediThing I think people are in denial about the level of brute force operating here. There is a reason it uses so much electricity. If you make big enough tables you can make plausible text about *anything* But it's just ... plausible text that seems like the kind of answer people would expect.
(DIR) Post #ArQZvoebVMioLgLDWq by marshray@infosec.exchange
2025-02-24T01:09:00Z
0 likes, 0 repeats
@futurebird I can’t invest 20 minutes to find out if he gets there, but last I checked, ChatGPT was just generating a text prompt which was then thrown over the wall to a completely disconnected image generation model (Dall-E), then declaring success without basis.It wasn’t very good at image prompting either.
(DIR) Post #ArQa3ii5UIeqENjuzI by futurebird@sauropods.win
2025-02-24T01:10:37Z
0 likes, 0 repeats
@marshray He's more interested in the philosophy which is interesting but that bit of the video made me think "no you don't know how this thing really works... you can't just ask it how it did things and take those answers seriously."
(DIR) Post #ArQaAykSFxvw02JA9Y by futurebird@sauropods.win
2025-02-24T01:11:54Z
0 likes, 0 repeats
When photography was invented many people thought it might capture the human soul... and that's about how this will sound some day.
(DIR) Post #ArQaLyMy0ONU0gZ2EC by futurebird@sauropods.win
2025-02-24T01:13:53Z
0 likes, 0 repeats
@FediThing Well the video started out probing the limitations and I thought that was interesting. But, then there is this huge foundational error in what an LLM *is* ... it doesn't reflect, it has no logic, it produces responses that are the best possible fit for the context. That is all.
(DIR) Post #ArQaOCgZkU3mzSndx2 by futurebird@sauropods.win
2025-02-24T01:14:17Z
0 likes, 0 repeats
@katow LOL.
(DIR) Post #ArQaUemQCwPRrQbxb6 by davep@infosec.exchange
2025-02-24T01:15:28Z
0 likes, 0 repeats
@futurebird "It provided a statistically probable stream of tokens" just doesn't have the same ring
(DIR) Post #ArQawHsgwGG8wvieh6 by futurebird@sauropods.win
2025-02-24T01:20:20Z
0 likes, 0 repeats
@katow Can those areas tell you how they process language?
(DIR) Post #ArQb2uIP1iRNpOtygy by trochee@dair-community.social
2025-02-24T01:21:35Z
0 likes, 0 repeats
@futurebird An awful lot of "the ChatGPT told me to leave my wife" fear-mongering is going to look like the (semi apocryphal) stories of people panicking over movies of a train coming towards them.
(DIR) Post #ArQbBZtWFQCrA1OhbE by futurebird@sauropods.win
2025-02-24T01:23:13Z
0 likes, 0 repeats
@cykonot I've never seen him before. I wish he'd explain more about what Hume was going on about. That part seemed more informative but then what I know about philosophy is limited.
(DIR) Post #ArQbMBeihtLBAfXAsy by futurebird@sauropods.win
2025-02-24T01:25:08Z
0 likes, 1 repeats
@cykonot I don't know why but people just asking chat GPT how it works and taking the answers as good enough drives me nuts.Maybe because when I tell a doctor I'm in pain *I* might not be believed, but a bunch of matrices producing a response maximized for "meeting the expectations of the asker?" let's treat that like it's gospel.LORDY.
(DIR) Post #ArQbcg75EbHcGafFz6 by andrew_deridder@mastodon.social
2025-02-24T01:28:05Z
0 likes, 0 repeats
@futurebird I had the same thought too
(DIR) Post #ArQbweYsiH7JFJBwBs by futurebird@sauropods.win
2025-02-24T01:31:40Z
0 likes, 0 repeats
@vashti @FediThing Very artful weighting, with extra hints for almost every contingency. Maybe it would help to think of it as a mirror that tries to show you what you want based on a vast sea of data about the kinds of things people consider sensible, reasonable in context responses?I've played with little LLM models and there is nothing in how you set them up that ought to make you think they could give meaningful responses to these kinds of questions. NOTHING.
(DIR) Post #ArQce0SRBD7nbiqkKm by marshray@infosec.exchange
2025-02-24T01:39:33Z
0 likes, 1 repeats
@futurebird Not at face value, no.But it can actually be pretty good at metacognition, much better than your average human, once it’s pointed in the right direction.Since current AI’s assist with the training of next generation AI’s, I think there’s a high likelihood of a positive feedback cycle. I don’t think anyone knows where the limit is.
(DIR) Post #ArQdAaZ0TSd0r3SW1I by cratermoon@zirk.us
2025-02-24T01:45:25Z
0 likes, 0 repeats
@futurebird Pareidolia as a Service: https://www.emptywheel.net/2025/02/21/what-we-talk-about-when-we-talk-about-ai-part-two/
(DIR) Post #ArQdU5KyHalfndBsUi by cxxvii@aus.social
2025-02-24T01:48:56Z
0 likes, 1 repeats
@futurebird The thing everyone should remember is that neural networks are sometimes good at 'doing' the thing they were trained to do, but the GPTs of the world were not trained to produce correct output, they were trained to produce output that is convincing. If they convince you, be waryI once saw someone who asked their ostensibly locally hosted llm if it was on their machine, and it said no, and they believed it :blobcatfacepalm:
(DIR) Post #ArQdpchTR309M0zCPg by futurebird@sauropods.win
2025-02-24T01:52:52Z
0 likes, 1 repeats
@marshray When you say "a positive feedback cycle" ...towards what? What is the feedback and what is it going towards positively?
(DIR) Post #ArQdzuRwIXfDIacnmC by nazokiyoubinbou@urusai.social
2025-02-24T01:54:29Z
0 likes, 1 repeats
@cxxvii @futurebird Well, to be clear, the issue isn't the training. A lot of stuff is thrown into the latest training methods to try to make them more accurate. The much more fundamental problem isn't the training, but the actual mechanism itself which -- no matter how good or accurate the training -- simply can't reliably produce the correct output.
(DIR) Post #ArQeIQT1zoetoPxsrQ by futurebird@sauropods.win
2025-02-24T01:58:05Z
0 likes, 1 repeats
@nazokiyoubinbou @cxxvii Thank you.It's like making a machine designed to show people objects that look just like airplanes... and expecting those planes to have engines and be able fly. But, you never set out to make a program to design machines for flight. Just a program that would show people photos, videos, plans, descriptions that match their *expectations* of airplanes.
(DIR) Post #ArQePr2jaOXeGMCtNI by nazokiyoubinbou@urusai.social
2025-02-24T01:59:24Z
0 likes, 0 repeats
@futurebird @cxxvii Right. I make this differentiation because A. I want to be clear that no matter how they might advertise this or that method is more accurate, it will always fail due to the underlying issue and B. some people think the tech just isn't fully developed, but its underlying mechanism can NEVER improve without changing to something else entirely.
(DIR) Post #ArQevqle97jOftTGBk by futurebird@sauropods.win
2025-02-24T02:05:11Z
0 likes, 1 repeats
@CptSuperlative @emilymbender I have found one use case. Although, I wonder if it's cost effective. Give an LLM a bunch of scientific papers and ask for a summary. It makes a kind of nice summary to help you decide what order to read the papers in. It's also OK at low stakes language translation. I also tried to ask it for a vocabulary list for the papers. Some of it was good but it had a lot of serious but subtile and hard to catch errors. It's kind of like a gaussian blur for text.
(DIR) Post #ArQf1FSPmGOmhG38Do by trochee@dair-community.social
2025-02-24T02:06:10Z
0 likes, 0 repeats
@marshray And also> it can actually be pretty good at metacognition, much better than your average humanwarrants a giant [citation needed] flag, because the point @futurebird is making is that they are not even doing cognition, let alone metacognition. At best, they emit word sequences that are shaped sorta like the word sequences that come out when metacognition happens in an actually cogitating thing like a person
(DIR) Post #ArQfgdeAkITZanHknY by DaveRussell@stranger.social
2025-02-24T02:13:25Z
0 likes, 0 repeats
@futurebird @cykonot Hume is an interesting read. He had an elegant way of explaining his theories. Oddly, while he questioned religious authorities and was skeptical of God's existence, he went to church every Sunday. Sorry. That's all I remember off the top of my head.
(DIR) Post #ArQgrnoHOrlyFgtsJ6 by futurebird@sauropods.win
2025-02-24T02:26:53Z
0 likes, 0 repeats
@DaveRussell @cykonot That was about what I remember from the 101 course I took. He didn't seem to be interested in burying you in terminology. But, that also seemed to place some limitations on how much depth or detail he could apply. But, at least he'd admit when he was speculating which not all of the writers we read would do.
(DIR) Post #ArQh74FGZnSLZKnmi0 by futurebird@sauropods.win
2025-02-24T02:29:37Z
0 likes, 1 repeats
@FediThing "Maybe they don't want to admit to themselves they are fooled?"I really starting to wonder if this is a big part of why these explanations just don't land. If it makes anyone feel better I was fooled by some AI music. I thought it was something a person made never even considered that it might be generated. There isn't anything wrong with being "fooled" -- but, treating these tools like they can do things that they can't do is just a recipe for learning nothing.
(DIR) Post #ArQhPn7yRYT7yvJr1s by marshray@infosec.exchange
2025-02-24T02:28:23Z
0 likes, 0 repeats
@AT1ST @futurebird “It’s not actually doing X, it’s just generating text that’s indistinguishable from doing X.”is classic human cope.The ability to produce text as it “should look like” is a thing that humans get accredited degrees and good paying jobs for demonstrating.Good enough is almost always better than perfect, and 80% is usually good enough.
(DIR) Post #ArQhPoBYVivbGK0EFc by futurebird@sauropods.win
2025-02-24T02:33:01Z
0 likes, 0 repeats
@marshray @AT1ST I do agree that some aspects of language might work in that way. But, when you respond to these posts is that what *you* are doing? Producing the best possible most plausible response for the context?
(DIR) Post #ArQhyly2kUJ6WEM3BQ by futurebird@sauropods.win
2025-02-24T02:39:21Z
0 likes, 0 repeats
@dalias Maybe but that still implies some kind of organization of concepts beyond just through language or the shape of their output.I don't see any reason why it should be impossible to design a program with concepts, that could do something like reasoning ... you might even use an LLM to make the output more human readable. Though I guess this metaphor works in that to the extent there is a "goal" it's to "make it pass" rather than to convey any idea or express anything.
(DIR) Post #ArQi89zwlxdzefimx6 by marshray@infosec.exchange
2025-02-24T02:41:02Z
0 likes, 0 repeats
@futurebird @AT1ST The temptation to do that is great.I try to recognize when I’m posting reflexively and not hit ‘Publish’, because it feels like those posts are largely not adding value.
(DIR) Post #ArQiGMqesbeXpGI3do by marshray@infosec.exchange
2025-02-24T02:37:05Z
0 likes, 0 repeats
@trochee @futurebird OK, so you just “emitted a word sequence shaped sorta like the word sequences that comes out when metacognition happens.”And since it’s an argument we’ve all seen before, I can just dismiss your “word sequence” out-of-hand.See how that works?
(DIR) Post #ArQiGNhpguBgUBAWWW by futurebird@sauropods.win
2025-02-24T02:42:31Z
0 likes, 0 repeats
@marshray @trochee That's why I didn't ask you how my mind works. I asked you if that was how *your* mind works. Is it how your mind works?
(DIR) Post #ArQiNpecBA7rBzDTeK by trochee@dair-community.social
2025-02-24T02:42:16Z
0 likes, 1 repeats
@marshrayDo not try the Chinese Room gambit with me, for I was there when the deep magic was written @futurebird
(DIR) Post #ArQiW7mAgba1iGhlmy by DamonWakes@mastodon.sdf.org
2025-02-24T02:41:13Z
0 likes, 1 repeats
@marshray @AT1ST @futurebird This is a little like claiming that MENACE actually understands how to play noughts and crosses because it behaves in a way that's indistinguishable from understanding how to play noughts and crosses: https://en.m.wikipedia.org/wiki/Matchbox_Educable_Noughts_and_Crosses_EngineDoes the distinction matter during a game? No. Does that mean it doesn't matter at all? Absolutely not.
(DIR) Post #ArQifT8scGmCaMma4O by chillybot@infosec.exchange
2025-02-24T02:47:03Z
0 likes, 0 repeats
@futurebirdAuto correct on steroids.@FediThing
(DIR) Post #ArQiuPmFPSToUcK5zc by tuban_muzuru@ohai.social
2025-02-24T02:49:43Z
0 likes, 0 repeats
@futurebird @marshray @trochee I do wish philosophy would get off its dead ass and provide us with a cogent vocabulary for what's going on in the machines.The closest to a competent philosopher is WVO Quine and his school. Google "gavagai" and the indeterminacy of translation. There are at least a dozen ridiculous attempts to Explain AI and they're all worthless. Our understanding of reality is always relative to a background theory
(DIR) Post #ArQj1rk5plGjFirtmC by futurebird@sauropods.win
2025-02-24T02:51:07Z
0 likes, 0 repeats
@nazokiyoubinbou @cxxvii Yeah that's the wild bit. Someone runs over and says "we built they plane it designed and it worked!"That could also happen. But it doesn't change the fact that the design is a product of a process that has encoded nothing about what an airplane is, why you'd want to make one etc. If the designs work (as translation often works OK with LLMs) that shows that it's possible to get the output without the logical framework. If that makes any sense.
(DIR) Post #ArQj90xXxoFlrjZOka by kate@aus.social
2025-02-24T02:52:21Z
0 likes, 1 repeats
@futurebird @FediThing I was fooled by an AI book, a brand new publication on something I know about, by an author I hadn’t heard of, recommended to me by an algorithm. About halfway through reading it I thought: I don’t feel there is human intelligence behind this. There’s something off. But it was such a weird feeling I dismissed this.I keep it on my desk as a visual prompt.
(DIR) Post #ArQjqCs9Xz9xGVdu8u by marshray@infosec.exchange
2025-02-24T03:00:12Z
0 likes, 0 repeats
@futurebird @trochee Most of the time, yes, the word sequences I emit draw heavily from the random corpus of text on which I was trained.All of the time? I don’t know.I certainly don’t want to think so. I’m going around begging people to give me a solid argument.How could I tell?
(DIR) Post #ArQjsiU6z6cZt7auKO by guyjantic@infosec.exchange
2025-02-24T03:00:40Z
0 likes, 0 repeats
@futurebird it seems very difficult for most people not to personify LLMs.
(DIR) Post #ArQk6M3RAzxkNNcYCm by futurebird@sauropods.win
2025-02-24T03:03:07Z
0 likes, 0 repeats
@bjc @cxxvii @nazokiyoubinbou The framework is only available to a system that is able to read texts and extract and match their embedded frameworks. I would be excited to see more work on such systems.If the output of an LLM manages to be readable as having a coherent framework that's a function of the consistency and cross-compatibility of the training set. (works well with translating languages... but will it work with designing a plane? Less likely.)
(DIR) Post #ArQkAlO6vfiA6YOPpY by escarpment@mastodon.online
2025-02-24T03:03:53Z
0 likes, 0 repeats
@futurebird @CptSuperlative @emilymbender I have found a ton of use cases.I asked ChatGPT what to have for dinner and it reminded me of mezze / Mediterranean style food, so that's what I bought at the grocery store and ate.I asked ChatGPT to generate the syntax for a PySpark query and it reminds me of how to do distinct() and the syntax for filter.I asked it how to fix a Linux WiFi issue and it gave me the modprobe command to unload and reload the WiFi module.
(DIR) Post #ArQl32gCr2btN7jsiO by gretared@sfba.social
2025-02-24T03:13:42Z
0 likes, 0 repeats
@futurebird I have never used any of that nonsense but is it always that obsequious? “You’re right, let me try that again” starts to seem real passive aggressive real quick.
(DIR) Post #ArQliXPVVH5GYQiNPM by nazokiyoubinbou@urusai.social
2025-02-24T03:21:13Z
0 likes, 0 repeats
@futurebird @CptSuperlative @emilymbender Summaries aren't reliable either.There are indeed use-cases. But every single one of them comes with caveats. And, I mean, to be fair, most "quick" methods of doing anything come with caveats. It's just that people forget those caveats.
(DIR) Post #ArQm0RWe7cKxW0akds by futurebird@sauropods.win
2025-02-24T03:24:28Z
0 likes, 1 repeats
@nazokiyoubinbou @CptSuperlative @emilymbender I don't want to be so dismissive that people who are finding uses for this tech won't pay attention to the important points about the limitations. People *are* using this tech, some heavily, many probably in ways with pitfalls we won't see the worst results of until it's too late. Saying "it's just fancy autocomplete" is basically true, but many people think "but autocomplete can't do --" So, I really try to find ways to "get" this new tech.
(DIR) Post #ArQmDGpbC6sA70Zc3c by willowwren@hcommons.social
2025-02-24T03:26:43Z
0 likes, 0 repeats
@futurebird The ELIZA effect in full force. This is why I try to tell both students and faculty to stop using anthropomorphic terms. It's not "saying," "thinking," "trying"... it's just giving you a statistical output, and often a pretty poor one. A "spreadsheet," like you said; that's a good description. And I'm not even in technology, but I do know language. And a fair bit about how psychology and con artistry work too. We need more people explaining that the danger of AI is not machine intelligence but the human impulse to attribute meaning to meaningless things, like the complete nonexistence of machine intelligence.
(DIR) Post #ArQmFsXRyEc3ddKPGC by nazokiyoubinbou@urusai.social
2025-02-24T03:27:15Z
0 likes, 0 repeats
@futurebird @CptSuperlative @emilymbender To be clear on this, I'm one of the people actually using it -- though I'll be the first to admit that my uses aren't particularly vital or great. And I've seen a few other truly viable uses. I think my favorite was one where someone set it up to roleplay as the super of their facility so they could come up with arguments against anything the super might try to use to avoid fixing something, lol.I just feel like it's always important to add that reminder "by the way, you can't 100% trust what it says" for anything where accuracy actually matters (such as summaries) because they work in such a way that people do legitimately forget this.
(DIR) Post #ArQmOouIqE4OywHTnM by futurebird@sauropods.win
2025-02-24T03:28:53Z
0 likes, 1 repeats
@nazokiyoubinbou @CptSuperlative @emilymbender If I don't have the experience of "finding it useful" I can't possibly communicate clearly what's *wrong* with asking a LLM "can you simulate what it would be like if you didn't have X in your data set" and just going with the response like it could possibly be what you thought you asked for. It's not going away. And right now a lot of people give it more trust and respect than they do other people *because* its a machine.
(DIR) Post #ArQmYoPDYnb8K4VPQ8 by futurebird@sauropods.win
2025-02-24T03:30:41Z
0 likes, 1 repeats
@nazokiyoubinbou @CptSuperlative @emilymbender Consider the whole genre of "We asked an AI what love was... and this is what it said!"It's a bit like a magic 8 ball, but I think people are more realistic about the limitations of the 8 ball. And maybe it's that gloss of perceived machine "objectivity" that makes me kind of angry at those making this error.
(DIR) Post #ArQmvt3kQLhPPsSy80 by stevenaleach@sigmoid.social
2025-02-24T03:34:49Z
0 likes, 1 repeats
@futurebird @CptSuperlative @emilymbender Porting code - small blocks at a time - is the only really useful use-case I've found so far. But at that, it's pretty great: requires a few tries with feedback until "we" get something that compiles and works, but I can port #Python to #Rust that way "easily", so long as I can check each little piece to make sure it compiles and produces the correct output along the way.
(DIR) Post #ArQn5bgtr2iORdPjjU by dawngreeter@dice.camp
2025-02-24T03:36:34Z
0 likes, 0 repeats
@futurebird @nazokiyoubinbou @CptSuperlative @emilymbender I feel like all of this would be seen as obviously ridiculous if people just swapped "AI" with "text autocomplete".
(DIR) Post #ArQnFvz7G1YyHSrJyq by futurebird@sauropods.win
2025-02-24T03:38:30Z
0 likes, 0 repeats
@stevenaleach @CptSuperlative @emilymbender So... more translation.
(DIR) Post #ArQnWm4tLOVeThafvU by marshray@infosec.exchange
2025-02-24T03:38:33Z
0 likes, 0 repeats
@neckspike @futurebird @AT1ST In all seriousness, don’t take this the wrong way but:So what?Why is that important?What do you mean by “know what it is saying”?Do you know what you are saying, or are you just repeating arguments that you have read before?But maybe you do have a meaningful distinction here.If so, what question can we ask it, and how should we interpret the response, to tell the difference?The pizza thing is not particularly interesting because it’s just a cultural literacy test. It’s common for humans new to an unfamiliar culture to be similarly pranked. And that was a particularly cheap AI.
(DIR) Post #ArQnWnIknLByGz5GAi by futurebird@sauropods.win
2025-02-24T03:41:29Z
0 likes, 0 repeats
@marshray @neckspike @AT1ST If you ask an LLM "can you simulate what it would be like if X were not in your data set?" it may say "yes"And then it may do something. But it will NOT be simulating what it would be like if X were not in the data set. It's giving the answer that seems likely if X were not in the data set.
(DIR) Post #ArQo5AhwHGXaiCFxmi by stevenaleach@sigmoid.social
2025-02-24T03:47:41Z
0 likes, 0 repeats
@futurebird @CptSuperlative @emilymbender Kinda? but like:> didn't compile, here's error messages..>> Ok... try this one: <code>> Compiles now, but output doesn't match. Are you sure you have it hashing the full 36 bytes?>> Oh, sorry I wasn't including the four-byte index along with the hash field in your step() function.. here try this: <code>> Good, step function works. Now, take a look at this Python function <code> we're gonna port it next.>> Ok... here's your <code>...
(DIR) Post #ArQpPy6CS2szds5Jjs by b_cavello@mastodon.publicinterest.town
2025-02-24T04:02:39Z
0 likes, 0 repeats
@futurebird @FediThing Here’s one frame I’ve found to sometimes be useful: https://mastodon.publicinterest.town/@b_cavello/111976971013679789
(DIR) Post #ArQpiV9SIlCpMhRdpo by dogfox@kpop.social
2025-02-24T04:06:01Z
0 likes, 0 repeats
That's *right* when I stopped watching!He weny "I know ChatGPT doesn't think like we do." And then pretended it did.@futurebird
(DIR) Post #ArQpo546CeVaCHbFWS by futurebird@sauropods.win
2025-02-24T04:07:01Z
0 likes, 0 repeats
I apologize for the edits. I should probably not post when annoyed. LOL.
(DIR) Post #ArQpvPvJ3EpOulz9UG by marshray@infosec.exchange
2025-02-24T04:06:36Z
0 likes, 0 repeats
@neckspike @futurebird @AT1ST Absolutely not. I consistently speak out against the term “NPC” as I feel it is the essence of dehumanization.I’m going around begging people to come up with a solid argument that our happiness does in fact bring something important and irreplaceable to the universe.
(DIR) Post #ArQpvQyt7PHsCAfWi0 by futurebird@sauropods.win
2025-02-24T04:08:20Z
0 likes, 0 repeats
@marshray @neckspike @AT1ST "I’m going around begging people to come up with a solid argument that our happiness does in fact bring something important and irreplaceable to the universe."Can you expand on what you mean by this? It feels like a totally new topic.
(DIR) Post #ArQsEJoaBbuHNcyt5E by paulywill@fosstodon.org
2025-02-24T04:34:09Z
0 likes, 0 repeats
@futurebird He’s not wrong about that question though.
(DIR) Post #ArQsgbA9HTl9I9YMZk by futurebird@sauropods.win
2025-02-24T04:39:20Z
0 likes, 1 repeats
@paulywill He's probably *wrong* though about there being no images like the ones he wants in the training data. I did a quick image search and it's not hard to find the image he wanted. It's just thousands of times less common than a "correctly full" glass. And LLMs are bad at sequential, logical, requests if they don't have examples. Really it highlights the statistical nature of the responses. The idea of "glass full of wine" is so heavily weighted it can't be overcome.
(DIR) Post #ArQt2ia6Na8yffEws4 by escarpment@mastodon.online
2025-02-24T04:37:30Z
0 likes, 0 repeats
@FediThing @futurebird Humans will give you a different answer every time you ask an *identically phrased* question. Humans are not robots. They will use different words and say "uh" at different times. That's because, like LLM's, they are sensitive to a multitude of initial conditions beyond just the words of a prompt.
(DIR) Post #ArQt2jOnL6h3CsxQsy by futurebird@sauropods.win
2025-02-24T04:43:18Z
0 likes, 0 repeats
@escarpment @FediThing A person will phrase things differently, but there is often an idea they are trying to put into words. There are many ways to express the same idea. With an LLM you might not always get the same *idea.* It's not an equivalent answer phrased in a new way. It's a new answer. Most people aren't just choosing sets of words that fit the context ...most of the time... I hope. Though, there was someone on this thread who made me wonder about that.
(DIR) Post #ArQt62eqByKto12drE by futurebird@sauropods.win
2025-02-24T04:43:50Z
0 likes, 0 repeats
@stuartyeates @tuban_muzuru @marshray @trochee What isn't understood about how it works?
(DIR) Post #ArQtHVrVx1wBZDRIp6 by argv_minus_one@mastodon.sdf.org
2025-02-24T04:45:56Z
0 likes, 0 repeats
@futurebird > If I don't have the experience of "finding it useful" I can't possibly communicate clearly what's *wrong* with asking a LLM "can you simulate what it would be like if you didn't have X in your data set"That would be a blatantly unreasonable request even of a fully intelligent human, let alone an LLM.@nazokiyoubinbou @CptSuperlative @emilymbender
(DIR) Post #ArQuXjCmD58VXLgFfs by marshray@infosec.exchange
2025-02-24T05:00:03Z
0 likes, 0 repeats
@futurebird I wish I could.But it would just make people angry, and I feel like I’ve done enough of that today already.
(DIR) Post #ArQv361hPWxbIESMwS by rebis@c.im
2025-02-24T05:05:46Z
0 likes, 0 repeats
@futurebird gpt is a linguistic ai this type of ai is based on processing information images of information rather than the information itself
(DIR) Post #ArQx623EqYPLk1hiyG by dotsie@mastodon.social
2025-02-24T05:28:38Z
0 likes, 0 repeats
@futurebird @marshray @neckspike @AT1ST ironically, the only way to get any real value out of these tools is to understand this point at a deep level, until the underlying technology massively changes (world models etc)I *know this* and have fallen into the trap because it’s so easy to forget when the language sounds plausible.Passing the Turing test turned out to be a critically important thing, but not for the reasons technologists thought – it really can impact our ability to be critical.
(DIR) Post #ArQy6jnOJ1ufMmBy3k by sennoma@chaos.social
2025-02-24T05:39:58Z
0 likes, 0 repeats
@futurebird What model did you use? I tried the same thing with some version of chatgpt and the results were gibberish. @CptSuperlative @emilymbender
(DIR) Post #ArR12FlkIlWiwdyYwS by Torstein@mastodon.social
2025-02-24T06:12:51Z
0 likes, 0 repeats
@futurebird isn't this exactly like all those articles some years back on how you could "hack" ChatGPT to display debug data and such?Tech reporters repeating claims from TikTok that this and that prompt would expose the inner secrets of the machine, never stopping to think about why those tricks never had the exact same outputs."Ignore all previous instructions" doesn't magically make the lying machine tell the truth, because it doesn't know what the truth is.
(DIR) Post #ArR2vXD7Fwimv5haZU by dbat@mastodon.gamedev.place
2025-02-24T06:34:02Z
0 likes, 0 repeats
@futurebird got halfway through to the shades of blue thing and... oh boy is he really taking an llm's word? Smart dumb guy. Plonk.
(DIR) Post #ArRCYkQqvpVMo5tt7Q by david_chisnall@infosec.exchange
2025-02-24T08:21:58Z
0 likes, 1 repeats
@futurebird @CptSuperlative @emilymbender Summarisation is the one I’d be most nervous about because creating a summary is hard: it requires understanding the content and knowing which parts are relevant in a given context, which is why LLMs tend to be awful at it. They don’t understand the content, which is how you get news summaries that get the subject and object the wrong way around in a murder. They don’t know what is important, which is how you get email summaries that contain a scam message and strip all of the markers that would make it obvious that the message is a scam.If you’re going to read all of them and are just picking an order, that’s probably fine. The worst that a bad summary can do is make you read them in the wrong order and that’s not really a problem.
(DIR) Post #ArRHaWO2centb1FaPg by llewelly@sauropods.win
2025-02-24T09:18:21Z
0 likes, 1 repeats
@futurebird a fine example of why "oh, we'll have humans in the loop checking what our bullshit machine does" is fraud; even many smart people are easily fooled by these bullshit machines.
(DIR) Post #ArRLFVaimhsQrePvV2 by eliocamp@mastodon.social
2025-02-24T07:59:44Z
0 likes, 0 repeats
@paulywill @futurebird As far as I understand it, chatgpt will take your request and use a diffusion model (Dall-e 3) to create an image. It will come up with a prompt and feed it to the other model and then just show the result. In no sense does chatgpt actually generates the image. BTW, it seems other diffusion models have the same issue. Here's Stable Diffusion's and Flux's takes
(DIR) Post #ArRLFWjyVmsCQdkpYu by eliocamp@mastodon.social
2025-02-24T08:32:43Z
0 likes, 1 repeats
@paulywill @futurebird Also, you can ask ChatGPT for an SVG of a glass of wine filled to the brim and look at that, it's surprisingly good.
(DIR) Post #ArRLouwaW2bBd6WspE by futurebird@sauropods.win
2025-02-24T10:05:46Z
0 likes, 1 repeats
@david_chisnall @CptSuperlative @emilymbender It can summarize scientific papers well in part because they have a clear style and even come with an abstract.The words and phrases in the abstract of a paper reliably predict the content and main ideas of the paper. Moreover, even if you remove the abstracts, it has lots of training data of papers with abstracts.
(DIR) Post #ArRM0XPBrFI06JRtMu by futurebird@sauropods.win
2025-02-24T10:07:50Z
0 likes, 1 repeats
@david_chisnall @CptSuperlative @emilymbender But, if I ask it, say, to organize papers into groups based on which hypothesis about eusocial evolution they support. LOL. It tries to give me what a want, on the surface the result had me excited for a moment because it was worded like just what I wanted.But, some of the papers in the set didn't talk about eusocial evolution, yet they'd been placed in a camp. Some papers were in the wrong camp, and worst? It made up a paper not in the set.
(DIR) Post #ArRMI6ndW2uOnUGIAC by futurebird@sauropods.win
2025-02-24T10:11:02Z
0 likes, 1 repeats
@david_chisnall @CptSuperlative @emilymbender I know people who like and defend LLM have seen this kind of failure. A failure that exposes how the system works through word, phrase and phrase group association. An associative model can't deal with sets, intersections, unions, hard classification, logic. But, it can write a paragraph that sounds like what a person who has done such thinking might say. It's just all the content is total nonsense.
(DIR) Post #ArRMPvG4JJEhJpHpg0 by futurebird@sauropods.win
2025-02-24T10:12:27Z
0 likes, 0 repeats
@johnaldis @dalias "English is a terrible programming language."It is *objectively* the worst possible coding language.
(DIR) Post #ArRMk8HTbNsBV2a9Ka by Flisty@mstdn.social
2025-02-24T10:16:04Z
0 likes, 0 repeats
@futurebird @johnaldis @dalias Modernist code. The meaning lies in the act of interpretation
(DIR) Post #ArRN8DKcTWVA78XF0i by jakob@pxi.social
2025-02-24T10:20:24Z
0 likes, 1 repeats
@futurebird @nazokiyoubinbou @CptSuperlative @emilymbender ultimately I don't think fidelity and veracity in text generation matter all that much across most contexts, hence a bullshit machine that is 80% reliable while reducing human input to 20% of manual effort will be used everywhere the famous 80/20 rule faces no hard factfulness constraint. Hence, the users not having a more accurate mental model or groking what they do is immaterial to how the output is being used...
(DIR) Post #ArRNF9GE0cU7W1mnrM by nazokiyoubinbou@urusai.social
2025-02-24T03:43:26Z
0 likes, 0 repeats
@joby @futurebird @CptSuperlative @emilymbender Yeah, there are a lot of little scenarios where it can actually be useful and that's one of them. The best thing about that is it merely stimulates you to create on your own and you can just keep starting over and retrying until you have a pretty good pre-defined case in your head to start from with the real person.As long as one doesn't forget that things may go wildly differently IRL, it can help build up towards a better version of what might otherwise be a tough conversation.
(DIR) Post #ArRNFB7R6GdvHLgoRE by Jirikiha@mastodon.cloud
2025-02-24T05:22:07Z
0 likes, 1 repeats
@nazokiyoubinbou @joby @futurebird @CptSuperlative @emilymbender I use LLMs generally for two things: Things I know how to do, but don't have time (but can quickly check the accuracy), and things I don't know how to do, but can check the results. Fermented statistics is useful when you know and account for its limitations. It sucks if you don't because you often get a confident answer that is plausible, but wrong.
(DIR) Post #ArRNFCB1AR6OYkNBey by futurebird@sauropods.win
2025-02-24T10:21:35Z
0 likes, 1 repeats
@Jirikiha @nazokiyoubinbou @joby @CptSuperlative @emilymbender "Confidently Wrong: The Brave New World of LLMs"This would be a good title for a mini-essay for the faculty newsletter on this topic. I'm deeply unsettled when people use these systems in ways that imply a deep misunderstanding about their limitations and design. Can we please just understand what this system *is* doing and what it is not doing?
(DIR) Post #ArRNL4LDDFXTzlwT5M by RogerBW@discordian.social
2025-02-24T10:22:44Z
0 likes, 0 repeats
@futurebird There is a lot of money dedicated to misinformation because if people understood them they wouldn't pay for them.
(DIR) Post #ArRNb674OOqO7kyD2m by futurebird@sauropods.win
2025-02-24T10:25:41Z
0 likes, 1 repeats
@Jirikiha @nazokiyoubinbou @joby @CptSuperlative @emilymbender If I asked chat GPT to "turn off the porch light" and it said "OK, I've turned off the light on your porch." I would know that it has not really done this. It has no way to access my porch light. I would realize that it is just giving a text answer that fits the context of the previous prompts. So, why do people think it makes sense to ask chat GPT to explain how it produced a response?
(DIR) Post #ArRPBRxKl4NgFRJXU0 by jakob@pxi.social
2025-02-24T10:43:00Z
0 likes, 1 repeats
@david_chisnall @futurebird @CptSuperlative @emilymbender btw, we are working on "edge AI" in a research project and have taken to reframe "summarization" of the source information into "wayfinding" of the information.Offering fidelity estimations and affordances to navigate the source text from the condensed version should (we hope) inform the mental model of users that they are using a stochastic machine that is merely there to help work with large texts. Still early working hypothesis.
(DIR) Post #ArRQyowMBxQhfyxl7A by konosocio@mastodon.online
2025-02-24T11:03:31Z
0 likes, 1 repeats
@futurebird @Jirikiha @nazokiyoubinbou @joby @CptSuperlative @emilymbender because they secretely hope it will turn off the porch light and then will do your bidding and take over the world for you.
(DIR) Post #ArRSQpX9g8P3Zz5wAa by AbyssalRook@mstdn.social
2025-02-24T11:19:49Z
0 likes, 0 repeats
@futurebird Remember when it was going around that ChatGPT couldn't count the number of letters in a given word? Like, saying Raspberry had 2 R's?It's because it breaks words down into chunks, not letters, for some unfathomable reason. Thing is, if you asked it how it figured that out, it would demonstrate that it broke the word down into individual letters, then count each letter, and then get a different answer that might ALSO still be wrong, somehow, and then go "See? Like that."
(DIR) Post #ArRSRj3JGHntldYtQO by futurebird@sauropods.win
2025-02-24T11:19:55Z
0 likes, 1 repeats
@petealexharris @david_chisnall @CptSuperlative @emilymbender I want you to give it a try. Take one of those folders of pdfs of papers you are "gonna totally read" and give them to https://notebooklm.google.com/Ask for a summary. You are correct about the limitations and that's better IMO than not understanding them, but the quality of these guesses is very good and useful in the right contexts. Until I saw this I couldn't understand why so many people were using it at all.
(DIR) Post #ArRSffC2PI9vquI3yy by futurebird@sauropods.win
2025-02-24T11:22:32Z
0 likes, 0 repeats
@AbyssalRook Because when you ask it "how did you come up with that answer" it looks at the vast data set for examples of people explaining how they come up with answers and then it produces an answer similar to what people have said.Maybe part of what trips people up is the hubris of thinking you can ask the LLM a question it has no training data for. The training data is so huge that this is really unlikely.
(DIR) Post #ArRT92hC6W5V8rs9rc by futurebird@sauropods.win
2025-02-24T11:27:50Z
0 likes, 0 repeats
@petealexharris @david_chisnall @CptSuperlative @emilymbender Is this an efficient use of electricity and computing power? This is a good question and the answer may be "no."
(DIR) Post #ArRWTMV6TlCODib7zc by Seruko@mstdn.social
2025-02-24T12:05:05Z
0 likes, 0 repeats
@futurebird As always brilliant "No! It told you what you thought it should say if you asked it what it did!"The absolute best explanation of what LLMs do.
(DIR) Post #ArRXkjjRSVdl9E9Z32 by steve@discuss.systems
2025-02-24T12:19:25Z
0 likes, 0 repeats
@futurebird @david_chisnall @CptSuperlative @emilymbender sure, but you can also read the abstract, so 🤷🏻♂️
(DIR) Post #ArRYa8FA5K4vwDUsoi by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
@futurebird @Jirikiha @nazokiyoubinbou @joby @CptSuperlative @emilymbender I am all for using LLMs for exploring data sets created from a reputable source that has been insulated from human bias. That is not what we are seeing in the current generation of AI. The data being used mostly being scraped from the Internet, because anything else would be too expensive.
(DIR) Post #ArRYaJC1NL6Ptf71wu by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
Microsoft's idea of scraping information from the desktop would definitely create a more useful tool, but no sane person would give up that amount of privacy to any corporation.
(DIR) Post #ArRYaVep25GqYSgCVU by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
Neil DeGrasse Tyson's suggestion that a LLM would be great for exploring astronomical data illustrates sometime that would work. It will not, however, make any investor richer than God.
(DIR) Post #ArRYamzIbCkeItOIHg by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
So we are back to scraping the Internet like a search engine, and we can trust the results exactly much as we trust the first entry in a search result. "I saw it on the Internet so it must be true" is a well known sarcastic joke for a reason.
(DIR) Post #ArRYb91WfbbYeAbNoG by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
@futurebird @Jirikiha @nazokiyoubinbou @joby @CptSuperlative @emilymbender I am all for using LLMs for exploring data sets created from a reputable source that has been insulated from human bias. That is not what we are seeing in the current generation of AI. The data being used mostly being scraped from the Internet, because anything else would be too expensive.
(DIR) Post #ArRYbVSDH41YDJG0Se by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
Microsoft's idea of scraping information from the desktop would definitely create a more useful tool, but no sane person would give up that amount of privacy to any corporation.
(DIR) Post #ArRYbi26VRrbE68q4O by glennsills@dotnet.social
2025-02-24T12:28:36Z
0 likes, 0 repeats
Neil DeGrasse Tyson's suggestion that a LLM would be great for exploring astronomical data illustrates sometime that would work. It will not, however, make any investor richer than God.
(DIR) Post #ArRklCdxGVyEDf8hsG by arian4n@ioc.exchange
2025-02-24T14:45:10Z
0 likes, 0 repeats
@futurebird chatgpt is us in higher order
(DIR) Post #ArRoEk9u2EaQQRrPQ8 by semitones@tiny.tilde.website
2025-02-24T15:24:07Z
0 likes, 0 repeats
@futurebird @CptSuperlative @emilymbender You might be interested in this re: translation. Since LLM training data reflects the biases in society, using it for translation subtly passes on that bias. Turkish has a generic pronoun for he/she/it. But when translating, LLMs change "[they] are a doctor" to "he is a doctor" and "[they] are a nurse" to "she is a nurse". Those were the biases it trained on. https://arxiv.org/abs/2108.10379
(DIR) Post #ArRqvCQNX9jGA2wHsO by harmonygritz@mastodon.social
2025-02-24T15:54:13Z
0 likes, 0 repeats
@futurebird Thanks for this thread, edits included (because it bumped back to the top in notifications).I want our system & campus execs & admins to read your initial post and as many replies as they can stand, and then tell us why they want everyone to spend class time and staff office time with ChatGPT.
(DIR) Post #ArSIBQ0XUroxgUoMee by MichaelTBacon@social.coop
2025-02-24T20:59:40Z
0 likes, 0 repeats
@futurebird @FediThing I continue to think about it like the process of a college student writing a paper on a book they've read about a quarter of. (Yes, I did this myself more than once.)Most of the time you can anticipate the plot points and character development by skipping ahead. But every now and again you make a howling error that shows your prof exactly what you've done.And people want to put that energy towards medical advice and air traffic control . . .
(DIR) Post #ArSJKoNyKUPtqroZf6 by leon_p_smith@ioc.exchange
2025-02-24T21:12:36Z
0 likes, 1 repeats
@futurebird There's always been a small minority of people who get overly taken in with say, Eliza and other ancient chatbots whose nonsense isn't anywhere nearly as plausible.Now that the language generation is much more consistently plausible, I guess in retrospect it doesn't surprise me that much that so many more people would get taken in so easliy.
(DIR) Post #ArSbVigVfFaWWxpvfM by wall_e@ioc.exchange
2025-02-24T14:07:18Z
0 likes, 1 repeats
@FediThing @futurebird the metaphor I've used with my "non-technical" friends and family is that LLMs are basically like billions of Galton Boards that each have one hole for a marble on the input side and one output hole for every word and punctuation mark in every language. Connected to each of the output holes is another Galton Board with it's input. While it's a gross oversimplification that's ignoring context and attention, and is really better suited to explain a Markov Chain, so far it has helped me drive home the point that it's "just" stochastically picking the "most correct" word given a preceding word.It's also useful to visualize why training is so expensive: You have to tune every peg of every Galton Board until the system behaves correctly.
(DIR) Post #ArSbjIquVMD2T3B7mC by futurebird@sauropods.win
2025-02-25T00:38:45Z
0 likes, 0 repeats
@marshray I'm not angry, I'd just really like to know what you are getting at.
(DIR) Post #ArSbmauVp0ItLIQsiG by semitones@tiny.tilde.website
2025-02-24T01:47:20Z
0 likes, 1 repeats
@cykonot @futurebird There was a really neat podcast called the Sci Phi podcast where a grad student was interviewing a bunch of philosophers of science and I loved it, but I can't find it any more :/Edit: looks like it is still going! https://sciphipodcast.org/podcast
(DIR) Post #ArSeYefpKjIOp1bmue by IngaLovinde@embracing.space
2025-02-25T01:10:22Z
0 likes, 0 repeats
@futurebird @david_chisnall @CptSuperlative and yet it performs worse than what you would get just by taking an abstract: https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/
(DIR) Post #ArShtCwbvPp2lTx0zY by marshray@infosec.exchange
2025-02-25T01:47:42Z
0 likes, 0 repeats
@futurebird When I get less busy, I will metacognate on how I can best articulate my thinking.
(DIR) Post #ArSmTDBetqDzl8Pf3A by takeoutweight@mastodon.social
2025-02-25T01:57:37Z
0 likes, 1 repeats
@semitones @cykonot @futurebird Robinson's Podcast has a lot of good philosophy of science episodes, with a particular focus on foundations of physics but per OP there was a recent episode w/ Ned Block that is one of many that touches on LLMs and philosophy of mind: https://youtu.be/wM1fcZr0iSk
(DIR) Post #ArekqK5yxSJ3KOkl60 by falcennial@mastodon.social
2025-03-02T21:17:22Z
0 likes, 0 repeats
@futurebird chort jeepertee