Post B12mvf8d9F9pV67W40 by malte@anticapitalist.party
(DIR) More posts by malte@anticapitalist.party
(DIR) Post #B12VlJyjGYjDl4yKYq by futurebird@sauropods.win
2025-12-08T13:10:24Z
1 likes, 1 repeats
“If the LLM produces a wild result, something that doesn’t meet with my expectations *then* I’ll turn to more reliable sources. I’m not blindly following just anything that says” People feel that this being a “responsible user of new technology”I think it actually proves the opposite.1/2
(DIR) Post #B12VuAeVJHkxRKjQky by NemoAndTheRats@wandering.shop
2025-12-08T13:11:56Z
0 likes, 0 repeats
@futurebird confirmation bias on the whole new level...
(DIR) Post #B12W0CfUjAlzQi4VE0 by futurebird@sauropods.win
2025-12-08T13:13:06Z
1 likes, 1 repeats
The most exciting and pivotal moments in research are those times when the results do not meet your expectations. We live for those moments.If an LLM is not reliable enough for you to trust unexpected results then it is not reliable enough to tell you anything new: it’s incapable of telling you anything that you don’t at (some level) already know.2/2
(DIR) Post #B12WSI5EBipSKgV8Xw by jhavok@mstdn.party
2025-12-08T13:18:06Z
0 likes, 0 repeats
@futurebird LLMs are designed to produce plausible bullshit. But if you know enough about the subject to reject plausible bullshit, why are you asking an LLM?
(DIR) Post #B12WYYY1t0PV8MxcJ6 by DrHyde@fosstodon.org
2025-12-08T13:19:16Z
0 likes, 0 repeats
@futurebird I suppose most people don't bother to check when *people* tell them things that match their expectations, so it's no surprise that they don't bother when a machine does the same.
(DIR) Post #B12WamLNZ2HwV66SkS by jmjm@mstdn.social
2025-12-08T13:19:37Z
0 likes, 0 repeats
@futurebird I'm surprised that it took so long, but i actually got some dangerously wrong medical advice from an LLM a week ago. I only realized it was wrong when i clicked through to the Mayo Clinic article. I miss the days when Google would just give me search results.
(DIR) Post #B12XQF7YKcGUWHAyie by Gotterdammerung@glitch.social
2025-12-08T13:28:56Z
0 likes, 0 repeats
@futurebird If we only trust AI when it agrees with us, we're not using artificial intelligence. Just an expensive echo chamber with typing animation.
(DIR) Post #B12XmkQwalf20Nyf7Q by cford@toot.thoughtworks.com
2025-12-08T13:33:00Z
0 likes, 0 repeats
@futurebird From another angle, I worry about the opposite, that we'll take its word when it confirms our preconceptions (which it's trained to do).
(DIR) Post #B12aXQvJ9FaX0vZKb2 by RegGuy@mstdn.social
2025-12-08T14:03:50Z
0 likes, 0 repeats
@futurebird That the LLMs have been trained by sucking up all the written works on the internet should tell us all we need to know about reliability.
(DIR) Post #B12cFdtYNfsPwOOPaa by Urban_Hermit@mstdn.social
2025-12-08T14:23:00Z
0 likes, 0 repeats
@futurebird 1/2Supposedly these things are good at finding correlations. But that is confusing narrowly focused, small data set, supervised research with generic LLMs.In my personal experience, the LLMs I have access to are likely to ignore all minority opinions, new research, and claim that scantly documented problems do not exist. They can not weigh the significance of any data, so they always default to what is frequently said is more true.
(DIR) Post #B12dVP5dIH5obw7dEu by MedeaVanamonde@beige.party
2025-12-08T14:37:07Z
0 likes, 0 repeats
@futurebird more simply:It tells you the lies you want to hear
(DIR) Post #B12fRjQpv8egRzPtzs by johnzajac@dice.camp
2025-12-08T14:58:51Z
0 likes, 0 repeats
@futurebird Any source of data that is wrong sometimes but you don't know when is worse than useless.
(DIR) Post #B12hAr5CQ9uLAINPPc by mcmullin@musicians.today
2025-12-08T15:18:03Z
0 likes, 0 repeats
@futurebird Related thread, from a while back. Using generic LLMs for science is terribly irresponsible, because they cannot integrate new evidence or evaluate the merits of new arguments. They only know what's been said more often. The glory of science is its ability sometimes to demonstrate that what everyone has always said is wrong, and a new idea is better.https://musicians.today/@mcmullin/115113940243762269
(DIR) Post #B12jIHSIxxmytTqueW by Urban_Hermit@mstdn.social
2025-12-08T14:23:32Z
0 likes, 0 repeats
@futurebird 2/2It is a stereotype disaster waiting to influence everything. They will rob us of Science. To an LLM, and the tech billionaires who want to influence us, what is stated frequently is always true, and what is new is always wrong or suspect.Intellectual stagnation. In the age of information, LLMs are prepping us for a new dark age. The largest LLM in the world might as well be called Aristotle.Robbing us of Science.
(DIR) Post #B12jIIjiCjJ6rl0KQK by Urban_Hermit@mstdn.social
2025-12-08T14:57:28Z
0 likes, 0 repeats
@futurebird 1/3I experimented with an LLM last year at the urging of a friend.I invented a game called "minority opinion" where we (me & the LLM) took turns identifying theories that could replace the dominant explination, and then asked the LLM to estimate a probability, based on supporting evidence, that the paradigm could be replaced with the new idea in the future.The LLM could list a dozen reasons why a new theory was a better fit, yet the probabilities were always astonishingly low.
(DIR) Post #B12jIJsFyRjiOY0fNg by Urban_Hermit@mstdn.social
2025-12-08T14:58:07Z
0 likes, 0 repeats
@futurebird 2/3And I could push those probabilities around by simply objecting to them. So it really is a people pleasing machine.I knew LLM logic was worthless when the LLM chose to believe that ghosts were a more likely explanation for haunted houses than carbon monoxide poisoning. Because of the many ghosts that people claim to have personally identified.
(DIR) Post #B12jILCr1Lo4Wied7o by Urban_Hermit@mstdn.social
2025-12-08T14:58:33Z
0 likes, 0 repeats
@futurebird 3/3Google can weigh a source for their own LLM, making it insist on a thing, but they can't weigh their own sources for credibility, other than frequency in the training data.So, the most commonly held beliefs are automatically true and will be referenced as such by an LLM.It's a confirmation bias machine for all of humanity.The end of Science.
(DIR) Post #B12jIMH92sphqJfZS4 by Urban_Hermit@mstdn.social
2025-12-08T15:05:00Z
0 likes, 0 repeats
@futurebird 4/3Make no mistake, the fact that I could have a conversation like this with a machine is a great accomplishment. Turning a huge data set and some language rules into a thing I could query for hours is astonishing.But, current AI has a credibility problem, and that means that it is not ready as a truth telling product. And the hype outweighs the truthiness by an uncomfortable margin.
(DIR) Post #B12jINSscjoXX0ASNk by Urban_Hermit@mstdn.social
2025-12-08T15:34:49Z
0 likes, 0 repeats
@futurebird A/CAnother experiment:I know of a data base that was populated by an early AI that 'hallucinated' details. An international kite museum, supposedly in Corpus Christi, Texas, was said to be populated by displays on "The Age of Mammoths" and "The Iron Horse" because the word "museum" took more precedence than "International Kite".It hallucinated a lot of other generic museum like details.A street view search of the address shows a hotel, and no museum at all for blocks around.
(DIR) Post #B12jIOVkjXhqmCWGUy by futurebird@sauropods.win
2025-12-08T15:41:54Z
0 likes, 0 repeats
@Urban_Hermit I think when some people are presented with these kinds of errors they think “the LMM just made a factual mistake” they think with more data and “better software” this will not be a problem. They don’t see that what it is doing is *foundationally different* from what they are asking it to do. That it has fallen to random CS educators, and people interested in these models to desperately try to impress upon the public the way they are being tricked makes me angry.
(DIR) Post #B12jIVVeiImuTvWmoq by Urban_Hermit@mstdn.social
2025-12-08T15:35:25Z
0 likes, 0 repeats
@futurebird B/CSo, I asked the LLM I was testing to evaluate this web page and list the supporting evidence that this museum was not a kite museum and did not exist.It had very profound problems evaluating this. That is when I saw definitive evidence that it did not understand words and would not look up the meanings unless prompted to.It mostly supported the idea that the museum must exist, but was simply not an "international kite" museum, and the address was wrong.
(DIR) Post #B12jIdUX2MdtEl40B6 by Urban_Hermit@mstdn.social
2025-12-08T15:37:45Z
0 likes, 0 repeats
@futurebird C/CThe format of a page describing a museum like thing overpowered the importance of the type of museum or that it did not exist at its reported location - two much more relevant facts.The number of words associated with "museum" meant, to the LLM, that a museum of some sort, somewhere else in Corpus Christi, must exist, AND that this page was still trying to describe it.So, it basically could not weigh the credibility of this source because it could not discount any words.
(DIR) Post #B12jPm0NmWkTfXMgBk by screwturn@mastodon.social
2025-12-08T15:43:19Z
0 likes, 0 repeats
@futurebird That is such a good insight!The QDA I use for qualitative and mixed-methods work has never yet hallucinated, but it does sometimes get implications or assumptions wrong. So, if it produced a very unexpected result, I would be excited and curious, but assume it was an error
(DIR) Post #B12kFTTbupU1GIgmDA by futurebird@sauropods.win
2025-12-08T15:52:41Z
0 likes, 1 repeats
@the5thColumnist @Urban_Hermit People, even people who have terrible mostly wrong ideas tend to have some guiding set of values. Even if you don’t learn much from their opinions you can learn about the philosophy that informs those opinions. Asking an LLM *why* it said any fact or opinion is pointless. It will supply a response that sounds like a human justification but the real “reason” is always the same “it was the response you were most likely to accept as correct”
(DIR) Post #B12kftCYtR7IZlJI36 by futurebird@sauropods.win
2025-12-08T15:57:29Z
0 likes, 0 repeats
@the5thColumnist @Urban_Hermit I’m trying to avoid loaded phrases like “bullshitting machine” I’ve had a lot of people roll their eyes and shut down on me because “well you just hate AI” as if this is a matter of personal taste. In reality I struggle to see how I could find value in exposing my curiosity to a system with these limitations. I will insulate myself from those time when a simple obvious question brings me up short— it just seems really dangerous to me.
(DIR) Post #B12l7op2ljQd6Ncxqy by janggolan@mastodon.social
2025-12-08T16:02:31Z
0 likes, 0 repeats
@futurebird Will be a sad day when LLMs steal all that on-the-job training our grad students were supposed to get —as well as appointments within The Groves Of Academe… https://en.wikipedia.org/wiki/The_Groves_of_Academe
(DIR) Post #B12lU1UQLu7klBZROC by futurebird@sauropods.win
2025-12-08T16:06:31Z
0 likes, 1 repeats
@alessandro @talya I think with topics where one isn’t an expert it can be more important to know what “most people” in your social circle *think* is true than it can be to know what is really true. Knowing an iconoclastic truth, but not having the expertise to explain it to others isn’t very useful. Moreover without that expertise you will struggle to evaluate the validity of the unpopular opinion. So, people reach for the popular option and hope it is correct.
(DIR) Post #B12lh44s2fNSYCpxNR by futurebird@sauropods.win
2025-12-08T16:08:55Z
0 likes, 0 repeats
@alessandro @talya No one is enough of a polymath and no one has enough time to avoid trusting others. This isn’t really a bad thing, but we have to be open to the reality that there are some things that “everyone knows” that are simply wrong.
(DIR) Post #B12mXzOUBba3NAFRy4 by vivasupeR@mastodon.social
2025-12-08T16:18:26Z
0 likes, 0 repeats
@futurebird I confirm. AI is wild when it makes things up, and sometimes what it invents doesn’t mean anything. For everything else, it’s handy for finding sources or doing search, but certainly not for making inventions. It only knows what it has been taught, and at best it rephrases things already said by the user. Still, it’s useful for many tasks.
(DIR) Post #B12mvf8d9F9pV67W40 by malte@anticapitalist.party
2025-12-08T16:22:42Z
0 likes, 0 repeats
@futurebird it also sounds a little bit like "once my car spins out and hits a tree, *then* i'll go and change the tires." 😬
(DIR) Post #B12nYqDDuP6J8jwyx6 by mloxton@med-mastodon.com
2025-12-08T16:29:47Z
0 likes, 0 repeats
@futurebirdI can describe how an LLM in my QDA helps me1. If I have a huge and new document corpus loaded, the LLM can give me a five-minute overview of the topics it discovered in the corpus, and also whether and where a topic I defined was covered. This is amazingly useful as a quick orientation, but also allows segmentation. Like if these are interviews of five different KINDS of people, the LLM can show which topics were discussed by which groups. /2 @the5thColumnist @Urban_Hermit
(DIR) Post #B12pLJjdXxaIYcn4r2 by futurebird@sauropods.win
2025-12-08T16:49:45Z
0 likes, 0 repeats
@elight @the5thColumnist @Urban_Hermit Because the popular models people will encounter have been trained to work this way.
(DIR) Post #B12vHRdmoMWfdvagV6 by aianhanma@toot.community
2025-12-08T17:56:18Z
0 likes, 0 repeats
@futurebird Not to mention all the cases where opposing answers could very well be reasonable.Woul not that even be very likely to happen often? Otherwise, why is a user asking the questions in the first place?
(DIR) Post #B12yo3NoJzIRpOMJMm by Urban_Hermit@mstdn.social
2025-12-08T18:35:49Z
0 likes, 0 repeats
@futurebird @elight @the5thColumnist Someone said, and I deeply feel this after my experiments, that "The only question that LLMs ever answer is 'What does a response to this look like?'"If the responses are always the same and always accurate, like a simple question in computer programming, then the LLM will be accurate, but not because it understands the question, it is just assembling most probable responses based on simple, accurate data.
(DIR) Post #B132fIb8maT6E7ftyq by mrcopilot@mstdn.social
2025-12-08T19:19:02Z
0 likes, 0 repeats
@futurebird Recognizing a source of unreliable information is an integral part of critical thinking tightly wed to mistrusting that source."I asked a known liar important questions all the time, sometimes I can even tell when answers are lies" is wild human behavior.
(DIR) Post #B136yOTFZVopl9LQmm by csstrowbridge@mastodon.social
2025-12-08T20:07:18Z
0 likes, 0 repeats
@futurebird I want to ask these people, "If you don't know the answer before you ask the LLM, then how would you know if if gave you the wrong answer?"
(DIR) Post #B138ggscaVmBYBtkFk by Arlodottxt@fosstodon.org
2025-12-08T20:26:33Z
0 likes, 0 repeats
@futurebird It works well to assume that AI has very little meaningful capability to bring to the table, but that it has a lot of capacity for whatever it CAN do.The solution, then, is for the human to bring their capability to the AI, and for the AI to bring its' capacity to the human.
(DIR) Post #B13P2q54et9NiwJ1g8 by Gorfram@beige.party
2025-12-08T22:41:41Z
0 likes, 0 repeats
@datarama @futurebird @alessandro @talya Now I'm wondering what lizards & spiders dream about (I'm guessing flies figure pretty heavily).So many things in science were right until they were wrong. I remember being taught that the earth's crust rested on a generally quiescent layer of molten lava, which occasionally broke through at weak spots in the form of erupting volcanoes (plate tectonics was known in academic circles by then, but hadn't made it as far as our 1960's-era 4th-grade earth science textbooks).
(DIR) Post #B13P2r0rC3N4c9LAkC by futurebird@sauropods.win
2025-12-08T23:29:47Z
0 likes, 1 repeats
@Gorfram @datarama @alessandro @talya Ants and bees probably dream too. I suspect they would review the things they learned about the space and resources around their nest, making memories of the locations of things more permanent. This is on the assumption that a purpose of dreaming is to refine and organize memories and things learned during the day.
(DIR) Post #B13PGeOqsvahYA8uLg by futurebird@sauropods.win
2025-12-08T23:32:21Z
0 likes, 1 repeats
@Gorfram @datarama @alessandro @talya We have a common ancestor with arthropods, but it was long long long ago. The fact that dreaming may be shared, even if it's very different by such very different creatures hints at something profound and fundamental about being an organism with a brain and an nervous system and bilateral symmetry who interacts with the world and makes choices. Having that in common is enough for both of us to need to sleep and also probably need to dream.
(DIR) Post #B13PLK4Ju3TpwSlbbk by futurebird@sauropods.win
2025-12-08T23:33:07Z
0 likes, 0 repeats
@ireneista @Gorfram @datarama @alessandro @talya It's all over the place I really doubt it's useless.
(DIR) Post #B13PzqON926fm5T5cG by lionelb@expressional.social
2025-12-08T23:40:28Z
0 likes, 0 repeats
@futurebird @Gorfram @datarama @alessandro @talya I suspect that personality, pleasure and empathy are characteristics of many more creatures than we generally imagine.
(DIR) Post #B14W5Ucvm2GgtLJZCq by danblondell@masto.nyc
2025-12-09T12:23:25Z
0 likes, 0 repeats
@futurebird A rule of thumb I heard is don’t use these things for anything you can’t competently do yourself. Not because you can’t be surprised by them, but so you can evaluate the output.It’s like having a research assistant who is replicating work that you could, but faster and worse. That arrangement could produce plenty of real insights and discoveries, but no responsible researcher would immediately release those results without verifying the surprising stuff personally.