Post AnhJaxA8xRCg1idOFM by emilymbender@dair-community.social
(DIR) More posts by emilymbender@dair-community.social
(DIR) Post #AnhJauc2RGja7Mf7oG by emilymbender@dair-community.social
2024-11-04T03:16:56Z
0 likes, 0 repeats
As OpenAI and Meta introduce LLM-driven searchbots, I'd like to once again remind people that neither LLMs nor chatbots are good technology for information access.A thread, with links:Chirag Shah and I wrote about this in two academic papers:2022: https://dl.acm.org/doi/10.1145/3498366.35058162024: https://dl.acm.org/doi/10.1145/3649468We also have an op-ed from Dec 2022:https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334>>
(DIR) Post #AnhJavpttDPtue9i3U by nanook@friendica.eskimo.com
2024-11-04T11:27:32Z
0 likes, 0 repeats
Fantasy is not the term I would use, fantasies are usually positive. I can't see anything positive in an all-known machine. What makes God good is that although all-knowing, he has an identity, a personage, and thus compassion, not so a machine.
(DIR) Post #AnhJaxA8xRCg1idOFM by emilymbender@dair-community.social
2024-11-04T03:17:11Z
0 likes, 0 repeats
Why are LLMs bad for search? Because LLMs are nothing more than statistical models of the distribution of word forms in text, set up to output plausible-sounding sequences of words.https://www.youtube.com/watch?v=qpE40jwMilU>>
(DIR) Post #AnhJazibSHxLxAlwEi by emilymbender@dair-community.social
2024-11-04T03:17:29Z
0 likes, 0 repeats
If someone uses an LLM as a replacement for search, and the output they get is correct, this is just by chance.Furthermore, a system that is right 95% of the time is arguably more dangerous tthan one that is right 50% of the time. People will be more likely to trust the output, and likely less able to fact check the 5%.>>
(DIR) Post #AnhJb2AKMBJzXjl6jA by emilymbender@dair-community.social
2024-11-04T03:18:39Z
0 likes, 1 repeats
But even if the chatbots on offer were built around something other than LLMs, something that could reliably get the right answer, they'd still be a terrible technology for information access.Setting things up so that you get "the answer" to your question cuts off the user's ability to do the sense-making that is critical to information literacy.>>
(DIR) Post #AnhJb4h0xcelNh4ExE by emilymbender@dair-community.social
2024-11-04T03:19:55Z
0 likes, 0 repeats
That sense-making includes refining the question, understanding how different sources speak to the question, and locating each source within the information landscape.>>
(DIR) Post #AnhJb6m3DrsDptRI5A by emilymbender@dair-community.social
2024-11-04T03:20:15Z
0 likes, 0 repeats
Imagine putting a medical query into a standard search engine and receiving a list of links including one to a local university medical center, one to WebMD, one to Dr. Oz, and one to an active forum for people with similar medical issues.If you have the underlying links, you have the opportunity to evaluate the reliability and relevance of the information for your current query --- and also to build up your understanding of those sources over time.>>
(DIR) Post #AnhJb8uHIFduRzItBQ by emilymbender@dair-community.social
2024-11-04T03:20:38Z
0 likes, 0 repeats
If instead you get an answer from a chatbot, even if it is correct, you lose the opportunity for that growth in information literacy.The case of the discussion forum has a further twist: Any given piece of information there is probably one you'd want to verify from other sources, but the opportunity to connect with people going through similar medical journeys is priceless.>>
(DIR) Post #AnhJbBf92yJuzvFLVo by emilymbender@dair-community.social
2024-11-04T03:20:55Z
0 likes, 0 repeats
Finally, the chatbots-as-search paradigm encourages us to just accept answers as given, especially when they are stated in terms that are both friendly and authoritative.But now more than ever we all need to level-up our information access practices and hold high expectations regarding provenance --- i.e. citing of sources.The chatbot interface invites you to just sit back and take the appealing-looking AI slop as if it were "information". Don't be that guy./fin
(DIR) Post #AnilFyxO5DwNNvPsAK by leberschnitzel@existiert.ch
2024-11-04T10:31:00Z
0 likes, 0 repeats
@emilymbender people sending me LLM suggestions in a field where I would probably count as "Expert" and me having to explain that it doesn't work because of xyz has become a big part of my work day.LLMs have helped me personally learn things adjacent to what I already know though. Or do tasks that I personally find tedious but can check the correctness quickly. 1/2
(DIR) Post #AnilFzpGqt2g52cu9Y by strypey@mastodon.nzoss.nz
2024-11-05T04:12:06Z
0 likes, 0 repeats
(1/2)@leberschnitzel> LLMs have helped me personally learn things adjacent to what I already know though. Or do tasks that I personally find tedious but can check the correctness quicklyWikipedia can also do this. At a fraction of the computational cost, with links to primary sources, and controlled by a network of peers, overseen by a not-for-profit. Existing web search engines quoting WP are just as useful as chatting with a #MOLE. But better because they attribute and link to the source.
(DIR) Post #AnilG0m7K6771Y9tsO by strypey@mastodon.nzoss.nz
2024-11-05T04:12:08Z
0 likes, 0 repeats
(2/2)One simple regulation that would reduce the potential harm is a law requiring any Trained MOLEs(1) to cite its sources, every time they give an output. Whatever kind of interface i connected to (chatbot or otherwise). Hold them to the same standard we expect of a scholar paraphrasing the work of others.As @emilymbender says, understanding where information is coming from, and having access to primary sources, helps people weight it appropriately.#PolicyNZ(1) https://disintermedia.substack.com/p/invasion-of-the-mole-trainers)
(DIR) Post #AnilG1DPgbwqOCvhQG by leberschnitzel@existiert.ch
2024-11-04T10:32:17Z
0 likes, 0 repeats
@emilymbender For example I know scripting, but I don't know Lua. So to make Tabletop Simulator automation I used GPT, and while it has many problems, my knowledge of scripting helped to make very quick progress and gain understanding of the language.And I use it some times to write comments in my scripts which is something I'm horrible in, but with GPT doing it I can quickly check if it makes sense but don't have to do all the writing myself in a way that another person can understand it. 2/2
(DIR) Post #AnirtF3ikVTKLMTy2y by escarpment@mastodon.online
2024-11-04T13:38:16Z
0 likes, 0 repeats
@emilymbender For code, it's often trivially easy to confirm whether you got the 95% of correct answers or 5% of incorrect. You plug the snippet of code in, execute it, and see if it passes your test or not. Chatbots have revolutionized data retrieval for programming languages. A chatbot can surface an esoteric piece of documentation much faster than a developer can find it and present the documentation in a more understandable way, e.g. by presenting a worked example.
(DIR) Post #AnirtG0DF2GBGlqgDY by strypey@mastodon.nzoss.nz
2024-11-05T05:26:44Z
0 likes, 0 repeats
(1/2)@escarpment> For code, it's often trivially easy to confirm whether you got the 95% of correct answers or 5% of incorrect. You plug the snippet of code in, execute it, and see if it passes your test or notThat tells you whether it's a catastrophic failure or not. It doesn't tell you whether it a good solution for the problem.@emilymbender
(DIR) Post #AnirtH49Ht0EZGhKzY by strypey@mastodon.nzoss.nz
2024-11-05T05:26:45Z
0 likes, 0 repeats
(2/2)It doesn't tell you whether it's computationally efficient. Or easy to read for the next developer who needs to work with it. Or whether it's going to create bugs when combined with other code it interacts with.The only way to find out things like that is to become a software engineer, or to ask one. Relying on a Trained MOLE to vomit up bits of code takes away your opportunities to do either. The likely result is making all the problems caused by under-audited code exponentially worse.
(DIR) Post #Anis0OrKi7NclW6LNg by strypey@mastodon.nzoss.nz
2024-11-05T05:27:22Z
0 likes, 0 repeats
(2/2)It doesn't tell you whether it's computationally efficient. Or easy to read for the next developer who has to work with it. Or whether it's going to create bugs when combined with other code it interacts with.The only way to find out things like that is to become a software engineer, or to ask one. Relying on a Trained #MOLE to vomit up bits of code takes away your opportunities to do either. The likely result is making all the problems caused by under-audited code exponentially worse.
(DIR) Post #AnjP4S4rz3aQOOyNIe by escarpment@mastodon.online
2024-11-05T11:38:24Z
0 likes, 0 repeats
@strypey I am a seasoned software engineer. I can look at the solution and decide those things for myself. I can write tests or profile the code to assert its efficiency. I can marvel at, "oh I didn't know about that new language feature." I can ask ChatGPT more general questions about the code, e.g. "what are some architectural tradeoffs about, say, running these tasks in serial vs in parallel."