Post AW6DMnqLgBc4mQilSy by SnoopJ@hachyderm.io
(DIR) More posts by SnoopJ@hachyderm.io
(DIR) Post #AW5cPBnxriFxN2sf6O by simon@fedi.simonwillison.net
2023-05-27T19:12:11Z
0 likes, 0 repeats
Lawyer cites fake cases invented by ChatGPT, judge is not amusedhttps://simonwillison.net/2023/May/27/lawyer-chatgpt/
(DIR) Post #AW5ckIOMZOYvshyCQq by scrambled1839@mstdn.party
2023-05-27T19:15:54Z
0 likes, 0 repeats
@simon citing chatgpt in front of a judge is arguably worse then using chatgpt to cheat on a final in college.
(DIR) Post #AW5fRoZoz0uK3rD0EK by simon@fedi.simonwillison.net
2023-05-27T19:46:24Z
1 likes, 0 repeats
There are so many weird and entertaining aspects of this story. One of my favourites is that the lawyer submitted screenshots from the *mobile web* version of ChatGPT showing how they confirmed its cited cases!
(DIR) Post #AW5fmEWkD4gIKvVQCO by drsbaitso@infosec.exchange
2023-05-27T19:49:11Z
0 likes, 0 repeats
@simon I was talking with a lawyer friend last night, and apparently this case has already made the rounds, along with "strong guidance" against stuff like this from essentially every source imaginable.
(DIR) Post #AW5fxtlHF2trPMrjjE by colindean@mastodon.social
2023-05-27T19:52:04Z
0 likes, 0 repeats
@simon I'm floored.I guess someone had to be the first.
(DIR) Post #AW5gA6lohY7zEFwMAS by SETSystems@defcon.social
2023-05-27T19:53:03Z
0 likes, 0 repeats
@simon "Lawyer cites fake cases invented by ChatGPT, judge is not amused"This should give everyone pause. The automated fabrication of falsehoods should be considered a systemic threat to civilization. Whether it involves deepfakes of evidence or as in this case, the whole cloth fabrication of case history, these pose serious challenges that must be remedied.https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html
(DIR) Post #AW5hagiVm6jaCOdGUq by SpaceLifeForm@infosec.exchange
2023-05-27T20:10:10Z
0 likes, 0 repeats
Re: ChatGPT legal bullshit that ChatGPT createdSteven Schwartz consults with well respected chatbots Eliza and Shyster:Eliza: Why do you feel like you are fucked?Shyster: You are fucked.hXXtps://en.m.wikipedia.org/wiki/ELIZAhXXps://en.m.wikipedia.org/wiki/Shyster_(expert_system)
(DIR) Post #AW5iEe3l4eWHXaci0W by djvdq@mastodon.social
2023-05-27T20:17:23Z
0 likes, 0 repeats
@simon and I think that it will happen more and more in (near) future.
(DIR) Post #AW5k98YEzfleHPNmOu by micmath@fosstodon.org
2023-05-27T20:38:48Z
0 likes, 0 repeats
@simon I asked chatgpt a couple months ago to list books on a particular topic. One caught my eye because I had never heard of it and I knew the supposed author. So I asked for the isbn. It gave a bogus isbn. I asked it for links to purchase the book, it gave me 3, all bogus. I asked it for a publish date, summary and the table of contents, it gave them. I asked it to cite sources, it gave bogus URLs it claimed were about this book. I checked WebArchive: no such sources ever existed.
(DIR) Post #AW6DMnqLgBc4mQilSy by SnoopJ@hachyderm.io
2023-05-28T02:06:19Z
0 likes, 0 repeats
@simon thanks for the write-up!I have a quibble: I find the framing of your question"What possible reason could LLM companies have for shipping a model that does this?"to be kind of concerning. The reason seems obvious to me: they have already made a whole bunch of money from this model.In my opinion, the better question is:"What possible reason could they have for *not* shipping this model?"The only reasons I can think of are ethical ones, and companies are not ethical actors.
(DIR) Post #AW6Nzok3GvHMOz1Ano by raj@tikirobot.net
2023-05-28T04:05:05Z
0 likes, 0 repeats
@simon I tried adding a sentence to your prompt which seems to prevent giving bogus information, and I’m wondering if there is a way to get this behavior by default?
(DIR) Post #AW6TC7b7UWgQ81WhiS by shinyeliza@mastodon.social
2023-05-28T05:03:33Z
0 likes, 0 repeats
@simon I think the explanation that ChatGPT improved its warnings between March and now is most compelling—I don’t think someone who so incompetently used ChatGPT is going to suddenly be able to fake ChatGPT logs convincingly.
(DIR) Post #AW6dr88nNMPLEATGvw by oatmeal@emacs.ch
2023-05-28T07:02:57Z
0 likes, 0 repeats
@simon thought it sounded familiar https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/
(DIR) Post #AW8EIqjHUo4oqK0v3Y by piccolbo@toot.community
2023-05-29T01:25:57Z
0 likes, 0 repeats
@simon Contrast this with the fact that chatGPT passes the uniform bar exam. What does it mean? 1: you can pass the bar exam and be a terrible lawyer or 2: tests only work in the context they are designed for (people with a law degree trying to become lawyers) or 3: other?
(DIR) Post #AW8GtWukKYUg0hzk00 by simon@fedi.simonwillison.net
2023-05-29T01:55:16Z
0 likes, 0 repeats
@piccolbo presumably the bar exam doesn't include questions that might cause you to invent cases out of thin air - I doubt it expects exam takers to memorize many obscure examples
(DIR) Post #AW9XcivMJkoKiLoU5I by piccolbo@toot.community
2023-05-29T16:37:20Z
0 likes, 0 repeats
@simon presumably it focuses more on the law (any lawyers out there who've taken it?). So why doesn't chatGPT make up laws with the same ease as it does precedent?
(DIR) Post #AWAAIhzXyHFCF8DeNM by simon@fedi.simonwillison.net
2023-05-29T23:50:59Z
0 likes, 0 repeats
@piccolbo interesting question - though I've seen it make up laws with prompts that encourage it to do thatI asked it for UK laws that made necromancy illegal a while ago and it invented some very convincing fake ones
(DIR) Post #AWCN8PfmRJAv8jE49Y by Jackivers@mastodon.social
2023-05-31T01:23:52Z
0 likes, 0 repeats
@simon As of this evening, ChatGPT+ 3.5 model still doubled down supporting the cases as real on my first try: https://chat.openai.com/share/e0a2a287-108d-4c49-9c95-d955fc5d5345
(DIR) Post #AWDGmhme1CzwN5f77A by tommorris@mastodon.social
2023-05-31T11:47:32Z
0 likes, 1 repeats
@simon We have an English equivalent!It's not quite the same - the person who relied on ChatGPT is a litigant-in-person. It's rather more understandable with a self-representing litigant than someone with (oh god) thirty years in practice.https://www.lawgazette.co.uk/news/lip-presents-false-citations-to-court-after-asking-chatgpt/5116143.article