[HN Gopher] A man sued Avianca Airline - his lawyer used ChatGPT
___________________________________________________________________
A man sued Avianca Airline - his lawyer used ChatGPT
Author : nlh
Score : 143 points
Date : 2023-05-27 15:00 UTC (8 hours ago)
(HTM) web link (www.nytimes.com)
(TXT) w3m dump (www.nytimes.com)
| Am4TIfIsER0ppos wrote:
| I can't wait for chatgpt equivalent of citogenesis
| (https://xkcd.com/978/) to happen.
| Szpadel wrote:
| archive.org link
| https://web.archive.org/web/20230527153017/https://www.nytim...
| metadat wrote:
| https://archive.today/pnEhz
| joshka wrote:
| @dang - the earlier article on reason.com is more thorough -
| perhaps merge these two posts?
|
| https://news.ycombinator.com/item?id=36092509
|
| https://reason.com/volokh/2023/05/27/a-lawyers-filing-is-rep...
| NelsonMinar wrote:
| OpenAI deserves some blame here. Their product UI does a very
| poor job conveying that it will just make things up when asked
| for specific information. And in the conversation there's no
| particular hint that the confidently stated "facts" are not in
| fact true. A large number of people get tripped up by this when
| first using ChatGPT. I did myself, it took me a day or two to
| realize the papers or web pages it was claiming existed didn't.
| It's pathological.
| scrollaway wrote:
| This drawback is presented front and center every time you open
| ChatGPT. A lawyer of all people should have known better.
| hartator wrote:
| > Judge Castel said in an order that he had been presented with
| "an unprecedented circumstance," a legal submission replete with
| "bogus judicial decisions, with bogus quotes and bogus internal
| citations." He ordered a hearing for June 8 to discuss potential
| sanctions.
|
| Disbarment should be a no-brainer and a minimum.
|
| Just "Southern China Airlines" should have raised eyebrows. This
| lawyer has shown obvious disrespect to the court, the court time,
| and to his client.
| paul_f wrote:
| The lawyer should lose his license. Imagine they had turned in
| this brief and follow up and ChatGPT had not been involved.
| Instant disbarment. ChatGPT is not an excuse for a professional
| to produce nonsense in front of the court. Bye bye.
| psychphysic wrote:
| Disbarment seems excessive to me.
|
| I don't think this is indictiative of a wider issue. Or
| likely to be repeated substantially.
|
| In terms of severity it's painfully foolish but then again
| ChatGPT is a totally new tool and a lot of people will be
| caught off guard.
|
| I am stunned the lawyer didn't at least look up the case
| notes or even prepare a pocket brief if he believed they were
| real but hard to find
| unyttigfjelltol wrote:
| Yeah, interestingly, being bad at lawyering isn't a typical
| reason for discipline. Discipline is integrity and process-
| based, things like stealing client funds and failing to
| communicate promptly and reasonably with the client.
|
| Arguably the leading goof was using a technology the lawyer
| didn't understand and failing to inform the client of the
| risks of using it. Between that and citing garbage
| precedent-- half the bar might be eligible for discipline
| on any given day. The judge might issue sanctions but bar
| discipline is a different ball of wax.
| not_a_shill wrote:
| If disbarment is the minimum, what's the maximum? I'm
| unfamiliar with other cases in which lesser offenses have led
| to disbarment.
| bombcar wrote:
| I wouldn't be surprised if prison is on the table.
| Intentionally filing bullshit to the court can get you some
| penalties indeed.
| zapdrive wrote:
| He swore under oath that everything in the affidavit is
| correct. So he lied under oath, means he committed perjury.
| Where I live, perjury has a 3 year jail sentence. However
| it is very rarely enforced.
| woodruffw wrote:
| The standard for perjury is (generally) a standard of
| _belief_ , which is much stronger than factuality.
| Ignorance of what ChatGPT is, even shocking ignorance,
| may not clear that bar.
|
| In other words: I would be very surprised if perjury was
| on the table here. This falls under the kind of basic
| competence and client obligation that the Bar exists to
| address.
| bombcar wrote:
| Without malicious intent it's highly unlikely he will get
| a perjury charge but he may be made an example of.
| woodruffw wrote:
| I don't think disbarment is out of the question (and that
| would indeed send a very strong message), but I disagree
| that this comes anywhere close to a serious risk of
| prison for this lawyer.
| eastbound wrote:
| I suspect a lot of lawyers should be disbarred, GPT aside. They
| don't always _care_ about their customers. I've often been more
| expert than my lawyers in France, and I've seen at least one
| guy going to prison where the lawyer, publicly shamed by a
| dozen of youtubers for not actioning the various correct
| levers, told the excuse that he "only had an hour to review the
| case".
|
| What are we paying for, if the guy spends 4 months in prison
| before the faulty judgement being overruled, if the lawyer says
| he didn't even work on the file.
|
| When you hire a lawyer, you have no guarantee he will work for
| you.
| LelouBil wrote:
| Do you have a name or links for the incident you mention ?
| raverbashing wrote:
| The title of the made up case seems to be "China Southern
| Airlines", which is correct. But it is misquoted on Document 31
| (Order to Show Cause) as China South Airlines
| croes wrote:
| Could be a clever method to make programs like ChatGPT
| untrustworthy for potential clients.
| cratermoon wrote:
| I was able to coax ChatGPT into inventing an entirely fake trend
| in cocktails and distilling, complete with stories about newly-
| opening bars and made-up concoctions (including recipes)
| supposedly created and served at these bars.
| acd wrote:
| I think it should be required to state if a written statement is
| written by a GPT.
|
| "This text section is written by GPT provided by company XYZ"
|
| Then humans would know that a particular part is written by an
| AI.
| GaggiX wrote:
| >Then humans would know that a particular part is written by an
| AI.
|
| Humans would know if a part is written by an AI or not only if
| there is a way to enforce the rule.
| adql wrote:
| Can't bullshit people who specialize at calling the other side's
| bullshit.
|
| Not that there can't be use for it,I'm sure it could work as
| great assisted search engine if it was made to cite sources.
| b1ue64 wrote:
| https://archive.ph/tHuXv
| andrewmg wrote:
| Some context: any litigator will have access to Westlaw or Lexis-
| Nexis to look up and verify cited authorities like cases. It's
| considered bad practice, at best, to cite authorities that one
| has not reviewed--for example, case citations drawn from a
| treatise or article.
|
| As a practical matter, it is inconceivable to me that the
| attorney here, at least upon being ordered by the court to
| provide copies of the cases he cited, did not look them up in
| West or Lexis and see that they don't exist. That he appears to
| have pressed on at that point, and asked ChatGPT to generate them
| --which would take some pointed prompting--was just digging his
| own hole. That, more than anything, may warrant professional
| discipline.
| Reason077 wrote:
| "Estate of Durden v. KLM Royal Dutch Airlines"
|
| _Tyler_ Durden? LOL, you couldn 't make this stuff up. Unless
| you're ChatGPT, obviously.
| julienchastang wrote:
| Related article: "End of the Billable Hour? Law Firms Get On
| Board With Artificial Intelligence, Lawyers start to use GPT-4
| technology to do legal research, draft documents and analyze
| contracts" [0]
|
| Critical thinking skills are more important than ever in the age
| of AI. Used correctly, ChatGPT(4) can sometimes be a huge time
| saver, but you cannot believe all the bullshit it serves you.
|
| [0] https://www.wsj.com/articles/end-of-the-billable-hour-law-
| fi...
| jaclaz wrote:
| Another thread where the poster linked to the relevant Court
| .pdf's:
|
| https://news.ycombinator.com/item?id=36092914
|
| Particularly relevant is the affidavit in which the lawyer tries
| to explain to the Court what happened:
|
| https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
| lionkor wrote:
| Hahaha that second link is brilliant.
|
| "Hm, maybe I should double check ChatGPTs output... Hey,
| ChatGPT, does your output make sense?" - "Yeah, my output
| definitely makes sense". "Are you sure?" - "Yeah".
|
| Well, then.
| namaria wrote:
| >Mr. Schwartz said that he had never used ChatGPT, and
| "therefore was unaware of the possibility that its content
| could be false."
|
| Why did he think it was true, if he had never used it before
| then?
|
| "I found it online" levels of competence here.
| montroser wrote:
| > Mr. Schwartz said that he had never used ChatGPT, and
| "therefore was unaware of the possibility that its content could
| be false."
|
| > He had, he told Judge Castel, even asked the program to verify
| that the cases were real.
|
| These two ideas are incompatible with each other. You can't claim
| that you didn't know to question the source, and then also that
| you questioned the source, even if it was done in the least
| effective possible manner.
| NeoTar wrote:
| Perhaps logically incompatible, but within legal proceedings
| you are allowed to use what is called alternative pleading, or
| alternative defence.
| https://en.m.wikipedia.org/wiki/Alternative_pleading
|
| To quote Richard "Racehorse" Haynes in the Wikipedia article:
|
| "Say you sue me because you say my dog bit you. Well, now this
| is my defense: My dog doesn't bite. And second, in the
| alternative, my dog was tied up that night. And third, I don't
| believe you really got bit. And fourth, I don't have a dog."
|
| So, here the defence is:
|
| * I didn't believe the content could be false,
|
| * Even if it is legally determined that I (beyond a reasonable
| doubt) knew the the content could be false, I asked the program
| to verify that the cases were real.
|
| There are more details in the wikipedia article, but I believe
| this is legally valuable because a defendant is required to
| legally enter a defence and cannot easily change this.
| codeflo wrote:
| I don't even think it's logically incompatible. Isn't this
| just ((A - B) [?] (!A - B)) - B?
| sowbug wrote:
| Perhaps, but it can be confusing if the listener interprets
| it as a witness making a factual statement, leading to the
| rhetorical question "were you lying when you said A, or
| else when you said not-A?"
|
| For legal arguments, it's more like "we contend that you
| can't prove A, and even if you can, you can't also prove B,
| and both A and B must be proven for legal liability." Which
| most people can understand isn't inconsistent at all.
| That's why the legal & ethical guidelines spell it out.
| AnthonyMouse wrote:
| What this comes from is that sometimes you claim something
| that you can't prove.
|
| Suppose you got rid of your dog a month ago. If that's true
| then the non-existent dog certainly didn't bite anyone. But
| you still have dog food and leashes and there is a dog
| registered to you, so that may be tough for you to prove.
| Even if it's true.
|
| But now suppose you can establish that there was a fresh
| pie on the doorstep when the plaintiff claims to have been
| there getting bit by your dog. If there was a dog loose in
| your yard at the time, the dog would have eaten the pie.
| Since that didn't happen, if you had a dog then it must
| have been tied up. It's also perfectly consistent with you
| not having a dog, but it doesn't help you prove that
| because it's equally consistent with you having a dog that
| was tied up.
|
| The reason this makes people uncomfortable is that the
| system is supposed to work, but you can easily imagine a
| case where you in fact don't have a dog but there was also
| no pie, so the only way for you to win is to establish the
| thing the jury disbelieved. People don't want to have to
| conclude that the system would arrive at the wrong outcome
| in that case, therefore how dare you claim you don't have a
| dog when there is some evidence that you do.
| shalalalaw wrote:
| We're a very tech forward law firm, and we're bullish on AI. The
| issue is that lawyers are traditionally tech illiterate, and they
| treat Gen AI like a search engine that puts results in narrative
| form. Realistically, I think AI generated motions and contracts
| are the future, and this instance will be looked at by every tech
| averse lawyer to try and stymie progress in the field. These
| lawyers deserve their sanctions for being so reckless with things
| they don't understand, but rather than take away that lawyers
| need to learn tech, lawyers will say tech is bad. I almost wish
| this was non-news so it wouldn't further push the legal industry
| into the past, but those clients were wronged and I guess people
| need to know what to beware of when hiring a lawyer.
|
| Personally, we get really good results from using AI, it's
| already present in all of our processes, but we tell it what to
| generate, rather than rely on it to know better.
| ouid wrote:
| >ai generated contracts are the future.
|
| Man, why do you guys get paid so much? Contracts need semantic
| correctness, ie the verification of the logical consequences of
| _natural language_. This is an AGI _determining_ problem. At
| the point this exists, humans are basically obsolete as
| workers, and you don 't have to worry about your law firm
| keeping up anymore.
| Zafira wrote:
| > [L]awyers are traditionally tech illiterate
|
| This is the main reason I think disbarment as the punishment in
| this specific instance may not be fair. There are people who
| are unaware of the limitations of these systems and the risk of
| these confabulations occurring.
|
| While I don't think disbarment is inappropriate, I would rather
| see the New York State Bar use this to require some better
| understanding of these emergent technologies or even better
| have all the State Bars start discussing some standardized
| training about this because it's easy to see a person trying to
| treat this as LexisNexis.
| NoMoreNicksLeft wrote:
| Is disbarment about fairness? Is the primary goal of such
| proceedings to rehabilitate and apply a sort of justice?
|
| Certainly, civil and criminal courts have those as their
| raison d'etre. But I thought licensing boards had an entirely
| different purpose. If I surgeon was a good guy who genuinely
| wanted to help people and who didn't engage in any sort of
| malfeasance... but even so, he just kept slicing aortas open
| accidentally through incompetence, the board should say "aw
| shucks, he's had some bad luck but he really wants to heal
| people".
|
| This is the same. The court system is replete with
| circumstances where a client does not get a second chance at
| pursuing justice. A lawyer that fucks that up, even if doing
| so in good faith, leaves them with zero remedies. This might
| have been a bullshit "Slippin' Jimmy" case this time, but the
| stakes could've easily been higher.
|
| I don't think I want to live in a world where fairness plays
| any part in the decision by the bar on this matter.
| Zafira wrote:
| Disbarment is usually considered a punishment of last
| resort. That the failure of the attorney to carry out their
| obligations is so absolute, that it justifies taking away
| their right to practice law in a given state. There are
| certainly other measures that can be done here that are of
| a similar rebuke, just not as final. A suspension or
| temporary disbarment is also possible.
|
| We don't know the full situation here, but a personal
| injury case against a bankrupt airline for striking someone
| in the knee with the serving cart seems remediable?
|
| Disbarment usually happens in cases where attorneys fail to
| file timely repeatedly at the expense of their clients and
| after multiple admonishments to stop that; utterly fail in
| their fiduciary obligations (i.e. they were acting like an
| escrow and then instead gambled the money away in Vegas).
| NoMoreNicksLeft wrote:
| > that it justifies taking away their right to practice
| law in a given state.
|
| This seems a little weird. As far as I understand it, no
| one has a right to practice law.
|
| There is a privilege that can be acquired, it one meets
| the requirements. If you somehow got through without
| meeting those, or if you start to fail to meet those...
| time for a new career.
|
| > We don't know the full situation here, but a personal
| injury case against a bankrupt airline for striking
| someone in the knee with the serving cart seems
| remediable?
|
| I don't know about this particular case, but many cases
| and circumstances can be a "one shot at it" scenario. You
| fuck it up, it's tossed and you can't refile. There are
| many reasons and details, any of which might be messed up
| by a lawyer relying on a silly chat program to draft
| motions. One might miss an absolute deadline. It might be
| dismissed with prejudice. Appeals might be exhausted.
| This could even be true of the case in question.
|
| In some cases, it might even be true if it was a criminal
| trial and your defense attorney was incompetent, that you
| don't get a chance to appeal. In California, I think,
| those are Marsden cases (someone correct me if I'm
| wrong). For those, you have to raise an objection during
| the trial.
|
| So, if someone found out that ChatGPT gave their lawyer
| bad advice the day after their conviction... well, oops.
| No appeal for you.
|
| I'll say it again. I do not want to live in a world where
| law license proceedings are decided on a "what's fair to
| the bad lawyer" basis. No one has a right to be a lawyer,
| if you're bad at it there are plenty of other occupations
| you might make a living with where incompetence doesn't
| threaten so many lives and livelihoods.
| ryandrake wrote:
| If your doctor asked ChatGPT to tell him how to remove your
| appendix, followed the directions, and subsequently removed a
| kidney instead, would you want him to lose his medical
| license?
| Negitivefrags wrote:
| For sure, but the difference there is that someone was
| actually severely wronged. The worst that happened here was
| some people had time wasted.
|
| I think a punishment where the lawyer had to pay for all
| the time he wasted for the judges and various legal clerks
| (and his client) would be sufficient personally.
|
| He is unlikely to make the same kind of mistake again I
| would think.
| cratermoon wrote:
| > The worst that happened here was some people had time
| wasted.
|
| At least one party to the suit, if not both, are going to
| end up spending extra money. Plus it wasted public
| resources - the time of the judge and court staff, and
| their salaries, and more - and cost taxpayer money. Your
| remedy of having the lawyer pay might bankrupt him, and
| it doesn't really make whole the other party. In cases
| where one party has limited financial resources or
| perhaps is close to death (thing capital punishment, or
| malpractice), this isn't just waste. Someone _could_ be
| severely wronged.
| nocoiner wrote:
| Out of curiosity, why do you think AI generated contracts are
| the future? Do you draw a distinction between contracts
| generated by AI and, say, contracts "generated" by first-year
| associates (i.e., using precedent to generate a first draft
| appropriate for the deal that's then iterated by more
| experienced lawyers)?
|
| Also, how is this incident going to push the legal industry
| further into the past? Do you think lawyers are going to, like,
| stop using email because of this?
| codeflo wrote:
| It's not just lawyers who think that ChatGPT is a search
| engine. I've observed this many times in my vicinity, people
| from all walks of life think that Star Trek is here and
| computers now respond accurately to natural language queries.
| For non-techies, "just asking the computer" is so much more
| convenient than translating your question into traditional
| search queries.
|
| So I guarantee you that stuff like this is happening daily
| across all industries. Depending on the profession, people will
| lose money or get hurt as a result of someone blindly trusting
| this technology. I can't prove it, but statistically, that's
| basically a certainty.
|
| In my opinion, you can't overstate the importance of articles
| like this, which point out the limitations and highlight the
| dangers. I'm also against banning. But lay people need to be
| informed what ChatGPT is and is not, and OpenAI won't do it
| because they want to ride the hype train.
| gpm wrote:
| In local-to-me politics we have a report on changing the
| admissions process for specialty programs in the public
| school system - which had a bunch of fake citations and
| people suspect was written with the "aid" of ChatGPT.
|
| https://www.thestar.com/news/gta/2023/05/26/tdsb-fires-
| resea...
| [deleted]
| cratermoon wrote:
| Computer, what's the formula for transparent aluminum?
| Seriously, I got ChatGPT to spit out a scientific-seeming
| paper on the formula and manufacturing process for
| transparent aluminum. It did note that there's a real thing,
| aluminum oxynitride, which is the closest thing we have to
| the Star Trek material. It even wrote the following abstract,
| based on my prompt:
|
| > This scientific description provides an overview of the
| formula and manufacturing process of transparent aluminum, a
| material used in applications where both structural strength
| and transparency are required. Transparent aluminum finds
| extensive use in diverse fields, including public aquaria,
| where it allows for the display of large marine organisms.
| The description outlines the chemical composition, key
| properties, and the manufacturing steps involved in creating
| transparent aluminum.
|
| Whether or not the six-step manufacturing process it came up
| with is correct or not, I haven't the expertise to say.
| Xenoamorphous wrote:
| > It's not just lawyers who think that ChatGPT is a search
| engine.
|
| Let's not forget that Google often puts incorrect information
| in their snippets/factboxes or whatever they call them.
| Der_Einzige wrote:
| ... retrieval augmented search is here today and is available
| in ChatGPT with plugins or integrations with vectorDB systems.
| A lot of AI systems are "search engines that give you narrative
| outputs"
| drumhead wrote:
| You sound like an AI generated post.
| SanderNL wrote:
| Lawyer uses completely new tech without so much as glancing over
| its properties and failure modes, also known as being completely
| incompetent.
|
| Breaking news: the results are not fantastic.
| isp wrote:
| I've been following this from legal circles
|
| Original court documents:
| https://www.courtlistener.com/docket/63107798/mata-v-avianca...
|
| The lawyer didn't only cite "bogus" cases, but when challenged
| attached entire "bogus" case contents hallucinated by ChatGPT
| (attachments to #29 on link above)
|
| In the second #32 affidavit, there are screenshots of ChatGPT
| itself!
| https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
|
| A legendary example for the legal risks of hallucination in LLMs
| https://en.wikipedia.org/wiki/Hallucination_(artificial_inte...
| asmithmd1 wrote:
| Thanks for the great context. The lawyer should be disbarred.
| He doubled down when he was caught, and then blamed chatGPT.
| What do you bet he was trying to settle _really_ quickly to
| make this all go away.
|
| Here is the direct link to the chatGPT hallucination the lawyer
| filed in response to the judge's order to produce the actual
| text of the case:
| https://www.courtlistener.com/docket/63107798/29/1/mata-v-av...
| BaseballPhysics wrote:
| Did he "double down" or did he genuinely not understand that
| ChatGPT was making stuff up the whole time?
| microtherion wrote:
| The original bogus citations may be excusable as a genuine
| misunderstanding of ChatGPT, i.e. he falsely thought he had
| a research assistant feeding him accurate quotes.
|
| But there is simply no good faith excuse for filing the
| transcripts of the cases without as much as skimming them,
| once doubts had been raised. I'm not a lawyer, but even a
| cursory look at the Varghese case transcript shows that
| it's gibberish: The name of the plaintiff changing
| inexplicably, the plaintiff filing bankruptcy (of two
| different kinds) as a tactical move, etc. Another
| transcript purports to be about a passenger suing an
| airline over being denied an exit row seat. As soon as you
| start reading the "transcripts", you see that something is
| seriously off about them, compared to the two real (but
| irrelevant) cases cited.
| NoMoreNicksLeft wrote:
| I think the big question is... what was this guy doing 2
| years ago? Was his stuff real work, or was he finding a
| less sophisticated way of phoning it in?
|
| It seems improbable that someone who did all the hard work
| and knew how to do it would suddenly stop doing that. Such
| work ethics tend to be habit-forming, or so I had thought.
| mnd999 wrote:
| Doesn't matter. He is responsible for what he files.
| woodruffw wrote:
| It matters in terms of remediation: incompetence implies
| that lawyers require better technical education on LLMs,
| while malice implies that the lawyer has violated an
| already established rule or law.
|
| Lawyers undergo continuing legal education throughout
| their careers; in many (most?) jurisdictions, it's
| mandatory. "LLMs are not legal search engines" as a CLE
| topic in the next decade would not surprise me remotely.
| adriancr wrote:
| Either way it's inexcusable, they should be disbarred if
| they are this incompetent.
| bbarnett wrote:
| Understand, some lawyers finished last in class. Cramming
| for a bar exam != intelligence.
|
| Don't let one dumbass, be a example of how all lawyers
| are.
| giardini wrote:
| Yes but be merciful to an unfortunate fool who believed
| in technology! ChatGPT proved, like the Ouija board, to
| be the very voice of Satan himself for this lawyer.
| Bwahahahaaaaaah!8-)
| asmithmd1 wrote:
| Of course I do not know, but he should have come clean.
| "Hey, I can't find this case in WestLaw, but chatGPT found
| it and produced it". Instead he just submitted it as-is
| right out of chatGPT. Alarm bells had to be going off in
| his mind that a federal court decision in a lawsuit was
| less than 5 pages
| Vespasian wrote:
| Forward thinking for him to try out ChatGPT for his work.
| Nothing wrong with experimenting with a potential helpful
| tool.
|
| But just as I review and correct code snippets it
| produces, he should have verified the results because
| nothing indicated to him that they are any good (besides
| the fact that they were well written).
|
| I'm pretty sure plenty of other lawyers are experimenting
| with ways to use ChatGPT without being quite as naive.
|
| This is 100% on this guys uncritical thinking.
| dirkt wrote:
| There are plenty of people on the internet (including here)
| who think ChatGPT is a "smart expert" and who don't
| understand that ChatGPT can easily make up stuff that looks
| very convincing at first glance.
|
| And if you challenge them, they also double down and say
| "ChatGPT is the future" etc.
|
| So the lawyer is not alone...
| intelVISA wrote:
| So you're telling me GPT lied when it disproved Church-
| Turing using only 3 lines of Coq?
|
| What purpose is a machine that cannot be trusted? If I
| wanted that I'd use the cloud...
| isp wrote:
| The above link wasn't the only hallucination(!)
|
| The lawyer kept digging the hole deeper and deeper, and (as a
| non-expert) I agree that it seems that the lawyer is at
| serious risk of being disbarred.
|
| Interesting documents are from #24 onwards:
|
| - #24 (https://storage.courtlistener.com/recap/gov.uscourts.n
| ysd.57...): "unable to locate most of the case law cited in
| Plaintiff's Affirmation in Opposition, and the few cases
| which the undersigned has been able to locate do not stand
| for the propositions for which they are cited"
|
| - #25 (https://storage.courtlistener.com/recap/gov.uscourts.n
| ysd.57...) & #27 (https://storage.courtlistener.com/recap/gov
| .uscourts.nysd.57...): order to affix copies of cited cases
|
| - #29: attached the cases - later revealed to be a mixture of
| made up (bogus) for some, vs irrelevant for others
|
| - #30 (https://storage.courtlistener.com/recap/gov.uscourts.n
| ysd.57...): "the authenticity of many of these cases is
| questionable" - polite legal speak for bogus. And "these
| cases do exist but submits that they address issues entirely
| unrelated to the principles for which Plaintiff cited them" -
| irrelevant. And a cutting aside that "(The Ehrlich and In re
| Air Crash Disaster cases are the only ones submitted in a
| conventional format.)" - drawing attention to the smoking gun
| for the bogus cases
|
| - #31 (https://storage.courtlistener.com/recap/gov.uscourts.n
| ysd.57...): an unhappy federal judge: "The Court is presented
| with an unprecedented circumstance. A submission filed by
| plaintiff's counsel in opposition to a motion to dismiss is
| replete with citations to non-existent cases. ... Six of the
| submitted cases appear to be bogus judicial decisions with
| bogus quotes and bogus internal citations" ---- this PDF is
| worth reading in full, it is only 3 pages & excoriating
|
| - #32 affidavits, including the ChatGPT screenshot
|
| - #33 (https://storage.courtlistener.com/recap/gov.uscourts.n
| ysd.57...): an even more unhappy judge: invitation for the
| lawyer & law firm to explain why they "ought not be
| sanctioned"
| asmithmd1 wrote:
| A dry quote from the defendant in #24 above:
|
| "Putting aside that there is no page 598 in Kaiser
| Steel..."
| [deleted]
| _-____-_ wrote:
| That the screenshots are from the mobile website for some
| reason makes this look even worse.
| spondylosaurus wrote:
| It does. It's like this lawyer is charging you his hourly
| rate just to prompt ChatGPT while he's on the toilet.
| BSEdlMMldESB wrote:
| [flagged]
| bandyaboot wrote:
| I'm quite amused that ChatGPT hallucinated a frivolous lawsuit
| brought by someone who was denied an exit row seat.
| m348e912 wrote:
| This problem will be solved with LawGPT which will be a
| generative pre-trained transformer trained on a vast array of
| legal cases, rulings, and briefings and designed with accuracy in
| mind and free from hallucinations.
|
| It will cost $3,400/month for a legal firm the size of Levidow,
| Levidow & Oberman, P.C.
| blibble wrote:
| and it'll still generate bullshit
|
| the technology is not capable of knowing when it is wrong
| karaterobot wrote:
| It's not always clear to me why people downvote things on HN,
| but a lot of the time I find it's because they misunderstood
| them.
| ChuckNorris89 wrote:
| Still waiting for SaulGoodmanGPT
| fhd2 wrote:
| Would probably lead to an uptick in tourism for Belize.
| hammock wrote:
| Write an amicus brief for a Supreme Court case in the style
| of Saul Goodman
|
| UNITED STATES SUPREME COURT
|
| Saul Goodman's Legal Advocacy Clinic 123 Shady Lane
| Albuquerque, NM 87102 Phone: (555) 123-4567 Email:
| saulgoodman@lawyerextraordinaire.com
|
| May 27, 2023
|
| JUSTICES OF THE SUPREME COURT
|
| Re: Amicus Curiae Brief - Case No. 12345
|
| Dear Honorable Justices,
|
| I, Saul Goodman, attorney-at-law and mastermind of legal
| brilliance, on behalf of the humble but feisty David
| Rosencrantz, submit this amicus curiae brief to the esteemed
| Supreme Court. This Court, known for its unflinching pursuit
| of justice, now has the opportunity to rectify a grave
| miscarriage of justice perpetrated by the lower courts.
|
| SUMMARY OF ARGUMENT
|
| The case before this honorable Court presents a classic tale
| of a downtrodden individual, David Rosencrantz, pitted
| against the formidable Goliath that is the United States
| government. Rosencrantz, a simple taxi driver trying to earn
| an honest buck in this dog-eat-dog world, has been unjustly
| accused of tax evasion. However, this Amicus respectfully
| submits that the government's case is nothing but smoke and
| mirrors, an elaborate scheme to crush the dreams of an
| honest, hardworking citizen.
|
| ARGUMENT
|
| I. Violation of Due Process
|
| The lower courts, in their zealous pursuit of victory,
| trampled upon the sacred principles of due process. Mr.
| Rosencrantz was denied his fundamental right to a fair trial
| when the government's star witness, a notorious jailbird with
| a penchant for singing, was allowed to testify without any
| corroborating evidence. The government's reliance on the word
| of a jailhouse informant, who likely had ulterior motives, is
| an affront to the Constitution.
|
| II. Reasonable Doubt
|
| Ladies and gentlemen of the Court, this case is riddled with
| reasonable doubt like a cheap suit in a thrift store. The
| government's case rests on flimsy evidence and a convoluted
| web of speculation. The alleged tax discrepancies are based
| on questionable calculations and fuzzy math, all while
| conveniently ignoring legitimate deductions and expenses
| rightfully claimed by Mr. Rosencrantz. Reasonable doubt casts
| its long shadow on this entire proceeding, and it is the duty
| of this Court to bring it into the light.
|
| III. Selective Prosecution
|
| This Court has never turned a blind eye to injustice, and I
| implore you to scrutinize the government's motives. Mr.
| Rosencrantz, an unassuming member of society, has become the
| target of selective prosecution. Why, might you ask? It's
| because he dared to stand up against the establishment,
| questioning the powers that be. This case is not about taxes;
| it is about silencing dissent and punishing those who refuse
| to play by their rigged rules.
|
| CONCLUSION
|
| In conclusion, the government's case against David
| Rosencrantz reeks of a witch hunt, an attempt to crush the
| spirit of an ordinary citizen who dares to question
| authority. The violation of due process, the presence of
| reasonable doubt, and the specter of selective prosecution
| are grave injustices that cannot be ignored. This Court,
| armed with the sword of justice, must rise above the fog of
| deceit and deliver a resounding verdict in favor of Mr.
| Rosencrantz.
|
| For these reasons, I, Saul Goodman, implore this Court to
| right the wrongs committed by the lower courts and restore
| faith in the sanctity of our legal system.
|
| Respectfully submitted,
|
| Saul Goodman, Esq. Attorney for Amicus Curiae Saul Goodman's
| Legal Advocacy Clinic
| nostrademons wrote:
| Same thing, but paid in Monero.
| seanhunter wrote:
| You don't want a criminal law AI... you want a _criminal_ law
| AI.
| jesusofnazarath wrote:
| [dead]
| simonw wrote:
| I tried pulling together a full timeline from the various
| documents, it's a fascinating story:
| https://simonwillison.net/2023/May/27/lawyer-chatgpt/
| dpifke wrote:
| Previous discussion:
| https://news.ycombinator.com/item?id=36092509
| dpflan wrote:
| Events like this are needed to show to society the flaws of the
| technology and possible misuse. They can then lead to regulations
| or guidelines, or even lawsuits that can further define how and
| when this technology can be used.
| jhncls wrote:
| Even if the lawyer would have carefully checked and corrected
| ChatGPT's answers, isn't he going against his professional duty
| by submitting private information to a website that doesn't
| ensure confidentiality?
| babelfish wrote:
| Don't use GPT for fact finding. Use it for synthesis when you
| already have the facts.
| asmor wrote:
| Or use it for fact _discovery_ if you 're not well-versed in a
| field. But always check if it hallucinated reasonable sounding
| gibberish.
| ttul wrote:
| Or use it to retrieve alleged facts from its own output, pass
| those alleged facts to another tool (like a legal search
| engine) for verification, and then get GPT to edit its output
| accordingly...
| whatever1 wrote:
| The thing is that if you are not an expert in the field you
| cannot tell gibberish from legit facts. Especially if the
| writing style and grammar are top-notch.
|
| I know it is a bad bias, but we typically associate good and
| clear writing with legitimacy. Here we have chat GPT, that
| can do exactly that, but spit out complete bs.
| hosh wrote:
| One thing I like about phind.com is that it ties the
| specific assertions to the specific web page it made it
| from. That allows me to check into the sources.
|
| However, like all generative AI, it's good at forming
| narratives, and not many people are aware how powerfully
| influential narrative frames are because people rarely step
| back to examine the frame itself.
| ayewo wrote:
| Yep. People rarely step back to examine how a narrative
| was framed. It is a cognitive bias [1] built into our
| brains.
|
| It takes a lot of mental effort to spot, which is why we
| don't do it often.
|
| 1:
| https://en.wikipedia.org/wiki/Framing_effect_(psychology)
| raincole wrote:
| If you pay for ChatGPT Pro you can ask it to link to real
| (not hallucinated) web pages from Bing's index.
|
| I'm not promoting it. Just in case you're not aware of
| this feature.
| rhino369 wrote:
| How do you do that? Is there a tutorial?
| babelfish wrote:
| Three years of law school?
| croes wrote:
| Don't take something as fact without at least checking sources.
|
| Couldn't be too hard for court decisions.
| Vespasian wrote:
| Yes he is guaranteed to have access to some case library
| software.
|
| Why he didn't verify the existence of these cases spit out by
| a tool he used for the first time is beyond me. It would have
| taken him a few minutes at most to discover the fake ones and
| skimming the bogus references (assuming some legal
| competence)
| naet wrote:
| In a thread on Reddit I saw some people who were using ChatGPT as
| their personal lawyer around some tech issues, and the community
| seemed to have absolute confidence it was providing good advice.
| I tried to comment that I wouldn't trust GPT as a legal reference
| and got downvoted into the deep negatives.
|
| I'm guessing we will see a lot more of these type of issues.
| ChatGPT is sometimes useful but sometimes full of misinformation
| and many people don't seem to be wary enough of the potential for
| error.
| misnome wrote:
| Unpossible! I'm a 20x coder who spends hours a day writing
| regex, tried asking it a high-school level coding question, and
| it answered perfectly. I'm 100% sure that its just as accurate
| everywhere else.
| [deleted]
| wseqyrku wrote:
| They should've used GPT-4.
| wseqyrku wrote:
| (The article does not say anything about this but I'd guess if
| they are as clueless as they claim to be by calling it "a
| source that has revealed itself to be unreliable", they might
| as well be using the free version without the knowledge of the
| latest version.)
| codeflo wrote:
| It's uncanny how precisely you've put your finger on what the
| actual issue is here.
| entwife wrote:
| I'm only peripherally involved in the law field, and I am aware
| that ChatGPT and similar do not consistently provide correct
| legal citations. It also gets things wrong some of the time; the
| only way to tell is to be an expert or to look up the facts
| yourself. Don't use ChatGPT for legal writing, nor for anything
| else that requires accuracy.
| mountainb wrote:
| When it's correct, it can be a good search aid. But for a lot
| of things, it is just incorrect with high levels of confidence.
|
| You can also ask it Bluebook questions and it will often get
| the right answer. At other times, it will get the right answer
| but cite to the wrong rule (not that it matters that much).
|
| Another issue is that it can cite to the correct case, but
| misunderstand what it is citing to. You can be really specific
| and ask something like "what is the x-factor test from Doe v.
| Doe" and it will get three factors correct and invent the other
| two.
|
| The thing with law, though, is that there are often already
| many quick reference materials that have already been
| extensively published that will get you the answer you are
| looking for more quickly than you can get it through either
| search or a chat interface. Many state bar associations make
| available the equivalent of a "practice area in a box" full of
| checklists, templates, and other material geared towards making
| it possible for you to start working in that area almost
| immediately.
|
| I have had it be useful in course correcting my research in an
| unfamiliar area of law. I was wasting a lot of time reading
| secondary sources and cases that were not relevant to my
| problem because I knew nothing about that area of law and my
| search queries were just leading me in unproductive directions.
| ChatGPT pointed me towards a more relevant case that opened up
| the rest of my research for me using conventional tools like
| Westlaw. It saved me a lot of time. But I did not use it at all
| for the final work product and never used it blind without
| looking at a source.
| visarga wrote:
| That's right, generative model outputs are worthless unless
| checked by a human. You are using it right. For the moment I
| don't think there is any single domain where AI can work on
| its own, autonomy was reached in 0% of fields. That makes me
| think the removal of the human in the loop will take a long
| time. We are still safe, AI will be our sidekick.
|
| It's crazy how AI seems to progress at incredible speed and
| yet we don't get closer to full autonomy anywhere. It's as if
| we discover new problems at the same speed we are solving
| them. Just 5 years ago nobody would think hallucinations will
| become a central issue in AI, we might discover other unknown
| unknowns that hide in our future.
| im3w1l wrote:
| I've been thinking that it's funny how these AI tools are
| framed as assistants, but it seems they are actually the
| opposite. They are great at big picture stuff but sloppy when
| it comes to details. So the more logical division of labor is
| to make the human the assistant.
| squarefoot wrote:
| Could he sue his own lawyer now?
___________________________________________________________________
(page generated 2023-05-27 23:02 UTC)