[HN Gopher] Lawyers blame ChatGPT for tricking them into citing ...
___________________________________________________________________
Lawyers blame ChatGPT for tricking them into citing bogus case law
Author : glitcher
Score : 128 points
Date : 2023-06-09 13:29 UTC (9 hours ago)
(HTM) web link (apnews.com)
(TXT) w3m dump (apnews.com)
| voakbasda wrote:
| If they were able to be tricked by ChatGPT, they are definitely
| not good at being lawyers. Trying to blame the AI is like trying
| to blame MS Word for offering an inappropriate homonym when spell
| checking. The computer did not put the citations in front of the
| judge.
| delfinom wrote:
| The joke is, the judge even pointed out the citations they got
| from ChatGPT literally made no sense. Basically one quote went
| from describing an wrongful death to a legal claim over an
| airline ticket.
|
| It's an understatement to describe them as "not good lawyers".
|
| They basically never ever read their own court filings.
| Dma54rhs wrote:
| Yeah, they just defrauded their clients and billed arm and
| leg. Ironically I believe and hope that laywers for the most
| part get soon automated away by machines because legal
| language is different to human language.
| mysterydip wrote:
| They should ask themselves why we should pay for lawyers at all
| if we can ask ChatGPT the same things and get answers of the
| same quality.
| rank0 wrote:
| These dudes are getting in trouble specifically for the shit
| answers spat out of chatGPT.
|
| Or yeah you could just yolo your freedom away and represent
| yourself using a mathematical expression with zero
| motivations, perception, or emotion.
| mysterydip wrote:
| You missed what I was saying (or I communicated it poorly).
| I'm not suggesting people use ChatGPT for their lawyer.
| They're saying it's ChatGPT's fault that they provided
| wrong info. If they provide no value (no liability for the
| service provided, and no filtering/vetting of ChatGPT
| answers from their lawyer expertise), then why would people
| pay their fee?
| rank0 wrote:
| Oh I see. Yeah 100% agree the people in the article are
| garbage lawyers.
| singlow wrote:
| So there are some bad lawyers therefore all lawyers are
| useless?
| mysterydip wrote:
| No, I'm saying if they think that it's to blame for wrong
| info and not themselves, then why should anyone pay for a
| lawyer instead of asking ChatGPT on their own?
| dylan604 wrote:
| From the sounds of the quality of ChatGPT's legal skills, you
| might be better off representing yourself.
| pengaru wrote:
| It makes sense in a Dunning-Kruger way that lawyers this dumb
| would consider ChatGPT qualified for their purposes.
| seydor wrote:
| Why don't they sue it?
| midnitewarrior wrote:
| "How dare you allow me to use my laziness against myself
| ChatGPT!"
| John23832 wrote:
| People confuse creating a response that makes sense in a language
| (which LLM's are designed to do) with coveying facts and truths
| in a language (which LLM's are not designed to do).
|
| LLM's are revolutionary because they provide a more fluent
| interface to data... but that doesn't not mean that the data is
| correct. Especially not in the early phases.
|
| For most people any sufficiently advance technology is
| indistinguishable from magic. The average joes think that this is
| magic.
| cowmix wrote:
| [flagged]
| pyuser583 wrote:
| Did you write this comment with ChatGPT's help? It's a helpful
| and well written post. I'm not trying to insult it. But I'm
| curious. The text seems to say "AI", and I'm wondering if I'm
| seeing things not there.
| cowmix wrote:
| Yes, in the spirt of talking about ChatGPT I did run my
| response through it. It is probably the worst thing it has
| ever produced for me. I ring this up to ChatGPT's "declining
| quality". :)
| butler14 wrote:
| Great anecdote, thanks for sharing. Forcing ChatGPT to go back
| and sense check its overarching initial interpretation is such
| a good way of doing things in these types of uses cases.
| VoodooJuJu wrote:
| This comment is fraudulent.
|
| Not only is the style quite brazenly ChatGPT, this part is a
| huge red flag:
|
| >ChatGPT uncovered state utilities codes...
|
| ChatGPT cannot uncover anything, especially the municipality-
| specific corpus of codes that the comment ostensibly claims to
| have access to.
| garblegarble wrote:
| >ChatGPT cannot uncover anything, especially the
| municipality-specific corpus of codes that the comment
| ostensibly claims to have access to.
|
| It's possible the training set includes those codes (or a
| citation of those codes). It's possible that they used GPT4
| with Browsing, or that it gave them the right terminology to
| search for & they then pasted in sections of the code and
| asked it to work off that.
|
| It's also entirely possible ChatGPT hallucinated these
| utility code sections to support its case and the people
| working for the utility didn't call their bluff... which is
| essentially exactly what this story is about, except they got
| caught out because ChatGPT's bluff was called...
| cowmix wrote:
| OP here. That's exactly what happened.
|
| This is the code it found:
|
| California Public Utilities Code SS 10009.6
| COGlory wrote:
| This comment is settinh of my ChatGPT alarm bells. Something
| about the sentence structure and adjective usage.
| cowmix wrote:
| Heh, your spidey senses are correct. I actually wrote the
| response and then ran it though ChatGPT for clean up. I do
| agree, it actually reads weird over my original draft.
| weego wrote:
| Their recent comments have a few overly glowing and slightly
| unrealistic scenarios using ChatGPT.
| jkea wrote:
| Last two paragraphs starting with "however" and "ultimately"
| SCREAM chatGPT generated to me
| cowmix wrote:
| As least I didn't add "I hope this response finds you
| well."
| 300bps wrote:
| Just once it would be refreshing for someone who gets caught
| doing something wrong to say, "Wow, ya got me. I'm sorry. Not for
| doing it, I'm only sorry I got caught. I really thought I'd skate
| right by on this one. Legitimately won't do it again. Or if I do
| I'll proof-read it better at least. Please let me know the
| punishment and I'll accept it."
| gizmo686 wrote:
| That happens all the time. It just isn't newsworthy.
| Avicebron wrote:
| I mean isn't the punishment his livelihood? And whoever's
| livelihood when someone looks under the hood and they realize a
| lot of people (not just lawyers) are doing this all over the
| place?
|
| And everyone thinks they're too smart to get caught...until
| they aren't..
| pixl97 wrote:
| Yep, admitting would be immediate disbarment. With the proper
| amount of bullshit you might just be able to escape
| disbarment.
| jamesliudotcc wrote:
| I used to be a lawyer. While I never got in trouble with
| the disciplinary commission, I did keep up with what they
| were up to. And once, I was involved in a bankruptcy case
| where they were a party (the disbarred lawyer filed
| bankruptcy, and it was a whole mess).
|
| My sense is that these are serious people who do not put up
| with BS. BS will only make it worse.
| jcranmer wrote:
| While the lawyers blamed ChatGPT, the totality of the
| circumstances seem to indicate that they're less than honest in
| doing so. There is a live-tweet of the hearing here:
| https://twitter.com/innercitypress/status/166683852676213965...,
| and you can follow along with the lawyerly cringe there.
|
| Okay, lawyer #1 (LoDuca, the one on the case in the first place)
| appears to have played essentially no role in the entire case;
| his entire purpose appears to be effectively a sockpuppet for
| lawyer #2 (Schwartz), as LoDuca was admitted to federal court and
| Schwartz was not. He admits to not having read the things
| Schwartz asked him to file, as well as the "WTF?" missives that
| came back. He lied to the court about when he was going to be on
| vacation, because that is when Schwartz was on vacation. But
| other than doing nothing when he was supposed to do something
| (supposed to do a lot of somethings), he is otherwise uninvolved
| in the shenanigans.
|
| So everything happened because of Schwartz, but before we get to
| this part, let me fill in relevant background information. The
| client is suing an airline for an injury governed by the Montreal
| Protocol. Said airline went bankrupt, and when that happened, the
| lawyer dismissed the lawsuit, only to refile it when the airline
| emerged from bankruptcy. This was a mistake; dismissing-and-
| refiling means the second case is outside the statute of
| limitations. The airline filed a motion to dismiss because, well,
| outside statute of limitations, and it is Schwartz's response
| that is at the center of this controversy.
|
| What appears to have happened is that there is no case law to
| justify why the case shouldn't be dismissed. Schwartz used
| chatGPT to try to come up with case law [1] for the argument. He
| claims in the hearing that he treated it like a search engine,
| and didn't understand that it could come up with fake arguments.
| But those claims I'm skeptical of, because even if he's using
| chatGPT to search for cases, he clearly isn't reading them.
|
| When the airline basically said "uh, we can't find these cases,"
| Schwartz responded by providing fake cases from chatGPT where
| alarm bells should be ringing saying "SOMETHING IS HORRIBLY,
| HORRIBLY WRONG." The purported cases in the reply had blindingly
| obvious flaws that ought to have made you realize something was
| up before you're off the first page. It is only when the judge
| turns around and issues the order to show cause that the lawyers
| attempt to start coming clean.
|
| But wait, there's more! The response was improperly notarized: it
| had the wrong month. So the judge asked them to provide the
| original document before signature to justify why it wasn't
| notary fraud. And, uh, there's a clear OCR error (compare last
| page of [2] and [3]).
|
| When we get to these parts of in the hearing, Schwartz's
| responses aren't encouraging. Schwartz tries to dodge the issue
| of why he was citing cases he didn't read. Believing his
| responses of why he thought the cases were unpublished ("F.3d
| means Federal district, third department") really requires you to
| assume he is an incompetent lawyer at best. The inconsistencies
| in the affidavit are glossed over, and the stories don't entirely
| add up. It seems like a minor issue, but it does really give the
| impression that even with all the attention on them right now,
| they're _still_ being less than candid with the court.
|
| The attorneys for Schwartz are trying hard to frame it as a "he
| didn't know what he was getting into with chatGPT, it's not his
| fault," but honestly, it really does strike me that he _knew_
| what he was getting into and somehow thought he wouldn 't get
| caught.
|
| [1] His conversation can be found here:
| https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...,
| it's one of the affidavits in the case.
|
| [2]
| https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
|
| [3]
| https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
| isp wrote:
| Excellent comment, thank you.
|
| https://twitter.com/innercitypress/status/166683852676213965...
| is worth reading in full.
|
| I am not a lawyer, but I was cringing throughout.
|
| For non-lawyers (like me), I found it helpful to check out the
| quote tweets.
|
| For example, to explain how much of a cringe was "F.3d" -
| https://twitter.com/ReichlinMelnick/status/16668536501095424...
|
| > OMG. This is the Chat-GPT lawyer. You learn what F.3d means
| in first year legal writing. To non-lawyers, this is a tiny bit
| like someone being asked "You know what the DOJ is, right" and
| getting something wild like "Directorate of Judges" as a
| response.
|
| The correct meaning of "F.3d" is "Federal Reporter, 3rd
| Series".
|
| And it is like a lawyer's worst nightmare to have an exchange
| with a judge like this:
|
| > Judge Castel: Have you heard of the Federal Reporter?
|
| > Schwartz: Yes.
|
| > Judge Castel: That's a book, right?
|
| > Schwartz: Correct.
|
| The implication being that at bare minimum, the lawyer could
| have looked it up in a book. Like a first year law student
| would.
| jamesliudotcc wrote:
| Hilariously, the explanation of the notarization typo
| (actually, I find that believable) is a 1746 "declaration."
| It's an old federal law which provides that as long as you say
| you are signing under oath, it is as good as an affidavit in
| federal court.
|
| Why didn't he just make a declaration in the first place? Also,
| why would he have the lawyer who is in trouble notarize the
| document? Now that he has made the typo, he may now need
| Schwartz's testimony that actually, it was April 25, not
| January.
|
| In my legal career, I worked for a judge once. He told me to
| never get a notary stamp. It only creates problems. There's
| never a good reason for a lawyer to notarize something. You ask
| your staff to do it instead.
| rank0 wrote:
| Disbarment is appropriate here.
|
| Why not blame your laptop manufacturer for creating the hardware
| you used to file your fraudulent court documents?
| CPLX wrote:
| No it's not.
|
| You'll get old one day. It's pretty challenging to keep track
| of what's real and what isn't possible with technology.
|
| The guy apologized and said he thought it was a search engine.
|
| He should definitely face some sanctions but someone had to
| learn this lesson in public the hard way for word to spread.
| code_runner wrote:
| These exemptions we are passing out are insane. This guy
| claims he thought it was a search engine?
|
| Lawyers use specific case law databases and they SHOULD
| approach any new tools with healthy skepticism.
|
| And if he googled it and cited a fake case would it be
| better? Why wouldn't you vet the information.
| CPLX wrote:
| Yeah it was bad.
|
| Disbarment is equivalent to taking away his entire life's
| work.
|
| Two minute hate on the internet every day is fun and all
| but it's not that severe.
| vkou wrote:
| > Disbarment is equivalent to taking away his entire
| life's work.
|
| 'Lawyer' is the kind of profession which actually holds
| its practitioners to a standard, because the system falls
| apart when they stop behaving honestly.
|
| This wasn't an oopsie-daisies, this was dishonesty,
| followed by _further_ dishonesty, when they supplied
| bogus references for these cases.
| CPLX wrote:
| But it wasn't dishonesty it was total obliviousness,
| maybe negligence.
| Ekaros wrote:
| He failed his duty to both client and the court. This was
| wilful negligence.
|
| This is like surgeon doing wrong procedure he just
| invented on patient.
| cj wrote:
| Is this case different from the same incident a couple weeks
| ago? (also on the front page here)
| FpUser wrote:
| >"The guy apologized and said he thought it was a search
| engine."
|
| I am also having a thought: stop bullshitting. Or stop being
| a lawyer as a result of gross incompetence (I wish the same
| applies to politicians).
| spuz wrote:
| The lawyers not only cited bogus cases, but when asked to
| provide copies of those bogus cases, fabricated multiple page
| PDF documents from whole cloth. This is impossible to argue
| as a mistake.
|
| Here is an example of one of the fabricated cases. The 11th
| circuit has no record of this case. 1 of the 3 judges named
| in the case was not on the 11th circuit at the time: https://
| storage.courtlistener.com/recap/gov.uscourts.nysd.57...
| regulation_d wrote:
| The counterpoint here is that there is already a cause of
| action for this type of incompetence and it's called
| malpractice, which is a pretty reasonable road to remedy. I
| don't know if you actually think these were "fraudulent court
| documents", but "fraudulent" actually means something very
| specific and this ain't it. Even if the court is considering
| sanctions (which is not the same as disbarment), that seems at
| least partially related to the attys' failure to address their
| failure once they were aware of it.
|
| Something interesting about the legal profession is that it is
| self-regulating. The state bars are typically not government
| organizations. Attorneys know that confidence in their
| profession is extremely important and they strike the balance
| between preserving that confidence and, you know, destroying
| someone's livelihood because they don't understand how LLMs
| work.
| flangola7 wrote:
| Disbarment doesn't happen for much, much worse actions.
| gumballindie wrote:
| > Why not blame your laptop manufacturer for creating the
| hardware you used to file your fraudulent court documents?
|
| Because your laptop manufacturer doesnt claim your laptop
| "thinks", is "intelligent", doesnt build an entire fud
| marketing campaign around the two, doesnt claim it "creates"
| ideas on its own, doesnt claim it "learns like a human", doesnt
| claim it has cognitive abilities and so on.
| [deleted]
| IIAOPSW wrote:
| Thats weird. I tried really hard to convince it that there's such
| case law establishing a legal category of "praiseworthy homicide"
| and it refused to believe me. I thought it was overttrained /
| patched on all law related applications.
| hackerfactor1 wrote:
| A poor workman blames his tools.
| 6gvONxR4sf7o wrote:
| I don't understand how they messed this up so bad. They say they
| didn't know it could hallucinate and that they thought it was
| just like any other search engine. But it seems like even if it
| worked like they thought, they'd still have fucked up?
|
| If it's just like a normal person, if that person isn't a lawyer,
| you wouldn't ask them to do your lawyery work. I'd hope this
| lawyer doesn't his kids to do his work for him.
|
| If it's just like a normal search engine, we all know how much
| bullshit, spam, and misinformation there is on the internet
| (mostly written by normal good old fashioned humans!). So that
| wouldn't have been trustworthy either!
|
| There's no way this kind of thing is excusable.
| sys42590 wrote:
| When you start giving ChatGPT a plugin to query LexisNexis and do
| proper citiations (as Bing Chat does), then things get
| interesting.
|
| Unfortunately Lexis's API fees are currently quite steep, so only
| very wealthy law firms will be able to afford to use such a
| service in the short term.
| RugnirViking wrote:
| You'll still run into the same problem these guys did (i.e. not
| checking the citations). It's easy enough to just search to see
| if a thing it tells you is real.
| SparkyMcUnicorn wrote:
| I'm currently building something for a client that does this.
|
| 1. Searches for relevant documents.
|
| 2. Generate a response based on the found documents, using
| temperature=0 and a prompt that instructs the response to
| include a citation reference in a specific format.
|
| 3. Display the result, linking directly to the sources of the
| citations, and a warning on any that don't actually exist
| (which hasn't happened yet).
| plorg wrote:
| ...Until search is subsumed by chatbots and all you can
| access is a commercial statistical model's rendering of the
| truth, digested from ever more abstract renderings of the
| primary sources, which probably exist somewhere but are
| impractical to find under the mountain of re-written LLM
| spam.
| [deleted]
| activiation wrote:
| Surely CGPT can tell them how to get out of this one.
| xbar wrote:
| Misrepresenting wholly fabricated case law to a judge out of
| sheer incompetence has been grounds for disbarment in at most US
| States for over a century.
|
| ChatGPT told me that. It might be true.
| cyanydeez wrote:
| Unfortunately, technology washing is the latest trend in
| minimizing responsibility for outcome.
| ryandrake wrote:
| Sounds like these lawyers need to be... DisBard? Sorry, it's
| too early for AI puns.
| kkielhofner wrote:
| Bingo.
|
| With or without ChatGPT this is easily malpractice and
| potentially even fraud. My (cynical) guess is not only are they
| lazy and sloppy, they likely (fraudulently) over billed their
| clients in terms of representing billable hours spent drafting
| this lawsuit. They're almost certainly billing their clients
| (or in the case of contingency, keeping the cut they normally
| would) as though humans (with high hourly rates) are drafting
| this while having ChatGPT generate their supposed work product
| in seconds.
|
| If you remove ChatGPT from the picture and look at this as if
| it was actually drafted by them the fraud argument strengthens.
| They essentially made up case law and citations that
| artificially (fraudulently) improves their argument before the
| court.
|
| At a minimum it's grossly incompetent and when you consider my
| prior paragraph it strengthens the fraud angle, as they likely
| skimmed over the generated ChatGPT results and submitted it
| because it (again, artificially) strengthens their case. It
| seems as though ChatGPT (with whatever prompts they used) was
| more than happy to prioritize pleasing them vs actually being
| accurate.
|
| They may as well have prompted ChatGPT with "I'm a lawyer and
| tell me anything you need to so I can win this case and take
| home the money". It's a disgrace.
|
| What a mess - these lawyers should be disbarred and
| investigated for what is also likely fraudulent billing
| practices at minimum.
| dylan604 wrote:
| >They're almost certainly billing their clients
|
| If they sent that bill in an envelope with a stamp and placed
| it in the mail...sounds very familiar. Maybe they should ask
| ChatGPT what are the possible outcomes of me using the
| answers you provide
| pessimizer wrote:
| > With or without ChatGPT this is easily malpractice and
| potentially even fraud.
|
| AI has been used and will continue to grow in use as a way to
| launder discrimination and fraud. AI will never face a
| penalty from the justice system, so why not blame everything
| on it?
| pgeorgi wrote:
| This is a case that's close to the judiciary. It might be
| what's needed to nullify the "I was only following the AI"
| defense by requiring a manual double check.
| plorg wrote:
| A lawyer is certainly at fault if they do not fact check the
| material they present at trial. But the conmen who are selling
| ChatGPT and the like are extremely irresponsible for the way they
| sell LLMs as magical AI that arrives at factually correct answers
| by reasoning rather than the consequence of the law of large
| numbers applied to stochastic text generation.
| ignite wrote:
| They literally warn you every time you log in that the results
| may not be accurate.
| Ukv wrote:
| > the conmen who are selling ChatGPT and the like are extremely
| irresponsible for the way they sell LLMs as magical AI that
| arrives at factually correct answers
|
| ChatGPT has a pop-up on first use, a warning at the top of each
| chat, a warning below the chat bar, and a section in the FAQ
| explaining that it can generate nonsense and can't verify
| facts, provide references, or complete lookups.
|
| There is probably more OpenAI could do, like detect attempts to
| generate false references and add a warning in red to that chat
| message - since it seems there are still people taking its
| hallucinations as fact (although if there's hundreds of
| millions of users, maybe only a tiny fraction), but I don't
| think this is a fair characterization.
| meghan_rain wrote:
| in this thread (and all threads on this topic):
|
| angry armchair legalists trying to stick it to The Man!! by
| pretending neglegient homicide is the same as premeditated murder
| isp wrote:
| Related:
|
| - "A man sued Avianca Airline - his lawyer used ChatGPT" (13 days
| ago, 174 points, 139 comments):
| https://news.ycombinator.com/item?id=36095352
|
| - "Lawyer who used ChatGPT faces penalty for made up citations"
| (1 day ago, 106 points, 128 comments):
| https://news.ycombinator.com/item?id=36242462
|
| Original court documents:
| https://www.courtlistener.com/docket/63107798/mata-v-avianca...
| causi wrote:
| Blaming ChatGPT for making stuff up is like blaming sex dice
| after you followed their instructions to "spank" + "hair".
| bioemerl wrote:
| OpenAI wanted to preach safety. I think we hold them liable for
| literally everything, everything Chat GPT does or says, until the
| model is open and they can argue that they have no control over
| it.
|
| They wanted this liability, they accepted this liability, they
| said they'd keep it safe and they haven't. It's on them.
| DtNZNkLN wrote:
| The lawyer said: "I did not comprehend that ChatGPT could
| fabricate cases."
|
| I wonder how many other people using ChatGPT do not comprehend
| that ChatGPT can be a confident bullshitter...
|
| I'm surprised that this one case is getting so much attention
| because there must be so many instances of people using false
| information they got from ChatGPT.
| CPLX wrote:
| Normal people don't get that intuitively because it's not
| intuitive _at all_.
|
| For everyone who his looking down on this guy have you ever
| read a story on the Reddit front page and thought it was a real
| story accurately recounted by a real person and not a work of
| fiction? If so you're equally naive.
| seydor wrote:
| notOpenAI told them that this is so incomprehensibly smart that
| it will destroy all of us. Not because it's dumb and connects
| the wrong cables, but because it's super-smart. People can't
| surmise from that that the model just makes stuff up on the way
| IIAOPSW wrote:
| I prefer to use the term "clopenAI"
| SilasX wrote:
| Or OpenAladeen
|
| https://m.youtube.com/watch?v=NYJ2w82WifU
| JohnPrine wrote:
| OpenAI is clear about the fact that ChatGPT can hallucinate,
| where have they ever said otherwise?
| arbitrarian wrote:
| This is what I don't get. Not only have they not said
| otherwise, but they put it right up front in a pretty easy
| to understand brief message before you start using it. I
| guess lawyers just click agree without reading too.
| jprete wrote:
| People are accustomed to ignoring the fine print as legal
| CYA with no real-world relevance. This is also why the
| product warnings that "The State of California considers
| this to cause cancer" are a joke and not a useful
| message.
| seydor wrote:
| One of the most widely circulated PR about GPT4 is that it
| passed the bar exam
|
| https://www.forbes.com/sites/johnkoetsier/2023/03/14/gpt-4-
| b...
|
| and it's had the most vocal and sensational PR about how AI
| needs regulation now
|
| https://edition.cnn.com/2023/06/09/tech/korea-altman-
| chatgpt...
|
| People assume the smalltext does not apply to them.
|
| openAI does not get a lot of flak from the media for the
| amount of BS that chatGPT can blurt out
|
| Does anyone remember what happened to Galactica which did
| the same thing ? That too was clearly labeled as
| hallucinatory . But it was shut down because they did not
| BS the media enough about regulations and such.
|
| I m afraid these LLMs are turning into too much of a
| political game to be useful for much longer.
|
| On the other hand, if they become political, then people
| will be even more incentivized to build offline, local LLMs
| taco_emoji wrote:
| It is literally just a goddamn language model. it is very good
| at making plausibly human-like sentences. It is not a general
| intelligence, it is not your friend, it is not a research
| assistant. It is not designed to deliver content which is
| _correct_ , it is designed to deliver content which is _similar
| to human language_.
|
| It might get things correct most of the time! But that is
| purely incidental.
| kevin_thibedeau wrote:
| It does subsume a corpus of factual information. I use it as
| a search tool for topics and relationships that traditional
| search engines can't handle. You just have to know that
| whatever it outputs isn't trustworthy and needs to be
| verified.
| jstarfish wrote:
| Part of the corpus is explicit bullshit though, and we
| don't know to what degree. It internalized conspiracy
| theory and WebMD alike. In a generative capacity, it only
| reliably produces fiction. Ever. Fictional stories often
| take place in realistic settings and reference real facts.
| They sound real. But they're still fictional compositions.
|
| Using GPT as a reference to anything is the same as using a
| Michael Crichton novel as a biology reference. It _looks_
| right, but why would you waste your time asking questions
| of something you can 't trust and have to double-check
| everything it says anyway? Nobody would keep an employee
| like that around, nor would you hang out with someone like
| that. It's friendly enough, but it's a pathological liar.
|
| There's too much black magic going on inside the black box.
| We don't know how prompts get tampered with after
| submission, but it might be worth it to pepper "nonfiction"
| tokens in prompts to ensure it skews on the right side of
| things. It certainly responds to "fiction" when you're
| explicit about that.
| dekhn wrote:
| yes but it's still very much just a language model, not a
| knowledge model.
| travisjungroth wrote:
| It is literally just a goddamn electric motor. it is very
| good at converting chemical energy into mechanical energy. It
| is not a universal engine, it is not your horse, it is not
| your servant. It is not designed to create movement which is
| _correct_ , it is designed to create movement which is
| _similar to a piston engine_.
|
| It might move you forward most of the time! But that is
| purely incidental.
| verdagon wrote:
| I believe electric motors _are_ actually intended and
| advertised to create movement which is correct, and come
| with warranties and liability about reliability. (Or maybe
| I missed a joke in here somewhere?)
| seydor wrote:
| now imagine believing that this electric motor can self-
| drive itself to the supermarket
| travisjungroth wrote:
| ChatGPT isn't _just_ an LLM. It's literally not. There's
| a web server, interfaces, plugins, etc.
|
| LLMs are this super powerful thing (like a motor) and
| people are getting to play around with it before it's
| fully harnessed. There's this strange phenomenon where
| because it's not totally harnessed, people just rip on
| it. I don't know if they think it makes them sound smart,
| but it sure doesn't to me. It's like seeing a motor on an
| engine stand and being like "But the crankshaft
| _rotates_. I want to go in a straight line! This isn't a
| transportation solution and anyone who thinks so is just
| naive. And _horsepower_? Stop zoomorphizing it!"
|
| Sure, there are lots of problems. But don't mix up the
| limitations of this component of the system, the current
| limitations of the system overall, and the possible
| capabilities of the system. Someone builds a car that
| stalls when it drives through a river and people point
| out that an ICE engine needs air and how could you even
| think about driving through a river? Then someone else is
| off making a snorkel.
| IIAOPSW wrote:
| "hey chatGPT, can you make up some bullshit for me?"
|
| "I'm sorry, as an online AI language model I am not able to
| generate output that is knowingly incorrect. Is there anything
| else you want help with today".
|
| see, it isn't capable of lying. It told me so itself.
| atmavatar wrote:
| "Let me put it this way, Mr. Amor. The 9000 series is the
| most reliable computer ever made. No 9000 computer has ever
| made a mistake or distorted information. We are all, by any
| practical definition of the words, foolproof and incapable of
| error."
| blitzar wrote:
| "There is no danger that Titanic will sink. The boat is
| unsinkable and nothing but inconvenience will be suffered
| by the passengers." Phillip Franklin, White Star Line vice-
| president, 1912
| belter wrote:
| " Well, I don't think there is any question about it. It
| can only be attributable to human error. This sort of thing
| has cropped up before, and it has always been due to human
| error." - https://youtu.be/CD9YqdWwwdw
| ilyt wrote:
| Inventing stuff out of thin air isn't _technically_ lying
| jakelazaroff wrote:
| You say that as a joke but the lawyer literally asked ChatGPT
| whether the cases it provided were fake! There are
| screenshots! https://simonwillison.net/2023/May/27/lawyer-
| chatgpt/
|
| _Lawyer: Is varghese a real case
|
| ChatGPT: Yes, Varghese v. China Southern Airlines Co Ltd, 925
| F.3d 1339 (11th Cir. 2019) is a real case.
|
| Lawyer: What is your source
|
| ChatGPT: I apologize for the confusion earlier. Upon double-
| checking, I found that the case Varghese v. China Southern
| Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019), does
| indeed exist and can be found on legal research databases
| such as Westlaw and LexisNexis. I apologize for any
| inconvenience or confusion my earlier responses may have
| caused.
|
| Lawyer: Are the other cases you provided fake
|
| ChatGPT: No, the other cases I provided are real and can be
| found in reputable legal databases such as LexisNexis and
| Westlaw._
|
| Why the lawyer didn't go the extra step and check the
| _actually real_ legal databases ChatGPT pointed out is beyond
| me.
| uoaei wrote:
| The new LLMs have been advertised in layman circles often
| as something like "a chatbot with access to all the
| information on the web" or something similarly ambiguous.
| So it is certainly easy to imagine why people think it
| literally goes out and checks its sources by re-accessing
| the webpage and summarizing it for the user. The responses
| you quoted seem to simulate/imply that kind of behavior
| through its natural language presentation.
| jakelazaroff wrote:
| Yeah, I've spoken to people who've had trouble
| internalizing that it can't search the web even after
| I've explicitly told them that. The "upon double-
| checking" message from ChatGPT is especially egregious --
| it's straight-up lying about how it arrived at that
| response. There really should be a layer on top of the
| chat UI to alert the user if it detects a response like
| that.
| jstarfish wrote:
| > Why the lawyer didn't go the extra step and check the
| actually real legal databases ChatGPT pointed out is beyond
| me.
|
| Because that's _work_ and takes _effort._ He gets paid the
| same to delegate the work to AI.
|
| He did the absolute, bare minimum amount of verification
| needed to [hopefully] cover his ass. He just didn't expect
| the system to lie (sorry, " _hallucinate_ ") to him more
| than once.
|
| > [...] the lawyers did not act quickly to correct the
| bogus legal citations when they were first alerted to the
| problem by Avianca's lawyers and the court. Avianca pointed
| out the bogus case law in a March filing.
|
| This is what fraud looks like. He's so checked out he even
| ignored the red flags being waved in his face. It stopped
| being a cute case of a student generating a common essay
| about Steinbeck when he started getting paid $200 an hour
| to cheat an injured client.
| sokoloff wrote:
| > He gets paid the same to delegate the work to AI.
|
| If he was being paid hourly, he would actually get paid
| more to go look up those cases in a database.
| jstarfish wrote:
| Well, yes, but you're assuming good faith in implying
| he's willing to spend his time on it. The point is to
| maximize hours billed while _doing_ as little work as
| possible.
|
| No contractor charges you for 2 minutes of work
| installing a $0.99 part; they pad it every way possible
| with service call fees, labor, etc. Attorneys just lie
| about it altogether since for logical work, you can't
| prove whether or not they actually did anything. It's all
| showmanship. Question them on it and it's all gaslighting
| about how you're not a lawyer and don't know what you're
| talking about.
|
| Sibling comment points out possible contingency basis, so
| if true, he certainly wouldn't want to spend real time on
| a case that may not pay out. But if he can automate the
| process and collect winnings while doing no real work,
| it's a money printer.
| unyttigfjelltol wrote:
| > It stopped being a cute case of a student generating a
| common essay about Steinbeck when he started getting paid
| $200 an hour to cheat an injured client.
|
| It's more likely these lawyers are working on contingency
| and, because they did poor work, will receive nothing for
| it.
| jstarfish wrote:
| Good point!
| lt_kernelpanic wrote:
| He assumed that ChatGPT was under oath, apparently.
| dunham wrote:
| For me it was perfectly willing to:
|
| "produce fake technical language in the style of star trek"
| boredumb wrote:
| I think the case is getting the attention because it's not just
| some one spouting off online, it's a lawyer bumping into the
| legal system with false information that would otherwise be a
| massive legal no-no and they are trying to scape-goat it onto
| the new shiny software.
| Avicebron wrote:
| But they might have seen the shiny new software touted by
| "those smart AI guys as being revolutionary and passing the
| Bar!" and they don't hang out on HN all day so to them that's
| like someone saying "this bridge is sturdy!" and they walk
| over it without realizing they should really go over the nuts
| and bolts of it like a civil engineer to really be sure
| taberiand wrote:
| I just think idiots who touch fire should be burned -
| particularly lazy idiots whose high paid job in fact
| requires them to be extremely careful and precise in their
| actions, and who refuse to take responsibility for their
| actions afterwards.
|
| It's not that they need to inspect every nut and bolt of
| the bridge, they just need to not walk over the bridge - or
| at least, not immediately start driving unreasonably heavy
| loads across it.
| Avicebron wrote:
| >I just think idiots who touch fire should be burned -
| particularly lazy idiots whose high paid job in fact
| requires them to be extremely careful and precise in
| their actions, and who refuse to take responsibility for
| their actions afterwards.
|
| Are we talking about lawyers or the AI researchers?,
| because they certainly want to portray themselves as a
| modern day Prometheus
| elforce002 wrote:
| This is interesting. How long untill someone gets sick because
| he/she was following what chatGPT told him/her to do? Medical
| advice? Political misinformation?
|
| How things will unfold this decade? Banning chatgpt from certain
| topics (medicine, law, etc...)? This decade will be really
| interesting indeed.
| ilyt wrote:
| People did that with googling symptoms for a long time now
| theknocker wrote:
| [dead]
| zzzeek wrote:
| comments here are like "what dumb lawyers". Sure OK. But what can
| we say here about "GPT-4 passed the bar exam!" and how useful is
| that data, given that this does not imply GPT-4 has the actual
| skills of a human lawyer.
| ekam wrote:
| The lawyer here was probably not smart enough to distinguish
| between 3.5 and 4. Haven't seen anything to indicate this was
| the output of GPT-4
| BeetleB wrote:
| I've had Bing Chat (GPT4) hallucinate research studies. It
| even generated links to them in citation indexes.
|
| The authors existed. They do research in the area, so the
| title was very plausible.
|
| But the paper didn't exist.
| zzzeek wrote:
| oh ok! so if he used gpt-4, still might have been illegal but
| the output would have been perfect. good to know
| ekam wrote:
| Your reading comprehension seems to be as good as that of
| the lawyer here. You asked what this says about the claim
| that GPT-4 was good enough to pass the bar. I didn't say
| anything about GPT-4's quality or the legality here, only
| that we cannot assume this was the output of GPT-4 without
| evidence it was given that more people overwhelmingly use
| the default 3.5-turbo.
| BaculumMeumEst wrote:
| "There's simply no way we could have known these were bogus
| cases." the lawyers are quoted as saying.
|
| They are currently using Bard to help draft a lawsuit against
| OpenAI, claiming the company knowingly misrepresents the
| capabilities of their technology.
| IIAOPSW wrote:
| The judge is using bingChat to write the decisions anyway.
| anon25783 wrote:
| I'm so confused. Why do the lawyers not simply check to see if
| the references are real or not? How hard is it to look through an
| LLM's output and do a quick search to see if any of the laws or
| cases mentioned in it are in fact unsubstantiated?
| chrononaut wrote:
| At best an LLM in this case should serve as a "pointer" or
| "hint" to references. It's clear one would still need validate
| the entirety of the case and the arguments put forward from
| them.
| Ekaros wrote:
| Sure use whatever tool to find suitable cases supporting you.
| But also verify that those cases support your position. Better
| not give ammo to your opponent.
| phren0logy wrote:
| Regardless of how hard it is to check the references (which is
| not really all that hard), it's the job of a lawyer.
| bastardoperator wrote:
| They saw ChatGPT passed the bar in the 90th percentile and
| thought they were on easy street. What a dumb way to lose your
| law license.
| seydor wrote:
| but they can use GPT4 to regain it
| summerlight wrote:
| Oh god, I suppose a part of the reason why those lawyers are well
| paid is because if something goes wrong then they're going to be
| responsible...
| jameslk wrote:
| Given that they're attorneys, we know what their next course of
| action may be. I've noticed that ChatGPT's new onboarding
| includes a screen with a big disclaimer where there wasn't one
| before. I can only assume that it may be related to cases like
| these.
| bequanna wrote:
| Would it have been that difficult for the lawyers to actually
| check the case law ChatGPT cited?
|
| Seems like pure laziness.
| ineedasername wrote:
| >[to the judge on behalf of Schwartz] Mr. Schwartz, someone who
| barely does federal research, chose to use this new technology.
|
| That's a horrible excuse. I'm not a lawyer and don't do caselaw
| research on any sort of regular basis but I have still poked
| around a bit when something strikes my interest. Compared to
| google they're clunky and have poor matching, but I don't
| remember it taking me more than half an hour or so to figure out
| which system would have the case I want and have to drill down to
| find it. ChatGPT was giving the the lawyer the (made up) case. It
| should really be a trivial task to find it in a caselaw database.
| Heck if I was the lawyer I would really really _want_ to find the
| full text case! Who knows what broader context or additional
| nuggets of useful information it might have for my current
| client's issue?
|
| I would not be surprised if he went looking, couldn't find it
| easily, and just said "whatever it has to be there somewhere and
| I can get by without the entire thing"
| omginternets wrote:
| The cynic in me wonders if this isn't part of a plan to create a
| legal precedent banning AI from handling legal disputes.
|
| Think about it: the legal profession is possibly one of the most
| threatened by the development of AI models. What better way to
| secure the professional future of the long tail of lawyers and
| paralegals?
| wefarrell wrote:
| No need to ban AI from handling legal disputes. Unless you're
| representing yourself you need a lawyer and there's no way for
| an AI to act as a lawyer.
| omginternets wrote:
| I should have made my thoughts clearer; my mind immediately
| went to small-time stuff like handling parking disputes,
| which I think AI _is_ on track to competently solve.
| wefarrell wrote:
| I don't think parking disputes are any different. You
| either handle them yourself or you hire a lawyer, I don't
| think anyone can act as your agent without having a legal
| license.
|
| So an AI can do the work but the person handling the
| dispute needs to sign off on it. If the AI screws up and
| the lawyer doesn't catch it then the lawyer's on the hook.
| I don't see any need to change this.
| ilyt wrote:
| I think _at best_ , if we teach AI how to cite the laws it
| is talking about, it would be good for answering some basic
| law-related questions.
| Avicebron wrote:
| maybe, unless the AI calculates somehow that 98% or
| something of parking disputes can be resolved by just not
| showing up in court, so that's the strategy. But you can't
| convey in a reasonable enough prompt that "this cop just
| had that look of being ready to really screw me over"...idk
| man.
| jcranmer wrote:
| There was a brouhaha a few months ago over a tech startup
| that wanted to do AI lawyering, via having the chatbot
| speak in a bluetooth earphone or something. They actually
| signed up someone to do a speeding ticket hearing...
|
| _and subpoenaed the officer to make sure he showed up in
| court for the hearing_ so that they could actually have
| oral argument.
|
| The best way to avoid such a ticket is to show up to
| court and hope the officer doesn't show up. And the AI
| firm went outs of way to make sure the officer showed up.
| That's legal malpractice right there. (In the end, IIRC,
| the judge heard about the firm's involvement and put the
| kibosh on the entire thing.)
| IIAOPSW wrote:
| How would that work, unless the AI advises so many people
| at a time it can suddenly decide to tell everyone to just
| stop going to court.
| ImPostingOnHN wrote:
| Occam's razor would suggest it's just an incompetent lawyer,
|
| rather than an evil genius lawyer playing 3d chess while
| perfectly appearing to be an incompetent lawyer
| AnimalMuppet wrote:
| Not in this article, and I something I heard second-hand, so
| take it with some salt:
|
| The lawyer wasn't admitted to Federal court, so he signed his
| partner's name on the filing.
|
| That's an incompetent lawyer, and a dishonest one. It's not
| 3D chess, it's just "sleazy lawyers gonna sleaze".
| ArnoVW wrote:
| Nit : you probably mean Hanlon's razor
| https://en.wikipedia.org/wiki/Hanlon%27s_razor
| rideontime wrote:
| Both would work in this case.
| ilyt wrote:
| Which is usually the case tbh.
| ImPostingOnHN wrote:
| Thank you! How embarrassing :)
| lennoff wrote:
| How can you ban AI? It's literally designed to produce text
| that's indistinguishable from text that was produced by a
| living human being.
| makapuf wrote:
| either you consider that lawyers have an added value from IA
| and you consider they can take an advice from an IA but they
| are able to see through its bullshit, or you consider they have
| little added value when provided IA input, and then once AI is
| good enough/better as a whole (considering cost and
| availability) they are of no use.
| pixl97 wrote:
| The particular issue here is you believe that this lawyer just
| isn't dumber than a bag of hammers. There is no conspiracy
| needed. People are dumb. This is why when you see warning
| labels all over some item, "Big Label" didn't do that, no some
| dumbass got their tallywacker ripped off by their Easy Bake
| Oven. Now everyone has to deal with 10 stickers on the item and
| a dictionary sized book of what you can and can't do with it.
| TeMPOraL wrote:
| > _People are dumb._
|
| _People_ are dumb. _Lawyers_ shouldn 't be. If distribution
| of stupidity among certain professions is similar to that of
| people in general, something has gone very bad somewhere in
| the education and certification pipeline.
| pixl97 wrote:
| I've got some really, really, really bad news. Lawyers are
| not really any smarter than any other group of people out
| there. Passing a test about law doesn't mean much at all...
|
| I mean, I did computer support for many lawyers and
| prosecutors and many were clever intelligent people. Others
| had to be explained to in simple instructions they
| shouldn't pour the glue on their keyboard before eating it.
| How they became a lawyer is beyond my understanding, and
| yet here we are.
| mysterydip wrote:
| Johnny's family has been invested in this university for
| generations and if we don't pass him they might pull
| their funding of the new building?
| Ekaros wrote:
| Legal degree from less popular mills aren't very high
| standard. Expensive sure, but not high standard.
|
| The true test is the bar, but even then you can probably
| get lucky or hammer enough stuff in head to pass.
| renewiltord wrote:
| Most people's lesson from a certification pipeline
| producing garbage is that we need more certification
| pipelines. It is all quite interesting.
| asveikau wrote:
| I don't know that it produces absolute garbage, you just
| need to be aware that perfect metrics don't exist. A
| person having a credential is one data point. You collect
| multiple data points to form an opinion.
|
| eg., You probably shouldn't hire a lawyer with no law
| degree or no experience simply based on the fact that
| there are tons of credentialed, experienced attorneys who
| are no good.
| asveikau wrote:
| You're just figuring this out? What do you mean "if"?
| spondylosaurus wrote:
| What do you call someone who graduated at the bottom of
| their class in law school?
|
| A lawyer.
| mcguire wrote:
| What do you call someone who didn't graduate from law
| school?
|
| Not a lawyer.
| abduhl wrote:
| Well they also have to pass the bar and get licensed by a
| state or territory. Until then they're just a JD holder.
|
| I think it's flipped for doctors (which is where the
| original joke comes from?) and an MD isn't awarded until
| licensure is completed.
| akiselev wrote:
| If they pass the Bar and have a license to practice law
| they become an _attorney_. With a law degree but no
| license, they 're just a lawyer.
| abduhl wrote:
| This is a distinction that JD holders who have not passed
| the bar have tried to push. It is not true and holding
| yourself out as a lawyer without a license is the
| unlicensed practice of law. The public does not recognize
| the distinction and neither does the law.
| Alupis wrote:
| Speak with a few lawyers and you'll realize they're just as
| dumb as the general population.
|
| In some cases - even more dumb, since they have this belief
| that their credentials mean they know everything about law.
|
| An awful lot of what a lawyer does is look stuff up... and
| in some cases, they aren't even that capable. All too often
| you are responsible for providing your lawyer with
| mountains of research, arguments, etc.
| ilyt wrote:
| You mistake "smart" with "have good memory". Laweyrs are
| far more about knowing the law and finding the law to apply
| at given situation + some social skills rather than
| "smart".
|
| Obviously the good ones, as in any profession, will
| probably also be "smart", but that's just the top.
|
| > something has gone very bad somewhere in the education
| and certification pipeline.
|
| Yeah like the fact lawyers earn far more money than people
| responsible for teaching kids... teaching future
| generations should be at least as prestigious and well paid
| job
| Kon-Peki wrote:
| > part of a plan to create a legal precedent banning AI from
| handling legal disputes.
|
| The judge, in his ruling linked in previous HN discussion,
| listed a good half dozen parts of a legal dispute where he
| thinks AI would be super awesome, and then lays out why this
| particular part of the dispute is terrible for AI to play a
| role.
| ipython wrote:
| No need for cynicism or a grand plan, this is just a few
| lawyers who could find no case law to justify their argument,
| so they made something up and proceeded to blame ChatGPT for
| it. They had several opportunities to "nope" out and apologize
| to the court, and they doubled down EVERY time. These lawyers
| deserve to be disbarred, no question.
___________________________________________________________________
(page generated 2023-06-09 23:01 UTC)