[HN Gopher] Detangle: AI-generated summaries of legal docs
___________________________________________________________________
Detangle: AI-generated summaries of legal docs
Author : Shpigford
Score : 96 points
Date : 2023-01-03 15:39 UTC (7 hours ago)
(HTM) web link (detangle.ai)
(TXT) w3m dump (detangle.ai)
| quaintdev wrote:
| The summaries are good but I wish it also had the ability to
| answer my questions about document like whether something is
| allowed or not or what are my rights, etc,.
| freejazz wrote:
| It says all documents and summaries are deleted after 24 hours,
| but are the contents of the documents incorporated into a model?
| I'd imagine for some kinds of legal documents (say those under a
| court's protective order) this would present issues.
| adamsmith143 wrote:
| Are you able to convert the weights of a Neural Network into
| the training text that adjusted them?
| freejazz wrote:
| Simply allowing someone to do this could violate the court's
| order, is the point. Maybe detangle should give a summary on
| the kind of documents it can help with and how to identify
| them, if it's going to pretend to be anything like a lawyer.
| adamsmith143 wrote:
| I'd question if that would hold up in court. Can the court
| order also be violated if I burn a contract and use the
| ashes to grow vegetables? Presumably about as much
| information in the contract is extractable from the
| vegetable as from a NN weight.
| freejazz wrote:
| If you gave the contract to someone (to burn) when the
| court order prohibited giving it to a non-party, yes,
| absolutely. No need to be obtuse about it and to fight
| the hypo. It isn't entirely unusual for a contract in a
| litigation to be marked as attorney's eyes only, or
| highly confidential, and for there to be a protective
| order limiting the distribution of such documents. Also
| would not be unusual for a party to produce such a
| document in the discovery process, to which a protective
| order would also apply.
| amelius wrote:
| Transformer based models have been shown to reproduce
| training text verbatim.
| muttantt wrote:
| I tested with a 4 page basic agreement. Price quoted was $50+.
| peter_d_sherman wrote:
| I uploaded a copy of the U.S. Constitution...
|
| It told me that it would cost $123 USD to summarize...
|
| It also told me:
|
| >"This document summary is not legal advice or legally binding in
| any way. You should consult a lawyer."
|
| Opinion:
|
| _A Lawyer would cost me less..._ <g>
|
| (Perhaps I could get a cheaper rate if I uploaded the Magna Carta
| or The Twelve Tables (of Ancient Rome) or the Ten Commandments or
| Newton's Three Laws -- or the Two Laws of Richard J Maybury:
|
| o Do all you have agreed to do
|
| o Do not encroach on other persons or their property
|
| )
|
| All I know is, $123 is too expensive for me -- and,
|
| _What if the AI takes the Law -- and interprets it all wrong?_
|
| ?
|
| ???
| nashashmi wrote:
| I work through a bunch of contract documents reissued with a just
| a few changes. I need a service that would identify what the
| changes are instead of me going through each document and reading
| it over again Or doing a comparison of previously issued
| documents with this one.
| yakshaving wrote:
| Interesting, what is your job? Contract lawyer? do you work in
| biglaw or in house counsel? or a vc firm?
| nashashmi wrote:
| Engineering marketing concentrating on government proposal
| requests. That involves contracts, described scopes of work,
| etc.
| NoboruWataya wrote:
| What do you mean exactly, are you just looking for software
| that will highlight the differences between two documents? That
| software exists already.
| nashashmi wrote:
| Yes that software does exist and currently my go to method
| (convert pdf to word document, compare changes in word).
|
| The ideal system would highlight (sets of) text that that's
| common between both documents and underline text that is
| original. Or vice versa.
| freejazz wrote:
| You can run redlines of PDFs in Adobe Acrobat pro, no need
| to conver to word.
|
| "The ideal system would highlight (sets of) text that
| that's common between both documents and underline text
| that is original. Or vice versa."
|
| Just change your settings in microsoft word.
| meindnoch wrote:
| git diff --word-diff
| Ryanb58 wrote:
| Love it! This is one of the best uses of ML/AI IMHO. I really
| dislike reading leasing agreements.
| Shpigford wrote:
| Thanks so much!
| mtlmtlmtlmtl wrote:
| How is this tested to make sure the summaries aren't wrong?
| denton-scratch wrote:
| I started wading through the Google example. I got about 0.25 of
| the way through.
|
| The glosses all seem accurate, if incomplete sometimes. I got a
| strong sense that they had been reviewed/polished by a human
| "expert". That might explain the pricing.
|
| I _didn 't_ find the glosses much clearer than the original, just
| shorter. The Google legalese I read was fairly clear. With the
| document being a legal contract, it's obviously vital to
| understand the legalese.
|
| I'd be impressed by an AI that can read an employment contract,
| and say "Whoah! Clause 31-B(ii) is implicitly asking you to sign
| your descendants into servitude in perpetuity! Perhaps you should
| take a look!", or "Did you notice that all benefits are
| contingent on stuff beyond your control?". I'd probably have paid
| $25 for that, because I've never had a lawyer scan my employment
| contract. I read them myself, but IANAL.
| goldforever wrote:
| [dead]
| tasseff wrote:
| Can this idea be turned on its head to engineer adversarial legal
| language that sounds reasonable but actually has terrible
| implications for the signee?
| [deleted]
| make3 wrote:
| I'm sure it would work with a LLM if you give it a few examples
| lordgrenville wrote:
| I've heard that in situations like this courts will usually
| uphold the spirit and not the letter of the contract (same as
| eg if you have a typo and add a zero somewhere). Not a lawyer
| and might be 100% wrong on this.
| awb wrote:
| INAL, but I think this is true to a certain extent.
|
| I'm pretty sure for a contract to be legal both sides need to
| receive "valuable consideration" (something of value). So, if
| you get completely screwed over, I think there can be a legal
| remedy.
|
| But, I don't think there's much a court will do if you come
| out on the losing end of a contract, just based on the terms
| themselves as long as both parties still received something
| of value.
| zopa wrote:
| There's a rule of thumb ("canon of construction") that
| ambiguities in an agreement are construed against the
| drafter.
| planede wrote:
| Uh, I wanted to try it with GPL v3, but no way I'm paying $111
| for it.
| ahepp wrote:
| How is this "not legal advice"? You're _selling_ summaries of
| legal documents.
| superfrank wrote:
| I assume their argument would be that they are just summarizing
| information that you provided and not giving you advice on what
| to do with this information. Similar to how summarizing a
| book's plot isn't an endorsement of that book.
|
| I'm not a lawyer though, so I have no idea if that logic would
| hold up in court. I'm just guessing that's the rational they
| are using.
| jmacd wrote:
| https://www.spellbook.legal is another one I have seen that is
| doing the opposite (helping with the writing of contracts)
| alexmolas wrote:
| With these prices they could have a lot of people in some cheap
| country summarizing the documents.
| xiphias2 wrote:
| It wouldn't be a bad idea either
| t_serpico wrote:
| anyone else worried about how tools like this will make us
| dumber?
| quaintdev wrote:
| It depends what you use the tool for. You can use tool for
| menial tasks while doing something highly creatives. Tools
| inherently don't make us dumber. The choice is always lies with
| the individual.
| [deleted]
| fellerts wrote:
| Who is the target audience for something like this? I'm not sure
| anybody is going to pay to have Apple's ToS simplified before
| clicking "agree". On the other hand, if you are squinting at a
| contract that actually matters, I don't think a service like this
| gives sufficient understanding of what all that legalese would
| actually mean if shit hits the fan. For example, changing the
| applicable law in a contract can yield wildly different
| interpretations without changing a word in the contract iself.
| This is why you should seek professional help when in doubt.
|
| You say "The summaries are to help inform conversations with
| actual lawyers", but would a lawyer give a rats ass about a
| layman's interpretation of an AI-backed translation of a legal
| doc from a company who "does not guarantee that the stuff on the
| website is good or works"? IANAL but I don't think so.
| fnordpiglet wrote:
| I assume lawyers could benefit from a tool that can summarize a
| large document rapidly. They're often faced with thousands of
| pages of documents, and in large class actions possibly
| hundreds of thousands to millions.
| NoboruWataya wrote:
| > You say "The summaries are to help inform conversations with
| actual lawyers", but would a lawyer give a rats ass about a
| layman's interpretation of an AI-backed translation of a legal
| doc from a company who "does not guarantee that the stuff on
| the website is good or works"? IANAL but I don't think so.
|
| If you are conversing with your own lawyer, you should be
| getting your summary from them, as that way you can be fairly
| sure it's correct (and you can probably sue them if it's not).
| That's part of a lawyer's job, explaining the law to their
| clients.
|
| If you are conversing with someone else's lawyer, and you don't
| have your own lawyer, you are dangerously close to relying on
| this software for legal advice, which, given how confidently
| incorrect AI can be, could end in tears.
| traceroute66 wrote:
| > You say "The summaries are to help inform conversations with
| actual lawyers", but would a lawyer give a rats ass about a
| layman's interpretation of an AI-backed translation of a legal
| doc
|
| Indeed.
|
| We all know what the medical profession quite rightly thinks
| about patients who turn up having (supposedly) self-diagnosed
| themselves with the help of Doctor Google.
| arcticfox wrote:
| > We all know what the medical profession quite rightly
| thinks about patients who turn up having (supposedly) self-
| diagnosed themselves with the help of Doctor Google.
|
| And we all know what artists think about AI art...they hate
| it, because it's pretty good and it's going to take a lot of
| their jobs.
|
| "Doctor Google" is pretty bad but there's absolutely no
| reason AIs can't get very good at most areas of medicine. For
| now doctors don't like Google because it gives bad results,
| but pretty soon they're going to start hating it because it
| gives _good_ results.
| bronco21016 wrote:
| I'm a unionized employee who is currently waiting on finalized
| legal language for a new contract.
|
| A lot of my time will be spent reviewing a diff between the
| current agreement and the proposed agreement but it's a 500
| page document. I might give this product a try just to see what
| it spits out. I could see this being incredibly useful if it
| could write a plain language, non-biased, summary of changes.
| muttantt wrote:
| A 4 page document prompted me for a $50 payment.
|
| Good luck running 500 pages through this thing...
| kkielhofner wrote:
| The pricing is absolutely disconnected-from-reality wild.
|
| At worst a junior associate at $BIGFIRM would review a
| document of that length with an understanding of issues
| specific to your situation, governing law/locality, etc for
| somewhere around $100 AND this would include a redlined
| response with suggested edits, annotations, any questions,
| etc.
|
| A local, friendlier more SMB or startup catering
| attorney/smaller firm would likely look at a bunch of these
| for free or close to it.
|
| In short the pricing needs to be at least 1/10 what it is
| currently so it can (at best) serve as a novelty, a basic
| filter to a series of potential agreements, etc.
| freejazz wrote:
| How much do you think they charge for associate time at a
| biglaw firm? It's well above $400/hr.
| kkielhofner wrote:
| I've worked with (and paid invoices from) several top-
| tier international > 500 attorney firms and if that's
| what you're effectively paying for junior associate time
| you need to look elsewhere.
|
| For rate sheet time that's still too much and negotiating
| down legal invoices is just about the best use of time
| from a cost-benefit standpoint.
| freejazz wrote:
| They are probably not the top tier biglaw firm if they
| are charging less than $400/hr for associate time. S&C
| charges over $800/hr for associate time.
| mc32 wrote:
| I would see if it could interpret Oxford commas appropriately
| such as in: https://www.nytimes.com/2018/02/09/us/oxford-
| comma-maine.htm...
| tluyben2 wrote:
| You would actually want to discuss the document with a lawyer
| (AI), for instance asking if not paying the 100k in point 3.4.7a
| is enough for termination of the contract, show the reasons
| (copied from the doc and explained) and if yes, are there any
| clauses that invalidate that clause or any logical
| inconsistencies regarding that clause etc.
|
| As someone (no legal background) who has to read legal documents
| sometimes (perks of being a founder _cough_ ) and would take
| advice (yet) from an AI over a lawyer; the above would give me
| enough of an idea how exposed we are or not and if a lawyer
| should be contacted. I can read legalise quite well (perks of
| having been a founder for 35 years), but if its 100s of pages, I
| need to be able to ask questions about the doc to get an idea of
| where I stand. Which is why I would hire a lawyer to do a review
| after I send them my questions anyway.
| traceroute66 wrote:
| I'm sorry, but in the nicest possible way, this is bullshit.
|
| Anybody who has been in business-roles that involve reviewing
| legal documents will tell you that.
|
| TL;DR three things: 1. The AI will NOT tell
| you if it is the right document for the context, only a lawyer
| will do that 2. The AI is unlikely to correctly
| analyse the document, clauses in a legal document have inter-
| relation with each other and with the general context (perhaps
| even with other legal documents you have previously signed). That
| sort of analysis is only something a lawyer can do.
| 3. Perhaps MOST IMPORTANTLY ... with legal documents, often the
| important thing is not what is IN the document, but what is NOT
| IN there. Only a lawyer can tell you what is missing in a legal
| document AND help you get that missing stuff negotiated into it.
|
| Really, I'd run away from something like this as fast as you can.
| jackbrookes wrote:
| Why do you think AI cannot be as good as a lawyer for all these
| things? Take the technology to its logical end and it can be
| better than a lawyer
| notahacker wrote:
| > Why do you think AI cannot be as good as a lawyer for all
| these things? Take the technology to its logical end and it
| can be better than a lawyer
|
| Because the "logical end" of an LLM that can match words with
| other words isn't even close to AGI that actually understands
| the current version of the relevant law and its specific
| implications for a specific entity in a specific
| jurisdiction. But it can spout convincing looking bullshit
| based on having parsed a load of text with a certain amount
| of semantic similarity which is _completely irrelevant_ in
| legal terms.
|
| A random teenager on Reddit who prefaces his posts with
| "IANAL, but I think..." is closer to the relevant level of
| understanding of legal documents and jurisdictions than
| GPT-3, but nobody ever wonders why people don't think
| teenagers on Reddit are better at offering legal advice than
| lawyers...
| kemitchell wrote:
| Whether you're using the right start point and picking the
| right optional pieces are definitely important. But for most
| kinds of agreements I see and advise on, there's nothing
| lawyers know that nobody else could know. What you need doesn't
| _have_ to come from a lawyer, though it often does.
|
| Meanwhile, specifics of the deal and the broader context can
| also matter, sometime more than the abstractions legal forms
| tend to deal in. A lawyer may or may not notice those and
| reason through them.
|
| Paving an AI path over ill-fitting or over-standardized legal
| terms isn't great for the industry. Neither is reinforcing
| lawyer monopoly over business knowledge.
| freejazz wrote:
| Lawyers don't have a monopoloy over business knowledge, they
| have a monopoly over providing legal services.
| okennedy wrote:
| Lawyers also have a _legally binding obligation_ to provide
| accurate, context-aware, complete advice.
| freejazz wrote:
| Which, when they don't deliver, gives you a cause of
| action for malpractice.
| Shpigford wrote:
| Detangle gives you AI-generated summaries of your legal docs so
| you can understand what you're signing.
|
| Here's the YC SAFE, for example: https://detangle.ai/examples/yc-
| safe
| andrewfromx wrote:
| I wonder, what happens when you read just the summary of a
| complex section, sign, and then years later are in a lawsuit.
| If Detangled's algorithm in shown to have clearly written the
| summary wrong... are they in any way liable? hehe notice their
| own terms with summaries: https://detangle.ai/terms
| Shpigford wrote:
| That was the big reason for adding the massive disclaimer
| everywhere, but I guess we'll see.
|
| I also intentionally didn't just give an output of the
| summary but instead showed it next to each paragraph so the
| implication isn't that the summaries _replaces_ the legal,
| but rather tries to clarify.
|
| Glad you noticed the Terms. :)
| NoboruWataya wrote:
| From the original text:
|
| > We may suspend or terminate your right to use our website
| and terminate these Terms of Service immediately upon written
| notice to you for any breach of these Terms of Service.
|
| From the summary:
|
| > We can end this agreement any time we want.
|
| I read these as being quite different? The original text says
| they can only terminate for a breach by the user of the ToS,
| not simply "any time we want".
|
| I get that the service isn't supposed to replace actual legal
| advice but surely differences as glaring as these limit the
| usefulness of the summaries.
| ahepp wrote:
| > AI-generated summaries of your legal docs so you can
| understand what you're signing.
|
| If you sell me a summary of a legal document, which was
| advertised as being useful to understand the legal document,
| that seems like just about the most straightforward case of
| legal advice I can imagine.
| simonista wrote:
| I'm excited to see where this goes because I think it is a cool
| application of AI. That said, two of the first three paragraph
| summaries are wrong in this example:
|
| > [Investor Name] gave [Company Name] the right to certain
| shares of its Capital Stock in exchange for [Amount] on [Date].
|
| This is backwards
|
| > The Post-Money Valuation Cap is a number that is written in
| Section 2.
|
| This isn't true, the number is here, "additional defined terms"
| are later.
| treis wrote:
| It's anywhere from slightly to very wrong for each section.
| Pretty cool as a tech demo but definitely should not be used
| to understand a legal document.
| ryanlitalien wrote:
| Absolutely fantastic! This is a great use of AI here,
| definitely using this going forward. A bit concerned though,
| the LLC name doesn't give confidence when reading through the
| output with confidence.
| bernardv wrote:
| The fact I have to upload to a server and my document will not be
| deleted for up to 24 hours, makes it a no go for most corporate
| users. Make this a standalone desktop app, with zero cloud
| dependency and it would be useful. This is very useful for a
| first pass through a legal doc, for example, to triage documents.
| binarymax wrote:
| Also, it uses OpenAI services (GPT3) - so now you also need to
| understand their TOS and data retention policies. Makes it a
| non starter for any private or confidential documents.
| falcolas wrote:
| Given how confident AI can be when it's incorrect, it feels like
| a bit of a problem to use AI with legal docs, since you can't
| really trust it to be correct (and the cost of being incorrect
| may be absurdly high).
| stronglikedan wrote:
| That's why all the results come with an "AINAL" disclaimer.
| falcolas wrote:
| Aah, yes. The old fine print switcheroo.
|
| "Here's a tool for this highly specific purpose!+"
|
| "+Tool not fit for this highly specific purpose."
| oa335 wrote:
| Agreed, I would still want a lawyer to take a look or anything
| of importance.
| Shpigford wrote:
| I also would want a lawyer to take a look. Hence the note
| saying to talk to a lawyer. ;)
|
| The reality is, even knowing what to ask a lawyer can be
| difficult and it's not economically viable to ask a lawyer to
| summarize a 100-page legal document into text you can wrap
| your head around.
|
| Detangle just helps give some clarity so that you can further
| a conversation with a lawyer, or even just ask for
| clarification from the person who sent you the document.
|
| "Hey, this paragraph seems to imply XYZ...is that the case?"
|
| I'm making zero claims that this should replace a lawyer or
| be used as a replacement for legally binding text.
| notahacker wrote:
| > The reality is, even knowing what to ask a lawyer can be
| difficult and it's not economically viable to ask a lawyer
| to summarize a 100-page legal document into text you can
| wrap your head around.
|
| A 100 page legal document (i.e. one with lots of stuff
| implied by structure and lots of specific subconditions and
| lots of case-specific stuff that might make more sense in
| conjunction with the examples in the appendix) sounds like
| the sort of thing AI would be wrong about far more often
| than it was right...
|
| At least reading a short contract or EULA I'd be pretty
| confident an AI-based system would have lots of similar
| contracts and plain English explanations in its corpus and
| correctly identify the boilerplate exclusions and even if
| it misinterpreted stuff like IP assignments and ability to
| modify the agreement it might be slightly better than
| Ctrl-F at figuring out which clauses do this. Although from
| what I can see the current incarnation is working on a
| paragraph by paragraph basis so doesn't even necessarily
| apply definitions defined in the section above correctly...
| jchw wrote:
| Question: how much would it cost to consult a lawyer about
| a 100 page document? How does one even go about doing
| something like this? I've never really needed to do that,
| so I'm genuinely curious.
| freejazz wrote:
| You call a lawyer up, tell them what you need, and the
| lawyer will either give you an estimate, or agree to a
| cost, and send you an engagement letter.
| traceroute66 wrote:
| > even knowing what to ask a lawyer can be difficult
|
| That's the lawyer's job, to guide you through ! If they're
| not doing that, then find a different lawyer.
|
| The whole point is we are talking about legal document
| review here.
|
| Reviewing legal documents necessarily involves an active
| discussion with a lawyer, it is not something you can turn
| up with pre-prepared questions for.
| jakelazaroff wrote:
| That's true, but you still should (try to) be prepared.
| Meeting with a lawyer will go much more smoothly if you
| have at least a rudimentary understanding of the subject
| matter, rather than having them explain everything to you
| from first principles.
| freejazz wrote:
| And you can also waste half the time having the lawyer
| explain to you that what detangle told you was wrong...
| drbwaa wrote:
| > That's the lawyer's job, to guide you through ! If
| they're not doing that, then find a different lawyer.
|
| Yes! And if you happen to know that, and have a good
| lawyer, then you're all set. But a hell of a lot of
| lawyers won't volunteer to do this, or aren't competent
| enough to have this conversation productively with a
| layperson, and most people don't know to ask for it.
|
| I have my doubts as to the practicality, but a tool that
| could summarize and then prompt laypeople go to their
| lawyer to start that discussion seems like it could
| definitely have value. I suspect most people interested
| in this tool may already be knowledgeable enough to ask
| for a walkthrough anyway, but you could imagine
| implementations where the summary is presented directly,
| and could guide more people toward the help they need.
| traceroute66 wrote:
| > competent enough to have this conversation productively
| with a layperson
|
| HUH ? That is literally their job !
|
| Legal document review is bread and butter for lawyers.
|
| Having productive conversations with laypeople is what
| lawyers do all day every day.
|
| I really don't get your point.
|
| If a lawyer is unable to have a productive conversation
| with a layperson about a legal document, then they need
| to be stripped of their qualifications.
| yunwal wrote:
| There are plenty of lawyers who are not competent at
| their jobs, but can be convincing to uninformed
| laypeople.
| babyshake wrote:
| This seems targeted at the low end of the market, where the
| alternative is simply to not read a ToS, EULA, etc.
| fnordian_slip wrote:
| Wouldn't no information be preferable to wrong information in
| most cases?
| NoboruWataya wrote:
| I highly doubt the low end of the market will pay anything
| close to what this service wants to charge.
| scrose wrote:
| I've had ChatGPT tell me how to do something one way, and then
| give an example that omitted the specific syntax it had called
| out. It was only until I told it that the example didn't
| reflect what it wrote that it basically said 'oops! Here's the
| corrected version'. I can't even begin to imagine how poorly
| that could go in a legal context.
| tluyben2 wrote:
| With chatgpt, with the little toy project I am trying that
| uses it, I default prompt it (automatically) that is is wrong
| and please provide a better version. I will generate a few
| versions that get better. Mind you that I don't tell it what
| was wrong saying 'this does not include all the points I
| asked'. It will say 'oops, here is a version that includes
| all you asked'. And that a 3-4 times in a row.
| make3 wrote:
| As a NLP researcher at a well known place & working on this, I
| totally 100% agree with you, this is ultra dangerous and should
| not be used, straight up, on anything remotely serious
| saliagato wrote:
| What is the best approach to building something like this? Do
| you work at HF?
| drbwaa wrote:
| Given how _poorly written_ a lot of real legal documents are, I
| would definitely not trust any current AI to summarize it
| accurately. How would it handle documents where terms or
| clauses are in conflict, or any other situation where a close-
| reading human would have to step back and say "okay, this is
| _semantically_ impossible?"
|
| I could see a tool like this being valuable as an outline
| generator for long documents, but I would be very reluctant to
| believe any statements it makes about the actual legal effects.
| CSMastermind wrote:
| I could see this being useful as a tool that scans legal
| documents in bulk like Terms of Service and looks for red
| flags that simple keyword searches would miss.
| intuxikated wrote:
| Could definitely be a useful tool to improve something like
| ToSDR
| jamesdwilson wrote:
| Does anyone know of a "Tangle" - it would be neat to do the
| inverse!
| nerdponx wrote:
| What are they doing with all the uploaded document data? I didn't
| see that in their terms of use.
| Shpigford wrote:
| We use GPT-3 to summarize it the document data. Then 24 hours
| later we completely delete all text, URLs and file uploads.
| mrstone wrote:
| You do nothing with the data in the interim? I find that hard
| to believe.
| muttantt wrote:
| No way any respectable lawyer thinks it's OK to upload a
| client agreement with confidential information to some fly by
| night website the promises to delete.
___________________________________________________________________
(page generated 2023-01-03 23:01 UTC)