[HN Gopher] Police relied on Clearview AI and put the wrong pers...
___________________________________________________________________
Police relied on Clearview AI and put the wrong person in jail
Author : danso
Score : 248 points
Date : 2023-03-31 16:35 UTC (6 hours ago)
(HTM) web link (www.nytimes.com)
(TXT) w3m dump (www.nytimes.com)
| diebeforei485 wrote:
| This is nuts. Automated image matching results should not treated
| the same priority as detective work. It should not result in
| warrants automatically being sent out.
| ripe wrote:
| Apart from the facial recognition technology, there's also a
| second technology that was used and possibly misused. From the
| article:
|
| The friction of getting a warrant has been eased by technology.
| The Jefferson Parish Sheriff's Office uses an "eWarrant" service,
| CloudGavel, for which it paid $39,800 last year. It's an app that
| allows officers to request digital signatures from judges. "Law
| enforcement officers can now get an arrest warrant approved in
| minutes," the company's website states.
|
| Many civil liberties advocates actually favor electronic
| warrants; they allow judges to more easily review decisions made
| by the police and eliminate a complaint from officers that it's
| too hard to get a warrant. But advocates said it would be
| worrisome if judges were simply clicking a button without asking
| questions or providing sufficient scrutiny.
|
| "There are real questions about whether it increases the
| incidence of judges rubber-stamping warrants," said Nathan Freed
| Wessler, a deputy director with the A.C.L.U.'s Speech, Privacy
| and Technology Project.
| alixj wrote:
| Excellent point. The judge should've known better than to
| approve this warrant on the basis that they photographed the
| whole country and found a resemblance 500 miles away.
| hungryforcodes wrote:
| For a moment I read it as police raided Clearview AI. Sadly this
| is not the case.
| danesparza wrote:
| Oh, this is simple. Clearview AI needs to get sued for defamation
| or slander.
|
| In the United States, falsely accusing someone can be considered
| defamation or slander, depending on the circumstances.
|
| Defamation is a legal term that refers to the act of making false
| statements about someone that damage their reputation. If the
| false statements are made in writing, such as in a blog post or
| social media post, it is called libel. If the false statements
| are made verbally, it is called slander.
|
| To prove defamation or slander, the person who was falsely
| accused must demonstrate that the statements were false (he can),
| that they were published or spoken to a third party (they were --
| to the police department), that they caused harm to the person's
| reputation (he lost a week from work and was put in jail -- not
| to mention countless articles that mention this fact), and that
| the person making the false statements acted with actual malice
| or negligence (they provided a service for money and didn't check
| their facts). Actual malice means that the person making the
| false statements knew they were false or acted with reckless
| disregard for the truth. I'm pretty sure 'reckless disregard for
| the truth' would be pretty easy to prove in this case --
| considering Clearview probably can't say specifically why this
| person selected for arrest.
|
| If a person is found guilty of defamation or slander, they may be
| required to pay damages to the person who was falsely accused to
| compensate for the harm caused to their reputation. The amount of
| damages can vary depending on the extent of the harm and the
| specific circumstances of the case.
| bigmattystyles wrote:
| I always wondered the same about credit reporting bureaus.
| mindslight wrote:
| A fine example of regulatory capture. The "Fair" Credit
| Reporting Act 15 USC 1681h (e):
|
| > _Except as provided in sections 1681n and 1681o of this
| title, no consumer may bring any action or proceeding in the
| nature of defamation, invasion of privacy, or negligence with
| respect to the reporting of information against any consumer
| reporting agency, any user of information, or any person who
| furnishes information to a consumer reporting agency, based
| on information disclosed pursuant to section 1681g, 1681h, or
| 1681m of this title, or based on information disclosed by a
| user of a consumer report to or for a consumer against whom
| the user has taken adverse action, based in whole or in part
| on the report except as to false information furnished with
| malice or willful intent to injure such consumer._
|
| This country desperately needs a GDPR equivalent. One that
| does _not_ except financial surveillance bureaus, the
| healthcare industry, or any other quasi-governmental
| organization that abuses our personal information.
| bnjms wrote:
| We need another word when the capture is beneficial to the
| government because it solves a quasi governmental problem
| which is easier without oversight.
| jessaustin wrote:
| This crap does not benefit any defensible purpose of
| government. Probably it does benefit certain government
| employees. The principal-agent problem appears again.
| maximilianroos wrote:
| > Oh, this is simple.
|
| If you're commenting on something you don't understand, maybe
| don't start out with this?
|
| > I'm pretty sure 'reckless disregard for the truth' would be
| pretty easy to prove in this case -- considering Clearview
| probably can't say specifically why this person selected for
| arrest.
|
| Sigh
| im3w1l wrote:
| I don't think it's nearly that simple. Do you want the same for
| other techniques? If someone says in good faith that a
| fingerprint matches and it turns out it was a false positive it
| should be slander? Shoeprints? DNA?
|
| Clearview AI is providing a service in good faith like all
| these other things. It's up to the police and courts to use
| that information correctly.
|
| Furthermore notice that this guy was only arrested, and the
| evidence bar for arrest is lower than sentencing. I don't think
| we can even say the system malfunctioned here actually. He was
| arrested because it seemed probable he did it. That's how it's
| supposed to work.
|
| Then he was released and should be given routine compensation
| for being arrested and turning out to be innocent.
| bb88 wrote:
| I think this is why I don't think slander would work.
| Clearview AI didn't force the cops to arrest him, or say with
| 100% accuracy it was him. CVAI could have an accuracy of
| 99.9%, and the defense would be "Well see you were that 0.1%
| incorrect!"
|
| It's the police that need to do the investigation to make
| sure the guy arrested is the same person on the video. But
| then they could say too, "Well CVAI said it was a near
| certainty -- we just took their word for it! It would have
| wasted the taxpayers money to do a more thorough
| investigation -- afterall there's always some implicit
| uncertainty in standard policework!"
|
| I think a more interesting question is something like: Does
| Clearview AI fudge their accuracy numbers? Would the true 80%
| likelihood of you being the perp be more or less slanderous
| than a 99% likelihood?
| tantalor wrote:
| It's going to be really easy for them to weasel out by claiming
| the match was like "99% confidence" or something, so not
| actually false per se. This is supported by the facts: "one of
| the alleged fraudsters looked like Mr. Reid"
|
| They can also claim their technology does not make an
| accusation, it provides a similarity score that LEO can use in
| their investigations. This is also supported by the facts:
| sheriff's officer insisted it was a "positive match".
|
| The sheriff's officer actually gives up the game here,
| revealing they improperly relied on the similarity score to
| deduce a suspects guilt, but an actually competent prosecutor
| would know better. The fault lies with the LEO in this case.
| Good luck suing them.
| konschubert wrote:
| The problem is that people's priors for "is this the same
| guy?" are not normalised to "we have used an AI to scan the
| whole country for lookalikes".
| tantalor wrote:
| Police already know about this. It's called a "dragnet".
|
| https://en.wikipedia.org/wiki/Dragnet_(policing)
|
| _Since the 1950s, such "dragnets" have generally been held
| to be unconstitutional as unreasonable search and seizure
| actions._
|
| Related:
| https://en.wikipedia.org/wiki/Reverse_search_warrant
| hypersoar wrote:
| My guess is that some other law or tort would be a better fit,
| but I'll note that the "actual malice" and "negligence" are
| different standards with the latter being a lower bar. The
| former only applies to public figures.
| darth_avocado wrote:
| You also want to sue the police department separately for
| wrongful arrest to keep them accountable and discourage a use
| of tech like this.
| treis wrote:
| This isn't a technology story. This is a police made up evidence
| story. They claimed a source told them that this guy did it.
| Which seems impossible according to the facts the NYTimes laid
| out.
| kayodelycaon wrote:
| A police made up evidence story wouldn't get as many clicks. A
| lot of people get arrested for the crime of "driving while
| black".
| sidewndr46 wrote:
| I thought it was because the officer "smelled weed". You
| can't just pull someone over for being a minority you know
| right?
| bsder wrote:
| You might want to ask some minorities about that ...
| voakbasda wrote:
| Any cop will tell you that they can come up with a "valid"
| pretext to make a traffic stop, after following a vehicle
| for a short time. The officer knows that their subjective
| claims will be treated as factual by the courts.
|
| So, yeah, a bigoted cop can pull over minorities and
| trivially manufacture some rational reason that will
| justify their actions. There's always a "reason".
| danso wrote:
| The subject of this story wasn't arrested for "driving while
| black". His arrest warrants, based on AI-face recognition as
| a source, was signed on July 18. His car was pulled over in
| November after an officer learned of the warrants when
| running his plates.
|
| "People get arrested on false pretenses since the invention
| of police" feels like an overly reductionist response to
| stories scrutinizing technology
| oceanplexian wrote:
| Officers don't run plates any more, it's all done with
| ALPR. All the various constitution-violating "tools" over
| the past few years have been successfully combined to
| create a fully automated, computerized police state.
| wahnfrieden wrote:
| Technology launders abuse from police. You can see for instance
| breathalyzer technology which is closed-source, rife with bugs
| which misclassify, and provide an obfuscating cover for piggy
| action.
| joelfried wrote:
| There is more to it than that if indeed he was flagged by a
| facial recognition pass triggered on a toll booth integrated
| into some warrant database and one policeman clicking "good
| enough" too easily. How deeply have these systems integrated
| themselves into the day to day of police investigations? Are
| any digital warrant requests ever denied? What sorts of
| protections are there to make certain that the person on the
| other side of the warrant request is actually even a police
| officer?
|
| A hundred years ago there was a real cost in time and effort
| and interpersonal relationships to get a judge to sign off on a
| warrant at a weird hour. Are the reductions of those costs
| brought about by using technology in this way a net win for
| society?
|
| The policeman in this actual case did something wrong. Did he,
| out of a motivation to increase his arrest record, find a
| random black man across state lines to try and arrest? Did he,
| out of malice, choose to target this person? Did he, out of
| laziness, not look too closely at two pictures side-by-side and
| click a "Request Warrant" button? How easy, exactly, is this
| mistake to make? That question only makes sense because of the
| technology in the story.
| danso wrote:
| How is a story about a misuse of technology not a "technology
| story"?
|
| The "evidence" would not exist if Clearview AI (or any similar
| vendor) did not purport to offer a reliable matching algorithm
| and expansive dataset. It's not as if the police randomly
| picked out someone to frame, and then used software to
| fabricate evidence for the warrant.
| treis wrote:
| But they did randomly pick someone out and framed them.
| That's the story. Not that they randomly picked someone using
| ClearAI rather than doing so using mug shots, or yearbooks,
| or by driving down the street.
| petsfed wrote:
| I know this outside of the scope of the article, but I've seen
| this a bunch of times when HN commenters claim that a given
| topic is not relevant for HN. The gist of the claim is
| basically "$badaction was already illegal, this new tech did
| not enable $badaction, therefore we don't need to discuss this
| further". The claim always seems disingenuous, because it
| ignores a crucial fact: prior to the tech, the friction
| opposing a bad action was sufficient all by itself to keep the
| rate of the bad action within acceptable limits. Its a "nothing
| wrong with nuclear weapons, if sharp rocks are still allowed
| too" sort of argument.
|
| Fine, yes, this is a police-made-up-evidence story. But its
| also a technology-enabled-a-psuedo-scientific-confidence-
| interval-to-make-the-made-up-evidence-more-convincing story.
| Dismissing it out of hand really downplays _why_ facial
| recognition (or algorithmically generated feeds or the banning
| of human content moderation or...) is so fraught. There may
| well be a solution to the problems appearing, but we 're not
| going to arrive at those solutions without discussing those
| problems as, well, problems.
| SN76477 wrote:
| We need to be skeptical of technology again.
| jimnotgym wrote:
| Maybe we should just say that all 'miracle' technology is ok as
| corroborating evidence, but not as prima facie evidence. DNA,
| fingerprints, ai facial recognition...
| danso wrote:
| non paywall:
| https://www.nytimes.com/2023/03/31/technology/facial-recogni...
|
| Excerpt:
|
| > His parents made phone calls, hired lawyers and spent thousands
| of dollars to figure out why the police thought he was
| responsible for the crime, eventually discovering it was because
| Mr. Reid bore a resemblance to a suspect who had been recorded by
| a surveillance camera. The case eventually fell apart and the
| warrants were recalled, but only after Mr. Reid spent six days in
| jail and missed a week of work.
|
| > Mr. Reid's wrongful arrest appears to be the result of a
| cascade of technologies -- beginning with a bad facial
| recognition match -- that are intended to make policing more
| effective and efficient but can also make it far too easy to
| apprehend the wrong person for a crime. None of the technologies
| are mentioned in official documents, and Mr. Reid was not told
| exactly why he had been arrested, a typical but troubling
| practice, according to legal experts and public defenders.
| barbazoo wrote:
| Yikes. And there is very little people can do to prevent this
| until someone makes this their election platform to change the
| system, right? I'm assuming this is all constitutional.
| sitkack wrote:
| How can non-official documents exist inside a government
| organization? What divides the two?
| [deleted]
| joe_the_user wrote:
| Bureaucracies operate by standard procedures. Official
| documents are produced by standard procedures and official
| documents are often available to the public on a standard
| search - in the case of police, official documents would also
| be available to judges, prosecutors and defense attorneys.
|
| Unofficial documents are produced by official not using
| standard procedures in any variety of ways (searching a
| private company's database in this instance). Generally,
| doing this is against the regulations of a bureaucracy. But
| American police view themselves and often treated as above
| regulations, even their own regulations. So American police
| often produce and keep unofficial documents with no
| consequences.
| lotsofpulp wrote:
| > Mr. Reid's wrongful arrest appears to be the result of a
| cascade of technologies
|
| This statement is clearly false.
|
| As evidenced by
|
| > and Mr. Reid was not told exactly why he had been arrested
|
| Malfeasance is the cause, the level of which should result in
| prison sentences for the police who made the decision to
| deprive a person of their freedom and more for a week.
| than3 wrote:
| I'm sure they would simply say it was from improper training
| since most police recruits are never taught the law, and are
| often only dealing with hardened criminals for the first 5
| years on-the-job.
| joe_the_user wrote:
| The idea that any police group (experienced, inexperienced,
| etc) _" are often only dealing with hardened criminals for
| the first 5 years on-the-job"_ seems absurd on it's face.
| Police drive around an area and deal with the people and
| situations that arise in the area, which is to say they
| will inherent encounter average people most often. Violent
| crimes are less common than other crimes virtually
| anywhere.
|
| Edit: wow, OK, things beyond even me.
| than3 wrote:
| In many areas, before you can be assigned to roles that
| interact with the public you generally have to have a
| number of years handling transfers, which often include
| hardened criminals for medical, court, jail/prison.
|
| It might be absurd, but it is common practice.
| joe_the_user wrote:
| Good call,
|
| The article is effectively saying "(accepting as a given that
| police charge and arrest people on entirely ad-hoc, hence
| lawless grounds...), Mr. Reid's wrongful arrest appears to be
| the result of a cascade of technologies". But naturally we
| shouldn't let that just go by.
| sidewndr46 wrote:
| Are police in the US obligated to tell you why you are being
| arrested?
| jfengel wrote:
| No, not constitutionally. But some states, including New
| York, do have laws that say that they must.
|
| Also, if there is a warrant, they generally have to show it
| to you.
| joe_the_user wrote:
| US Police aren't required to say anything when a person is
| arrested. They can just grab you and hustle you into a car
| without a word.
|
| When a person is charged with a crime, the police expected
| to supply evidence but they can come up with excuses not
| to. Once the case goes to trial, all the evidence is
| supposed to be available to defense attorneys. But since
| plea bargaining is common, police may not have to come up
| with the evidence ever.
|
| Overall, US legal procedure is full of things that are
| absolute rules for civilians but just sloppy average
| suggestions for cops.
| than3 wrote:
| I'm not sure about that, but they certainly aren't required
| to provide information needed to validate a warrant is
| legitimate.
|
| California recently had a gang round-up where warrants that
| were provided as the basis for searches for properties in
| question had almost all information needed to verify the
| validity of the warrant redacted (even the address for the
| property in question being presented at, at least according
| to local news).
|
| No one had any idea what was going on or whether the
| warrants were even legitimate, seemed like a play out of
| East Germany's playbook before the wall came down, just
| short of the gestapo.
| radicaldreamer wrote:
| Everyone should opt-out of Clearview AI here:
| https://www.clearview.ai/privacy-and-requests
|
| Much easier for EU and California residents
| JohnFen wrote:
| > This tool will not remove URLs from Clearview which are
| currently active and public. If there is a public image or web
| page that you want excluded, then take it down yourself (or ask
| the webmaster or publisher to take it down). After it is down,
| submit the link here.
|
| That's not an opt-out. That's only a request that they remove
| the index to an image that has already been removed from the
| web. You can't ask them to remove all photos of you, nor will
| they remove a photo that is still up on the web.
|
| In other words, it's the sort of bullshit that we can expect
| from an evil company like ClearView.
| radicaldreamer wrote:
| You need privacy laws like GDPR and California's in your
| state.
| stevenjgarner wrote:
| Yes my cousin's pet ferret's website desperately needs the
| burden of a GDPR-compliant Cookie Consent Notice. It has
| made European online web presence so burdensome.
| Dylan16807 wrote:
| How about the ferret website doesn't track people.
| bluefirebrand wrote:
| Your cousin's pet ferret's website likely has absolutely
| no business setting anything related to cookies in the
| first place.
|
| So.. yeah.
| JohnFen wrote:
| Yes, very much so.
| realce wrote:
| What incredible arrogance, it makes my blood boil. You're
| supposed to either know what all their sources are and do all
| of your own work one at a time, or just give them links to
| the things you most want hidden, an extreme vulnerability.
| JohnFen wrote:
| Not only that, but ClearView also incorporated driver's
| license photos into their database -- and there's literally
| no way that you can have those removed.
|
| So even the meagre facility they offer here is completely
| meaningless.
| Aransentin wrote:
| Bayes' theorem is ruthless. Even if your AI is 99.99% accurate,
| if you have one true positive and scan everybody the _vast_
| majority of the people you flag will be perfectly innocent. The
| people deploying the system and police are ignorant of the
| statistics and base rates, thinks that it 's only 0.001% likely
| you are innocent, and chuck you in jail because of it.
| giraffe_lady wrote:
| This is the real risk of AI in the short and medium term.
| Everyone who fronts like a malicious superintelligent being is
| the big problem should take a couple steps back and refocus on
| the harms being caused by it now. And particularly should
| understand the damage as being a complex combination of human
| social factors exacerbated by AI, rather than a fundamentally new
| thing we've never seen before.
___________________________________________________________________
(page generated 2023-03-31 23:01 UTC)