[HN Gopher] Innocent woman jailed after being misidentified usin...
___________________________________________________________________
Innocent woman jailed after being misidentified using AI facial
recognition
Author : rectang
Score : 225 points
Date : 2026-03-12 20:55 UTC (2 hours ago)
(HTM) web link (www.grandforksherald.com)
(TXT) w3m dump (www.grandforksherald.com)
| rectang wrote:
| > _facial recognition showed she was the main suspect in what
| Fargo police called an organized bank fraud case._
|
| > _Her bank records showed she was more than 1,200 miles away, at
| home in Tennessee at the same time police claimed she was in
| Fargo committing fraud._
|
| > _Unable to pay her bills from jail, she lost her home, her car
| and even her dog_
| Jtsummers wrote:
| https://archive.is/yCaVV - Archive link to get around the
| paywall.
|
| https://www.theguardian.com/us-news/2026/mar/12/tennessee-gr... -
| Another article on this without a paywall.
|
| It's annoying that both articles are calling this AI error. This
| was human error, the police did the wrong thing and the people of
| Fargo will end up paying for this fuckup.
| superkuh wrote:
| > https://archive.is/yCaVV
|
| When I load this URL I get "One more step Please complete the
| security check to access" and I cannot get past the archive.is
| computational paywall.
|
| But the guardian article actually has text! Thanks.
| hrimfaxi wrote:
| That's a common issue if you use cloudflare dns.
| janalsncm wrote:
| I would argue it was both. No doubt this company was marketing
| it in a way to make it seem very reliable. And all of the
| procedural things afterwards made the error so much more
| damaging.
|
| But imo this is why local police departments should not have
| access to this kind of tool. It is too powerful, and the
| statistical interpretation is too complicated for random North
| Dakota cops to use responsibly. Neither the company nor the PD
| have an incentive to be careful.
| mitchbob wrote:
| https://archive.ph/2026.03.12-183903/https://www.grandforksh...
| jauer wrote:
| AI or not, it's unconscionable that victims of compulsory legal
| processes by way of mistaken identity are not made whole.
| ryandrake wrote:
| People will defend this, too, saying "well, she was eventually
| exonerated, right? So the system works!" Ignoring how she'll
| never be fully reimbursed for the time, money, and grief of
| going through the system.
| munk-a wrote:
| We also need to question how many people might go through the
| same process without eventual exoneration and how much going
| through this process costs individuals. Being falsely
| prosecuted comes usually imparts a permanent black mark in
| search results about the person (outside of places with sane
| laws like the EU) as well as causing stress or permanent
| injury.
|
| Wrongly arrested individuals with mental disabilities have a
| history of physical abuse in jail potentially to the point of
| death.
| janalsncm wrote:
| > In all criminal prosecutions, the accused shall enjoy the
| right to a speedy and public trial
|
| This is from the Sixth Amendment. Where the rubber hits the
| road is what "speedy" means.
| neaden wrote:
| I hate this headline (not blaming submitter). Police incompetence
| and negligence jailed her for months and left her stranded in a
| North Dakota winter. The AI is no more responsible than the cars
| and airplanes they used.
|
| Edit: this is in reference to the original headline "AI error
| jails innocent grandmother for months in North Dakota fraud case"
| not the revised title that it was changed to.
| conartist6 wrote:
| I disagree. Clearly the police felt the AI was "responsible
| enough" to be the only thing they needed to trust.
|
| The AI made the call and humans licked its butthole
| nkrisc wrote:
| And that is a complete failure of the police and authorities.
| They made the decision to extradite her with such flimsy
| evidence.
| conartist6 wrote:
| If it didn't erase accountability, how would it create any
| value?
|
| Many people are treating this as a matter of philosophy,
| which it isn't.
|
| At a primitive, physiological level if you delegate to AI
| and most of the time you don't get in trouble for it, the
| resulting relationship you have with the AI could only be
| called "trust".
|
| If you're expected to be 40% more productive at your job,
| your employer is making it crystal clear that you will
| trust the AI or you will be fired. Even if nobody ever said
| it, the sales pitch is that AI does the work and people are
| mostly there to be their servants whose role is to keep
| them fed with decisions we want made but don't want to be
| responsible for making.
| dmurray wrote:
| Even if she was guilty, they shouldn't have imprisoned her
| for 3+ months without interviewing her. The AI didn't tell
| them to do that.
| throw-the-towel wrote:
| I think you actually agree with the GP? As I understand them,
| they're saying that it's not the AI tool that takes the most
| blame, it's the police.
| rpdillon wrote:
| And the police were wrong, which is why they're the culpable
| ones.
| Chris2048 wrote:
| Even if the id was correct, why would they leave her in jail
| for 5 months before the first interview and/or court
| appearance?
| like_any_other wrote:
| > Clearly the police felt the AI was "responsible enough" to
| be the only thing they needed to trust.
|
| Yes, that's what the OPs "incompetence and negligence"
| referred to.
| PTOB wrote:
| No indication that the licking was consentual.
| add-sub-mul-div wrote:
| Your picking apart the words doesn't matter if police are more
| incompetent with AI than without it. AI being the catalyst to a
| worse society is a more interesting and worthwhile topic than
| whether "AI is responsible" is the right way to phrase it.
| _m_p wrote:
| A jury will probably decide the AI company's level of
| responsibility at trial. It is an open question til then!
| mirekrusin wrote:
| Brave police officers wanted to show us all the dangers of AI
| slop.
| mmooss wrote:
| If you make the AI software, then your software malfunctioned.
|
| If the laser printer screws up a page in the middle of the
| document, and the user doesn't catch it and includes it in the
| board of directors binder, the laser printer still
| malfunctioned.
| tony_cannistra wrote:
| Completely infuriating, but more of a commentary on the sad state
| of incompetent power-hungry law enforcement with tools they don't
| know how to use than the tools themselves.
|
| Though, the question remains: are the tools built in such a way
| as to deceive the user into a false sense of trust or certainty?
|
| _Some_ of the blame lies on the UX here. It must.
| sidrag22 wrote:
| It must land as human's fault or this will become more and more
| of a pattern to avoid accountability.
| paulhebert wrote:
| It's both.
|
| The cops need to be held accountable.
|
| But it's glaringly obvious that if you build tools like this
| and give them to the US police this is the outcome you will
| get. The toolmakers deserve blame too.
| hsbauauvhabzb wrote:
| Spoken like someone who isn't built for a sales role at said
| company.
|
| Sales will sell the dream, who cares if the real world outcomes
| don't align?
| ImPostingOnHN wrote:
| _> are the tools built in such a way as to deceive the user
| into a false sense of trust or certainty? _Some_ of the blame
| lies on the UX here. It must._
|
| Are AI code assist tools built in such a way as to deceive the
| user into a false sense of trust or certainty? Very much so
| (even if that isn't a primary objective).
|
| Does any part of the blame lie on the UX if a dev submits a bad
| change? No, none.
|
| _You_ are ultimately, solely responsible for your work output,
| regardless of which tool you _choose_ to use. If using your
| tool wrong means you make someone homeless, car-less, and also
| you kill their dog, then you should be a lot more cautious and
| perform a lot more verification than the average senior
| engineer.
| tony_cannistra wrote:
| I agree with all that. Maybe the word isn't "blame," then.
| Surely there must be some code, perhaps moral or ethical, but
| ideally more rigorously enforcible, which ought to prevent
| the development of intentionally deceiving tools. Sure you
| could say this about all software, but that which can cause
| actual physical harm ought to be held to a higher standard.
| ImPostingOnHN wrote:
| Yes, unfortunately technology is advancing faster than the
| average human brain evolves more neurons, so it will only
| become less comprehensible to the average person.
|
| That's setting aside the tendency for police to hire from
| the left side of the bell curve to avoid independent
| thinkers that might question authority, refuse to do bad
| shit, etc.
| throw_m239339 wrote:
| > they don't know how to use than the tools themselves.
|
| No, the tools work perfectly as they were design to work. The
| problem is that the tools are flawed.
|
| Ultimately, every single of these decisions should be approved
| by a human, which should be responsible for the fuck up no
| matter what the consequences are.
|
| > _Some_ of the blame lies on the UX here. It must.
|
| No, the blame lies with the person or the group who approve the
| usage of these tools, without understanding their shortcomings.
| Pxtl wrote:
| I miss the days of earlier AI image-recognition software that
| would emit a confidence percentage.
|
| New LLM-related AIs are all supremely confident in every
| assertion, no matter how wrong.
| janalsncm wrote:
| I don't know what tool they used, but it was very likely
| not an LLM. They probably have some database of drivers'
| licenses and they ran a similarity search against the
| surveillance footage. This poor lady happened to be the top
| match.
|
| Even if it also output a score, that score depends on how
| the model was trained. And the cops might ignore it
| anyways.
| jolmg wrote:
| >> are the tools built in such a way as to deceive the user
| into a false sense of trust or certainty? _Some_ of the blame
| lies on the UX here. It must.
|
| > No, the blame lies with the person or the group who approve
| the usage of these tools, without understanding their
| shortcomings.
|
| The person who approved the tools might've understood, but
| that doesn't mean the user understands. _Some_ of the reason
| why the user doesn't understand the shortcomings of the tool
| might be because of misleading UX.
| rpcope1 wrote:
| There's no way this isn't a slam dunk case to sue the piss out of
| the Fargo Police, probably the US Marshals and maybe other orgs.
| The woman in the surveillance phone clearly looks way younger,
| among the many other obvious signs this woman didn't do it. I
| hope she wrings at least several million dollars out of the
| government.
| Blackthorn wrote:
| With all the lovely qualified immunity doctrine? That's wishful
| thinking.
| Jtsummers wrote:
| That may protect them personally, but not the city and the
| department itself from being sued.
| blagie wrote:
| Nope.
|
| https://abovethelaw.com/2016/02/criminally-yours-
| indicting-a...
|
| You can be arrested, indicted, and held in jail on
| pretrial, and there is literally no recourse. There are
| many other ways jail can happen without due process. Where
| I live:
|
| * Civil contempt. Absolutely immunity. No due process.
| Record is about 16 years. Having a bad day? Judge can toss
| you in jail.
|
| * "Dangerous." Half a year. No due process. He-said she-
| said.
|
| * "Insane." Psychiatric hold. Three days. Due process on
| paper, not in practice. Police in my town can and do use
| this if they don't like you.
|
| Absolutely no recourse. You come out with a gap in income,
| employment, and, if you missed rent/mortgage, no home.
| Landlords will simply throw your stuff away too.
|
| You're also basically damned if things do move forward,
| since from jail, you have no access to evidence, to
| internet (for legal research), and no reasonable way to
| recruit a lawyer (and, for most people, pay for one).
|
| Can happen to anyone. Less common if you're rich and can
| afford a good lawyer, but far from uncommon.
| mothballed wrote:
| >* "Insane." Psychiatric hold. Three days. Due process on
| paper, not in practice. Police in my town can and do use
| this if they don't like you.
|
| A friend of mine was committed longer than 3 days without
| council or the ability to represent themselves in the
| hearing. Apparently the whole process of being committed
| is ex parte in practice in some states.
| abduhl wrote:
| This is a bit hyperbolic and the exaggerations really
| undermine what I think is your broader point (that there
| is rarely recourse when you're held for short to moderate
| amounts of time). It is hard for me to believe that
| someone was held for 16 years on civil contempt without
| due process or that someone was held for half a year
| without due process after being deemed dangerous. The
| reason that is hard for me to believe is that the due
| process is implicit in the action you describe. Civil
| contempt is from a judge which implies that you're
| already in court - that's due process. Someone being
| labeled "dangerous" implies that a finding was made by a
| neutral party - that's due process.
|
| Just because you disagree with the outcome doesn't mean
| that due process wasn't given.
| mothballed wrote:
| Yeah it's "due process." In civil contempt the judge is a
| witness and prosecutor in the very "process" they're
| judging. That's the most perverted form of due process
| imaginable.
|
| A judge should have to recuse themselves if they are
| acting as witness to the supposed infraction.
| abduhl wrote:
| Civil contempt isn't some roving criminal charge that
| jumps out of the jury box randomly. It's meant to make
| somebody comply with a court order. Anybody in civil
| contempt holds the keys to the jailhouse door in their
| own hands, all they have to do is comply.
|
| This statement should make you uncomfortable. It makes me
| uncomfortable because it is a pure expression of the
| power of the state. But it's still due process.
| Jtsummers wrote:
| I don't know what you're responding to, but I don't think
| it's my comment.
|
| Qualified immunity protects individuals, _not_
| departments, from liability.
|
| The particular thread (in this thread) that I was
| responding to:
|
| >> I hope she wrings at least several million dollars out
| of the government.
|
| > With all the lovely qualified immunity doctrine? That's
| wishful thinking.
|
| I was responding to the claim that qualified immunity
| protected the government, it does not.
| djfobbz wrote:
| Criminal immunity? Sure. Civil immunity? Nope! She could
| definitely make a nice buck.
| theLiminator wrote:
| Off of taxpayer money sadly. Imo we really need a fix for
| this. When cops are grossly negligent the money should come
| out of their aggregate pension fund (or at least
| partially).
| SunshineTheCat wrote:
| Yes, this is the key point. Tax payers get a nice big
| bill while the people who caused the problem get a nice
| paid vacation while they conduct an internal
| "investigation" that typically finds they did nothing
| wrong.
| vkou wrote:
| There is a fix to it. Elect people who will hold them
| accountable.
|
| As long as you keep electing clowns that let the police
| do whatever they want, the police will... Do whatever
| they want.
| theLiminator wrote:
| Yeah, of course they need to held accountable, and we
| need to vote in people who will do so. What I'm
| suggesting is an alignment of incentives that will ensure
| that police will try to do their best to not be
| negligent.
|
| Of course there's a balance that has to be struck so that
| police are empowered enough to act. So perhaps something
| like settlements against the police being 30% borne by
| the police pension fund and 70% by taxpayers is
| sufficient. I think this will also make police very
| enthusiastic about bodycams and holding each other
| accountable.
| rectang wrote:
| "Tough on crime" -> lenient on police -> innocent
| grandmas in jail.
| GuinansEyebrows wrote:
| despite this being something practically everybody wants,
| the fact that it hasn't happened is not a coincidence and
| speaks to the power of police unions/guilds and their
| lobbying arms. outside a few toothless instances, those
| groups are extremely good at reframing these attempts and
| mobilizing their bases to vote against the broader public
| interest.
|
| it sucks.
| vkou wrote:
| > despite this being something practically everybody
| wants,
|
| No, everybody does _not_ want police accountability. Half
| the population will fall on a grenade to prevent that.
| They know that the purpose of the police is to keep the
| undesirables in line, and they never envision that they
| will ever fall in that category.
|
| The brutality _is_ the point for them.
| GuinansEyebrows wrote:
| oh, i generally don't disagree with you on that point; i
| specifically meant that when presented with the question
| "do you want your tax dollars to pay for police
| liabilities?" the answer is probably almost always "no".
| JumpCrisscross wrote:
| > _we really need a fix for this. When cops are grossly
| negligent the money should come out of their aggregate
| pension fund_
|
| This is on us as voters. If we didn't piss our pants
| every time a police union sneezed, we'd realize wholesale
| restarting police departments is precedents in even our
| largest cities.
| lotsofpulp wrote:
| Almost all taxpayer funded pension funds are already
| underfunded. It makes no difference if the funding
| decreases or increases, the government employee will
| still get their benefit. The government would have to go
| through bankruptcy to get the benefit amount reduced.
| opo wrote:
| Qualified immunity doesn't apply to criminal cases. It is
| used to defend against civil suits. It is unfortunately
| very easy to find many cases where it leads to injustice.
| For example:
|
| >...Abby Tiscareno, a licensed daycare provider in Utah,
| was wrongfully convicted of felony child abuse when a child
| under her care suffered brain hemorrhaging. After calling
| emergency services, subsequent medical tests supported
| these findings. However, during her trial, requested
| medical records from the Utah Division of Child and Family
| Services (DCFS) were not provided. It wasn't until a civil
| suit that Ms. Tiscareno saw pathology reports suggesting
| the injury could have occurred outside of her care. She was
| granted a new trial and acquitted. Her subsequent lawsuit
| for due process violations, alleging that DCFS failed to
| provide exculpatory evidence, was dismissed due to lack of
| precedent indicating DCFS's obligation to produce such
| evidence.
|
| https://innocenceproject.org/news/what-you-need-to-know-
| abou...
| anigbrowl wrote:
| imho the US Marshals are the only innocent party here, as my
| understanding is they don't do investigations and just serve
| warrants without any knowledge of the underlying case.
| john_strinlai wrote:
| > _I hope she wrings at least several million dollars out of
| the government._
|
| which the citizens end up footing the bill for. yay.
| eek2121 wrote:
| Maybe the citizens will learn to elect better leaders.
| phendrenad2 wrote:
| Maybe they'll realize votes have consequences.
| JumpCrisscross wrote:
| "Unable to pay her bills from jail, she lost her home, her car
| and even her dog."
|
| Who stole her dog?!
| wvenable wrote:
| It literally doesn't matter -- you're focused on the wrong
| thing. She could be that woman's exact twin and it wouldn't
| matter. Spending six months in jail and losing your house, your
| car, and your dog with the flimsiest of evidence is ridiculous.
| OutOfHere wrote:
| A lawsuit is exactly what matters. They learn only the hard
| way, and no other way. If you want them to not be ridiculous,
| a lawsuit with large punitive damages is the only practical
| way to get there.
| wvenable wrote:
| I disagree. The city or state gets sued and they pay the
| result from the taxpayer funds and literally nobody learns
| anything, especially not the hard way. Everyone is so
| completely divorced, and in some cases immune, from
| consequences that this will change nothing.
| quickthrowman wrote:
| It's obvious from the one photo they posted of the actual suspect
| that the lady they arrested is about 20-30 years older than the
| woman in the bank photo. The woman in the photo is maybe 25-30
| years old, this grandma looks like she's 65-70 (actual age of
| 50).
|
| Absolutely ridiculous, I hope she wins her civil case.
| Aardwolf wrote:
| This reminds me of the British Post Office Scandal:
| https://en.wikipedia.org/wiki/British_Post_Office_scandal
| pdpi wrote:
| I followed the inquiry when it was ongoing -- all of the
| depositions were live on YouTube. The level of both hubris and
| incompetence involved in that case was breathtaking.
| kawsper wrote:
| If you can get your hands on it, I recommend the 4 episode
| BAFTA-winning mini-series about it:
| https://en.wikipedia.org/wiki/Mr_Bates_vs_The_Post_Office
| anigbrowl wrote:
| It is an AI error, but also an error on the part of the cops, the
| prosecutors, the judge, and the county sheriff (who is
| responsible for the jail inmates). I hope everyone involved in
| this travesty is sued into oblivion and unable to hide behind
| their immunity defenses. Facial recognition should never be the
| sole basis for a warrant.
| causal wrote:
| This x1000. We need to suspend this shared fiction that AI has
| any agency. Only humans can be responsible. Full stop.
| irishcoffee wrote:
| ICE detains innocent woman 1200 miles away based on AI
|
| Same comment?
| GuinansEyebrows wrote:
| respectfully, can you elaborate on why the answer would not
| be yes? or am i just misreading your comment?
| idle_zealot wrote:
| > It is an AI error, but also an error on the part of the cops,
| the prosecutors, the judge, and the county sheriff
|
| Yes, it's critical to remember that _multiple parties can be at
| fault_. In a case like this, it is true that
|
| a) law enforcement misused a tool and demonstrated extreme
| negligence
|
| b) the judiciary didn't catch this, which suggests systemic
| negligence there too when it comes to their oversight
| responsibilities
|
| c) the company selling/providing this AI tool _should have
| known_ it was likely to be misused and is responsible for
| damages caused by such predictable usage
|
| We cannot have a just world until our laws and norms result in
| loss of jobs and legitimacy as punishment for this sort of
| normalized failure, from all three parties. Immunity is a
| failed experiment.
| recursivecaveat wrote:
| Even if she was a read ringer (clearly not the same person to
| any human who glances at the image), common sense should tell
| you that among 340,000,000 Americans there are a lot of
| lookalikes. Clearly there's a kind of stupid belief in the
| mystic powers of an AI and a callous disregard for the well
| being of suspects. No one should be dragged 1000 miles and held
| for months based on a facial match, especially when exculpatory
| evidence was easily available.
| mekoka wrote:
| > It is an AI error
|
| _The software identified the person as Angela Lipps. According
| to the court documents, the Fargo detective working the case
| then looked at Lipps ' social media accounts and Tennessee
| driver's license photo.
|
| In his charging document, the detective wrote that Lipps
| appeared to be the suspect based on facial features, body type
| and hairstyle and color._
|
| The software worked exactly as intended. It's a filtering tool
| that sifts through data for common patterns to provide _leads_
| , not matches. It raises a flag on persons of interest. You can
| be a "match" anywhere between 0 and 100% and only relative to
| some specific input (like that picture taken from the top of
| the woman at the teller). In that sens mismatches are within
| acceptable parameters and have been known to happen.
|
| A "match" is a pronouncement ultimately made by the humans that
| uses the tool, after they've checked out the leads. Someone
| slept at the wheel here.
| jchama wrote:
| The movie "Brazil" was right!
| _doctor_love wrote:
| We do the work, you do the pleasure!
| Pxtl wrote:
| Except in "Brazil" it was a mechanical error in a deterministic
| machine caused by an invasive outside actor. It would be
| reasonable to trust that the autotypewriter/printer would
| faithfully output the correct text.
|
| Modern AI seems incapable of any respectable amount of accuracy
| or precision. Trusting that to destroy somebody's life is even
| more farcical than the oppressive police in "Brazil".
| hsbauauvhabzb wrote:
| Why the fuck does a newspaper need a 'notifications' icon in the
| top right hand corner?
| acuozzo wrote:
| How else can they report on BREAKING NEWS if it doesn't at
| least break your concentration?
| kazinator wrote:
| Because it has an updating-feed-like structure, in which new
| items can appear.
|
| Knowing that there are (N) new items is so useful (to some
| people), that as far back as the 1990s, we developed technology
| called "RSS" to give you this superpower over a website that
| doesn't provide anything of the sort. One that simply updates
| with new stuff when you hit refresh, with no UI to indicate
| what is new/changed.
| holman wrote:
| Me: Whoa, cool, my hometown is on atop Hacker News!
|
| Also me, reading further: Uh-oh.
|
| The chief of police also resigned today; wouldn't be shocked if
| this was part of the reasoning.
| JumpCrisscross wrote:
| > _chief of police also resigned today_
|
| Source?
| waterhouse wrote:
| Googling "fargo police chief resigns":
| https://www.inforum.com/news/fargo/zibolski-announces-his-
| re... among other results.
|
| That said, it's portrayed as a retirement, and doesn't seem
| to give any hints that it's connected.
| JumpCrisscross wrote:
| Out of curiosity, was the guy known for being fast and
| loose with the rules? Put more simply, was he a good cop?
| Or did he have a history of being a rogue.
| Sl1mb0 wrote:
| Are authoritarians good? That's basically what you are
| asking.
| sumeno wrote:
| There are no good cops
| PTOB wrote:
| I am from a town that gets national news coverage only for
| Shenanigans like this.
| causal wrote:
| Wait - what was the AI tool and how did it have her face to begin
| with? If small-town police are doing face-matching searches
| across national databases then nobody is safe because the number
| of false positives is going to be MASSIVE by sheer number of
| people being searched every day.
|
| Pretend the tool is 99.999999% specific. If it searches every
| face in the USA you're still getting about 3 false positives PER
| SEARCH.
|
| You will never have a criminal AI tool safe enough to apply at a
| national scale.
| whack wrote:
| > _According to the court documents, the Fargo detective working
| the case then looked at Lipps ' social media accounts and
| Tennessee driver's license photo. In his charging document, the
| detective wrote that Lipps appeared to be the suspect based on
| facial features, body type and hairstyle and color._
|
| > _Once they were in hand, Fargo police met with him and Lipps at
| the Cass County jail on Dec. 19. She had already been in jail for
| more than five months. It was the first time police interviewed
| her._
|
| How is this the fault of AI? It flagged a possible match. A live
| human detective confirmed it. And the criminal justice system,
| for reasons that have nothing to do with AI, let this woman sit
| in jail for 5 months before doing even interviewing her or doing
| any due diligence.
|
| There's a reason why we don't let AI autonomously jail people.
| Instead of scapegoating an AI bogeyman, maybe we should look
| instead at the professional human-in-the-loop who shirked all
| responsibility, and a criminal justice system that thinks it is
| okay to jail people for 5 months before even starting to assess
| their guilt.
| RobRivera wrote:
| I think it's more nuanced; it is one error in a Tragedy of
| Errors.
| themafia wrote:
| > How is this the fault of AI?
|
| It could be the fault of the company that's selling this
| service. They often make wildly inaccurate claims about the
| utility and accuracy of their systems. [0]
|
| > There's a reason why we don't let AI autonomously jail
| people.
|
| Yes we do. [1]
|
| > and a criminal justice system that thinks it is okay to jail
| people for 5 months before even starting to assess their guilt.
|
| Her guilt was assessed. That's why she had no bail. It assessed
| it incorrectly, but the error is more complicated than your
| reaction implies.
|
| [0]: https://thisisreno.com/2026/03/lawsuit-reno-police-ai-
| polici...
|
| [1]:
| https://projects.tampabay.com/projects/2020/investigations/p...
| rglover wrote:
| > How is this the fault of AI? It flagged a possible match. A
| live human detective confirmed it.
|
| Because we're seeing the first instances of what reality looks
| like with AI in the hands of the average bear. Just like the
| excuse was "but the computer said it was correct," now we're
| just shifting to "but the AI said it was correct."
|
| Don't underestimate how much authority and thinking people will
| delegate to machines. Not to mention the lengths they'll go to
| weasel out of taking responsibility for a screw up like this
| (saw another comment in this thread about the Chief of Police
| stepping down but it being framed as "retirement").
| pj_mukh wrote:
| I'm sorry but this is a piss-poor excuse. When I Claude code
| broken features, I'm responsible 100%.
|
| Why are cops not treated the same way? OP is right, AI is
| totally irrelevant in this story.
|
| If the point is "cops can't be trusted". Why do they have
| GUNS?! AI is the least of your problems.
|
| I feel like I'm going crazy with this narrative.
| wat10000 wrote:
| When are cops ever treated the same way as the rest of us?
| stego-tech wrote:
| You're going crazy because up until this exact moment
| you've never had to confront the reality that these tools,
| placed into the hands of the common man, are viewed as
| authoritative and lack any accountability or consequence
| for misuse.
|
| For anyone who has been victimized by law enforcement or
| governments before, we've been warning about this shit _for
| decades_. About the lack of consequence for police
| brutality. The lack of consequence for LPR abuse. The lack
| of consequence for facial recognition failures and AI
| mismatches.
|
| You need to understand that by using these systems
| correctly and holding yourself accountable, _you are in the
| minority_. Most people do not think that critically, and
| are all too happy to finger the computer when things go
| badly.
|
| And until you accept that, and work to actually hold folks
| accountable instead of deflecting blame away from the tool,
| then this won't actually change.
| pear01 wrote:
| It's called qualified immunity. Many support its repeal. I
| hope you join them, and convey the same to your local
| representatives and candidates. Until it is reformed few if
| any officers or administrators of criminal justice in the
| United States will ever feel any type of accountability.
|
| Short of video evidence of blatant gun to the back of the
| head style homicide qualified immunity means most law
| enforcement officials are never held accountable for their
| miscarriages of justice. Criminal charges against officers
| are exceedingly rare. She should be able to sue this
| detective directly. Of course she can sue the government
| too, and should. But without any personal consequences for
| the people carrying out these acts, taxpayers will continue
| to bail out these practices without ever noticing. Your own
| government should not be a shield for a police officer who
| has violated you or your neighbors.
| caconym_ wrote:
| As soon as we start to see a pattern of shitty vibe-coded
| software actually harming people via defects etc. (see:
| therac-25), I would hope that the conversation is about
| structural change to mitigate risk in aggregate rather than
| just punitive consequences for the individual programmers
| who are "responsible". The latter would be a fantastically
| stupid response and would do little or nothing to reduce
| future harm.
| pj_mukh wrote:
| all accountability need not be punitive, we can certainly
| talk about systemic guardrails. What I find disbelief in,
| is someone saying the Chief of Police saying "We are not
| going to talk about that today?" is not the biggest
| scandal, but the AI is.
| obviouslynotme wrote:
| It's not. This is just an acceleration in the unraveling of
| society facilitated by AI. As someone whose childhood included
| so many "robots will kill humans" books and movies, I am
| flabbergasted that the AI apocalypse will be dumb humans
| overtrusting faulty AI in important matters until everything
| falls apart.
|
| Most humans cannot distinguish AI from actual intelligence.
| When you combine that with bureaucrats innate tendency to say,
| "Computer said so," you end up with bizarre situations like
| this. If a person had made this facial match, another human
| would have relentlessly jeered him. Since a computer running AI
| did it, no one even cared to think about it.
|
| Computers are wildly dangerous, not because of anything innate
| but because of how humans act around them.
| blitzar wrote:
| computer said yes
| caconym_ wrote:
| This particular "AI bogeyman" isn't just AI; it's cops with AI
| and in particular cops with facial recognition tools, dragnet
| LPR surveillance tools, and all this other new technology that
| essentially picks somebody's name out of a hat to have their
| life temporarily (or [semi-]permanently) ruined by shithead
| cops who won't ever face any real accountability.
|
| This keeps happening, and the reason it keeps happening is that
| shithead cops have these tools and are using them. Until we can
| find a reliable way to prevent this from happening, which may
| or may not be possible, cops who may or may not be shitheads
| should not have access to these tools.
| throwaway314155 wrote:
| There's nothing wrong with your comment per se, but it's
| almost as if you didn't even read the comment you're
| responding to.
| Retric wrote:
| Seems like a direct response to me.
|
| >> How is this the fault of AI?
|
| > This particular "AI bogeyman" isn't just AI; it's cops
| with AI
| caconym_ wrote:
| Let me help you out with this comprehension issue. The
| point of my comment is that I disagree with the apparent
| premise of the comment I replied to, which is that "AI" is
| some generic investigative tool that we can neatly snip out
| of the picture to blame this incident on human factors at
| the individual level ("the professional human-in-the-loop
| who shirked all responsibility"). Said comment also implies
| that people are fixating on the AI aspect of this issue
| while ignoring the human factors, which IMO is a strawman.
| To me, the existence of AI in its current incarnations and
| the ways in which law enforcement will _inevitably_ abuse
| it are, together, inseparably, the problem. AI (in the most
| general sense) opens up entire new dimensions for potential
| abuse.
|
| As a concrete example:
|
| > And the criminal justice system, for reasons that have
| nothing to do with AI, let this woman sit in jail for 5
| months before doing even interviewing her or doing any due
| diligence.
|
| Let me state what should be obvious: without AI (as in, the
| facial recognition systems involved in this case), _this
| woman would not have sat in jail for 5 months, or indeed
| for any length of time at all_. So saying that it has
| "nothing to do with AI" is totally ridiculous.
| hinkley wrote:
| It's also cops Making the Numbers Go Up by marking down a
| case file as having progressed because someone is in custody.
| Which isn't about justice.
| mothballed wrote:
| They don't seem to give a single iota of a fuck about that
| when a private regular person has their money stolen or
| their car totaled by hit and run driver. Finding some
| innocent person to arrest would indicate they are at least
| pretending to give a fuck, yet they seem to only be
| bothered to even keep up appearances when it is the bank
| being robbed.
| rightbyte wrote:
| > How is this the fault of AI?
|
| The false positive rate combined with scanning millions of
| pictures might make the chance of arresting the wrong person
| really high.
| stego-tech wrote:
| It's the fault of the tool because our society treats the tools
| as superior judgements than humans and to be trusted completely
| as a means of deflecting accountability - something any and
| every minority group has been warning about _for fucking
| decades_.
|
| The reason everyone rushes to defend the tool's use is because
| holding humans accountable would mean throwing these tools out
| entirely in most cases, due to internal human biases and a
| decline in basic critical and cognitive thinking skills. The
| marketing has been the same since the 80s: the tool is superior
| (until it isn't), the tool shall be trusted completely (until
| it fails), the tool cannot make mistakes (until it does).
|
| If folks actually listened to the victims of this shit,
| companies like Flock and Palantir would be gutted and their
| founders barred from any sort of office of responsibility, _at
| minimum_. The fact so many deflect blame from the tool like the
| marketing manual demands shows they don 't actually give a shit
| about the humans wrapped up in the harms, or the misuse and
| misappropriation of these tools by persons wholly unaccountable
| under the law, but only about defending a shiny thing they
| personally like.
| RobRivera wrote:
| >Unable to pay her bills from jail, she lost her home, her car
| and even her dog. Fargo police say the bank fraud case is still
| under investigation and no arrests have been made.
|
| I smell a lawsuit
| api wrote:
| It's not an AI error. It's a human error in mis-using AI in this
| way. Saying it's an AI error is like saying a hole in your
| drywall is a hammer error.
|
| Unfortunately we'll probably see a trend of people using AI and
| then blaming AI for cases where they mis-used AI in roles it's
| not good for or failed to review or monitor the AI.
| munk-a wrote:
| It's both. It's good to acknowledge that AI is easy to misuse
| in this manner but it doesn't detract from the fact that the
| ultimate responsibility lies in those that should be verifying
| the tool output.
|
| There is far too little skepticism around the magic box that
| solves all problems which is causing issues like this. It's not
| the fault of the AI (as if it could be assigned liability) for
| being misused, but this kind of misuse is far too common right
| now so scare stories like this are helpful and we should
| highlight the use of AI in mistakes like this.
| api wrote:
| I worry that blaming AI at all actually incentivizes humans
| to offload things to AI that should not be offloaded, since
| it lets them escape blame.
| zingar wrote:
| "Computers don't argue" seemed charmingly wrong about how
| computers work until a few short years ago.
|
| https://nob.cs.ucdavis.edu/classes/ecs153-2019-04/readings/c...
| layer8 wrote:
| This quote from a 1979 IBM training manual remains applicable:
|
| "A computer can never be held accountable, therefore a computer
| must never make a management decision."
|
| (https://www.ibm.com/think/insights/ai-decision-making-
| where-...)
| bethekidyouwant wrote:
| I read the article and I don't really understand... she was held
| in a jail in Tennessee but the article states they flew her to
| North Dakota? And somehow she's a fugitive so that's why she
| doesn't get bail? but she's a fugitive held in her own state in a
| holding facility? But then when they release her, she's in North
| Dakota? So if some state says you're a fugitive your home state
| will just hold you in jail until they come and put you on an
| airplane? Is that correct?
| janalsncm wrote:
| I read it as her arrested and held in Tennessee temporarily
| then flown to North Dakota.
| wvenable wrote:
| I think you have the interpretation correct. It seems like any
| state can say you're a fugitive from their state and now you
| have even fewer rights. Every day I learn some new fact about
| "justice" in the United States.
| jmyeet wrote:
| We are rapidly becoming a world where every person is one
| inscrutable LLM decision from having their life ruined with no
| recourse.
|
| This type of incident isn't new and is only going to get worse.
| The problem is our governments are doing absolutely nothing about
| it. I'll give two examples:
|
| 1. Hertz implemented a system where they falsely reported cars as
| being stolen. People were arrested and went to jail for rental
| cars that were sitting in the Hertz lot. Hertz ultimately had to
| pay $168 million in a settlement [1]. That's insufficient. If I,
| as an ordinary citizen, make a false police report that somebody
| stole my car I can be criminally charged. And rightly so. People
| should go to jail for this and it will continue until they do.
| These fines and settlements are just the cost of doing business;
| and
|
| 2. The UK government contracted Fujitsu to produce a new system
| for their post offices. That system was allowed to produce
| criminal charges for fraud that were completely false. People
| committed suicide over this. This went on for what? A decade or
| more? But resuted in a parliamentary inquiry and settlements.
| It's known as the British Post Office scandal [2]. Again, people
| should go to jail for this.
|
| The choice we as a society face is whether to have automation
| improve all of our lives by raising everyone's standard of living
| and allowing us to do less work and less menial work or do we
| allow automation to further suppress wages so the Epstein class
| can be slightly more wealthy.
|
| [1]: https://www.npr.org/2022/12/06/1140998674/hertz-false-
| accusa...
|
| [2]: https://en.wikipedia.org/wiki/British_Post_Office_scandal
| temp0826 wrote:
| Even in Idiocracy they didn't have this problem
| chrisjj wrote:
| There's an opportunity for an "AI" app here. Takes your photo,
| compares with mugshots on police databases, quotes you for
| requisite cosmetic surgery.
|
| /i
| puppycodes wrote:
| They do not care.
|
| End qualified immunity and see how fast cops start to do their
| jobs with care.
|
| Winning a lawsuit literally ends in your own community members
| (not the cops) paying the bill.
| jawns wrote:
| John Bryant, aka The Civil Rights Lawyer, recently did a piece
| about a similar case of mistaken identity. The consequences
| weren't as severe, but the willingness to trust the AI over any
| other evidence was the same:
|
| https://thecivilrightslawyer.com/2026/03/11/ai-software-tell...
|
| In the video, it shows a police officer blindly trusting a
| casino's AI software, even when a cursory investigation should
| have given any reasonable person enough of a reason to question
| whether the man he arrested was the same man accused of a crime.
| (And then even after it was confirmed he was not, the prosecutor
| continued to charge him for trespassing!)
| Lerc wrote:
| This problem predates modern AI.
| https://en.wikipedia.org/wiki/Computer_says_no is built upon the
| deliberate abdication of responsibility to processes that cannot
| be held accountable. AI is just letting them do it at scale.
|
| That doesn't mean we should accept it from AI. We should fight
| the blind yielding to the facade of authority regardless of
| whether the decision was made by an AI or an insect landing on a
| teleprinter at the wrong time.
| stego-tech wrote:
| I really, _really_ need folks to understand that deflecting blame
| away from the tool and trying to hold the human accountable feeds
| right into the marketing playbook of these companies in the first
| place.
|
| The cops _cannot_ be held accountable because the laws basically
| give them immunity. The politicians _cannot_ be held accountable
| beyond being tossed out at the next election, because the laws
| otherwise give them immunity. The people operating the system
| _cannot_ be held accountable, because the systems are marketed as
| _authoritative_ despite being black boxes and lacking in
| transparency; they trusted the system just as they were told to,
| _and thus cannot be held accountable_.
|
| And so when _every human in the chain_ cannot be held accountable
| for these things, and the law prevents victims from receiving
| apologies, let alone recourse, then _the tool and its maker is
| the only thing we can hold accountable_. By deflecting blame away
| from the tools ( "it wasn't AI, it was facial recognition"; "the
| human had to sign off on it"; "humans made the arrest, not
| machines"), you're protecting quite literally the _only possible
| entity_ that could still potentially be held accountable: the
| dipshits making these stupid things and marketing them as
| superior and authoritative when compared to humans.
|
| You want accountability? Start holding capital to account, and
| this shit falls away _real fucking fast_. Don 't get lost in
| technical nuance over very real human issues.
___________________________________________________________________
(page generated 2026-03-12 23:00 UTC)