[HN Gopher] FTC orders 'gun detection' tech maker Evolv to stop ...
___________________________________________________________________
FTC orders 'gun detection' tech maker Evolv to stop overstating
effectiveness
Author : shakna
Score : 128 points
Date : 2024-12-30 15:01 UTC (8 hours ago)
(HTM) web link (www.techdirt.com)
(TXT) w3m dump (www.techdirt.com)
| idontwantthis wrote:
| Clark County School District recently signed a $5 million
| contract with a different AI "Weapon Detection System". There is
| no way any of them are real is there?
|
| Edit: CCSD uses Remark. Maybe their AI actually knows what a
| person walking with a gun somewhere on their body looks like in
| all situations and for all skin colors.
| pavel_lishin wrote:
| Seems like it would cost less than $5m to independently test
| this unit before purchase.
| Aeolun wrote:
| There's definitely some form of filtering you can apply to
| metal detector results to make a relatively good guess on
| whether it's a false positive or not.
|
| Just need to train it with a variety of hidden and unhidden
| items. I kinda like the idea of a team of test people hiding
| their weapons everywhere and then taking multiple trips through
| the thing just to teach it.
| idontwantthis wrote:
| It's not a metal detector it's optical. What does a person
| with a pistol strapped to their inner thigh walk like? Or a
| knife? There is no possible way to detect that visually.
| xrd wrote:
| I cannot find any summaries for the cost of these detection
| devices. What did this cost, anyone know?
| ceejayoz wrote:
| Probably a lot.
|
| https://en.wikipedia.org/wiki/ADE_651 made tens of millions of
| dollars at ~$5,000/unit.
|
| > The laboratory found that the card contained only a standard
| radio frequency ID tag of the type used in stores to prevent
| shoplifting. According to the laboratory's Dr. Markus Kuhn, it
| was "impossible" for the card to detect anything and it had
| "absolutely nothing to do with the detection of TNT". The card
| could not be programmed, had no memory, no microprocessor and
| no form of information could be stored on it. Despite the high
| cost of the devices, the cards were worth only about two to
| three pence (3-5C/) each. Kuhn commented: "These are the
| cheapest bit of electronics that you can get that look vaguely
| electronic and are sufficiently flat to fit inside a card." The
| "card reader" was found to be an empty plastic box.
| gs17 wrote:
| > Promotional material issued by ATSC claimed that the ADE
| 651 could detect such item as guns, ammunition, drugs,
| truffles, human bodies, contraband ivory and bank notes at
| distances of up to 1 kilometre (0.62 mi), underground,
| through walls, underwater or even from aircraft at an
| altitude of up to 5 kilometres (3.1 mi).
|
| Could they have at least tried to make their lies believable?
| pavel_lishin wrote:
| I assume it's like Nigerian Prince emails - if they were
| believable, they might have appealed to buyers who would
| have done something like purchasing one and _testing it_ ,
| instead of spending millions of dollars on garbage.
| jandrese wrote:
| This is it exactly. All of the serious people left before
| they finished the sales pitch. If they had sold one to
| someone competent that would be grounds for a lawsuit or
| fraud charges. They had to filter out the customer base
| to just two kinds of buyers:
|
| 1. The people who fail to realize that it's a big scam
| and will be fat dumb and happy forever.
|
| 2. The people who don't want something functional, they
| want a "probable cause" generator they can pull out when
| there isn't any evidence to go on.
| mrandish wrote:
| Sounds like they're basically making minimally plausible
| props for customers more focused on security theater
| performances than any actual security.
|
| It's scummy and dumb but legally the only real problem I
| see is lying and collecting money under false pretenses
| making it fraud. Alternatively, they could have chosen to
| market these products confidentially as essentially
| security placebos. There's a market for things like fake
| security cameras, which arguably have some value for
| deterrence and reassurance. I suspect many of their current
| customers would probably have been just as happy buying
| these products knowing they didn't work.
| jandrese wrote:
| I don't think it's quite as innocent as security theater.
| I think these are more for generating probable cause when
| they need to shake down someone who isn't doing anything
| illegal.
| mrandish wrote:
| Ah, well if the design allows manual triggering as
| opposed to some random percentage, then that would
| obviously be bad.
| ceejayoz wrote:
| It's mostly based on the ideomotor effect. It's _all_
| manual triggering, just mostly unconscious.
| recursivecaveat wrote:
| > He told The Times that ATSC had been dealing with doubters
| for ten years and that the device was merely being criticised
| because of its "primitive" appearance. He said: "We are
| working on a new model that has flashing lights".
|
| Damn, why am I not a scam artist? This worked so well for so
| long. If he put $20 of random electronics and tiny glass
| tubes of chemicals in there I think he could've gotten away
| with it.
| giantg2 wrote:
| I mean, the company has never really said how their systems
| worked. It's basically left the users/scanned with the impression
| that it's magic. It seems to me that it's basically some of 3D
| metal detection paired with cameras. It would be cool to
| understand how they make it work, but I guess it's a trade
| secret. They haven't readily put up numbers about false positive
| rates, and even less data about false negatives. I get the
| impression that this relies heavily on placebo effect - "they
| have scanners so I better not try to take my weapon in".
| unwise-exe wrote:
| I thought advertising and marketing materials were _expected_ to
| exaggerate things to the point of blatant falsehood.
|
| Is that changing, or is this company being singled out for some
| reason, or are they really that much worse than everyone else?
| Zak wrote:
| It is, and has long been illegal to make false statements of
| fact in marketing materials.
|
| _Puffery_ , on the other hand is allowed. That usually entails
| non-falsifiable statements like "Evolv is the _best_ way to
| detect guns ". "Best" doesn't really mean anything because
| there are a bunch of tradeoffs that go into designing a
| security screening system.
| conover wrote:
| Right. "The world's best cookie". I think the idea is that no
| reasonable person would understand that statement to mean
| that literally, of all the cookies in the world, this one
| right here is the best.
| Zak wrote:
| At least equally important to whether a reasonable person
| would take it literally is that there's no way to prove it
| isn't. If I call something the best cookie, the closest
| thing that has to a concrete meaning is "I like it better
| than any other cookie".
| skywhopper wrote:
| It's illegal, and has been for many decades. It's not enforced
| nearly as often or as strictly as it should be.
| kjs3 wrote:
| In the US (YMMV) there's a distinction in advertising between
| exaggeration for effect, like "this expensive cream will make
| you _beautiful_ ", and straight up falsification like "we can
| detect all guns everywhere" when you demonstrably can't.
| too_pricey wrote:
| Multiple concert venues in my city use these, so I interact with
| them all the time. They have replaced standard metal detectors,
| bag searches, and manual patdowns w/ hands and/or metal-detecting
| wands. Security checkpoints are the primary point of delay for
| getting into venues, and places that have rolled these out
| process people through about 95% faster. It's a huge difference.
| If it does trigger, you just get the manual patdown you would
| have gotten anyway, so the false positive cases aren't any lost
| time.
|
| The article and settlement seem to only mention the false
| positive rate, which is a bad thing to focus on. Every true
| positive is a much faster experience. Only subjecting 110 out of
| 3000 people to a longer search is a big improvement. Given the
| negative outcomes of a gun slipping through and the lack of a
| cost of a false positive, we probably want it to be tuned to be
| more false positive prone anyway. We don't need these to detect
| guns THAT well, we just need them to weed out people who
| definitely don't have them.
|
| I do have concerns about what its false negative rate is relative
| to the standard practice it replaces. I do not really trust
| whatever psuedo-AI they're bolting to their metal detectors; it's
| probably easier to get a gun through. That said, the false
| negative rate probably isn't good already. TSA isn't great on
| their false positive rate, does more intense screening, and isn't
| being staffed by hungover 20-somethings. So maybe the false
| negative rate didn't actually increase by much?
| ggreer wrote:
| An acceptable false negative rate really depends on the
| consequences of getting caught.
|
| If someone's nefarious plan depends on smuggling a gun in, they
| want to be confident they won't be arrested or shot at the
| entrance. Even failing to detect 20% of firearms means there's
| an 80% chance they'll be caught before they can do whatever it
| is they plan on doing. This is also why it's important to have
| armed guards alongside the scanners. Scanners aren't very
| useful if the only armed person is the bad guy.
|
| If the consequences of getting caught are negligible (as is the
| case for anyone trying to bring a box cutter through airport
| security), then the attacker can try as many times as they want
| without issue. Even if the false negative rate is low, they
| only have to get lucky once.
|
| Annoyingly, I can't find any info about false positive/negative
| rates for various scanners. There doesn't seem to be the
| equivalent of Consumer Reports or Underwriters Labs for
| scanners. My guess is that the numbers must be pretty bad if
| companies aren't willing to go through public 3rd party
| testing.
| mrandish wrote:
| The TSA's own security tests clearly show a significant
| percentage of guns, knives and explosives regularly get
| through. This is further confirmed by the number of travelers
| who, after arriving, discover the handgun they accidentally
| left in some pouch in their suitcase that was never detected.
|
| > There doesn't seem to be the equivalent of Consumer Reports
| or Underwriters Labs for scanners. My guess is that the
| numbers must be pretty bad if companies aren't willing to go
| through public 3rd party testing.
|
| Of course they are but the main reason there's no publicly
| available objective testing isn't _only_ that sellers don 't
| want it. In reality, no stakeholder in the security market
| wants it. The vast majority of high-volume public security
| like airports, concerts and sporting events is largely
| unnecessary and mostly ineffective but our current
| political/media environment requires appearing to "do
| _something_ " to "make it safe". The Vice-President of "Make
| it (Seem) Safe" knows that their shareholders, politicians
| and the public aren't willing to pay more or be even more
| inconvenienced than they already are for 800% better "Make it
| (Seem) Safe"-ness.
|
| Metaphorically speaking, the tiger repellent is working just
| fine, thank you. Those truly worried about tiger attacks feel
| safer and those being well-paid for preventing tiger attacks
| can claim virtually 100% effectiveness. So, if you start the
| world's best Tiger Repellent Testing Laboratory, you'll find
| a shocking lack of interest in buying your test data from
| both sellers and buyers in this brisk, profitable and growing
| market. Much like the lack of interest in objective testing
| data for lie detectors, astrology readings and placebo pills.
| The smaller minority of customers actually willing to pay
| more for improved detection (like Tel Aviv airport), do their
| own in-context performance testing anyway. In fact, a good
| proxy for doing your own effectiveness testing is available
| for free. Just look at what those under constant active
| threat with real consequences actually pay for and do.
| yonaguska wrote:
| I unknowingly transported ammo both to and from Mexico. I
| used an old backpack that I had previously used as a range
| bag from years ago. I ended up finding several 223 rounds
| in Mexico, then when I got back, even more 22lr.
| bwilliams18 wrote:
| Others have not been so lucky
|
| https://www.independent.co.uk/news/world/americas/kyle-
| busch...
| mrandish wrote:
| > Security checkpoints are the primary point of delay for
| getting into venues, and places that have rolled these out
| process people through about 95% faster. It's a huge
| difference.
|
| I assume expediting peak crowd throughput at low labor cost is
| the primary, if not entire, value of the device. I hate that
| it's being marketed dishonestly but I also assume most concert
| venue buyers know (or suspect) it probably doesn't work all
| that well in practice. However, in a concert context accurate
| detection isn't their main priority. They need to get more
| bodies per minute into the venue at lower cost while appearing
| to conduct security checks sufficiently 'real' enough to act as
| a deterrent to get those who care about getting 'caught' to
| leave their knife or concealed carry handgun (or whatever) in
| the car.
|
| The only hard and fast requirement is meeting the contractual
| security requirements of the venue and promoter's insurance
| carriers - because no insurance = no concert. It's a bonus if
| the 'security' also looks plausible enough to reassure the
| small fraction of perpetually fearful people statistically
| challenged enough to actually worry about terrorists or an
| active shooter killing them while at a Taylor Swift concert (as
| opposed to the infinitely more likely chance of dying in a car
| crash on the way to the concert).
|
| In a perfect world, everyone would be rational and numerate
| enough that we wouldn't need to maintain the pretense of
| 'security theater' in contexts where actual security isn't
| necessary. But in the imperfect world we live in, I prefer
| having concert (and airport) security be as minimally
| disruptive and inexpensive as possible regardless of
| effectiveness (since it's unnecessary and mostly ineffective in
| those contexts anyway). I just wish companies would sell these
| products as 'security placebos' instead of lying about it
| because fraud is wrong.
| mlyle wrote:
| I don't know anything about the devices that are being sold,
| but it doesn't seem impossible to me that with better signal
| processing from a metal detector, you could reduce false
| positives a bit while maintaining or slightly improving the
| false negative rate.
| mrandish wrote:
| Sure, I agree that's an interesting and likely solvable
| technical problem. However, the vast majority of the
| addressable market in today's over-secured society don't
| really need improved detection. Concert venues, sports
| arenas and similar customers buy massive volume and they
| are much more concerned with faster throughput enabled by
| shorter cycle time and minimal false positives. Of course
| the head of security at Madison Square Garden can never
| publicly admit they don't care about better detection
| enough to pay more for it, but I'm confident the sales
| managers at these security vendors understand exactly what
| their largest market segments really care about.
|
| Customers like Tel Aviv International Airport, who actually
| care to some meaningful extent about improved detection,
| are a small minority segment of the overall market.
| Creating new technical measures able to demonstrate
| improved performance in rigorous objective tests on the
| metrics these customers care about (some sweeter spot on
| the matrix of false pos, false neg, true pos, true neg, net
| throughput, cost) would be valuable but only to that small
| segment.
| HeyLaughingBoy wrote:
| Very likely, but who's going to pay for the engineering
| efforts? If the customer doesn't care about having a better
| device, or a better device would actually make their job
| harder, then it's wasted effort on the part of the
| manufacturer.
|
| Engineering exists to solve a problem. It's entirely likely
| that your definition of the "problem" differs from that of
| the paying customer.
| lupusreal wrote:
| > _get more bodies per minute into the venue at lower cost
| while appearing to conduct security checks sufficiently
| 'real' enough to act as a deterrent_
|
| Or they just need to convince most of their customers it will
| be safe to attend, while covering their ass by following
| "best practices" if something slips through, people get hurt
| and they get sued.
| mrandish wrote:
| > convince most of their customers it will be safe to
| attend
|
| Apparently, you've never met my Aunt Sue. She has a
| graduate degree in innumeracy with a minor in illiteracy
| and a specialization in worrying about whatever the media
| tells her to worry about. However, she _always_ votes.
|
| More seriously, it's not cost-effective to "convince most
| customers it will be safe _enough_ to attend. " The game
| theory around fallacious public perception makes it a
| losing proposition for a politician or company to ever
| appear to reduce security requirements because as soon as
| "rare bad thing happens", they will be blamed - even though
| their reduction in pointless measures had no bearing on it.
|
| Most independent experts agree that securing cockpit doors
| in 2002 made subjecting every passenger to the TSA's
| increased security measures unnecessary and, objectively, a
| very poor ROI in both cost and disruption. However, the TSA
| will never, ever go away - even though it could and should.
| Not only is reducing security politically costly, the TSA
| is now a multi-billion dollar federal bureaucracy, paying
| hundreds of vendors with lobbyists and employing tens of
| thousands of unionized workers spread across the most
| populous congressional districts. Yes, this is frustrating.
| Zak wrote:
| > _The only hard and fast requirement is meeting the
| contractual security requirements of the venue and promoter
| 's insurance carriers _
|
| I think it would be a good idea to create an explicit carve
| out in the law saying that there is no premise liability for
| a property owner or event organizer due to a third party
| committing a crime.
| staticautomatic wrote:
| So, do away with all negligent security cases?
| Zak wrote:
| Yes. In general, business owners aren't expected to
| prevent crimes against their customers. If someone
| attacks me at a bar or grocery store, I probably won't
| get very far trying to sue the owner for failing to check
| everyone for weapons on entry. I'm not sure I'd have more
| success with a concert venue, but it appears insurance
| companies perceive enough risk to demand certain
| procedures.
|
| Codifying that expectation in law would reduce costly and
| obnoxious security theater. Of course, a business
| advertising a certain level of security could be sued for
| failing to provide it.
| staticautomatic wrote:
| Ok, but it seems like a bit of a non-sequitur to say "
| business owners aren't expected to prevent crimes against
| their customers" when there's a body of law to the
| contrary.
| Zak wrote:
| Is there? In most US states, the concept of premises
| liability seems to be derived entirely from case law, not
| statute. Some states appear to have statutes limiting its
| scope, such as
| https://colorado.public.law/statutes/crs_13-21-115
|
| Edit: to be clear, I don't think there's anything
| actually stopping someone from attempting to sue a bar or
| grocery store over a crime committed there, but it
| usually doesn't happen and would likely be an uphill
| battle for the plaintiff.
| staticautomatic wrote:
| So what? It's not like common law has less effect. "Body
| of law" is understood by lawyers to include both common
| and statutory law.
| Zak wrote:
| I will concede the technical point: there is a body of
| law that sometimes expects business owners to prevent
| crimes and sometimes doesn't, with a whole lot of
| ambiguity about exactly what any given owner is actually
| expected to do. I think that ambiguity should be reduced
| by putting criminal acts out of scope.
| potato3732842 wrote:
| The point is that it can easily be overridden with
| statute
|
| "business owners meeting definition X are only liable in
| conditions Y"
| LordDragonfang wrote:
| >TSA isn't great on their false positive rate, does more
| intense screening, and isn't being staffed by hungover
| 20-somethings. So maybe the false negative rate didn't actually
| increase by much?
|
| TSA is abysmal on the false negative rate for things that
| actually matter. The FNR for actual weapons and explosives is
| somewhere between 80 and 95%[1]. It's because they waste all of
| their attention looking for nail clippers and water bottles.
|
| Even an FNR of 50% would be a massive improvement.
|
| [1] https://abcnews.go.com/US/tsa-fails-tests-latest-
| undercover-...
| dawnerd wrote:
| Theme parks are switching to these too and it's oh so much
| nicer to just walk through and if you're unlucky having your
| bag searched.
| scrose wrote:
| Interesting how a guy whose entire administration is being
| investigated and indicted by the FBI ran almost entirely on a
| 'law and order' platform, with this pilot being one of the
| staples.
| stronglikedan wrote:
| The "investigated and indicted by the FBI" that lead to nothing
| for the "law and order" candidate? Interesting that the
| "lawfair" candidate would have had the complete opposite
| outcome, had the laws been equally enforced.
| bwilliams18 wrote:
| You're confusing Eric Adams with Donald Trump.
___________________________________________________________________
(page generated 2024-12-30 23:01 UTC)