[HN Gopher] The Dark Past of Algorithms That Associate Appearanc...
___________________________________________________________________
The Dark Past of Algorithms That Associate Appearance and
Criminality
Author : nkurz
Score : 67 points
Date : 2021-01-16 02:45 UTC (1 days ago)
(HTM) web link (www.americanscientist.org)
(TXT) w3m dump (www.americanscientist.org)
| kodah wrote:
| > that criminals are under-evolved, subhuman beasts
|
| This has everything to do with common attitudes. Ask any person
| about what kind of person does the kind of crime or moral
| aberration they are most passionate about and you'll get a
| similar response. People can't imagine that criminals might have
| had some understandable train of thought prior to committing a
| crime; or worse that they may have done exactly the same thing in
| their shoes.
|
| > These algorithms reportedly make false accusations against
| students with disabilities who move their faces and hands in
| atypical ways, and Black students have indicated that they have
| been required to shine bright lights in their faces so as to have
| their features detected at all.
|
| Warning: I will be using the term police quite broadly here, but
| it's anyone involved in any kind of policing or detective
| activity.
|
| Any algorithm used in policing that produces false negatives
| needs to be retired until it can be proven they can be used
| provably equally on people, which I suspect is never. The problem
| is that most people believe that policing should be more
| efficient. More efficient police work means violating your
| privacy, breaking algorithms, and using lazy surveillance and
| data collection. If you want your freedoms intact then good old
| fashioned police and detective work needs to be just that; the
| police need to be smarter, not have smarter tools.
|
| > Yet equating convictions with criminality seems to register
| with the authors mainly as an empirical flaw, in that using
| pictures of convicted criminals, but not of the ones who were
| cleared, introduces a statistical bias that skews the results.
| They said they were "deeply baffled" at the public outrage in
| reaction to a study that was intended "for pure academic
| discussions," which also suggests an unawareness that their
| work's flaws go beyond sloppy statistics.
|
| Absolutely a fair criticism. I'm surprised anyone was baffled
| that the criminal justice system is highly flawed, other than
| some mathematicians who lead quite prestigious lives. I've long
| thought it's pretty suspect that there's a morality bar forming
| (or that has formed) in programming. A force beyond laws can make
| you unhireable, what a precedent! Really, I find the idea that we
| might deny entry to programming even for convicted felons that
| have served their time problematic. Their insights might be quite
| useful here.
|
| Edit: Changed "Americans" to "people"
| nooyurrsdey wrote:
| > Ask any American about what kind of person does the kind of
| crime or moral aberration they are most passionate about and
| you'll get a similar response > .... > Especially Americans
| can't imagine...
|
| Why is this targeting Americans? In my home country these same
| biases exist. There's a "group of people" that are deemed more
| likely prone to criminal behavior.
|
| For some reason America seems to serve as a lightning rod for
| these types of criticisms, as if the rest of the world is
| better somehow.
| Aerroon wrote:
| The American context is easier to understand for most people.
| We're surrounded by American culture on English-speaking
| websites. The context of <other country> isn't as easy to
| understand for people that don't know <other country> well.
| kodah wrote:
| I'm certainly not saying your struggles don't exist. I live
| in America and it'd be disingenuous and probably counter-
| intuitive for me to create an exhaustive list of all people's
| who suffer these circumstances.
|
| That's to say, I can really only speak in earnest about the
| devil that I know.
| [deleted]
| TheOperator wrote:
| This article was eye rollingly heavy on moralizing ultimately to
| only eventually hit on the point that what the algorithms are
| measuring is the odds somebody will be convicted and not the odds
| somebody does illegal things. Appearance is known to dramatically
| affect your chances at trial.
| jetrink wrote:
| I remember reading about a similar result in software that
| judged job applicants. The system was trained on resumes of
| past applicants and the hired/not-hired outcome. Unfortunately,
| instead of providing a neutral guide, it just learned the
| biases of the humans who had made past hiring decisions. Of
| course, the designers of the system had excluded demographic
| information, but the system found proxies like extra-curricular
| activities that correlated with race, gender or social class.
| MaximumYComb wrote:
| I seem to recall reading somewhere that good looks affect the
| sentence but that they play a lesser role in if you're
| convicted.
| dane-pgp wrote:
| Of course you'd say that, you have the brainpan of a stagecoach
| tilter.
|
| https://www.youtube.com/watch?v=ySXpRf3ggJE
| CWuestefeld wrote:
| I had previously written a critique of this article, for a
| different forum. I'm reproducing it here, slightly edited for
| this venue:
|
| With concerns of systemic racial bias such a hot topic today, the
| subject of hidden bias in machine learning systems driven by "big
| data" has been a frequent question. An article "The Dark Past of
| Algorithms That Associate Appearance and Criminality", by
| Catherine Stinson, appearing in the January-February 2021 issue
| of American Scientist, attempts to make the point that "Machine
| learning that links personality and physical traits warrants
| critical review", and largely succeeds at this task. However,
| most of the piece attempts to provide that critical review, and
| at this task it fails.
|
| The article starts off offering phrenology as a close cousin to
| contemporary efforts to use facial recognition to identify
| criminals, or as "gaydar". A series of examples of it (mis)use in
| Chinese surveillance, or to monitor schoolchildren to ensure they
| pay attention in class, capping this off with the claim that
| Black students have had to endure bright lights pointed at their
| faces to enable such systems to work. Such applications do seem
| unfair, and potentially so disruptive as to be counterproductive.
| So in this way the author is successful in showing a need for
| critical review.
|
| Stinson goes on to discuss how some of the derided foundations of
| phrenology - the idea that particular cognitive functions can
| each be seated in a particular locale within the brain - has now
| become "a standard assumption in mainstream neuroscience".
| However, she then claims that despite this, the science has
| suffered from sloppy methods. One example study is cited that
| looked for criminality based on photographic input had biased
| training data by using police photos to represent the convicts,
| versus professional photos scraped from the web for nonconvicts.
| It's easy to see how this would skew the results.
|
| The author first claims, "using pictures of convicted criminals,
| but not of the ones who were cleared, introduces a statistical
| bias that skews the results", and that seems a fair point. She
| also says, "the researchers also admitted that taking court
| convictions as synonymous with criminality was a 'serious
| oversight'", but offers no explanation for why that should be. My
| best guess is that we should be able to assume that a "guilty"
| verdict is the best judgment we can get as to whether someone is
| guilty of a given crime; but that other people who are guilty of
| a crime but never caught will be left out of the data set, so our
| training would bias the system to give a pass to people who don't
| get caught. I suppose that's fair, too.
|
| It's at this point that I think things start to derail. Stinson
| goes on to say,
|
| _their framing suggests that criminality is an innate
| characteristic, rather than a response to social conditions such
| as poverty or abuse, or a label applied to exert social control.
| [...] Part of what makes the data set questionable on empirical
| grounds is that who gets labeled "criminal" is hardly value-
| neutral._
|
| That's largely a philosophical question that she doesn't attempt
| to back up. But the fact that she does make this point, about
| extrinsic forces being an important contributor, actually defeats
| her later argument.
|
| Along the way, she feels the need to throw in the standard anti-
| HBD argument that "psychologists who studied the heritability of
| intelligence, such as Cyril Burt and Philippe Rushton, had to
| play fast and loose with their data to make it look like they had
| found genuine connections between skull size, race, and IQ." I
| don't intend to debate that here, there's ample material out
| there showing the correlation between race and IQ (although I
| don't know about skull size). Either way, it doesn't seem to be
| relevant to her actual argument about revelations of criminality
| from facial characteristics.
|
| Her main argument starts
|
| _Complex personal traits such as a tendency to commit crimes are
| exceedingly unlikely to be genetically linked to appearance in
| such a way as to be readable from photographs. First, criminality
| would have to be determined to a significant extent by genes
| rather than environment. There may be some very weak genetic
| influences, but any that exist would be washed out by the much
| larger influence of environment._
|
| This is just wrong. Genes do not need to be the determinant here.
| There are any number of external factors that can point to
| someone's criminality:
|
| + A person's age is a strong clue to criminality, and we can
| judge age based on facial appearance quite readily
|
| _" more than half of all homicides in 2014 were committed by
| 15-to-29-year olds, who only made up 20 percent of the
| population"_ [1]
|
| _" The relationship between age and crime is one of the most
| robust relationships in all of criminology. This relationship
| shows that crime increases in early adolescence, around the age
| of 14, peaks in the early to mid 20s, and then declines
| thereafter."_ [2]
|
| + Probably the best clue as to criminality is past criminal
| history. Jailhouse tattoos (like the tear thing) are probably
| excellent hints here. There are even studies that claim to relate
| tattoo type and location to the type of crime that an inmate is
| incarcerated for
|
| _" The effect is even more pronounced for those with tattoos on
| the head or face, who are around 30% less likely to be murderers.
| Similar associations can be found for perpetrators of domestic
| crimes."_ [3]
|
| + Property crimes correlates with poverty, and poverty leaves its
| marks on a person's visage as well. (this is where her previous
| point about criminality not being an intrinsic trait actually
| undermines the argument). See, e.g., [4]
|
| She continues:
|
| _Second, the genetic markers relevant to criminality would need
| to be linked in a regular way to genes that determine appearance.
| This link could happen if genes relevant to criminality were
| clustered in one section of the genome that happens to be near
| genes relevant to face shape. For a complex social trait such as
| criminality, this clustering is extremely unlikely. A much more
| likely hypothesis is that any association that exists between
| appearance and criminality works in the opposite direction: A
| person's appearance influences how other people treat them, and
| these social influences are what drives some people to commit
| crimes (or to be found guilty of them)._
|
| She's doubling down on the social influence thing, but see my
| rebuttal about poverty just above.
|
| But more importantly, her argument is again false. As improbable
| as it may seem to her, we know for sure that at least some
| important factors linked to criminality are, indeed, found
| relatively localized in the genome. It's established that genes
| on the Y chromosome lead to greater criminality: in other words,
| it's well documented that men are more likely to be criminals
| than are women.
|
| _Males were convicted of the vast majority of homicides in the
| United States, representing 89.5% of the total number of
| offenders._ [5]
|
| Her conclusion compares these technologies to cold fusion, saying
| that continued work on these questions is worse than useless
| because it's not just wasted, it also serves in "propping up
| colonial and class structures" - bringing non-scientific
| politicization into the argument. The trouble is, while she's
| shown that - like any tool - this facial recognition technology
| can be misused, she hasn't even tried to look into how it might
| be used for good.
|
| I've been a subscriber to American Scientist for many years, ever
| since Scientific American became so overtly political. Over the
| past few issues, American Scientist has been debating with itself
| about whether it should go in this direction. The editor's note
| in the previous issue made me worry that it would be going in
| this direction, and this article solidifies that. Anybody know of
| a good outlet for scientific information, a couple notches more
| in-depth than Discover or Popular Science, but still approachable
| to a well-educated generalist?
|
| - - - -
|
| [1]
| https://onlinelibrary.wiley.com/doi/pdf/10.1002/978111851963...
|
| [2] https://www.sagepub.com/sites/default/files/upm-
| binaries/602...
|
| [3]
| https://webcache.googleusercontent.com/search?q=cache:https%...
|
| [4]
| https://www.researchgate.net/publication/221674142_The_dynam...
|
| [5]
| https://en.wikipedia.org/wiki/Sex_differences_in_crime#Stati...
___________________________________________________________________
(page generated 2021-01-17 23:01 UTC)