[HN Gopher] AI recognition of patient race in medical imaging: a...
___________________________________________________________________
AI recognition of patient race in medical imaging: a modelling
study
Author : aunterste
Score : 71 points
Date : 2022-05-17 12:33 UTC (10 hours ago)
(HTM) web link (www.thelancet.com)
(TXT) w3m dump (www.thelancet.com)
| HWR_14 wrote:
| A lot of people are proposing simple reasons why this could be
| the case. They did so last year when the study that inspired this
| got published.
|
| Maybe this needs to be updated from physicists:
| https://xkcd.com/793/
| ppqqrr wrote:
| So there's material differences that supports certain prejudices;
| big surprise, turns out human societies have been (and still is)
| working very hard for thousands of years to craft those
| differences - isolating, separating, enslaving, oppressing,
| exiling their scapegoat "others". The question is not whether the
| differences are real, but whether we can prevent AI from being
| used to perpetuate those differences. TBH, we don't stand a
| chance; we live in a society where most people cannot even wrap
| their heads around why it _shouldn't_ perpetuate those
| differences.
| tejohnso wrote:
| "This issue creates an enormous risk for all model deployments in
| medical imaging: if an AI model relies on its ability to detect
| racial identity to make medical decisions, but in doing so
| produced race-specific errors, clinical radiologists would not be
| able to tell, thereby possibly leading to errors in health-care
| decision processes."
|
| Why would a model rely on its ability to detect racial identity
| to make decisions?
|
| What kind of errors are race-specific?
| matthewdgreen wrote:
| https://www.hopkinsmedicine.org/news/media/releases/er_docto...
| amarshall wrote:
| Just because the model relies on race in some way doesn't mean
| that _we_ know it relies on it. I.e., the model is, unbeknownst
| to us, biased on race in inaccurate ways.
| codefreeordie wrote:
| Presumably the model would actually be biased on race in
| _accurate_ ways, if it found the correlation itself
| amarshall wrote:
| Maybe, maybe not. Hard to say--which is the problem they
| call out in the paper
|
| > efforts to control [model race-prediction] when it is
| undesirable will be challenging and demand further study
| codefreeordie wrote:
| The correlation being "undesirable" to the individuals
| doing the research does not mean that the correlation is
| inaccurate.
|
| I mean, sure, there are _tons_ of ways for garbage data
| to sneak into ML models -- though these guys tried pretty
| hard to control for that -- but if the model actually
| determined that "race" is a meaningful feature, then
| that might be because it is, and science should be
| concerned with what is, not with what we wish were.
| amarshall wrote:
| If one believes and proclaims that they have controlled
| for variable X, but they haven't actually done so, then
| their results and analysis may well be invalid or
| misleading because of that. Whether they actually should
| have controlled for X or not is orthogonal.
| codefreeordie wrote:
| Oh, yes, sorry. If by the correlation being possibly-
| undesirable you meant that it was possibly-spurious due
| to incompletely controlling for some bias in the source
| data, then yes, conclusions based on a model which found
| such a spurious correlation caused by incomplete input
| control might be undesirably biased in a not-accurate
| fashion.
|
| This study appears to have done a good job controlling
| for known biases that could have been proxies for race,
| but it is presumably possible that they missed something
| and tainted the data
| amarshall wrote:
| Right, and that's pretty much the conclusion: our
| explicit goal was to control for race, and yet, we appear
| to have failed and don't know why (so don't know how to
| adjust the control yet). So likely others using similar-
| enough methodologies and techniques are unknowingly
| failing to control.
| Ancapistani wrote:
| I could be entirely wrong here, so if you've got more
| context in this area by all means correct me.
|
| Consider an "AI" that rates the probability of recidivism
| for prisoners nearing their parole date. That score would
| then be presented to the parole board, and taken into
| consideration in determining whether or not to grant
| parole. If this AI were accidentally/incidentally
| accurately determining the race of the prisoner, then the
| output score would take that into account as well. Black
| men have a recidivism rate significantly higher than other
| groups[1]. The reasons for the above aside - it's a complex
| topic, and outside the scope of this analogy - this is
| extremely undesirable behavior for a process that is
| intended to remove human biases.
|
| You might then ask, how does this relate to medical
| imaging? Medical decisions are regularly made based on the
| expected lifespan of the individual. It makes little sense
| to aggressively treat leukemia in a patient who is
| currently undergoing unrelated failure of multiple organs.
| Similarly it would likely make sense for a healthy 30-year-
| old to undergo a joint replacement and associated physical
| therapy, because that person can reasonably be expected to
| live for an additional 40 years while the same treatment
| wouldn't make sense for a 70-year-old with long-term
| chronic issues. This concept is commonly represented as
| "QALY" - "quality-adjusted life years".
|
| Life expectancy can vary significantly based on race[2].
|
| An AI that evaluates medical imagery that considers QALY in
| providing a care recommendation may result in a positive
| indicator for a white hispanic woman and a negative
| indicator for a black non-hispanic man, with all else being
| equal and with race as the only differentiator.
|
| In short - it's not necessarily a bad thing for a model to
| be able to predict the race of the input imagery. The
| problem is that we don't know why it can do so. Unless we
| know that, we can't trust that the output is actually
| measuring what we intend it to be measuring.
|
| 1: https://prisoninsight.com/recidivism-the-ultimate-guide/
| 2: https://www.cdc.gov/nchs/products/databriefs/db244.htm
| codefreeordie wrote:
| At the risk of discussing sensitive topics on a platform
| ill-suited:
|
| If, in your hypothetical recidivism case, an AI
| "accurately" determined that a pattern of higher
| recidivism-related features was correlated to race, and
| was able to determine "accurately" that the specific
| subset of recidivism-related features predicted race, why
| would it be wrong to make parole decisions using those
| recidivism-related features?
| gadflyinyoureye wrote:
| If you decided on race, in this instance, you would be
| making people much more deterministic as a result of the
| power of race. Race is too broad a concept to reliably
| say that all white people are at X chance of recidivism.
| Instead we want to know if Marlowe is at risk of high
| recidivism based on her character.
| codefreeordie wrote:
| Both responses address the problem with a human making a
| biased decision based on race, which I think mostly we
| all agree would be bad.
|
| The question I was posing is different, though, because
| this was discussing an AI system that looked at the
| underlying [in this case, recidivism] data _which had
| race and race-adjacent information removed_ , and the AI
| has effectively rediscovered the concept of "race" by
| connecting it to some set of attributes of the actual [in
| this case, recidivism-predicting] features. If the AI
| were to determine such a link, that doesn't make its
| results biased, it just makes them uncomfortable. It's
| not clear to me that in such a case that would mean that
| we should remove those [recidivism-predicting] features
| from the dataset just because they ended up being
| correlated to race.
| pessimizer wrote:
| Because both the original conviction and any recidivism
| is determined through the decision-making of people who
| are aware of race and racial stereotypes. The AI would
| just be laundering the decisions you were already making,
| not improving them.
|
| edit: imagine I was a teacher who systematically scored
| people with certain physical characteristics 10% lower
| than people who didn't have them. Let's say, for example,
| that I was a stand-up comedy teacher that wasn't amused
| by women.
|
| If I used an AI trained on that data to choose future
| admissions (assuming plentiful applicants), I would end
| up with an all-male class. If this happened throughout
| the industry (especially noting that the all-male
| enrollment that I have would supply the teachers of the
| future), stand-up comedy would simply become a thing that
| women were seen as not having the aptitude to do,
| although nobody explicitly ever meant to sabotage women,
| just to direct them into something that they would have a
| better chance to succeed in.
| eklitzke wrote:
| Let's say you're trying to train an model to predict if a
| patient has a cancerous tumor based on some imaging data. You
| have a data set for this that includes images from people with
| tumors and people without, from all races. However, unbeknownst
| to you, most of the images from people of race X had tumors and
| most of the images from people of race Y did not have tumors.
|
| If the AI is also implicitly learning to detect race from the
| images, it's going to learn an association that people of race
| X usually have tumors and people of race Y usually do not.
|
| The problem here is that the people training the model and the
| clinical radiologists interpreting data from the model may not
| realize that race was a confounding factor in training, so
| they'll be unaware that the model may make racial inferences in
| the real world data.
|
| If people of race X really do have a higher incidence rate for
| a specific type of cancer than race Y, maybe this is OK. But if
| the issue is that there was bias in the training/validation
| data set that was unknown to the people building the model, and
| in the real world people of race X and race Y have exactly the
| same incidence rate for this type of cancer, then this is going
| to be a problem because it's likely to introduce race-specific
| errors.
| [deleted]
| SpicyLemonZest wrote:
| Using race as an independent factor to make medical decisions
| isn't unheard of today. The medical community is largely trying
| to stop doing that as a matter of social policy, so it's a
| problem for that goal if an AI model might be doing it under
| the hood.
|
| See e.g. https://www.ucsf.edu/news/2021/09/421466/new-kidney-
| function...
| ars wrote:
| If this is true I suspect a human could be trained the same way.
|
| I read once that a radiologist can't always explain what they see
| in an image that leads them to one diagnosis or another, they say
| that after seeing many of them they just know.
|
| So I suspect the same could be done for race. This would be a
| super interesting thing to try with some college students - pay
| them to train for a few days on images and see how they do.
| uberwindung wrote:
| .."In this modelling study, we defined race as a social,
| political, and legal construct that relates to the interaction
| between external perceptions (ie, "how do others see me?") and
| self-identification, and specifically make use of self-reported
| race of patients in all of our experiments."
|
| Garbage research.
| [deleted]
| groby_b wrote:
| And you are qualified to make that assessment because...?
| axg11 wrote:
| Perfect example of citations-driven research. The authors
| aren't motivated by a genuinely interesting scientific question
| ("are anatomical differences between genetically distinct
| groups of people visible in X-rays?"). Instead, the authors
| know that training a classifier to predict race will generate
| controversial headlines and tweets. All publicity, positive or
| negative, leads to more citations.
| colinmhayes wrote:
| > genetically distinct groups of people
|
| Is race a genetically distinct marker though? I guess if you
| limit the sample enough it is, but I've always thought of
| race as more of a continuous quality than a distinct one.
| axg11 wrote:
| Race _is_ a spectrum but genetic differences themselves are
| distinct (SNPs). It's trivial to train a classifier to
| distinguish race from genetic data, hence, I'd argue they
| are distinct groups.
| Imnimo wrote:
| The fact that the model seems to be able to make highly accurate
| predictions even on the images in Figure 2 (including HPF 50 and
| LPF 10) makes me skeptical. It feels much more probable that this
| is a sign of data leakage than that the underlying true signal is
| so strong that it persists even under these transformations.
|
| https://arxiv.org/pdf/2011.06496.pdf
|
| Compare the performance under high pass and low pass filters in
| this paper on CIFAR-10. Is it really the case that
| differentiating cats from airplanes is so much more fragile than
| predicting race from chest x-rays?
| kerblang wrote:
| > Importantly, if used, such models would lead to more patients
| who are Black and female being *incorrectly* identified as
| healthy
|
| I think this is the point a lot of people are missing; they
| think, "So what if 'black' correlates to unhealthy and the model
| notices? It's just seeing the truth!"
|
| However, I'm still wondering how this incorrectness works; can
| anyone explain?
|
| Edit: Clue: The AI is predicting _self-reported_ race, and the
| authors indicated that self-reported race correlates poorly to
| _actual_ genetic differences.
| KaiserPro wrote:
| My guess is that they are using an american dataset. This I
| would suspect encodes socioeconomic data into the samples. ie
| rich people, have access to better diagnostics, get seen
| earlier and are treated sooner. Conversely poorer present later
| and with more obvious symptoms. _also_ the type of system used
| to take the images would also be strongly correlated.
| jl6 wrote:
| > Models trained on low-pass filtered images maintained high
| performance even for highly degraded images. More strikingly,
| models that were trained on high-pass filtered images maintained
| performance well beyond the point that the degraded images
| contained no recognisable structures; to the human coauthors and
| radiologists it was not clear that the image was an x-ray at all.
|
| What voodoo have they unearthed?
| civilized wrote:
| I think there's a significantly greater than zero chance that
| they simply botched their ML pipeline horribly and would get
| their 0.98 AUCs from completely blank images.
| JumpCrisscross wrote:
| > _What voodoo have they unearthed?_
|
| Curious for the take not of a neuro-ophthalmologist. If they
| too are stumped, this may be a path to a deeper understanding
| our visual system.
|
| Simple transformations obviously discernible to us blind
| computer vision. (CAPTCHAs.) There may be analogs for human
| vision which don't present in the natural world. Evidence of
| such artefacts would partially validate our current path for
| artificial intelligence, as it suggests the aforementioned
| failures of our primitive AIs have analogs in our own.
| 6gvONxR4sf7o wrote:
| I think it's pretty straightforward. Imagine the fourier
| transforms of some recognizeable audio signals. Maybe a
| symphony and a traffic jam. They'll look totally different,
| even to the naked eye. If you chop off the low frequency
| components, you can still probably tell which fourier spectrum
| is which. But now do the same thing in time domain (high-pass
| filter the audio). It probably won't be clear that you're
| listening to a symphony anymore.
| proto-n wrote:
| I tend to not believe unbelievable results in machine learning.
| It's too easy to unintenionally cause some kind of information
| leakage. I haven't read the paper in detail though, so their
| experimentation setup could be foolproof, this is not a
| critique of this paper specifically.
| dragonwriter wrote:
| > I tend to not believe unbelievable results
|
| That seems tautologically true.
| sidewndr46 wrote:
| This reminds me of the ML research that could predict sex
| from an iris. It turns out they were using entire photos of
| eyes to do this. There are so many obvious cues to pick up on
| in that case, like eyeliner, eyelashes being uniform (or
| fake), trimmed eyebrows, general makeup on the skin, etc.
| Der_Einzige wrote:
| ... Adversarial examples.
|
| It's a whole field of research, and it's pretty trivial to
| generate them for most classes of ML models. It's actually
| quite difficult to create robust models that DON'T have this
| problem...
| tech-historian wrote:
| The interpretation part hit home: "The results from our study
| emphasise that the ability of AI deep learning models to predict
| self-reported race is itself not the issue of importance.
| However, our finding that AI can accurately predict self-reported
| race, even from corrupted, cropped, and noised medical images,
| often when clinical experts cannot, creates an enormous risk for
| all model deployments in medical imaging."
| Animats wrote:
| "Predict self-reported race". Not race from DNA. (That's
| routinely available from 23andMe, and is considered an
| objective measurement.[1]) They should have collected both. Now
| they don't know what they've measured.
|
| [1] https://www.nytimes.com/2021/02/16/opinion/23andme-
| ancestry-...
| nerdponx wrote:
| I suspect this is a "tank vs sky" problem. The article says
| that the bright areas of bone are not the most important for
| predicting race. What if it's some features of different
| hospitals and x-ray setups?
|
| Also did they release their code and anonymized data? If not,
| it's impossible to tell if this is a bug.
|
| If I got this result in my work, I would check it 10k times
| over because it defies belief. Even allowing subtle skeletal
| differences in different ethnic groups, the differences in this
| case are not in the bone and at least sometimes not visible to
| the human eye. Unless there is an undiscovered difference in
| radio-opacity across ethnicities, the result doesn't make
| sense.
| nerdponx wrote:
| Replying to my own post because I can't edit it anymore.
|
| Apparently this is a known and persistent affect across a
| variety of other medical images, tests, and scans. Not just
| for a "race" but for ethnic groups in general, as well as
| biological sex. So this might actually just be an "AI hit
| piece" that otherwise confirms an unpalatable but persistent
| and strong effect in the literature. The causes seem to be
| badly understudied, in part due of the obvious need for
| delicacy and respect around such topics.
|
| This result is tremendously implausible to me, but I am
| finding quite a few articles documenting similar phenomena
| across things like retina scans and brain MRIs.
| prometheus76 wrote:
| What you are experiencing is cognitive dissonance. Take
| your time. It's never fun.
| aulin wrote:
| what's this enormous risk they're talking about? racial bias in
| x-ray reading? race can be a risk factor in plenty of diseases,
| why should we actively try to remove this information from
| medical images?
| [deleted]
| KaiserPro wrote:
| > racial bias in x-ray reading?
|
| no, it implies there is a signal in the dataset that could be
| something other than clinical. This means that until they can
| pinpoint the cause, or the thing the AI is detecting, all the
| other things it predicts are suspect.
|
| ie if the AI thinks the subject is west african, then it
| might be more inclined to diagnose something related to
| sickle cell.
|
| Or north western european woman in her mid 60s vs a japanese
| woman might get widly different bone density readings for the
| same level of "blob" (most medical imaging is divining the
| meaning of blobs and smears )
| sim7c00 wrote:
| soon they will want to remove race indicators for photographs
| and tik tok videos. who knows, maybe its racist to be of a
| race >.>
| matthewdgreen wrote:
| "This issue creates an enormous risk for all model
| deployments in medical imaging: if an AI model relies on its
| ability to detect racial identity to make medical decisions,
| but in doing so produced race-specific errors, clinical
| radiologists (who do not typically have access to racial
| demographic information) would not be able to tell, thereby
| possibly leading to errors in health-care decision
| processes."
| ibejoeb wrote:
| Typically? It's coded in the standard. There's a DICOM tag
| for it.
|
| https://dicom.innolitics.com/ciods/procedure-
| log/patient/001...
| matthewdgreen wrote:
| Unlike the authors of this research paper I am not a
| trained clinician, so I can't tell you. However I would
| note that the first exemplary value in the link you gave
| me is "REMOVED".
| ibejoeb wrote:
| It doesn't provide example data, but there's still a spot
| in the standard for it. The values can differ by modality
| or manufacturer. Sure, it's not required, but certainly
| it's very important in some situations. Consider
| dermoscopy.
|
| If interested, searching for "dicom conformance" should
| yield lots of docs that probably contain specific values
| for those things.
| ska wrote:
| FWIW, the standard printed out is multiple linear feet of
| shelf space. There is a spot for a lot of things.
|
| One common issue is a lot of these kinds of tags rely on
| optional human input and are inconsistently applied. As
| opposed to say, modality specific parameters produced by
| a machine, which are consistent.
|
| DICOM is a great example of design by committee, with the
| +'ve and -'ves that implies.
| aulin wrote:
| ok, maybe it's an US specific thing, why wouldn't a
| clinical radiologist have all the information he can gather
| about his patient including race to help the diagnosis?
| codefreeordie wrote:
| Because in the US we are required to pretend that there
| is no such thing as race and no such thing as gender, and
| all people are exactly and precisely the same and there
| can be no differences.
| sandworm101 wrote:
| >> Because in the US we are required to pretend that
| there is no such thing as race
|
| Then you are not pretending very well. When I lived in
| the US I was shocked at how often it was an issue. It
| permeates nearly every aspect of US culture.
|
| The icing on that cake: A government-run interactive map
| so you can lookup which races live in which
| neighborhoods. Some versions allow you to zoom in to see
| little dots representing clusters of black or white
| residents. https://www.census.gov/library/visualizations/
| 2021/geo/demog...
| nradov wrote:
| Actually, the US federal government specifically
| recommends that healthcare providers record patients'
| race, ethnicity, assigned sex, and gender identity. Most
| of those elements are self identified.
|
| https://www.healthit.gov/isa/uscdi-data-class/patient-
| demogr...
| Loughla wrote:
| Not to get into a flame war, but I want to present an
| alternate option to yours.
|
| Because in the US some people have a hard time
| understanding that all races and genders deserve to be
| treated equally as humans with the same access to goods
| and services. Further, that there are disparities in care
| based on race/ethnicity[1][2] and gender[3][4] because of
| that racism/sexism present in the systems. This then
| leads to requiring that race/ethnicity and gender data be
| scrubbed sometimes to keep people from impacting outcomes
| based on their own biases.
|
| [1] https://www.americanbar.org/groups/crsj/publications/
| human_r...
|
| [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1924616/
|
| [3] https://www.americashealthrankings.org/learn/reports/
| 2019-se...
|
| [4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2965695/
| codefreeordie wrote:
| It sometimes makes sense to scrub race/ethnicity/gender
| information from certain types of data, typically when a
| human is going to be making individual decisions.
|
| For example, not having race data on resumes is generally
| productive, because that categorization can't provide a
| meaningful input to the decision associated with an
| individual person. Even if it were to be the case that
| there was some correlation between race and skill at
| whatever job you're interviewing for[1], the size of the
| effect is almost certainly small, and in the meanwhile
| you've also controlled for any bias in the person doing
| the reviewing.
|
| If you're having a machine look at a dataset, and the
| machine determines that race or ethnicity is a material
| factor in determining some attribute in that dataset,
| you're not doing anybody any good by denying that fact
| and destroying the result.
|
| [1]Let's ignore for the purposes of this discussion,
| fields (like certain sports) where extreme competition
| combines with a position heavily dependent upon racially-
| linked physical characteristics. Though even in this
| case, there is still a (different, weaker) argument for
| suppressing race data in "resumes" (yes, I know,
| ballplayers don't submit resumes to their local NBA
| franchise)
| bumby wrote:
| > _If you 're having a machine look at a dataset, and the
| machine determines that race or ethnicity is a material
| factor in determining some attribute in that dataset..._
|
| I think the trickiness is in providing the machine
| unbiased data to begin with so that it doesn't incorrect
| associations between features like race. The most
| egregious examples I'm aware of are the machine learning
| systems used to suggest criminal sentencing, but, apropos
| to this topic I believe there are cases where it may
| produce erroneous associations in something like skin
| cancer risk.
| nerdponx wrote:
| Then you should be looking at ethnicity and not "race" as
| such. For example, Ashkenazi Jews as an ethnic group are
| genetically very distinct from other Europeans, but are
| generally considered "white" on self-reported race
| surveys.
| pessimizer wrote:
| Race is a rough, subjective, culturally-bound summary of
| characteristics. If you're already evaluating
| characteristics, adding either your guess of race or a
| self-reported race is like injecting gossip into good
| data.
|
| If the outcome that you're trying to predict is also
| affected by perceptions of race, you've built a gossip
| feedback loop.
| nradov wrote:
| I don't understand that part. All modern EHRs have a field
| for self-reported race, and clinical radiologists do
| typically have access to that information. (Whether they
| actually look at it, or whether it's useful when reading
| images, are separate issues.)
| towaway15463 wrote:
| Without knowing the actual outcome, isn't there also a
| possibility of error due to not knowing the race of the
| individual? They used mammogram images in the study and it
| is well known that incidence of breast cancer varies by
| race. Removing that information from the model could result
| in worse performance.
| cameldrv wrote:
| Well one thing you wouldn't want to do is take the output
| of this model and then apply a correction factor for race
| on top of it, because the model is already taking that
| into account.
| towaway15463 wrote:
| Is that true or would it help as a tie breaker in cases
| where the confidence was just at or below the threshold?
| Retric wrote:
| AI is driven by the training sets, but the goal is to find
| the underling issues.
|
| Suppose AI #1 got a higher score on the training data and AI
| #2 had a more accurate diagnosis. Obviously you want #2 but
| if there is bias in the training data based on race and the
| AI has access to race then eventually you overfit into #1.
| dekhn wrote:
| yep, the case for "enormous risk" hasn't been well
| articulated. It's been repeated a lot, but of all the
| problems in medical care, this isn't one of the larger ones.
| pdpi wrote:
| ML models are great tools, but they're way too much of a
| black box. What you have here is a model that's predicting
| something you think it shouldn't have been possible to
| predict, and you can't simply ask it where that prediction
| comes from. Absent an explanation for how the model is doing
| this, you have to consider the possibility that whatever is
| poisoning that prediction will also poison others.
| axg11 wrote:
| > ML models are great tools, but they're way too much of a
| black box.
|
| A human doctor is also a black box, in meat form.
| fumblebee wrote:
| My first thought here is to relate this to the problem of
| early colour film, which was largely tested and validated
| with only light skin tones in mind. Once it was put out into
| the wild, folks with darker skin tones found the product to
| be total crap. Why? Because there was a glaring OOD (Out of
| Distribution) problem during testing.
|
| Similarly, if the train/test sets used here - for X-ray based
| diagnostics - using Machine Learning relies only on specific
| races, then the performance _might_ be worse for other races,
| given that there 's a new discriminatory variable in play.
|
| The obvious solution here is to reduce bias by ensuring race
| is part of the dataset used for training and testing. Which,
| due to PII laws in play, may actually be quite challenging!
| Fascinating tradeoff imo.
| [deleted]
| unsupp0rted wrote:
| What if it turns out that humans have identifiable biological
| differences among genetic sub-groups, ethnicities, etc? It
| would be anarchy in the social sciences.
| ibejoeb wrote:
| I don't get it either. It's accurate. It would be a problem
| if it got it wrong, which could, for example, underweight
| quantitative genetic data and adversely influence
| differential diagnosis.
| [deleted]
| oaktrout wrote:
| I recall seeing a paper in the early 2010s with an algorithm that
| could discriminate between white and Asian based on head MRI
| images. I'm having trouble finding it now, but this finding to me
| is not too surprising.
| bitcurious wrote:
| I would guess a causal chain through environmental factors, given
| how much archeologists are able to tell about prehisotric humans'
| lives based on bone samples.
|
| Bone density, micro fractures and deviations in shape. The
| mongols had famously had bowed legs from spending a majority of
| their waking lives on horseback.
| wittycardio wrote:
| I don't trust medical journals or experimental AI research to be
| particularly scientific so I'll just throw this into the
| meaningless bin for now.
| mensetmanusman wrote:
| What does this mean in terms of race being a social
| construct/concept?
| badrabbit wrote:
| Still is? The AI is correlating biological features with self
| reported race. There are biological differences between people
| who have different ancestors. Finns are different from brits.
| The spanish are different from russians. Nigerians look
| different than somalians. The Japanese look differnet than
| filipinos.
|
| Race picks specific and arbitrary differences , for example
| hispanic is a different race in US society but black and white
| based on skin color are as well, indians and east asians are
| also one "race".
|
| Ethnicities are not social constructs but race is. The AI finds
| ethnic differences and correlates them with self-percievied
| social/racial classification.
|
| "Race" as the evil social construct it is, takes ethnic
| differences and intrprets them to mean some ethnicities are
| different races of humans than others as in not just different
| ancestors but differently created or evolved despite all
| evidence and major religion saying all humans are one species
| (homosapiens) that have a common homosapien ancestor.
|
| I thought all this was obvious but the social climate recently
| is very weird.
| fullstackchris wrote:
| Race as an entire concept to me has always been stupid at
| best. Sure, there are vast swaths of biological similarities
| (typically, though not necessarily) according to general
| geographic regions of the globe, but the real mistake was
| trying to give this vague concept a label. Can anybody give a
| definition of what "white" or "black" REALLY means? It's an
| impossible task. If we're talking just visually about skin
| color, congratulations, that represents citizens of some 100+
| odd countries on the planet and just as many (if not many
| more!) cultures and languages. But leave it to humans to try
| and over optimize and try to denote evermore meaningless
| abstractions...
|
| Let the social "culture war" rage on. The only war I see
| going on in the west (U.S. mostly) is a _lack_ of culture.
| dijit wrote:
| I believe you're inverting race and ethnicity.
|
| From national geographic: "Race" is usually associated with
| biology and linked with physical characteristics such as skin
| color or hair texture. "Ethnicity" is linked with cultural
| expression and identification.
| badrabbit wrote:
| I am not. Historically ethnicities and ancestoral lines
| aligned so ethnic differences and biological differences
| due to generational mating choices largely influenced by
| the culture that is a component of the ethnicity are
| aligned as well. Race is not a biological classification
| because it is appearance based but arbitrary. Appearance is
| not the same as biology. A husky appears similar to a small
| wolf but it might be correct to consider them (dogs) a race
| of wolves.
|
| The deceptively evil part of the concept of race is, it
| does not simply differentiate biological features but it
| goes on to impose a fork at the root of the ancestoral tree
| where people of that race share the same origin and same
| differences. In reality biological differences are a result
| if what a culture considers attractive multiplied by
| mutations that help people adopt to different environments
| (e.g.: skin color being a result of adaptation to sun light
| and vitamin d levels instead of a being a feature that
| shows ancestoral forks in creation or evolution).
|
| It is simply inaccurate to label people by race but it is
| useful to impose social evils. But biological differences
| due to mating and cultural choices are very real and can be
| examined at a granular level that takes the actual factors
| for the differences into account instead of the lazy+evil
| correlation that is the concept of race.
|
| Ethnicity is not what culture you identify with. You don't
| become ethnically african american because you like african
| american culture and grew in a specific neighborhood. It is
| the marriage of culture and ancestry.
| nradov wrote:
| Medical software used in the US classifies Hispanic as an
| ethnic group, not a race. Those are separate fields in a
| patient's chart. Here is the official federal government
| guideline.
|
| https://www.healthit.gov/isa/taxonomy/term/741/uscdi-v2
|
| https://www.healthit.gov/isa/taxonomy/term/746/uscdi-v2
|
| (I'm not claiming that this is an optimal approach, just
| pointing out how it works in most software today.)
| badrabbit wrote:
| Yes, but socially when you ask people their race they will
| say black or hispanic or white. And with little consitency.
| It is more of a way to justify and impose social classes.
| towaway15463 wrote:
| So isn't the "evil social construct" part actually the
| invalid extension of the theory that biological or phenotypic
| differences mean that someone is more or less human? You can
| remove that part and still acknowledge that there are
| biological differences between people based on their genetic
| lineage without invalidating their basic humanity.
| badrabbit wrote:
| The evil part is not differences but considering people as
| part of a different race of humans because of those
| differences.
| [deleted]
| ibejoeb wrote:
| I'm just going to abandon the term race because nothing
| constructive is going to come from it. It is not contentious
| that there are various physiological developments among groups
| of humans.
| pessimizer wrote:
| > various physiological developments among groups
|
| This is a very contrived way to say that people share
| characteristics with other people. The real question is why
| people don't say that I belong to the six-foot tall bad-knees
| race.
| ibejoeb wrote:
| Really? It's very contrived?
|
| I'm not here to tell you what to do. Use race then. I
| offered up why I think this article is only generating
| interest is because race is a loaded word, and if it
| weren't used, it'd be passed over.
|
| > The real question is why people don't say that I belong
| to the six-foot tall bad-knees race
|
| This is an article about ML accurately predicting self-
| identified race. This is not even on the spectrum of real
| questions.
| HKH2 wrote:
| What term are you going to use instead? Subspecies? Breed?
| ibejoeb wrote:
| I don't know. My hunch is that these suggestions, though,
| will be received poorly.
| dijit wrote:
| Race, in terms of physiology has never been regarded by science
| to be a social construct.
|
| In fact it can be medically harmful to think this way.
| wutbrodo wrote:
| Unfortunately, that's not quite true. Here's the AMA[1] with
| a press release entitled "New AMA policies recognize race as
| a social, not biological, construct".
|
| They discourage using race as a source of any physiological
| signal. They do allow using genetics, but the relevant
| situations are the many many ones where genetic testing isn't
| possible or doesn't yet provide useful signal.
|
| Unaccountable institutions get captured very easily, and the
| race cult that's swept through our educated class has been a
| very powerful one.
|
| [1] https://www.ama-assn.org/press-center/press-releases/new-
| ama...
| PartiallyTyped wrote:
| One of the reasons certain communities were hit harder with
| Covid was vit D deficiency as a consequence of skin color.
| dijit wrote:
| That is one hypothesised cause for the disparity, social
| factors in those cases need to be controlled for.
|
| A better discussion is around sickle cell anaemia[0] which
| is exclusively carried by people of African or Afro-
| Caribbean descent.
|
| [0]: https://en.wikipedia.org/wiki/Sickle_cell_disease
| PartiallyTyped wrote:
| That's a better example, thank you. Reminded me that I
| know quite a few people with Thalassemia/Mediterranean
| anemia.
| pessimizer wrote:
| Sickle cell disease is exclusively caused by genetics,
| not race. The vast majority of people of African or Afro-
| Caribbean descent aren't carriers, so have the same
| likelihood as everyone else who is not a carrier to
| develop it.
| pessimizer wrote:
| Skin color isn't race.
| tom_ wrote:
| Same as it always did, as humans have long claimed to be able
| to distinguish race simply by looking.
| bb123 wrote:
| Perhaps there is some quality of the x rays themselves that is
| different? Maybe white people tend to visit hospitals with
| newer, better equipment or better trained radiographers and the
| model is picking up on differences in the exposures from that.
| [deleted]
| shortgiraffe wrote:
| They mostly accounted for this: >Race prediction performance
| was also robust across models trained on single equipment and
| single hospital location on the chest x-ray and mammogram
| datasets
|
| Sure, it's possible that bias due to the radiographer is the
| culprit, but this seems unlikely.
| oaktrout wrote:
| From the paper "Race prediction performance was also robust
| across models trained on single equipment and single hospital
| location on the chest x-ray and mammogram datasets"
| gbasin wrote:
| Interesting hypothesis but I can't tell if you're being
| serious
| polski-g wrote:
| It means that is a lie.
| [deleted]
| PheonixPharts wrote:
| I've noticed people in all parts of the political spectrum have
| a hard time understanding the term "social construct". It
| doesn't mean the same as "completely made up".
|
| Nations are uncontroversially recognized as a social
| constructs. However I'm certain that AI could also detect
| images taken outdoors in Mexico vs those in Finland.
| Additionally I, a US citizen, cannot simply declare that I am
| now a citizen of France and expect to get a French passport.
|
| However it also means that what a nation is, is not set in
| stone for eternity. It means that different people can debate
| about the precise definitions of about what defines a
| particular nation. It means that Czechoslovakia _can_ become
| the Czech republic and Slovakia. It means that not everyone
| agrees if Transnistria _is_ an independent nation. It means
| that the EU _can_ decide that a German citizen can have the
| same passport as a French citizen.
|
| As a more controversial example, this is also the case when
| people talk about gender being a "social construct". It doesn't
| mean that we can simply pretend like the ideas "men" and
| "women" doesn't exist (as people both declare and fear). But it
| does mean there is some flexibility in these terms and we as a
| society can choose how we want these ideas to evolve.
|
| Society is a complex and powerful part of our reality, arguably
| more impactful on us from day to day than most of physics
| (after all we did survive for hundreds of thousands of years
| without even understanding the basics of physics). Therefore
| something being classified as a "social construct" doesn't mean
| it "isn't real". Even more important is that individuals cannot
| choose who social construct evolve. I cannot, for example,
| declare that since taxes are a social construct, I'm not paying
| them anymore. We can however, as a society, change what and how
| these constructs are interpreted.
| mlindner wrote:
| Science does not claim that race is a social
| construct/concept...
| SpicyLemonZest wrote:
| While I agree with you that "social construct" isn't the
| right way to think about it, the authors in this very paper
| say that it is.
| emddudley wrote:
| There is no scientific, consistent way to define race. The
| groups we put people into is fairly arbitrary. They don't
| correlate to appearance, genetics, country of origin, etc.
|
| An interesting question in the U.S. is "who is considered
| white?" There was a Supreme Court case in which someone who was
| literally from the Caucasus was ruled not white. This is why
| it's sociological, not scientific.
|
| https://www.sceneonradio.org/episode-40-citizen-thind-seeing...
| orangepurple wrote:
| Alloco 2007 looked at random locations of single nucleotide
| polymorphisms(SNPs) and found that, using random SNPs, you
| still get very good correspondence between self-
| identification and best fit genetic cluster. Using as few as
| 100 randomly selected SNPs, they found a roughly 97%
| correspondence between self-reported ancestry and best-fit
| genetic cluster.
|
| https://pubmed.ncbi.nlm.nih.gov/17349058/
| pessimizer wrote:
| Were the formulations of genetic clusters created through
| marking samples with self-reported race? If so, why
| couldn't you create an entirely different rubric of race by
| choosing a few arbitrary features to define each of them
| and find exactly the same thing?
| daenz wrote:
| If there's no scientific, consistent way to define race, how
| is it that a machine learning model is able to pick the race
| that somebody self-identifies as consistently? The model is
| simply using rules based on math to deduce an accurate guess.
| greenthrow wrote:
| It still is. Just because it includes physical signifiers that
| can be measured doesn't mean it isn't still a social construct.
|
| To give a contrived example; if I say people with ring fingers
| over 3 inches long are Longfings and people wkth ring fingers 3
| inches or less are Shortfings, and then out society treats
| people differently based on being Longfing or Shortfing, this
| is a social construct that is causing problems for people based
| on a contrived criteria that has no real meaning. The same is
| true of race.
| inkblotuniverse wrote:
| What if shortfings tend to be drastically taller, and the
| longfings are complaining that they're overrepresented in
| jumpball?
| pessimizer wrote:
| > What if shortfings tend to be drastically taller
|
| What does it mean for shortfings to be dramatically taller?
| Are you saying that shortfings must transmit height along
| with finger length; some sort of race invariance? Or are
| you saying that most shortfings you meet are also tall?
|
| If a black person is a pale as a white person, they're
| still considered black (and may share many other
| characteristics that many black people have.) If some of
| your shortfings have long fingers, does the distinction
| still make sense as a scientific category?
|
| > the longfings are complaining that they're
| overrepresented in jumpball?
|
| Is admission to jumpball determined by finger measuring, or
| through social factors?
| inglor_cz wrote:
| Ever more complicated attempts to bridge the gap by muddying
| the waters.
|
| Frankly, even a freshly arrived alien from Mars or Titan could
| easily tell Icelanders, Mongols and Xhosa apart, without
| knowing anything about our culture. The fact that there has
| been a lot of interbreeding/admixture since the Age of Sail
| began, does not mean that there aren't meaningful biological
| differences between the original groups, which still obviously
| exist.
|
| An analogy: much like the existence of twilight does not render
| the concept of night and day a 'social construct' either. We
| attach certain social meanings to those natural phenomena, and
| a 'working day' can easily stretch into 'astronomical night'
| (all too often!), but that does not mean that 'night' and 'day'
| do not exist outside of our cultural reference framework.
|
| There is a social concept of 'race' which corresponds to the
| 'working day' concept in this analogy, e.g. 'BIPOC', claiming
| Asians as 'white adjacent' or classifying North Africans or
| Jews as 'white', even though they may not necessarily look
| white. But this is almost certainly not what the AI identified.
| This social concept of race _would_ confuse a Martian alien
| unless he started to study the social and racial history of the
| U.S., and possibly even afterwards. It definitely confuses me,
| a random observer from Central Europe.
| daniel-cussen wrote:
| It could actually be the skin, it's designed to block rays, it
| might also have a different x-ray opacity, and that can be judged
| from the whole picture in particular where there's several layers
| of melanin, or there's transitions from melanin to very little
| like on hands and feet. Eyelids too, if they're retracted. And at
| the perimeter, the profile, different angle for the ray.
|
| And the intention is for melanin to block x-rays too, block all
| rays, not just UV but deeper. Well it has a spectrum, that cannot
| be denied. And if you're taking all the pixels in an image, there
| might be aggregate effects as I described. You get a few million
| pixels, let AI use every part of the buffalo of the information
| of the picture, and you can get skin color through x-rays.
|
| The question is what this says about Africans with light-skin
| strictly because of albinism, ie lack of pigmentation, but
| otherwise totally African.
| mathieubordere wrote:
| I mean, if color of skin, form of eyes and other visible,
| "mechanical" characteristics can be different it's not that big
| of a leap to observe that certain non-visible characteristics can
| differ too between humans.
| hellohowareu wrote:
| Simply go to google image and search: "skeletal racial
| differences".
|
| subspecies are found across species-- they happen based on
| geographic dispersion and geographic isolation, which humans
| underwent for tens and hundreds of thousands of years.
|
| Welcome to the sciences of anatomy, anthropology, and forensics.
|
| other differences:
|
| - slow twitch vs fast twitch muscle
|
| - teeth shape
|
| - shapes and colors of various parts
|
| - genetic susceptibility to & advantages against specific
| diseases
|
| Just like Darwin's finches of the Gallapogos, humans faced
| geographic dispersion resulting in genetic, diet (e.g. hunter-
| gatherer vs farmer & malnutrition), and geographical (e.g.
| altitude) differences which over the course of millennia affect
| anatomical differences. We can see this effect across all biota:
| bacteria, plants, animals, and yes, humans.
|
| help keep politics out of science.
| airza wrote:
| The article is pretty fascinating and I recommend that you
| actually read it. For example:
|
| >"We found that deep learning models effectively predicted
| patient race even when the bone density information was removed
| for both MXR (AUC value for Black patients: 0*960 [CI
| 0*958-0*963]) and CXP (AUC value for Black patients: 0*945 [CI
| 0*94-0*949]) datasets. The average pixel thresholds for
| different tissues did not produce any usable signal to detect
| race (AUC 0*5). These findings suggest that race information
| was not localised within the brightest pixels within the image
| (eg, in the bone)."
| jcranberry wrote:
| So just from the silhouette of a skeleton, if I understand
| that correctly?
| airza wrote:
| Even after being munged into a nearly-uniform gray by high
| pass the effect seems pretty robust.
| shakna wrote:
| One of the primary ways of identifying possible race from
| bones in anthropology involved calculating ratio from
| lengths. Good for an estimate or fallback, but not completely
| accurate. Removing the density would do absolutely nothing to
| obscure that method. Any image will allow you to measure
| ratio of bones sizes.
| ZeroGravitas wrote:
| The problem with 'race' as a concept isn't that you can
| genetically tell people apart.
|
| Our tools are so precise you can tell which parent a set of
| cousins had with DNA tests, this doesn't make them a different
| species/sub-species or race from each other, even if one group
| has red hair and the other has black.
|
| It's the pointless lumping together of people who are
| genetically distinct and drawing arbitrary, unscientific lines
| that's the issue.
|
| Presumably the same experiments that can detect Asian Vs Black
| Vs White could also detect the entirely made up 'races' of
| Asian orBlack, AsianorWhite and WhiteorBlack since those are
| logically equivalent.
|
| So are the races I made up a moment ago real things? No. But a
| computer can predict which category I'd assign, doesn't that
| make them real and important racial classifications? No it
| means my made up classifications map to other real genetic
| concepts at a lower level, like red hair.
| nerdponx wrote:
| Then problem is that human experts sometimes can't tell the
| difference while the model can.
| inglor_cz wrote:
| AI is also able to determine your sex from your retinal scan
| with very good levels of certainty (provided that your retina
| is healthy; its ability to tell sexes apart drops in diseased
| retinas). [0]
|
| Which came as a surprise to the ophthalmologists, because
| they aren't aware of any significant differences between male
| and female retinas.
|
| [0] https://www.researchgate.net/publication/351558516_Predic
| tin...
| chopin wrote:
| I am surprised that this is a surprise. At least color
| vision is encoded in the X-Chromosome so there should be
| variation as males have only one which can be expressed.
| inglor_cz wrote:
| It is a surprise, because the retina as an organ is very
| well visible and observable in living people, so we have
| a ton of observational data and practical clinical
| experience. But despite that, humans haven't noticed
| anything.
| hugh-avherald wrote:
| I've never seen a retina which had been separated from a
| human.
| KaiserPro wrote:
| > skeletal racial differences
|
| PS10 says that its not that. Anatomy is extraordinarily hard,
| and AI isn't that good, yet. Sure different races have
| different layouts, but often that's only really obvious post
| mortem. (ie when you can yank out the bones and look at them,
| there are of course corner cases where high res CAT/MRI scans
| can pull out decent skeletal imagery in 3D) There are other
| cases, but that should be easy to account for.
|
| If I had to bet, and I knew where the data was coming from, I'd
| say its probably picking up on the style of imaging, rather
| than anything anatomical. Not all x-rays have bones in, and not
| all bones differ reliably to detect race.
|
| > keep politics out of science.
|
| Yes, precisely, which is why the experiment needs to be
| reproduced, and theories tested through experimentation. The
| reason why this is important is because unless we workout where
| this trait is coming from, we cannot be sure the diagnosis is
| correct. For example those with sickle cells have a higher risk
| of bone damage[1] which could indicate they are x-rayed more.
| This could warp the dataset, causing false positives for sickle
| cell style bone damage.
|
| [1]https://www.hopkinsmedicine.org/health/conditions-and-
| diseas...
| knicholes wrote:
| I wonder if some communities use certain x-ray machines
| verses which machines are commonly used by other communities
| and this has nothing to do with race but the machine being
| used. I read over the paper but didn't really understand it.
| Maybe all this is doing is identifying which machine was
| used.
| MontyCarloHall wrote:
| >I'd say its probably picking up on the style of imaging,
| rather than anything anatomical
|
| Certainly possible! They do control for hospital and machine
| ...
|
| >Race prediction performance was also robust across models
| trained on single equipment and single hospital location on
| the chest x-ray and mammogram datasets
|
| ... but it's also possible that different chest x-rays were
| being used for different diagnostic purposes and thus have a
| different imaging style, which a) may correlate with
| ethnicity and b) does not appear to be explicitly controlled
| for.
| ASalazarMX wrote:
| The weak factor in these AI studies seems to be data set
| normalization.
| tedivm wrote:
| >If I had to bet, and I knew where the data was coming from,
| I'd say its probably picking up on the style of imaging,
| rather than anything anatomical. Not all x-rays have bones
| in, and not all bones differ reliably to detect race.
|
| This was my guess as well. I've spent a lot of time around
| radiology and AI (I used to work at a company specializing in
| it) and we read a lot of the failure cases as well. There was
| one example where the model picked up on the hospital, and
| one hospital was for higher risk patients- so it learned to
| assign all patients from that hospital to the disease
| category simply because they were at that hospital.
|
| There are a ton of cases like this out there, especially when
| using public datasets (which in the medical field tend to be
| very unbalanced datasets due to the difficulties of building
| a HIPAA compliant public dataset).
| krona wrote:
| > _one hospital was for higher risk patients- so it learned
| to assign all patients from that hospital to the disease
| category simply because they were at that hospital._
|
| That just sounds like poor feature selection/engineering.
| Garbage in, garbage out.
| tedivm wrote:
| Yeah there are definitely ways they would have avoided
| that, but it's just one example of many. The whole point
| of ML is that it picks up on learned patterns. The
| problem is that it can be difficult to identify what it
| is learning from- this paper itself says they do not know
| what is causing it to make these predictions. As a result
| it is difficult to validate that the model is doing what
| people think it is.
| scandox wrote:
| andrewmcwatters wrote:
| The reality is more humbling: humanity is vast and knowledge
| is not uniformly dispersed.
| rmbyrro wrote:
| Why's that?
|
| Does Google have a filter that leaves all good science out of
| its indexes?
| codefreeordie wrote:
| Yeah, they remove anything they consider "misinformation"
| scandox wrote:
| I object more concretely to the word "simply". Everyone who
| has some sort of agenda and doesn't actually read stuff
| starts with the word "simply". I'm tired of "simply" -
| meaning "listen to me and stop reading the actual content".
| samatman wrote:
| Physiologies are created by genetics, and differences in ancestry
| are the basis for self-identified race.
|
| Ordinary computer vision can also identify race fairly
| accurately, the high pass filter thing is merely pointing out
| that ML classifiers don't work like human retinas.
|
| It's astonishing how many epicycles HN comments are trying to
| introduce into a finding that anyone would have predicted.
| Research which confirms predictable things is valuable of course,
| but no apple carts have been upset.
| omgJustTest wrote:
| Given the complexity of datasets, and what is known about the
| quality of medical scanners, is it possible that underserved
| communities (ie higher noise scanners) serve a specific community
| that is heavily skewed in race distributions?
| cdot2 wrote:
| "our finding that AI can accurately predict self-reported race,
| even from corrupted, cropped, and noised medical images"
|
| It doesn't seem like noise in the images is a factor
| orangepurple wrote:
| Imaging artifacts may persist despite corruption
| bb123 wrote:
| One idea is that there is some difference in the x-rays
| themselves that could potentially be explained by racial
| disparities in access to (and quality of) healthcare. Maybe white
| people tend to visit hospitals with newer, better equipment or
| better trained radiographers and the model is picking up on
| differences in the exposures from that.
| MontyCarloHall wrote:
| They mostly accounted for this:
|
| >Race prediction performance was also robust across models
| trained on single equipment and single hospital location on the
| chest x-ray and mammogram datasets
|
| Sure, it's possible that bias due to the radiographer is the
| culprit, but this seems unlikely.
| Beltiras wrote:
| That's an interesting confounding variable. I think it's
| disproven by the fact that the AUC is too high given your
| hypothesis.
| redox99 wrote:
| These results seem too accurate to be explained only by a
| correlation to the medical equipment used.
| krona wrote:
| > _We also showed that the ability of deep models to predict
| race was generalised across different clinical environments,
| medical imaging modalities, and patient populations, suggesting
| that these models do not rely on local idiosyncratic
| differences in how imaging studies are conducted for patients
| with different racial identities._
| MontyCarloHall wrote:
| Not too surprising that physical differences across ethnicities
| are literally more than skin deep. It wouldn't be shocking that a
| model could identify one's ethnicity based on, for example, a
| microscope image of their hair; why should bone be any different?
|
| I'm more surprised that the distinguishing features haven't been
| obvious to trained radiographers for decades. It would be cool to
| see a followup to this paper that identifies salient
| distinguishing features. Perhaps a GAN-like model could work--
| given the trained classifier network, train 1) a second network
| to generate images that when fed to the classifier, maximize the
| classification for a given ethnicity, and 2) a third network to
| discriminate real from fake X-Ray images (to avoid generating
| noise that happens to minimize the classifier's loss function). I
| wonder if the generator would yield images with exaggerated
| features specific to a given ethnicity, or whether it would yield
| realistic but uninterpretable images.
| hemreldop wrote:
| eklitzke wrote:
| I think it's more likely the case that (a) most radiographers
| aren't trained in medical school to look for distinguishing
| racial features (why would they be?) and (b) in most cases the
| radiologist knows or can easily guess the race of the patient
| anyway so there's no need to try to guess it from X-ray imaging
| data. There are a lot of anatomical features related to race
| that have been known since before radiology has been a field,
| it's just not pertinent to the job of most radiologists.
| tomp wrote:
| If you're interested in "hard to describe features that can be
| learned with enough expiration", look up _chick sexing_
|
| https://en.wikipedia.org/wiki/Chick_sexing#Vent_sexing
| Beltiras wrote:
| Interesting field. You have to breed a couple of males to
| maintain the species. If you were to pick those from the mis-
| sexed group I suppose natural selection would reduce the
| classifying feature over time. I wonder if poultry farms pick a
| couple of the male-classified birds to maintain a stock of well
| identifiable males and kill all the mis-classified males.
| dang wrote:
| The submitted title ("AI identifies race from xray, researchers
| don't know how") broke the site guidelines by editorializing.
| Submitters: please don't do that - it eventually causes your
| account to lose submission privileges.
|
| From the guidelines
| (https://news.ycombinator.com/newsguidelines.html):
|
| " _Please use the original title, unless it is misleading or
| linkbait; don 't editorialize._"
| gus_massa wrote:
| It's the title of the Vice article about the same topic.
| https://www.vice.com/en/article/wx5ypb/ai-can-guess-your-rac...
| (It was posted last year.) (No idea why the OP used one title
| and another URL.) (The title of Vice is a bad title anyway.)
| dang wrote:
| Good catch!
| civilized wrote:
| It would be nice to see more genuine, enthusiastic scientific
| curiosity to understand how the ML algorithms are doing this,
| rather than just abject terror and alarm.
| SpicyLemonZest wrote:
| It seems like the reason the researchers in this paper are
| concerned is precisely that they tried and failed to understand
| how the ML algorithms are doing this. If they'd discovered that
| white people have a subtly distinctive vertebra shape the model
| was detecting, it would have been much more of "oh, we
| discovered a neat fact".
| civilized wrote:
| I don't think they tried very hard at all. I see no
| meaningful use of modern explanation tools.
|
| There are lots of known ways in which people of different
| races are different physiologically. Probably even more
| unknown ways.
|
| There could also be differences in imaging technology used in
| different communities, as others have suggested. I'd be a bit
| surprised if something like that could create such a strong
| signal but it's on the table.
| SpicyLemonZest wrote:
| For those of us less familiar with this space, what are
| these modern explanation tools? (I certainly agree that
| it's plausible the model is seeing a physiological
| difference, and the researchers seem to have considered a
| few concrete hypotheses on that dimension.)
| civilized wrote:
| Here's an introduction to one technique:
| https://cloud.google.com/blog/products/ai-machine-
| learning/e...
|
| This is a cutting edge subfield of ML, so it's
| understandable that one paper in a medical journal isn't
| going to be on that cutting edge, but I think they should
| at least acknowledge that their investigations barely
| scratched the surface.
| SpicyLemonZest wrote:
| Thanks! This looks pretty neat, I'll have to dig more
| into their explainable AI product later.
___________________________________________________________________
(page generated 2022-05-17 23:02 UTC)