[HN Gopher] Resident physicians' exam scores tied to patient sur...
___________________________________________________________________
Resident physicians' exam scores tied to patient survival
Author : Bostonian
Score : 117 points
Date : 2025-02-25 16:18 UTC (4 days ago)
(HTM) web link (hms.harvard.edu)
(TXT) w3m dump (hms.harvard.edu)
| Bostonian wrote:
| 'Board exam performance was powerfully linked to patient risk of
| dying or hospital readmission. For example, there was an 8
| percent reduction in the odds of dying within seven days of
| hospitalization in patients of physicians who scored in the top
| 25 percent on the exam, compared to the patients of physicians
| who scored in the bottom 25 percent on the exam, which was still
| a passing grade.'
| goodluckchuck wrote:
| Controlled hospital quality? I figure the best credentialed
| doctors go to the best hospitals, where patients receive a lot
| of other care aside from the MD.
| sebmellen wrote:
| This is the most important question in the thread.
| pcthrowaway wrote:
| Did they also control across types of medicine? If the
| higher-scoring doctors go into types of care which are more
| competitive, could those practices have lower patient
| mortality within 7 days?
|
| For example, maybe burn unit care is high-mortality and
| low-barrier, compared to sleep medicine which is low-
| mortality and high-barrier (I don't know how accurate this
| is, just providing some hypotheticals for clarity)
| helsinkiandrew wrote:
| From the article:
|
| >The researchers compared outcomes for patients within the
| same hospitals who were cared for by doctors with different
| exam scores. This allowed the researchers to eliminate, or at
| least minimize, the effect of differences in patient
| populations, hospital resources, and other variations that
| might influence the odds of patient death or readmission,
| independent of a doctor's performance.
| ETH_start wrote:
| Reducing standards to meet DEI requirements is therefore a killer
| practice:
|
| https://www.aei.org/carpe-diem/new-chart-illustrates-graphic...
| colechristensen wrote:
| I'm all for diversity but that admissions gap is just racism.
|
| You can't have separate entrances for your establishment based
| on what folks look like, the group you prefer getting better
| service doesn't make it equality.
| sagarm wrote:
| I know right? Black applicants with high MCAT scores were
| rejected in favor of white applicants with low MCAT scores!
| Just unbelievable.
| NotYourLawyer wrote:
| Unbelievable because it's the opposite of what the link
| shows?
| sagarm wrote:
| 96% acceptance rate for black candidates with high MCAT
| scores, but a nonzero acceptance rate for white
| candidates with low scores.
|
| Maybe there are other factors, and they're correlated
| with the buckets being used here?
| NotYourLawyer wrote:
| It's not the case that every single black applicant gets
| admitted before a single white/Asian applicant does. The
| point is that it's much, much easier for a black
| applicant to get admitted.
|
| A black applicant with GPA and MCAT scores in the lowest
| bucket still has a 56% chance of admission. That's on par
| with an Asian applicant who has GPA and MCAT scores in
| the _highest_ bucket.
| sagarm wrote:
| So do you think that if the acceptance rate for high MCAT and
| GPA are below 100%, then the other bars should be zero? i.e,
| these are the only admissions criteria that should be
| considered?
| zameerb1 wrote:
| its easy to hide data behind percentages and say 94% of the
| blacks who had a certain GPA where admitted. look at the raw
| numbers, study after study have shown improved care for colored
| patients and outcome better when treated by black physicians
| which indicates we have to have proportional numbers of black
| and hispanic physicians representative of their population. If
| whites and asians disproportionately apply to medical schools
| their admission rates are going to look different. The systemic
| advantage afforded to affluent kids by being brought up for 18+
| years by highly educated parents is not level playing field.
| evulhotdog wrote:
| This is a study my wife wrote regarding this exact scenario,
| trying to see if patients think they're getting better care
| if they're similar to the doctor (and team) treating them!
| evulhotdog wrote:
| Forgot the link: https://pubmed.ncbi.nlm.nih.gov/37801560/
| NoImmatureAdHom wrote:
| This study has nothing to do with the claim being made by
| the grandparent comment.
| NoImmatureAdHom wrote:
| "study after study have shown improved care for colored
| patients and outcome better when treated by black physicians"
|
| This is false. You're probably getting this idea second hand
| from this study:
| https://www.pnas.org/doi/abs/10.1073/pnas.1913405117
|
| Probably because it was famously misused by SC Justice
| Ketanji Brown Jackson, who got it wildly wrong https://statmo
| deling.stat.columbia.edu/2024/06/14/statistics...
|
| Anyway, that study is bogus:
| https://www.pnas.org/doi/abs/10.1073/pnas.2415159121
|
| The only evidence _for_ your claim that I know of is an NBER
| paper https://www.nber.org/bah/2018no4/does-doctor-race-
| affect-hea...
|
| Where they randomly assign black male patients to white or
| black doctors, IIRC, and patients get advice on preventative
| care. Outcomes for black patients are better because they are
| more willing to take black doctors' advice. Obviously,
| newborns in the first study, so it's about doctor competence
| straight-up.
| derbOac wrote:
| MCAT != board exam, for one thing.
|
| There have been studies suggesting that elimination of the MCAT
| does little to nothing to prediction of student performance
| beyond the second year or so.
| ETH_start wrote:
| I would be willing to place money on there being a very high
| correlation between MCAT results and board exam results.
| derbOac wrote:
| My prediction is the correlation is about 0.30-0.40.
|
| As others have pointed out, there are a lot of unmeasured
| variables not being controlled for in this finding as well.
|
| I'm not surprised board exam scores predict outcomes, I
| just think there's lots of other variables along that path
| from one to the other, and even more from MCAT -> board
| exam.
| _aavaa_ wrote:
| I'm a skeptical of the interpretations. All we have are
| percentages without knowing the size of each group.
|
| Among other things, it reeks of of Simpson's Paradox.
|
| https://en.m.wikipedia.org/wiki/Simpson's_paradox
| ETH_start wrote:
| What relevance would the group size have to any of this, and
| how would this possibly be a result of the Simpson's Paradox?
| _aavaa_ wrote:
| > black applicants were more than 9 times more likely to be
| admitted to medical school than Asians (56.4% vs. 5.9%),
| and more than 7 times more likely than whites (56.4% vs.
| 8.0%)
|
| If the number of Asian applications is 10x the number of
| spots available, their admittance rate can never be higher
| than 10%. No "discrimination" required. Same for white
| applicants.
|
| If you only have 10 black applicants and you accept 5 of
| them that's a 50% admittance rate. Which looks huge and you
| can scaremonger about how much bite and Asian people are
| unfairly getting sidelined.
|
| Until you see there were 10,000 white applicants with a 8%
| admittance rate, ie 800 people.
|
| 800 from 8% vs 5 from 50%.
|
| Again without absolute numbers the percentages can be very
| deceiving.
| crooked-v wrote:
| With the absolute absurdity the residency process, and the focus
| entirely on new doctors just after that residency, I have to
| wonder how much of this just corresponds to whoever's lucky
| enough to be the kind of high-powered mutant who can survive
| multiple years of 80- to 100-hour week schedules designed by a
| man who was high on cocaine and morphine 24/7 (seriously, look it
| up, it's true). There are going to be a lot of people who need an
| extended sabattical to recover from that before they'll be
| effective at anything at all, which makes any kind of baseline of
| test scores really suspect to me.
| derbOac wrote:
| Yeah I wondered how much of this is accounted for by some
| general resiliency thing or circumstances during residency or
| something along those lines.
| SpicyLemonZest wrote:
| Does the difference matter in this context, though? Medicine
| isn't like other professions where it's no big deal to have
| some fraction of the workforce be bad at their jobs. I'm not
| so status-quo-biased that I'd _support_ 100 hour residencies,
| but I 'm skeptical of reform proposals that focus on doctors'
| working conditions rather than patient outcomes. If some
| filtering process leads to better patient outcomes, I think
| we should retain it, even if it's quite stressful for the
| doctors who have to go through it.
| the_d3f4ult wrote:
| Fair point. There's some data showing patient outcomes are
| worse when managed by overworked residents-in-training, but
| I think you're referring to outcomes post-residency. i.e.
| Physicians should squeeze as much training as possible into
| the allotted years. This is reasonable, especially for
| surgical specialties where procedural reps are a commodity
| for trainees.
|
| I'd be more open to this line of reasoning if physician's
| salaries had kept pace with inflation over the last 30
| years and if if we hadn't tacitly accepted a much, much
| lower standard of training in the form of DNPs, CRNAs and
| PAs who are now practicing independently in a lot of
| regions. You can't demand that people make extraordinary
| sacrifices without extraordinary compensation.
|
| For contrast, most European countries have a much longer
| post-residency training process that is more humane. Caveat
| being that students enter medical school directly from high
| school and don't have student loans.
|
| It's also worth pointing out that in the US a LOT of those
| 100 hours are not spent in direct patient care. They're
| spent doing chores ('scut') that are not directly tied to
| patient care. Think: Calling insurance companies for prior
| authorization for your supervisor or filling out FMLA
| paperwork for one of your supervisors' patients. As a
| resident you don't have the ability to say "no" to these
| tasks.
| pcthrowaway wrote:
| Your comment sounds reasonable, but it doesn't allow for
| nuance.
|
| If a hellish residency improves patient outcomes by 0.1%,
| at the expense of every single resident suffering twice as
| much as they need to (and likely leading to some stimulant
| addictions and deaths among the resident/doctor
| population), that's not a fair tradeoff.
|
| Medical workers don't exist solely to sacrifice themselves
| for others; they are humans also and their needs should be
| weighed as important like everyone else's.
|
| As it so happens I think some of the strain of medical
| residency is related to supply shortages in the health care
| industry. If it's not _crystal clear_ that working 80+
| hours per week is necessary to significantly improve
| patient outcomes, and it is clear that working 80+ hours
| per week makes a lot of people choose other careers
| (limiting supply artificially), then reform here is
| imperative.
| ioblomov wrote:
| Am I missing something here? How could a hellish
| residency--with all the stress and sleep deprivation that
| implies--possibly _improve_ patient outcome?
|
| Apart from the bad real-time cognitive effects, long-term
| memory retention is dependent on regular, sustained
| sleep.
| polio wrote:
| The idea is that the stress and sleep deprivation are not
| sources of permanent impairment (even though they are),
| but rather a filter that selects the strongest
| candidates.
| pcthrowaway wrote:
| Oh I'm not saying it does, the person above seemed to be
| suggesting that we should focus on figuring out the
| residency conditions that lead to the best patient
| outcomes, rather than improve the conditions for
| residents, which suggests they believe worse conditions
| for residents may be better for patients.
|
| Just to point out the obvious, people doing 80 hrs/week
| for 2 years (lower end of residency term I believe) are
| going to have twice as much 'experience' as people doing
| 40hrs/week for 2 years.
|
| I suspect most of us here know more hours worked doesn't
| directly correlate with more retention of information and
| best practices, but that's the thinking.
|
| I'm arguing that even _if_ 80hrs /week residencies _was_
| the optimal amount of pressure to turn our fledgling
| residents into battle-hardened physicians, if you can get
| 99% of the effect with 40hrs /week, maybe do that
| instead. And again, I'm not even suggesting this is
| actually the case.
| thijson wrote:
| One of the guys that founded the modern medical education
| system was a coke head:
|
| https://magazine.columbia.edu/article/cocaine-addict-who-
| cha...
|
| The Mayans used coca leaves to get more work out of their
| people.
|
| I guess medical residency is kind of like a hazing
| ritual. Today's doctor's are like I went though it, why
| can't you?
| derbOac wrote:
| I don't necessarily think the relationship is "worse
| residency conditions predicts higher board exam scores"? It
| could be that residents with more time to study or whatever
| score higher. It could be examinees with scores close to
| the threshold are accounting for the association. Or maybe
| it is resiliency. I have no idea.
|
| My general impression is that the evidence overall is
| really not supportive of harsher residencies in terms of
| patient outcomes. I also think that rigor does not have to
| mean masochism or hubris; there seems to be this assumption
| that any change to residencies would mean dumbing it down
| or making it easier, as opposed to improving things
| overall. I'm also a little skeptical of minor tweaks to
| residency that might have happened somewhere now being
| representative of a more wholesale restructuring.
|
| The often unacknowledged factor in the background is that
| hospitals and residency locations are getting free labor
| with no chance of repudiation of their situation by
| workers. Hospitals are getting physicians whose salaries
| are paid for by the federal government, where those
| physicians are essentially unfree to move if they're
| unhappy. So of course there's going to be an attempt to
| milk them for everything. It gets whitewashed as
| "selflessness" and physicians are encouraged to boast about
| it or something, instead of calling it out as exploitation.
| No physician wants to make that claim, for a whole host of
| reasons, even if it is true.
|
| Imagine what would happen if hospitals had to bear the
| costs of residency training completely, like just about any
| other healthcare profession, and residents were able to
| move freely like most employees.
|
| I get despondent about so much in US healthcare. There's so
| much focus on invoice costs per se, and payment by
| insurers, and not enough on monopolies in service delivery,
| and problems with educational structures. Any attempt to
| address these issues is met with resistance by various
| groups with conflicts of interest, who aren't called out on
| these conflicts of interest.
|
| Another thing about residencies constantly on my mind from
| other settings (institutional tracking hours in the moment
| versus recalled hours later) as well as personal
| experiences with residency in the past is that people are
| notoriously bad about reporting past work hours and
| conditions, and tend to exaggerate. I'm not saying that
| anyone in particular is necessarily being dishonest in
| describing their residency experience, but I suspect there
| has been drift over time in conditions that reflects a kind
| of biased memory of things on the part of residency
| directors. "I worked 120 hours a week" when that wasn't
| actually the case, or is distorted, then becomes residency
| policy for the next generation.
|
| Sometimes I feel like the logical conclusion, given the way
| these discussions go sometimes, is the only one being
| legally able to practice is someone with an MD who has
| completed a residency working 140 hours a week for 6 years,
| with perfect board exam scores. It just doesn't add up.
| maxerickson wrote:
| Well how much of it is just initiation rituals and
| accidents of history? How fast do effective new practices
| propagate throughout the industry?
| lurk2 wrote:
| Please tell me more about the man who was high on cocaine and
| morphine 24/7.
| femto wrote:
| Probably referring to this?
|
| https://pmc.ncbi.nlm.nih.gov/articles/PMC7828946/
| dawatchusay wrote:
| This doesn't show that he was "high on cocaine and morphine
| 24/7" as the relevant commenter suggested; just that he
| struggled with addiction
| ToValueFunfetti wrote:
| It does say
|
| > he was able to hide his addiction under a veil of
| eccentricity and a pyramid of residents
|
| Which means "created an environment to allow himself to
| be high at work" to me. It's not impossible that he held
| it off at home, but I don't see why he would.
|
| Also, he's clearly Dr. House; Ctrl-F "Leaving much"
| PaulKeeble wrote:
| Medicine is now absurdly complex, it's far more than a person can
| possibly learn especially if trying to be up to date with modern
| research. The more you can memorise correctly and pattern match
| the better. Many patients are failed in the current system, most
| not fatally but their lives are damaged and it's not uncommon for
| more complex diseases to have 90% of sufferers never getting a
| diagnosis until they die from the disease.
|
| Something has to change drastically in how medicine is organised
| because it's not working in its current iteration as the
| difficulty goes up and up.
| smgit wrote:
| Its the environment that compounds the complexity. Go down the
| list of Largest companies by revenue in the US and 8 in top 20
| are related to "health" - are they running hospitals? are they
| pharma companies? No.
|
| They run pharmacy benefits management, health insurance and
| drug distribution.
|
| The estimate is 4-5 Trillion flows throw these firms. Which is
| larger than the GDP of India. So this gigantic structure has
| emerged that doesn't really make too much profit btw (very
| similar to Amazon Platform Economics) but is layer upon layer
| upon layer of cash flow passing through middlemen.
|
| Drastic change requires new ideas about what do we do about all
| these middlemen who shape the environment on top of which
| everything exists.
| ethbr1 wrote:
| The biggest problem with the US health system? Complexity.
|
| It's impossible to fix overly complex systems.
|
| Simplify, simplify, simplify, and then the fixes become
| trivial.
|
| In the US case, that means banning most of the middle-layers.
| zdragnar wrote:
| Alas, independent middle layers have long been the US
| solution to avoiding monopolies. This is the whole reason
| car manufacturers can't sell directly to consumers, and
| micro breweries can't sell to consumers except for on-site
| purchases. Breweries in particular have to sell to
| distributors, who sell to stores.
|
| Banning the middle layers here (absent other changes) just
| means that the companies that replace their spots in the
| top 20 will be vertically integrated conglomerates that
| manufacturer, distribute, prescribe and provide insurance
| (i.e. payment plans) for pharmaceutical drugs.
| vitus wrote:
| > Banning the middle layers here (absent other changes)
| just means that the companies that replace their spots in
| the top 20 will be vertically integrated conglomerates
| that manufacturer, distribute, prescribe and provide
| insurance (i.e. payment plans) for pharmaceutical drugs.
|
| Except these companies are already vertically integrated,
| to a large degree. All the biggest insurers have their
| own in-house PBMs.
|
| CVS (the parent company of Aetna) has Caremark.
|
| Cigna has Express Scripts.
|
| Anthem (fine, Elevance) has CarelonRx.
|
| UnitedHealth Group has Optum.
| analog31 wrote:
| Indeed, it may be the case that the middlemen aren't
| individually all that profitable, but if the money passes
| through several stages and each one skims off a few percent,
| you end up with the present situation where health care costs
| twice as much as it does in any civilized country.
| vineyardmike wrote:
| Revenue is an incomplete signal of the complexity and waste.
| It's just a signal of the money flowing through. A "single
| payer" system would probably also show a huge revenue number
| even if the profit was <=0. There's just a lot of money and a
| lot of people who are patients.
|
| I don't disagree that the system requires change and is
| extremely complex, however.
|
| The real problem is that it's nearly impossible to "scale"
| healthcare and keep it personalized, and people want
| personalized healthcare - because that's shown to be more
| effective healthcare. Doctors can only see a limited number
| of patients a day, and they need to be paid some compensation
| commensurate with their skills and efforts. That alone makes
| it hard for everyone "healthy" to see a doctor often enough
| and for long enough to get deeply personal care. Most people
| realistically can pay out of pocket for preventative care.
| $100-200/yr for an American isn't crazy. Even _most_ drugs
| are super affordable out of pocket if the profit margins are
| kept low (which is started to be available, bits at a time).
|
| The real complexity, of course, is the long-tail where a few
| people get cancer and car accidents and other serious
| conditions which swamp the costs of everything else.
| RadiozRadioz wrote:
| Based on hours of past experience, the solution to this
| particular problem seems to be to give all the doctors a cane
| and a bottle of Vicodin.
| rscho wrote:
| Things are changing to accomodate the increasing complexity,
| same way as ever: specialization. There are now
| subsubspecialties, and 'cardiologist' or 'nephrologist' have
| become incomplete qualifiers. It may not look like that from
| the pov of outsiders, but medicine is becoming more and more
| secure by the day. Things were much worse before.
| mouse_ wrote:
| if something's wrong, I just don't go to the doctor
| anymore...
| rscho wrote:
| Since you're still alive, it seems the wrong things aren't
| severe enough to kill you. Good for you!
| choilive wrote:
| But now you have the problem of being too specialized - Ive
| seen many specialists that think a problem lies within their
| specialty- when it does not. And how do you deal with
| problems that are multi-disciplinary (problems that require
| multiple organ systems) when you have an army of specialists
| that are each fighting for their own fiefdoms? When all you
| have is a hammer everything looks like a nail. Comes to mind.
| rscho wrote:
| Well, the model is migrating to one of hyperspecialists
| collaborating together. Problem is, this isn't compatible
| with private practice where you're operating mostly alone
| and results in what you describe. The model has to evolve,
| yes. Good news is, it is in fact evolving (slowly). We
| can't evolve faster than science anyway, and while medical
| science is evolving much faster than it used to, we're far
| from the exponential acceleration we've seen in other
| domains, e.g. computers.
| jmcgough wrote:
| Edit: Well, that's embarrassing. I hadn't realized that the link
| is to a new 2024 study on IM board scores and patient outcomes.
| My post is in regards to a 2023 study on USMLE scores and patient
| outcomes that was pretty widely discussed.
|
| Healthcare worker here. Sheriffofsodium did a great video poking
| holes at this study:
| https://youtu.be/JKS9Y-nCnKs?si=VPsUNSoepltbg4Hu
|
| It's 45 minutes so I don't expect people to watch it, but he
| makes several important points, including:
|
| - This study was performed by USMLE insiders, the only ones with
| access to this private data. USMLE does not share this data
| publicly so it's impossible to verify.
|
| - As the USMLE makes millions of dollars from these exams, they
| have a clear conflict of interest.
|
| - The differences in patient outcome are AT BEST of marginal
| clinical significance, which the authors of the study even state
| in the paper.
|
| There is better scientific evidence that female surgeons have
| better patient outcomes on average:
| https://pubmed.ncbi.nlm.nih.gov/37647075/
| zameerb1 wrote:
| the OP is referring to a different study about Board not USMLE
| eigenblake wrote:
| Doctors aren't machines, they're humans. I have not yet read the
| full paper, only the article, but I already see something really
| big and important to look out for. When I read the full thing,
| the question I'll be asking is "what's the likelihood that the
| self-esteem of doctors was directly intervened on by the exam
| taking process itself." How do you control for the loss in
| confidence that learning of your test performance gives you? How
| are we certain that learning your score on the board exam doesn't
| make you more conservative (or riskier) with how you treat
| patients as a psychological effect?
| the_real_cher wrote:
| loss of confidence? lol what?
| AStonesThrow wrote:
| Yeah but the patient is just a biological machine. This machine
| can easily be divided into organs and apportioned among
| specialists. The machine is easily understood by a corpus of
| research and laboratory experimentation.
|
| . Many inputs can be placed in the machine by physicians, and
| the outputs are known. The biological machines can easily be
| isolated from environment, or monitored with high technology,
| and assigned numbers in databases to be processed in data
| centers.
|
| Value is extracted from the biological machines mostly from
| government and 3rd party sources, so there is no real need to
| rely on machines having a means or will of their own.
|
| There is no compelling reason to treat humans any different
| from automobiles for the purposes of medicine and medical
| treatment. In fact humans are less genetically diverse than
| motor vehicles, and A new model year will always produce a
| bumper crop of lemons to work on.
| rscho wrote:
| The common misconception of someone with a hard science
| education.
|
| > Many inputs can be placed in the machine by physicians, and
| the outputs are known. The biological machines can easily be
| isolated from environment, or monitored with high technology,
| and assigned numbers in databases to be processed in data
| centers.
|
| We aren't even close to that level of understanding.
| zemvpferreira wrote:
| And still, the model works. Lives are saved. We might save
| many more with a fully integrated non-simplified approach,
| but it's not necessary to keep seeing growth in positive
| outcomes.
| NoImmatureAdHom wrote:
| Soon they will be!
| eigenblake wrote:
| This appears to be an observational result, so I'm genuinely
| perplexed by the reception here. I genuinely thought this
| comment shows a healthy amount of curiosity and asking
| important questions. Asking "what control group did this study
| use?" is usually well-received here.
| iancmceachern wrote:
| I worked for the "Father of Robotic Surgury" and once in a
| company-wide meting he said "The general public would be pretty
| happy with the average surgeons results, but they would be
| horrified by the below average surgeons results". Their goal was
| to bridge that gap with robotics.
| AStonesThrow wrote:
| Like bridging it in this sense?
|
| https://en.wikipedia.org/wiki/Brooklyn_Bridge#Culture
| Calavar wrote:
| I'm not a surgeon myself, but when I was in medical school the
| program director of our local general surgery residency told me
| that in terms of hand skills 90% of surgeons are more or less
| average, 10% are masters, and 10% are horrific. (So basically a
| bell curve with very thin tails.) He also said the correlation
| between test scores and surgical hand skills was pretty week.
| MichaelZuo wrote:
| That adds up to 110%...
| kinggrowler wrote:
| He's a surgeon, not a mathematician.
| throwaway127482 wrote:
| Lol literal first four words of the comment said he's not
| a surgeon
| SlightlyLeftPad wrote:
| He's a brick layer, not a linguist.
| rscho wrote:
| Which must make it the most MD comment I've seen in a long
| time. It's hilarious!
| solardev wrote:
| That's just the normal level of performance people expect
| of their doctors, 24 hours a day, 8 days a week.
| 3eb7988a1663 wrote:
| How much of surgery is based on dexterity vs
| knowledge/attention-to-detail? I sort of assumed that most
| operations are basic plumbing (A connects to B) while there
| are a few specialized domains that require exquisite
| deftness.
| rscho wrote:
| The question is too general. Depends a lot on the kind of
| surgery you're doing. I guess the answer you're looking for
| is that anyone could be a surgeon, but not for all kinds of
| surgery. Also 'basic plumbing' with no room for error is
| not an easy thing at all.
| Tade0 wrote:
| I'm just relaying what my friend who is studying to become
| a doctor told me, but by his account there's a wealth of
| techniques for each procedure or even parts of it, like
| tying up the dangling bits after kidney removal.
|
| Ultimately it boils down to what a given surgeon practiced
| in their career.
| iancmceachern wrote:
| Most complex or endoscopic surgeries are (dependent on
| dexterity)
| iancmceachern wrote:
| It's more in that if you improperly connect a to b someone
| dies, and it's not just a snap fit like a pipe. You need to
| do things like suture two blood vessels together, using
| pliers (not the technical term), inside a dark box lit up
| by a tube, while looking at an upside down TV image of
| what's going on.
| devilbunny wrote:
| It doesn't have to be upside down; you can just rotate
| the camera to any angle you like.
| thaumasiotes wrote:
| > (So basically a bell curve with very thin tails.)
|
| Is this a concept that exists? I thought "thin tailed" and
| "fat tailed" were defined by contrast to a normal
| distribution.
| Ma8ee wrote:
| I don't think that "Bell curve" should be interpreted
| strictly as a Gaussian in this context, but more literally
| as a curve with a shape resembling a bell.
| rscho wrote:
| From a doc, it was just a figure of speech. Most docs have
| no idea what a statistical distribution is.
| Unlisted6446 wrote:
| I'm pretty sure, yes? Cauchy distribution and student-t
| have fatter tails than a standard normal distribution.
| hibikir wrote:
| It's not just that there are low performers, but that systems
| aren't always built to send them to do something else quickly.
|
| When my mom need her second urgent brain surgery, one of the
| nurses, a friend of a friend, warned us that one of the
| surgeons on staff that night had dreadful results with
| basically any procedure. and that if we were stuck with him,
| that refusing the surgery until another one was able to come
| in. If the floor nurses know, the head on neurology knows, and
| yet, they were still OK letting them operate.
|
| Any functioning system just has to eject surgeons that get bad
| outcomes. It's not as if they have to stop practicing medicine,
| but move to something a little less dangerous.
| Calavar wrote:
| I agree with the sentiment. The specifics of the
| implementation are hard.
|
| When it comes to bad doctors, I think a lot of people in the
| medical field would agree "you know one when you see one."
| But when you're talking about putting professional
| restrictions on someone, you need objective criteria.
|
| The naive way, counting up bad outcomes, leads to a system
| where surgeons are incentivized to decline any case that
| looks technically difficult or where the patient has lots of
| preexisting conditions that put them at risk for
| complications after the surgery. We already see this to a
| degree in transplant surgery, where outcomes are followed
| closely to avoid wasting organs.
|
| That said, I think true incompetence is pretty rare. I can't
| think of a single doctor I 've worked with professionally
| where I'd be concerned if I found out they were taking care
| of one of my immediate family.
| dartharva wrote:
| Is this surprising?
|
| High exam scores are an indication of discipline and good
| prioritisation - factors that evidently reflect on the
| physician's professional performance.
| marcosdumay wrote:
| It's evidence that those exams are doing something right.
|
| Whether it's surprising or not, it's up to you. But it's
| something that should be measured once in a while.
| zdragnar wrote:
| We (in the US) have spent years deprioritizing standardized
| testing for college admissions on the (public) justification
| that they don't reflect potential for success.
|
| Likewise, there's been an element of testing = racial
| discrimination in all sorts of fields, such as:
|
| https://fairtest.org/article/legal-attack-biased-firefighter...
|
| And:
|
| https://www.theguardian.com/world/2009/jun/29/connecticut-fi...
|
| The fact that this study alleges a direct link between exam
| scores and performance is itself bucking the zeitgeist.
| NoImmatureAdHom wrote:
| It isn't exactly news that doctors with better test scores are
| better doctors, but this is additional evidence. The article
| doesn't touch on race, but very deliberately. To anyone on the
| inside, the silence is deafening.
|
| In the U.S. med schools been matriculating many unqualified
| "underrepresented minority" (black, hispanic, native American,
| Hawaiian) medical students for a long time. This is unfair to
| patients and doctors, especially competent brown doctors, because
| it is now the case that you get a very strong signal about how
| how good a doctor is simply by the color of his or her skin.
| Which is messed up.
|
| AAMC has the data (https://www.aamc.org/data-reports/students-
| residents/data/fa... , table A-18). This is _after_ the 2023
| Supreme Court decision, so the spreads are a little wider in e.g.
| 2022 data. MCAT scores range from a minimum of 472 to a max of
| 528, which is stupid and a deliberate tactic to make the
| differences between groups seem small. Subtracting 472 from each
| average score, 2024 average MCAT scores look like this for
| matriculants:
|
| 41.9: Asian
|
| 40.2: White
|
| 36.9: Hawaiian
|
| 34.4: Black
|
| 33.9: Hispanic
|
| 31.3: American Indian
|
| These are _very large differences_ which you can absolutely
| expect to show up in doctor performance. Everyone has to pass the
| same boards during / after med school, but that's just going to
| cut out some of the worst. Among those who pass, the unqualified
| minority students who were admitted to med school because of
| their skin color will still be concentrated at the bottom of the
| distribution.
|
| Do you know what they call the guy who finished last in his med
| school class? "Doctor".
| dqv wrote:
| > The article doesn't touch on race, but very deliberately. To
| anyone on the inside, the silence is deafening.
|
| ??? The NPI registry doesn't indicate the race of registered
| providers, only their sex. Really bizarre to call a limitation
| of the available data "deliberate".
| NoImmatureAdHom wrote:
| It's possible to put together multiple data sources. There
| are certain things everyone reading this will already know.
| It's like reporting "educational attainment" rather than _g_
| or IQ in studies...everyone knows what it implies, you just
| can 't say it. Anyway:
|
| 1) Board scores are strongly linked to patient outcomes (this
| paper)
|
| 2) We already know test scores vary strongly with observable
| characteristics like race
|
| 3) It's a very safe bet that board scores vary with race in
| the same way that MCAT scores vary with race
|
| Therefore,
|
| 4) We can have a very good idea of how good a doctor is based
| on observable characteristics like race
|
| Which is a thing the article immediately, obviously, and
| loudly implies but of course couldn't say for fear of
| censorship, losing jobs, etc.
| Der_Einzige wrote:
| Was wondering how long I'd have to scroll for this. The reality
| is that it's unhealthy not to be "racist" when selecting health
| care providers right now due to historical policies like this.
|
| When the right takes swipes at "DEI", going after bar lowering
| in medical school is very high on the list of legitimate
| targets for them to attack. I don't want to care about the race
| of my doctor, but do gooders gave me no choice by passing so
| many bad doctors.
| alistairSH wrote:
| Did they pass bad doctors? The first post referenced entrance
| exams but cited no data about those that actually complete
| their medical training.
| NoImmatureAdHom wrote:
| You could probably back out at least some bounds from the
| data here: https://www.aamc.org/data-reports/students-
| residents/report/...
|
| This: https://www.sciencedirect.com/science/article/abs/pii
| /S00904... Suggests _MEDIAN_ USMLE step 1 scores for White,
| Asian, Hispanic /Latino, and Black applicants were 242,
| 242, 237, and 232. It's urology specific, and practice
| specific, though.
|
| This:
| https://onlinelibrary.wiley.com/doi/full/10.1002/hsr2.161
| Says The mean (+-SD) USMLE step 1 score was significantly
| greater among non-[Black or Hispanic] applicants as
| compared to URiM applicants (223.7 +- 19.4 vs 216.1 +-
| 18.4, P < .01, two-sample t-test). This is at a specific
| medical school.
|
| But more generally...imagine what would have to be true for
| us to go from BIG differences in g-loaded test performance
| to small / no differences. Either people fundamentally
| change somehow (get smarter / dumber), people's test scores
| systematically differ because they e.g. got better / worse
| at "tests" or something, independent of their underlying
| knowledge of the content or abilities, or it's attrition
| (e.g., very many minority med students wash out, leaving
| only those who should have been admitted in the first
| place).
|
| None of those things seem plausible to me. The little
| glimpse we have from the two studies above is consistent
| with the obvious thing happening. Things are mostly the
| same, though I'd bet URM have higher wash-out rates, so
| differences get attenuated somewhat by the time they're
| practicing. Of course, URM vs non-URM will sort differently
| into specialties and geographies so there's that...you'll
| see bigger or smaller differences depending on how they
| sorted. A good question, as well, is why the USMLE people
| don't split reporting by race. I bet one of the reasons is
| they'd get a lot of flak because there would be big
| disparities. And good on them (maybe!) because one reason
| they might care about that is they want to produce good
| doctors, and watering down their test won't help with that.
| jedberg wrote:
| This is the data on entrance exams, not exit exams. Is there
| any data that actually shows minorities that _finish_ med
| school and pass boards are any less competent?
|
| The entire point of these programs is to make up for the lack
| of educational access for minorities by giving them a chance to
| prove themselves by admitting them with lower scores. But if
| they complete the same program, doesn't that mean they are just
| as good?
|
| Now, in light of this study, it would be super interesting if
| this divide holds up in exit exam scores. But until we actually
| have that data, I'm not sure your claim is valid.
| doctorpangloss wrote:
| A chatbot can also score very highly on these tests. What do you
| think the survival rate of ChatGPT's patients will be?
| rowanG077 wrote:
| Probably pretty high in diagnosis at least?
| rscho wrote:
| Most patients are unable to communicate their symptoms
| accurately enough. Which is why you need to see them in
| person, talk with them, and examine them. Not saying a robot
| couldn't perform, but certainly not a simple chatbot. Despite
| what some papers say.
| rowanG077 wrote:
| I wonder how much this is simply. Smart people do better on
| tests. Smart people make better doctors.
| rscho wrote:
| I don't think smart people make better docs. I'm USMLE 90+
| percentile, and not particularly clever. It is however,
| important to be clever enough to understand what you read.
|
| Good docs are humble, meticulous and knowledgeable. Stellar
| docs are excellent communicators.
| rowanG077 wrote:
| The study at least proves better test taking strongly
| predicts outcomes, test scores are correlated with
| intelligence as countless studies prove. It may be the case
| that some non-clever people get high test scores. That
| doesn't dismiss the general conclusion.
| rscho wrote:
| No, no contradiction. I said: high USMLE score != smart. GP
| said: good doctor = smart. Study says: good doc = high
| USMLE score. As I also said: good doc = understand what you
| read.
| rscho wrote:
| The article says 'board exam' which is quite different
| from USMLE. So, it's established: I can't read, and I'm
| not especially clever. It all checks out ! :-)
| gadders wrote:
| Reminds me of the old joke:
|
| What do you call the person that came bottom of their class in
| Med School?
|
| Doctor.
| apognwsi wrote:
| i did not read the study. an obvious confounding factor is that
| doctors with better board scores are hired into better hospitals
| with better patient populations, and thus better outcomes.
| dh5 wrote:
| This was controlled for as stated in the article:
|
| > The researchers compared outcomes for patients within the
| same hospitals who were cared for by doctors with different
| exam scores. This allowed the researchers to eliminate, or at
| least minimize, the effect of differences in patient
| populations, hospital resources, and other variations that
| might influence the odds of patient death or readmission,
| independent of a doctor's performance.
| HelloMcFly wrote:
| This is also pretty much the easiest thing the factor out
| through mixed effects modeling (among other methods if
| required). But your statement that higher scoring physicians go
| to places with healthier patient populations is not correct
| across all disciplines. Often it can be the opposite: the best
| physicians go to the major hospitals (usually but not always
| university affiliated) located in major population centers that
| draw in the sickest/worst/rarest cases from the surrounding
| geography.
| palijer wrote:
| The people who ran the study also thought of this and
| controlled for it.
| fn-mote wrote:
| > Gray, Lipner, McDonald, and Vandergrift reported that they are
| employees of ABIM [American Board of Internal Medicine]. Landon
| reported receiving consulting fees from ABIM for ongoing work
| during the conduct of the study.
|
| A study that shows the board test is effective, sponsored by the
| board?
|
| I might be reassured by more detailed statistics about the
| analysis. Even top 25% vs bottom 25% - how much actual variation
| in score are we talking about? What is the probability that
| someone scoring in the top 25% is actually in the top/bottom 25%?
| We imagine a big gap but that's not necessarily true. Consider
| exam scores of 85 90 90 95...
___________________________________________________________________
(page generated 2025-03-01 23:01 UTC)