https://www.nytimes.com/2024/11/17/health/chatgpt-ai-doctors-diagnosis.html

Skip to contentSkip to site index
Health
 
 
Today's Paper
 

Artificial Intelligence

  * Art World Takes On A.I.
  * Putting A.I. in Charge
  * A.I. and Hollywood
  * Microsoft-OpenAI Partnership Frays
  * Turning Off A.I. Tools

Advertisement

SKIP ADVERTISEMENT
You have a preview view of this article while we are checking your
access. When we have confirmed access, the full article content will
load.

Supported by

SKIP ADVERTISEMENT

A.I. Chatbots Defeated Doctors at Diagnosing Illness

A small study found ChatGPT outdid human physicians when assessing
medical case histories, even when those doctors were using a chatbot.

Listen to this article * 9:25 min Learn more

  * Share full article
  * 
  * 

A view from a hallway into an exam room of a health care center.
In an experiment, doctors who were given ChatGPT to diagnose illness
did only slightly better than doctors who did not. But the chatbot
alone outperformed all the doctors.Credit...Michelle Gustafson for
The New York Times
Gina Kolata

By Gina Kolata

Nov. 17, 2024, 5:01 a.m. ET

Dr. Adam Rodman, an expert in internal medicine at Beth Israel
Deaconess Medical Center in Boston, confidently expected that
chatbots built to use artificial intelligence would help doctors
diagnose illnesses.

He was wrong.

Instead, in a study Dr. Rodman helped design, doctors who were given
ChatGPT-4 along with conventional resources did only slightly better
than doctors who did not have access to the bot. And, to the
researchers' surprise, ChatGPT alone outperformed the doctors.

"I was shocked," Dr. Rodman said.

The chatbot, from the company OpenAI, scored an average of 90 percent
when diagnosing a medical condition from a case report and explaining
its reasoning. Doctors randomly assigned to use the chatbot got an
average score of 76 percent. Those randomly assigned not to use it
had an average score of 74 percent.

The study showed more than just the chatbot's superior performance.

It unveiled doctors' sometimes unwavering belief in a diagnosis they
made, even when a chatbot potentially suggests a better one.

And the study illustrated that while doctors are being exposed to the
tools of artificial intelligence for their work, few know how to
exploit the abilities of chatbots. As a result, they failed to take
advantage of A.I. systems' ability to solve complex diagnostic
problems and offer explanations for their diagnoses.

A.I. systems should be "doctor extenders," Dr. Rodman said, offering
valuable second opinions on diagnoses.

We are having trouble retrieving the article content.

Please enable JavaScript in your browser settings.

---------------------------------------------------------------------

Thank you for your patience while we verify access. If you are in
Reader mode please exit and log into your Times account, or subscribe
 for all of The Times.

---------------------------------------------------------------------

Thank you for your patience while we verify access.

Already a subscriber? Log in.

Want all of The Times? Subscribe.

Advertisement

SKIP ADVERTISEMENT

Site Index

Site Information Navigation

  * (c) 2024 The New York Times Company

  * NYTCo
  * Contact Us
  * Accessibility
  * Work with us
  * Advertise
  * T Brand Studio
  * Your Ad Choices
  * Privacy Policy
  * Terms of Service
  * Terms of Sale
  * Site Map
  * Canada
  * International
  * Help
  * Subscriptions

  * Manage Privacy Preferences