1. Home
  2. tech
  3. news
  4. ChatGPT might replace your doctor — and it will actually do a better job of caring for you

ChatGPT might replace your doctor and it will actually do a better job of caring for you

Adam Rogers   

ChatGPT might replace your doctor — and it will actually do a better job of caring for you

Nobody expects touchy-feely niceties from a chatbot. Mediocre writing, sure, along with some made-up facts and a bit of random racism. I've written about those AI downsides myself.

But these big, new chatbots can, as we know, also generate human-sounding responses to prompts and questions. And in a recent head-to-head test, that capability gave the bots a surprising superiority in what ought to be one of the most intrinsically human of all activities: serving as a doctor.

To conduct the test, a team of researchers from the University of California in San Diego lurked on r/AskDocs, a Reddit forum where registered, verified healthcare professionals answer people's medical questions. The researchers selected nearly 200 representative questions on the forum, from the silly-sounding ("Swallowed a toothpick, friend said I'm going to die") to the terrifying ("miscarriage one day after normal ultrasound?"). They then fed the questions into the virtual maw of the bot ChatGPT, and had a separate group of healthcare experts conduct a blind evaluation of answers from both AI and MDs.

The results were shocking. For one thing, ChatGPT came out well ahead of the human doctors on usefulness. Almost invariably, the chatbot answers were rated as three or four times as reliable as the ones from the poor wee humans. What's more, the bots didn't show any of the distressing tendency to make stuff up that they often have in other circumstances.

But here's the most striking part: The chatbot answers, on average, were rated seven times as empathetic as the ones from humans. Seven times! They provided just what you want from your doctor: care and an emotional connection. It's as if the unfeeling android Mr. Data figured out how to convincingly emulate Dr. Crusher's comforting bedside manner.

Now admittedly, the bar is low on beating human doctors for displays of empathy. Still, the apparent facility with which the bot could handle medical concerns, in both style and substance, presages an actual, real-world use for these things. I'm skeptical that AI bots driven by large language models will revolutionize journalism or even make internet search better. I suppose I'm open to the idea that they'll accelerate the coding of software and the analysis of spreadsheets. But I now think that with some tinkering, chatbots could radically improve the way people interact with healthcare providers and our broken medical-industrial complex.

The point of the empathy experiment wasn't to show that ChatGPT could replace a physician or a nurse. It was to show that a chatbot could have a role in the provision of care. Our for-profit healthcare system doesn't hire enough doctors and nurses, and it expects the ones it does hire to treat more and more patients, assembly-line style. Nobody likes it, except the people getting rich.

"People are disconnected from healthcare, and they're desperate," says John Ayers, a computational epidemiologist at UC San Diego who was lead author of the new paper. So they're looking for answers on forums like r/AskDocs. "This is how patients do this now. And doctors didn't sign up for it."

The pressure to answer those messages has become intense. The COVID pandemic accelerated remote, online contact between doctors and patients — and even in the pandemic's first year, research suggested docs spent almost an hour every workday dealing with their email inboxes. Add in dealing with other electronic medical record technocracy and you end up with some doctors dedicating half their time every day to these back-and-forths. It's enough that insurance often bills for time spent answering messages, making them a potential source of revenue above and beyond face-to-face interactions..

Prior studies asked whether patients and doctors like using these messaging systems; Ayers looked at whether the system actually work. "We used real messages," he says. "Nobody has ever done that before." The result, based on the quality of the interactions, was definitive. "ChatGPT won in a landslide," Ayers says. "This stuff is probably ready for prime time."

Based on the bot's initial success, Ayers is ready to see what more it can handle. "We want to begin randomized controlled trials, where you evaluate patient messaging against patient outcomes," he says — not just whether the messages are accurate or empathetic, but whether they help keep people healthier or alive longer. What if a chatbot could help someone recovering from a heart attack stay on a low-salt diet, remind them to take their meds, keep their treatment updated? "A message in that case could save that patient's life," Ayers says.

For all the tech-world promises of robot pets and AI psychotherapists, the idea of a caring chatbot still feels destabilizing — maybe even dangerous. Nobody thinks ChatGPT actually cares, any more than they think it's actually smart. But if our current, broken healthcare system makes it impossible for humans to take care of one another, maybe fake taking-care will save real lives. An artificially intelligent assistant may not be more human than human, but maybe it'll be more humane.

Specialized AI systems — not dumb chatbots — are already pretty good at diagnostics. They're highly trained to detect one thing, like a tumor or sepsis, using specific test results as input. But they're expensive and hard to build. So the medical establishment is jumping on chatbots as a cheaper, more ubiquitous tool. Dozens of companies are working on applications, aiming for uses from diagnosing illnesses to helping with the slog of paperwork that has somehow become the responsibility of both doctors and patients alike. If you're lucky enough to have health insurance, your insurance company probably already has some kind of dumb chatbot for you to talk to before you can get a human on the phone.

If you ask people whether they're into that idea, they mostly say no. Sixty percent of Americans surveyed recently by the Pew Research Center said they wouldn't want an AI system diagnosing what's ailing them or proposing treatments. But they're probably going to get it anyway. Don't tell anyone I said this, but a lot of what healthcare workers do is already a bit formulaic — at least at the lowest-level, patient-facing interface. You feel icky and call an advice nurse; they ask preset questions to determine whether you should get to an ER or just go take a Tylenol. That's called triage. Or, if you have electronic access to your medical records, maybe you get results back from a new set of tests, and you email your doc to ask what they mean. Either way, you don't expect to enjoy these encounters. They're perfunctory. Robotic, even.

A human clinician backed by the knowledge base and processing power of AI systems will only be better

Well, you know who's very good at robotic things? Robots! Recently a team of researchers from Harvard showed dozens of descriptions of health problems to three groups: physicians, people with no medical training, and ChatGPT. They asked everyone (and -thing) to diagnose the illness, and then for triage recommendations.

The non-doctor humans were allowed to do an internet search — what healthcare folks call, with dread, "Dr. Google." But even with the online assist, the untrained humans were terrible at diagnosis. No shock there. But, as the researchers report in a recent preprint — meaning it isn't peer-reviewed yet — the chatbot was almost as good at diagnosis (scoring over 80%) as the human physicians (who scored over 90%). And on triage, ChatGPT got a little better than 70% correct. That sounds crummy compared with the physician's 91%, but still! This is a general-purpose chatbot, almost as good as a fully trained doctor.

Now imagine adding to that skill set the sort of mundane, time-consuming healthcare tasks that chatbots should be able to handle — things like scheduling appointments, requesting prior authorization from insurance, and dealing with electronic medical records. "These are tasks nobody went into medicine to do, and they are massive headaches, massive time sinks, physically and emotionally draining," says Teva Brender, a medical resident at the University of California in San Francisco. Maybe a chatbot could generate at least the beginnings of this kind of bureaucratic traffic, along with all the emails to patients. "The physician could skim it and say, 'Yes, this is right,' and send it off," Brender says.

That seems like a likely scenario. Highly trained chatbots will work in tandem with physicians, nurses, and physician assistants to deliver more empathetic and more complete answers to people who need care. As Ayers' team wrote in 2019, people are so desperate for medical help that they post images of their own genitals to the subreddit r/STD in hopes of getting an accurate diagnosis. That is just sad beyond belief, and a staggering indictment of our truly shitty and inhumane system of healthcare.

In a system this poor, AI could actually be an improvement. "A human clinician backed by the knowledge base and processing power of AI systems will only be better," says Jonathan Chen, a physician at the Stanford University School of Medicine who has been studying AI systems. "It is entirely likely that patients will reach for imperfect medical advice from automated systems with 24/7 availability, rather than waiting months for an appointment with a human expert."

To make those AI-driven systems better, lots of folks — including Ayers' team — are now working on smaller language models finely tuned with medical information. ChatGPT's attraction, such as it is, is that it's a generalist drawing input from everything on the internet. But that's how biases and misinformation sneak in. Give those medical chatbots access to people's individual medical records, and they could offer more precisely directed advice. "When this tech gets access to electronic health records, that's the real game changer," Ayers says.

If a future of AI-driven health advice — complete with access to your medical records — makes you worried, I don't blame you. The bad science-fiction outcomes here are pretty dystopic. After years of work, the Food and Drug Administration still doesn't have a framework that's ready to regulate AI and machine learning in medical devices. Someone will have to figure out all the liability questions surrounding chatbot advice, especially when it's bad. Healthcare AI startups will want the cheapest versions with the most financial bang, which won't necessarily have the best patient outcomes. And if a healthcare company manages to fine-tune a chatbot with state-of-the-science medicine, then any company can do the same thing with homeopathy or scented candles — or anti-vaccine nonsense. Those chatbots will spew dangerous misinformation, both eloquently and empathetically.

"That's the worst case," says Greg Corrado, the head of Health AI at Google. "This is not something for Silicon Valley folks to do in isolation." That means developing these systems in conjunction with healthcare experts, not just healthcare executives — to make sure they're private and safe and that they actually help people.

It won't be easy, but it might be necessary. Our healthcare system, sadly, isn't built to provide everyone with decent human caregivers. And until that changes, it'd be nice to have robots that could help us stay healthy. If they can simulate caring about us at the same time — maybe even better than human doctors do — well, that'd still be a nice message to receive.

Adam Rogers is a senior correspondent at Insider.

Popular Right Now