Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Physician in the Loop

Physician in the Loop artwork

Special Topic: The thinking is the job

Oct 27, 20267 minAudio
ShareXLinkedInFacebookWhatsAppEmail

A new paper says medical training should teach doctors when to defer to a machine. It has the order backwards, and almost nobody is counting what the deferring is worth. Read the special topic and its sources: https://physicianintheloop.org/specials/mastery-and-the-machine/ The voice on this episode is generated by AI. It reads a special topic from Physician in the Loop as written; every fact in it links to its source.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show.

This is a Physician in the Loop special topic.

The thinking is the job

A new paper says medical training should teach doctors when to defer to a machine. It has the order backwards, and almost nobody is counting what the deferring is worth.

Four emergency physicians at the University of Texas Southwestern Medical Center published a perspective on Oct. 13 with a line that should stop any doctor cold. One day, they write, the most dangerous actor in the room may not be the machine making a recommendation. It may be “the human who insists on overruling it simply because being in the loop still feels like being in charge.”

They may be right about that doctor. The trouble is the paper they wrote to get there. It asks medical training to stop prizing struggle for its own sake and refocus it from recalling facts toward “interrogating machine reasoning, detecting failure modes” and knowing when a person should step in.

Deference is not a skill of its own. It sits on top of one. The same paper says so. A trainee who has never had to defend a differential diagnosis “may be poorly prepared to recognize when an AI system is incomplete, overconfident, or wrong”. You cannot teach a young doctor to tell a good recommendation from a confident wrong one until she can tell those apart on her own. The struggle is not a teaching method we picked and could swap for a better one. It is the thing being taught.

Which raises the question almost nobody is answering. If the doctor in the loop is the safeguard, how much does it catch? A systematic review of these systems in primary care, published this month, looked at eleven studies. Not one reported whether the clinician agreed with the machine or overrode it. Three of the perspective's six references are commentary, and the five that can be read follow no patient past the moment a doctor accepted or refused a machine's advice.

The one randomized trial among them ran at the end of 2023. 50 physicians, nearly half of them residents, got an hour and up to six written cases, with ChatGPT or their usual references. Blinded scorers graded the reasoning they wrote down. With the model, a median of 76 percent per case. Without it, 74. Two points, and the trial could not separate that from chance. The model working the same cases alone scored 16 points higher. Now read what that trial is not. Nobody took a history. Nobody examined anyone. Nobody found out what happened to a single patient. The model's score came from a prompt a researcher wrote and ran three times, in what the authors call a secondary exploratory analysis. They say the rubric score is a signal and that “broader clinical trials are necessary to assess for meaningful differences in downstream clinical impact.”

Deskilling and automation bias are visible wherever someone bothers to measure them. 27 radiologists read 50 mammograms with ratings attached that they were told came from AI. When the suggestion was wrong, the least experienced readers went from getting about four in five right to one in five. The most experienced dropped below half. In four Polish endoscopy centres, adenoma detection in colonoscopies done without the software fell from 28.4 percent to 22.4 percent in the three months after it arrived. Its authors call that a possible effect of continuous exposure. Not forgetting how to find a polyp. Not looking as hard. One arrangement has held up in real care. A Swedish trial screened nearly 106,000 women. Software sorted the mammograms and flagged findings. Radiologists still decided whom to call back. Detection rose from five cancers per thousand to six, with no more false positives, reading work almost halved and cancers surfacing between screens no worse. Public money and the Swedish Cancer Society paid for it. What was tested there was the arrangement. Not the override.

Rachael Bedard, a geriatrician who uses OpenEvidence every day, wrote in a New York Times guest essay on Sept. 25 that it has made her a better doctor and may be making her trainees worse, and argued for teaching them to think critically with a machine in the loop. Adam Rodman, quoted in the same piece, put the line where it belongs. A little deskilling in experienced hands probably does not matter much. In someone with far less skill, “the cognitive offloading can cause never-skilling.” One of those is a bad habit. The other is a doctor who never arrives.

The blunt case for getting out of the way is not Bedard's. In JAMA in August, Ezekiel Emanuel, Abe Baker-Butler and Neal and Vinod Khosla of Curai Health and Khosla Ventures asked whether autonomous AI will beat AI-aided physicians. Where that turns out to be true, a doctor standing in the way costs patients something real, and the perspective is right that a profession should not dress up its own indispensability as a moral principle. But look at the job it offers instead. Orchestration, it says. Among other things, routing requests, reconciling contradictory outputs, watching for deterioration. That is the most automatable work in the building.

So count it. A hospital that puts a doctor between a model and a patient should record what the doctor did and what followed, and how those patients fared against the rest. A monthly chart sample is within reach of a small hospital. Short of a trial, it is the only way to learn whether the reviewer is a safeguard or a signature. Put a plausible, wrong recommendation in front of a trainee and grade the catch, as the perspective itself asks. The Association of American Medical Colleges has AI competencies targeted for this fall, and that belongs in them. Funders should pay for the prospective trials these authors keep asking for.

Teach the foundations first. They are the only thing that separates a doctor from a layman with a search box, and a profession that hands them over will not be asked to stay. The perspective may be right about the doctor who overrules a better system out of habit. Nobody can say how often, because almost nobody is counting. We are not defending mastery. We are not yielding wisely either. We are guessing, and telling students it is a curriculum.

This special topic was read by an AI voice. Its sources are linked at physicianintheloop.org.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Special Topic: What changes on Jan. 1

    Nov 20, 20267 minAudio

    Laws in at least eight states that touch AI in health care take effect on Jan. 1. Most bind insurers and chatbot companies. A few, in California and Colorado, reach the practice itself. Read the special topic and its...

    Credit the people on this episode

  2. Friday Letter: On a short leash

    Nov 20, 20267 minAudio

    Software has begun to act for doctors, not just advise them. It should be kept on a short leash, because the doctor watching it rarely notices when it slips. Read the letter and its sources:...

    Credit the people on this episode

  3. Daily Briefing for November 20, 2026

    Nov 20, 20266 minAudio

    Hospitals race to patch a Java flaw found with AI in an hour The voices on this episode are generated, and so is the script, from the day's post at Physician in the Loop. Every fact in the post links to its source. Read...

    Credit the people on this episode

  4. Special Topic: Not the doctor's to sign

    Nov 19, 20267 minAudio

    Free AI tools for doctors come with a HIPAA contract one doctor can accept alone. A hospital's records need the hospital's signature, and the companies should make the safe path the default. Read the special topic and...

    Credit the people on this episode

  5. Daily Briefing for November 19, 2026

    Nov 19, 20267 minAudio

    AI hacking agents seize some networks in under 3 hours, FBI warns The voices on this episode are generated, and so is the script, from the day's post at Physician in the Loop. Every fact in the post links to its source....

    Credit the people on this episode