Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Becker’s Payer Issues Podcast

Becker’s Payer Issues Podcast artwork

Beyond the Doomerism: What Safe Healthcare AI Looks Like in Practice

Oct 16, 202626 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this episode, Munjal Shah, Co-Founder & CEO of Hippocratic AI, discusses how healthcare AI can improve patient access and outcomes through rigorous safety testing, clinical safeguards and responsible deployment. This episode is sponsored by Hippocratic AI.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

- Hello, everyone. This is Molly Gamble with Becker's - Healthcare. Thank you for tuning in to the - Becker's Healthcare podcast. - Today, we're going to talk about beyond the - doomerism, - what safe health care AI looks like in - practice. - Joining me for today's conversation is Munjal Shah, - cofounder and CEO of Hippocratic AI. Munjal, thanks - for joining me today and being my participant, - my guest for this big topic. - Oh, yeah. I'm, really excited to be here. - Thanks for having me. - Likewise. - For our audience who hasn't caught up with

- us before, Munjal, can you share a bit - more about yourself and your work in health - care just to get them up to speed? - Yeah. I'm a serial entrepreneur. This is actually - my fourth company. So, - I'm a glutton for startup punishment, but, - and - what we've built here at Hippocratic is really - we build, - agents that call patients on the phone and - conduct clinical patient facing activities, - largely within the scope of a registered nurse - and and below. So we don't diagnose and - we don't prescribe, but, but we do everything

- else. - Okay. - And we are sitting down to discuss this - at a pretty interesting time. The conversation about - AI safety - right now is is so loud, - and a lot of this discussion and these - headlines - and consternation, it's happening in altitude - among Frontier Labs, policymakers - that are somewhat removed from the realities of - delivering care. - You've seen leaders from OpenAI and Anthropic specifically - call for shared regulation - to ensure greater safety.

- You, on the other hand, have built Hippocratic - specifically around deploying AI safely in health care. - This has been part of your your mission, - your charter since day one. So maybe to - to start us off, how is the work - you and your partners are doing with clinical - AI agents - just fundamentally different from what we're hearing out - of the Frontier Labs lately? - Yeah. You know, we took a very different - approach. - I think when we started the company, you - know, we named it Hippocratic. We made the - tagline do no harm. And we said... I - actually said to the investors, I said, I'm

- not going to ship until it's safer than - a human. - And, they're like, great. When when is that? - And I was like, I'm... I don't know. - I'm like, and if you're not comfortable with - that, don't give me your money. - And I think, you know, today, - and and if you look at the Frontier - Labs, you know, they're like, well, we need - to all slow down and make it safe. - I'm like, you coulda just made it safe - yourself. - Like, that's what we did. We just didn't - ship. It took us eighteen months to actually - cross the threshold of being safe enough to - even begin our first patient facing calls. And,

- it's just a a personal responsibility. - Like, I don't I don't think you need - to have some... You know, it's like it's - like peer pressure in in middle school. Like, - you don't need to do the right thing - only because if the other guy does the - right thing, you just need to do the - right thing. And so we approach this problem - very differently from a place of personal responsibility - and commitment because... Look. I mean, we're all - patients. We all have family members who are - patients. - Like, you you just wanna build something that - truly helps people and doesn't hurt them, and

- and the only way to ensure that is - is to take the care upfront. - Let's unpack that a little bit. When you - say - safe, let's make it more concrete. You mentioned - it took you eighteen months to to cross - a threshold into what you would deem as - a safe product or safe AI, - with the name Hippocratic and do no harm - in... Intact. - But when you say you're engineering for safety - back in that that time and still today, - what does that actually involve? Can you speak - to that in a bit more detail? - Yeah. You know, safety is a multilayer

- thing. In fact, you know, Jensen said this - best when this whole debate was occurring, and - he said, - safety is about engineering. - If you want safety, you engineer safety. But - the way you do it is redundancy - in every way you can think of. So - we start first where every employee takes the - Hippocratic - Oath, - which is a version of the original one, - but one we made for for kind of - our company. - But it says, hey. Look. I'll I'll make - sure I put patients first. Why? Because you - want people in the right psychology. - Second, we decided that,

- what we would do is limit our scope. - You know? People say they wanna be safe. - I'm like, look. The best measure of being - safe is - self editing. You know? And we said we're - not gonna diagnose. We're not gonna prescribe, but - we also went deeper. We said we're not - gonna do kids under two. We're not gonna - do high risk pregnancies. We're not gonna do - mental health. We're not gonna do hospice care. - Like, there's just areas that we knew - it just wasn't the right, - you know, area for AI to be used. - And then we said, alright. We're not gonna - build one LLM. We're gonna build 31 LLMs

- and have 30 of the LMs double check - the main LLM all the time. In fact, - some of them double check another LM that - double checks the main LM. - And we said you need this redundancy of - systems. I mean, that's how you design an - airplane. You don't just have one way to - control the rudder. You know, you have fly - by wire. You also have a mechanical connection - in some planes. You have a wired connection. - Like, you just because if one goes down, - you wanna make sure the other two were - there. And so we took a similar approach. - Then we said, hey. - That's not enough. Let's make sure it'll transfer - the call to a human if needed if

- you're having chest pains or shortness of breath. - And so we built an escalation engine that'll - actually transfer to humans. We've now done that - 38,000 - times. So we've done, you know, 300,000,000 - plus patient interactions, - but, - in those interactions, 31,000 times, we had to... - Or 38,000 times, we had to transfer it - to a human. And so, - and then we said, let's output test. Let's - hire 7,000 nurses to make, - test calls to the AI, and let's not - ship until,

- and in those test calls, one out of - 50 times, they got a human on the - other line, and we said, alright. We're not - shipping till we're safe as the human. - And and then we monitor the first thousand - calls, every single call within an hour, - and call the patient back. If there's a - problem, we sample half a percent of the - calls after it's live. Like, we've just done - every last thing. - You know, this is belt, suspenders, - whatever comes after suspenders on your jacket. Like, - this is this is every last thing we

- could think of to just ensure safety. - Yeah. It sounds like restraint and redundancy has - been - two really important things among others. But I - I guess just to turn and flip the - seats a little bit, Munjal, like, if you - were sitting in a CMIOs - or CIO seat, what are the questions you'd - want a vendor to answer before letting their - agents anywhere near patients? - Yeah. You know, I think that, - there's this trope out there that's constantly, like, - all the models are commodities.

- I don't think so. - Like, there's a reason Harvey in legal is - doing really well - despite Claude legal launching because people want a - built for purpose system. Because when you get - a built for purpose system, - it closes the last two, 3%, - last five, 10%, which in safety is a - big deal. - Like, you can't tell your boss, hey. I - I bought this thing. It was half the - price, but I'm gonna kill 2% more people. - Like like, that's not okay. - It might be okay in, you know, taking

- your order at Taco Bell with the voice - AI. Like, okay. 2% of the time, I - gave you three tacos instead of two. Okay. - Fine. Like, you know, it's not the end - of the world. But I think for us, - the first question they need to ask is, - you know, what's the architecture? - Is it a multimodel safety architecture? - How has it been tested? Did they do - output testing? - So some of the points I just made. - But one of the ones that I find - everybody is missing at the moment - is they've forgotten why this revolution happened.

- It's called the large language model. - And yet, so many times today, people are - not asking how big are the models you're - using. - So, - you know, most people don't know this, but... - Or have forgotten, but we we measure models - by the number of parameters they have. The - number of parameters to use is quite literally - the number of connections between the neurons in - the neural network. - And just like the human brain, the more - neurons an animal has, - the more connections it has and the smarter

- it is. Right? Meaning, - you know, your mouse brain is not as - smart as your dog brain, is not as - smart as your dolphin brain, - and obviously onto the chimpanzee and and humans. - And so - this is actually the breakthrough that happened where - we got the first models at 7,000,000,030 - parameters that really created breakthroughs, and then CHAT - GPT came - and was a much higher number. But, you - know, today, the the fable you use is - 10... Is rumored to be. They haven't published - the numbers, which is why we lost track,

- but it's rumored to be 10,000,000,000,000 - parameters. - Now why do you care in health care - about the size of the brain that's being - used in the model? In fact, a lot - of companies are running around saying, we have - small language models. You know they're really better. - I'm like, no. They're not. The whole darn - breakthrough was large language models, guys. - And the reason is that we found the - large language models catch subtlety. - So if I ask you, you know, Molly, - are you short of breath? And you don't - say yes. You don't say no. You say, - feels like there's an elephant on my chest.

- Our model better darn well say that's a - yes. - Shoot. You know? And, - what we found is the small models will - miss that. So there's a lot of subtly, - a lot of ambiguity, - that's critical in health care that you want. - And so you want this intelligence. In fact, - the new mythos, - one that's finding all those bugs, is rumored - to be 27,000,000,000,000 - parameters. - And so, you know, at Hippocratic, we're about - now 5,100,000,000,000 - parameters, which - I would use more except you have to

- make it fast enough for voice, and so - that's why we use 30 in parallel. So, - is we we basically push the number of - parameters up by firing them in parallel because - the voice latency means you you can't take - too long. And Fable takes a long time. - Like, you put in something. The other day, - I put in something, and, - you know, I had enough time to go - get a cappuccino and come back before it - was done. Like, you can't use that in - voice, so you you have an inherent trade - off there. But the size of the model - matters because it's basically the amount of intelligence - you're bringing to bear.

- Mhmm. You just touched on something, which is - that this work... This... The the work of - safety - comes intention to other goals to speed and - or... And urgency being one of them. - But Munjal, I mean, nearly every AI company - in health care right now, it... It's making - some version of of the safety claim. And - for a leader - trying to separate - real rigor, like you're discussing, from good marketing, - are there any sniff test or litmus test - that you would put forward as a way - of verifying what what you're... They're actually hearing

- about safety? - The thing about safety is - it... It's only measurable backward looking at some - point. Right? So, - like, you are safe because you have been - safe. - And so, - you know, we've now done these 300,000,000 patient - interactions, - and we've not had one major safety incident. - So I'm like, you can bet on somebody - who maybe - hasn't proven that, but you don't. Like, why - take that chance if you don't have to?

- Second, - I think that - there's another element, - and you can get into the architecture, and - you can get into the model size, and - you can get into some of those properties. - But - in the end, to me, there's only two - things that really will help you. One, did - we do output testing? - Did we test what's coming out of the - model, not just say it's safe because of - what went in? - You know, drugs are done this way. They're - not... The FDA is not like, oh, yeah. - You you built that drug. You're just following - the process, so, therefore, we approve it. Like, - they're like, no. No. No. We're gonna test

- the output. - Mhmm. So look for the output testing. Look - for the history of it. But, you know, - this is a new field, so there's a - ton of history for anybody. - And then I think that, you know, the - other element, - if you're a leader just trying to parse - through it all, - is, you know, have your own teams - go at it clinically - and look for corner cases - that are there. I mean, I'll I'll tell

- you an example. We had a patient calling - in - on, - he was, calling in a schedule with a - neurologist. - He said, oh, I need an appointment with - a neurologist, and and the AI is like, - hey. Why why do you need an appointment? - And he's like, - or or what? Why do you need an - appointment, and what type of appointment do you - need? And he's like, oh, well, I was - struck by lightning yesterday at work. - Like, so you would think scheduling doesn't need - a clinically safe thing, but, actually, every now - and then, it does. - And you don't... You know? And that that - one's not so easy because it's not like

- the answer is he should've called 911. - It's the next day. Like, he's he's okay. - Mhmm. - But at the same time, you better not - book him an appointment with a neurologist six - weeks from now. - Like, somebody's gotta squeeze this guy in tomorrow. - Right? So, - I think that there's a lot of different - elements, but in the end, - you really wanna build a safer system. You - really need to realize almost every call in - health care - can... Has a risk of being a clinical - call. So even when you think

- it's a nonclinical use case, you're you're not - always right, and so you you want this - capability to be there at any time to - jump in. - Mhmm. Mhmm. - The stakes are just so much... They're already - higher in this... In in health care. And - then even there, there can still be, like, - being struck by lightning. My goodness. Like, what - a - cascading... The stakes are just even higher than - you could even imagine. - And and this is where you need the - judgment. Right? Because Right. You you think struck - by lightning means scheduled this way is anywhere

- in the protocol? - Like like, there's no business rule that was, - if struck by lightning, then do this. Like, - you know, even the escalation protocols don't have - probably struck by lightning. So - the model just had to have judgment. And, - basically, the bigger the model, the better the - judgment. - Working off of of that story, I mean, - so much of the public, - narrative or commentary or attention - right now is really looking at advanced AI - as something to fear.

- It sounds like from these stories and the - work that you've been putting forward at Hippocratic - AI, you and your team are just clearly - operating from a different premise altogether. - What would you say, Manjal, is the counter - case you'd make to that type of doomerism? - And maybe more to the point, what are - you seeing in live deployments that actually supports - optimism? - Yeah. - This thing drives me nuts. - You know, the... For... I'm I'm gonna start - at the highest level. - Why are we listening to doomers? These people

- are owing 5,000 - in the history of the world. They've never - been right because we're all here. - I'm like, oh, but this time for sure, - like, you know, it's it's... There's must be - something inherent about human beings beings that we - want to believe the doomer, framework because literally - these people have always been wrong. And I - I I thought about it deeply. I was - like, why are they always wrong? I mean, - as it's on the surface, I could see - why some of these arguments, you know, might - play out. It's... I actually think it's because

- you can never predict the secondary and tertiary - implications, - only the first. So you're like like, let's - say you and I had owned farms on... - When the tractor came out in the industrial - revolution. People might have said, oh my god. - The farmer's gonna do so amazing. - You guys own all the good fertile land. - Like, you're gonna make a fortune now. You're - gonna make food for so much less, and - you're gonna make so much more of it. - Did the farmers end up super rich in - society from the tractor? No. Because they all - made a ton more food, and the cost

- of food dropped. - And, actually, they ended up some of the - worst off. And so... But people could anticipate - kind of the first phase of it, but - not the second implication and the third implication. - And I think we keep - missing that. The second thing that I've seen - is we're wrong about the job loss thesis. - In fact, I'll I'll tell you one of - the key learnings that we've seen with Hippocratic - is we have this framework called Copilot, Autopilot, - Infinipilot. - And Copilot is things you do with the - human in the loop. Autopilot is things you

- do today that we can now do cheaper, - can staff where you couldn't staff. And infinite - pilots, things you never thought to do till - you had an infinite supply of clinicians, you - know, - at at a very low cost. And - almost all of our... We knew Copilot was - gonna be hard because the human in the - loop, you only get 10% efficiency gain, and - it kind of evaporates on you. So we - focused on autopilot. Actually, every one of our - autopilot use cases has been a bit of - a struggle. - Like, we've gotten people there, but it's kinda - ground warfare. Like, we have had to fight

- and fight to make the use cases work. - And every one of the infinite pilot, the - things you never thought to do that are - emergent have worked brilliantly. - Mhmm. - And - that's where I realized, okay, the job loss - thesis is false. So I'll give you one - example. You know, we recently did a loss - to follow-up, - for a large health system where we called - 1,700 patients that had not gone in for - their low dose CT scans for lung cancer. - And we didn't get all of them, but - they were gonna be at zero because they

- gave up. That's what loss to follow-up means. - And, - they had tried to text. They had tried - an email. They had tried to letter. And - we got 02:50 to schedule. We got a - 180 to show up. We found nine significant - findings, three cancers, like, - you know, like... And one of them was - caught so early. It only needed a re - resection. It did not actually need a, - chemo or radiation. I mean, - you know, what a great - outcome - that we only could've done with the abundance

- of of AI. And so - I think that the company should make great - products and not put them out till they're - safe. - They should also, by the way, take liability, - which we do at Hippocratic. - We're saying, look. We'll stand behind our products. - And I think any vendor that doesn't - is is a vendor one shouldn't use because - taking liability means I'll make my product better - because - I'm forced to financially as well as morally. - So, - anyways, those are those are some of the - things I think that, you know, we really

- should look at the balance of payments because - we do that with every other technology in - health care. - Mhmm. I think it's a really good point - of our... Just human psychology and and biases - we might have to our, like, not - not anticipating those secondary and tertiary effects like - you had said in your opening. - The part of that response, Minjal, it's very - important to to consider. I love the farmer's - analogy. - As we head into 2027, - maybe we can wind down here and leave - our listeners with one thing that you'd want - the health care c suite to understand about

- adopting AI safely. - And then the second part of that... My - question is what you think will separate leaders - who get this right from the ones who - who don't. - Yeah. So I think one part is, - don't - think there's such a thing as clinical and - nonclinical. - We call this omnidopic. - It says, hey. When a patient calls in - to schedule, if you have them on the - phone, - you better darn well - talk about - the discharge follow-up that they needed to do

- today, - even if that's not why they called in. - And you can only do that if you - have a clinical and nonclinical - single platform, single framework. You should look for - that in the market and try to build - that because it'll take advantage of kind of - the opportunity, - much better. I think that the second element - is, - you know, - architect... - Look for things that are architected for safety - from day one. - Everybody thinks, oh my god. These frontier models - are amazing, and they do everything. I think

- they've forgotten one very important insight we've gathered, - which is... - I call this the jagged intelligence insight. - These models can solve a 150 year old - math proofs. - But the other day, by the way, one - of these frontier models we were using, - we were testing out actually was, like, it - offered a woman a pap smear over telemed. - Said, ma'am, for your pap smear, would you - like telemed or in person? - And I'm like, pretty sure you can't do - a pap smear over telemed. - But, - these models are really smart and really dumb

- at the same time. It's just we don't - realize the dumb things because we don't have - evals that tell us those those troughs. - And so you really wanna look for things - that, - that go... But I think you will see - the specialist companies in general. - Really, the question you're gonna wanna ask them - is what are all the troughs they fixed, - versus just, you know, what's your absolute score - on how you can solve, you know, the - next math proof. And so - I think that that's a... That's an element. - And then the last part is - patients are getting used to talking AI.

- Mhmm. This question that you used to ask - about, oh my god. Are they really gonna - talk to it? - They're doing it all the time now. They're - talking to MetaMuse. - I mean, the other day, I I ordered - flowers for a family member on one of - the on on MetaMuse, - and, - they're talking to it in in Taco Bell - drive thrus. They're talking to it on their - bank. Like, they are very much getting used - to it. And so - you're soon gonna see in the next, I - bet, in '27, - every single consumer you'll start seeing will know

- the difference, say, hey. This is not like - talking to IVR. It's way smarter. It really - answers my question. - It really completes my action, - and they'll just be comfortable. In fact, even - prefer it over many other forms of interaction, - and I think that folks need to get - ahead of that, - because the patients are doing it all day - long. I mean, what, 25% - of all searches on ChattGPT now are health - care? So people are like, are they gonna - really use AI in health care? I'm like, - they're already using it.

- Right. And, by the way, there's almost a - billion weekly act of uniques. That means 250,000,000 - people are doing health care searches - today. Like, what do you mean? Like, it's - already happening. And so I think they need - to get ready for this new world, - and think about what sort of... How does - that change access? How does that change follow-up? - And then the next phase is now already - coming, which is they're not even gonna call - you. Their agent, their MetaMuse is gonna call - you

- and book the appointment and - follow-up on things. - And are you ready to have an interface - where you can talk to their agent, - and their agent will talk to the patient? - I love this. So for our listeners, it's - just twenty twenty seven omnitopics, so not distinguishing - or not separating the clinical versus nonclinical. - I love that challenge for those interactions. If - you have a patient on the line, - everything in... Is somewhat... Is is in clinical - in nature. - And then look for things that are built

- with safety - day one. Beware of that jagged intelligence. - AI can be so brilliant and so - so dumb. Like you remind us with that - PAP smear anecdote, Mujal. And then patients getting - used to talking to AI. Sometimes it feels - like there's a lag to how we're still - seeing AI and moving into this industry. But - to your point, consumers, - patients, they're not only using it and talking - to it, but sometimes at length, - much much longer and with more detail than - you might assume. - Munjal, I just wanna thank you for your

- time and your insights today. Anything you wanted - to add in closing here? - No. I, you know, I I think that... - Look. These things are really great helps in - health care. You know, I had a family - member diagnosed with stage four cancer actually this - week, and it was awful. - But I will tell you the large language - models are... Have real... I mean, in one... - In literally the last seventy two hours, I - was able to - look at every clinical trial, look at, you - know, every treatment plan, understand what it means,

- and prepare for my meeting with the oncologist, - later on today, actually, - in a way that I could have never. - Like, you know, I could have never... It - would have taken me a month of research - to get what I got. - And, - I truly... And even... It prepared me a - 71 - bulleted checklist - to to, - go through and make sure... And even took - the 10 that I gotta make sure I - ask the oncologist today. - Like, it's really helped me and my family

- zero in on what's needed. And, - this is, - you know, this is a gift we never - had before, and I I think the whole - world already kind of is starting to get - that if they've been through a health crisis - since these LMS came out. - Mhmm. - In response to some of the doomerism is, - unfortunately, when health care hits close to home - like it has for you and your family, - which I'm sorry to hear, - Munjal, about your family member, and I I - wish them and you the best. - But nothing can quite counterdumerism,

- like, being reminded of what's capable now, with - the technology like this at the fingertips of - of consumerisms - and patients and families and organizations. - So thanks for sharing that. - Listeners, you can tune into more podcasts from - Becker's Healthcare by visiting our podcast page at - beckershospitalreview.com. - Munjal, thank you again. Always a pleasure hearing - from you and learning more about the work - you and Hippocratic AI are building and doing - with organizations. - Thank you for having me, Molly.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Payment Integrity Can Protect Savings Without Creating More Friction

    Nov 20, 202619 minAudio

    In this episode, Jon Grandstaff, Senior Vice President of Payment Integrity at Zelis, discusses how payers can reduce payment friction, use AI to support more accurate claims reviews and improve payer-provider...

    Credit the people on this episode

  2. From Learning Collaborative to Gap Closure: How Wisconsin Medicaid Is Telethinking Colorectal Cancer Screening

    Nov 19, 202612 minAudio

    In this episode, Amy Conlon, MPH, Population Health Sr. Manager at Exact Sciences, and Jeff Huebner, MD, FAAFP, Executive Medical Officer, Division of Medicaid Services, Wisconsin Department of Health Services, discuss...

    Credit the people on this episode

  3. What Health Plans Miss When They Look at Markets in Isolation

    Nov 18, 202617 minAudio

    In this episode, Denis Mrkva, Senior Vice President of Analytics Solutions at Data Axle, discusses how health plans can combine consumer, employer, geographic and healthcare data to identify growth opportunities, reduce...

    Credit the people on this episode

  4. Strengthening Provider Networks Through AI, Data and Strategic Partnerships

    Nov 17, 20268 minAudio

    In this episode, Bob Tavernier, Solutions Executive at Quest Analytics®, discusses how health plans can leverage AI, provider data and strategic partnerships to improve network accuracy, strengthen regulatory compliance...

    Credit the people on this episode

  5. Martina Lee Strickland, Head of Marketing, PR and Brand at Clever Care Health Plan

    Nov 16, 202617 minAudio

    In this episode, Martina Lee Strickland, Head of Marketing, PR and Brand at Clever Care Health Plan, joins the podcast to discuss the importance of building trust when serving underserved communities. She also shares...

    Credit the people on this episode