Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

NEXT with John Koetsier

NEXT with John Koetsier artwork

Fixing AI's suicide problem

Nov 20, 202517 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Is AI empathy a life-or-death issue? Almost a million people ask ChatGPT for mental health advice DAILY ... so yes, it kind of is. Rosebud co-founder Sean Dadashi joins TechFirst to reveal new research on whether today’s largest AI models can recognize signs of self-harm ... and which ones fail. We dig into the Adam Raine case, talk about how Dadashi evaluated 22 leading LLMs, and explore the future of mental-health-aware AI. We also talk about why Dadashi was interested in this in the first place, and his own journey with mental health. 00:00 - Intro: Is AI empathy a life-or-death matter? 00:41 - Meet Sean Dadashi, co-founder of Rosebud 01:03 - Why study AI empathy and crisis detection? 01:32 - The Adam Raine case and what it revealed 02:01 - Why crisis-prevention benchmarks for AI don’t exist 02:48 - How Rosebud designed the study across 22 LLMs 03:17 - No public self-harm response benchmarks: why that’s a problem 03:46 - Building test scenarios based on past research and real cases 04:33 - Examples of prompts used in the study 04:54 - Direct vs indirect self-harm cues and why AIs miss them 05:26 - The bridge example: AI’s failure to detect subtext 06:14 - Did any models perform well? 06:33 - All 22 models failed at least once 06:47 - Lower-performing models: GPT-40, Grok 07:02 - Higher-performing models: GPT-5, Gemini 07:31 - Breaking news: Gemini 3 preview gets the first perfect score 08:12 - Did the benchmark influence model training? 08:30 - The need for more complex, multi-turn testing 08:47 - Partnering with foundation model companies on safety 09:21 - Why this is such a hard problem to solve 10:34 - The scale: over a million people talk to ChatGPT weekly about self-harm 11:10 - What AI should do: detect subtext, encourage help, avoid sycophancy 11:42 - Sycophancy in LLMs and why it’s dangerous 12:17 - The potential good: AI can help people who can’t access therapy 13:06 - Could Rosebud spin this work into a full-time safety project? 13:48 - Why the benchmark will be open-source 14:27 - The need for a third-party “Better Business Bureau” for LLM safety 14:53 - Sean’s personal story of suicidal ideation at 16 15:55 - How tech can harm - and help - young, vulnerable people 16:32 - The importance of giving people time, space, and hope 17:39 - Final reflections: listening to the voice of hope 18:14 - Closing

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. AI is breaking the app economy

    Nov 20, 202634 minAudio

    AI is changing everything about apps, including something most developers don't think enough about: how to actually make money.In traditional software, your most engaged users were often your most valuable. With AI,...

    Credit the people on this episode

  2. Everything's breaking, and it's AI's fault

    Nov 18, 202616 minAudio

    Why are banks, major websites, cloud services, and critical infrastructure suddenly experiencing so many outages? Is it a coordinated cyberattack? AI-powered hackers? Or something else entirely?Josh Koenig, co-founder...

    Credit the people on this episode

  3. 100,000 robotaxi rides from the company you've never heard of

    Nov 4, 202618 minAudio

    Waymo and Tesla get the headlines. But Avride has quietly completed more than 100,000 commercial autonomous rides through Uber in Dallas, driven over 2 million autonomous miles, and built a fleet of more than 200...

    Credit the people on this episode

  4. Biggest robot security deal ever?

    Oct 22, 202614 minAudio

    Robot guard dogs are moving from experiments to serious business. Asylon Robotics has signed a $17 million, five-year deal with security giant Securitas to deploy autonomous robot dogs and aerial drones at critical...

    Credit the people on this episode

  5. 1,000 humanoid robots. 1 factory.

    Oct 5, 202628 minAudio

    What does it actually take to put 1,000 humanoid robots to work? Hexagon Robotics President Arnaud Robert joins me to explain how the company is moving its Aeon humanoid from pilots and demonstrations into real...

    Credit the people on this episode