Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Many Minds

Many Minds artwork

From the archive: What does ChatGPT really know?

Jul 24, 202455 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Hi friends, we're on a brief summer break at the moment. We'll have a new episode for you in August. In the meanwhile, enjoy this pick from our archives! ---- [originally aired January 25, 2023] By now you've probably heard about the new chatbot called ChatGPT. There's no question it's something of a marvel. It distills complex information into clear prose; it offers instructions and suggestions; it reasons its way through problems. With the right prompting, it can even mimic famous writers. And it does all this with an air of cool competence, of intelligence. But, if you're like me, you've probably also been wondering: What's really going on here? What are ChatGPT - and other large language models like it - actually doing? How much of their apparent competence is just smoke and mirrors? In what sense, if any, do they have human-like capacities? My guest today is Dr. Murray Shanahan. Murray is Professor of Cognitive Robotics at Imperial College London and Senior Research Scientist at DeepMind. He's the author of numerous articles and several books at the lively intersections of artificial intelligence, neuroscience, and philosophy. Very recently, Murray put out a paper titled 'Talking about Large Language Models', and it's the focus of our conversation today. In the paper, Murray argues that - tempting as may be - it's not appropriate to talk about large language models in anthropomorphic terms. Not yet, anyway. Here, we chat about the rapid rise of large language models and the basics of how they work. We discuss how a model that - at its base - simply does "next-word prediction" can be engineered into a savvy chatbot like ChatGPT. We talk about why ChatGPT lacks genuine "knowledge" and "understanding" - at least as we currently use those terms. And we discuss what it might take for these models to eventually possess richer, more human-like capacities. Along the way, we touch on: emergence, prompt engineering, embodiment and grounding, image generation models, Wittgenstein, the intentional stance, soft robots, and "exotic mind-like entities." Before we get to it, just a friendly reminder: applications are now open for the Diverse Intelligences Summer Institute (or DISI). DISI will be held this June/July in St Andrews Scotland - the program consists of three weeks of intense interdisciplinary engagement with exactly the kinds of ideas and questions we like to wrestle with here on this show. If you're intrigued - and I hope you are! - check out disi.org for more info. Alright friends, on to my decidedly human chat, with Dr. Murray Shanahan. Enjoy! The paper we discuss is here. A transcript of this episode is here. Notes and links 6:30 - The 2017 "breakthrough" article by Vaswani and colleagues. 8:00 - A popular article about GPT-3. 10:00 - A popular article about some of the impressive - and not so impressive - behaviors of ChatGPT. For more discussion of ChatGPT and other large language models, see another interview with Dr. Shanahan, as well as interviews with Emily Bender and Margaret Mitchell, with Gary Marcus, and with Sam Altman (CEO of OpenAI, which created ChatGPT). 14:00 - A widely discussed paper by Emily Bender and colleagues on the "dangers of stochastic parrots." 19:00 - A blog post about "prompt engineering". Another blog post about the concept of Reinforcement Learning through Human Feedback, in the context of ChatGPT. 30:00 - One of Dr. Shanahan's books is titled, Embodiment and the Inner Life. 39:00 - An example of a robotic agent, SayCan, which is connected to a language model. 40:30 - On the notion of embodiment in the cognitive sciences, see the classic book by Francisco Varela and colleagues, The Embodied Mind. 44:00 - For a detailed primer on the philosophy of Ludwig Wittgenstein, see here. 45:00 - See Dr. Shanahan's general audience essay on "conscious exotica" and the space of possible minds. 49:00 - See Dennett's book, The Intentional Stance. Dr. Shanahan recommends: Artificial Intelligence: A Guide for Thinking Humans, by Melanie Mitchell (see also our earlier episode with Dr. Mitchell) 'Abstraction for Deep Reinforcement Learning', by M. Shanahan and M. Mitchell You can read more about Murray's work on his website and follow him on Twitter. Many Minds is a project of the Diverse Intelligences Summer Institute (DISI) (https://disi.org), which is made possible by a generous grant from the Templeton World Charity Foundation to UCLA. It is hosted and produced by Kensy Cooperrider, with help from Assistant Producer Urte Laukaityte and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by Ben Oldroyd (https://www.mayhilldesigns.co.uk/). Our transcripts are created by Sarah Dopierala (https://sarahdopierala.wordpress.com/). You can subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you like to listen to podcasts. **You can now subscribe to the Many Minds newsletter here!** We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com. For updates about the show, visit our website (https://disi.org/manyminds/), or follow us on Twitter: @ManyMindsPod.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. The puzzling story of human sleep

    Nov 21, 202659 minAudio

    Sleep presents a puzzle - layers of puzzles, in fact. Evolution prepares us for action - it tunes our senses, sharpens our wits, makes us alert to opportunity and danger. But then for huge chunks of our lives, we throw...

    Credit the people on this episode

  2. Why the brain is not a computer

    Nov 6, 20261 hr 16 minAudio

    It's one of the most powerful ideas in the history of science. That a brain is, in essence, a computer. That we can look past its gelatinous flesh and see through to its underlying workings. That these workings are best...

    Credit the people on this episode

  3. Culture, from birth to death (metal)

    Oct 23, 20261 hr 48 minAudio

    Culture can be a little hard to wrap our minds around. It's seemingly everywhere, enveloping us - but where is it exactly? What are its core properties? How does culture change? Why does it change? What tools and...

    Credit the people on this episode

  4. Island of the tool users

    Oct 9, 20261 hr 27 minAudio

    Tool use is rare in the animal kingdom - but maybe not as rare as we once thought. Since Jane Goodall famously observed a chimpanzee fishing for termites, many new examples of animals using tools have come to light....

    Credit the people on this episode

  5. Cognitive offloading and its consequences

    Sep 25, 20261 hr 15 minAudio

    For thousands of years - maybe for as long as we've been human - we've offloaded our thinking. We've counted on our fingers. We've carved tallies into bone. We've invented a long series of tools to help us reason,...

    Credit the people on this episode