Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

062: AI's Sputnik moment? DeepSeek R1 has entered the chat(GPT).

Jan 27, 202519 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Bonus breaking news episode of the Deep Learning with PolyAI podcast! Hosts Damien and Nikola chat about the recent release of DeepSeek's open-source R1 model, which has sparked widespread reaction across the tech world. The R1 model, created with limited resources, has matched or exceeded benchmarks of leading models from OpenAI and Meta, prompting discussions on its potential impact on AI development and market dynamics. The episode explores DeepSeek's engineering feats, the subsequent market responses, and its implications for the future of open-source AI movements and global competition. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

NikolaThis is not strictly speaking, an algorithmic model advancement. It's not a new architecture, but the really supremely executed, you know, pirouette where different tricks were used to get to a composite thing. That's as good as anything else, which is a real feat of engineering, right

DamienChinese AI company DeepSeek has dropped its open source R1 model, and it's got the world in an uproar from massive tech sell offs to bloodthirsty chatbot arena crowds. It's hitting number one in consumer app stores, and yet it's all built on old chips and around what 7 million in capital. Welcome to the deep learning with poly AI podcast. The only podcast built on a lot more than old chips and 7 million with me as usual is Nicola Marchick, co founder and CEO of poly AI. How are you, Nicola? I'm great. How are you doing?

DamienI'm doing pretty well. Kind of doing a little bit of a breaking story here with all the hullabaloo that's going on around deep seeks, dropping of our one. First of all, kind of diving into what exactly is our one? Why is it causing such a stir? You know, where's some of the differentiators between it and other LLMs and and, and then kind of going into some of its effects that we've seen so far in these, in these, you know, seven days since its release. So without further ado, what is. Deep seek. And what is there are one model?

NikolaYeah. So, I mean, look, I think in a nutshell, it's a new open source model that has matched or even exceeded on a bunch of benchmarks models from open AI and tropic meta very small model trained on gaming GPUs where, you know, I think they claimed that the cumulative investment is something like 5 million bucks. So I think it's kind of like sent the whole world in a bit of a panic, I think Andreessen called it AI Sputnik moment, I mean, from the reactions on, on X, you know, it looks more like the moon landing.

NikolaRight. But you know, I think that through a lot of really smart engineering. They built a model that's really, really good, right? Like they use things like quantization, a mixture of expert model you know, kind of like training on paragraphs, not just on like single tokens. You know, then kind of like if we were to use analogies, it's You know, are you using your whole brain to like do every single task or training a single person to be a doctor or a lawyer, a poet, whatever, a mathematician, or do you just kind of like have different parts of brain, right?

NikolaDifferent parameters used for different tasks. And then you learn kind of like to evoke them and you create more specialized models that need. Less fewer parameters, therefore less data, less training hardware required. Cause they train for a shorter amount of time to learn faster. Right.

DamienSo, so is this, is this model when we, when we think of traditional LLMs and, and sort of the anti LLM movement towards SLMs, which was more or less taking a restricted data set, but still that. Constructed the entirety of the models training. Is this saying you still have, it still is trained on a massive amount, but for specific use cases, it's sort of like fine tuned to those use cases more effectively and efficiently.

NikolaIt's more elegant in that it's not really like a model, a set of disjoint models used for different things. It is trained that once it's just kind of like turning different parameters on and off. And it's learning to do that on its own, right? So it's still an LLM, right? It's still a large model. This model is no joke, right? It's still a very large model. It's just not as large and demanding or trained in the same way as others. Right. I think, you know, the other trick that is often used with other LLMs at inference time. So at runtime is something known as quantization, where you basically don't use say 32 bit or 64 bit floating point representation.

NikolaAnd instead you basically rounded up to like. Eight digits of a decimal digits, right? Which means that basically the computations are a bit less accurate, but still good enough right now. It's usually used. At inference time to allow it to run faster or like, you know, weaker devices, maybe in your phone, right. Or computer rather than you know, collection of hundreds of, you know, 40, 000 GPUs that are used for training. It's a form of like compression. What they use the force for training as well.

NikolaAnd they still got it to work, right. Whether it's because of this, like, you know, mixture of expert or, you know, just like multi token kind of like training procedure they use. But I think the coolest thing is they've released the whole thing. Right. So it's not the classic kind of like closed open AI model where it's like, here's the model. Here's how it performs. You know, it can validate and use it to confirm the evaluation, but you can't really see how it works under the hood. You know, this thing is out there. So, you know, I think that. All the Western AI companies, you know, I've seen Yann LeCun, who's head of research at Meta posting about it.

NikolaYou know, for them it's a moral victory because Meta was always the one pushing open source models. You know, it's a lot of people, a lot of media are framing it as like, you know, a geopolitical victory for China. I think it's a victory for open source. And a victory for, Oh,

Damien100%.

NikolaYeah.

DamienYeah. But the question is, is, and you mentioned this, they sort of released it all. Why? It doesn't seem like a very Chinese move for a nation. That's, that's notoriously protective of its IP and sort of secular, you know, insular about it. What's the play here?

NikolaIf you're asking Nicola, the Serbian conspiracy theorist, I'll disappoint you. I don't really have one. I think the only way you can interpret it is it's a real kind of like, You know, show force in that. It's like, not only are we going to show that we are on par with less resources and, you know, the scarcity breeds that innovation, but we're going to go a step further and just like show you how it's done. We're going to like release it all. And you know, I mean, if you really, you know, the Serbian conspiracy theorist is waking up it's a bit like the Belt and Road Initiative, where I think China is winning hearts and minds in large parts of the world by not like talking about rule of law and similar things, but instead just providing, you know, infrastructure.

NikolaWell, you know, what better way than to provide for free to everyone the infrastructure of the future that Also runs on anyone's laptop a lot more easily than the bulky American model. That's really, really, really expensive. And you have to pay someone to use it. Right. So it's going to get interesting. It's very interesting, right? I mean, there's a world in which I actually think that the drop in share prices of those with the most AI exposures. Actually not even that serious. Right. It's definitely a blow to like the foundational layer.

DamienYeah. Let's well, let's, let's talk about that drop in share prices. It obviously had some market effects domestically here in the states. NVIDIA is. Stock price took a tank and a massive tech sell off. It's going to bounce back.

NikolaLike it's not, they're like one thing or something like that.

DamienI mean, by scale, you know, they're, they're topping the top 15, like loss of market value, just based on their size alone, but greatest

Nikolaof all time. I think actually, yeah, exactly.

DamienDo you think these are longstanding effects? Do you think this is a gut reaction to the fear that's imposed by, you know, again, China, like dropping an open source model that's this performative on, on basically built on, you know, duct tape. Is this something that we should like, that is going to have longstanding concern or is it more of a, AI industry?

NikolaI think it's, well, I mean, I think it's a wound for adoption and like a big step towards like just, I mean, it's a step towards also multipolar world in a sense that, you know, there are multiple parties capable of these kinds of advances. I'm no Wall Street analyst, so

DamienI mean,

Nikolathey're going to, they're going to price this in however they think is suitable in the next. You know, news hype will have a different effect, right? I mean, AI is here to stay and video provides the thing that was used to train these things as well. Right. Maybe just four of them. So, you know, the house always wins except, you know, in this case, the gambler didn't come in with a lot of money and came out with a lot

Damienof winning, right? Well, you said, well, you said that it would, you know, it was reported that these were built on gaming GPUs. Is this really the house's advantage still to Nvidia or can cheaper chip makers now kind of throw their hat in the ring?

NikolaYeah, I mean, absolutely. Right. I think it really, you know, if sustainable, if we don't get like a major advance by throwing more computer, the problem you know, God knows. Right. I mean, in terms of how it was trained, I don't know if that's. Whole thing and data set was released. Altman, you know, continues to kind of say like, you know, the hardest bit is like getting to it and then, you know, imitating is a lot easier, right? It's kind of like that anecdote about the first person to run a hundred meters and under God knows how many seconds.

NikolaAnd then the next year you had like seven more people doing it, right. Cause then you, it was possible. So I still, I don't think this in any way, like reduces the achievements of previous LLMs of 4, right? Or one, but it definitely, you know, we don't know what it was trained on because if it's a distilled version or even, you know, if you had a training procedure that got there, but you know, maybe in the meantime used data or, you know, just kind of like overall meta level guidance from other work that might've done, but not disclosed.

NikolaThat used GPT 4 or other leading LLMs to guide their training process. That then would be mischievous, right? And definitely still impressive in that it's like a really, you know, it's an achievement to do it with less hardware. If it is with less hardware, in any case, they released something really good to open source. They, if you flip it on that side, then really, you know, In that game of like competing for a better model, then method is the loser, right?

NikolaCause they've done the most in the open source game to release those models and now someone else is better, but I don't think so. I think they're like, you know, they wrote the game and, you know, imitation is the greatest form of flattery. So I think this is, I think this is good for the ecosystem broadly, right. For adoption, for interest in it, for getting even more people to work, because, you know, it's, you don't just have to. Get a job at open AI or Anthropic to have the kind of resources to play the game. Right. And that's cool. Right. I mean that was always, I think the appeal of computer science, right.

NikolaThat's unlike say in physics, where if you want it to be an experimental physicist, you got to get a job at CERN or, you know, whatever, go build your own large Hadron collider. And that's pretty expensive and difficult. And you know, the appeal of computer science was always that, you someone's sitting in the room cause like think of something clever and, you know, make an algorithmic advancement. This is not strictly speaking, an algorithmic model advancement. It's not a new architecture, but the really supremely executed, you know, pirouette where different tricks were used to get to a composite thing.

NikolaThat's as good as anything else, which is a real feat of engineering, right? This is a really, really inspiring feat of engineering by those guys. No matter if they use GPT 4 or not.

DamienRight. I'm just having a hard time wrapping my head around communist nation makes massive move to democratize AI. I just, I can't, I it's, it's, I want to know like what's behind the curtain. I want to know what the next move is, what this, it, what's Was it just an industry disruption? Was it just to sort of shake up the West, the existing you know, technology powerhouses and say, Hey, not, not for, not for Meta's benefit, but kind of to spite the others instead. I'm just, it's, it's hard for me to conceptualize what the lesson they mean

Nikolaas a, as a grandson of two Yugoslav partisans. 1 of whom was actually, I think, ideologically communist and, you know, many people think that she, and especially kind of like, in these later terms really is. Moving away from a kind of like a free market capitalist China into a communist one. There is definitely a, you know, a number of profound communist beliefs in the current Chinese regime, right? One of them would be, you know, like, egalitarianism and, like, The rights for all to have the access to the same thing.

NikolaRight. So releasing this is very much in line with that. Right. And I think that, especially with it's non commercial pursuits. Yeah. I mean, like also, you know, it's hard to know exactly what's on the, on the other side, but this is a startup, right? This is not Baidu. This is not Tencent. This is not Alibaba, right? Like this is a quant hedge fund, which is not by itself a very communist thing to help. Right. So again, I think China's model is interesting.

NikolaI mean, it's like, I don't know, an 8 billion quant hedge fund. I mean, that sounds more like something like Jane street or jump or GSA than you know, Citadel Renaissance. And, you know, those places, I mean, like throughout my undergrad and PhD, you know, smartest kids were getting like quant hedge hedge fund jobs and decided to opt out of the whole thing. It wasn't easy. 'cause not only did you have to be smart, you had to be fast. Right? Right. You had to be really good at probability and maths and, you know, classical like problem solving. You know, I mean, those guys are the highest, those are the highest paid entry level jobs that exist on the planet.

NikolaRight. So it's not really a surprise. And I think, someone from Atropic wrote about. Like, Hey, look, I mean, regardless of anything, the consideration of talent, which is, you know, if you look at like the future vector of an LLM company, Talent is arguably more important than the supply of just LLMs. Well, I mean, now these guys have proven at least for this level in the race, right? For the Sputnik moment, right? So, you know, they have like serious talent. So, you know, on that, maybe this is,

Damienmaybe this was winning a race where there was no prize money, but knowing that the sponsorships are going to be coming in

Nikolathe compensation of those guys working on it, right? 200 people. There's a lot of, I mean, What would media be if not sensationalist, right? So, you know, they're like, Hey, open has 3000 people and these guys only have 200, I mean, deep might have 200 people. Maybe the last one would be a hundred, right? Like opening up until recently had like a thousand people and then fewer, right. They were already what they were when they were a few hundred people, right. And you know, the num, what's that law?

NikolaIt's a, it's a permutation of the Pareto principle, where basically if you have N people in the company. The root of n people do have to work. Right? So, you know, if you've got, you know, 400 people, 20 people to have to work, right? And then, you know, it's it's interesting, right? Because it means basically the scaling wall of adding people. And I think many people in many companies feel this has to grow. The scaling law is definitely not linear. It's definitely like sub linear and adding people actually slows you down in the first instance, because instead of doing their best work, your superstars are figuring out how to manage people.

NikolaAnd, you know, half of them realizing that they really hate it or that they it's a waste of time or not what they'd like to do. And, you know, it's just a world where those guys are just working on what they're really good at. Is a world where they're more productive initially, right?

DamienYeah, no, very good point. Hey, earlier, you mentioned Mark Andreessen calling this AI's Sputnik moment. Obviously referencing that sort of technological sleepers. And in this case for the international space race and cornerstone moments therein, is this the one, or is this the first of many with regards to Western shakeups to what's possible without those Western resources?

NikolaI mean, on an engineering level, it's a split moment and that would fire less resources, the achievement was matched on a profound, like capability level. It is not a split moment because the capabilities are not. More advanced, right. It's just more efficient. Right. So yeah, I mean, it's what it means for the future. I'm not really sure. It means that China is a force to be reckoned with and that the perceived you know, at least two to five year advantage, which many people claimed is not really there.

NikolaNot for now. Right. So the race is on, I think any companies in the West are working to, you know, compete with each other and not with these guys. So. No, there's definitely a new kid on the block, so we'll see, we'll see who wins the most grand slams here. But I think if they were all committed to open source, it would probably be. More interesting. Right. But we'll see, we'll see what it means. We'll see how other Chinese groups deliver. I mean, I, you know, going back to like my high school programming competition days, I mean, the Chinese are fiercely competitive, right?

NikolaI would not be surprised if another five groups in China now start beating these results in a matter of months. Right. And then I think you have to like hire. I mean, you know, if you were really to be a propagandist here, like you could do wonderful meme wars on behalf of CCP by talking about the American company's ecological footprint in what they're doing with spending, you know, 500 trillion or whatever, 500 billion to to train these things while, you know, these guys do pirouettes and match the performance.

NikolaSo look, I think it's fun, right? It's good for the field.

DamienYeah, definitely. You mentioned a little bit about being good for the open source movement as a whole. Is this going to redefine AGI as traditional AGI, which is more or less performance based as opposed to what open AI and Microsoft established AGI to be, which is technology that's worth a hundred billion dollars. I

Nikolamean, I don't think this decreases the value of AI at all. It might decrease. You know, the share price upticks of those companies that depended on huge upfront CapEx, right. It may get I mean, I can tell you that mind box exploded from venture capitalists who are now like, Oh, wait, maybe it's not all being disrupted. Maybe there is differentiation and, you know, like that specific modeling in the application layer. So look, I think it's it's all going to go back and forth a few times. The world is different.

NikolaWe have models and capabilities that we couldn't dream of like three or four years ago. This is all good.

DamienAll right. Any other thoughts on DeepSeek R1?

NikolaNo, I see that, you know, I was I didn't even manage to download it. I think like at the moment they've suspended registration because they can't cope with the demands. So, you know, maybe, maybe they'll get some more gaming GPUs so they can service the whole world, but we'll see.

DamienWe shall see. All right. and for the rest of you, we'll be back to our regular scheduled programming here soon, thanks for joining us on another episode of deep AI,

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode