053: Generative AI's Power Paradox. Can it prevent its own energy crisis?
20 minAudio
In this episode of Deep Learning with PolyAI, join Damien and Nikola as they kibbitz over the immense energy demands of modern data centers powering advanced AI models. They explore how tech giants like Microsoft and Google are turning to nuclear energy, including the potential of fusion power, to meet these needs. From comparing the roles of nuclear and other renewable energy sources, to the impact of data, chips, and better algorithms in AI advancements, they paint possible tableaus for the future of sustainable tech and energy-efficient solutions as AI continues to evolve. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Nikolathe needs of data center used to train a modern LLM. Is, a few months of, I don't know, the city of LA. So it's pretty serious
Damiennow.
NikolaModern data centers are about two to 3%, I think of global energy consumption and AI will probably be the lion's share of that, like in two to three years from now.
DamienEverybody and welcome to another episode of deep learning with poly AI's podcast, the only podcast that promises to hand out Brussels sprouts this Halloween. No, it's not going to happen. We want good candy. What's your favorite candy? Nicola. No Brussels sprouts. Boy, that's tough. I think I'm going to probably go with a Heath bar. I know it's not a popular favorite, but that is probably my favorite. It's toffee with chocolate on it. It's like English toffee with chocolate on it. Yeah. I see. I see. Okay. Okay. I'm not too much of a candy person now, but come Halloween and everyone's a candy person.
DamienAbsolutely. Anyway I'm I'm Damian. I'm your host. I'm here as usual with Nicola Marchick, CEO and co founder of poly AI. And today we're talking about power. Something that's scary to many but more specifically how power fuels the next generation of AI and generative AI, simply because. It uses so much. Now we've talked about this in past episodes as well, just with regards to general consumption, but there's been a lot of activity in the news lately with some of the big tech companies and how they're pursuing alternate forms of energy to keep up with the demand that this, groundbreaking technology with its Iterations and advancements is consuming with its compute needs.
DamienMicrosoft obviously just announced that they are recommissioning three mile Island. To start under constellation energy's ownership to start doing some fusion power for for their AI needs. Sam Altman has founded our, he's a chairperson for a chairman for a company that is a Helion energy that is fusion based. And it's got like a 4 billion market cap. And then Google has worked with Kairos energy to promise a handful of of reactors here in the States.
DamienSo obviously there's a lot of leaning towards nuclear. Why are so many tech companies looking at nuclear as opposed to some of the other more. popular or iterative energy especially on the renewable front energy solutions out there.
NikolaWell, I think first, if you just look at the power of considerations we're building bigger and bigger models well, they take a lot more energy and data to train, build, et cetera. And the main hypothesis that proved that, ushered in the era of LLMs is that, bigger models trained for longer and more data. Now that's kind of like played out, right? So I think the hypothesis that everyone is like thinking about the future and just say, Hey, can we jack it up further? Can we have more data and, train for longer, more chips and see where it leads us.
NikolaSo the needs of data center used to train a modern LLM. Is, a few months of, I don't know, the city of LA. So it's pretty serious
Damiennow.
NikolaModern data centers are about two to 3%, I think of global energy consumption and AI will probably be the lion's share of that, like in two to three years from now. I'm not actually checking the numbers, but at this rate, it has to be. And when you think about that, it's okay, well, if you think that you're competitive advantage, if you're Microsoft or any of the other big guys you're like, okay what do I need to fund the war effort? Well, I need to like, have data. They all have a lot of data and they always work on having more and they synthesize and create a lot of data that they need to build the next generation of these things.
NikolaSo data on the one hand, they already have a data advantage. These few companies, then you're like, okay do I have the chips? And that's why NVIDIA skyrocketed. They're currently a monopoly. And in the foreseeable future I think that they'll stay there. I was recently at a dinner where the CEO of an AI company said that he thinks that the only person who can stop AGI from arriving in about three to five years is Xi Jinping invading Taiwan. That otherwise everything is already there, right? The hardware's coming, the data's there or being created.
NikolaAnd, we have the algorithms, I don't know. I thought it was very, so just attack
Damienthe supply line there. It's I attacked the production.
NikolaWell, if he were to do that, it would disrupt it. It would just delay it. All right. Again, like we've talked about it many times what is AGI? How do you define it? What does it mean? Different discussion, non scientifically well founded, infinitely fun. Yeah, no doubt. For those of for
Damienthose of you not in the know, artificial general intelligence, check the episode couple episodes back, give it a watch and then come back and pick up.
NikolaYeah, absolutely. But look, I think that in that cold war, I know funds. I know that a lot of the large, like Holtzman is like in the spotlight, so we hear about all this, but I think that many of the large public tech company CEOs have been buying up land. that will be used for data centers quite well in advance. And it's like a thing that it's like a fundamental part of the puzzle for, making greater use of AI. It's also what kind of data center and worse, because if you have these clusters and you've organized them in a certain place, you also need like concentrated power.
NikolaSo another interesting data point is that most of the energy used for AI today, comes from non renewables. It comes right. Coal gas, coal gas, oil, right? Yeah. Carbon, and I know just looking in prep for this one, you and I just looked France is, I think the only country which has the majority. Of its energy in nuclear, I think a lot of people, as they just, as tech people do, right? Like we think immediately, like, how do you scale it to infinity?
NikolaWell, nuclear is something that we can very predictably and reliably just build more reactors, place them somewhere. Hopefully they're good forever. And we'll have as much energy as we need. But obviously nuclear,
Damienbut obviously nuclear has a little bit of a bad rep. You don't hear about solar accidents or solar panels causing, the sort of devastation that fall out from like Chernobyl's and the three mile islands have. And so that's just something where it seems like it's got a bit of a marketing problem. Why is it that even so much as to say that right now, nuclear power in the media, in the headlines is being referred to as fusion energy, because it's the same, it's the same process. It just maybe has a, eschewing the old rebrand, right? A total rebrand. So why is it maybe not why, but does this sudden approach into a previously like somewhat taboo energy production system?
DamienI don't think it's
Nikolajust it depends on where you look. America, what do we say? I think Close to 20 percent of us energy comes from nuclear. So I don't think it's not out there like
Damienscreaming it from the mountaintops. Again, it's got a consumer brand issue. It's got a consumer identity issue.
NikolaPerhaps. I think that it really depends on where you look. Okay. We're in the, like the U S is in the double digits. The UK is France at 70% South Korea at 30. So, I think it's just about what you have available, right? Sure.
DamienSure. I guess I'm just saying there's a lot of, there's a lot of efforts on the, from the eco conscious perspective of, preservation and renewable energies, solar, wind, water, all these like hydroelectric. It's just, that seems to be the direction. If nuclear is the power de jure largely driven by data center production needs, et cetera. Does that put less eco conscious nations like China, India On a more competitive standpoint from other nations, we've been a bit
Nikolahamstrung. I don't think so. Not, yeah, we can play a grand strategy game and say, maybe, but the truth is the bottleneck is first and foremost, the hardware, and they're like, that's like foundational and most like important bit right now. The second one is data. I think we've discussed previously how like a China could. Have a better grip over more of this data in one place and concentrate that effort over the West. But equally, I think that we have these very large companies in the West already have formidable data advantages.
NikolaAnd I think more advanced know how in terms of how to create synthetic data, although there, I don't think China is as far behind as it is in chip making. I think in terms of energy, well, these are well understood projects, right? So if that were the only bottleneck, I think that. Each maybe not even large, like modern economy would know how to reinvest there. So there, I think if you don't have nuclear as an option or you're unwilling to, my, my father in law will tell you that the Netherlands is insane for just stopping like natural gas production, which it has like ample amounts.
NikolaI think the UK is doing something similar where Even that most, I think, well, the cleanest form of carbon is, has a bad rep. And I think that's purely a political issue. If it became survival to have, well, I dunno, Germany in the wake of the Russo Ukrainian war, like they've not turned on their nukes. In fact, they were notorious for turning them off as the whole thing was happening. And they saw that the energy production in Germany will. Well, the equation is like irreversibly ruined by having more expensive energy, right?
NikolaSo, I think that largely today still like Europe is heated over the winter through like LNG terminals from, America and the Middle East. So I don't know. It's interesting, right? I think that If it's two to three percent of our overall consumption and it's likely to grow, nuclear is an easy way to go about fixing it. But I don't think we're at a bottleneck yet. I just think that, these infrastructure investments are being made and people make a lot of money. It did me at the time. Well, you've talked,
DamienSo you've talked a little bit about, Hey, this is one piece of the equation, right? When it comes to being competitive in the space, the other is hardware. It's the chips, the, and so pushing those aside, is there any other advancements in hardware software that you think could help to tackle this potential Energy crisis that could erupt by increased demand aside from the energy production piece. Do you think there's anything else that could be done from a hardware and software perspective to help to refine the energy needs of and the computing needs of generative right now?
NikolaAbsolutely. Look the whole motion of trading with more data and that being the main way that we produce technological advancement. Like it's the most non elegant thing that exists, right? It is brute force in its purest form. And counter to that would be like, can we have algorithmic advancements, better models, models that learn faster, that, generalize faster, that use data more effectively that, you're basically upping that learning rate.
NikolaHow can you learn faster? Transformers were the last meaningful thing that really changed the course of deep learning, right? Like the. We eradicate the need to have like recurrent models and, more complicated cells. And it was fantastic that they worked or they did. So now we're at a point where, I would think, especially when you look at deal one model, like that's being further now with like hundreds, annotators who are creating examples of.
NikolaData conversations and just not necessarily conversations, but creating data in the right format at large scale for these models to learn from, right? The latest estimate is that we've got a thing that's probably got an IQ of one 20, which is pretty serious. That's one deviation above average, right? That's smarter than what like 70 percent of the people on the planet or something, but. Is that right? Yeah. Something like that. So, when we think about that, it's okay, like advancements might be able to come just from like better data, and if we don't have data structured in the right way, well, then we're not going to get better things.
NikolaLike the natural point of just like seeing more data, like it will flatten the curve. Right now, the bet you can make is can we do it exponentially yet again? And I think a lot of people think that the current LLM Without transformer would have just come five, 10, 20 years later when the technology was just so much bigger. And the first like deep learning advancements modern deep learning advancements and 2012 onwards happened because, we figured out how to train things on GPUs and to really just crank a lot more like.
NikolaIterations through the multi layer perceptron. I think that's like 1967, right? I remember my, first, my bachelor's thesis and stuff. I remember support vector machines and Gaussian processes and, more hacky things like random forests. And it was quite diverse statistical machine learning back then. There were a lot of things and, if it was a computer game, you'd be choosing like. How mathematical do you want to be like, here's like a Gaussian process, which will give you decisions based on data. It's trained on that are very explainable.
NikolaYou'll be able to break down the kernel of that process and see kind of like which factors, which features contributed to the decision that was made. So I remember predicting like diabetes and it was like, well, weight matters and age matters, a few other things. And that was interesting, but it was really, Weak because it wasn't a black box. It wasn't there to just get maximum performance. It was like forcing itself to reason the way that a human would. So rather than just being predictive, it was trying to like really build a model that is like an interpretable model of why something works
Damienand more rational.
NikolaWe've thrown that into the fire, right? They're at the other end. We're like, these parameters, we don't know what they mean. They're a bunch of like sequential nonlinear mathematical operations. Produce text and or they make decisions if they're trained as classifiers or whatever, running the internet, if you're Google. So yeah, I think that we should really think about, well, we should, we don't like, we're just focused on solving our problems, but when we think about it, like it's really this like mix of like better data. Better algorithms and better hardware, right?
NikolaHardware really matters if you're going to consume a lot of data, right? So I think, for us internally, when we think about our datasets, they're crazily large from the perspective of five, 10 years ago. But they're not that large compared to what these people shoving nuclear reactors and, LA sized data centers powering their training for months, right? So, I think the best thing we can do for the environment is not do this. Of course, we're humans and we're capitalists. It's going to be done. We compete, it will be done.
NikolaSo then until the shareholder stops believing, which
Damienwill never,
Nikolanever, I think, I think it's very likely that if you saw like a flat lining of improvement and, you work the gardener, right? Like the trough of the delusion men will come and people, but I think that while we see such shifts into Models that can do a lot more than we ever believed was possible. People will be building data centers to jack it up, right?
DamienWell, one of those things then, and this ushers in the power paradox is can the reasoning and the advanced deep learning and the neural networks that are being demonstrated in current models, can they basically, and obviously AI, one of the chief applications for generative AI is in software development and coding. Can basically it train itself to be as efficient as possible? And if not, can. It basically map out the most efficient from a physical perspective, the most efficient energy grid and sourcing that can be done to power it's ever growing needs.
NikolaLook, I think that can AI be used to improve designs of these things for sure. Can it do it itself? That's the eternal question of like singularity. If you're imagining like. True. Skynet sitting there improving itself. Sure. But, it probably go after the French nuclear reactors and the American nuclear reactors first. Because that would claim the land and invade Taiwan and GPUs. There you go. Those would be, it's for you. Jinping.
DamienYou lost your chance.
NikolaOh, yeah. But I, I think for real though, it's maybe. But it's still going to have a natural physical limitation. And if it's that demanding, it will spread geographically, it will spread, vertically, horizontally, and in every other way it can. So, scaling it by. Producing more energy and more chips and, that whole supply chain all the way from like real estate to like the raw materials all the way to like the land that will be built on and right against the regulatory environment, which allows it to do all that.
NikolaYeah. Like we're doing it. Right now as a civilization, right?
DamienSo course the course is the course until it gets proven. Otherwise, is that what I'm hearing from you?
NikolaWell, look, I think that we consume more and more technology. AI is the conduit. It's like the new thing that everything will be built on. So will that 3 percent of energy going into yeah. Training and stuff increase. Oh, for sure. Yeah. I think where will it like reach? Look, as you think about hopefully a new age of abundance where we will just, you know, watch Netflix and play computer games and pontificate, albeit in ways inferior to modern AI we'll probably consume a lot more entertainment, right? And if you think about what drove, you To produce the GPUs we have today.
NikolaThat a lot for AI it's gaming, right? What was your first GPU? Do you remember?
DamienMy first GPU, I think it was a Texas instruments. Oh boy. I couldn't tell you the model. It was basically for word processing.
NikolaOkay. Okay. Mine was a, a G force too. I'm trying to see I think it was an MX 400, but yeah, this is ah, wow, you can buy them for a hundred, a hundred pounds. This is basically yeah, but that's like a lot smaller and weaker than their modern variants, but like they were part of getting there. So I think again even if, the. The drive of the tech industry to produce better LLMs, to give them a competitive advantage over others. Even if that thing calms down a little, we'll still be doing entertainment, right?
NikolaAnd we'll be producing. Yeah, I guess that's true. I guess that's true.
DamienSo, so maybe the energy isn't going towards other facilitated production, but it's going towards us feeding our natural need to always be entertained. And maybe we do have more of a Wally type future at that point, or we're just going around our own personal entertainment devices.
NikolaAbsolutely. So I think we'll need a lot of energy for whatever we do. So it's probably not wasted. I don't know enough about the distribution storage and all that, but, if you think about just draw electricity production and like EVs, well, it's not a bad thing that we're doing all this. Right.
DamienAll right. So we basically find another use for the energy that we end up saving by becoming more efficient and we end up staying pretty much flat with regards to consumption and it spikes anyway, right? So we're, we're building our own failure because of
Nikolawhatever's still left. You use it to, train GPT 11 or you mine Bitcoin, right? So
Damienyeah, very, very true. Boy, that's another episode. All right, cool. Well, thanks very much, Nikola. This was awesome to talk about. I love painting these futures with you. It's always interesting to see where we end up. And this time it was, what was your first GPU? Which is awesome. Now I'm going, I'm thinking back waxing historic. I appreciate your time. A pleasure as always. And we'll talk to you next week. Okay. Yeah, have a good one. All right. Take care
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...