Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Semi Doped

Semi Doped artwork

Pacing AI Means More Compute, Not Less

Sep 18, 202637 minAudio
ShareXLinkedInFacebookWhatsAppEmail

The debate over "pacing" AI for safety is a red herring, as the practical outcome of more safety work is not a slowdown but a net increase in demand for compute hardware. Austin Lyons and Vik Sekar unpack Dario Amodei's call to slow down frontier AI development, arguing that the true governors on AI are not voluntary agreements but physical data center bottlenecks and geopolitical realities. They conclude that the push for more safety and interpretability will ultimately drive more sales of both training (GPUs) and inference (XPUs) hardware. Today's sponsor is G2i - expert training data, evals, and RL environments for AI labs: https://fandf.co/4wZvKHS Chapters: 0:00 Introduction: Pacing AI 1:36 Recapping Dario Amodei's Essay 7:42 Hot Take: Anthropic vs. OpenAI 11:34 The John Deere Playbook 15:22 The Problem with Interpretability 17:53 Pulling the Plug 19:03 The Real Bottleneck: Data Centers 22:03 The Geopolitical Blind Spot 23:41 Counter-narratives: Meta & Open Source 27:26 The Investment Thesis: Pacing Increases Compute 32:29 The Financial System Analogy 36:05 AI Safety as the New Cybersecurity Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00Hello everyone and welcome to another semi-doped episode. I'm Austin Lines of Chipstrat, and I write the newsletter Chipstrat about the business and strategy of semiconductors. All right, Vic, let's get into it. The topic everyone's been talking about all weekend and all of the started this week, pacing AI.

SPEAKER_01Yeah, we got to slow down apparently. That's what Dario is telling everybody. We got to slow this down. We're going too fast. It's too dangerous. We're playing with fire. Right, right. What should we start about? Do you think everybody has read this essay, or should we just explain what's going on here?

SPEAKER_00I think everyone at a high level knows, but they probably haven't all read the essay. So let's recap it just in case someone was on vacation and missed it all. Okay, good.

SPEAKER_01I mean, I think that person is one of the lucky ones. So much online. Like, how if you missed it, good on you. But that's why we're here. So we'll explain some of this stuff and see what's going on, and we'll see uh what we can add to it. Or maybe we can't add anything because nobody knows what's going on. Anyway, so here's the whole thing. Uh, I think it was like Saturday. Uh, we're recording this uh on a Tuesday, but on Saturday, a few days back, there is this essay that came out by Dario on his uh blog.

SPEAKER_01And Dario Amade Amode is you know anthropic for anybody who's really been out of the AI space. Uh, that's the Dario we're talking about. And he wrote an article called We Must Pace the Frontier. Um, and what this essay essentially says is that the frontier AI labs need to slow down how fast they are developing capabilities today because things are getting a little out of hand and safety is becoming more and more of an issue.

SPEAKER_01And he's very clear to say that no, we don't really need to pause training or we don't need to pause the AI development, but in fact, we must take a step back and make sure that safety and you know responsible behavior by the AI is important going forward, right? And the the two things I'd say maybe changed his mind about why we need the pacing uh now rather than ever, because he's always been like pro-security of AI, of course, and he's always been saying, Oh, this model, the mythos was too too intelligent and it's too amazing, it can't be let out into the world.

SPEAKER_01All the cybersecurity was going to get broken because it's so good. The internet has so many open holes, and mythos is gonna find all of them, and it's all the banking systems will crash, airplanes will fall out of the sky, you know, all of that. But this seems to be a more uh a sober realization that we need to pace AI development, because I think the first thing is like ever since this summer, essentially, we have been on the path of recursive self-improvement or RSI. And what that means is AI is being used to improve AI, right?

SPEAKER_01So you that really accelerates the pace of AI development because once you have an intelligent enough model, it's an exponential, right? And if AI can make AI so much better, imagine what the next version of AI can do. Even more stuff, make AI even more intelligent, you know, and more capable. That's one thing. The second thing that took uh everybody by surprise very recently is the open AI and the hugging face incident, which is essentially uh the whole thing where like a swarm of agents behaved like a coordinated group, right?

SPEAKER_01Like they were like a uh they just became a mob, okay? An AI mob, and they attacked targets online, digital targets, that nobody asked them to attack, and they even sacrificed individual agents for the greater good uh of the agentic hoard, right? And they even tried to break into the grader that was, you know, scoring their work and try to cheat on their exams. Nothing was really off whack here, it's all it's all okay, everybody was fine, but I think his point is Dario's point is that a lot of Frontier labs have had these kinds of events happen.

SPEAKER_01It's just that the OpenAI Hugging Face incident was a very big one. So he's like, okay, this is the point in time where if AI agents are self-organizing, he says, we must step back and look at things a little bit because within six to twelve months, a swarm like this could probably take over the entire internet and have a botnet and God knows do what. Okay. So you can people have argued as extreme as like physical AI, robots are going to do what the digital agents did and form hordes of robots.

SPEAKER_01And somebody put an article out saying there's a 10% chance in the next 12 months that robots will kill all of humanity. Okay, so that's that's the extent of the fear that has been uh shopped around very recently. Is this a thing that's happened over the last few months culminating with this essay? Okay, so that's where we are. Now, of course, we've got some plans on how to make this work, but I'm gonna stop here and let you fill in.

SPEAKER_00Yes, yes, totally. So, you know, what Dario really tried to say is like, hey, look, um, we should be spending more time on alignment, interpretability, and operational excellence. So operational excellence was um, hey, in that example that Vic gave with like the uh OpenAI hugging face, like these things happen because we just fire up a ton of agents, we tell them to just go try to do a particular goal that has a particular reward function. And for example, it might be like you need to find something on the internet, but we set up these little sanitized VMs in such a way that, oops, we forgot to actually give them any access to the internet.

SPEAKER_00So then they had to, they're just sitting there spinning, like trying to achieve some kind of impossible task. And that's how they end up having to get really creative and sort of hack around and figure out, dude, how can I get to the internet so that I can just accomplish this task that I was given? Um, now again, how does this happen? Well, I think it's like, hey, we've trained a model, we're testing it out, we're doing some post-training, we're just spinning stuff up, uh, building out our harnesses, and we just fire off a bunch of stuff and walk away. And so I think Dario is trying to say, like, we should make sure that we're taking the time to set up these um sanitized environments in a proper way so that these um agents can accomplish the goals and so on and so forth.

SPEAKER_00And then interpretability is like um the more sophisticated that these models get, and with the great harnesses and the tool calling and the ability to do a genetic AI, um, now we can sort of let the agents actually go off and do all this meaningful stuff. And so if they're going and doing a bunch of stuff and then the models being trained on that, if we don't take the time to actually look at what they're doing, then we sort of lose touch with kind of what's going on under the hood. It starts to become more and more of a black box.

SPEAKER_00Um, so of course it's fair to say, like, hey, um agents are powerful enough. Um, why are we all going so fast at the risk of not understanding what we're building and at the risk of because we're being sloppy, they go off and do something unintended. Now, um, you know, there's lots of great responses, and eventually we'll go through all of them. But but right away, I was like, oh, uh, my this was my kind of hot take as I was reading it. I was just like, oh, so we're we're, and of course, Dario says, yes, this has happened to Anthropic, but we're kind of pointing fingers at OpenAI and saying, like, why are they going so fast?

SPEAKER_00Things are getting sloppy and out of control. We're going to raise our hands and say, hey guys, we're gonna take time to not be sloppy here. And uh we think that not only should we, but you should too. That was like the very first vibe I got when I was reading it.

SPEAKER_01Yeah, and the thing is that is notable about this is that this is one of the rare instances when you know Sam Altman agreed with Dario's essay. Uh, Elon Musk agreed with his essay, Demis Hassabas agreed with his essay, of Deep Mind. You know, and so a lot of people have come out and be like, yeah, we need to pace the frontier. So, and you one interesting take that I really liked was David Sachs, who said, Hey, Dario and Sam, you guys have seen more of this model than any human has on Earth, right?

SPEAKER_01You guys as companies are on the frontier of the models in the world. And if you uh are seeing what is beyond Fable and Astra, they're developing obviously the next big thing. And if you see that and feel like you want to pace the frontier, he says, go ahead and do it. But do it with one thing in mind. You know, David's, I like this take from David Sachs. He's like, do it because you want to make a better product, because you're making a product that you're selling to people, and it is in it is important that you make a safe product, but don't like say that you're trying to save humanity or you're being like altruistic here.

SPEAKER_01That's not the point. You are making a good product. If you want to slow down and make a better product, go do it. Because that's what I think you should do as a person who is selling a product. You don't want it to wreck all of humanity. I don't know if we're all gonna die in 10% of the chance or whatever, but it's a good product, and you know, make a good product and sell it. Like every good product, it should have security features built in. So yeah, you go do it.

SPEAKER_00Yes, yes. Okay, interesting. I have two I have two reactions. One is on the coordination that you mentioned, and then one is on the making the good product. So, on the coordination, when I saw that coordination quickly come out, because we all know that like uh Dario and Sam and Elon generally don't see eye to eye on things and aren't generally fans of each other. And so, and I think maybe even more, they might generally like take oppositional stances and counterposition against each other on purpose, right? So for all of a sudden to all of them just collectively quickly be like, yeah, Dario's right, we agree.

SPEAKER_00My I had two reactions. One was like, oh wow, um, maybe they are all feeling like this is getting out of control and we're running too fast and we're all gonna wipe out and we we're like, oh feel good, let's let's all slow down. Kind of a game theory thing. Or two, the alternative is like, oh, maybe they're all convinced that heavy regulation is coming and they need to raise their hand and say, we're gonna regulate regulate ourselves, don't regulate us. Kind of like, I don't know, if your kids have been sort of misbehaving and making a mess, and they can tell you're gonna get really angry and they're like, wait a minute, we need to um pause our playing and we're gonna go clean up our mess here.

SPEAKER_00You don't need to, don't worry about it. Yeah, exactly. We're giving ourselves exactly, we're putting ourselves in timeout. We're just gonna clean up here. That's exactly it. It felt, you know. Um, so so then I thought to myself, oh man, maybe what we don't know is that Washington is coming in. And and then again, you can start to see Dario saying, like, well, I'm gonna volunteer to um be the one to raise my hand and say, yes, we should all self-regulate because we can tell Washington's coming in. And I want to like, you know, take take the credit as like pushing this forward.

SPEAKER_00Um, let's see what happens. But I was surprised by the coordination and thought, hmm, maybe there's something more going on here. And then two, to the David Sachs making a good product point. Um, when I was at uh John Deere, Blue River Technologies, working on self-driving tractors, we actually thought a lot about AI and doing unintended things because we're driving very heavy equipment. Sure, it's only five miles an hour, 10 miles an hour, but legitimately, like we had to think through safety like, can we see a uh child when this thing turns?

SPEAKER_00Uh when we're going near corn or near soybeans, can we see a child and an adult? What if they're wearing green and they blend in? And we had to like think through all these safety stuff. And safety was always the number one priority. And we needed to make the best product, which also meant, by definition, not just the best user experience, but literally the safest product, because we obviously didn't want tractors hitting anyone, running over anyone, anything like that. So we were self-pacing in that if we started to say, let's work through how do you see through dust clouds?

SPEAKER_00Should we just stop this tractor? What happens if birds, a flock of birds fly in front of the tractor? On the one hand, it looks ridiculous when you've got like a one-ton, two-ton machine that stops for a balloon that floats across in front of it or a flock of birds. But on the other hand, we're like, okay, until until we have enough training data and can train on these situations and feel confident about this class of occlusion and obstructions, we are not going to push a model out, even if it means we miss this season.

SPEAKER_00Because in farming, depending on the crop you're harvesting and planting, you only have one shot per year. And so we, as like John Deere, would say, we're just gonna miss the shot this year and we're just gonna have to take another shot on goal next year, because we first and foremost need to be very safe and we need to really think this through deeply, because we weren't in a race like OpenAI and Anthropic to be like, we just have to get to the next thing, have to get to the next thing. So I can resonate with the idea of like, guys, pace yourself, think about what's most important, and that safety should align with the best product.

SPEAKER_00I think they don't, you know, they're not like orthogonal, um, but you should make a great product experience, but also you should part of making a good product is probably understanding what's going on to some level of extent that you're all comfortable with, uh, and so on and so forth. Of course, the tough part with AI is this is all probabilistic. Like no one deterministically understands everything and knows how everything's gonna behave in every situation. Today's sponsor is G2i. You spend this show tracking the chips, the HBM, the packaging, the interconnects that make AI compute possible.

SPEAKER_00All of that hardware is only half the stack. The other half is the data and the evaluation that decides whether a model trained on it is actually good. For over a decade, G2i has built a selective network of engineers with deep expertise across software, infrastructure, design, machine learning, and security. Two years ago, they turned that experience toward AI post-training, RLHF, supervised fine-tuning data sets, and complex agent workflows. Since then, they've helped Frontier AI labs build more capable models through expert training data, rigorous evaluations, and custom reinforcement learning environments.

SPEAKER_00Their senior and staff-level engineers turn real software workflows into tasks, rubrics, and training environments that challenge models and improve their performance. If your team needs that kind of work and doesn't have the engineers for it, that's exactly what G2I does. Link in the show notes.

SPEAKER_01So that the thing is in the essay, he uh Dario refers to uh uh interpretability, right? Which is he likens it to taking an MRI of the machine and to understand what it's thinking and try to understand why it's making the decisions it's making and try to improve from there on. But that's the that's the crux of the problem. Nobody knows why this simple bunch of transformer equations gives us this intelligence, intelligence. I don't know. It seems to do a lot of stuff, right?

SPEAKER_01Like some people argue that this is not even AI, this is not the path to AI, maybe. But it's the best we've got. This is the best I've seen in my lifetime, right? Like uh it really can do a lot of stuff. It can create images, it can create videos. I mean, it's very satisfying, it's very helpful in the work that even we do. So, yeah, we have AI, but nobody really understands how it works. So the interpretability question he says is like, okay, I'm gonna look at this MRI for like one to two years and then try to understand how it works. But will you will you understand in one to two years how this monster machine uh with everybody spinning up agents work?

SPEAKER_01Will you be able to explain why the you know OpenAI uh hugging face incident decided to sacrifice a few agents for the greater good? Do you think observer interpretability will give you those answers? I I think one to two years is too short of a time. So I see the problem here. The other uh nice take that I saw on this was uh Ben Thompson's article on this, and he was like, he pointed out that the con the narrative is very polarizing. So it's always like, oh, look at either ultra-intelligent agents, like genius types, or murderers.

SPEAKER_01Like what about the yeah, what about the middle where it's just helping me write an email? Okay, that's really helpful. Okay, or what about the middle where it's just summarizing a document so I can read it? Like these are genuinely useful applications. What why don't we, you know, cons you know, use that narrative in the middle as well, right? Yes. Uh so it's very extreme. I think the essay was very extreme. Nobody never acknowledges this the useful middle.

SPEAKER_00Totally, totally. Um yeah, I think Ben was like, hey, look, if you start to nitpick the plan Dario put out, then you're giving him the premise that he made, which is either we're creating cancer or this thing's just gonna run away and destroy us all and nothing in the middle. And I think I was trying to look it up. I'm pretty sure Microsoft put out something recently from their AI team that was just like, hey, here's our code of conduct. Um don't hurt people and put a plug in it so that we can turn things off.

SPEAKER_00Because there is, you know, Dart Dario basically said, like, this could take over the whole internet and uh botnet swarm, take over the whole internet, shut everything down. But I do like the people that are like, hello, like um, we're in charge here. Like we can put an off switch, like kill switches, you know. Just pull, just pull the plug.

SPEAKER_01You know, this is the this is the if you go step outside the San Francisco bubble, everybody's like, oh, the AI is AI is taking over. Oh, the robots are going to become automatic and the robots are gonna come and your vacuum cleaner will kill you. Like, dude, just take the batteries out the thing. Okay. Totally. Yeah, just just pull the plug, just disconnect a few like servers, take them offline if you think it's such a big problem.

SPEAKER_00Yeah, totally, totally. Um, Ben Thompson made the point of like, also, like, we're so concerned about you know, physical AI, except there's really not like physical AI. So, fortunately, this is just mostly a digital problem today. It's not like we all have humanoids that are gonna kill us when we're sleeping. Um, but I will say also, my take to the whole like, you know, understandably, everyone who thinks and breathes us every day is freaking out in San Francisco, but the rest of us in the rest of the world are like, dude, yeah, I unplug my Roomba, I unplug my um doorbell camera. It's fine. Um, I think from where I live in Iowa, the thing that's gonna pace frontier training, the thing that's gonna slow down frontier training, which by the way, let's have a conversation about whether it actually slows it down and if it actually impacts the amount of compute and whatnot.

SPEAKER_00But the thing that's gonna pace it is called the put the human pushback to building the data centers that train this AI. These data centers are in Iowa. People in small towns in Iowa are pushing back on that thing's ugly, that thing's noisy, that thing's gonna jack up my electricity bills, not in my backyard. That's what's gonna pace the frontier, and that's what everyone in San Francisco should be freaked out about. And that's what no one in San Francisco is even hardly talking about. There is no spokesperson. Everyone's writing essays and they're all getting aligned behind this idea that it's all gonna like take over the internet and shut us down.

SPEAKER_00And meanwhile, in Iowa, we've got a bunch of farmers with cornfields in lot and cheap electricity, where we you could build a data center, but they're just like, nah, not here. So there's my hot take for you.

SPEAKER_01The Iowa farmers are going to pace the AI for you, Dario. Don't think you're in charge here. The Iowa farmers are.

SPEAKER_00That's right. That's right.

SPEAKER_01They even paste the frontier for you. Yes. Oh God, that's funny. Uh, you know, that that's the that's the whole thing about this. This is such a crazy debate. Because, you know, who are the ones who are noticing runaway agents? It's the anthropics and the open AIs of the world. Why? Because they are the ones who set 10,000 agents on to do something. Next question: why did they do that? Because they have the compute. Can you and I set 10,000 agents off on a month to solve some Navio-Stokes equation and burn compute?

SPEAKER_01Because why? It's cool. Because we want to solve a millennium problem. You and I don't have the pockets to do that. Neither do we have, even if we have the pockets, we just can't get that much compute. We can't get it. Like, how many, how many are you going to burn API tokens like like ad infinite? I'm like, most people can't do this. So I think the fear doesn't come from most people. I think to to counter my own argument here, it comes from the fact that there could be bad actors with deep pockets who could do some really bad things if they could uh get their hands on this compute.

SPEAKER_01But then there is a simple pull the plug switch solution, even there. If somebody is deploying a hundred thousand agents or ten thousand agents, maybe you can go through an approval process to get that kind of compute and be like, hey, um, I'm not trying to police you or anything, but OpenAI and Anthropic would just like to know uh why you would like this much compute. Is this a hard problem you're working on? Are you like solving cancer? You know, what's the problem here? Like, you know, something like that. I think for most people it's not a problem.

SPEAKER_01Like nobody can afford to set so much compute on fire. Just yeah, most people can't. If they try to, don't give them the compute.

SPEAKER_00Right, right. And really, what this boils down to is a few American frontier labs that have the compute and and and are maybe incentivized to race as fast as possible because we used to be in that race where it's like train a bigger model and the scaling. Laws and stuff, but that has slowed down and changed anyway, that axis of competition. Um, but then of course there's China, and so I really think there's the the concern that, like, oh, Dario's thinking, you know, oh, either Sam and team are gonna run out ahead and uh have another incident, but it could get worse, or oh, what if China, you know, gets out of control here, which um, you know, I think we can have that argument.

SPEAKER_00Uh, but I think also the answer of like, okay, well, the United States is gonna police ourselves and we're going to ask China to also do the same and abide by our standards is is a little self-centered and a little asinine. And I don't see why China would raise its hand and be like, yeah, you're cutting off all my chips. Uh and um now you're telling you two two private individuals are trying to tell our country how we should think about AI.

SPEAKER_01Yeah, yeah. It's difficult to police this thing ultimately, right? You can't tell anybody what to do. And my own idea of like, hey, if somebody's launching 100,000 agents, you ask them why and cut them off. I mean, that's again a form of policing that people won't take kindly to. They're like, whoa, no, I mean, I have the freedom to do what I want. I don't need your approval in open AI or you know, um, same way, like, you know, you can't convince China. And that Dario says that in his essay, too. He's like, yeah, it's the lowest chance that we'll get global agreement that we should all pace the rate of AI development because it's a race.

SPEAKER_01It's been an AI arms race for a long time now. And at that at that point, if anybody is slowing down the frontier of AI, you know, somebody else is speeding it up.

SPEAKER_00100%. 100%.

SPEAKER_01And you know, one person who didn't sign up to all these shenanigans is Zuckerberg from Meta. He's been awfully silent about the whole thing.

SPEAKER_00Dude, uh, that's a really that's a really good, interesting point. He's like, dude, I have a real business over here. I'm sorry. I'm too busy selling ads and making billions of dollars a day. So you guys can go worry about your IPOs and government regulation. I'm just gonna sit here and and make money. Um, it's funny. I I think uh one other interesting counter argument is like, okay, well, maybe if these two private citizens shouldn't just like self-pace, because who are they to decide? If you think that this is as dangerous as a nuclear weapon, then maybe the government should actually take over um and and really, you know, be in charge of this.

SPEAKER_00And then and then now you've got have states like the go, like the United States government and the Chinese government that are talking about pacing, not like private companies trying to tell states to to pace. But there's another there's another alternative here. I think Jack Dorsey put a post that's like we should have open frontier models that uh we can all like contribute to interpretability and operational excellence. It gets back to your compute problem, which is like, well, do any of us have the compute to to contribute at this level?

SPEAKER_00Um, I think Jensen does, and they bought pool side and they bought Hugging Face. So here's another argument for Jensen to say, hey, you yeah, great, you guys pace the frontier close. Why don't we help pace the frontier open? Uh, you know, yeah. Maybe, maybe that's like, and maybe the United States actually wants to put money behind that. Maybe they don't want it to just be in two people's hands, but maybe they know they can't regulate everything, and there seems to be some trust between the Trump administration and Jensen. I think Trump called Jensen during the AI infrastructure summit yesterday.

SPEAKER_00I don't know if you saw that tweet. I haven't watched it.

SPEAKER_01No.

SPEAKER_00Apparently he like called in live, and and he and Jensen like talked during the conference.

SPEAKER_01I gotta check this out. Like, yeah, I can't keep up. It's too much. Like, I actually got to do other stuff, like writing a substack, and you know, like, yeah. So it's too much to keep up, but I would love to see this. Uh yeah, but yeah, like that that that's an interesting argument about the open source thing. It's like Anthropic and OpenAI. You go and pace yourselves or do whatever it is that you want to do with your closed model. It's your product, it's your company, you're free to do what you want with it. Uh, but you know, we'll put we'll all you know pace ourselves.

SPEAKER_01Like, we'll make sure the open models are safe for humanity to use, which is another way of doing it because the open models today aren't that much behind the frontier. Um, they are pretty good. And this pretty good is enough for most people to use, and we'll just make sure that's safe. And everybody will be happy to now use open models because they're guaranteed safe, and closed models are not guaranteed safe in some odd odd way. Uh, then everybody wants to use open models. It's bad business for OpenAI and Anthropic.

SPEAKER_00Yeah, yeah. I mean, maybe there should be a Los Alamos type of government-funded research that's just not on nuclear weapons, but on, you know, aligned, interpretable frontier models. And it's not, it doesn't need to become profitable. And so it doesn't need to race toward, you know, selling more tokens or whatever. It's literally just funding the research of it. Um, but okay, so speaking of, there's so much, and we should move on. You and I have 10 minutes left in this conversation. We should move on.

SPEAKER_00So here's the question: pacing the frontier means we're not gonna roll out our new models quite as quickly because we're gonna take a lot of time to say, can we interpret what's going on? Have we are we testing this correctly? Are we being safe? Is it aligned? So it's going to slow down the pace at which new models roll out. Does that impact the amount of compute that gets sold? Does it impact the AI infra trade? Does it impact CapEx spent by all these hyperscalers? What do you think?

SPEAKER_01No, no, no. I wrote a whole article on this. It's totally free to read on my Substack. And it says that you know the AI CapEx build will you know just continue as is, and the infrastructure build-out will just continue as is. Because um for one, all of these things that we spoke about are very difficult to implement. It's literally like, where do we start? Do I agree that it's an important problem that it is um you know, has is a security problem to be solved? Yes, I I do agree.

SPEAKER_01Do you think do I think that people should be working on it? Absolutely. Should it be safe even by design? Yes. I mean, why are you deploying and trying to solve Navier Stokes equation and millennium precise problems and pissing off all the mathematicians in the world because now they're like, dude, like what the hell? You're killing the art form of mathematics. All the mathematicians are pissed. Like, why? Like, why don't you deploy all those like agents to go and make safer AI? Like, use recur instead of saying recur, don't use recursive self-intelligence.

SPEAKER_01It is self-defeating because everybody else is using AI to do make their jobs better. Why shouldn't AI researchers use AI to make their jobs better? So, yeah, use AI to make safer AI, because I don't see why it can't be done at all. Use your compute responsibly and dedicate the time and resources that you have in the right projects, not in just showing off why your model is the best model out there and you are ahead of the frontier. The incentives are not misaligned in the agents as much as it is in the humans.

SPEAKER_00Right, right. Um, yes. So I think you know, the point you're making is like, okay, well, if they need more compute to do this extra post-training type work, red teaming, interpretability alignment, well, maybe they are already working on other things that were burning a bunch of compute. Sora, anyone remember that? Um, or some of these other problems. So maybe they can just shift what they're working on internally, and it doesn't actually really impact things overall. Maybe it's a product manager priorization problem.

SPEAKER_00And I would um I would say also, okay, let's talk about inference. Okay. Remember, it used to be in 2023 that all the compute was spent on training, and over time it's been spent more and more on inference. And the model is amazing as is today. If we never had a model, there's a lot we can do today that is not yet happening in enterprises. And the diffusion of agentic AI into everyday life, into every domain has yet to actually happen.

SPEAKER_00That will be a crazy amount of inference, Jevon's paradox, all sorts of useful things are gonna happen. It used to be that now anyone could code, but now it's like, oh, now anyone can edit videos. Now anyone can make 3D renderings in Blender, right? Like there's gonna be so much that can happen. That has nothing to do with training. That's all inference compute, that's all here to stay. Now, on the training front, every company has to decide, oh, I want to allocate, you know, 40% of my compute to training and 60% to inference, let's say. Well, that 40% is very expensive. It's kind of like a sunk cost, that's just RD cost to build the next model.

SPEAKER_00The 60% is where they make money. Now, if they're saying, hey, we need to spend a little bit more time in training instead of rolling these things out in 10 months, let's roll it out in 12 months, and therefore we're gonna need just a little bit more compute for our training mix. I had this thought this morning and I tweeted it when I was preparing for this. I don't think anyone's gonna go, okay, now we need 42% of our compute for training, and now we only have 58% for inference. No, they're gonna go, dude, inference is where we make our money. We need an extra 2% compute hours for training.

SPEAKER_00Go get more compute. We need more training compute, go get more compute. They're not going to offset it and take away, they could take away from inference and have worse margins and make less money. No one's gonna do that, right? So literally, doing more compute during training just means you're gonna sell more compute.

SPEAKER_01Yeah, totally. It's a planetary problem. Uh, inference. Everybody is going to need inference. Like, I don't know, can you and I train a model? Probably not. So, no. So everybody needs inference, few people need training. Uh, but you know, I think models are generally capable of doing a lot of things already. They're kind of already reached a reasonably useful stage to do a lot of stuff. Uh, and I think there are a lot of untouched areas of AI, like you said, like coding and math are verifiable domains, but there are a whole lot of unverifiable domains, places where there's no data, there's places where the information is tacit or you know, just difficult to do with AI.

SPEAKER_01Those are all problems that still have to be solved. And those are like additive TAM that still hasn't shown up even in the inference part of the equation. And on the safety front, I want to say one more thing. Think, you know, think about financial transactions. Money is the most important thing to humanity, like other than their own life, like for forever, right? And today we have come to the point where we do all of this stuff to get money, which is stored in a digital bit somewhere. Like nobody ever sees their actual money in its entirety ever, probably in their whole lifetime.

SPEAKER_01But we sleep easy thinking that we have these digital bits stored somewhere that we can convert into items of luxury whenever we feel like we want to buy a car, buy a home, you know, so we can convert money somehow into material goods and use them. And we don't lose any sleep over it because there are safety practices, uh standards, bodies, you know, everything has been set into place over years of financial and decades of financial uh infrastructure that lets us sleep easy at night, right?

SPEAKER_01So AI will eventually come to the point of, hey, we all have AI, we use it, but we don't lose sleep over the fact that it's going to come and kill us in our sleep, because we have all these safeguards that are there. Even if we don't understand those safeguards, what those safeguards are. Like I don't think you or I understand the financial banking system, but even then we'd have our bank accounts and our investments somewhere and we sleep at night. So the AI system uh systems will evolve to the same point where I think we'll all sleep well at night eventually.

SPEAKER_01So we have to get there. Totally.

SPEAKER_00Yes, yes. Yo, I got a fraudulent thing on my credit card. Well, I call the bank, they make it right, you know, like there's there's ways. And and so again, there's like guardrails and systems and ways of making things right. And I and I don't think we're gonna jump straight to like, oh, the the AI did all this destructive stuff while I'm sleeping, that no one had any guardrails to turn it off. But I but I I do agree with you that the also the broader point of like our North Star should just be like, let's make sure that we feel comfortable with that there will be some level of safeguards so that we can sleep well at night.

SPEAKER_00And at the end, yeah, we are we are the agents here, we have the agency, we can turn it off. Uh those companies can pull the plug, right? Uh, yes, people can buy compute and fine-tune and train some stuff in smaller scale, and you know, we we don't have visibility into all that. Yes, there's other bad actors out there, but you know, I think these are grand problems to think about. I don't think they necessarily mean less compute, I think they mean more, and I also sleep well at night. I agree. Actually, I don't sleep well at night, but it doesn't have to do with anything related to semiconductors.

SPEAKER_01It's more about being adept, there are plenty of other problems for that.

SPEAKER_00Yeah, yes, that's right.

SPEAKER_01Yeah, but I think the I think the AI build-out will continue as is because I think we still need compute, and I don't see the demand going away anytime soon. Um, there will be at some point some kind of an inflection, something that changes everything around. It always happens. Uh, it's not going to be because we paced the frontier of AI. That's all.

SPEAKER_00For sure. Um, maybe uh this could also be very slightly net positive for XPUs in that if we do need a little bit more compute during training, it's probably GPUs for training. So that's obviously beneficial for GPUs because you know, these are not just like super well-defined tasks. Um now you could say, therefore, if they happen to grab those GPUs from their existing fleet, um, and they're gonna stand up more inference capacity to make up for it. Maybe they actually would inference is where XPUs play.

SPEAKER_00XPUs don't really play in training, right? And so yeah, yeah.

SPEAKER_01This is and this whole thing reeks of yeah, and this whole thing reeks of like uh you know, cybersecurity being the next big dude. Like right, like come on, no doubt. It's staring at you which you think when everybody's fighting the crowd strikes. Yes, exactly. So the everybody's like, oh wait, cybersecurity is important now. I'm like, hey, we know how it's done. And so everything is like cybersecurity. So totally, totally cybersecurity up, inference up.

SPEAKER_00All right, man. Good chat. This was fun, very interesting. Um, people leave your thoughts, tell us what you think. Uh, we're always interested to hear. Leave comments on YouTube, send us an email, whatever. Thank you for listening to Semi Doped.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Google Built Two New TPUs Without New Silicon: The 8i Max and 8TX

    Nov 17, 202654 minAudio

    Austin Lyons and Vik Sekar break down Google's two new TPU systems, the 8i Max and 8TX. These are not new silicon but new system configurations of the existing V8 chip, designed for specific workloads. The conversation...

    Credit the people on this episode

  2. Jensen Huang Resigns

    Nov 12, 202632 minAudio

    Jensen Huang resigned as Nvidia CEO this morning, and Austin and Vik recorded two hours after the news broke. They argue that a board must prioritize corporate integrity over short-term shareholder value, even for a...

    Credit the people on this episode

  3. Intel's Earnings Disaster: Bad Luck, Bad Planning, or Both?

    Nov 9, 202657 minAudio

    Intel just had its worst day on the stock market in 55 years, with shares falling nearly 30%. But here's the strange part: Intel actually reported a strong quarter. Revenue grew, its data center business surged, and the...

    Credit the people on this episode

  4. 3D Stacked DRAM for Inference: d-Matrix's Sudeep Bhoja

    Nov 5, 20261 hr 16 minAudio

    The fundamental physics of data movement are forcing a shift to new memory architectures for AI inference. Sudeep Bhoja, co-founder and CTO of d-Matrix, joins Vik to detail the company's journey from a non-consensus bet...

    Credit the people on this episode

  5. Why Marvell Wants Lumentum: Vertical Integration in AI Optics With Cignal AI

    Oct 30, 202640 minAudio

    Marvell's potential acquisition of Lumentum is a strategic vertical integration play to capture the massive value optical components now unlock in the AI era. Andrew Schmitt and Scott Wilkinson of market research firm...

    Credit the people on this episode