Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Semi Doped

Semi Doped artwork

Google Built Two New TPUs Without New Silicon: The 8i Max and 8TX

Nov 17, 202654 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Austin Lyons and Vik Sekar break down Google's two new TPU systems, the 8i Max and 8TX. These are not new silicon but new system configurations of the existing V8 chip, designed for specific workloads. The conversation covers the 8i Max's use of advanced packaging to increase density and the 8TX's swap to an AMD CPU to enable confidential computing for multi-tenant cloud customers. Presented by Crusoe: Fine-tune and serve open models without managing a single GPU. Deploy on Crusoe in one click, or download and run it anywhere. Upload a dataset, pick an open model, and pay per token rather than per GPU hour. Accounts are free, and get $5 in credit when you sign up. Try Serverless Fine-Tuning: https://semidoped.com/crusoe Key Takeaways: - Google's new TPUs are system-level innovations, not new silicon. The 8i Max and 8TX are both built around the existing V8 chip, signaling a strategy of diversifying offerings around a single generation. - The 8i Max is a packaging play. It places two 8i chips on one module to double on-chip SRAM to ~768 MB, creating a single logical processor for long-context and agentic workloads. - The 8TX system is built for security, not performance. It combines 8T training silicon with an inference network (Boardfly) and an AMD EPYC CPU to deliver confidential computing via AMD's SEV technology. - The choice of AMD's EPYC CPU over Google's own Axion is a time-to-market decision. It gives Google's cloud customers a mature, flight-tested hardware security feature without waiting for an in-house version. - The 'Max' branding signals a roadmap. Austin and Vik speculate that this year's premium dual-die 8i Max configuration will likely become the baseline for next year's TPU 9i. - Software unlocks the hardware's value. The new TPU Sync layer manages the memory hierarchy (HBM, DRAM, SSD), cutting KV cache transfer overhead by 90% and improving time-to-first-token. This episode is supported by Meter. You have the GPUs and the power. Now who builds the network, and who do you call when it breaks? Meter designs, deploys, and runs frontend, storage, and management networks on its own switches and its own network OS. One call reaches the team that designed the hardware and wrote the software. https://semidoped.com/meterGoogle's two new TPU systems, the 8i Max and 8TX are not new silicon but new system configurations of the existing V8 chip. The 8i Max uses advanced packaging to increase density and the 8TX swaps to an AMD CPU to enable confidential computing for multi-tenant cloud customers. Presented by Crusoe: Fine-tune and serve open models without managing a single GPU. Deploy on Crusoe in one click, or download and run it anywhere. Upload a dataset, pick an open model, and pay per token rather than per GPU hour. Accounts are free, and get $5 in credit when you sign up. Try Serverless Fine-Tuning: https://semidoped.com/crusoe Key Takeaways: - Google's new TPUs are system-level innovations, not new silicon. The 8i Max and 8TX are both built around the existing V8 chip, signaling a strategy of diversifying offerings around a single generation. - The 8i Max is a packaging play. It places two 8i chips on one module to double on-chip SRAM to ~768 MB, creating a single logical processor for long-context and agentic workloads. - The 8TX system is built for security, not performance. It combines 8T training silicon with an inference network (Boardfly) and an AMD EPYC CPU to deliver confidential computing via AMD's SEV technology. - The choice of AMD's EPYC CPU over Google's own Axion is a time-to-market decision. It gives Google's cloud customers a mature, flight-tested hardware security feature without waiting for an in-house version. - The 'Max' branding signals a roadmap. Austin and Vik speculate that this year's premium dual-die 8i Max configuration could become the baseline for next year's TPU 9i. - Software unlocks the hardware's value. The new TPU Sync layer manages the memory hierarchy (HBM, DRAM, SSD), cutting KV cache transfer overhead by 90% and improving time-to-first-token. This episode is supported by Meter. You have the GPUs and the power. Now who builds the network, and who do you call when it breaks? Meter designs, deploys, and runs frontend, storage, and management networks on its own switches and its own network OS. One call reaches the team that designed the hardware and wrote the software. https://semidoped.com/meter Chapters: 0:00 Not New Silicon, New Systems 7:28 Refresher: 8T vs. 8i 12:42 Defining the 8i Max 13:59 The 'Mirror Chip' Mistake 18:47 Packaging vs. Optics 22:42 The 'iPhone' Naming Strategy 23:09 Defining the 8TX 28:51 How Confidential Computing Works 30:03 Why AMD? SEV Technology 39:58 Fine-Tuning on the 8TX 42:37 TPU Sync: The Software Layer 51:48 Dogfooding as an Advantage Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/ Chapters: 0:00 Not New Silicon, New Systems 7:28 Refresher: 8T vs. 8i 12:42 Defining the 8i Max 13:59 The 'Mirror Chip' Mistake 18:47 Packaging vs. Optics 22:42 The 'iPhone' Naming Strategy 23:09 Defining the 8TX 28:51 How Confidential Computing Works 30:03 Why AMD? SEV Technology 39:58 Fine-Tuning on the 8TX 42:37 TPU Sync: The Software Layer 51:48 Dogfooding as an Advantage Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00Google just announced two new TPUs. Except they really aren't new chips. They're just derivatives of their V8 chips announced earlier this year. So Google is taking existing silicon and building two very different systems around it. We'll also discuss TPU Sync, which looks a lot like NVIDIA Dynamo. So what is really Google up to here? Let's get into it.

SPEAKER_01Hello listeners and welcome to another Send Meok episode. I'm Austin Lyons of Chipstrat. I write a newsletter for institutional investors and technologists.

SPEAKER_00I'm Vic Shaker and I write a substack called Vic's Newsletter. And I also am the founder of Semi-Exponent, which is an independent research firm for institutional investors. And you should check out our daily newsletter on uh daily.semidope.com. We cover all the daily news around the AI and semiconductor industry, and that is an entirely free newsletter to read on Substack. So you'll have that delivered every day to your inbox, along with our takes on the news.

SPEAKER_00So sign up for that at daily.semidope.com.

SPEAKER_01Yes, and which by the way, I haven't published ours today. I think you put your takes in there and you're waiting on me. But uh we're recording this on Thursday, November 12th, and we were gonna record this TPU thing, but then of course there was some big news this morning about NVIDIA and Jensen Huang retiring. So instead, we just recorded a hot take, and then so people will have already heard this, and now in the same day, we're wearing the same clothes, and it's like almost midnight for Vic. Uh, and we're recording this TPU one as well, which means I didn't get the Sendy Dub Daily out yet.

SPEAKER_01So let's record this, and then if I remember, I'll try to add my takes and hit send.

SPEAKER_00Yes, the AI world never stops, time zones do not matter, so we have to get to it. So, what's with the whole uh Google thing? Um let's get into it actually, because if you saw the announcement, Austin, like it's like suddenly there were two new TPU uh SKUs that landed. And the the earlier ones announced this year for the version eight uh TPU chips was the TPU uh V8P for training and the V8i uh for inference, which was uh kind of a nice way to name it, and it was like very understandable, and we were very happy with it when we discussed uh the networking that went around these chips at the time, which was basically they introduced this new network topology called Board Fly, which we explained in a YouTube video, uh a podcast here, and it was seemingly one of our most watched podcasts.

SPEAKER_00People really seem to like it. So anybody who wants to check that out should be there in our back catalog. But the recent announcement uh a few days ago uh is that we have two more versions of this eighth generation chip, which is the TPU 8TX and the TPU 8i Max. Okay, so we're gonna go and discuss like what is happening with these four versions of the same chip in one generation, that is like version eight.

SPEAKER_00Like, so what do you think is going on? What's what's happening here?

SPEAKER_01Yes, yes, let's dig into it. So so first of all, I barely saw the news because I think this came out like at the start of the week, and obviously there's all the NVIDIA stuff in the news, and so most people's eyes and attention was on the NVIDIA stuff, and so this was kind of like a as Claude would say, it was it was quietly announced that you know Google had some extra chips here. Um and I was surprised, I was like, wait a minute, there's four. Two was kind of a big deal because it used to be back in the day that for most all the generations, the TPU just had one and they use the same one for training and inference.

SPEAKER_01And then I wrote an article when the 8T and 8i came out. Uh, around that time, I'd been talking about like, hey, it's the right, you need the right silicon for the right workload these days. And we've talked a lot about um different memory hierarchy decisions and different networking decisions, um, not only with the TPU, but also with like other startups, AI accelerator startups, that are like, oh, these they're choosing more SRAM here and more HBM here, or also using um DDR and all these different decisions. And so it made a lot of sense when um Google came out with the TPU for training and the TPU for inference, because the training one was like for the more compute-bound workload, and the inference one was for the memory bound, and so it had more HBM, and then to your point, they had different networking, and so you could see, oh, like uh topologies, whether it's the 3D Taurus or the board fly.

SPEAKER_01And so you could see, oh, hey, this is pretty cool. Google is designing not just chips, but entire system for training, an entire system for inference. So then when I saw that news and I saw it just kind of fly past on my screen, and it was like, oh, also they have the 8TX and the 8i Max or whatever. Um, I was just like, wait, we already have a system for training and a system for inference. What are these two other systems for? So, with that, what are they for? Let's start with the 8i Max. Should we start there? This episode of Semi-Doped is presented by Crusoe, where you can fine-tune and serve open models without managing a single GPU.

SPEAKER_01When we sat down with Crusoe's serverless fine-tuning product team, we asked the question every AI user wonders, what will you do with my data? Crusoe has a simple answer. They don't touch it or see it. Your data is yours to fine-tune models with or simply store for future use. Their fine-tuning is designed so that both your training data and the fine-tuned model itself remain your property. Your training data is encrypted, so even Crusoe's cloud engineering team can't see it. You can deploy it on Crusoe Cloud in one click or download the checkpoints and run it anywhere you like.

SPEAKER_01The fine-tuning process is fully managed. So you upload a data set, pick an open model, and Crusoe takes care of the GPUs, orchestration, and all the infrastructure management. You pay per token processed rather than per GPU hour. So run a few versions, compare them, and download the one that actually works best for your use case. Create a new Crusoe account for free and get$5 in credit to start testing today. This part is super important, guys. Support semidoped by going to semidope.com slash Crusoe, click on that link in the show notes, type it in yourself, and you'll get directed to a sign-in page.

SPEAKER_01Sign in with Google or create a new account, and Crusoe will immediately create that account for you with$5 in it. Boom. Again, just click that link in the show notes, sign in, five dollars for you, and you support semi-dopes. So thank you for your support. Go check out Crusoe, try to fine-tune your own data. Again, the link to try serverless fine-tuning is in the show notes. So hit that now, click through, and make sure you claim that$5. Give it a try and support us. Back to the show.

SPEAKER_00I want to go and just set the stage for what the original chips were so that we could discuss like what the new ones are. Because I want to just remind listeners what the old ones, like the old, you know, the previously announced ones this year were. Because so let's start with the 8T. The 8T was basically for large-scale pre-training, right? It's a training chip. And the 8i was for inference, right? That much is clear. The 8i had more HBM than the 8T.

SPEAKER_00So it had like 288 GB for the inference chip and 216 GB for the training chip. And the HBM bandwidth per chip was also higher in the inference chip. What was significantly higher in the inference chip was on-chip SRAM. It was like almost three times as much. There's 384 MB of SRAM on the inference chip versus the training chip. And you can see the chip too. It's like a humongous chip. The inference chip is humongous compared to the training chip. And at Hot Chips, the speaker from Google put up a picture and had a quiz and said, like, which one is the training chip?

SPEAKER_00And everybody feels like they want to say that the training chip is a bigger chip. For whatever reason, you think the training

unknownis

SPEAKER_00like a hard problem and therefore bigger chip. No, no, it's completely the opposite. It's actually the inference chip because of the massive SRAM capacity in it. You know, was actually the inference chip. So no, the training chip was smaller. So the the what Nvidia's design choice for the inference chip was that you they put on a lot of memory. So whether it's HPM or SRAM, it's like memory. And this is a good reason for that, is because inference is a memory-bound problem, and we will need this piece of information to discuss the 8I max.

SPEAKER_00That's why I'm setting the stage. So we need to have a lot of memory because you know the inference has basically two stages, the pre-fill and the decode, and the decode is an entirely memory problem. So it makes sense for an inference chip to have a lot of memory. So that's why they did it that way. And the other way is that the training chip actually has more compute, okay, so it than the inference chip. Okay. So that's the internals of the chip. There's like a little bit, like you said, there's a system involved here, really. So there's a little bit more detail, and that is for the scale-up networking of the training chip, they use the 3D torus topology.

SPEAKER_00Okay, so this is like what they have used in for the longest time, really, with all their optical circuit switches. It's a very complicated 3D interconnection that involves both copper and optics, right? And then the 8i actually uses board fly. And this is something to remember, and we did the whole video on board fly, but it's important to remember that the inference chip used board fly. And we'll see why this is important when we get to the 8TX. Okay, because it's very, very confusing. Um and then finally, both of these C these 8T and 8i, as a head node, they used a Google Axion CPU.

SPEAKER_00Okay, so this is like an ARM-based CPU that acts as the head node. Now, this is the fundamental data set you need to understand what these new chips or these new variations of these chips are going to do. Right? So far with me?

SPEAKER_01Yes, I'm with you so far. And yeah, I was just gonna add in the 3D Taurus, people can remember. I think of the Rubik's Cube where it's like, oh, I want to talk to my neighbors in different directions, and that makes a ton of sense for training because there's very consistent communication patterns. And then the board fly is good on inference because these days it's we've got a mixture of experts, and you don't know exactly who you're gonna talk to. It kind of depends on which experts getting fired for which token. So you do need a different topology that's suited for that. Um, and then I'll add in lastly, I'm glad you called out the Axion CPU and that it's ARM-based.

SPEAKER_01And it reminds me, whenever I listen to ARMS earnings calls, you know, we we I listen to Intel's earnings call and then AMDs and then ARMS, and uh in some somewhat of that order, maybe ARMS came first, it's they all run together. Um, but you know, there's always like an x86 one that'll say, like, oh yeah, we sell, you know, that we still have the majority of CPU server sales or whatever. And then ARM comes in and they're like, Yeah, we we sell, you know, or 60% of the CPU server sales are ARM, and you're always like, What? I'm so confused. These add up to more than 100%.

SPEAKER_01Um, but it's usually like one is focused on just x86, and then ARM is focused on everything. Um, not you know, and and so headnode, I bring this up because head node, because we tend to think like, oh yeah, axion and it's like in Google Cloud, but you also forget they can count head nodes. So if you see, oh, a bazillion TPUs got made, then you know, I guess well, there's a bazillion head nodes to go with it, or half a bazillion or whatever. And that his right now, for this generation, it has mostly been their ARM-based Axion, but we will get into that as well in a little bit.

SPEAKER_00Yes, it's very, very important uh fact that like axion is going to be replaced by something else. But yeah, so this is the difference. And the boat flight topology was used for inference fundamentally over the 3D torus as a refresher from the other video if you haven't seen it, is that it reduces the overall number of hops required to go from one point to another within the networked GPUs or TPUs. And reducing the number of um hops is important because you can reduce the latency. The lesser networking you use, the better.

SPEAKER_00So that's the reason board fly is good for inference because we want latency to be low, right? So that's where we are now. Now is a good point to jump into, like you said, the 8i max. So the 8i max is nothing different from the 8i. I mean, the silicon is exactly the same. They just took two 8i chips and put them together on a single package, right? Is that that's pretty much it, right?

SPEAKER_01Yes, yes. And there's an amazing um graphic that they made that we will try to include here that shows, yes, I think one server node or like one you know pizza box, if you will. I believe it used to have four individual chips, and now it has those same four columns, if you will, but two chips kind of per column. So there's eight in a pizza box instead of four. At least that's what their graphic illustrated.

SPEAKER_00Yeah. So now the question is like, why would somebody like put take the same chip and like put two copies of it in there and package it together? Exactly. What happens? Yeah.

SPEAKER_01Why didn't they do that in the first place? That was my thought.

SPEAKER_00So there's always an evolution because now what happens is like normally people don't want to jump generations like that because you want to make sure one chip works and before the other chip uh comes up and screwed, you know. Because now if the package doesn't work, two chips don't work. By the way, I have to tell you this funny story. This is a true story from my personal experience in the semiconductor world. I worked at a company once that uh had to make two chips like this. I think this is a great time to tell the story. And they decided that for whatever reason, um, because RF is very sensitive stuff, they decided they will make the mirror version of the same chip because the package required two chips.

SPEAKER_00So they decided they'll just mirror the chips so that you could create a symmetric like package layout to put both the chips on. And they said, fine, this is a beautiful approach. Look at the symmetry and the beauty of this. RF guys love symmetry. Right? And then what happened was the first iteration of the chip, something went wrong. Now the problem is the mirror image of the chip is not the same chip, they're two different chips.

SPEAKER_01Oh, what? So

SPEAKER_00every iteration, yeah, because the the complete mask is a mirror mask. You can't so even the transistors are mirrored, they're not like everything is not the same. So they had to run the same chip uh twice every time and pay for all the masks from beginning to end because they decided to use a mirror without determining if the first chip worked.

SPEAKER_01Oh, oh no. So they okay, so the first chip, if it's broken and then you create a mirror of it in an expensive way, it wasn't just a rotation of the first chip, it's an actual mirror. Yeah, it's a mirror. It's all new masks. And then, oh crap, the first one didn't work. And so now it probably doesn't work.

SPEAKER_00Yeah. So the development cost for this chip was double what it should have been because they mirrored the chip, right? And the chip didn't work once, so every time you made a fix on one, you have to make the mirror chip fix and you had to double everything, mass costs and everything. It's like hilarious. Oh my god. So you know what? Make sure one chip works first before you make two of them. So like I have seen this in my own little world. Uh, and so Google obviously knows what they're doing. So, yeah, so obviously the 8i is a good way to do this.

SPEAKER_00But then the question is, why would you put like two of them together? I think the advantage is that you double HBM, you double the SRAM. Remember, like now it was like 384 MB of SRAM, right? Uh per 8i chip. Now you have double that. Like you have like 760 whatever, like, you know, MB of SRAM. Imagine what amount of stuff you can do with that, right? So now your bandwidth is like enormous bandwidth. Um, and uh your uh HBM capacity is so much more. It's it's it's really good to hook up two chips.

SPEAKER_00Everything uh, you know, in terms of hardware specs kind of doubles almost. Yeah. So and that's a it has its advantages.

SPEAKER_01Yeah, let me ask you a question about that then. I guess you're making the assumption that the program has access to all the chips that are on the board, and therefore, to the programmer, suddenly there's just twice as much SRAM and twice as much HBM. Um, and is that how how you're thinking about it? Because you know, if if there are individual programs like running on each chip, then it's like, oh, well, they're just in proximity to each other. So clearly these have to be like the system, the kernel can see now double the HBM, double the SRAM.

SPEAKER_00Yeah, I think that's how it is, because the way when you package things together in an advanced package or something like this, uh the idea is that they behave more as like one chip working together. Like it's it's like more like one huge chip

SPEAKER_01that's twice the size of the old chip. It just happened to be disaggregated, but then reconnected through advanced packaging.

SPEAKER_00Yeah, when you use advanced packaging, the benefit of that fine-grained nature of advanced packaging is that this 8i Max chip is literally like uh a chip that is like two times bigger than the 8i. And you can't make two times bigger chips because they are already very reticle limited. Remember, I was saying how big the 8i chip is to begin with. Like now you want to make, you can't make it, you become reticle limited on the wafer anyway. So Nvidia does this too, right? Like with their Ruben packages, you know, they have two uh Ruben dye in a package, and they were talking about putting four Rubin dye in a package, in an advanced package and things like that.

SPEAKER_00So you'll burn a lot more power per uh per eighti max, but you get so much more out of it. And this is good because you know, this is all towards a trend of rack densification. Like, how do you put more TPUs or GPUs into a rack? Like it's very important to get as much compute as possible in a rack because networking to get the same kind of uh you you end result is a very difficult thing to do. And what I mean by that is you could just put two chips instead of uh advanced packaging them together, you can just put two separate chips and hook them up with even optics, right?

SPEAKER_00Like, why do people not do that? Because there is no match to putting two chips in an advanced package, you know, optics can't match that kind of latency. No, it's always going to be slower.

SPEAKER_01And it costs more power.

SPEAKER_00Yes, exactly. If you're moving bits over a longer distance, you're gonna burn more power. So nobody wants that. So you want to uh put everything into a single rack and densify the rack as much as possible because another benefit is that maybe you can do it with copper, you know, like the reach is limited. It's the reach limitation means that you can save on energy and you can use other technologies, like a lot of things open up. And in an in a world where people don't want to build data centers, you better put more compute in a rack, right?

SPEAKER_01Yes, I was gonna get there too. I had had the exact same thought, which was essentially, yes, we haven't talked about the no, no, no, no, you it's good. Great minds thing alike. We haven't talked about moratoriums and stuff like this, but that's where my head went. It was like, well, I guess if it's going to be harder, which time will tell, um, if you've already got space and you can take one rack out and put another rack in and get twice as many, of course, that is a good thing for continuing to scale the AI build out for sure.

SPEAKER_00Exactly. So, yeah, so it's a good thing, right? And the benefit is that the AI Max now has uh so much more, it can hold so much more of the model, or it can hold so much more of the KV cash. Um, and uh you know, all that kind of stuff. Like you have to move data less so you can save power. You can run long agentic tasks because you have so much storage lying around, you can run fast inferencing because you have double the SRAM. Uh so it gives a lot of benefits uh in terms of like what kind of performance you can extract out of a huge chip like 8I Max.

SPEAKER_00So I think it's definitely built for like long context agentic workloads.

SPEAKER_01Yes. So I guess okay, two thoughts. One, does this I remember NVIDIA was you know gonna go to tying four reticle limited GPUs together, but then there was a question about warpage from KOAS because it means you needed a bigger substrate. And then it made me think, oh, is this just going from one to two? But I'm I'm assuming that the TPU is also built using COAS. I don't know if it's CoAS S or CoAS L. Um, and and so then I'm thinking, like, oh, what's uh the TPU gonna do in the ninth generation?

SPEAKER_01Do you think that they will still have the one and two variants, like the 9i and 9i max, or do you think they'll just go with this max? Because don't we always just want more memory for inference? And then of course, then I also wondered, are they gonna scale it even bigger to four? Are they is ninei gonna have two, and then ninei max actually has four? Like, what do you think?

SPEAKER_00Yeah, well, I think they'll they should go. The 9i should be whatever the eighti max is, right? That's the logical way to go. And I don't think they'll, I mean, the 9i max might be the four version. Total guess, by the way. I have no idea what the roadmap is. Like, I have no inside information or any such thing. I'm just totally guessing here. 9i Max, you know, may have to go and solve those warpage issues, and uh, you know, then you know.

SPEAKER_00We can put more chips. And I'm guessing Nvidia will also reach that point where they will put four G Ruben die into a I think Ruben Ultra, right? That was supposed to be for the Ruben Ultra. And uh so I'm they may reach that point too where we can make this big enough. All these advanced packaging houses are working towards making bigger and bigger packages, right? Like we're talking about glass interposers and so many things are coming down the line. Those are all going to be solved eventually. And so we will go to four four die systems. But yeah, that that's the way to push the technology forward.

SPEAKER_01Yes. Come on, Intel, EMIB, get in the game here, work with Marvell and whoever, Media Tech, make these TPUs, try to package them. Oh, and I'll also say the Macs naming reminds me of the iPhone. And it feels like we're talking about iPhone generations because it's like, oh yeah, this year's Max will be next year's base general.

SPEAKER_00Yeah.

SPEAKER_01Yeah. Yeah. So

SPEAKER_00I think they're losing the naming game. The 8T and 8i was like nice.

SPEAKER_01Yes.

SPEAKER_00But now it's going to get out of hand. Do we have to cover anything else about the 8i Max? Or do we go to the other crazy name?

SPEAKER_01I think it makes sense. Let's go to 8TX. Okay, 8TX. What does that mean, Vic?

SPEAKER_00Okay, so what is the 8TX? So let's get into that. It's a weird name, and I have no idea what the name means, but Austin, you will suggest a better name at the end, I'm sure, because you have a flair for these kinds of things, but I'll just explain what it is. So it takes the 8T silicon, right, which is basically the smaller chip. Remember, this is the one with less SRAM, less HPM, and then throws out the Google Axion host node CPU, and instead puts a fifth gen AMD Epic CPU as the host node CPU.

SPEAKER_00So it uses the 8T, the training version of the silicon, and it uses the AMD chip instead of the ARM-based axion chip. And funnily, it uses the board flight topology. So remember the 8T silicon that we spoke about used 3D torus because it was a training chip. But this uses the 8T silicon with board flight topology and an AMD CPU. You see the confusion here is like a complete re-jigging of everything.

SPEAKER_01Right, right. It's like the training chip with the inference network and then with the different CPU. So 8 TX. So but uh yes. I I would not name it, I would not put a T in it. Now, well I guess okay, so T is use case to me. It's training, right? So what did they say the use cases are for this, by the way?

SPEAKER_00The use case seems to be what they call confidential inference. And we'll get to it, but that comes from the air use of the AMD Epic CPU. And uh they also said this use for private fine-tuning of models.

SPEAKER_01Okay. Is that training?

SPEAKER_00You think that's training?

SPEAKER_01I think the first bit was inference, and then the second bit is training.

SPEAKER_00Um

SPEAKER_01but security sounds like the most important thing here. Um so maybe the best name would be 8S.

SPEAKER_00Yes, good. Because then it doesn't go with that.

SPEAKER_01It doesn't conflict the tr the use case and and then also the topology and stuff. So maybe so Google, if you're listening, maybe next time chat with us. Maybe something like 8S. 8 secure chip.

SPEAKER_008 secure is so much nicer.

SPEAKER_01It's a system. Yeah.

SPEAKER_00Yes, it's a secure system. Yeah.

SPEAKER_01See,

SPEAKER_00you we just have to use S. And you know, 8TX. I don't even know what the X stands for, like anything, like 8T and all the other crap. Or like, yeah, I don't know. Well, I mean,

SPEAKER_01for an RF guy, TX probably means transmission.

SPEAKER_00Yeah, that's what I thought. Yeah. But it's not that. Okay, so confidential in in so what is the use case for this? I think that uh essentially the this chip is designed to make sure that you can use these um chips to do AI in sensitive in information-sensitive environments. Like a trading firm might be uh training on their data to make like stock purchase decisions. It's very important for their business and it involves a lot of money that they're training um uh towards making maybe these trading decisions, uh maybe the hundreds of millions, billions, requires security.

SPEAKER_00Nobody can access this. You know, I think of somebody like uh Jane Street or a big company like that, you know, doing a 0.72 or somebody like that, I don't know, doing something like you know, um quantitative trading, like um high frequency trading, things like that. Well, the other use case I can think of is like uh, you know, think about Nvidia, for example, or it's probably a bad example, but anyway, think of any company that has like a super secret way to design a chip. Like this is their secret sauce, and they want to now train AI on their secret sauce chip design.

SPEAKER_00And they don't want to make, you know, run it on a server that's shared by anyone else. The thing is, if you go to the cloud and you start running inference, your hardware is being shared with other people who are also running har their inference on yours. So, how do you guarantee that your tokens or your information of whatever is not going to mix in with somebody else's or leak in any way? There's no way to guarantee that. Yes, I think this chip is based towards that.

SPEAKER_01I think that is actually the key and primary use case, which is neo clouds or clouds or whatever, and having neighbors potentially sharing the same system that you don't want to see their information. Because prior to this, you know, if the first and best customer was Google, well, of course, all the when they're trading Gemini and all this stuff, like wouldn't they want it to be secure? Why didn't they build this security and hardware security in mind in the first place? Well, yeah, of course they want to be secure, but they were the only customer.

SPEAKER_01It was all just internally used compute. So now when they're trying to give their compute to someone else who's going to be a landlord and rent it out to other people, I think now is where you want to say, oh yes, you can put your data on our accelerators, and when it moves from the accelerator to HBM or from the accelerator to the CPU or whatever, it will move encrypted data, is my understanding. My understanding, and tell me what you you've studied this more than me, is that you don't want in in these use cases, you don't want the memory or the data sitting in memory to be unencrypted per se, but you want it to be sort of password protector, if you will, until it's being actually computed on.

SPEAKER_00Correct. So the main purpose of this is to make sure that the data only reaches the whole the target it is intended to reach. And by that I mean like the data is encrypted, and so there is a key. Okay, the key is held only by the correct party who can unlock the data. So even if this data goes to some other place by whatever means, uh they can't do anything with the data. There's no key to unlock it. So that's what this whole confidential computing comes to.

SPEAKER_00So the only the intended recipient can interpret that data, and that kind of encryption is very important for uh trusted multi-tenant situations.

SPEAKER_01Yes. Absolutely, totally, that makes sense. Okay, so why an AMD CPU then? Why not the the Axion?

SPEAKER_00So Axion does not have the feature to uh basically encrypt this information to the level at least that AMD can do it. I don't know if that feature exists or not, maybe it does, but I don't I don't think that um it's as good as what AMD uh can do with their Epic systems.

SPEAKER_01Right, right.

SPEAKER_00So the AMD chip has something called a secure encrypted virtualization or SEV. Right? So essentially you can have a virtual machine uh that is uh completely isolated uh from other virtual machines and from the underlying operating system that the virtual machine sits on. That underlying operating system is often called a hypervisor. So the VM is completely insulated from the hypervisor underneath.

SPEAKER_00Hypervisors are shared by many VMs. So you don't want any information leaking from VM through the hypervisor to the other VM. So this capability uh allows uh not just any kind of VMs hosted on these CPUs to be secure and encrypted, but also it allows information that is coming in and out of the CPU or devices that are connected to it, like the TPU in this case, the 8T training TPU, it all of them inherit this uh secure encrypted virtualization feature, which is what uh allows secure transmission of data between these various uh parts of the system.

SPEAKER_01Gotcha. So so the 8x is using the 8T silicon and nothing has changed. So all of this is done in software managed by the CPU, but the TPU does do it must have some sort of software to be able to handle this as well.

SPEAKER_00I'm not sure of that software layer within the CPU, but like there's nothing new in the hardware because the silicon is exactly the same as what came up earlier. So the CPU orchestrates the confidential computing. That's the key thing to take away here. So all that the encryption process, all that is handled by the CPU, right? And uh that's why they had to go and use this um ARM CPU instead. And actually, what is funny is that during the announcement, uh actually uh AMD actually put out a press release as well talking about their uh SEV feature, uh which I don't know, never made it into the news, but if you see it's there, yeah.

SPEAKER_00Yeah, yeah. They announced that it's there, this is the SEV is why it is inside the Google Eight TX, right? So

SPEAKER_01the natural question, by the way, to that is okay, is this just a temporary lead like AMD got it because they have the SEV? And will ARM end up saying, okay, that's on our roadmap now? So that the next Google Axion can have it? Or is this like a sustainable advantage for AMD?

SPEAKER_00No, no. So I think that um one of the reasons is that ARM does have client confidential uh computing. I think they call it client confidential, confidential computing architecture or something. If you see the ARM AGI, this phrase is in there. So ARM already has a CPU that can do this. Um I don't think it's like has that kind of like flight history that AMD has, because AMD has had this feature for a while. So I think it's not just it's it's not the right time.

SPEAKER_00I don't think it's locks somebody out forever. Other people have it too. Uh Intel has a version of this as well. And I wrote it down here, and it is actually called Trusted Domain Extension or TDX. And uh there was, I was looking up, like, why did uh you know Google actually choose AMD over Intel uh for this chip? Why don't you get an Intel chip? I just saw that there were some you know security concerns and stuff with the trusted domain extension, so I figured maybe they just didn't want to deal with it and they went to the AMD one because it's like well established and nice to go.

SPEAKER_01Yes, yes. Or maybe they weren't so sure maybe they weren't sure there would be enough supply from Intel. Hope that's not true. But you know, maybe they just said, hey, AMD seems to have the supply stuff figured out and we're gonna need a bunch of these. So uh

SPEAKER_00Exactly, exactly. I guess

SPEAKER_01we'll have to see. Now, yes, to your point, I agree. Clearly, the Axion was made at a specific point in time with certain design decisions that made sense for the Google Cloud. And so in this case, when they're like, hey, we need this security stuff, why not just pick one up off the shelf from a merchant silicon vendor who already has a portfolio of CPUs? So obviously AMD already had silicon that can do this, so does Intel. Um, maybe in the future we'll see a portfolio of Axion, you know, the this is our head node, one for the secure environment and stuff like that.

SPEAKER_01But point in time, easiest to just pick something off the shelf, plop it in.

SPEAKER_00Yeah, what is so funny about this announcement is that that this time I got an uh X invitation or uh somebody followed me. I'm like, oh, who is this person who followed me? Oh, it's ARM. Oh, wait, ARM is following me. That's cool. And then I'm like, oh wait, ARM DM'd me. Like, what did they say? They're like, oh wait, ARM DM'd me to say that the 8i Max actually uses the ARM-based Google Axion because it wasn't clear from the press release what that was using. The 8TX uses the AMD, yes, we know.

SPEAKER_00But what was the 8i Max using? So they they just didn't do a bunch of people on X.

SPEAKER_01Okay, so they were just proactively telling people on X? Had you been okay, okay. I was like, did you talk about it? And then they DM'd you, but no, they're just proactively.

SPEAKER_00They're proactively telling people on X.

SPEAKER_01Good job, ARMS uh social media team, for proactively saying, like, just in case it wasn't clear, we're also in this slot here.

SPEAKER_00Yes, please uh send it out on your channel just so that we know. So I mean, I'm doing my part. Semi-doped is where I talk about this stuff. Totally. And uh yeah, so 8i Max uses the Google Axion like everything else. And the simple reason is simple. You there's there's no you know encryption-based confidentiality computing here, right? It's only an 80x feature. Can we just call it like 8S, right? So that's that's the whole point. So that's where we are right now with these chips.

SPEAKER_00I think it's a good addition. I think it unlocks uh uh you know the 8I Max is honestly a bit more exciting to me because of the sheer size of that chip.

SPEAKER_01Okay, so yes, I agree with you. Technically, it's very interesting and it's it's very compelling. Now, from a sort of business model and even business strategy, business model perspective, the confidential compute one's interesting because that enables partnerships like OCI. Oh, hey, now we have a chip that you neo clouds, or I you you can argue that um OCI is just a cloud in its own right, um, which by the way, they had an investor day like two weeks ago. And I tried to start listening to it on a run, but it sounded like they were showing a lot of slides and stuff, so I need to watch it sometime.

SPEAKER_01Um, but OCI is trying all sorts of very interesting things, like you can even stand up a rack of their hardware in an existing in another cloud like AWS, which is kind of fascinating. Um, but anyway, uh OCI is being thoughtful and trying to partner with Google and stand up some TPUs in their own infrastructure. So be a first external buyer of these TPUs. And so clearly, yeah, by having um the secure enclave type stuff, I look at it and I think, oh, good, Google can sell these externally, which is super interesting because that means Google's sort of straddling the line between like using silicon for their own first party workloads to just be vertically integrated and bring their costs down, to being a cloud and renting out this first part party silicon to third parties who can run it like bare metal, who helps them amortize the cost of building all these TPUs and everything, to now sort of like a merchant silicon vendor in some regards, it's you know, selling TPUs externally to other customers.

SPEAKER_01It's just very interesting to think. It's also it's kind of even hard to wrap your head around. It's like, okay, is Google competing with uh AMD NVIDIA, or are they competing with AWS and Trainum and stuff like that? Or is Google competing with OpenAI and Anthropic? Uh or is Google competing with Meta and MetaMuse? It's like, yes, the answer is yes.

SPEAKER_00Answer is yes, everything is. Yeah, the answer is yes to everything. Yes. Uh maybe they shouldn't, maybe they shouldn't be. Maybe they should be focusing on something that is uh more near-term for them. But you know, even though it's the 8T, what is important to note about it is the 8TX naming system, it still uses the board fly network. And so it's very much an inference chip. So don't mistake it for like a big training chip. The naming system is like very weird, but uh A, it's essentially for secure inference.

SPEAKER_01Yes, but you can fine-tune on it because it is just an accelerator at the end of the day, and like a GPU, you can run several workloads. And I think the interesting point too is that really one way to think about the 3D Taurus is it's also very good for large-scale training when you're like like pod-level training, you know, like I need to talk to the chips that are closest to me on this board and then the chips that are closest to me in this rack, but then I also need to be able to talk to, you know, maybe other pods using RDMA.

SPEAKER_01And of course, yes, board fly is more about minimizing the hops because you don't know who you're gonna talk to yet. Um, you don't know who you're gonna talk to next. But if you're doing small um fine-tuning, you've got an uh open, like uh some medium-sized open model, and you're just taking your own internal data that you want to be secure, uh, and you are doing some training, it will still work with Boardfly. It's not like it won't work without the 3D Taurus. It might not be the most efficient thing in the world, but at the end of the day, you're not the kind of person that's like, oh, I'm putting a hundred thousand TPUs together, and if this takes me an extra day, it costs an extra, you know, a couple million bucks.

SPEAKER_01No, you're just like fine-tuning small stuff that probably lives all within one rack or maybe even within one server node or pizza box, so to speak. Um, so it can, it's inference first, but it can be used for fine-tuning.

SPEAKER_00Yeah, yeah. Part of this slew of blog posts that came on the Google uh cloud blog. I I mean, if anybody wants to look it up, it's very interesting. They had so many other blog posts that like was relatively lost in the noise. One of them was called um uh the Google Kubernetes engine. It's called the GKE, and it does exactly what you're talking about. It's called dynamic slicing. And you know, the whole point is very nice, actually. There are really good animations. We'll leave links in the show notes. But essentially, you can connect these like groups of TPUs together to form a bigger pod, right?

SPEAKER_00Like you can put, take many smaller pods of GPUs and you can hook up to them to become what they call a super slice, and then you hook up all the external connections via optical circuit switches, switches, OCS, right? So you can connect smaller pods into huge big pods with OCS, and that's called super slicing. But you can do the reverse too. You can go into a single pod and just carve out a little, you know, a few TPUs. So basically, you have complete flexibility. You know, you just throw a bunch of TPUs into a data center.

SPEAKER_00You can slice them any way you want. You want to put them together to make a bigger thing, you can do it. You want to take a single one and break into it to sub-slicing, you can do it. It's like fascinating the kind of things that you can do with these things. Um, but yeah, all that is like really detailed. That like the blog posts, as usual from Google, have beautiful animations and explanations. They're not even very long, so I highly recommend people read the links in the description. But uh yeah, I think one other point we should touch on now that you mentioned that, you know, other people like Oracle can go and use Google and they have some announcements, some plans around this too, is that what makes this happen?

SPEAKER_00Like, how does Oracle like use Google TPUs without, you know, the CUDA was the biggest thing for Nvidia. Like everybody could program on CUDA and use GPUs. I think Google is making it a lot more easier now to do those kinds of things with TPUs. And I'm no software expert, so I can't go into all the details, but I think one thing stood out to me nicely that straddles the world between software and hardware, and that's called um TPU Sync.

SPEAKER_00So TPU Sync is now an open source project that um essentially I feel like it's Google's version of NVDS Dynamo. Maybe I'm wrong, but somebody can correct us. But basically, what it helps you do is that it helps you manage the memory hierarchy. It hand it allows you to handle HBM, DRAM, uh SSD storage, and even network storage. Uh, I think Google has like a special version of network storage uh that they use. Um so all of that, this software layer, which is open source, uh, allows the inference engine to really like distribute work to the right places at the right times, right?

SPEAKER_00Like, you know, it doesn't recompute anything. It there is always a place in the memory hierarchy to store something. And so that's what uh TD TPU sync is. And we'll get into a little bit more detail. But what do you think of the software layer?

SPEAKER_01Yeah, yeah. No, I think it's very important. Obviously, you know, if you're trying to sell your silicon, as AMD knows, if you want to sell accelerators, but people aren't happy with your software, you're not gonna sell accelerators. That was the rock um thing that they had to deal with like two years ago. They're pretty much past it now. No one talks. About Rocum anymore because it just works. So clearly, if you want to use TPUs, you want to sell them to external customers, the software has to be very easy to use. And to your point, a big part of LLM workloads at scale on accelerators today is disaggregation.

SPEAKER_01And then it's also memory management. And actually, I think this probably fits in nicely too with having the 8i Max. Is like, oh, some of the chips have you know a ton of HBM and a ton of SRAM, and how could you make sure that you put sort of like the hot KV cache stuff where you need it and offload some of the colder stuff to still have it something that you can access, but not as close. And it actually reminds me of the conversation you had with Val Burkovici of Weka. You guys talked a lot about this, like LLMs needing to think very carefully about having as much essentially having a memory hierarchy, and can you also can you do tricks in software to make it feel like some of the stuff like flash or something is lower latency than you might expect because you're thoughtful about what you put where and and how you fetch it in time.

SPEAKER_00Yeah. It feels like

SPEAKER_01TPU sync helps with that.

SPEAKER_00It helps with that. And the uh the blog post I was reading says that uh the amount of the KV cash transfer overhead drops by like 90%. There are some time to first token improvements that span anywhere uh from 10% improvement all the way to like 90% improvement, depending on you know various configurations and model sizes and things like that. Um also uh the the time between inter-token latency, it's called ITL, it's just the time that is taken between tokens, you know, that comes down as well with the use of TPU sync.

SPEAKER_00And there's a nice analogy in the article, which is basically like think about uh a very fast executive chef at a you know uh at a restaurant. Now the guy is like, you know, say cutting all the vegetables and cutting all the meats and uh cooking, and but he has a very small countertop. So he can't literally keep all the meat and vegetables and the food or whatever that he's cut, you know. So what he has to do is he has to cook it uh and then when the next batch comes, he has to cut everything from scratch and you know store it in the small countertop.

SPEAKER_00He needs to keep doing this, it's very slow. So the TPU sync idea is that now uh you don't have this small cutting board where he is operating. Now you you roll up a cart next to him where he can store all the food that he's cut. And if there's meats, then you have a cold storage there where you can store cut meats even if you wanted to. So everything is in its right place when you need it. So the chef only needs to get stuff from the right place. Sometimes, yes, it takes time to go to cold storage to get the meat, but it's far better than cutting the meat from scratch.

SPEAKER_00Like it's ready to go, you just have to go get it, right? So TPU sync across its memory hierarchies allows for that. So I like that analogy.

SPEAKER_01I love the analogy. And it just goes to show when you're operating these workloads at such big scale, it's very important to have the right stuff at the right place at the right time. And co-design and co-optimization from the hardware into software can make this possible. So, what I just thought when you were explaining the analogy, even it just made me think like, oh, what a cool job for some people out there, hardware and software engineers working on this, to think about like, oh, how do we make sure that the KV cache is in the right place at the right time and how can we write software to make that even easier?

SPEAKER_01Because obviously it feels very technical and kind of like a niche problem, but you know, even like inner token latency, that kind of stuff could impact, you know, millions or billions of users. Like they they would literally legitimately feel that. Of course, that's a whole nother conversation of if Google can get people to use Gemini and uh where's Google's agent and stuff like that. We won't go there. But very cool, very cool technical problems to work on.

SPEAKER_00Today's sponsor is Meter. You've got the GPUs and the power. Now, who builds the network and who do you call when it breaks? With Meteor, it's one call. Tell them your GPU or server account, and they design, deploy, and run your front-end storage and management networks. Meter builds its own networking hardware, switches and all, and writes its own network OS. So that call goes to the team who built it. Running a new cloud or clusters in a colo, go to semidope.com slash meter.

SPEAKER_00Links in the show notes. Yeah, I'll just explain maybe two use cases that are very interesting to me. Um is think about enterprise use cases. Like before a prompt is sent by any employee, right? Uh companies say they want to put a lot of guardrails and rules and everything into place. So what they usually do is they they prepend the entire prompt that you send with all the guardrails first.

SPEAKER_00So it's like you send an instruction manual of what you can and cannot do before Austin can send a request, right? Yes. Read the

SPEAKER_01every email that I send. Yes.

SPEAKER_00Yes, that's attached to every email that you send, right? Uh so this kind of stuff can be stored uh with TPU sync in the right locations so that you know you don't have to like re-compute or redo this. You have the instructions already, always, and you can always process uh requests quickly. That's one use case. Yes. The other use case is that I think uh one way to drop inference cost is to make sure that KV cache is not recomputed. Like you want to have KV cache hits as much as possible.

SPEAKER_00You want to use tokens that are already uh existing from the past or things like that. So if you have all the storage hierarchy, uh you will have a much higher KV cache hit rate, which means you will drop your inference costs by an order of magnitude uh in this environment. So it's a it's a nice, nice thing, I think. They also have a nice thing called uh host-to-host streaming, which means if you're doing prefill on a separate track, you're doing decode on a separate track, they have these like uh high-speed interconnect, not it's not interconnect, it's like a protocol that they call host-to-host streaming, edge-to-edge, that makes sure that like the entire weights from the pre-fill goes to decode quickly.

SPEAKER_00So there's this kind of stuff. Like they've thought through every part of the memory hierarchy nicely. But I feel like this is not unique to Google. I feel like Google uh NVIDIA has thought about a lot of this as well. So but it's good to see the progress, right?

SPEAKER_01Agree, agree, agree. Yeah, and I'm sure AMD, for example, thinks about this too. But uh good to see Google doing it and to know if you're asking yourself, can they compete in selling these to people? Clearly, this shows that yes, they can build this, people can be confident that they can run on top of this. And obviously, I would like to think that Google's using this all internally too, so you know, dog fooding it, so to speak, which would be uh different, and that actually could be an unfair advantage that they have compared to a merchant silicon vendor like an AMD or NVIDIA, where AMD and NVIDIA don't necessarily, yes, they do their own training.

SPEAKER_01Um this could be a reason, a good reason why AMD bought world labs, pushing into the having your own internal team to really understand the you know uh 3D world model type use case. But NVIDIA, obviously, with Nemotron, they train their own model, and it's not because they want to undercut their buyers and say, uh, yes, we're gonna try to be anthropic and be open AI. No, no, no. They train their own model um so that they can make these intelligent decisions um about how to architect their system, but just saying, if if Google's not so big and hierarchical that they don't talk, but hopefully they're the Gemini team, uh the DeepMind team, et cetera, is talking to the TPU team a lot, and so that they're using this software and they're talking and sharing all that knowledge as they're building the systems, that they're consuming this all right away, too.

SPEAKER_00Yeah, yes, exciting. When people build exciting hardware, we are excited to talk about it. So if you are uh listening to this podcast and work in one of these companies doing all this cool stuff, you know, keep at it.

SPEAKER_01Indeed. Yeah, let us come by, show it to us sometime. Have semi-doped, have the semi-dope tour sometime.

SPEAKER_00Absolutely.

SPEAKER_01Cool. All right, well, Vic, anything else on the TPU front?

SPEAKER_00No, that's it for now. We'll leave all the blog uh articles on the show notes, but yeah, you can also go just find them. They're really nice. Uh I love reading Google Cloud blog articles because they have like the best pictures and the neatest explanations, and they're kind of short. So it's very much fun to read it. I always have fun.

SPEAKER_01Awesome. Yes, I love those too. Good job, whoever makes them. All right, with that, um, everyone, thank you for listening. Send this to your friends. Uh, we appreciate recommendations and like, comment, subscribe, whatever. Vic and I read it all, so please, you know, send us your thoughts.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Jensen Huang Resigns

    Nov 12, 202632 minAudio

    Jensen Huang resigned as Nvidia CEO this morning, and Austin and Vik recorded two hours after the news broke. They argue that a board must prioritize corporate integrity over short-term shareholder value, even for a...

    Credit the people on this episode

  2. Intel's Earnings Disaster: Bad Luck, Bad Planning, or Both?

    Nov 9, 202657 minAudio

    Intel just had its worst day on the stock market in 55 years, with shares falling nearly 30%. But here's the strange part: Intel actually reported a strong quarter. Revenue grew, its data center business surged, and the...

    Credit the people on this episode

  3. 3D Stacked DRAM for Inference: d-Matrix's Sudeep Bhoja

    Nov 5, 20261 hr 16 minAudio

    The fundamental physics of data movement are forcing a shift to new memory architectures for AI inference. Sudeep Bhoja, co-founder and CTO of d-Matrix, joins Vik to detail the company's journey from a non-consensus bet...

    Credit the people on this episode

  4. Why Marvell Wants Lumentum: Vertical Integration in AI Optics With Cignal AI

    Oct 30, 202640 minAudio

    Marvell's potential acquisition of Lumentum is a strategic vertical integration play to capture the massive value optical components now unlock in the AI era. Andrew Schmitt and Scott Wilkinson of market research firm...

    Credit the people on this episode

  5. How Intel Took EMIB From Science Fiction to High Volume: Intel Foundry's Ravi Mahajan

    Oct 28, 202646 minAudio

    Intel's EMIB technology represents a scalable, cost-effective advanced packaging solution that now serves as a core strategic differentiator for its foundry services. Ravi Mahajan, a distinguished engineer at Intel...

    Credit the people on this episode