0:00And when a grandmaster is planning his or her chess move, it consumes close to 20 watts of power. The same level of sophistication and planning exercise running on an AI inference cluster, whether it is Claude's AI agent or ChatGPT's AI agent, it consumes close to 1 megawatt of power. Welcome to Impact Quantum.
0:40Hello and welcome back to Impact Quantum, the podcast where we explore the emerging industry that is quantum computing. And you don't have to have a PhD in physics, you just need to be a little bit curious. And with that in mind, I have the most quantum-curious person I know, Candace Cooley. How's it going, Candace? It's great. Thank you for asking. Today's a beautiful day. Blue, blue sky. I'm very excited. I'm sorry, go ahead. No, no, it's— I'm— it's a little gloomy here today, but that's okay because it's been like
1:10well above 95 and humid all week. The— if you're watching this, you'll notice my background is a little different. I am in Hilton Head Island, South Carolina on a family vacation, and it's been spectacular. So a little warm, but spectacular. And I just told you today, in fact, that now it's like summer gets turned off in mid-August, usually here in Montreal, Quebec, and it just happened. So, like, we're having the 70s now. So I'm in, like, the mid-70s, which is beautiful weather, right? But
1:41you're not swimming anymore, but it's still beautiful. And I'm not complaining. I'm not complaining at all. So today we have the co-founder of Escape Photonics, Vivek. We're really excited to talk with you and find out more about the company and what you're doing there. How are you today? I'm good. Thanks for asking, Candace and Frank. Enjoying the weather in California myself. So— Yeah, you get to enjoy the weather like, what, 11 months out of the year?
2:15Yeah, it's been hotter than we expected, but today it's been fairly good weather. Cool. So tell me about Escape Photonics. Escape Photonics is a company that is looking to solve the next frontier of hardware that is looking to mimic human intelligence. Really? Oh, so it's not just photonics is the story. It's It sounds like
2:46there's a little bit of AI in there. It's— everything today is about solving the quest for human intelligence and what is the most efficient hardware to build it. And today's data centers are glorified computers that are mimicking human brains. And the way we are thinking about it at Xscape Photonics is how do we make that AI brain, which is trying to meet human brain's efficiency, as efficient as possible using a
3:20multicolor laser platform. Interesting. So not just photonics, but because usually in my understanding of photonics, traditionally it's been some one-color laser with different polarization of it. But now you're mixing colors into the mix. Sounds like there's a lot more bandwidth. And if you're able to use color. That's absolutely right. I think the close— the reason I keep talking about human brain and human intelligence is partly
3:50because today's human brain consumes 35 times more power in communication than in computation. And the reason that happens is because human brains can contextualize a different inputs in the most efficient manner. And when a grandmaster is planning his or her chess move, it consumes close to 20 watts of power. The same level of sophistication and planning exercise
4:23running on an AI inference cluster today, whether it is Claude's AI agent or ChatGPT's AI agent, it consumes close to 1 megawatt of power. Yeah, there is no efficiency. Yeah, 50,000x in efficiency difference between the two. And the reason for that is the communication between the processor is not efficient at all, and it is not fast enough. And using multicolor lasers to improve the communication between the processors
4:54and memory is a way to get closer to how a human brain operates and how the efficiency of AI inference cluster can improve. Interesting. That could be big because a lot of the complaints about data centers is that they need so much power, they need so much cooling, and they need so much land. And one of the things that I've always said was there's a lot of room for efficiency improvements in artificial intelligence. Because if you look at the human brain, it's something like on a regular baseline, it consumes
5:27about 25 watts of power. Right. And that's because nature had a lot of constraints, right? Consuming calories to the point where no, no biological system that I'm aware of could possibly consume 1 megawatt worth of calorie, right? It's a luxury really only machines have. And I think also too, another example I like to use is the crow. Crows are intelligent, very intelligent. They are frequently rated close to a 5 or 6-year-old child in terms of intelligence.
6:01But they— I don't know what their calorie consumption is, but they have to pack all of that in a platform that could fly. Yeah. Right. So you have a dense platform. Yeah. You have evolutionary pressure on biological systems that thus far we've not really had in, in kind of artificial systems. So I find that interesting, and I'm glad somebody It's thinking outside of the traditional, I just throw another rack of GPUs at it. Which is how the industry is doing it because no one has actually thought
6:31outside the box like what you're talking about. And we just think the answer is staring right at your face. The human brain is all about communication. And when you focus on building a very efficient communication platform, then you end up solving that efficiency problem as well as you can. And today, all the problem solving has been focused on processing. How can I process as fast as possible? That's where the GPU investments have
7:02been coming through. And without realizing the fact that these GPUs still have to talk to one another and they still need some memory. And inter-process communication. Exactly. So the investment in that had been like an afterthought until recently. Now they just have started thinking about it. Yeah, it's interesting because like, it's funny you mentioned that because, you know, on the first watch of a Jensen Huang keynote, he'll talk about the processors. On the second— in the second
7:33watch of it, about 2-3 minutes in, he'll make a passing reference to the networking of them together, which, as I heard you say this, it's like Maybe, maybe we need to put equal emphasis at least. Yes. Yes. And now it's actually becoming more important because he talks about Moore's Law, which is the law that governs how well the processor can improve over time, is getting saturated, which means it's no longer about how fast you can
8:03process if you cannot contextualize and communicate that to the neighboring processor. And today the bottlenecks have shifted from processing to memory access and how well you can actually communicate between multiple processors and multiple memory. And that's why you start seeing optical technologies and memory companies getting all the limelight in the last few months, mainly because the realization that now
8:33I actually need a memory to think like human brain is kind of happening now. And while the solution has been staring right at our face for a long time, and the industry has started embracing that inevitability, and there is a shift in focus on how we think about the future of AI as we know it. So I'm taking all this in and my mind is blown. And so I want to dial down for a
9:04second and Just go back to the idea of what does light allow us to do in computing that electricity simply doesn't do as well? That's a great question. So today, electricity is very good in processing information. When you can switch on and off, that gets in some— that information of a transistor where you can tie, you can store bits of information in zeros and ones. That
9:34can be done extremely efficiently when you're using electricity and transistors. That's what is the basic building block of a processing unit. What electricity has limitation is transmitting that information. So, uh, it just turns out that when you have a lot of information that you are processing, you can, you can do it instantaneously with electrons. But then when you have to communicate from one processor to another processor, the— it becomes
10:06extremely difficult to transmit that information over a copper wire beyond a certain speed. And the reason for that is electrons are extremely lossy if you ask the electrons to communicate at high speed. So think of it as like it runs out of steam. when it is actually running at fast speed. So, and for a long time, the industry has been trying to figure out how fast I can communicate over a copper wire without
10:37it losing stream, losing its strength. And it has tried to navigate that problem by actually coming up with advanced circuit compensation techniques where you all the information that gets lost when you're transmitting through the copper wire, you end up recovering that information using advanced encoding technologies. And this is where technologies like SerDes technology and electronic copper wire technology came into picture. It
11:10just turns out that when you're doing that, you can only transmit— over time, the industry has evolved to a level where it can transmit, say, information at 16 gigabit per second to 32 to 100. And today the industry can transmit information at 100 gigabits per second over 2-meter copper cable. But when it goes to 200 gigabit per second on a single copper wire, it cannot transmit for more than half a meter or even say
11:40less than say 10 meters. And as the speed continues to go up, The physical limitation is electrons just cannot travel long enough over a copper wire. And this is the same problem that happened in the telecom world long time back when the industry went away from copper wire to fiber optical cable. The reason is, when, when you replace copper wire with fiber optical cable, the photons don't have the same energy loss when it is transmitting at high speed. that electrons do.
12:14So photons can carry information across like kilometers and kilometers of fiber optic cable without losing energy. So then you are not spending more energy on recovering that information anymore. So you kind of get a double boost in efficiency. Exactly. Because the original problem goes away. And then because the original problem goes away, you don't need the fixes. And what you're talking about, I know my dad worked in electronics and I mean, this goes back to the 1880s when they were wiring telephone poles. Right? Like, exactly.
12:45Every— I forget what the exact number is, but every X number of feet, they had to amplify the signal. And, you know, the, the repeat. Exactly. Those, those amplifiers break down. They have to be replaced. They have to be maintained. Right? So, like, the, the switch to fiber was a, a no-brainer, so to speak, back in '80s, '90s, I guess. Yes. Which is exactly— we are seeing a revival of that in the datacom world where the bandwidth— because now the processor are communicating, processing information a lot, but then you are not able to
13:18communicate that to the neighboring processor or a memory. So then what's happening? You are just waiting on the communication to happen between multiple processors, and it's not happening fast enough. So then your tokens are getting limited by how fast you can communicate back and forth. And now your entire performance is limited by how fast you can communicate. And that is hitting a wall with copper wires. And now with optical cables, all we are reviving
13:49is the telecom industry's invention, which has replaced copper wire with optical cable. And with optical cable, photons can travel longer. And now I can, um, and it can travel faster. So you can travel longer and faster without needing additional energy. And now you can increase the speed of communication by increasing number of colors, because every color can transmit one, like, high-speed data. So now not only can I send 200 gigabit per
14:19second on a single color, I can also send 100 such colors. So that means I end up getting a 100x boost in the speed of communication, and I also get 100x boost in the distance of communication, which means the entire data center today can be wired up with copper wire, with optical cables that can make it behave as one big processor unit rather than Chunks of small
14:52processing units waiting on one another for communication. I remember reading something about this. I don't know, maybe like a couple of years ago, they were, they were talking like, if you have an optical cable connecting CPU to CPU or GPU to GPU, right? Because you don't have the speed limits and limitations, like they could be across the room, right? Now, again, there is a latency cost because it's the speed of light. But honestly, Whether it's— for most applications, I'm sure someone will— you can correct me— whether it's across the room or like a football field away,
15:25the speed of light is fast enough that it's going to be mostly negligible. That's right. It's actually— you cannot even realize that because it's a nanosecond. So the speed of light is such that it takes only 5 nanoseconds over a meter. So if you're talking about a kilometer, You're talking about a microsecond. You might not even realize that it's taking time anymore. Right. And this is not new. The data center already realized that this is what
15:56they need as the distance increases. So they have been using optical cables within the data center too, when they're connecting multiple racks for a long distance, like 2-kilometer cable and 3-kilometer cable. What has changed with AI is the fact that even within a rack, which was all copper cable, that is becoming all optical cable, which means optics is becoming a core part of the compute. It's getting closer and closer to the processor. And
16:26when it gets closer and closer to the processor, that division of boundary between processor memory and optical cable is diluting to a point where you need to treat each, like all of them together as a single building block. So the definition of compute is no longer just processing. The definition of compute becomes processing, communication, and memory. I have a bunch of questions. They need to be designed together. Yes. I'm sorry, Candace. I don't mean to— No, no, no. Go ahead. I have
16:57a bunch of questions. Great. First of all is, so if you take out a typical motherboard, I build— if you build PCs, right? Or you open up a laptop, right? You see a motherboard. you have each of the chips and there's some kind of copper and some kind of breadboard. You're talking about replacing the copper there with optics? That's exactly right. That is the difference between what has been happening in the data center till now, where there are optical cables, but they are transmitting much lower speed, let's say 100 gigabit per second. But when you keep going closer
17:27and closer to the processor, and if you're replacing that motherboard, the copper wires on a motherboard with optical cable, now you do optical cable becomes a new motherboard. So, I mean, then, uh, then what you need is, uh, think of it as, um, you have, uh, 2 chips right next to one another with an optical cable that becomes your unit. And you can think of this as like a mesh, like an optical mesh of like a fabric of processor
17:58units and memory units that is all tied together. with like a multicolor fabric where you can essentially communicate any processor to any memory at speed of light. Effectively, instantaneously. Instantaneously, effectively with a pretty wide bandwidth. Exactly. Which means the speed of communication within an internal processor today happens at petabits of bandwidth. Petabits is 10 to the power of 12. It sometimes— And
18:31also over a wider distance. So you could have, you know, let's just, let's come up with a ridiculous scenario, right? I can get like 3 or 4 old PCs and then wire them together with optics. Obviously there's a lot of practical engineering that would, but let's put that aside for now. I could have that as one massive supercomputer 'cause they'd all work together. Interesting. Exactly. Because now I'm just like weaving them together and now the speed at which they communicate within a chip and speed at which they communicate outside the chip is exactly the same. So then the
19:04boundaries don't matter anymore. Right. So that's what optics bring to table is the distance and speed that copper cannot achieve. So now distance is a non-factor, whether there are 2 chips at the 2 ends of the football field or whether they're inside the same device, they are talking at exactly the same speed. So then it views them as you are essentially building the internet of the GPU world where you are— we are able to have a real-time conversation with Candace in Montreal and you in
19:38North Carolina. And there is no, absolutely no latency. Today, that world in the processor world is still happening through postal lines where there is a lot of, I'm still sending you a return letter and I'm waiting for a post box and USPS to like figure out where it needs to go. And I'm still waiting for Candace and hoping that she's actually, she has time to read that letter and then respond. And I'm still waiting and twiddling my thumbs until that happens. There's a lot of idle time.
20:12And this is basically, I think the easiest analogy is I'm introducing, um, the real-time video conferencing, and that cannot happen with copper wires. It has to happen with optical cables. So then the next question is, I suppose since you're dealing with photons, you can also encode quantum-level information in how the thing spins. Do you do that too, or is for now color wavelength is enough? Yeah, we are currently focusing on let's just make the industry navigate the
20:42postal era to a video conferencing era first. And then we can really talk about augmented reality version of it, which is the figuring out how can I encode qubits with these multicolor lasers. And our co-founders like Professor Alex Gater and Yoshi, they have been doing this comb laser technology for quantum applications for the longest of time. So they are very well aware of the application as it relates to
21:12quantum computing. And they do believe that the underlying platform that we are currently building at Xscape can be easily applicable there in terms of multi-qubit encoding and being able to control multiple qubits at the same time and being able to use that as a pump for low noise qubit generation. And I wouldn't claim to know anything about quantum, well enough to comment on what the
21:44technical implications would look like, except the fact that the application of comb lasers in quantum has been a deep field of research for my co-founders for over a decade. And there is a lot of interest in that field when that becomes a reality. And once the fiber is there, it can carry— if it can carry the same photon with or without extra quantum information thrown into it. That's exactly right. So you would— you're building the infrastructures, and then whatever you do on top of that,
22:14which presumably would be maybe another order of magnitude more information across from point to point. That's interesting. Sorry, Candace, I'll stop hogging the mic now. No, no, this is fantastic. So does photonics change what is computationally possible, or does it mainly let us do the same things faster and more efficiently? In today's data center, it is mainly focused on efficiency and speed. And with the applications like quantum
22:47computing, it changes the way how computing is done. And photonics and the platform that we are building for that is certainly the base infrastructure on which people can reimagine the way the computing can be done. I still think that's going to be the next wave of innovation that is going to ride on top of the existing AI wave. Would the— would it run cooler slightly? Because obviously CPU heat is going to be CPU heat.
23:18But the fact that it seems like it would run slightly cooler, right? If not chips themselves, the interconnecting parts would not heat up as much. 100%, because you are— you no longer need that many overhead to communicate anymore. Right. You don't need additional overhead on figuring out where the loss of the signal comes from, recovering that loss, and spending additional power on that. So, and you don't have to necessarily worry about errors associated with sending it through copper wire. So when you're using
23:52optical cables, you end up like removing a lot of additional signal recovery circuits that consume some significant portion of the communication power. So you're looking at roughly 10 to 20x improvement in overall energy efficiency of the system. And in the future, you can even unlock an additional vector with multicolor lasers as well. Interesting. So what
24:23happens when you combine photonics and electronics? Rather than treating them as competing technologies? The short answer is they are not competing technologies. I still think you need electronics for processing the information. And my view is today the most efficient way of doing it is to marry electronics and photonics and use electronics wherever you can and use photonics wherever you must.
24:54which is in the communication of multiple electronic circuits at high speed and still rely on electrons for processing information. Even when you're looking at quantum, you still need quantum error correction and additional circuits that are still electronics-driven. And electronics are something that is extremely efficient when it comes to doing math operations and doing complicated signal processing techniques. So
25:27that's where the strength of electronics lies, and I will continue leveraging electronics for that, which is information creation, and using photonics for information communication transfer, and using memory for information storage. So I do think each of them have their own strengths, and we just— the ideal platform should use the best of all 3 of them. That makes a lot of sense.
25:58So if AI keeps demanding more computing power, where do you think light becomes essential rather than simply advantageous? Today, at this point of time, the inflection point of light becoming essential is happening And we are living through that transition phase as we speak today. The— it turns out when you are processing large
26:29context information and when you are trying to do reasoning applications where the model has to think through different scenarios, contextualize different scenarios in its head, and think about what is the right optimum solution for a particular query. What the AI model is actually doing is mimicking the way humans are thinking through different scenarios and
27:00thinking about what is the right answer and what should I really respond and how I should respond to a particular question and why. And that becomes more and more complicated when you are getting a lot of multimodal inputs. What I mean by that is I'm, I'm watching you speak. I'm also hearing Frank's feedback about certain technology. And at the same time, I'm also thinking about what does the AI model would look like under these scenarios.
27:32And then I'm coming up with my response. And when the level of sophistication and the context continues to grow. In other words, as the AI model is going to grow from being a toddler to being an educated scholar, the experience that it is going to get and the context it's going to continue to have when it is responding is going to evolve. And when that is happening, it just turns out that the efficiency with which it is able to recover information
28:02is becoming the key factor that makes light essential rather than a nice-to-have. And the right figure of merit of how to actually evaluate it comes from thinking about how many tokens per second, which is the way AI models respond, am I able to generate? Like, how many tokens per second am I generating and how much of calories am I consuming, which is the megawatts? So that tokens per second
28:33per megawatt today is, say, 10,000x when you are thinking about very short context length. If you are just asking what is an apple, then it can do it at like a speed of like 10,000 tokens per second per megawatt. But when you are asking it to process a certain slide deck and asking you this, asking AI this question, then it is generating tokens at 10 tokens per second per megawatt. So as the context length is going
29:06larger and larger, the agents are not able to reason out in a timely manner that is economically viable because it cannot continue to consume that much power when it is providing that response. So the key is for this large context application, when you want the AI robots to behave like humans, light becomes essential. It's no longer necessary. Like, it is a fundamental building block that needs to be factored in
29:37when you're designing these systems. But if you are really just focused on one-off, like, questions on what is an apple, what is a mango, and describe the weather today, those are all, like, much smaller context, simple applications where you probably don't need light. So it's really a matter of how smart you want the AI to be. Light makes it way smarter and way more intelligent. If you just want the models to be like a toddler, then you don't need light anymore.
30:10Interesting. Yeah, because I think that we're— as we go push the frontier models and things like that, we're really running— we're outrunning our ability to improve infrastructure. in terms of the chips. And this seems like would be a good solution to that problem. Yes. Fundamentally today, that's why the optics industry is becoming a hot topic of conversation across the entire industry, mainly because it
30:42is supply constrained and the demand is outgrowing supply at a pace that the industry is not able to meet And that's one of the reasons why the data center power consumption is becoming unsustainable. So there is no smoke without fire. And the way the industry is handling it is just throwing more data centers and inefficient cores because at the end of the day, I can only process 10 tokens per second, but I still need to process 10,000 to get to a revenue. How do I do it? I will just buy 1,000 more GPUs.
31:16If I'm buying 1,000 more GPUs, now I don't have supply. Like, who's gonna make 1,000 more GPUs? I need to install more foundry capacity. So now it becomes a supply-constrained world where to meet an inefficient ecosystem design constraints, you have to throw money and supply at it. And then you start hitting the supply-constrained world that the industry is currently navigating. So what advances in optics have made photonic
31:47computing more practical today than 10 years ago? The ability to process light-based devices, the devices that can process light encoding and convert electrical data to optical data. All these devices traditionally were manufactured in a very discrete, small volume manufacturing line. Today, the industry can process and manufacture all of them in a
32:1812-inch wafer, which is a silicon CMOS wafer that the industry typically uses for electronic consumer industry. So if you take an iPhone or an iPad or even a Pixel phone, the chips that go inside it are manufactured on a 12-inch CMOS wafer. And that's why they were able to bring down the cost and increase the supply and being able to meet the demand of the
32:48entire world. The photonics industry was never able to do that until like a few years ago when we demonstrated the ability to manufacture these things on a 12-inch wafer. Since then, I think there has been a clear inflection point in the ability to drive unit economics down and drive volume up to a point where photonics has become more viable at scale, and it unlocks new applications like photonics computing as well.
33:26Interesting. How do you actually use the properties of light to perform computation rather than simply transmit information? Yeah, that's a good question. Today we use it for only transmitting information, but the way to think about how do you, uh, how to convert that for computation is to think about how, um, what are the building blocks of computation would look like. The first building block is
33:58zeros and ones. Can you actually process zeros and ones? And the— in the analog world, like when you're talking about light, the way to think about the equivalent of it is whether when there is light, there is 1, and when there is no light, it is 0. So the equivalent of zeros and ones is literally light and no light. So you can make a device that can transmit light and that can transmit no light. And depending upon that, you can actually process and convert that as a compute
34:32element. Interesting. So, there are devices that you can make where you can actually send in signals, and depending upon the signals that you send, either electrical signals or just optical signals, when you're sending in optical signals, depending upon the interference between these optical signals, you either get a response out, which can either be a 0 or a 1. And you can use that property of interference between 2 optical signals as a compute element.
35:06Interesting. And this is very similar to how, at the end of the day, everything is a wave. Photonics is nothing but a wave. So if you are essentially taking 2 streams of information and sending it through a through a device that can take both of it in, the output of it is essentially a compute output. And the industry has been looking at that for quite some time. Those are like some of the fundamental building blocks of what is being used in communication as well, and also what is being used in quantum
35:39as well. Just with the right constraints and right inputs, you can also make it as a compute element. I remember reading a science fiction short story in Omni magazine. So shout out for anyone who remembers Omni magazine. as a kid, and they were talking about— it was set, you know, some cyberpunk type future on it— was they were basically saying like, you know, well, if you use light instead of like electrons, interference is different. You have a lot more efficiency. You can pack things more densely. And it was an interesting concept. And
36:13I wasn't sure if it was still science fiction, but it sounds like it's less science fiction today than it was when I was a kid. 100%. Yeah. I mean, things are moving in that direction. And it's just the max linear equation, Maxwell's equations have been around for a long, long time. And the industry has just not figured out how to make it efficiently at scale and how to leverage the properties of those. And now we actually have a way to unlock that. That's always the research and the development, right? Like some
36:46physicist comes up with some crazy, figures something out crazy. And then it could take like 50 years or 100 years to figure out how to, how to engineer that in a practical way. Yeah. Yeah. Yeah. Interesting. And it's really just figuring out, realizing at the end of the day that all these are just zeros and ones. Once you realize that the way to think about zeros and ones is just, and you add colors to it and you add circuits to it and you, you do math around that. But the
37:19way computers do math is very different than the way we do math because everything is zeros and ones. So you just have to line them up in a way where it meets the same outcome as we do. So do you need new network protocols defined? Like, is there gonna be like an 802.11 something? Like, or does it already exist? Like, how much of this— so obviously the hardware stack changes, But how much of the other higher-up stack? I'm thinking transport layer stuff, like what else has to change because of this? Yeah, I mean, every—
37:52today the electronic world has like a certain stack, and then there's a quantum world that has a different stack. And I think the photonic computing world will essentially be an intermediate between them. So there will be certain level of changes, like you might end up reusing similar hardware in the electronic world. But then you will end up like bringing in photonics closer to the electronics and make it look like a photonic computing hardware, which means there are devices that is— that can do both processing and
38:23communication and memory storage, all with optics. And in the future, that will get morphed into all optical, which is for quantum computing. So I do view The transition from the current world to the quantum computing world is where the photonics computing is going to act as that intermediate step to get us there. So in terms of the stack that has to change, it's likely layers that are on the software protocol
38:55level, like how you write the software, how you write the communication between them. There's a chance that the communication transport layer might not change, But there is likely that the coding, the control logic, like which information goes where, those need to be like rewired because you are physically training your brain to communicate. You are putting the structure in place saying that when you have this calculation, go there. When you have this calculation, go
39:25there. Which today everything is done electronically. by encoding a packet header. What I mean by that, think of it as, um, an address that you write on every data saying that this is the address you need to send this particular data to. And an electronic circuit reads that address and says, okay, if this came from GPU 1, I'm reading this address, it says I have to send it to GPU 1 million. So let me send it
39:57out, let me figure out what is the path I need to take. Do I need to catch a flight to this particular switch? So it's usually not the same. It might not connect into the same country, right? It could be like the analogy that I'm drawing is a million GPUs within a cluster can be connected via multiple switching layers. And you can think of those switching layers as terminals. that an airport would take. So the message has to first go to one switch, one terminal, it processes it, then it says, okay, I'm
40:30currently in Newark. Now I need a way to go to, say, Belgium. I could either take a direct flight, but then I have to wait for one more day to take the direct flight, or I could take one stop to a different flight. So it optimizes based on the header And, um, and constraints in the address also tells you which, um, what is the latency constraint you have to follow. So you have to follow the protocol of that communication.
41:02So, um, then it says, okay, I'm going to take single hop, or I will take 1 stop or 2 stops. So every information that you do or think through when you're optimizing a route through kayak.com is what a switch is trying to think through in terms of where it needs to send the packet and how it needs to send the packet. Now, it's like IP routing tables, so to speak, right? Yes. It's going to give you the shortest path or it is going to optimize for something else. And then eventually it goes there.
41:32Now, if you go all optical, this electronic packeting layer is going to get replaced with optical circuits. Now, you still need that intelligence somewhere. And that intelligence will end up happening on the node level where the buyer, like, so instead of like relying on a switch or like an airport to figure out what is the fastest path, as a sender, I need to figure out what is the fastest path and pick that route and send it out so that I'm
42:03guaranteed that it is reaching the destination. So think of this as like 2 different methods of how do you communicate communicate when you're communicating to optics versus communicating through electronics. You just, so it's essentially like coming up with communication protocol is really like figuring out how to connect point A to point B and who controls it. Yeah. Is it the airport? Is it the networking layer that is gonna control it? Or is it the sender that is gonna control it? So then you get into this philosophical discussion
42:33on who should actually own it, right? As an end consumer, would you rather rely on yourself to come up with the best strategy to reach point B, or would you rely on someone like kayak.com to tell you, or would you go to a different one, right? So then, so this is the same thing that the analogy to what is happening in the data center today. And what is the most energy efficient way of doing that? That's an additional constraint. And who owns what piece of the stack and what piece of the value chain?
43:05And those are all the discussions that are happening as we speak, which is the control layers, the transport layer, the communication between multiple GPOs, the communication between one device to another device, multiple vendors. So all these conversations is what the hyperscalers are actively having with vendors like us and end users like— Anthropic and OpenAI before they have finalized the right solution for every generation. And that's a good point because,
43:39you know, because of the particulars of deployment, there are probably people who have no experience with AI, so to speak, working on these types of hardware and routing and switching problems, which I think traditionally we would have called just networking kind of— not network engineering, more network design. Or protocol definitions, right? There's probably a better title for it. Yeah. Like the people who decided what Ethernet is, right? How Ethernet should look, how it should work. I suspect that those conversations in the standards bodies
44:10are working on that now. Exactly. And it is happening. And now, to your point, until now, that has been a different conversation from who decides to make processors because they view exactly the way you view it, which is like, oh, it's just a networking problem. Let's, let's actually like throw it over the wall. Let me just make the most efficient processor that can process a lot of information. Now the conversation has shifted from being a device and networking, like separate buckets and combining all of
44:41them to a cluster-level definition, saying that it's no longer your problem versus my problem. It's, it's an entire cluster-level problem where networking is part of this decision. Because networking owns which processor is going to talk to who. And today it just happens that networking choice is going to govern how fast you can think. So now if tokens per second is determined by networking performance, you can argue that networking is the new compute.
45:12So that's why the hyper— the conversation is no longer a separate conversation with a processor, separate conversation with networking. With the AI world, this entire conversation with networking, memory, and processing is happening at the same time. And the winners have been those who have actually treated them as a single entity, like NVIDIA. Sure. NVIDIA actually designs the entire cluster as an entire system. And now to your point, Jensen no longer talks about GPU roadmap alone.
45:42He starts off with NVLink cluster. He talks about NVLink as a separate business unit that is generating certain revenue, and that has been growing 200% year over year. And the networking component of the business is outgrowing the processor component of the business for, in certain, in certain companies. So what you start to realize is when the distance becomes, when you're starting to design an entire cluster as a unit of compute, it's no
46:12longer chip boundaries, no longer geared towards what you're designing on silicon. Chip boundary is that entire data center, and everyone involved is part of the chip design. And that is creating this huge complex web of decision-making that can potentially delay deployment cycles. And that's where vendors like NVIDIA have a huge advantage because they're fully vertically integrated. where they're not relying on so many vendors to come
46:44together and arrive upon a decision before they can actually agree and handshake and start the development. Right. So, right. So the vertical aspect of integration. So yeah. So it sounds like the future is a lot more discipline, multidisciplinary. Yes. Kind of organizations have a significant advantage on this. Exactly. Yeah. I remember It must have been the keynote back in March for one of the GC, the big GTC in San Jose. And he had
47:15said InfiniBand, I think, or something like that, whatever it's called. But he had said what the bandwidth was on their networking stack for their cluster. And he said, like, you know, it can handle all the network traffic for the internet, entire internet for like 2 minutes or something like that. And I was like, that sounds almost impossible. But now I can kind of see, like, Maybe it's not impossible. And I also struggled with why would you need that much bandwidth on a particular cluster? And then now this makes sense. Yeah. Today, within not the— they are
47:48absolutely right. The given cluster, one rack unit communicates as much amount of information that the entire internet has communicated over the last 20 years. Wow. Yeah, it sounds unbelievable, but Yes. Yeah. Yes. That's why it's like, I mean, the volume of information is just going through the roof to a point where you have to really rewire these cables to make it really efficient. Otherwise, you are just going to continuously
48:20dissipate a lot of power that is going to make it unsustainable. Yeah. Wow. Yeah, I suppose for hyperscalers in these types of businesses, This is no longer a nice to have. I mean, this is straight up survival. 100%. Yeah. I mean, it is— there are multiple narratives of the data center power consumption story that you must have read and seen. But there is certainly one report from McKinsey that I find to be closest to reality, which
48:51is predicting close to 10 to 12% of the entire US power being consumed by data centers by 2030. And I actually think it's on, it's on target, like it's actually on course for that. And that's a lot of power. 10 to 12% is a lot of power. I mean, it's 1 out of 8. Yeah. I mean, yeah, yeah. It's, um, I think Bloomberg, somebody on Bloomberg was saying it was going to be 25%. Like, yeah, I wouldn't be surprised. Like 20, 35%. Yeah. If they
49:22don't fix it soon, power is going to become a premium and, uh, We all might have to revisit our plan to Mars, right? Yeah, seriously. So, but I do think efficiency is the next frontier of innovation. That is a must-have, not a nice-to-have, where you basically treat power as a hard constraint and say, I'm not going to give you more than 10% because we have people to feed and we just cannot afford to give more than 10% of
49:53US power for these data centers. And imposing some hard constraints as a budget would force innovations on efficiency rather than— rather than taking it for granted that we just have to figure out a way to make more power. Well, it was always, it's somebody else's problem. It's somebody else's problem. Eventually, you kind of run out of— Until you start seeing power fluctuations in your house because there is a data center close by. Well, there's always controversy around
50:24electric bills going up, and sometimes that's not always the data center's fault, right? Sometimes there's other reasons for that. But, you know, certainly they are getting a lot of the blame, and ultimately it's going to make it harder to get a data center approved, right? If the— Yeah, 100%, I think. And it is also, I think it's unfair to limit the innovation and limit AI access because there is no power. So, um, the, the real solution is you cannot stop people from asking, uh,
50:55um, uh, uh, them to use AI applications because once you start using them, it's like using internet. So once you start using internet, you are not gonna go back. You're not going back. Like I, it, at that point of time, it becomes a part and parcel of your life. So the question is, uh, tomorrow if we just said the only solution is stop using your iPhones and stop using internet, people are not gonna like that either. So I almost think the real solution is how do you continue innovating and providing performance improvement and enable these AI applications
51:29without consuming a lot of power? Interesting. While staying within this. And there are a lot of solutions there. I certainly think to your point, human brains consume only 25 watts of power. And we have 300 million humans in US, if I'm not mistaken. So it's certainly like, so all I'm, trying to get towards the calorie— the power consumption of these data centers can be brought down drastically, and optics is a way to do that
51:59for sure. And the industry is starting to recognize that, and certainly there is a lot of progress that is being made in that direction. Interesting. I could talk to you for another hour, but I want to be respectful of your time. Where can folks find out more about you and about your company? We have a website, escapephotonics.com. where we talk about our laser products for data center application. And they can certainly join us. They can apply for job openings there or just reach out to us on
52:33any product briefing that they would need or just reach out if they just wanted to chat. Excellent. Fantastic. Awesome. Any parting thoughts, Candace? This was fantastic. I'm telling you, it was a brilliant conversation. Thank you so much for your brain. It was great. It was awesome. I won't look at fiber optic cables quite the same way again. Totally, exactly. We'll let the outro music play. Awesome. Thank you.
53:24The multiverse is skanking, skanking in time. Black holes are wailing in a horn line so fine. From Planck scales to planets, they're connecting the dots. Candace and Frank, they're the cosmic hotshots! Quantum hotshots! Turn it up fast. Candace and Frank blowing my mind at last. Quantum Podcast, they're breaking the mold. Science has got beats. It's bold and it's gold.