Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

ADSP: Algorithms + Data Structures = Programs

ADSP: Algorithms + Data Structures = Programs artwork

Episode 192: Systems Programming & More with Kevlin Henney

Jul 26, 202434 minAudio

In this episode, Bryce chats with Kevlin Henney about systems programming and more. Link to Episode 192 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Twitter ADSP: The Podcast Conor Hoekstra Bryce Adelstein Lelbach About the Guest Kevlin Henney is an independent consultant, speaker, writer and trainer. His software development interests are in programming, practice and people. He has been a columnist for various magazines and websites. He is the co-author of A Pattern Language for Distributed Computing and On Patterns and Pattern Languages, two volumes in the Pattern-Oriented Software Architecture series, and editor of 97 Things Every Programmer Should Know and co-editor of 97 Things Every Java Programmer Should Know. Show Notes Date Recorded: 2024-07-11 Date Released: 2024-07-26 Kevlin Henney ACCU 2024 Talk Intro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons - Attribution 3.0 Unported - CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

BryceLike I I can I can dream in Xenophy Assembly.

ConorWelcome to ADSP the podcast episode 192, recorded on July 11th, 2024. My name is Connor, and today with my co-host Bryce, we chat with Kevin Henney in part three of our five-part series about systems, programming, and more.

BryceUm sorry, I gotta uh this uh the the dog was throwing up this morning, which is why I've been I was tasked with keeping her in my eyesight, which is why I've been. Uh-huh. Right.

Kevlin HenneySo this is I was just assuming, given that we were uh, you know, we're we're we're all on Teams here, for those of you listening in Parcast Land. And um and so therefore, I always thought it was compulsory when you are uh you know you're on Teams, you're on Zoom, or whatever. There has to be an animal. And I just thought that's I thought the price was was okay, it's my turn. I you know here is here is the uh uh the meeting dog. Okay.

BryceSo I would do that for a few minutes, but in this particular case, I if I see the thing is she's so quiet, if I didn't sit here on the bed with her, she would wander off and potentially throw up again, and then I wouldn't know about it, and then I would get in trouble with mom. So you gotta stay here with me. I know you would like to go elsewhere, but sorry. Anyways, the fundamental question. Fundamental question. Does software get better with time? Like, like, or sorry, do we get better at software over time?

BryceAre things better now than they were and will they continue to get better?

Kevlin HenneyI think that's actually more than one question, interestingly. Although you're offering different perspectives on it. I think um I think collectively, we as individuals and as developers and development organizations, I think we do get better. I think there are a number of things that are taken for granted now and things that people just do that they never realize were a struggle, that they never realized they that you know was a thing to worry about. In fact, uh I I in digging around, I did a did a talk at um the ACU conference um in April, um basically uh on data abstraction.

Kevlin HenneyI wanted to kind of focus on the history of that, but also to bring it up today. And one of the things I looked at was Barbara Liskoff's I've been a fan of Barbara Liskoff's work for for many years in terms of language design, the clue language in her original work on data abstraction. But I'm nerdy like that, so I already knew this stuff. But I saw a couple of her talks, and she said, yeah, when she was given the um Turing Award, I think 2008, um one of the um one of the things that uh one of the critics online said is why is she getting an award for this?

Kevlin HenneyEverybody knows this stuff, and they know it because she did this. You know, prior to 1973, 1974, data abstraction was not a thing. Um, and pretty much everybody that, you know, uh although there was an idea of object orientation around her work on abstract data types, um, had the greatest influence, particularly on statically typed languages and the stuff that Bjana later did, and some of the evolution of it. And and everybody just regards it as like, yeah, yeah, of course, it's always been like this. It's like, no, it hasn't.

Kevlin HenneyThat somebody had to do this and had to do this. And so people are able to build on a lot of features in languages, in libraries, in ecosystems, but also in the way we disseminate knowledge. Um, I think you know, if we we can kind of okay, slight s slightly egotistical from the point of view of the uh the developer view. So collective egotism is a thing. Um, a lot of things have happened as a result of developers wanting to communicate. Um the it the internet, uh, email.

Kevlin HenneyI mean, literally, it is developers trying to talk to other developers. How do we communicate? So uh usenet and therefore ultimately all forms of um uh uh sharing of data. Um, open source, the idea, oh guess what? I've got a piece of code. I wonder if there's a standard way we could share that and others take advantage of it. How can we share our knowledge? Well, tell you what, let's put this on a place that we can all get to. It was FTP servers and then it was World Wide Web, and then it was just like it's just that we're very good at solving some of these problems. Our ability to disseminate knowledge is actually really good.

Kevlin HenneySo I think in the 2020s we have we are definitely building on a lot of benefits that uh somebody from um uh former eras will go, like, wow, you know, you you guys have really got it together. But when they look closely, they might be also disappointed by, yeah, you're a bit better, but not as much better as we thought you could be, given all of this. So I think that's the thing. I think that we are we have a benefit, but also we are not in a position where we're making the best of it. And then what is what's the statistic?

Kevlin HenneyI don't know, it varies, but half of all developers have been developing for less than a year, it's something like that, you know. So the influx of people is huge, and then trying to communicate this. If we if we were if we were uh slightly older and stuffier, and that was the whole of the demographic, then actually uh we might we might have a different story. But the point is that we've got a lot of people who are coming into development and they don't have the backstory, they don't have the history, they don't see all of this, or understand why such a thing is a big deal.

Kevlin HenneyAnd and so therefore they are not in a position, they're learning for the first time. And so, therefore, most developers are on a on the on the earlier part of their learning curve. And I think that's the that might characterize our profession perhaps in a different way to many other professions, where there's much more of a steady state. I think we we're operating much more on a power law. So so that's that's what may prevent us from taking full advantage of all the benefits that we actually have created.

BryceBut there are also all of that, all of this progress, all you know, this greater interconnectedness, it comes with downsides too. Um like like take one like security, like in this new bold world where we have all this open source code and everything's connected, like one, like when everything's on the internet, like the internet itself like opened up like these attack surfaces to like almost all the digital technology. And things like things like um it's okay somebody at the door, particularly.

BryceUm things like open source and package managers have opened up supply chain attacks. That's like a whole new thing that we have to think about that we didn't have to deal with before.

Kevlin HenneyYeah. And I think I I think that's an important that is an important thing because yeah.

BryceI'm gonna have to this may be my fridge, which they were supposed to deliver a new fridge yesterday. And there's a whole story about why we have to get another new fridge because the fridge that we had was a new fridge, but I'll be right back. Talk amongst yourselves.

Kevlin HenneySo I'm gonna pick out the point of the price made there. That that whole I think that uh and we've seen it on the people front as well. Um, you know, it's not it's it's it's there's a social implication here as well. When you start increasing connection, there's a whole lot of benefits. There's a lot of really good network effects, but it's also an amplifier for other things and it creates new possibilities, and security is one of them. And the the the the language or the focus on security in modern development has shifted hugely, even in just the last um uh even just the last five years.

Kevlin HenneyAnd that that is that's not going to change, but that comes as a consequence of um yeah, connecting everything together. But the assumptions that we have, it's really interesting. So, yeah, do you remember I mentioned I worked in the electricity industry? And that was in the 1990s. And uh you know, one of the things that uh we were working on SCADA systems, um, so supervisory control and data acquisition systems that sit in substations, they monitor stuff. Every electricity network has these things. Okay, so you've got all of that.

Kevlin HenneyAnd we were developing one of these, and we uh we were literally communicating between you know uh between um substations and um uh kind of headquarters using wet pieces of string. We you know it was kind of like modem dial-up speed. Wet pieces of string. Wet pieces of string. Yeah, modem dial-up speed, okay. Um so therefore, for us, having a compact um representation for our wire-level protocols was really important because you don't have, you know, we we that we worried about bandwidth in a way that um people now simply wouldn't understand.

Kevlin HenneyUm so we worried about that, that was our one of our main things. It's like, okay, it's got to be really compact. And then the question of security came up, and it's just like I remember uh pretty much the decision was well, we probably don't need to worry much about security because nobody would be stupid enough to put their um uh their you know their fundamental infrastructure uh onto the internet where it could be attacked. Um, we're already aware of worms and and and other issues, and it's just like, yeah, but you wouldn't put the electricity grid online, would you?

Kevlin HenneyNo, not not so it's accessible. You'd make sure that was yeah, now yeah, that that's that's charming from the modern perspective, um, but also it does portray the assumptions that we had, because our driver was, well, the minute we start using any security, um, then that's gonna involve encryption, that's gonna that's that's potentially oh, that's extra stuff. If we were to use um uh secure sockets, that actually we lose a whole load of bandwidth by doing that. So we uh having made everything really, really compact, uh we'd immediately lose that, and there's a whole load of issues that we'd have with that were performance related.

Kevlin HenneyNow, we uh we took those decisions um consciously, and hindsight suggests you know what, it turns out people do put this stuff on. You know, the stocks net worm was an example of the fact, yeah, we can we can you can disable another country's nuclear power stations uh via the internet. Um consequences of connectivity, that connectivity went even further than people anticipated, and that you know there are all of these issues, but it does mean everybody now has to worry about security in a way that they were only paying lip service to, even only five or ten years ago.

Kevlin HenneyUm, it's one of those things of like yeah, everybody's concerned, your language has to be safe, you're you you're you're top top to bottom, your whole stack, we have to know what's going on. Um, because all of the the fact is that most of your software build is not yours, it comes from other places. Who knows what's there? Um, and we're seeing that um that has changed the landscape of what people are wary of, but it's also changed the way they build, and yeah, you're gonna get one with the other. Um but that it I think is a that is an important consequence, and we're gonna see that for for pretty much any language.

Kevlin HenneyThat again, if I if I enter the uh if I enter the language space today with a new language, you can bet, you know, 20 years ago nobody would have said, so tell me about security and memory safety of your language. It's not likely that would have been a big issue, but now that's gonna be one of the first five questions they're gonna ask. Hey, you've got a new programming language, tell me about this. How do you do that? So the priorities have shifted, uh, which is obviously some languages in a better position to take advantage of that or to be able to say, yep, already sorted. Other languages, oh, we have to do a bit of catching up.

Kevlin HenneyUm, but that has now become such a concern that if you were creating a new language that you wanted people to take seriously, you have to answer that question almost on, you know, uh uh, you know, if if almost on by the time you've read the uh the first half of the landing page for that uh uh for that language, you've got to have answered that question.

BryceUm, it's interesting how Yeah, yeah, it's exactly the the the priorities and also the constraints have shifted. You know, if you think about it in the 70s, the 80s, the 90s, um we were far more heavily resource constrained. Um there were there were computing problems where um you know even if you had all the money in the world, um you couldn't necessarily solve them. Um and you know, like even if even if the power company had had uh the money, they may not have been able to get um the sort of connection between substations that they would have that the programmers would have wanted because the technology may not have existed um for that.

BryceUm whereas so we we used to computing was uh like started off in this era that was very heavily resource constrained. Today we are much more cost constrained. And that that that cost can be either um on the uh the actual like dollar cost for compute, like okay, like if you know we we have an unlimited amount of compute, we can get as much from AWS as we want, but you know, it'll cost you. Um and the the cost can also be on the power side.

BryceLike, okay, like you could do a whole lot of compute, but if you do a whole lot of compute, you're gonna drain the power on your phone real fast. Um and so those are like it's much more of an efficiency uh uh more than resource constraint uh limits us. And then also we have this you know safety, security, and reliability concern that is much uh more of a uh a priority than it than it used to be. Um and I mean may maybe it's that like all of the things that we deal with today, like the cost and the um uh you know energy uh uh concerns and safety, maybe those would have been more of a priority back in the day, but they just weren't because you know there were bigger problems like you know, oh, we only we only have this much memory, you know.

BryceWe only we only have this much bandwidth.

Kevlin HenneySo yeah, I I but I think that that kind of speaks to something else, is that we're always going to hit some kind of constraint. And I would also argue that um power is itself a resource. If anything, it's the original resource. Um I and I love the fact that you know my um the the the mobile phone, I've got a Samsung, um it's got better battery life than my previous Samsung, and we're now actually almost still not quite at the levels that my Nokia in the mid-90s had. Yeah. In other words, you know, in the sense of being able to, how long can I go without recharging?

Kevlin HenneyUm you know, it's we hit different boundaries at different times. Um, and as you said, memory was an issue. In fact, there's been some really interesting cases where we've ended up with this curious mismatch, and we saw it um with um the PC at one point, suddenly you've got all this memory, but it's not addressable because it's a 16-bit system and you have to do weird things. And then we kind of got over that, and then we hit in the 2000s, we hit that issue of like, well, yeah, I've got loads of memory, but I've only got 32-bit addressable. Um and uh and then it's a case of like, well, maybe actually it's cheaper to run things across multiple machines.

Kevlin HenneyUm so you've got the kind of the Hadoop kind of approach of like, yeah, let's just break this break this computation up, scatter it across the network. Um, and then it's a case of people saying, Well, actually, if you could it's always faster if you can put it on the same machine. We've got 64-bit addressing. So that the I worked with one team, their limit, they'd hit the 32-bit limit, even though their machine had clearly a lot more than four gigabytes. Um, and so therefore, we had to solve the problem with various optimizations of compacting, you know, ch really, you know, messing up the C data structure to kind of like, oh, we're gonna save a bit here and a bit there, so that when we multiply it up to large data, um, so not big data, but large data, it it will still fit in memory.

Kevlin HenneyBut also the other option of one of the other systems they used was like, uh we're gonna have to run it across separate processes because that's the only other way we can take advantage of that. You hit 64 bits, suddenly you can address it. But now we also start hitting the other limits. Each time we do something, you hit the other limits and go back to connectivity. Um, we hit light speed as an issue. You know, it's that it's one of those things that, you know, it's it's um it's 20 like milliseconds between London and New York. Um so if you've got a trading system, you know, if you've got a trading system and somebody says, oh, we need to have New York and London in sync with respect to 10 milliseconds, you need a Nobel Prize because you are not going to break that limit.

Kevlin HenneyThis is one of those laws you don't get an option on. It's not a you can't open up a config file and say, you know, I'm just gonna change the speed of light today. I think it should be faster. Well, let's let's reboot the system with a faster speed of light. So we found and and this the light limit is genuine in the sense of um the the systems we're working with. It's um not I'm not just talking space travel and stuff like that, because that messes with a whole load of protocols and timeouts. You can't have a TCPIP connection to the probes on Mars because you'll have timed out with those. But even if I've got something like a geosync satellite, um that incurs a round trip of over 100, 150 milliseconds.

Kevlin HenneyAnd suddenly, ooh, that's really slow. Um, that's noticeably slow for certain classes of system, and even for phone calls. So we're always gonna we're gonna build a system and then we're gonna hit the limits. And as you say, there's there's there's power. Um, we're hitting light speed limits on a number of things that actually prove to be impressive.

BryceYeah, it it's it's not just an expectation too. It's also on the the um on the you know how how many transistors, how densely can I pack transistors? We're starting to reach the point with process technology where there's just not a lot of headroom for us to make smaller transistors. We start to to reach a uh uh a scale, we've started to reach a scale where uh just the physical limitation. Yeah, quantum effects are becoming it says just like no, like this. Yeah, you cannot build smaller transistors.

Kevlin HenneyAnd that and that's the point, is that the and therefore we have to kind of like, as it were, kind of squeeze the toothpaste in a different direction. Um multi-core, etc. And I think that um that's the point is every every era is going to hit its own limits and regard something as normal and given. Um and uh so I did uh so I mentioned Hadoop earlier on, the the whole idea, yeah, the MAC Reduce architecture, the idea of pushing stuff out onto the network, just parallelising the task. Yeah, we'll break it up into small tasks and do that.

Kevlin HenneyAnd I I highlighted um something that um something to somebody I was running a workshop um running a workshop for a company, it was uh when was it? About 2017, and I made an observation to uh uh to somebody, I said, okay, so you know, we could break this up and look at this. And then I said, look, somebody just wrote a script to solve this problem, and they they they did it in memory, and it was based on a blog in 2014, and they said they did it all in memory, whereas historically they would have used a Hadoop solution, and they'd have said, Okay, we're gonna use a compiled language and we're gonna spread it across the network.

Kevlin HenneyAnd they said, Yeah, they just used a scripting solution, and because it was all in memory, it was it was over 200 times faster than the so-called fast optimal uh solution. And I said, that's simply down to things like the speed of light and you know, or you know, the the cost of the network. And I had one person in the workshop say, Oh, that's because we you know our architectures are better now. And I said, I want you, you know, we we know more, going back to this idea. And I said, Yeah, you know, that that's not a problem. I said, actually, had you couldn't have done the solution 10 years in the past, it was not available to you, because you wouldn't have been able to address that memory.

Kevlin HenneyUm, uh it simply wasn't available to you. But I said, and he said, Yeah, we we know about we're doing good architecture now. And I said, I want you to write a letter to yourself, send yourself an email in 10 years' time, and tell you, describe to your future self what your current architecture is, and then your future self will laugh and say, Oh, you thought you had it all worked out. And that's the problem, is that uh it's not that we're getting some things are improving, but other things we're just chasing the horizon. We're always gonna be chasing the horizon. The shape, the the problem space moves.

Kevlin HenneyUm, and what one generation thinks, and so going back to the fact that a lot of developers are coming into the industry now, they think there's a whole load of things that are normal. And in 10 years' time, they're gonna go, well, that's not normal anymore, or what I assumed was normal is now weird or antiquated. Um, and I just dug out this wonderful quote from Douglas Adams. Um, and he said, I've come up with a set of rules that describe our reactions to technologies. Um anything that is in the world when you're born is normal and ordinary and is just a natural part of the way the world works.

Kevlin HenneyAnything that's invented between your uh when you're 15 and 35 is new and exciting and revolutionary, and you can probably get a career in it. Anything invented after you're 35 is against the natural order of things. Um and and the point there is that this is also, yeah, this is a social thing and uh you know, uh and causes uh uh older grumpy people to go around and go, oh, in my day, you know, just like, oh, this is unnatural, this is unnew, this is new and fangled and is not necessary. But being inside the technology space, I think this is also quite interesting, it's probably on smaller timescales, um, that a lot of things that people regard as a given somebody else had to fight for, but also in some time in future, you're gonna have to recognize is either part of the furniture, very much given, or is actually no longer relevant.

Kevlin HenneyUm, because not because we've progressed, but because we've kind of moved up and sideways. We're constant, constantly moving up and sideways. If the if the problem space had remained fixed, we'd always be moving up. But we're moving up and sideways, we're we're fine, we're hitting new boundaries, we're hitting new constraints. Um, every time we solve a particular problem, somebody creates software in a different way. You know, and I think that that or we Use software in a different way. Going back to the beginning of this, where we talked about connectivity, the fact that we're also connected changes the very software we're trying to create, but it also changes the problems we're experiencing.

Kevlin HenneyHad we stuck in ourselves in the early 90s kind of mode, then actually that would all be solved and very stable. But we didn't. We didn't just move up, we moved sideways as well.

BryceYou know, one uh one one interesting trend in in processing, to sort of talking about that whole distributed versus keeping things in local on a local system, is that the the meaning of local is rapidly changing out from under us. Because if you look at every modern CPU and GPU architecture, not necessarily for CPUs yet, but it'll it'll come. Um every modern GPU is a multi-chip module. And by that I mean it is separate little chips that are stuck together on some interproser.

BryceAnd so everybody remembers, you know, Numa systems used to be all the rage back in the day in HBC before GPUs. You know, you'd have a four-socket, uh, you know, an eight-socket system connected with some you know crazy interconnecting. You'd have all of these, you know, you'd have this flat memory space in your process, but in reality, there'd be some memory that some chips could access faster than others. And that's you know, rapidly becoming how we build each individual processor now.

BryceAnd so much of the engineering into building a modern chip goes into the not the chips themselves, but also this interconnect that connects each one of those individual uh modules uh on on the interposer into a single thing. Um and uh what what we take for granted today of this sort of flat view of memory, um uh which is which has been an incredibly useful abstraction, um, incredibly potent abstraction throughout the life of programming.

BryceAnd and we've had chips where it's broken down, and just the utility of having a single view of memory um has been so great that the performance penalties for that not actually being how the hardware works under the hood has not been worth the the complexity of having to deal with it. Ten years from now, we may live in a very different world where where people who are doing systems programming have to think a lot more about uh you know, not just allocating memory, but like when I allocate this memory, where is this memory being allocated?

BryceUm you know, it it could be it could be the case that that uh 10, 15 years from now, the the way we program systems will be very different than they are today.

Kevlin HenneyYeah, I think that that point about systems programming is really key because that that idea, the the illusion of a flat memory model, consistent memory model, is you know, is baked into something that we find in C. And that's why C and C had had such a difficult time on um segmented architectures historically. Yes. There's kind of an implicit notion uh there that you have this. And exactly as you say, what we're doing is maintaining an illusion. We're maintaining this very powerful but convenient abstraction, but that illusion has to be propped up on stage.

Kevlin HenneyThere's a there's a whole load of stuff that happens behind it, and there's a there's a cost for that. But in practice, we're also looking at uh, I think another idea that um uh yeah, as you described, every single um chip is a distributed system. So we normally think of distributed systems, we think we look out to the world and there are the distributed systems. It's like actually it's it's it's it's running there, it's running right next to you. Um that's your you've got a distributed system, and it's got this very different view of how memory is organized. And that I think is a um from the if you're not at the systems level, then the illusion is okay, it's fine.

Kevlin HenneyBut if you are at the systems level, this is the system, and it it challenges you with a lot of complexity, and it's having the languages or the the the paradigms that will actually align with that and say, yeah, here's an easy way to think about it or work with this. Um and yeah, as you say, another decade could shift that at the lowest level, um, it's definitely not standing still. Um, but at the higher level, people might not have to worry about it. In other words, it the upper level is might is actually potentially more stable in this respect.

BryceYeah. You know, it's interesting, we we I think we're all people who identify to some degree as systems programmers. And uh it is, I think, to some degree a nebulous and vague term. But to me, uh systems programming is is any form of programming in which you need, at least to some degree, to care about the underlying architecture of the platform. And that might be operating system architecture, it might be hardware architecture, but it's it's it's a type of programming where you need to think about the the whole view of the system, not just application logic.

BryceYou need to think about what it what is it that I'm programming to.

Kevlin HenneyYeah. How does this get organized? When when this is when this is run, how is this organized at runtime in terms of um sequence of instructions, the fact that pipelines matter and cache lines matter and stuff like that. Um and how is this organized in memory and then to start worrying about that? And this and I think it's interesting from the C perspective, is that C, although we say it's a systems language and it allows us a lot of this access, it doesn't allow us a lot of this access.

Kevlin HenneyIn other words, you know, we're up at L4 caching at the moment, I think, and you know, 10 years' time, who knows? L57 caching, who knows what my process is going to have. Um but C has no opinion on this. It can't talk about that because it doesn't have an idea that that's actually what's going on. Um and how my memory is organized. So it can't speak to that. In other words, it's written the system that C ⁇ is uh the system that C is built against, so to speak, is is is the flat memory model that C grew up under with processes that really what you saw is what you got.

Kevlin HenneyUm, you know, uh you didn't have to worry uh about all of this magic happening, um, you know, this illusion being maintained. Uh and so C has yet to cut through that. Um, we now talk about that as that's the that's the hardware people that worry about that. We don't we we no longer think of it systems program, but in one sense it is the system, and we often worry about you know the cache friendliness of our data structures, um, and yet we can't talk about the cache in the language. Right. We have to step outside the language to say, oh, caching matters, I profiled it, but I can't talk about the caching inside the language.

Kevlin HenneyUm it's kind of like removed from it. So you know, we're systems with systems programming, yet without enough access to the system, which I think is an interesting paradox.

BryceWell, and and I I have uh an interesting story uh uh uh on that subject, which is um I I was at Berkeley Lab back when um uh gosh, I can't remember the Corey. Corey was the the name of the supercomputer was installed at Berkeley Labs, and this was a uh a Xeon Phi system, and it was the second or third generation of Xeon Phi. And um, this was the first one that was a standalone uh uh system, so the Xeon Phi would boot up uh the OS and run the OS.

BryceIt wasn't an accelerator card, and it was basically an x86 architecture, but with a lot more cores. Um and this particular chip that we had in Corey, um, it had this uh HBM memory, this high high bandwidth memory, in addition to its sort of regular memory. And there were a few different modes for the HBM memory. One mode was an explicitly programmable mode where you could explicitly allocate this memory and like use it for like fast memory, um uh because it was uh higher bandwidth, although maybe a little bit lower latency than other memory.

BryceUm and then there was uh another mode where it just acted as like another layer of caching. Um and uh uh to switch between the modes, you need to get to like reboot the nodes. It was like a boot time option. And there's this um this big prize uh in the HPC space. It's basically the Nobel Prize for the HPC world called the Gordon Bell Prize. And uh to win a Gordon Bell prize, you have to do some runs on a big, big supercomputer.

BryceAnd so when Corey was installed, it was a top 10 supercomputer. And so right after installation, there were a number of teams, I think it was eight teams that uh got exclusive access to this supercomputer for a period of time to do full-scale, full system runs on this 10,000 node cluster to try to get results for a Gordon Bell submission. And out of those eight, and these so these teams, these were the best of the best. These were the ninjas, they had a ton of developers, these were folks who who had been spending two years prior to this machine being installed learning this architecture.

BryceUm like I I could I can dream in Xeon Phi assembly. You know, I I knew this chip very well, and everybody else who was working on one of these teams knew this very well. Um and yet, out of those eight teams, seven of the teams use this chip in the cache mode. Seven of those teams were like, it's not worth the effort of us having to change our code to explicitly use this separate pool of memory. Well, like we get a good enough performance boost just using this as cache, just using this implicitly.

BryceAnd only one of the teams tried to use it explicitly, um, but they just they ran into too many issues and too many quirks, and they they ended up just using it in the cache mode too. And it was it was such a smart move of Intel too. They were like, you know what, we're gonna try something new. We're gonna have this explicitly programmed. But somebody at Intel was like, hey, hey, wait, hang on a second. You know, we gotta have a fallback just in case this is really hard for people to use and they don't switch to using this thing. We gotta have something so that this thing will be usable.

BryceAnd boy, was that a smart decision because I'm sure almost everybody who used that chip used it in the implicit mode. And um it just speaks to how pervasive the flat memory space abstraction is and like hiding caches from users, hiding differences in memory hierarchy from users. Um so much work in hardware goes into that. Um and it's worth it because it's very hard to program otherwise.

ConorBe sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions. Thanks for listening. We hope you enjoyed and have a great day.

BryceLow quality, high quality. That is the tagline of our podcast.

ConorThat's not the tagline. Our tagline is chaos with sprinkles of information.

Open the transcript file

Hosts and Guests

Guests

  • Kevlin Henney Guest Inferred

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Episode 313: C++26 Reflection vs CRTP, AI, Kids & More with Zach Laine

    Nov 20, 202647 minAudio

    In this episode, Conor and Bryce chat with Zach Laine about C++26 Reflection vs CRTP, AI, kids & more. Link to Episode 313 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP:...

    • Apple Podcasts

    Credit the people on this episode

  2. Episode 312: The C++26 Contracts Fiasco

    Nov 13, 202652 minAudio

    In this episode, Conor and Bryce chat with Zach Laine about the C++26 contracts fiasco. Link to Episode 312 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The Podcast:...

    • Apple Podcasts

    Credit the people on this episode

  3. Episode 311: Career Changes, GPUs & More with Zach Laine

    Nov 6, 202628 minAudio

    In this episode, Conor and Bryce talk with Zach Laine about his career and the career changes he has made. Link to Episode 311 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials...

    • Apple Podcasts

    Credit the people on this episode

  4. Episode 310: Leaving NVIDIA for... ?

    Oct 30, 202624 minAudio

    In this episode, Conor and Bryce talk about Bryce's next chapter and where he is leaving NVIDIA for. Link to Episode 310 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The...

    • Apple Podcasts

    Credit the people on this episode

  5. Episode 309: Default Construction & CppCon 2026 with Jason Turner

    Oct 23, 202629 minAudio

    In this episode, Conor and Ben chat with Jason Turner about default construction in C++. Link to Episode 309 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The Podcast:...

    • Apple Podcasts

    Credit the people on this episode