Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

ADSP: Algorithms + Data Structures = Programs

ADSP: Algorithms + Data Structures = Programs artwork

Episode 227: Re: The CUDA C++ Developer’s Toolbox

Mar 28, 202539 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this episode, Conor and Bryce chat about Bryce’s talk The CUDA C++ Developer’s Toolbox from NVIDIA GTC 2025. Link to Episode 227 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The Podcast: Twitter Conor Hoekstra: Twitter | BlueSky | Mastodon Bryce Adelstein Lelbach Show Notes Date Generated: 2025-03-20 Date Released: 2025-03-28 NVIDIA GTC 2025 NVIDIA GTC Trip Report ⭐ The CUDA C++ Developer’s Toolbox - GTC 2025 - Bryce Lelbach Thrust RAPIDS.ai CUTLASS CUB nvbench How to Make Beautiful Code Presentations Intro Song Info Miss You by Sarah Jansen https://soundcloud.com/sarahjansenmusic Creative Commons - Attribution 3.0 Unported - CC BY 3.0 Free Download / Stream: http://bit.ly/l-miss-you Music promoted by Audio Library https://youtu.be/iYYxnasvfx8

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

ConorLike, I I think honestly, this is your best work, Bryce. It's your best work, maybe of your career. And because CUDA, let's be honest, uh, has a lot of work to do when it comes to uh its onboarding and education experience. Yeah. And I think that this is the start of something beautiful. Welcome to ADSP the podcast, episode 227, recorded on March 20th, 2025.

ConorMy name is Connor, and today with my co-host Bryce, I ask him questions about his GTC 2025 talk entitled The CUDA C developers toolbox and more. One, two, three, four, five screenshots. Five screenshots. So not not eight or nine, but definitely the most I've taken of any other talk I've watched is one. Which leads me to, I mean, I'm not gonna share screen share because uh we don't need to.

ConorYou'll know the slides I'm talking about, and maybe we'll include it in the show notes. By far, my favorite slide. Arguably better than Jensen's equivalent, folks. And Jensen said it was his favorite slide if you watch the keynote. And Jensen, if you're listening to this for some reason, your slide was beautiful too. But for the discussion that's about to ensue, uh, you'll understand why I like this one better. It is the four by six grid of logos of libraries. And this is why I've been up, I've lost sleep. I think I've had like four or five hours of sleep. I couldn't get to sleep for like an hour. I woke up at another point in the night, like couldn't get back to sleep for an hour, because all I am doing is thinking about this slide deck.

ConorFirst of all, uh where did you get all the logos from?

BryceOh my man, so much work went into that slide.

ConorI know, I know. I saw the slide and I've never I've lost sleep over this slide. So uh just so that the listener is not like, what are they talking about? We've got a four by six grid on a single sli uh slide of twenty-four different libraries, and well we'll list off a couple Thrust, LibQ, Rapids, uh, QFFT, uh, Q Tensor, uh, QDNN, uh, Cub, Cutlass, etc. Some of them have logos, some of them have modified versions of their logos to make it look better, some of them just have text.

ConorAnyways, and and I have I haven't seen most of these logos.

BryceWe do have to clarify that they are not logos, they are graphic signifiers.

ConorAlright, fantastic. This is exactly what I was hoping to get into. Some of them are logos. Thrust? Is Thrust a logo? That's a logo.

BryceIt is a graphic signifier.

ConorOkay, none of these are logos. Uh even rapids? Rapids doesn't count?

BryceNo.

ConorThey're all graphic signifiers. Alright, I'm sure that's for some legal reason that uh I'm uh I'm not involved. Um, but walk me through what went into this slide. And so so none of these are official logos, and why do some of them don't have graphics? Where'd you get these graphics from? Was AI involved?

BryceSo well, so actually um AI was involved, but the the place where AI was involved was actually a different place where there's graphic signifiers. The place where I use AI most heavily was on the slide introducing Thrust and Cub, where I I um I list the four different kinds of things that are in Thrust and Cub. And there's a little graphic signifier for each one. And those were all generated by 01.

BryceUm, although it required me holding a baseball bat over the model. Um it's an interesting question of why would I generate images with a large language model, right? Like what shouldn't show or let me put it differently. Um there are there are large language models specifically designed for generating images. Um I there's one I've used in the past, I forget the name of it. I did not use one of those models. And the reason I did not use one of those models is because I wanted to generate very simple vector graphic images.

BryceIn particular, I wanted to generate SVGs. And so I viewed this as being more of a coding problem than an image generation problem. Um and so I thought it would be better to use something like O1 or Claude. Um I I just used O1 because convenience, that's I it that tends to be the day-to-day model that I use for things. Um so I had it uh generate uh those SVGs.

BryceUm and it took a bit of back and forth. Um I wanted them in a specific color scheme, which it was able to do, but then things like I told it like make a 200 by 200 SVG. And it would make something, but it would put padding around around it, right? So that that what what I want in these little graphic signifiers is I want either in the horizontal or vertical dimension of the graphic, I want it to touch the boundaries.

BryceUm, because I'll add padding myself. And uh it took a little work to convince it that I actually I know I don't want any padding. Um and uh there was a bunch of iteration back and forth of uh getting it like one of the mistakes it made frequently was um it would get the um the layering wrong. So like the uh if there's uh some of these graphic signifiers have arrows and dots, like like dots and then arrows going between them.

BryceUm and sometimes it would have the arrows be on the layer in front of the dots. So you'd see the start of the arrow like in the middle of the dot, and like it doesn't look good. And I would be like, no, like they put them in like this order. And I was surprised that if I if I told it like um a complex instruction, like have the dots in front of the arrows, and then the the the arrows in front of the other element, and then the other element in front of this, it kind of got got the idea. Um and then the the of the logos on that page that that that you saw, um, so some of them come from uh uh some of them have like all the CUDA math libraries have their own little graphic signifier.

BryceThose are the ones on the left, and they I love those because they have a very nice uniform style. And for slide um uh for slideware, I love having these like simple um uh um vector graphic uh images. And I I'm trying, I've been trying to think about how do I describe the property of what I'm looking for in a slide graphic. And I think it's like one, I'm not looking, I want something with no shading, um, no like shadows or anything like that.

BryceUm sometimes they're 3D, sometimes they're 2D. Usually it's like a a 2D flat perspective. I usually use very few colors, um, and I want it to be a vector graphic so that I can scale it and have it look um uh very nice. And I want it to be simple. Um and one of the reasons that I like it to be simple is because if it's if it's simple, then I can make it as small or as large as I want, and I don't have to worry about there being a loss of the detail of it.

BryceUm some of the libraries didn't have uh graphic signifiers, some of the libraries didn't have graphic signifiers, and I just made the graphic signifier, like that little one for NVBench. Um did not exist, did not exist a week ago. Um and then some of them, like the coup DNN and the TensorRT ones, uh, I found there was a blog post where there was a uh on codie and there was a blog post on Tensor RT. And those don't have a like those are not official logos for those libraries, but the blog post had a nice image that was at the front at the header of the blog post.

BryceUm it was an official NVIDIA blog post, and I was like, this is a nicer way than just putting the coup DNN you know text up on the on the slide. Um so yeah, and then uh a couple, uh at least a couple of these libraries, I just I just went to the person who maintains the library, and I'm like, hey, look, your library is gonna be on this slide. Do you have a graphic signifier that you want me to use? And uh I think the next time I give this talk, there will be more of these that will actually have graphic signifiers and not just text. Because at least one person promised me that they would have that they would create the graphic signifier, um, and I didn't have a chance to check back in with them, uh, but but they have an idea of what it would look like.

BryceWe just haven't haven't uh uh produced it yet. But I know I know how much you, Connor, love having um programming language logos. And um I was talking with some people internally about this, and so CUDA C ⁇ does not have a programming language logo. And um one of the points that I I made to people is that I think the best programming language logos and the best graphic the best graphic signifiers in general are the ones where you don't have to put text below it to say what it is.

BryceLike the C logo, the C logo, the Fortran logo, um you don't have to put the text below it saying that it's C or Fortran or C. Now the Python logo doesn't tell doesn't like if you don't if you have no familiarity, you're not gonna know. But I feel like Python is ubiquitous enough that maybe it's not a problem. I don't know about the like the Rust logo, like maybe it's fine, but the ones that I like best for programming languages are the ones where you can, even if you don't know the logo, you can tell what the programming language is.

ConorI mean, I've got so many, so many, so many thoughts and follow-up questions, and uh this is clearly uh gonna turn into two parts. So maybe the start of my questions was part two of this episode, so you got the GTC recap, which probably was only 25 minutes, but at this point, this is gonna blow way past the 35-minute like max one episode. And I'm about to twight, I'm I'm about to twite. I'm about to tweet. Low on sleep, folks. I I did mention I lost sleep over this slide, and I still got other questions to get to. Um I'm about to tweet uh with your permission, Jensen versus Bryce slide deck.

ConorIs that fine? I'm not gonna at uh Jensen.

BryceThat that's that that's fine. I'm just gonna pretend that I didn't hear you ask me the question.

ConorUm but it needs to be done mostly. I mean, I don't need to put Jensen versus Bryce slide deck. Uh it's just this slide that I'm talking about versus the one because do you know the slide that I'm referring to in Jensen's talk that it actually then I don't need to tweet it, but now that I said I will, I will. Um because I needed you to, and actually it probably will be good. So head to Twitter. I'm about to tweet this right now. Actually, should I just put Jensen versus Bryce? No commentary, people can infer what they want. And uh once you're at Twitter, and this is this is why I've lost so much sleep, uh, I'm about to explain.

ConorSo uh for once again, for the audio listener, Jensen's slide is, we will say, much more consistent in that every single one of the what I'll what I'll assume is a cell phone screen with some uh artistic art on it, it looks roughly the same. Unlike Bryce's slide, there are no missing graphic signifiers. Uh that being said, they're also way less meaningful, because these are all just artistic art things that have nothing to do with the actual underlying technologies. And on top of that, it uh it does some weird things or weird, or it's maybe the future.

ConorAll of the coup prefixed libraries are followed by uppercase letters, which is not the case in your slide deck, which is also not the way that these libraries are named. Um, but at least on Jensen's slide, that is the case.

BryceAnd so so I I have I have slightly mixed feelings about that. I um, you know, the math li I I I do feel like we ought to be consistent in the style and the spelling. And um the math libraries all consistently do coup in all uppercase letters now. Um and I think part of that is because CUBLAS, which is one of the first ones, CU BLAS and CU FFT were the first two CUDA libraries. Well, Blaz is an acronym, and FFT is an acronym.

BryceSo I think that started the trend. And then originally CUSOLVR, which SOLVR is not an acronym, SOLVR is a word. Um CUSOLVR was originally Q capital S, and then lowercase the rest of SOLVR. And the same for Q Sparse. Um there were periods of times where it was was spelled differently, but I think that for consistency with the Ku Blaz, the CU FFT, uh, the coup DSS, that it uh it it became this way.

ConorI mean that's fair enough. I completely agree that we should be consistent. I'm just merely pointing out that there are there's a delta. And the one of the one of the biggest things that I've lost sleep over is the fact that on your slide, rapids is there. And one, first of all, rapids is purple, and I'm gonna get in so much trouble.

BryceI'm gonna get in so much trouble.

ConorIt's there and it's been greenified. I I don't necessarily I'm not too upset by the fact that it's been greenified because you want to make the slide look nice. If it was purple, it would look awful. But um on Jensen's version of the slide, he doesn't mention rapids. He just puts QDF and QML, which is and so the the for folks outside of NVIDIA not familiar with rapids, which is where I started my in uh NVIDIA career on the Rapids team before switching to research, is Rapids is basically an umbrella term for a bunch of these libraries, which include QDF QDF.

ConorDF is also an acronym for data frame, which is the pandas equivalent, and ML for machine learning. And it's kind of it's it's the reason I I've lost sleep is that it it is indicative of this kind of I don't want to say marketing problem, but like uh and also too, there was a another slide from uh the Python talk where they show this graphic of um all the different Python technologies of which NVIDIA was not a part of it shows PyCuda in 2010, then Numba, Kupai, PyTorch, Jax, and then Rapids Warp, CudaPython, and Triton Lang.

ConorRapids warp and CudaPython being only the NVIDIA uh built projects, all the other ones were outside, which I thought was kind of curious that for the first decade NVIDIA didn't have any um direct like uh ownership over these projects. But that uh I'm not sure about warp because I actually haven't used it, but I I believe Rapids requires conda at the moment. Koupai numeric requires conda, whereas like Jax and PyTorch you can just pip install. And uh anyway, so that there's just this like uh what do you call it?

ConorUh like disconnect of like you've got in the first two verticals all the coup libraries, and then you've got rapids, which is kind of like an umbrella for a bunch of coup. So I'm getting to my question. What are your thoughts and feelings about this?

BryceUh yeah, so so you know, I actually so one, I I do have to say uh I liked Jensen's slide, and um I think that his slide and my slide are trying to do different things because they're trying to speak to different people. Um my slide in my talk is is um targeting purely a developer audience, and Jensen's speaking to um, you know, the the not just developers, but also people who are in these particular industries.

BryceHe's speaking to um to analysts, to everybody who cares about GPUs. And I think that his slide is far more effective for a broader audience. But I also I love that um despite the fact that Jensen's got a much wider audience that he has to speak to, that he's still able to have a talk that's so developer focused. I think it's great that he's still got a talk where um he's got a slide there where he calls out um, you know, uh and speaks about like a lot of the great libraries that we built.

BryceUm like that's that's I think really special. I think there's not a lot of tech CEOs who will talk about, you know, developer libraries, like specific developer libraries that their company's building. Um so I I I actually was kind of excited by by Jensen's slide. I debated whether I should, and I actually I think if I had had time, I would have taken Rapids off of my slide and I would have put um QDF there instead. Um because and it's not not solely because of the purple logo nature of Rapids.

BryceUm it's because Rapids isn't a library. Rapids is a collection of libraries, and to some degree it's like it's more of a product. And I wanted this slide to be not product focused, but you know, library focused. Like the the point of my talk was to, you know, or the point of this slide in my talk was to point people to all of the tools that are available to them. Um and so I think that I I I think that it when I give this talk in the future, I'll probably instead of Rapids, just have QDF there.

BryceUm the problem is uh there's not enough real estate in the slide for me to do what I would really want, which would be to list all of the various Rapids um libraries, um, of which there are there are many. But maybe what I would do is I would take off one or two of the uh math libraries and put um some of the rapids libraries uh up there.

ConorWell, I'm thinking of like this slide I know because I know you will clearly evolve over time and the iterations that you give this talk, but like I think you know one of the questions at the end of the talk was about docs and and um you know best practices in terms of education, which uh CCCL and NVIDIA is one of their primary focuses now. But what would be amazing is if like this was there was a JavaScript. I don't know if you need D3 for the kind of animations that I'm thinking of, but like imagine this is a landing page website where you hover, you take your mouse and hover over these, and it does a little jiggle every time.

ConorYou you go over each one, and then for rapids, you would get this little balloon that expanded into QDF and QML, and and it would be a good starting point of like. The other thing is too is uh which is a whole other layer of this of like what is the language that you are interoping with? Technically, Rapids at the highest level is Python, right? There are CUDA C libraries that you can target that lie belief beneath them, but it seems a bit odd to have Rapids sitting right on top of Thrust. I mean, actually, it doesn't seem odd because Rapids is kind of built on top of Thrust, but like Thrust is a CUDA C library, whereas Rapids is primarily a C or a Python library that has C lower level libraries that you can target.

ConorAnyway, so depending on the uh what do we call them? Graphic signifier, you know, you have access to different languages per tool or per library.

BryceSo the hardest part about this slide was um figuring out the ordering and the grouping of these. And there's a couple different axis upon which I needed to group them. First, there were libraries that do similar things. Um so there's on the left-hand side there are eight libraries that are typically known as the CUDA math libraries. Um and uh and then um we have the three C core libraries, Thrust, LibQ, and Cobb.

BryceUm then there's NVBench, which is maintained and developed by the C core libraries team, but it's really it's a benchmarking framework. Um I think of it more of like a developer tool. Um and then there's the CUDA runtime, which is what everything here is built on top of. Um then some of these libraries are device-side libraries. Some of these libraries are libraries where it's a thing that you call from your CPU that launches work on the GPU.

BryceSome of them are things that you purely use in device code on the GPU. Um, and some of them are a mix of both. And so I try like like there's a couple on here, cutless, cooperative groups, and cub. I needed those three to be next to each other. Um, but also Cub needed to be near all the other CCCL libraries. But then the last constraint, um, and then there there were libraries like NVSM and Nickel, which are communication libraries, that I wanted to have together, and then like Q files and I.O.

Brycelibrary. I wanted to have that near like the CUDA runtime and some of the other like like it's not a compute library. There's there's one one way of dividing these libraries is is it a library that does, you know, that does math or science or physics, um, or is it a library that does like, you know, runtime management or or comms or IO or stuff like that? Um and then there's the the machine learning libraries, QDNN and Tensor RT, and I wanted those three libraries to kind of be centered in the image, um, I decided that the it was best for them to either be at the top or the bottom because I have four rows.

BryceIf I'd had three rows, I would have put them in the center row. But I feel like with four rows, that would have been hard. Um and I actually I um the way I made this slide is I started off with just um uh raw boxes um and I figured out how much like how many of these like you know blank boxes did I want to have. Um And then based on that, I tried out different configurations. Go off of that.

BryceHere, let me see if I can find the the first version of this this slide deck went through it's on R4 right now. And the the way that I version my slide decks is I bump to another R version of them when uh when either I give the talk or if I delete content. Like if I change the content plan substantially, then I will make a new version of the slides because I want to record, I want to save what my old content plan was.

BryceI'm gonna share share my screen with you very briefly, but it's okay because the visual will be easy to explain. This is what the slide originally looked like, where it was the four by six grid in a slightly different configuration. Um and it is just uh boxes, square boxes that say the library names, um, and some of them have different colors to indicate the grouping, and there's a giant like text box on the slide that says to do this, but do better.

BryceAnd I think I did better than the you know colored text boxes, one hose.

ConorAlright, I mean uh I'll have to think on this, but like I'm I'm coming to realize the reason I have lost sleep and uh we're discussing this at length. I have and also I put the ho cart in front of the horse. I didn't even explain Bryce's talk, which I'm gonna take a uh 10-second digression. The title of Bryce's talk was the CUDA C developers toolbox. It is at a high level uh showing some examples of Thrust, uh Cub, and what was the other LibQ?

ConorDo you show that at all? LibQ ⁇ , yeah. Yeah. Yes. And then also it it uh covers the universal vector, which I have a question about, which we'll get to. But anyways, high level, that's what it's about. I was supposed to say that at the beginning, but I got too excited about the slide. And the reason I've lost sleep, I think this is your best slide um you've ever made, but it's gonna be it's gonna be so much better. Like, I I think honestly, this is your best work, Bryce. It's your best work maybe of your career, and because CUDA, let's be honest, uh, has a lot of work to do when it comes to uh its onboarding and education experience.

ConorYeah. And I think that this is the start of something beautiful. And so my recommendation, I already had this in my mind um before you showed me uh what I'm looking at right now, which is I'm I'm staring at six different colors. We've got yellow, red, blue, purple, pink, and green. When you were describing that on the uh the two left columns of the six columns on this slide are the CUDA math libraries, I was thinking, man, what you should have added to that talk was like having a rounded, not not uh pointy, but a rounded rectangle that hot that kind of briefly you know goes over and says, you know, these are the math libraries, and then that one disappears, and then the next one pops up, and that one has a red coloring or whatever.

ConorSo basically, have like six slides that transition and fade from like one rounded rectangle to another rounded rectangle that basically visually and then you're you would vocally describe because uh I think to folks at NVIDIA, you know, we see thrust next to LibQ next to Cub, and and we know that like okay, you know, thrust uh is is kind of a layer above Cub and it interacts with. Anyways, we see the connections, but for folks that are maybe even experienced or not experienced, those connections are invisible. They have no idea that you know this grouping, this implicit grouping uh kind of exist exists there.

BryceUm well and also and also like it, you know, it it was uh they may also not know what these libraries are. And it's it's interesting that you say that because I as I'm we talked a little bit earlier about that, um this is a 35-minute talk, and um this is a talk I plan to give a lot this year, and for the most part, speaking engagements are hour-long talks. So the reality is the GTC version of this talk was really just a preview. There is an hour-long version of this talk, and there was content that had to be cut. Um, and I think of the content that had to be cut, the parts that are probably of the most interesting to you is one, this the reason I put so much effort into this slide is because this slide, something just like what you described, will happen with this slide.

BryceI haven't built that part of the deck yet, but I'm gonna spend five to ten minutes going through this overview of the libraries and um giving people the the, I'm not gonna go into details, but giving people the pointers about what are these libraries, when should you use them? Because it's the CUDA C developers toolbox, the name of the talk. We want to teach people what are the tools in the toolbox. Um and the other thing that got sadly cut from this version of the of the talk is I spent some time talking about the buy-key algorithms. And the reason I'm I was very sad to cut that from this talk because I think if you're looking at thrust versus the standard library algorithms, um the the part of thrust that's most compelling for people is the that there's all these very useful extended algorithms.

BryceUm like you could use C standard parallelism, uh, which we have an implementation that supports GPU acceleration, but if you wanted to do a segmented reduction, that's not in standard C yet. Um and so I had to I had a couple different examples of doing some nice uh uh segmented reductions that I had to trim. Uh those will be in the next version of this talk, and also the deep dive into, or not the deep dive, the overview of some of these libraries will be in the next version of this talk. And I suspect that I will do something like what you described with the boxes.

BryceUm it'll probably just be like uh like drawing a circle around these. Um and uh I actually like the idea that you just had about rapids that to to have it expand out into uh uh all of the various rapids libraries. Um I I should say this is the first time I've given a talk where I have used uh transitions and I used the morph words transition uh that uh that Connor taught me about.

BryceUm but then I I I I think that I maybe do this more than you, but because when I've seen you use the morph words transition, you usually do it on a pure code slide. But I often have a code with diagrams slide side by side. And I found that when I used Morph words, that it um uh it would fade the diagram to black. And the reason is because PowerPoint in a lot of cases, if I had the diag if I had even if I had the same diagram on one slide versus another slide, the group of objects would have a different name from slide to slide.

BryceAnd so PowerPoint would get confused. But I learned that if you go into this magical thing called the selection pane in PowerPoint, you can give your group of objects a specific name, and if they have the same name from slide to slide, then the PowerPoint transitions know that they're the same thing. And so I that's how I was able to use that trick to build some really cool morph transitions, and I really love the morph transitions. Um I uh I I for so long I resisted having transitions in my slides, and part of it is that transitions take time and they can throw off by timing.

BrycePart of it is that if you rely heavily on transitions and animations in your slides, then the PDF version of your slides, um uh the the the like if somebody's just reading through your slides, it can be a little bit harder. But I I am a convert. I loved the um the morph uh words transition. And to explain for people who don't know that what the morph words transition does is um if you've got like two like two slides of text that are very similar, but there's some changes between the two, the morph words transition will um uh it will show the words, the the text between the two slides like evolving into each other.

BryceUm maybe you maybe you have a better way of explaining it than me.

ConorYes, sure. I will uh better than me explaining it, although I will briefly, uh the first thing I have to say is we need to we need to wind down this part two of our conversation because uh I don't have a super hard stop, but I do have a hardstop-ish, and I definitely want to get to the uh stay tuned to next week, listener.

BryceYou know what my hard stop is? My hard stop is after a certain hour, I will be unable to park anywhere in downtown San Jose.

ConorAll right. Well we'll put our hard cap at 30 minutes from now. So if we want a 25 minute uh part three, we've got to wrap this up in five minutes. So uh and and I have I have like four questions. We're gonna have to rapid fire, and then you get like 15 seconds per answer, um, and maybe we'll we'll revisit this topic. So uh I I have my third most popular YouTube video is how to use basically morph, because it's the number one question I got. Welcome to the club. It's been here since 2019, you're only half a decade late, and it's the most beautiful thing ever. There's three different versions objects the default, words is the second setting, and characters.

ConorSo most people don't know how to use it effectively for code because they leave it set to the default option of objects, and that does nothing for your code. You have to change it to words, and yes, it basically just automatically finds the closest number of words. It's magical, and also that little hack that you sent me. I've no I've watched basically uh this morph um uh not keynote, but presentation where you can do crazy things with it uh even more than what you showed me. So I've been a I've been a Morph pro. In fact, I don't want to name the person at Microsoft, but a very senior person at Microsoft one time emailed me and asked, How are you doing that in PowerPoint?

ConorAnd I was like, bruh, you work for Microsoft. Like email the PowerPoint team. But wait, you take 15 seconds, 15 seconds for whatever you're gonna say, because we got a rapid fire questions after this. Do I know what?

BryceDo you know do you know anything about the algorithm that they use to do the matching?

ConorNo, but I would love to talk to the PowerPoint folks. I imagine it's just kind of some like closest uh you know thing because it does some weird stuff every once in a while. But uh if you're if you work for Microsoft and you know or happen to work on that algorithm, get in touch with us. I usually block anyone that emails me asking to be a guest, uh, but we're reaching out this time.

BryceSo the I'll uh I'll briefly say the thing that I love the most about the Morph transition is nothing about the visual itself. It's about the structure of the code. I used to, if I had a progression of chain of code evolutions, it used to, like where there was different things on different slides, it used to be that I would add additional new lines and additional spacing so that when I went from one slide to another slide, elements would stay in the same place. And that meant that if you saw, if you looked at the first code slide in one of these evolutions, the spacing would be all weird and unnatural.

BryceAnd the thing that I love about Morph is that with Morph, I don't have to do that because with Morph, if I need to insert a line at the top of the code in the next slide, what Morph will do is it will gracefully shift all the code down. I don't like it if it's jarring, if like one slide it's you know nine lines of code, and then the next slide, those same nine lines of code are there, but they've moved down one line because I've inserted a tenth line of code in front. I don't like that. So what I would do is on the nine lines of code slide, I would put a blank line at the top.

BryceAnd now I don't need to do that because now with Morph it will just beautifully just shift everything down. And that that is that is for me the killer feature.

ConorAlright, it's very graceful, and we've used up three of our five minutes left in this episode, folks. Um so maybe I'll limit myself to one question and you have 30 seconds to answer. The number one thing uh that I was confused in your talk, how come MVC and uh Studpar didn't get mentioned as a CUDA C? I hope this isn't breaking news that like that technology is uh has been like backburnered or something. Uh what is uh what's the reason for not mentioning it?

BryceNot not that it's been backburnered. Um uh just that we wanted to give a talk about how to do CUDA programming. Um uh studdPar is is great. I like I it's not in any way something that we don't support. Um uh you know, we certainly want people to be using it, but we wanted to give a talk about uh if you want to write CUDA code, like how should you write idiomatic CUDA C code? And I think part of the problem is that people that consider themselves CUDA programmers, um they don't think that a talk about C ⁇ standard parallelism is for them, right?

BryceAnd so we want we wanted to reach a particular audience. And so, you know, this talk is or this the the whole curriculum that our our our colleagues Georgie and Toshenko and much of this talk is based on material that Georgie made for the tutorial. So I can't take too much credit for it. But Georgie proposed that, you know, we shouldn't start teaching people the low-level things. Like the way that you typically teach CUDA is you start off by teaching people about kernels and device code and just separate host and device memory and you know warps and blocks.

BryceThis talk doesn't mention warps or blocks once. In this talk, I only talk about distinct host and device memory in the last part of the talk. Right? I introduce the idea that there's different CPU and GPU memory as the last thing. And and I don't even say the word kernel anywhere in here, really. I just talk about launching work on the GPU. And Georgie came up with this idea that we should teach people to use libraries first, to use abstractions first, use libraries first, try that out, use the the things like thrust first, and uh then you know, only as a more advanced topic would you go and write your own kernel.

BryceYou should only do that if you've tried to use the libraries and the libraries haven't worked for you. Because honestly, you're gonna get better performance if you use the abstractions. And so if you think about how do you just teach like modern C versus C. Well, the way that people teach modern C is like you don't start with teaching them pointers, right? You don't start teaching them pointers and C style arrays. The way that I've seen people teach modern C to beginners these days is you start off by teaching them, you know, the standard library, by teaching them containers.

BryceYou only introduce the notion of pointers much later on. And that's what we're trying to do with CUDA C ⁇ here, is we are teaching the content from the high level to the low level instead of from the low level to the high level. And I had two instructors during GTC come up to me and they were like, wow, this is really surprising. This is a very different way of teaching. And one of them, he sort of like asked this to me and then just paused as if he expected me to say something like, uh, oh no, this isn't how you should actually teach it.

BryceAnd I was just like, yeah, no, like that's what we want you to do. And he just paused and he just like processed. And he he was clearly expecting me to say something to um make him feel reassured that he didn't have to completely rethink how he was teaching this material. And then he realized that, like, no, wait, you really are telling me to completely rethink how I've been teaching this material, and that's exactly what we want. We want to teach from the high level to the low level.

ConorAn amazing answer, if too long. We have now exceeded our our threshold, and I have uh follow-up stuff, but we're gonna we're gonna table this. Uh remember I asked about uh you know MVC ⁇ . Um we will we're gonna revisit MVC ⁇ , um, which you know the short version of what Bryce said is this was a CUDA C ⁇ talk, MVC ⁇ is ISO C with you know inherent uh or automatic parallelism. A topic for another day, and I did have questions about the uh universal vector and unified memory, but we will table those questions for a follow-up discussion.

ConorWe are now transitioning to part three because at this point, if we stop at, you know, five minutes past the half hour mark, it's only going to be a 22-minute episode, folks. So part three of this recording, which I believe puts us on like episode two twenty-eight, potentially. Be sure to check these show notes either in your podcast app or at adspthepodcast.com for links to anything we mentioned in today's episode, as well as a link to a GitHub discussion where you can leave thoughts, comments, and questions. Thanks for listening. We hope you enjoyed and have a great day.

BryceLow quality, high quality, that is the tagline of our podcast.

ConorIt's not the tagline. Our tagline is chaos with sprinkles of information.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Episode 313: C++26 Reflection vs CRTP, AI, Kids & More with Zach Laine

    Nov 20, 202647 minAudio

    In this episode, Conor and Bryce chat with Zach Laine about C++26 Reflection vs CRTP, AI, kids & more. Link to Episode 313 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP:...

    • Apple Podcasts

    Credit the people on this episode

  2. Episode 312: The C++26 Contracts Fiasco

    Nov 13, 202652 minAudio

    In this episode, Conor and Bryce chat with Zach Laine about the C++26 contracts fiasco. Link to Episode 312 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The Podcast:...

    • Apple Podcasts

    Credit the people on this episode

  3. Episode 311: Career Changes, GPUs & More with Zach Laine

    Nov 6, 202628 minAudio

    In this episode, Conor and Bryce talk with Zach Laine about his career and the career changes he has made. Link to Episode 311 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials...

    • Apple Podcasts

    Credit the people on this episode

  4. Episode 310: Leaving NVIDIA for... ?

    Oct 30, 202624 minAudio

    In this episode, Conor and Bryce talk about Bryce's next chapter and where he is leaving NVIDIA for. Link to Episode 310 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The...

    • Apple Podcasts

    Credit the people on this episode

  5. Episode 309: Default Construction & CppCon 2026 with Jason Turner

    Oct 23, 202629 minAudio

    In this episode, Conor and Ben chat with Jason Turner about default construction in C++. Link to Episode 309 on Website Discuss this episode, leave a comment, or ask a question (on GitHub) Socials ADSP: The Podcast:...

    • Apple Podcasts

    Credit the people on this episode