Ben's friend is locked out of their AWS account and Matt's AI has decided he's a hacker. The escape route runs through a cheap computer, a mesh network, and Neal Stephenson.
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Matt GodboltHey Ben.
Ben RadyHey Matt.
Matt GodboltIt has been almost a month to the day that you and I have sat in front of a microphone and tried to talk.
Ben RadyMm-hmm.
Matt GodboltAnd at least I'm sat in front of a microphone now and not actually recording on my earbuds in my wife's bedroom, childhood bedroom. So back in the right continent.
Ben RadyRight. Right.
Matt GodboltAlthough I do have a funny story on that. I set myself up for a work recording the other day and, you know, for an internal like chat kind of video casting, and I had my pompous microphone in shot obviously because, you know, I'm one of those kind of podcasty people. And of course, I forgot to actually select it in the dropdown. So it was using my terrible earbuds as the recording. And so that's going to go out internally and I'm going to have the absolute... Mickey taken out of me by being this, you know, hey, I thought we got somebody who was like media savvy, but no.
Matt GodboltNo I'm not. But that's not what we're going to talk about today. We don't even really know what we're going to talk about today, but something happened to me this morning. It was the first time. That was fun. I was intrigued by a super cool new exploit that has been discovered. It exploits [pretty strong]. It's like...
Ben RadyNo.
Matt GodboltDo you know, I can't remember the dude's name, unfortunately. XOR, E-A-X, E-A-X, E-A-X is their handle, his GitHub handle. He did SandSifter, which was like a comprehensive search through the entire space of all possible x86 opcodes looking for undocumented opcodes, which was fun. He did M/o/Vfuscator, which was, hey, every instruction... the MOV instruction is Turing complete on its own with certain caveats, and he wrote a compiler that compiles to just MOV instructions, which is fun. And then most recently he's discovered this cool facility there. If you have ring zero access, like, you know, if you're like the lowest level access, you might just think, hey, yeah, I've got root, what more do I need to know? There are layers below that, that like the firmware of the system runs at.
Matt Godboltand other things that you're not supposed to be able to see about your own computer that you own under your desk. And he's found a way to get into it by misconfiguring the RAM system in some particular way, in a really cunning way. Anyway. I was like, this sounds cool. I'm going to... While I'm in the shower, I gave a quick query to Claude saying, can you just summarize this for me? I want to learn about it a little bit. Give me some pointers. And then I had my shower and I came back out and I looked. And for the very first time, I had a massive alert saying, no, you can't do this. You are clearly some evil hacker.
Matt GodboltWe will not allow you to use our AI to hack things. You're a bad person.
Ben RadyI hope.
Matt GodboltI don't think that's exactly what it said. I'm sorry if anyone from Anthropic is listening, but it was something like that.
Ben RadyMm-hmm. Yeah, you're paraphrasing a little bit.
Matt GodboltAnd it gave me the option to use, you know, like a watered down model, which I did. And it was perfectly good for summarizing a document, which I then went off and read anyway. So, you know. It was fine, but it made me think. Are we entering a new age where we're being gatekept by things that... we don't have any control about.
Ben RadyMm-hmm. Yeah. You know, this is, it's sort of like a new problem and also not a new problem.
Matt GodboltRight.
Ben RadyYou know, I had a similar experience this week where a friend of mine was...
Matt GodboltOh.
Ben Radylocked out of their AWS account for reasons that are kind of opaque to them.
Matt GodboltThat...
Ben RadyAnd...
Matt Godboltis a nightmare situation for me because yeah, that goes down and then you're like, well, how do I... who do I call?
Ben RadyRight. Right, right. Right. And I mean, we have this sort of digital dependence problem in many facets. Like, I'm going to give you a hypothetical scenario here.
Matt GodboltMm-hmm.
Ben RadyYour choices are... You can continue to own and use a mobile device, a mobile phone. Or not. But if you continue to use a mobile device, then you need to cut off one of your fingers. It can be any finger. Whichever one you want, you can use whatever method you want to cut it off, but you need to cut off a finger.
Matt GodboltWhat? What? This was not on my bingo card.
Ben RadyYeah.
Matt GodboltRight! Okay.
Ben RadyWhat do you choose? You lose the left pinky. Like that's the choice that 99% of people are making.
Matt GodboltThere's some...
Ben RadyIt's like, yeah, I kind of need a phone.
Matt GodboltYeah, I... I don't like the way this analogy is going, but I understand where... I think I understand your point. Yes, I get it. Yeah. Okay.
Ben RadyYeah, it's this dependence. And it's like, it does... it's not even good enough to have the device. You need...
Matt GodboltRight.
Ben RadyLike the network. Right. You need the connectivity.
Matt GodboltYeah. You know, teleport me back to 1970 with the top of the line phone and it's useless.
Ben RadyRight.
Matt GodboltIt's an interesting...
Ben RadyRight.
Matt Godboltyeah, device, but it doesn't really... yeah.
Ben RadySo we have this digital dependency, right? And it comes in many forms. Like AI models are the most recent form of it, you know, cloud services.
Matt GodboltRight.
Ben RadyAll of these, like, you know, your Google account, right, with your email and everything else. And you've got probably a lot of identity tied up in that.
Matt GodboltYeah. Oh my gosh. Shit, I have a story about that, right? Like, my vanity domain godbolt.org is a Google Apps for Your Domain, when that was a thing. That was like a thing that if you were a Google fanboy like I was, and I was an early adopter of everything, you could say, hey, I would like to bring my own domain to a Gmail account effectively.
Ben RadyMm-hmm.
Matt GodboltSubsequently, when I worked at Google, I discovered that that is not how it works. There's a massive internal hack to make that work. And then it sort of became Google Apps Enterprise-y type thing. But unfortunately, those who early adopted, like I did, are stuck in this... it's this hinterland between it's not really an enterprise account and it's also not a Gmail account. So I can't like do half the things in my house that are internet connected with my godbolt.org account. And so I want to just move. And I can't just move because so many things are tied into my login identity as matt@godbolt.org that goes through like the Google login process, that now I can't... I'm stuck.
Matt GodboltSo I'm like... what do I do? But yeah, sorry, I interrupted.
Ben RadyYeah.
Matt GodboltYou were in the middle of the flow of talking about other digital dependencies and...
Ben RadyYeah. No, well, I mean, that's another great example, right? And so like, I don't think it's realistic to try to eliminate all of these things. But to the extent that you can be... more resilient to some other entity just unilaterally deciding maybe, in a way where they think you're acting in your best interest, like my friend in his AWS account, maybe.
Matt GodboltRight.
Ben RadyMaybe it was. Or not.
Matt GodboltYeah.
Ben RadyIt's sort of like, you don't know, and you don't really have a way to know, and you don't have a way to like validate what it is that they're doing and be like, actually that...
Matt GodboltI know.
Ben RadyIt's not how that works. Like, to the extent that you can limit that liability, it sometimes makes a lot of sense. And so like, you know.
Matt GodboltRight.
Ben RadyI feel like open source was like a great example of this, where you know I certainly remember early on in my career the pain of having to deal with, you know, third-party software vendors, and you would try to understand how their software worked. And they would give you answers that were clearly wrong. And then, you know, you maybe tried to dig into it a little bit. And then, you know, I had the experience very early on in my career of like, you know. I don't even think that it was any more complicated than like reading through like the scripts they had sent us. But they were like, don't modify those. Right. And I'm like, okay, I'm not modifying them.
Ben RadyI'm just reading them. Like I'm trying to understand like this thing that you sent me that we paid a bunch of money for that doesn't seem to work. And like, God forbid, if I had actually tried to like reverse engineer any of their software, right? I can't imagine what their reaction would have been to that. Like maybe some kind of lawsuit.
Matt GodboltYeah.
Ben RadyBut... like you're stuck in this place where you're like, you know. I don't understand how this works. I'm not empowered to change it. And yet I am dependent on it. Right?
Matt GodboltEmpowered is the word, the active word. That's really interesting. Yeah. It occurred to me that like another thing, it was like early Google search engine optimization once Google became the monopoly. Like, you know, you had this thing where an algorithm would change and people's would drop off of the listings, and again it's like, but they couldn't choose that. That's not something they could choose to be in or out. They were like dependent on something, but not because they chose it. Which is even... but yeah, the empowerment is an interesting aspect of... I do wonder, yeah, so what can we do about this, Ben? Well, I mean, I'm kind of stuck, right? I mean, what... I'll have my own internet search, I'll just, you know, my own data center, my own... yeah, what. Yeah.
Ben RadyRight. Mesh network internet for everybody. That's the solution. Is we're all just going to get some powerful antennas and put them on the top of our houses and mesh all of the houses together.
Matt GodboltDon't tell me. Yeah, mesh.
Ben RadyAnd that's how the internet's going to work.
Matt GodboltIt's going to end up in some crypto scam though, isn't it? At some point, like some...
Ben RadyYeah. At some point, yeah, there's like, you know.
Matt GodboltAnything distributed nowadays ends up in some... Well, there's a... we can make a chain, some kind of...
Ben RadyRight. There's a coin. You got to buy the coin.
Matt GodboltHey there.
Ben RadyAnd we're issuing the coins.
Matt GodboltYou got AI coin.
Ben RadyAnd who gets the money from these coins?
Matt GodboltUgh.
Ben RadyYes, exactly.
Matt GodboltOh, token coin.
Ben RadyYeah.
Matt GodboltI'm sure it's been done. I'm sure...
Ben Radya billion times at this point.
Matt GodboltIt has already been done.
Ben RadyThe terrible idea that launched a thousand slide decks.
Matt GodboltYeah.
Ben RadySo, you know, like... I think it sort of depends on how far down the sort of, you know, crazy tinfoil hat, you know, internet libertarian, you know, route that you want to go. And I say that recognizing that like a lot of the things that people, you know, from like the Electronic Frontier Foundation and, you know, some elements of the open source community, you know, 20, 30 years ago... a lot of things that they were saying, you know, with panic in their voices would come to pass, have come to pass, and nobody seems to care.
Matt GodboltRight.
Ben RadyLike the amount of spying that like all the devices that you pay for do on you is like off the charts.
Matt GodboltSomehow that's okay when... I mean, the thing is, there is... so I've always thought of it as there is some kind of implied social contract between me and... I'm going to pick on Google again just because I'm a bit of a fanboy for it, and I feel like having been there I've got some standing in there, some... And I've always felt that like, yes, it serves ads, yes, it reads my emails or does something with them, and I don't mind that because I get a decent email service that I don't have to pay for and it's integrated with everything. And like the slightly spooky face recognition is great when, for example, you are preparing a family
Ben RadyMm-hmm.
Matt Godboltboard. And I'm like, hey, let's just find pictures of my father-in-law. Here's hundreds of pictures of Tony. And like, oh, that's great, right? That's kind of an amazing thing. But yeah, I do realize that I've given up something for that, but I've probably given up more than I think. than I think about. And I've probably given up more than I would if someone sat me down and said, well, what about these things? And I was like, oh, yeah, I hadn't really thought. Oh, that's a good point. And, you know, but I'm getting something back out of this.
Ben RadyMm-hmm.
Matt GodboltAnd like some of the things, though, I'm not getting something back out of it. So like, you know, automatic number plate recognition stuff, the Flock camera stuff that's going on around, you know, like there's a lot of things where it's like, yeah, that's... I'm uneasy because it is not... ironically, it's not directly a company making money. I mean, it is obviously, but it's like... Law enforcement, a government thing that is for the benefit of the people has decided that they would like this information and then they would be happy to pay someone to put cameras up to do all this kind of stuff. And that feels like, no, that's a step too far somehow. Now, I don't know where the boundary is there. I mean, it feels obviously... but one I'm sort of choosing.
Matt Godboltor at least choosing... not choosing, not choosing to. But then... I get so much benefit, and how will I get the same benefit without, you know, putting together a whole bunch of open source software, which is maybe where you're going with this. As you mentioned, the mesh network of like internet stuff.
Ben RadyWell... Yeah, I mean, this is kind of what I'm saying, is it sort of depends on how far down the rabbit hole you want to go. Right. And I think. For me personally, first recognizing that 90... nine... I'm just going to put two nines in that, percent of people are not going to want to go down the rabbit hole at all. Right. Like they're going to want to do the minimum amount that they need to do to keep themselves safe and secure from like scammers and phishers and, you know, people like that. And you know, some of them don't even really care that much about it, from some of the things I've seen. But you know, that's like the minimum amount that they're going to want to put into this, and the rest of it, they're like perfectly happy to be completely dependent.
Ben Radyand exposed in like a data sharing type sense. Like, I'm very happy to trade my data for free email. Right? Like, it's fine.
Matt GodboltYeah. That's, I guess, both. Yeah, yeah.
Ben RadyRight. Like they're very happy to trade their data for just about anything. And in fact, many of them... What was the... Was it Snow Crash that had the gargoyles in it? Do you know what I'm talking about?
Matt GodboltI...
Ben RadyDid you read Snow Crash?
Matt GodboltI did read Snow Crash, and now I'm just trying to think about...
Ben RadyI think it was, I think it was Snow Crash.
Matt GodboltI don't remember gargoyles. Was it?
Ben Radythat had basically like there was a whole profession where you just walked around with a bunch of like data gathering equipment strapped to your body.
Matt GodboltAh.
Ben Radyright like cameras and like infrared and like, you know, microphones and all these things
Matt GodboltRight. Does ring a bell.
Ben RadyAnd like that was your job. You just walk... it was like you're like a little mini like Google Maps car, but you were just a person.
Matt GodboltPut a person doing that, yeah.
Ben RadyRight. And that's what you did for, you know, for your livelihood. And they referred to these people as gargoyles, right? And I think like the great trick, it's like, you know. So much of what is being built is like 90s cyberpunk dystopia. Right? We're just literally using those books as like a user manual for like how to build a society.
Matt GodboltYeah. Oh. Yeah.
Ben RadyAnd I feel like one of the great tricks of social media is basically like turning everyone into gargoyles for free.
Matt GodboltNow I'm sad.
Ben RadyLike you don't even have to pay them. Right. Like you just walk around and you say like, hey, you should post this to your Instagram, and then people take pictures of everything around them all the time.
Matt GodboltYeah.
Ben Radyand just post it with, you know, geotags on it to social media, and then it's like, all right, cool. We've done it. We've gathered all the data that is.
Matt GodboltThat was... I remember like,
Ben RadyWe didn't even have to pay for it.
Matt Godboltthe like the surveillance things that like found all of the photographs of like a monument somewhere in Paris or something like that, we're able to reconstruct a full high detailed 3D model from like just thousands of tourist photographs of it, for example, that they've posted to their Instagram. So yeah, we're in that world, aren't we? Yeah, gosh.
Ben RadyThere you go. Yeah. So my point is that 99% of the people, maybe 95, I don't know, some very high percentage of the people out there are not going to be interested in going down this rabbit hole at all. But... I think it is important. Personally, I think it is important to be able to defend the ability to do it if you want. Because A, it just might make you sleep better at night. Like, I know a lot of anxious people. I know a lot of paranoid people. And they're just like, that's who they are. And they just want to know that these things are going to be safe and secure. And then they reduce their dependency as much as possible.
Matt GodboltRight, fair.
Ben RadyAnd they just want to be able to do that. And I think that you should. And also, there are lots of situations in which... you know... people who... maybe... in situations where they can be easily exploited by governments, by other people, need to be able to keep themselves safe and secure. And having the tools available to them to do that is important. Right? And you know, I don't know that I can go in and enumerate all of those situations, but I think we could certainly imagine lots of situations in which that might be not only something that someone would want to do, but be, you know, reasonably morally justified in wanting to do it.
Ben RadyAnd so, for all of those reasons, I think it's important to be able to do this. But it is also really hard.
Matt GodboltRight. I mean, so like... Going back to the beginning of this, or not nearly the beginning, as you mentioned your friend's AWS account. I mean, so...
Ben RadyRight.
Matt GodboltIf they can't recover that account, do they go and buy servers and rack them? Or, you know, is there a happy end to this story, by the way, I should say?
Ben RadyRight. I mean, immediately what you probably do is hope that you did everything with infrastructure as code and then try to port it all over to... you know, Google Cloud or... Azure, I guess. I don't know.
Matt GodboltLet's just switch it one... one problem for the same category.
Ben RadyYeah, you know. One feudal lord to another feudal lord, right?
Matt GodboltFeudal lord.
Ben RadyYeah. Like, okay.
Matt GodboltJeff Bezos, feudal lord.
Ben RadyYeah.
Matt GodboltYou heard it here first.
Ben RadySo, you know, like that's like on day one, that's probably as good as it gets.
Matt GodboltYeah.
Ben RadyBut I think this is why, like, you know, a lot of times, like especially in our industry, people will wind up building their own data centers, not because they strictly have to, but because, you know, when things go wrong or when they have problems or when they want to try to do things that are maybe not standard. They want the level of control to be able to do it, right? Like they want to be able to say like, yeah, we can put whatever hardware in these machines we want. We can, you know.
Matt GodboltMm-hmm.
Ben Radyconfigure them however we want. We can use whatever network technology that we want. We're not limited by what a third party vendor is willing to offer to us. Right.
Matt GodboltYeah. Yeah.
Ben RadyObviously that is out of the reach of... most typical people.
Matt GodboltYour average.
Ben RadyBut I will say, I think one thing that is overlooked... you know, kind of like not that expensive way to get yourself a lot of digital independence is just get a cheap computer and put it on your network and leave it on all the time.
Matt GodboltRight.
Ben RadyGreat. And, you know, if you want to take that up a level, like get some remote access for it, right?
Matt GodboltTailscale, something something, to make it a very... yeah, exactly, yeah.
Ben RadySo you can SSH into it. Yeah, yeah. Most of those types of computers that you can buy use very low power, so you don't really need to worry too much about how much power they're drawing. They're quite... powerful in terms of like the compute that they can, you know, supply.
Matt GodboltYep.
Ben RadyAnd you will probably find that most of the things that you would want to run in some other service provider, you could run on those types of machines today.
Matt GodboltYep.
Ben RadyAnd in a weird sort of like everything is backwards and I don't understand how anything works anymore... The absolute trick that you can pull is, you know, use the LLM of your choice, which you are absolutely dependent on these days, for
Matt GodboltAt least.
Ben Radyto build you the software that you need to run on that computer to replace whatever it is that you're dependent on the third-party services to do.
Matt GodboltPut a pin in that.
Ben Radyright, so if you have some like, you know, awesome note-taking app that depends on some cloud service and you're like, wow, if this ever disappeared I'd be totally screwed
Matt GodboltRight. Interesting.
Ben RadyAnd you want to go down the rabbit hole, which again, 95% of people don't. Something that you could do is you could build your own.
Matt GodboltYeah.
Ben Radyand run it on this computer that runs in your house. And maybe, you know, have some backups that, you know, write to a USB drive or, you know, go back to the cloud and encrypt them and upload them.
Matt GodboltYeah, I was going to say, that's one thing where like the cloud maybe isn't holding you to ransom unless...
Ben RadyThat's another way.
Matt GodboltTwo things happen. One, you need your backup. And two, the cloud provider decides that they're not going to let you.
Ben RadyRight. Right.
Matt GodboltSo that seems like a dual uncorrelated failure or whatever we used to say. So, yeah, that is an interesting call to action. Now I wonder. And we have touched on this before because I'm getting a slight deja vu feeling here. But I wonder, it is...
Ben RadyRight.
Matt GodboltThe fact that we have been so lucky for so long, like 20, 30 years in the sort of software profession that we're in, where essentially we've always been able to just, inverted commas, quotes, just build it ourself. And with open source, that has been free. Like, is that spoiling us? Because like if I worked in the manufacturing industry and I made grommets on a machine, and then the vendor that made the machine said, no, you can't use this machine anymore, that's a real problem that many grommet manufacturers may have always had, and they might have to switch to another provider at short notice because they've gone out of business and now they can't get the parts to repair them. So there's a little bit of like a, maybe, you know, other people have always had to deal with this dependence. It's like, unless you want to become a core... your core competency is building lathes to make grommets, as well as the grommets themselves, then you've always had to outsource some significant part of your endeavor. Whereas you and I have been always like, oh, an email client, yeah, I can't be bothered to write an email client, but I...
Matt Godboltcould in principle. I feel like I could. Is it... are we just spoiled? And then does an LLM now move us back to where we were to start with, you know?
Ben RadyYeah, I think a lot of the difference there is that once you've bought the grommets, they're yours. Right. The company who built the grommets never shows up and says, your license to these grommets has expired. And we're going to go into your...
Matt GodboltOr the lathe. Maybe in that instance for the grommet manufacturer. But yeah, either way, both of those things are true. Yeah.
Ben RadyRight. Like it is possible to own the thing and it's yours.
Matt GodboltIt's really interesting. I mean, obviously, again,
Ben Radyright? And it's kind of like servers, right? Like you buy a server. You own the server.
Matt GodboltYeah.
Ben RadyModulo maybe some weird things that happen when you ask LLMs about how those chips work.
Matt GodboltAh. Ha ha!
Ben RadyYou know, you can do pretty much whatever you want with them.
Matt GodboltYeah.
Ben RadyAnd like, you know, you want to screw them up and break them to experiment, go ahead and do that, right? Like there will be no ramifications other than the fact that you lost however much money you spent on the server.
Matt GodboltYeah.
Ben RadyRight? So I think that is sort of like the fundamental difference. And some of it gets back to like software as a service type models where it's like, you know.
Matt GodboltYou never quite own. And, you know, like, you know, video games these days, right? You know, obviously our hobbies are, my hobbies are old computers, and I've got like buckets and buckets of cartridges and CDs and DVDs, even, that I can put into various old machines and they work fine, but like. But nowadays, if you want to play a game for like PS4, PS5, it's like you don't really own it in any meaningful way. It exists as bits on the disc, maybe it doesn't even exist on a hard disk anymore, it just exists in the cloud, which means that it goes away.
Ben RadyYeah.
Matt Godboltwhen you don't own it anymore. You're renting. You've got a license to own something, which is like
Ben RadyRight, right.
Matt Godboltwe've gone... Yeah.
Ben RadyYeah.
Matt GodboltI mean, you buy a book and you read the small print in the front copy about, you know, you may not, you know, give it away or whatever, or there's some level of restrictions for what you can use it for.
Ben Radyand reproduce it. You know?
Matt GodboltBut like, that seems like... those seem very reasonable compared to end user license agreements in software or whatever else. Yeah, I don't know. How did we get here? How do we let this happen?
Ben RadyWell, the 90s cyberpunk set the model, and oh, we're just going to do all these things.
Matt GodboltYeah.
Ben RadyI mean, I don't know. I'm being a little tongue-in-cheek there.
Matt GodboltIs this Neal Stephenson's fault? Is that what you're saying?
Ben RadyYeah, exactly. Right.
Matt GodboltOh man.
Ben RadyWilliam Gibson can be blamed.
Matt GodboltWilliam Gibson. Oh, yeah, yeah, yeah. Is that Neuromancer as well?
Ben RadyYeah, yeah, Neuromancer, Mona Lisa Overdrive, and...
Matt GodboltThank you.
Ben RadyOkay, what was the middle one?
Matt GodboltMy nerd credentials are definitely going down here. I don't think I've ever read Neuromancer.
Ben Radylike Counter Zero, Count Zero.
Matt GodboltCount.
Ben RadyIt ends with Zero.
Matt GodboltCount Binface.
Ben RadyCount Binface.
Matt GodboltWho sadly lost today at Clacton-on-Sea. I love my countrymen.
Ben RadyOh, did he? Oh man.
Matt GodboltYeah, it didn't work out. 26% of the vote though.
Ben RadyThat's pretty good.
Matt GodboltIt's not bad. I mean, the other guy, the guy who resigned and who won't be named here, did win.
Ben RadyYes. Nigel Binface.
Matt GodboltYeah, no. Don't besmirch the Binface. Use that one. Yeah, I mean, it's... Yeah, you've got to love the Brits for not taking things too seriously, but still.
Ben Radyyeah
Matt GodboltYou know, I still...
Ben RadyI mean...
Matt Godboltsomeone pointed out that like the people of Clacton, the place where... you know, this is real for them. So we're all treating it as like a bit of fun, but you know, and it is making some political points, but like, yeah, it's a thing. Anyway, that was not on my list of things to talk about with you today. But you gave me an opportunity to talk about it. So, Count... what was he saying? Count Basie?
Ben RadyCount Chocula.
Matt GodboltNo, Count... Count Zero.
Ben RadyI think it's...
Matt GodboltThe books, that's right.
Ben RadyI think it's like counters. I forget what the middle book is called.
Matt GodboltI love to count.
Ben RadyBut all of those, I mean, you know, every...
Matt GodboltOkay, I think...
Ben RadyWhat is it? Who's your favorite vampire? The one from Sesame Street. He doesn't count. Oh, I assure you that he does.
Matt GodboltThat's all right. That's brilliant. I don't know how we're going to top that now. We've peaked. We've peaked. 20 minutes into this conversation, we peaked.
Ben Radyyeah Oh yeah, no, we're talking about 90s cyberpunk and the dystopian future that we live in.
Matt GodboltThanks. Yeah, so you were saying that the 90s... this is the blueprint. So what's coming in the future? Well, actually, no. Is there like, so. I think... we've talked a lot about how folks feel about LLMs. And I think, you know, you and I have both been on various rollercoaster-like ride journeys with this is great, to this is terrible, to this is I'm going to lose my job, to now I'm the most powerful programmer, to now this is what a mess I've got, and everything in between, right? And that's not even taking into account all of the moral and ethical aspects, which are also super important but I have sort of compartmentalized.
Ben Radyokay
Matt GodboltBut... you know, not being able to run an LLM. If, for example, tomorrow Anthropic decides that I've been a bad boy. or my company decided that, you know, they're not liked by Anthropic or whomever else they're providing, or whoever, you know, whatever. That's a concern.
Ben RadyYeah.
Matt GodboltSo you mentioned about running your own stuff. Can we do that with LLMs? Do I just need a warehouse-sized computer to be able to run something worthwhile?
Ben RadyYeah, you know, I think it's arguable that the end state of all information is to be free. Right. You know, it's like all of these models of like copy protection where you have like a private key that only three people in the world have. It's like someone is going to figure that out. Like, ask Sony.
Matt GodboltYeah.
Ben Radyright, like someone is going to figure that out. This is not going to work.
Matt GodboltYeah.
Ben RadyRight. And, you know, the model weights and LLMs are just too valuable for people not to try to figure out what they are. Right. And especially when you consider the fact that there is no. like world government entity that can, you know, enforce any sort of protection for any of this information in any sort of like.
Matt GodboltRight.
Ben Radyglobal way.
Matt GodboltAre model weights different though, in the same way? They're more like a compiled program. Right. If you, you know, if you imagine I give you the EXE
Ben RadySure.
Matt Godboltof my code but not the source code to it. I give you the EXE and you can go away and use that program as much as you like, but you can't really do anything more with it. You can't extend it or build on it or anything like that because it's kind of like already been compiled. It's very difficult to reverse engineer.
Ben RadyYeah.
Matt GodboltAnd I feel like the weights, or you know even the open weights that you can go and get on the internet, have that flavor. It's like, sure. Here's the result of compiling, but you know, we won't, you can't buy the compiler because it's like 45 data centers full of machines that ran for like two years. to compile the program. And now here's the program. So, you know, the fact that the program is dressed up to me as a service is, you know, problematic. But even if they handed me the weights, I guess that's what I asked you, in fairness, but like, I'm just now thinking ahead, right? You know, you could give me the, you know, 24 terabyte disc that contains the weights to the best model and I could somehow cobble together running it, but I'd be stuck with that until the next time somebody hands me the 48 terabytes of the next model.
Ben RadySure.
Matt GodboltIt feels like. I don't know. I've always balked at the term open weights because it's like, open? Yes, I guess. I suppose so, but...
Ben RadyRight.
Matt GodboltYeah, sorry. Anyway, I cut you off. Where were you going with this?
Ben Radywell no, I mean, you know, that's a fair point and that is definitely, you know, an avenue that we could explore here, is like how open are open source models, if all you're getting, if you're, yeah.
Matt GodboltWhat does it mean? I mean, because like, yeah, open source. I suppose that's it. I compare it to open source, and open source is like, hey, here's the source code. Now that might as well be alien.
Ben Radyyeah
Matt Godboltto most people on the planet, but in principle, you can look at it and go, this was meant for us to build on and continue, and in fact the people who gave me this or gave me access to this, they are using this to continue and get better. and the next version of the product will be built.
Ben RadyMm-hmm.
Matt GodboltAnd it's most likely, you know, obviously there are edge cases to these. Whereas the weights feels like so derived. It's not really the cool and interesting bit of it from the point of view of continuing and being a builder and not being locked into this brain forevermore.
Ben RadyYeah.
Matt GodboltI wonder what... but then the resources are so, so massive that I can't imagine.
Ben RadyYeah.
Matt Godboltthat you could do much with it. But at least you could, right? So in principle... And have you tried this, by the way? Have you experimented with local models or anything?
Ben RadyRunning local models? No, it's been on my list of things. It's like... there's going to be a tipping point probably in the next 18 to 24 months for me where it's like, part of my day job is probably going to be figuring this out. And so like, I'm kind of just waiting for that to happen because then it'll be like more of a focus, right?
Matt GodboltWoof!
Ben RadyYou know what I mean? Like, it's not just like something I'm monkeying around with. It's like, oh no, I need to make this work for real.
Matt GodboltYou actually have to understand it. To be able to do it, yeah.
Ben RadyYeah. You know, and like, the world will be completely different in 24 months.
Matt GodboltWhat?
Ben RadyTechnologies will be different. Everything will be different. So I might as well figure it out for real when I need to figure it out for real.
Matt GodboltYeah. Yeah, it's interesting. I suppose... I mean, we're, sorry, excuse the dog. We're meandering all over here, but like, you know, like even getting the equipment to run these things on, obviously one can find GPUs around or, you know, in the house I have a few around, but they're not really up to scratch for this kind of stuff.
Ben RadyMm-hmm.
Matt GodboltAnd then, you know, like I know from my own day job that sourcing, you know, top end GPUs and the RAM that goes with them is incredibly difficult right now, and I wonder if that's going to change. But one can only hope that there are some advances in the way that we...
Ben Radyright
Matt Godboltevaluate, not evaluate, to make better how we run inference or whatever
Ben RadySure, yeah
Matt Godbolton hardware, because GPUs are designed well for graphics. I still feel sad for rows and rows of graphic processing units in racks that have never had a video display plugged into them ever.
Ben RadyRight.
Matt GodboltThey've never generated a frame of a game, which is the point of a GPU, in my opinion. But, you know, it makes me sad. It's like battery-farmed hens, you know. It's like, oh, God, it's not right. Be free. Do what you're supposed to do.
Ben RadyYes. I need some free range GPUs, please.
Matt GodboltYeah, well, I've got a drawer of... I've got a Voodoo 2 in a drawer somewhere.
Ben RadyI wouldn't.
Matt GodboltI think that'll help, right?
Ben RadyWe feed them a steady diet of Doom Eternal and...
Matt GodboltOh. That's too new for my GPUs, more like VGA adapters with a bit.
Ben RadyYeah. Uh-huh.
Matt GodboltYeah, anyway.
Ben RadyYeah, no, there's, I mean, well, and this is kind of what I'm saying about the, you know, the information wants to be free argument on models is like, yes, it does take a lot of resources to train them and generate those weights.
Matt GodboltRight.
Ben RadyBut once you've done it... like, they're not going to disappear. And imagine a world where there is an open source agentic coding model that is better than, say, Fable 5. Right. Like.
Matt GodboltRight.
Ben RadyThat is probably going to happen at some point. Right? Like. Do you maybe have some situations in which you want to use a better one? Sure. But if you're just trying to like vibe code the notes app to run on your personal server, to like, it was like.
Matt GodboltYou don't really.
Ben RadyThat model will be more than sufficient, right?
Matt GodboltAll right. And I can see a world in which that kind of open... yeah. The thing that I'm interested in, and now this is going to be part two of a massive thing, is that like, you know, after the base training of a model, like where you've just fed some subset of all human knowledge in through however you've acquired it, which is its own massive asterisk and footnote and ethical dilemma, but you've got some amount of knowledge, you train a neural net on it, and then there are... so much post-training that happens to like make it valuable, make it aligned, make it safe. And then tuning after that to make it, you know, like, hey, don't write code that's terrible. And that's the bit I'd be interested in. I'd like to be able to go like, hey, I don't want to have to explain in text over and over again to a model that...
Matt GodboltDon't write comments like this, for God's sake, stop it.
Ben RadyMm-hmm.
Matt Godboltbecause you know you and I, I'm sure, have done that a million times. But I want to be able to say like, why didn't we just tune it? Can I personalize the weights so that it doesn't do that in the first place rather than me having to exhort it every single time to do it the way I want? And that feels like the last steps of an open model where you're like, okay, that's tractable. Here's the 20 terabyte intermediate model. You can add layers of tuning onto it. And then like, maybe that is something I could do. Maybe I'm just beyond, and you know, yeah.
Ben RadyI have to admit I don't know nearly enough about how.
Matt Godbolthow that process works now.
Ben Radyhow that process works to know whether that's a feasible thing or not.
Matt GodboltNo, but it seems like I can imagine open companies starting that process, or you know maybe maybe this stuff exists on Hugging Face.
Ben RadySure.
Matt GodboltMaybe this is the kind of thing that, you know, I would, if I spent a bit more time looking into it, I would discover it can actually be done.
Ben RadyYeah.
Matt GodboltBut now it's interesting. And yeah, just so, and the irony here that the sort of leader in open weight, as far as I can tell, is the Chinese government.
Ben RadyRight. Yeah.
Matt GodboltThat seems like, wow.
Ben RadyYep.
Matt GodboltThat was not on my bingo card for this year, is like champions of openness and freedom.
Ben RadyYeah.
Matt GodboltYou know, I'm sure you can find a, as you say, a 90s sci-fi novel that...
Ben RadyWe need an AI company that's open. What would we call it?
Matt GodboltNo.
Ben RadyI don't.
Matt GodboltNo.
Ben RadyI just can't think of a good name.
Matt GodboltNo, there's no... I don't think you can pick one.
Ben RadyI wonder if the domain, I'll just check some domains and see if anything's available.
Matt GodboltYeah, see whether you can find one.
Ben RadyYeah.
Matt GodboltYeah.
Ben RadyBut yeah, like it's entirely possible that in the limit, the only thing that matters is GPUs, power and cooling.
Matt GodboltAnyway.
Ben RadyRight. Because like. Models will get good enough. Their information will leak out into the world as all information does. And you will reach a point where. you know, you don't necessarily have to be beholden to a company to use their models as a service. You can go and either purchase or maybe just acquire for free because they're just freely available.
Matt GodboltMm-hmm.
Ben Radya model that you can run on your own hardware and then you can purchase that hardware and then you own that hardware.
Matt GodboltYeah.
Ben RadyAnd no one can take it away from you. And then you could have, you know, it's like that server that's running in your house maybe it becomes a little bit more beefy and uses more power.
Matt GodboltRight. but it is yours and no one can turn it off for you other than
Ben RadyBut it's yours, right? And no one can turn it off and no one can tell you not to use it. And, you know.
Matt GodboltAnd you're using solar panels so that the electricity company can't cut you off.
Ben RadyRight. Further and further down the rabbit hole.
Matt GodboltDown the rabbit hole, yeah.
Ben RadyAnd you got the mesh internet that you're not connected.
Matt GodboltThat's it.
Ben RadyYou don't have an internet service provider.
Matt GodboltExcept... mesh internet that only works when I'm not inside the tinfoil wrapped house, where like, yeah, no, it is.
Ben RadyYes, exactly. Right.
Matt GodboltGosh. Wow. Well, friend, we've gone round and we've covered an awful lot of things that I wasn't expecting to, but you and I haven't spoken for ages, so this is not too surprising. that rambling and in some ways but I think perhaps
Ben RadyIt's been... It's been a little bit.
Matt Godboltin the tinfoil wrapped house, with our own utility providing systems in place, is where we should leave this, with our own intelligent.
Ben RadyMm-hmm. Yes.
Matt GodboltWell, what I would say is about the power and cooling thing is that at least there is an existence proof of something which doesn't require that much cooling because I'm using it right now to talk with you. And I don't mean my mouth. I mean, but although that is the primary way in which I think, to be fair, as you've probably realized, but like, so it has to be possible to have something which has at least human intelligence that runs on, what, 80 watts. 100 watts, right? You know, something like that. Maybe I'm not as fast as Claude is, but like somewhere in there there's a scaling thing, so I'm sure we can make it happen along the way. And that's the hope obviously, is somebody will come up with some cool 3D
Ben RadyYeah, right. Yeah.
Matt Godboltsilicon hardware thing that is vastly more efficient at computing whatever it is, but
Ben RadyRight. Or just fusion power. That works too. That'd be awesome.
Matt GodboltFusion. Yeah. We still got a lot of exhaust heat to put somewhere, which doesn't seem like the best thing to have knocking around, you know.
Ben RadyCould be true. Yeah.
Matt GodboltSo. Yeah. Anyway, with that, on that bombshell... Yeah. on our fusion powered tinfoil wrapped home.
Ben brings a 1976 letter from Bill Gates, and Matt brings the after-effects of last night's pub. The hat gets out of the rabbit, a squirrel is briefly involved, and nobody downloads a computer.
Matt records from the wrong continent and explains how he taught his computer to be a 1950s television. Ben guesses his way to the secret of the colo(u?)r burst and wins today's podcast.