338 - Code Review, Testing, and Staying Sharp in the AI Era with Joel Hawksley
1 hr 5 minAudio
In this episode I talk with Joel Hawksley, Staff Software Engineer at GitHub and creator of ViewComponent, about whether AI is making us dumber, why blocking code reviews struggle under a flood of AI-generated code, and how automated tests can replace human reviewers as a safety net. We also discuss what conferences are really for, how to stay focused while waiting on agents (hint: fold laundry), and Joel's e-paper family dashboard, Timeframe. Links: - Timeframe - Timeframe (open source core) - ViewComponent - Refactoring English - Joel Hawksley's Blog - Nonsense Monthly
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_03Hey, it's Jason, host of the Code with Jason podcast. You're a developer. You like to listen to podcasts. You're listening to one right now. Maybe you like to read blogs and subscribe to email newsletters and stuff like that. Keep in touch. Email newsletters are a really nice way to keep on top of what's going on in the programming world. Except they're actually not. I don't know about you, but the last thing that I want to do after a long day of staring at the screen is sit there and stare at the screen some more.
SPEAKER_03That's why I started a different kind of newsletter. It's a snail mail programming newsletter. That's right. I send an actual envelope in the mail containing a paper newsletter that you can hold in your hands. You can read it on your living room couch, at your kitchen table, in your bed, or in someone else's bed. And when they say, What are you doing in my bed? You can say, I'm reading Jason's newsletter. What does it look like?
SPEAKER_03You might wonder what you might find in this snail mail programming newsletter. You can read about all kinds of programming topics like object-oriented programming, testing, DevOps, AI. Most of it's pretty technology agnostic. You can also read about other non-programming topics like philosophy, evolutionary theory, business, marketing, economics, psychology, music, cooking, history, geology, language, culture, robotics, and farming.
SPEAKER_03The name of the newsletter is Nonsense Monthly. Here's what some of my readers are saying about it. Helmut Kobler from Los Angeles says, thanks much for sending the newsletter. I got it about a week ago and read it on my sofa. It was a totally different experience than reading it on my computer or iPad. It felt more relaxed, more meaningful, something special and out of the ordinary.
SPEAKER_03I'm sure that's what you were going for, so just wanted to let you know that you succeeded. Looking forward to more. Drew Bragg from Philadelphia says, Nonsense Monthly is the only newsletter I deliberately set aside time to read. I read a lot of great newsletters, but there's just something about receiving a piece of mail, physically opening it, and sitting down to read it on paper that is just so awesome.
SPEAKER_03Feels like a lost luxury. Chris Sonier from Dickinson, Texas says, just finished reading my first nonsense monthly snail mail newsletter and truly enjoyed it. Something about holding a physical piece of paper that just feels good. Thank you for this. Can't wait for the next one. Dear listener, if you would like to get letters in the mail from yours truly every month, you can go sign up at nonsense monthly dot com.
SPEAKER_03That's nonsensemonthly dot com. I'll say it one more time. NonsenseMonthly dot com. And now without further ado, here is today's episode.
SPEAKER_03Joel, welcome.
SPEAKER_00Hey Jason, thanks for having me back. What is this? The fourth time?
SPEAKER_03Something like that, yeah. And I think our podcast episode uh stuff spans years now.
SPEAKER_00Yes, several years. I would guess this is the fourth appearance in five years.
SPEAKER_03Oh wow.
SPEAKER_00I've had some kids. I've had two kids since we started doing this. So yeah, my availability changes.
SPEAKER_03How old are those kids now?
SPEAKER_00My kids are a little over one and a little over three.
SPEAKER_03Oh wow. My kids are 13 and 16.
SPEAKER_00Oh wow. Okay, I have a question for you. Yeah. This is relevant to your audience, but like your kids, especially your 16-year-old, is probably starting to think about like what they're going to do with their lives once they leave the nest. How are you feeling about the world they're going into with all the upheaval in our industry?
SPEAKER_03I'm not unconcerned. At the same time, they are very like well-adjusted, capable, confident people. Of course they are. Yeah, and so I don't have huge concerns for for them as people. Like if anybody can handle the world, they can, I feel like. Sure. Yeah. Um, but like, what is the world gonna look like economically and from a job market perspective and stuff like that?
SPEAKER_03It's a weird world.
SPEAKER_00It is. And you can imagine that my I'm asking the same questions about my kids in a much more vague uh manner because they're so much younger. I just can't even imagine if we even have half the pace of innovation we've had in our field in the past three years for the next 13, what it will look like when my oldest son is 16.
SPEAKER_03Yeah.
SPEAKER_00It's unreal to imagine that. Versus, like, for comparison, if you were to imagine the preceding 15 years before that, you know, a good chunk of my career, things were ultimately pretty stable. And I wonder how anomalous that was.
SPEAKER_03Right. Yeah, like there's it's it's easy to think of the way the world currently is at any given point in time as the way the world turned out. You know, like the world went through all these shifts and changes and stuff, but like that's done now. This is just how things turned out, this is how things are. And obviously that's not the case. We're just seeing a snapshot of the way things happen to be at this moment, and it could be that um things are gonna get way different, and things that seem like a progression culminating in something um are actually just like a blip.
SPEAKER_03Um like yeah, like the whole like uh, you know, a couple hundred years ago the whole like job thing was not it wasn't it it it didn't work the way it does now. And now it's like you go to college and get a job and stuff like that, but like that doesn't mean that in the future you go to college and get a job. It might be something we can't even imagine.
SPEAKER_00Right. And I think that one of the things I've been thinking about lately with all the AI-related things going on in our industry is whether you know most technological progress is forward progress and that like it doesn't regress. But I have to wonder whether there will be certain things we're doing today that we weren't doing a year ago that we won't be doing two years from now.
SPEAKER_03I'm sure.
SPEAKER_00Like maybe we'll find that some of these things we're trying weren't the best for the companies we work at, for example, or for our profession. Like I think there's a lot of experimentation going on right now, and I think that we're more likely to be in that kind of environment where something we did just turns out to not be a great idea. I mean, I think you can see this in some larger technological trends. I pay attention to technology and education, for example, because we're about to have kids in schools, and I'm curious, and it's interesting to see the school districts in our area outside of Boulder, Colorado backtracking on their use of technology.
SPEAKER_00And that to me just points to I think some really good intellectual flexibility on the part of the people running our school districts because it implies that they don't just view every single improvement or every single thing that they thought was an improvement to be durable in the long term.
SPEAKER_02Mm-hmm.
SPEAKER_03Yeah, I see what you mean. My mind went to a slightly different place uh a minute ago. I was thinking of like this is kind of a different thing, but like if you plop me in the woods or the middle of the ocean, like I'm probably not gonna be able to figure out how to get uh to get out of there. Um I can't navigate um or like hunt and forage to find food to survive for like a week or something like that.
SPEAKER_03Um, because there's so much like knowledge that has been lost because we depend on technology now. Um, and like I've experienced that with programming. Like I feel like I've gotten at least a little dumber.
SPEAKER_00Yes, deintellectualized is what I've been liking to say lately.
SPEAKER_03Okay.
SPEAKER_00I definitely feel like I'm a little dumber too in certain respects. I feel like I'm smarter in others, but I feel like there is like a very real finite limit to our human capability, and you kind of have to budget it.
SPEAKER_03Yeah. Yeah, and okay, so like I've gotten dumber because I don't need to do as much intellectual grunt work. I've gotten like dumber and lazier. Because when there's I don't know, there's like a failing test or something, and rather than even look at it, I just copy the URL and I'm like, hey codex, why is this test failing? Right. Because why would I not do that? But now I'm not like using my brain very much, I'm just like copy-pasting something, and then I'll pass along that message to some other person, and that's it.
SPEAKER_03And then if there's an instance where I need to use my brain, I'm like, ugh, I don't want to do this, and it's harder than it used to be.
SPEAKER_00Absolutely. I think it's because more of our work is not thinking. So I I I've been thinking about the saying that you are like the five people you spend the most time with. Well, like, I spend a lot of time with AI, so I feel like I am becoming at least not necessarily becoming more like AI, but I'm becoming more of the person that I am when I use it. And that is not a totally positive thing. I think there are some positive aspects to it. For one, having to become a potentially more clear writer, or if you're dictating speaker, you know, like I feel like there is a shift towards natural language in our in our work.
SPEAKER_00But yes, there is also a decomplecting force where like what we're needing to do is potentially less complex, it's requiring less deep of thought.
SPEAKER_02Mm-hmm.
SPEAKER_03Yeah, and that's that's a bit scary. It's I I I I'm sure that it's kind of analogous to like when calculators were first introduced, or even like when writing was first introduced. Like Socrates wasn't a big fan of writing because he thought it would make people like dumber and lazier.
SPEAKER_00Right.
SPEAKER_03And it's kind of funny because like maybe it did, but that doesn't mean that it's like a net loss. It's I certainly think writing is a net win, and I think AI is a net win, but it's not without casualties.
SPEAKER_00Right. There's a a saying, uh automation is amputation. That anytime you automate something, you are taking away your ability to do something. And I think, especially in the long term, that is true. That like there are certain skills like deep debugging, for example, that are less exercised now. So when you go to do them, you are less good at them. They're also less necessary, though. So I think that that is like where AI is both the poison and the cure.
SPEAKER_00And I think that is an uncomfortable place to be because it's a new technology. And I'm like, do I really want to be putting all of my faith in this thing? Especially given the fact that from like a very rationalist perspective, a lot of our practices were built on this tower of like first principles knowledge, where like you would start a software project. I'm thinking about View Component, just for example, that like day one, 100% test coverage. Day one, I knew every line. Like, literally, in the last major release we did, I forget how long ago it was, I read every single line of code.
SPEAKER_00I like reread the whole library. I reread every line of documentation, I wrote, re-wrote everything that was under version control. I didn't read the image binaries, but you get the idea. Like, and I was able to understand all of what it did and remove as much as possible. Like, I in that release, my goal was to have as few lines in that repository as possible, just as a mental exercise, if nothing else. And I don't know how good I would be at doing that exercise at this point.
SPEAKER_00I also don't know if that's a great point of optimization anymore. I just had a really interesting experience this week using Astra for what, like five or six sessions in a row. I basically just gave it a simple prompt, which was like, here's view component, find bugs. And it found 12 bugs each time.
SPEAKER_03Oh wow.
SPEAKER_00So it found like I might be exaggerating a little bit, but I think it might have found 50 bugs this week. And they were all some of them were more likely to be hit than others, but they were all the kinds of things that I think we would have hit eventually. There's a lot of people using the project in a lot of different ways, and you know, eventually the the holes in the cheese, Swiss cheese line up, but it really just made me think about how you know I was reviewing changes to all these bugs, looking at all the test cases. My foundation of trust in that code was lower.
SPEAKER_00My reasoning towards the truth in that code was lower. And it's just a different place to be in. And I don't know how comfortable I'd be taking that approach without excellent test coverage. Like excellent, excellent test coverage. Um but it's fascinating to see that it found that many bugs with that much test coverage. And making me wonder about like what does that say about where we were, where we had like all this like pretty good metrics for this application, you know, for this library or framework, I should say.
SPEAKER_00And uh it still had all these problems.
SPEAKER_03Hmm. Yeah, um, you know, it's not limited in the way that we are. Um like it can just go into every nook and cranny and very easily find all these problems. Um I've I think I've had similar experiences with some of my projects. Um, and that's a great benefit. It's a great benefit for it to be able to find those problems and bring them to your attention.
SPEAKER_03Um a little bit different of a thing. Um I've been thinking the last few days, like I don't know, just kind of really big picture, like what is going on? What are we all doing? What are people interested in these days? Like, because okay, I myself am a content creator for lack of a term that I like more. Um and it used to be that I would write blog posts about like uh I don't know, software design, abstraction, coupling, cohesion, stuff like that.
SPEAKER_03Um but I I haven't written anything in a while, partly because I had like a weird stretch in my personal life for the last few months. But now I'm kind of coming back and being like, all right, let's let's resume this work. And I sit down in front of the keyboard and I'm like, I don't know. Like, what do people want to read about these days? And I like look at Reddit, I'm like, okay, what is like upvoted on the Ruby and Rails subreddits?
SPEAKER_03And like it's all almost all stuff about AI. Um, and it's like, what are people wanting? I don't know. Like, what are people thinking about? What are they wanting to talk about? I'm not sure.
SPEAKER_00I mean, it's introduced a certain amount of uncertainty across the entire discipline. Like I think we're finding kind of across the board and all the things we've done in our all the professional activities we've done habitually have been affected by this change. And I view both that as both exciting and terrifying. I view it as exciting. I'll say this, and I and I have to wonder if this is true for you, but for me, I was in a little bit too much of a groove.
SPEAKER_00Like I had kind of set myself on a pretty steady path. Like I'm not trying to get promoted or anything at work. And you know, having all these kids, like I wasn't trying to like push myself as much professionally as I might have in the in the previous years. But really, right around when these, you know, the more revolutionary agentic kind of models landed was right about when I was kind of coming out of the woods a little bit. And I was like kind of dreading going back to just doing more of the same of and I say this like really politely, like I still really like like I still really liked what I was doing, still like what I'm doing.
SPEAKER_00Um but AI has been such a disruptive force in the positive for me, and that it's just m forced me to reassess a lot of things, like you're saying. Like I and I think that it's a healthy exercise in general. Like I'm a very habit-driven person. I like put so much on the calendar, like I put all my workouts on the calendar, I put like I have a really good to-do list ritual, like I'm really regimented, but that can limit me cognitively into certain modes of thinking.
SPEAKER_00And I think that what is the healthiest part of this revolution is the way that it it has made us question old practices. So I'll pick one off the shelf for you that is a great one to talk about. Code review. Like hindsight is 2020, but man, we do some weird stuff with code review. Like, code review is really weird. Like, think about it. Like, really strange.
SPEAKER_00You write this thing, and at one point in time, you get one other human who they look at your tests and they're like, okay, the tests exercise us a certain way. They read the code and like, okay, I learned something from what you did. They also run the code in their heads, was a thing a lot of us did. We'd like read it and be like, oh, what about this edge case? That is like an interesting task to put on humans. And it already did not scale well. Like, we would have to limit our PRs to like what 100-line diff was like roughly the comprehension level for like functional changes.
SPEAKER_00Um there are a lot of these things that like are now falling over, falling on their face as we have a 10x increase in in code volume, right? And that and this is and this is trickling down to other parts of the system too.
SPEAKER_03Yeah, all this a lot of the effects of or a lot of the things that this AI revolution has revealed, um I found extremely depressing. Um like it has revealed to me like how many things um how many things, how many people just deeply don't understand or misunderstand.
SPEAKER_03Like the code review thing, ask ten people, like why do we do code review? And you'll probably get different answers, and sadly, probably not very good ones. Um, and it's kind of like um theater. Like a lot of times you you put up a PR and somebody looks at it and they rubber stamp it, and a second person looks at it, and they just like find some trivial thing to comment on.
SPEAKER_03Meanwhile, the code is garbage, and it's like, what is this really accomplishing? It's it's Nah, and and then uh for code reviews that are blocking, like it has to get reviewed and approved before it can be merged, that introduces such an enormous um time bottleneck. Like I was trying to um I I added a new Grafana graph to a service the other day, um and it took me like 30 seconds to have Codex write the code for it, and then a number of days to get the approvals that I needed, and then I noticed that I like named a file, something I didn't want, and so I renamed the file, and then I had to go ask the same person for a review again.
SPEAKER_03Um, and it's I've thought to myself, like, if I had a competitor and I could wish any curse upon them that I could, I would wish for them to have blocking code reviews. Although I probably wouldn't need to add that because almost everybody already does it. But it's one of those things that's just like frankly, really dumb, and like uh um it's just dumb, yet everybody does it.
SPEAKER_00Well, and it wasn't a bottleneck as much before, it still was to a certain extent, so I think we didn't notice it. Now that we have this like way, like it's like a tsunami of code because it's not a wave, because a wave comes and goes, but a tsunami just keeps coming, right? And I think that it's forcing us to consider what those truths are and like what the goal was. Like we have we have the same required reviews things, and there's a there are some legal reasons we do it, like as they're common at most companies.
SPEAKER_00Like you have to say that someone signed off on it. The thing that's made me think about is like what does it mean to really attest to a PR? Because that's what you're doing, is you're saying I approve this thing, and it's there's some you're you're committing some act of testimony about a diff. And what's been interesting to me is to study and start to look at what comments are being left.
SPEAKER_00Starting to study like what changes lead to production issues. And these are great things to put AI on. So like I know I've worked with Copilot, we use Copilot on top of you know any number of models, and had it summarize code review comments for a few peers for the past like six months, just to like specifically the comments that led to changes in the code, just as like a study of what is the function of code review for these people. And what it was finding for me is that we had really poor test coverage in some really key areas.
SPEAKER_00And like, I don't say poor because like there was still coverage, but like the systems we had in place were not adequate defenses. We needed people as a defense. And that premise is a very tricky one to maintain in this era of higher code throughput. Because what what premise is? That pe people as a defense, people as a safety mechanism. Like if you're truly expecting, you know, we had this comp just this week. I was working on I I uh my my job at GitHub right now is to keep Ruby and Rails up to date, which means keeping really just keeping our gem file up to date, which is a massive undertaking.
SPEAKER_00Uh and we had an upgrade that like two of us agreed on, but one person very rightfully disagreed and caught an issue that was like somewhere in between us and Rails. It really wasn't us, but it really wasn't Rails, it was like the combination. It's kind of like I forget, there's like a complicated like Git puzzle you can have where like tests pass on branch A and branch B, but the combination of them creates a failure. It's with that kind of like logic puzzle. Um so it just so happened that like this person saved us from like a production issue.
SPEAKER_00Now it would have been the kind of thing that we would have like caught mid-deploy and just rolled back. Like it's not like a huge risk kind of thing. But it really highlighted for me that the safety factor in our systems is under a lot more stress in this environment. And I I don't know if we can like code review our way out of it at this volume with the same number of people. You'd almost need to like change the ratio of staffing on your team to be like one author, five or you can't have like every author can't be reviewing.
SPEAKER_00You get what I'm saying? Like you almost need like to dedicate more of your time to review.
SPEAKER_03Yeah, I I'm familiar with this challenge. Um, and I I've seen people say things like, I'm just drowning in reviews now. It's it's so cheap to comp it's so cheap to generate code and so expensive to review it, and now there's like way more code than there was before, and so there's just this extraordinary burden of reviews. Um, but to me, that is a sign um it's a sign of something.
SPEAKER_03Um, it's it's kind of like a bit of a um okay. I've made this analogy before where like in the universe nothing can move faster than the speed of light, and there's so many like downstream consequences because of that fact. Um and in the systems, um you can't make changes to the system faster than the people responsible for the system can understand those changes, or else you're gonna have some trouble.
SPEAKER_03Um, you know, maybe not immediately, but you're like you're accumulating potential trouble. And you can only accumulate potential trouble for so long before some of that potential trouble turns into actual trouble. Right. And so I think what is happening when people are are buried under this flood of code review is that the people in the organization are not understanding um that even though we have the ability to generate great amounts of code very quickly, the like fundamental laws of the universe still apply.
SPEAKER_03Um and you're not you're not like gaining anything by writing more code that you're not gaining anything by writing code at a higher rate of speed than it can be understood. It's like going to a restaurant and ordering 10 entrees. It's like it's good to eat food, but it's not good to have ten times as much food as you can eat, because like you're gonna eat one of those entrees and then the other nine, you can't do anything with them.
SPEAKER_03They're just waste. Right. So that's that's the same with all this extra code. I I don't know why that's not more like widely understood, but that's that's my take on it.
SPEAKER_00I mean, you do see though, more code is landing. And I and I think that just like the I think the non-controversial thing you can say is that there are more lines of code going to production, more diff lines going to production per capita than there were a year ago. Which means that factually, I believe, the confidence in those lines is lower because I view confidence as like a one-to-one relationship with number of lines of code per capita.
SPEAKER_00Like the more you have to read, the less confident you're going to be in it. Unless we are changing the equation, like or shouldn't say changing the equation. I think we can change that equation. And that I think we can change the factors in that equation. One is you can change the automated levels of confidence you have in the system. You can also change your ability to respond to bad changes in the system.
SPEAKER_00So, like, we're getting really good at rolling back changes. We're getting really good or have gotten really good at deploying smaller changes or feature flagging, for example. Like all these things can make your mean time to recovery, MTTR, is what a lot of people call it, faster. And that can permit certain lower levels of verification, of like manual verification. I think those things kind of have to be true.
SPEAKER_00The nice thing is that AI is great at building those things. Like, so for example, we had a we had a PR, I had a PR the other day. I was upgrading our a library we use to generate PDFs, just a standard, like kind of dependabot kind of thing. And the reviewing team said, Great, this this looks good. I need to go run a script to generate these PDFs and just make sure that nothing is wrong. And I was like, Oh, well, I didn't know to do that. CI didn't tell me to do that. CI was green. You know where this is going.
SPEAKER_00Like, surely we can write a test for that. So I did. And it caught bugs that were already there. Because when you ran the test a couple times, like some things there was an unordered select, classic, like easy thing to miss, where you don't have an order by ID. And then the database doesn't guarantee that it returns things by ID. It returns them by page in the database. So if you have like deletions or additions, things that can come in quote unquote out of order. Like they're not coming unless you like order by the ID or ideally by like a created ad or something.
SPEAKER_00Um so that test uncovered that bug, and that now means that we can move faster when we change that code because we don't need to run that manual validation step. And I think that that is like where AI is the poison and the cure. Yes, it is poisoning us with this huge with tenantres on the table. I don't know if I have the right metaphor here, but because you're still not going to eat the tenantres. But it's I think it can, in certain circumstances, allow us to test our way out of these things by making it cheaper to write the tests that we might not have previously.
SPEAKER_00Maybe even the more useful tests. Like I think it would have been easier to write like a unit test for some of those PDFs, for example, but actually running the code that generates the PDFs, doing an image diff, that was more complicated. Like standing that all up took, you know, changing like our base image for things. And that was something that would have been maybe too expensive to justify, but now is something that can be done as like a very, very, very side hustle kind of side thread of my main work.
SPEAKER_03I want to put a nuance on or point out a nuance in what I'm saying. Um, I'm not saying that I don't think we can ship changes to production faster. I think we can and we have. Um it's that I think before we were making changes to systems at a rate that was slower than the maximum rate we could understand. And now some people are releasing changes at a rate that's faster than what they can understand.
SPEAKER_03At the same time, I agree with what you're saying, which is like if in the I mean, I'm not saying that you're saying this, but if in the past we were releasing changes at 20% of the rate of the maximum rate that we could understand the changes going out the door, then we have room to do 5x that much, and we're still okay. We just can't do anything more than that without accumulating some potential trouble.
SPEAKER_03I think. I just I want to asterisk all that with I'm not a hundred percent sure that I agree with myself.
SPEAKER_00Well, I think that what I would say is that this is another thing that I think has been very enlightening for me is realizing that what I think you're saying was true, but perhaps only at the point of authorship.
SPEAKER_03Only at the point of authorship.
SPEAKER_00Because these systems change outside of your purview. Like you land your code, someone else changes it a year later. Or you land your code, you leave. Like the ability to do things with confidence was always based on the test suite. Because at least in the long term. In the short term, yes, code review is like a kind of a friction point for this where it's like, okay, we're gonna review this change.
SPEAKER_00But your confidence in not making that same mistake that was caught in the code review was entirely almost entirely predicated on the test suite and you know, tribal knowledge, right? Which, like, you know, we have a multi-thousand engineer company. You know, we have a what I want to say, I think it's safe to say that we have maybe around a thousand people contributing to our monolith. But how many people have contributed, how many individuals have contributed to that monolith in the history of the company? It's probably several times that. So, like, those people don't work at GitHub anymore.
SPEAKER_00They can't be coming back to review their code and telling you why they did that thing. And when you build on top of what they've done, you are hoping that you are kind of guessing on top of that foundation. And that's why, to me, I'm starting to feel even stronger feelings against things like code comments, for example, and documentation, because these things are not durable truth. Like we had a situation the other day where we had a code comment that was incorrect, and I went down a thread on that code comment for half a day.
SPEAKER_00And luckily the author of the comment came in and corrected me, but it was a fairly time-sensitive thing. And I think that that's why what this moment has really led me to is studies of automated verification. Like I think that if there's anything where we need to expand our perspective creatively, it's in that realm. Because there are so many things we did implicitly, organizationally, that are just falling over, or really we're kind of already falling over, but we were like staying ahead of them enough, like we were picking up the pieces fast enough.
SPEAKER_00And I think that that is where we're most likely to fall behind as we ship more code, is that is you're kind of saying we're like we're accumulating these potential issues at a higher rate. And I think that to keep the same safety factor, which I think is a totally reasonable goal, that we should say that we have the same number of customer-facing defects in 2027 as we did in 2024, that means that our defect rate might need to be 50% of what it was in 2024 because we're shipping so many more changes.
SPEAKER_00And that shifting that uh equation uh I think is basically a requirement unless our customers are going to accept lower quality software.
SPEAKER_03Yeah, okay. Yeah, you're bringing a perspective that I hadn't really thought about. And you're making me rethink some of my prior positions, uh, which is good. Um, and I'm I'm curious Okay, switching topics yet again, if I may. Um what are you personally like paying attention to these days?
SPEAKER_03Is it different from that's not how I meant to put it? It's like how are you like how are you doing professional development? Um, is maybe the question that I'm that I want to ask. And how has that changed, if it has changed, since all this stuff has happened?
SPEAKER_00Okay, so I have actually had something I wanted to talk to you about in this specific space that I would actually like to ask you, which is what is a tech conference in 2027?
SPEAKER_03Oh man. Like, what is the point?
SPEAKER_00And I say this because I'm on the program committee for RubyConf next year. Oh and we're starting to think about you know, how do we position, like what are the topics we want to do, like what do we want this conference to look like? And I I know you are planning, you've planned conferences in the past. Um Yeah, I and that is professional development, right? Like, how do you think about that in this current age?
SPEAKER_03And did you know that I was involved with RubyConf 2026? Yeah, yeah, yeah. Yeah. Uh in case in case you didn't know, dear listener, I I co-chaired RubyConf 2026. And so, yeah, I faced some of these same questions. Um, for a long time I've thought that conferences are like ostensibly something that exists for professional development. Um, and it's kind of a polite fiction, but really, you know, it's it's like employers often have viewed and still view conferences as like an investment.
SPEAKER_03But I don't think it's exactly it's not mostly investment, it's part investment, but I th I don't think it's mostly investment. I think it's mostly perk. Um like getting to go to conferences is a perk of working here rather than sending your employees to a conference is a way to invest in their professional development. Um and this question, like I think this question only makes sense through the lens of the people who are attending the conferences.
SPEAKER_03Because, like, what is a conference for depends highly on who you ask. Like, for maybe, I don't know, 80% of developers, the answer to what are conferences for is nothing because they don't go to conferences and they never will. People who do go to conferences, why do they go to conferences? Um, I think a lot of them go to like um to like put gas in their tank.
SPEAKER_03You by going to conferences over a period of years, you tend to see the same people over and over, and you develop a social circle that's made up of people who live all over the country and world. Um, and it's funny, some people like end up seeing that group of people more often than they see their like local friends. That certainly has been the case with me, and so that's a big part of it. But I don't want to I don't know if like just calling it socialization is the right label to put on that.
SPEAKER_03It's more like some kind of um fulfillment or something like that. Like we have a deep psychological need to be like part of a tribe, um and so we go to conferences for that, but then as far as like the content of the conference, um like I like I like to be made aware of new things, maybe especially things that I didn't know I didn't know.
SPEAKER_03I'm not a big talk attender myself, and when I do the the talks that I like, it's usually like I see the talk, and there's a little tidbit in a talk that inspires me to go like dig deeper into something once I get home. It's not necessarily that the c the talk itself is a a piece of education that is like complete and usable. It's more like a hint that that I'm and I'm gonna go use that hint later on.
SPEAKER_03Anyway, that's those are some thoughts.
SPEAKER_00No, no, I uh I resonate with a lot of what you said. I think that there is for one, so for example, like I as one of the keepers of GitHub's gem file, I have to do a lot of open source stuff. So the more people that I know in the community, just the more enjoyable that is. Like it's nice to that you know, I've had to do some actually over the past year I've done quite a few like uh open source contributions, and they've a chunk of them have been to projects owned by people that I've met at conferences.
SPEAKER_00And it's just been really fun to have that connection and like they know who I'm I am. I'm not just like a stranger coming in to make a contribution. And I think that there's like something magical about that in our industry specifically, because I think Ruby has always had such an open source bent to it. I'd also say that your comment about like filling up your tank really resonates with me. I find that conferences to me, first and foremost, are like a source of creative energy. And I think it's the spaces they create being around a lot of people who are also who also care and are the Type of people who would go to a conference just fills up fills me up.
SPEAKER_00It also drains me in other ways. But um it's a lot of the same reasons I go to our meetup really regularly. Like we have a Ruby meetup in Boulder that's excellent, that's been going on for like 20 years now. It's incredible. Um and it's yeah that I need that like almost every month. And I think this is especially true in the era of remote work.
SPEAKER_03Oh yeah. Yeah, I think having a remote job, frankly, is like pretty psychologically unhealthy. Like our whole modern way of living is you know, it it's it's not like a novel insight or something, it's like a widely held view that like modern life is not psychologically healthy healthy, even going back, you know, a few decades. Um but uh especially now. Um but something something you said made me think about another reason for going to conferences, which is like intellectual stimulation.
SPEAKER_03I think something you said was like you didn't use those words, but like that's what came to mind. It's like not exactly necessarily learning something new, but being able to have interesting conversations about the things that you're interested in with other people who like are in that world. I don't have personally, I don't have that locally. Um I have to go to a conference to have that, and it's a really nice it's a really nice treat.
SPEAKER_00Yeah, and certain professional situations can provide that. Like I'd say I'm really fortunate to have some great colleagues that the pair programming we do feels like that quite often. You know, like I work with uh my colleague Daniel uh Colson and we like sometimes we'll be looking at like Ruby heap dumps, and I'm just like, whoa, this guy, this guy knows some stuff I don't know. But it's just it's like one of those things where it's like I can just the act of trying to understand what someone else is doing or like learning something interesting someone else did, even if it's not completely relevant, does something for me cognitively.
SPEAKER_00And I think that like in this age of AI, maybe that is like something that is more interesting as like as a counterforce to that kind of deintellectualizing feeling we were both lamenting at the beginning of our conversations today. But you so you asked about like professional development. Like, like what do I actually do? Well, these days I'm reading every PR that lands to Rails, every PR that lands to Ruby, because that's just like part of my work. But I find that to be interesting in and of itself because I'm seeing what types of things are changing in these projects, because those changes are in response to larger forces in the industry.
SPEAKER_00Like I see all these Ractor-related changes going into Rails, and it's just interesting to see. Like we don't have any explicit plans for using that code, but it does affect our contributions, say, to Rails. The other thing is that I'm starting to look more towards maybe some maybe a little more non-technical sources of learning inspiration. So like I'm reading a book now called Refactoring English by Michael Lynch. It's excellent and just talks about writing techniques.
SPEAKER_00I think they matter more and more, especially in an age where like basically all we're doing is writing. I think being a really good writer almost matters more because there's just like so much text to be consumed that like making sure your writing respects the readers in your life, I think is super important. So making that more a part of my practice has been beneficial. The other thing that I'm kind of curious from you that I've been trying to learn about is like, what the heck does like a flow state look like in this day and age?
SPEAKER_00Because it used to be that you could be like single tasking, deep into something, headphones on. And now it's like conversely, because agents are oddly slow, they're both fast and slow. Like the feedback loop is longer, even though they're getting more done. So you're ending up with these different size gaps that I usually fill with working with another agent. Like I'll sometimes have like five or six agents going at a time. Um, but I'm trying to, that's where a lot of ultimately my inquisition is professionally, is trying to find maybe not a new routine, but like uh something that is now sustainable in this new world because I knew what it was to be a sustainable professional programmer, you know, working normal hours two years ago.
SPEAKER_00Answering that question now, I don't have such a solid found, like solid idea of what that should look like in a way that's gonna like make me feel like I'm not burning out slowly or quickly.
SPEAKER_03Yeah. Yeah. Um, so at my day job, I don't do a huge amount of coding anyway, and so that's been like the timing kind of coincided with like when I got a different job and when AI hit an inflection point. Like there's kind of consensus among people I know that like the early part of 2026 was a big inflection point for AI, and that is when did I start my job?
SPEAKER_03I started my job like nine months before that, huh? So I guess that comment doesn't really hold up. But anyway, I don't write a huge amount of of code at work. Um but on Saturn CI, I write a lot of code, my um continuous integration platform that I've been building for the last few years. And I used to uh fill the waiting time with working with other agents, but then I went back to single tasking because I have found that I would get confused.
SPEAKER_03And I've even before AI, I've like tried to switch between two tasks when like, I don't know, you're waiting for tests to run or something like that. And I find that I forget which task is like the primary task and which one is the secondary task, because there's almost always one that's more important, and the other one is just kind of filler, something that is not unimportant, but you picked it just to pick something rather than because it's the thing you need to do today or whatever, and I forget which is which, and then I get like I get my priorities screwed up, and I discover that I've been working on the secondary task for 45 minutes and neglecting the primary task, and I'm the bottleneck, it's just waiting on me.
SPEAKER_03So I'm not being more efficient, I'm just being busier. And so I've gone back to monotasking, and it you know, this is this is a statement that is generally true. Um oftentimes being efficient feels inefficient, and doing things that feel inefficient are actually very inefficient.
SPEAKER_00I agree. I wish that I could have A-B tested myself for the past like year. I feel like this is like a this is something I've been meaning to do. I have a I have a friend, Brooke, he might have met Brooke Coolman, brilliant engineer.
SPEAKER_03I have met a man named Brooke, which is probably not a huge pool of people, so it might be the same person.
SPEAKER_00And uh he doesn't use AI much at all. And what I've been meaning to do is to do a bake off with him. Like I'd love to just like sit on a Zoom call with him for half a day and work on the same like open source project problem, but like see who actually is faster or better. And just kind of really take a step back and evaluate like what the real trade-offs are. I suspect that multitasking can be more effective, but I don't think it is out of the box.
SPEAKER_00Like, I think this is kind of what I've been getting at in our conversation that I feel like there are ways of working that we have yet to totally discover, or maybe we've discovered them, but we we haven't fully isolated them that will allow us to better thrive in this new environment. So, like, for one, figuring out how to manage the code review workflows is a is a new challenge. And I think that if we we might find a more clever solution, for example, that like makes that feel different.
SPEAKER_00Or we make our like so, for example, one thing I've been doing now is I have when an agent generates a PR, I have it do a self-review explaining all of the code with comments, like what change what necess necessitated each change. Uh like examples of like why each test case was written a certain way. And that makes the act of code reviewing easier. And I have to wonder whether for a lot of these things, whether we're thinking about that them in the old way, which is why they feel and probably are to a certain extent less efficient than we think.
SPEAKER_00Um I view that as a temporary phenomenon. I think that we will learn new ways of working within these systems, and we will come out the other side more productive.
SPEAKER_03Something else that I expect to happen is I expect LLMs to get faster. Right. And so you'll give it a prompt and it'll say done. And then the the ball is back in your court and it it spends basically all its time in your court because every time you pass it over to the LLM side, it just hits it back immediately instead of taking a minute or whatever it might take now. So that's something that I'm always thinking about because I don't want to, you know, remember when there was that whole prompt engineering thing and people were constructing these big like JSON prompts and stuff like that, and it's like I've seen those kinds of things as they have come and gone, and I'm just like, guys, like the AI is just gonna get better and make what you're doing irrelevant sufficiently soon that everything you did was pretty much a waste of time, and that's that's what's been happening.
SPEAKER_03Um, and so I'm consciously trying not to get too comfortable with the way things are because they're probably gonna get different soon. Okay, having said all that, uh I do a similar thing to you. Like I'll ask for a change, um, I'll look at what the change is, which may or may not mean looking at every single line, but I at least want to get the essence of the change. Um, and I will often have sometimes I just look at it and it looks fine and good.
SPEAKER_03Um, sometimes I'll say, give this a review, and I have a specific like test design review skill that I wrote, and a software design reviews skill that I wrote. Um, and part of the reason I I put it through those is because like sometimes I see code that I know is problematic and I want to experiment and see like, okay, this thing that I have in my head of what I don't like about this test, for example, will you find it if I tell you just to review it?
SPEAKER_03And if not, can I like change the review criteria so that from now on you will find such things? I find that to be a very interesting exercise. And so I've been accumulating these things, and I found that it gets pretty good. So that's that's how I do that part.
SPEAKER_00So when you're working this way, when you have little gaps, when you're waiting for the LLM, what do you do? What do I do? Do you look at the wall?
SPEAKER_03I don't look at the wall. I don't know what I do. Maybe I watch a YouTube video?
SPEAKER_00That's what I do. Or I have been doing. I'm I'm trying not to now because this is the thing I'm I'm fascinated with is like these kind of cognitive timelines we put ourselves on. Like what is your brain doing? Like what is your time budgeting for your like during your pro like a pr an hour, like the best hour of your day? What does like the 10 second blocks look like? Almost like a you know, a uh flame graph.
SPEAKER_03Yeah, I don't know anymore. Um but going back to what you asked a second ago, a habit going back many years is that sometimes instead of task instead of context switching on the screen, I will have a book in front of me. Yeah. And I'll leave the screen where it is and just move my eyes down to the book, and that makes and there's two reasons why that why I find that to be better.
SPEAKER_03One is because I can still monitor the screen continuously, and the other is because the secondary task is so different from the primary one, it doesn't like compete for headspace.
SPEAKER_00Agreed. Like the it's almost like the more different, like the like the most distracting thing is gonna be another programming task. The least distracting thing might be folding laundry. So my answer, I I wrote a blog post about this recently, is like how I find that finding non-work distractions makes my work productivity much higher.
SPEAKER_03Yeah. Mm-hmm.
SPEAKER_00So, like for example, uh I clean my garage once a week because the kids, and I have headphones on, and I can hear the pings from co-pilot. So I'll like break down a box, I hear a ping, go over, type the input. Enter. Yes, go do another like little thing. Or like I I ride my bike a lot, so I go, I have to do a ton of maintenance on it because I'm riding like a hundred miles a week right now. Um, so I'll like put the air in a tire, hear a ding, go over and do it.
SPEAKER_00Like it's a really easy, interruptible, non-technical task. And I find myself having a way higher level of mental clarity in that mode than when I'm filling it with anything that involves technology.
SPEAKER_03Yeah.
SPEAKER_00And that my like my my reasoning level is so much higher.
SPEAKER_03Yeah, I wasn't thinking about that for some reason, but I definitely do that. Like I'll have my computer on the kitchen table while I'm making dinner, and it's debatable whether this is like a healthy way to live, but it's like put some onions in the pan, that's gonna take a minute, go over, type a prompt, go stir the onions, go back to the computer, type another prompt. I have a feeling like a Buddhist would not approve of that way of living.
SPEAKER_00Okay, so you're reminding me there is, I mean, this is the note we can end on, is that there's a book called The Zen. Is it the Zen of Software Engineering? The Zen of Computer Programming, something like that. And it's a short book, but it talks about so the author talks about some sort of regimen that involves meditation and exercise as part of the programming habit. And this is old-ish, so it definitely predates the AI world, but it would like encourage you to do like sit-ups after like complete writing a commit or something like that.
SPEAKER_00Or like go for a walk, or like it like it created, it was like some sort of like practice around doing physical things as part of your active computer programming. And I find that to be way more useful now. Like I have a I have a thing my eye doctor makes me use, which it blanks my screen every 20 minutes, and it because I'm supposed to look at least 20 feet away every 20 minutes for 20 seconds because it reduces like the headaches I get. But what I've been doing now is I'll just do like literally go walk down the hallway or back, or I'll do some sit-ups or some push-ups or something that uh helps my body move a little bit.
SPEAKER_00And I find that is cognitively beneficial.
SPEAKER_03Yeah, everything is connected, like getting enough sleep is it has a huge bearing on how you put how you will perform at work the next day. Um we should probably start heading toward a conclusion soon. Um I I want to ask you about one more thing. We could probably do a whole nother episode at least on this, but um we've been exchanging emails for the last I don't know how long, couple months, because I keep not responding to your emails for a long time, which I apologize for.
SPEAKER_03It's all good. It's all good. But you're working on a device, right?
SPEAKER_00Yeah, so I have uh a project I did what started what 10 years ago. The premise is that having the physical totem of your cell phone near you has a psychological effect on you. And this has been proven in studies. It's been proven that people are less attentive, literally having their phone within reaching distance in social situations. Like if you're having dinner with your wife, for example, like phones on the table, your brain is in a is functions differently than if that phone is in the car.
SPEAKER_00So when my wife and I got married, we decided that we were not going to have any electronics in like our bedroom area. But that quickly fell over because we realized that we still needed, or at least wanted some information from the internet, like our calendar, like what the weather was like, basically stuff that helps you like pick out your clothes, talk about like these days, like who's gonna have which kid when, that kind of thing. So I started with some prototypes using old jailbroken Kindles, which are really nice because they don't have a light.
SPEAKER_00Like you, they just get lit by whatever lights you have in the room. And that has evolved over time to some more finished devices. I now sell using some open source hardware. Uh the project is called Timeframe, and it's this kind of e-paper-based solution. We have a couple of the screens in our house of various sizes and fed fidelities, and it's a really interesting system. And I'm starting to like play with having it be a product. It's kind of in a very prototype phase.
SPEAKER_03Yeah. Well, I look forward to um getting my hands on one of those. The only the only thing stopping that from happening is me responding to your email you sent me about it. Um do you do you have something online that people can go look at?
SPEAKER_00Yeah, it's at timeframe.app. And especially for your audience, they should know that it's open source, or at least the core of it is open source. So you can run it inside Home Assistant and it will use like all the weather and calendar data, et cetera, from Home Assistant. It will also display Home Assistant status things. Like, for example, we have it set up so it tells us whether all our exterior doors are locked, which when you have kids is really important because our kid, our especially our toddler likes to open doors. So it's really nice looking when you go to bed at night that like it shows that all our doors are locked.
SPEAKER_00Um, so yeah, you can run the core software for free inside Home Assistant. I also have a cloud version, that's what I charge for. And then you can run it on hardware that I sell, or you can run it on it's there, uh there's some firmware you can run on your own devices, or you can run it inside a web browser and you can run it on basically anything then at that point, a TV, uh, you know, computer monitor, etc.
SPEAKER_03Any other links you want to share before we go?
SPEAKER_00Just my website is hoxa.org. I blog there occasionally.
unknownYeah.
SPEAKER_03All right. We will put those things in the show notes. And Joel, thanks so much for coming on the show.
SPEAKER_00Hey, thanks for having me. Look forward to the next time.
In this episode I talk with John Athayde, designer-developer and co-author of The Rails View, about what architecture school taught him about software design. John tells the story of a user test at LivingSocial that...
In this episode I talk with Riccardo Carlesso, Google developer advocate and co-founder of Rubycon, a Ruby conference held in Rimini, Italy in May 2026. Riccardo explains how a joke about PyCon became the conference's...
In this episode I talk with Travis Dockter, organizer of Blastoff Rails in Albuquerque. When Sin City Ruby and RailsConf both ended, Travis decided to start his own conference. We discuss how cheaply you can run a...
In this episode I talk with Julien Danjou, CEO of Mergify, about merge queues: why PRs that pass CI separately can still break main together, and how testing changes in a queue keeps main green. We discuss how Mergify...
In this episode, I talk with Kevin Dias about his journey from investment banking to programming in Japan, building Ambiki for speech language pathologists, and tackling feature requests with a problem-first mindset....