Lada Kesseler's approach to loop engineering is brutally simple: give the agent one step, make it write the result to a file, then read it back and go again. She calls it a centrifuge. Spin fast enough and the stupid comes out. The Logic20/20 principal engineer has spent the last year and a half building the ground rules, skills and refinement loops that fight the defaults coding agents ship with, starting with an instruction to stop trying to please her. Along the way she explains why she trusts AI-written tests less than AI-written code, why the description field in an agent skill was never meant for humans, and why most teams chasing a software factory are about to drown in garbage. What we cover: - Why agent ground rules have to give the model a mission to disagree with you - The refinement loop that gets to quality when the first attempt never does - Why an agent harness only works if it does one thing at a time - Sketch prototypes: replacing your code with a markdown file and an agent - Why AI-generated tests are more dangerous than AI-generated code - Software factories, and what happens when you try to switch into one overnight *AI DevCon is coming back to New York this November* Get 15% off your ticket with code POD15 https://tessl.io/devcon#tickets Chapters: 00:00:00 - Introduction 00:03:38 - Using AI for everything, not just code 00:06:15 - Ground rules: fixing the defaults agents ship with 00:10:02 - Agent skills and why the description isn't for humans 00:13:09 - Loop engineering: how you actually get to quality 00:16:03 - The biggest misconception about coding agents 00:20:43 - Sketch prototypes and the limits of AI architecture 00:27:48 - Reverse direction: you are the decider 00:31:58 - Why she trusts AI tests less than AI code 00:35:00 - Software factories, or drowning in garbage Build your software factory, one workflow at a time, with Tessl: https://tessl.co/k2t 🔔 Subscribe for weekly episodes on AI-native development Where are you on the loop engineering curve, still hoping for a good first try or already spinning? Tell us in the comments.
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_00Whatever AI produces should be good for the first try and never never the case. So how do you get the quality refine? This thing is only good at doing one thing at a time. You have to almost give it permission to disagree and fight that in your rules. Hey, tell me honest things, don't lie to me, don't try to please me. When I started doing that, I noticed that I was like, ooh, it's less nice to me. I call it like a centrifuge. Yeah. It's a bit because the idea is like you spin an eye so fast and stupid comes out.
SPEAKER_04The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding. Join your host, Sky Pajani, and meet Simon Maple every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today. This is the AI Native Dev. Yesterday, you were writing code. Today, you're prompting agents. Tomorrow, you'll be running a software factory.
SPEAKER_04Agentic coding is moving fast enough to give anyone whiplash. This June, AI DevCon London sold out. Hundreds of developers, engineering leads, and VPs packed in to hear speakers from Anthropic, OpenAI, Netlify, and GitHub cut through the hype and get to what actually works. This November, we're bringing it back to New York. Bigger, bolder, and full of people actually building the agentic future. Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams, or setting the strategy.
SPEAKER_04Our early bird pricing ends on September 30th, so make sure you lock in your place before prices increase. As a podcast listener, you can get 15% off your ticket using promo code POD50. We have so much planned, and we can't wait to have you join us in New York. We actually wanted to uh have a session with uh Lada uh Kessler a number of times actually, and we tried to schedule it a number of times, and then we bumped into each other in San Francisco at AI Engineer in San Francisco, and so we thought, hey, why don't we just do this in person?
SPEAKER_04So welcome to the podcast, Lada. How are you?
SPEAKER_00I'm doing great. Thank you so much for having me.
SPEAKER_04Oh, it's an absolute pleasure. Um now, Lada, you are uh uh working as a principal um engineer at Logic 2020, right? Tell us a little bit about that.
SPEAKER_00Yeah, basically I'm working at a contracting company. Um it's kind of a niche one, uh, but nowadays we're all over the US, and I get to go do a lot of various things. Um right now I'm trying to do everything in my power to make sure that they are about to.
SPEAKER_04Yeah, amazing. And you of course gave a session at AI Native Dev. Uh, which one was that? That was in that was in New York last year, wasn't it? In November uh last year. Really loved by uh um all of our attendees, and you got some super high scores. I remember I was chatting with Patrick afterwards, and he was like, we have to get Larda back, we have to get Larda back. So um, so yeah, we'll see what we can do for this year. Um, but it's wonderful to have you on on the podcast. And by the way, uh, we are today is the first of July. It's currently 10am, just gone 10 past 10 um in San Francisco, and England are playing live, and they're playing so poorly.
SPEAKER_04It's actually 1-0 to Congo DR right now, they're playing so poorly. We saw let's not watch the second half here, let's let's have a podcast. That'll be more interesting, but we'll keep you up to date on on scores as we go.
SPEAKER_00We're still watching them.
SPEAKER_04Uh I'm well, we have the kind of score there, but we're not watching the game. Uh that would be rude. That would be rude. Um, so so um so tell us a little bit about you know your involvement with AI. How how um how are you using AI today uh in your day-to-day or your hobby work?
SPEAKER_00Pretty much completely at this point, really. Not just software development, so I do pretty much everything with AI, or the majority of things I do, I do with AI. Um so like it starts with code, obviously. Uh, and then I do a lot of code uh that like the systems I code have potentially some agents in there as well, but not always. And what I'm actually passionate about is figuring out how to how to get the coding bit, right? So and I find that oftentimes when you actually like working with a coding agent, you also get much better intuition for to how to write a genetic system.
SPEAKER_00So I think it's very, very much linked. Um but then things I do on top of that are like uh so I do a lot of note taking with AI, but like I used to be very disorganized, and like Obsidian and things like that didn't work for me very well. But now I have basically a folder where I have a drop folder where I have my notes, and it has a system of organizing things for me. So I'm like, hey, process today's notes and goes and uh like okay, ask me some questions sometimes or like organize it. So now I'm like, oh, so well organized.
SPEAKER_00I like that.
SPEAKER_04I've totally just built this for myself as well. So I actually connected it with Slack, my granola notes. It's it's I've I've I've completely opened up now. It knows everything about me and it tells me what to do. And it's like, you know, what should I do today? And things like that. So it's good to see it's not just me.
SPEAKER_00So far, I'm I'm restricting it from accessing my email.
SPEAKER_04I'm trying, I'm trying very hard not to go this way because I know its temptation is there, but I've I've I even allow it to write some of my emails, but I haven't yet got to the stage where I'm comfortable with it sending at all. And when it does write my emails, I tell it what I want to write and it just words it for me, and then I go into that and kind of like adapt. But it's it's curious the the path that we're all on here and how much we're relying, you know, we're relying on this to almost be uh, you know, we initially kind of like focus on the code that we review very heavily, and we're probably gonna see the exact the same uh levels of of validation and verification in the in the way that we present ourselves as well.
SPEAKER_04So if I write an email, that's me talking to someone else, and they're expecting that email to come from me.
SPEAKER_00Right.
SPEAKER_04Uh and so I want that to be validated to make sure it's not just AI slot that's being sent. Otherwise, yeah it's making, you know, it's giving me a bad reputation by me sending shocking content to someone else that I, you know, it's it's almost rude for someone to for me to expect someone else to read some AI slot that I've generated.
SPEAKER_00That's very rude.
SPEAKER_04So so let's talk a little bit more about that, because by default, agents aren't always the best at getting at nailing something that we you know want.
SPEAKER_00Yeah.
SPEAKER_04So how much does like the enablement of agents uh matter?
SPEAKER_00Basically, I think about it as like AI comes with defaults that are not very helpful. And you you kind of have to adjust them in your systems. So one way to do this is basically so they all have like this thing that I call ground rules, basically. This is your cloud MD or something like that. And you can basically define here's my here's my here's the things that are most important for me. Like you need to be careful to not just put all everything in your history and and your TDD process and your like everything else in there because it also has two not enough memory, right?
SPEAKER_04Um and your ground rules though, when you talk about your ground rules, how much of that is specific to the project versus almost like more your organization or your preferred way of development that you can just share across all your projects.
SPEAKER_00Alright, so I have both levels, right? So, first of all, my my my big one, my my big one that is applied to everything. This is like the ground rules that are fighting the most like uh the most problematic things that I see across all the AIs, no matter when they are. So several things. So for example, if you think about like so we're dealing with a black box, we don't know what's inside. The ment there's a mental model of some sorts in there. It when you talk to it, it understands something, but it's completely invisible. You don't know what's what's going on.
SPEAKER_00And it's also like trained on purpose to please you, right? So what I have a lot in my ground rules is like, hey, you actually like you have to almost give it permission to disagree and and and and fight that in your rules. So that's what I do. One of the things is like, hey, tell me honest things, like don't don't don't lie to me, don't try to please me.
SPEAKER_04Does it does it follow those? Or because it almost like feels like that.
SPEAKER_00So it's a little bit like prompt engineering. It's much better. So so when I started doing that, I noticed that I was like, ooh, it's less nice to me. It feels less nice. Like this initial, like, it's actually being like, oh, you see you're so great and so on. Like, I have allergy to this to the like I think I have I need therapy for this at this point. But but like I noticed the DOP. Yeah. It was it was substantial. Yeah, and now it's actually like before the latest opuses, it was pretty great. Nowadays it also like tells me that it's being honest, which is really annoying.
SPEAKER_00It's like honestly speaking, like, oh my god, stop doing this. Uh so yeah, but and then and then I have productivity rules, and productivity rules are basically hey, if you notice some problem, tell me now. Yeah, and don't actually commit to the thing that's happening.
SPEAKER_04And so those like checkpoints? Um or does it naturally do that as part of everything it's it's so the twofolds, right?
SPEAKER_00So first of all, it's in my ground rules, like this is like basic, like here's our contract, here how we work with Lada. And also I have to reinforce it or I have to do checklists when I do the work on on time. Yeah. And I kind of feel like I need both layers. Okay. Depending on the size of context, this second thing helps a lot as well.
SPEAKER_04And why do you put all that up? Did you mention you put that in your Claude MD or that's the same thing?
SPEAKER_00Yeah, the ground rules, the basic ground rules are in Claude MDT in the user level for everything.
SPEAKER_01Yeah.
SPEAKER_00Yeah, yeah. But like my Claude MD, I know every line there. It's 72 lines, and I'm very careful about how much stuff in there. Yeah. Right. And then um and unlike system prompts, it feels like um Anthropic just puts a lot of things in there. System prompts, it's it's a bit, yeah, not great. Um, but yeah, I also have things on on like I slash commands when we just invoke in time.
SPEAKER_03Yeah.
SPEAKER_00Right. I sometimes have some things on the project level, but I'm I'm careful what I put there again. And I'm more like, here's a menu of what you have, and then you you have you here's a way you discover more. It's a more skill-like, right? Somewhat something something like that.
SPEAKER_04And it's more of a progressive disclosure. Um in terms of skills, what what kind of what kind of skills do you feel like you rely on most heavily in your day-to-day engineering?
SPEAKER_00TDD, I think, is the best.
SPEAKER_04So why is TDD on a TDD is presumably a one-liner in your clawed MD and then Oh it isn't?
SPEAKER_00So it's so No, it's it's a scale right now. It's a skill. It used to be a slash uh slash command.
SPEAKER_04Yeah.
SPEAKER_00Uh so thing like so many people are saying, oh, TDD doesn't work with AI. Yeah. And the way when I asked like how are you holding it? And they're like, oh, we just tell it to TDD and it doesn't, because it like duh, it doesn't work, right? Um what works really well is like what what I what I have is basically a process. Hey, do this, do this, then do this, and this. And basically I just what I did how I got to it is I basically TDD'd with it uh for for like a session. And then I was like, hey, record everything we just did. And like so my I micromanaged it at first and then record everything they had.
SPEAKER_00Now I have it outside. Now I say, hey, clear session, let's TDD. And I watch every everything where it messes up and I correct.
SPEAKER_04So you you ask it to TDD?
SPEAKER_00I ask to TDD.
SPEAKER_04Ah, okay. So I was expecting a one-liner in your Claude MD that says make sure you always do follow the TDD rules, and then a skill that explains the TDD rules. But if you if you're explicitly invoking it to the TDD, you're kind of in control.
SPEAKER_00No, it's it's it's a little bit of both. So I have like um basically I have some rules about code, but not too much in my in my main thing. I uh mostly like I like simple things. If you can do things more simply, prefer that. We prefer that to hockey solutions and stuff like that. But um with TDD, basically I have a skill, right? And the skill basically has this uh this little front matter at the top, right? And that's the thing actually that's written for the agent for it to know when to invoke it. And the whole point is like, unlike the old MCPs that just you know shopped all the instructions for itself in your context and just garbaged it completely, you have just this little thing over there, and then the agent sees this, they're like, Oh, I have a TDD skill.
SPEAKER_00I'm supposed to do it every time I write code. Oh. So basically I have that, and that's what makes it automatically do TDD.
SPEAKER_04That's right, that's that's the activation, right? And that's the bit, I think the description is the piece that most developers, when they look at a description, they'll just they think it's almost like that's for humans. Yes. I'll just write a very lightweight, a very light rate description, and they completely mess up how that skill then gets activated. So that's how the agent determines whether it should use that skill or not.
SPEAKER_00Exactly.
SPEAKER_04And when you have many, many skills, it's also there to help the agent decide. And if they're very generic, then they're gonna overlap so so much. The agent's got no chance.
SPEAKER_00Precisely, precisely. That's why I have like a skill factory basically that focuses on that explicitly. Because I I find that anthropic default one doesn't work very well.
SPEAKER_02Yep.
SPEAKER_00So and and I sp like I see like many lines of uh in front matter, it produces like all this kind of stuff. It feels like it's not nine, nine, it's not good. So I basically have like a one of the processes in the in the skill factory is like, hey, iterate over this. And in this context, like the this is what it's supposed to be like, and and try many times and like go into that direction doesn't isn't good enough. And then it shows it to you, and and and it's usually much better than the defaults. Nice.
SPEAKER_04So it's just an iterative way of making sure it's uh it's concise.
SPEAKER_00I call a thing into a file, so basically this is a a refinement loop. This is just super powerful. So the process goes like this. You basically have a goal, some goal with the eye, it doesn't matter what, then you have it take one step towards the destination you want. Just one step, which is really important. And then you commit or you put it into a file. So for example, like I actually write with AI. And I'm a bit scared to say that because so many people are making a point to say that they are not writing with AI. Because I think it's expected that whatever you produced at this point is first of all not your voice and slop.
SPEAKER_04Right? When you say write, like author, like write something.
SPEAKER_00I don't know, like for things for conferences, for example, like a little blob for for myself. I would I would write it with the eye. And the end result is much better, but like it's so much my work.
SPEAKER_01Yeah.
SPEAKER_00There's so much more my work there. But like the way I get to that is like, hey, here's the thing, drafty things, thoughts, like maybe something. I don't like this, I quite like this. Go it to write, go think into a file. I call it go meditate.
SPEAKER_01Yeah.
SPEAKER_00Basically, it writes the first iteration to a file. Yeah. And I say, stop, read it back. Here's the goal still. Is it good enough? Or this is something to tweak, write it back, and so on. So I have like seven, it goes on like I call it like a uh centrifuge. Yeah. So basically that is like you spin-eye so fast and stupid comes out.
SPEAKER_04Yeah. I love it, love it. Yeah. That's really interesting. I and I think actually I I do the I do exactly the same from the point of view I use it to write a lot. And I feel guilty as well when I'm like, uh, you know, if I send this to someone, you know, do I need to de-LMIFI the uh the text so that it it's it feels like someone really thinks I wrote it myself. Right. And you know, the amount it's it's the guidance that we add in and the direction that we add in and the reading it and and and telling it, no, this is wrong, that's right. That's what is the important piece rather than the fact that we're whether we're writing every single word or not.
SPEAKER_00Right.
SPEAKER_04I think there's a there's an amount of us as humans that need to kind of like get over that that it's a mental barrier of that. And I think a lot of people, if they if they say they're not doing any of that with AI, they're probably lying to themselves a little bit more in terms of maybe, maybe they're trying to make a point of not doing that for real.
SPEAKER_00Perhaps, perhaps. And maybe that's a good thing. I think there's just like this um this expectation that I see a lot of people having, yeah, which is like whatever AI produces should be good from the first try, and never th never the case. It's just never the case. Like, so let's stop expecting that. So, how do you get to quality you refine? So this kind of processes of refining it together with human input. A lot of steering. So I have basically a lot of centrifuge, and then okay, I don't quite like this. Uh this doesn't sound quite right. Go go meditate again. And so on. And after like five loops of that, I'm at a place where it actually sounds much more like me than I would have done other myself, I think.
SPEAKER_04I can I can I can hear in Tom's head right now, he's thinking, how do I put Simon into a centrifuge right now to get the stupid out of him? But that's a that's another question. But let's let's talk a little bit about, you know, in order to get the best out of AI, it's super important to understand the problems. Right. Um understanding the problems and working around those problems or or you know circumventing those problems will help us get that better solution. So when we think let's let's let's think more about coding now. What would you say are the most common pro problems in and around coding agents that we should understand better so that we can actually produce a better output?
SPEAKER_00I think there's twofold. Uh so I think um so going a little bit back to what we discussed about like the problems with the eye, so there's some problems with the eye, and there's I find that there's some problems with mental mental expectations of people around how EI is supposed to work. So, like a lot of things I see right now, like especially new people when they come into it, they're like, oh, um, there's ground rules, I'm just gonna put everything I care about there. So let's just put my TDD process, let's just put all my like best practices and do this, and make sure not to screw up and make no mistakes and do this.
SPEAKER_00Uh so thing is that that thing doesn't work this way in practice. Like I'm I'm I'm not trading models, so I actually am getting more and more curious about how they work internally, but it feels like they have a concept of attention, and that's probably the biggest thing that I see people not understanding. Um so even like yesterday that was an interesting, interesting, really, really good talk from Human Lair, two talks from Human Layer, I really like them both. And one of the things that I was disagreeing with uh was that the person was like Dax, not the person, uh he was like um AI, if AI knew how to write good code, it would have done it.
SPEAKER_00And I think that's so wrong. Because what what I see a lot is basically so people are trying to say, hey, uh write write me, do this task for me, first of all. This is a task, and also uphold my standards. And they add a lot of things on top here. So uh in my experience that doesn't work, and I think why it doesn't work is because this thing is only good at doing one thing at a time. So you can't expect to just enforce your standards like this way.
SPEAKER_00So what you do instead that works amazingly well. And and and like he was basically here's something that he he knows so much about this, like I read. But he's like if if AI knew how to code, write good code, it would have done it. Uh thing is if you go and ask AI after it read something to find all the problems with it, it will. And if you go into different directions, you can get so much better like reports about what's wrong, yeah, and you have get it to fix it there. But you don't do it in one go, you do it in many goes.
SPEAKER_04This this resonates so so much. Right. Um it actually reminds me that so this week we actually um launched Tesla Agent. And and I'll I'll mention this for one reason that um there's a cool, really cool thing that I love called verifies uh in in Tesla Agent. Yes. And you can write you can use Tesla Agent to kind of like kick off code reviews. We have a basic, well not a basic, it's a it's a reasonable code review in there as well. Um and so you can use it to kick off your code review. Now in your code review, you might have a hundred things that you're trying to check for.
SPEAKER_04And if you say go and do a code review, it'll it'll do a reasonable job. But you want to show yeah, exactly. You you would if you look through every single piece, you might think, oh actually, you you didn't do this. Whereas if I was to say to an agent, just do that, it will do it and it will do it well. Um now a verifier is very interesting because it takes a very specific thing and sometimes triggers deterministically on various files. Like I need a front-end file to do this on or or whatever. Right. And it will say, Did you, you know, can you check, can you verify that you're adhering to these standards?
SPEAKER_04And maybe it even points to the standards in this in the skill. And it will go back and it will do that. And when you have each of those verifiers that are all doing that, you actually get a far, far better result, and it's faster as well. Yes. Uh, because it's it's it's uh the verifiers are, like I say, deterministically triggered and they run very short short, kind of like um processes. Really, really interesting. That really backs up what you're kind of saying. That's like very similar experiences.
SPEAKER_00I think I'm going in a similar direction with friends. Like we're doing something similar, so we have deterministic triggers that detect some code smells, like long method or a dead code, and then it triggers something, and then that that thing can be pluggable, so it can be like, hey, plug an agent to that to be like, hey, here's a prompt to do this for long methods, here's a long method, let's make it better, and here's how recipes so one focused do this, or it may be invoking a linter or a formatter, even right. So I'm trying to do as as much automation there as was possible as well. But yeah, and focus I think is something on the focus.
SPEAKER_04Yeah, absolutely. Let's now maybe turn more practical. Uh, and if we think about um, you know, various um kind of methods of of writing software. Um before AI as well, this was potentially a problem. We would I you know, even by myself at times, would just think, oh yeah, I know I know what I'm doing here, and I just write, just knock out as money, lines of code and you know, say here's the app, I didn't even need to think about it. And sometimes that works. Now, with agents, we can do that like that.
SPEAKER_04We can do it very, very fast and just say, This is what I want, just go and do it. But in larger systems, maybe we need to think about that a little bit more. Maybe we need to plan it and architect it. And I know you do a little bit of architecture in your role as well. You play with architecture as well as uh engineering. So um what's the what's the right process as a as a as an engineer to um to to approach a larger, more complex uh task? Do we just go to the agent and do a do like a you know rapidly rapidly applicate a rapidly developed application first, prototype, then do architecture?
SPEAKER_04Should we whiteboard first? What's the what's the approach you use?
SPEAKER_00Oh it depends so much on what's going on, right? And how much I understand the problem is the biggest the biggest thing here, the b biggest variety, like what would I actually do? For established projects, um, I would usually go talk to AI and like prototype think together. So there's a there's a whole prototyping session before.
SPEAKER_04But and then I I tend to plan mode thinking together or just like knocking an app out and learning through it?
SPEAKER_00It's a lot of back and forth, and I asked it to produce me artifacts. So like I I really like HTML right now. So it produced me HTML artifacts, and I can I can comment on each individual one, and then I can send all of that feedback to the agent, and it's much easier. It's like a whiteboard a little bit. So I more and more like I want this whiteboard with agent, and I'm trying to build one as well. But like that that works pretty well on the established thing that I all all things that I understand. I also have another process entirely for the things that don't understand, because I find sometimes like, oh, we're trying to do this thing.
SPEAKER_00Uh, neither the person I'm doing it with nor me have a good understanding of the problem space yet. So not let's not even just go coding at all. Because like when if you start coding, you're like overcommitting to a specific path, just this one, and you're blind to the other varieties, and you're like you're so focused on solutioning early on that you like forget about like what are you actually trying to do? And maybe there's a better solution over here, maybe it's a completely different solution that wants to come out, but you're not looking at the problem yet. So I see so many people looking at the jumping into solutioning before thinking through the problem.
SPEAKER_00Uh so one thing I do with some people like that was that's pretty fun, is like I call it basically like a sketch prototype. Like, I don't know, it's marked marked down. Basically, I I take my code and I replace it with an agent. In no way put it into production, please ever. But basically, this is for very draft. So basically, and and the code is instructions in a file. So whole my whole system is marked down file plus agent, and the agent goes through processes, and I get to experience the user experience from that. And I learn so much. So basically, then I actually commit to code or like go into some different directions.
SPEAKER_00So that's been working pretty well in some some cases.
SPEAKER_04So you take that you you you're essentially the user trying to try to understand how a user would go through that and then from that build the better.
SPEAKER_00And the agent basically can be stand-in for code, for any code whatsoever, yeah, if you want to, but like it won't be deterministic, it won't be good, but I don't need good at this point.
SPEAKER_04So it's almost like you're you're mocking the current, you're mocking with an agent.
SPEAKER_00Yeah, it's like a pretend product, it's like your paper prototypes I used to do, but it's much faster and much easier.
SPEAKER_04Yeah. Very interesting. And when obviously, you know, from an architecture point of view, a lot of people talk about, and myself included, talk about how developers are kind of more going to be driving teams of agents and moving more into the kind of, I guess, the architecture style role. Why can't an AI do good architecture?
SPEAKER_00So it might on the scope of like one specific task do a pretty good job. But what I find is like there's twofold problems, right? First of all, um oftentimes we are not able to keep up with it, right? So the question is like many people are doing different things, but many people don't read code. And many people are, I think, finding like a vacuum in their heads. Like where there used to be a problem and solution space is now nothing. So so but but I know that if you just let something happen, uh it will probably degrade, you know.
SPEAKER_00We live in the universe where entropy is a thing. So and AI kind of does a decent job, maybe if you do the the iteration on one task, but then it can't hold everything in his brain, it can't do all the things all at once, right? Like what we discussed, right? So maybe you can iterate, and like I haven't figured this out with AI definitely. So but maybe you can like run several iteration loops and think about this, think about that on this level. And you have to do it on the different levels of architecture, right? You can't just do it on one yeah, even like on this scope, it's too much.
SPEAKER_00Um, so it's it's a problem of complexity management, I think, quite a lot, and also like holding this mental model. And I don't know who holds this mental model right now. So I I kind of have to.
SPEAKER_04And do you see that long term? Short term when when like how how I know it's really hard to actually, it's a it's a tough question because it's impossible to really guess what's gonna happen in three months. Uh but I'll ask anyway, because I can do that as a host. Uh like how close do you feel we what we are to AI actually being able to do a good enough job, which is the danger, right? Because as soon as it can do a good enough job of the time, it'll it'll it'll try and do it.
SPEAKER_00All right. I I have no idea about time scales and what will happen. It feels like we're building a system that builds other systems, right? Yes, yeah, it very much goes and it feels so tempting and it feels so interesting right now. It's like the most interesting problem out there, I think, at least for me.
SPEAKER_01Yeah.
SPEAKER_00To build that system. And like, can I bake my engineering practices into that system so it does all this stuff for me? So I actually build things, but but on a high level, I think that was an allure that like people were trying to get to for a long time. I think it's actually might be reachable. I don't know if it's reachable. So there's some problems that I think are unsolved right now. Um will they be solved? Maybe.
SPEAKER_01Yeah.
SPEAKER_00Especially if we find something else that works much better and can actually learn as we go.
SPEAKER_04Hey everyone, hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development. Whether that's talking about the latest tools, the most efficient workflows, or defining best practices. But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favour and hit that subscribe button. It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
SPEAKER_04Alright, back to the episode. So, a quick game update. Harry Kane just scored in the 75th minute, so it's one all, England still have a chance.
SPEAKER_00Going land.
SPEAKER_04Let's get back to practical matters. Um, let's talk about um maybe some other tips of what you think developers uh should be doing, could be doing, that they tend not to uh naturally do in their agentic coding.
SPEAKER_00Right. I think what I see a lot of people are like, I think it sounds like a conversation, like you're having a conversation, so you kind of behave like in a conversation. So like it asks you a question and you're like, oh yeah, sure, let me answer you AI. And I feel like many people are not thinking high level, and like, hey, you don't have to actually go where it asks you to go. You actually can steer quite a lot. So I call it reverse direction. You basically decide like you're the decider here, you don't have to uphold. And also if you you don't like something that the eye does, like it gives you a list of like 10 10 questions, and some people sit there and answer every one of them.
SPEAKER_00This is so painful. Don't like one at one at once, or HTML with uh all the files, much, much easier. So it's kind of steering. Another thing I see people not doing is like using it almost like instrument of vision, I find, because like what I do with it a lot is um so uh I was just at craft conference and Goiko Ajitsh, I I hope I didn't murder his name horribly just now. Um but he he has an amazing talk, and one of the points that he was making, it's much easier to complain than to explain what you what you want.
SPEAKER_00So like seeing something helps you understand what you want. And it's just much much easier. So like uh have it show you something, and then it's uh much easier to see what you're not like, you get more ideas, you you st you you can borrow different ideas, combine them, and you can go into a different direction entirely by just doing doing that. Um and uh a lot of it is like I think about it as like imagine I'm at a crossroad with many, many, many, many paths be before me. Right now they're invisible to me, but AI can make them visible to me at zero cost.
SPEAKER_00Yeah, like show me all alternatives that are in front of me. One thing that I really like is like um when you have a choice that you have to make, uh give me a choice, like choose yourself, show me with a little emoji, like I have a star emoji, this is a chosen option, very succinctly. And then choose choose me the paths are not taken.
SPEAKER_01Yeah, yeah.
SPEAKER_00I I I was doing it for a long time kind of manually, but then Arlovel showed me this trick that he's doing it on one move. So basically, this is like four patterns at least applied all together, and then then I can basically like, oh, this is the decisions before me. I want this one. No, no, yeah, you're actually wrong. This one is good because blah, and so on. So I can go into that direction now. And and I actually broadened my horizon what I don't know before me. Looked into that briefly. No, no, good.
SPEAKER_04So it's very similar to kind of like uh we we're doing a lot right now thinking about uh sorry to bring it back to Tesla Agent, but um in Tesla Agent we have this uh this change risk, and and the key is it will determine or it will try to determine whether a change is high enough risk to be to be raised to say, hey, this needs human review, or whether it can is good enough to do LM review. And I think it's it kind of like makes me think about you know, we use LLM as a judge for a lot of things, and it's it's it's almost as it builds, it might be nice to essentially say, hey, this is a this is an important decision that I think you need to make versus this is an important decision that I need you to know that I have made, and you just need to kind of like look through it, check it over, and just say, Yeah, I'm I'm fine with this.
SPEAKER_04It then avoids that almost needless at times, you know, back and forth, back and forth and that too much interaction. Yeah.
SPEAKER_00Yeah, exactly. So I I run into this a lot as well. So like a lot of my conversation have started to seem like the same thing, right? So would I have like a defaults, right? So like my default technology stack is this. Let's just do all of that every time, let's not just decide it every time.
SPEAKER_01Yeah, right.
SPEAKER_00And also like when you say AI review versus human, you like why why not both, right?
SPEAKER_01Yeah.
SPEAKER_00So absolutely get AI to spin all the stupid out and then do human and and make sure that the AI positions it in a way that's easy for human to follow because it's not easy right now.
SPEAKER_04So in that case, uh is there almost like an over do you feel an overload of review?
SPEAKER_00Yeah, absolutely. Or do you almost like feel like it's I don't like myself to go there, right? I I don't let it. So what I do is basically there's twofolds. Um I think it's important to be able to answer a question. To me, it's important. Does my system work? Right? And for that, uh, first of all, I have tests. I have tests on two levels. I have BDD tests, I have TDD tests, right? TDD tests are for Asian. This is a cross-check for across against reality, so this is not letting it go too far from reality. Uh and uh it also uh is helping in, first of all, it produces pretty good code.
SPEAKER_00Surprisingly, so like I would love to go there a little bit a little bit further about that. And then um it also um it also makes sure my tests are not completely horrible because like when you have AI write tests, a big danger, like I trust my AI with my tests even less than I trust it with my code. Right because I absolutely cheat online, horror do horrible things there and common tests, which is like a crime scene, like it should be punished by something bad.
SPEAKER_01Yeah.
SPEAKER_00Um so okay, I have this TDD stuff. This is for Asian mostly, and for code quality and so on, and check against reality. And then I have BDD tests about does my system work?
SPEAKER_01Right?
SPEAKER_00And I want them high level. I don't want them like I don't even want them on Cucumber level because Cucumber feels too detailed for me. So what I do a lot is like I write BDD level tests, I basically write it like a little whiteboardy domain language. So like how do I think about this problem? Make it extremely scannable so I can look at my tests, are they complete? Like, and I see things like depends on the domain, but like if I do API, I see here's my API, here's what happened, that that's what API returned, for example.
SPEAKER_00And it's all like a bunch of tests like this. Or if it's like more visual, like it's a game, so somebody made this move, this is what happened. So I want to see that, and I want to make sure that this is not something that Asian can edit very easily. But this internal stuff he can edit as much as it can as it can.
SPEAKER_04And your your BDD style tests, is that is that kind of does it use a similar Gherkin style approach to I use approval tests pretty much. Is it deterministic?
SPEAKER_00No, absolutely.
SPEAKER_04It says you can rely on even when you read it, yeah, you can rely that those tests aren't going to be online to me.
SPEAKER_00Absolutely. That's crucial, right?
SPEAKER_04Yeah.
SPEAKER_00That's crucial. Yeah, I I use approval tests for that, basically. This is uh like a good technique to know for kind of legacy code if you're trying to um pin the behavior. You basically can can be like, hey, what does this thing return when I do this and it outputs? And you say, Oh yeah, it looks good. This golden standard, so approval tests, you approve this. Um there's a little bit of risk to that. You need to be careful what you're approving. But basically, you can pin your whole system and then you can refactor inside. That's one of the ways to use this. And I use approval testing for this kind of BDD level test as well.
SPEAKER_00It's very easy to define the main languages there.
SPEAKER_04Yeah, amazing. So we've we've covered the low level, really nice practical tips there for for engineers to listen to and follow. Let's take it one step higher now. Let's talk a little bit about patterns and maybe kind of like a little bit more about your environment to how you actually how you actually build. Um so software factories. First of all, what's your what's your thoughts on software factories? Uh is it a destination for you? Do you believe it's a destination? And and I guess if you do, where are you on that journey right now to to becoming a software factory uh worker?
SPEAKER_00I'm not to be a worker, but to be like an architect.
SPEAKER_04No, software factory architect. When I said factory, I thought, oh, a factory workers, like a software factory worker. Yeah, the agents of the software factory workers.
SPEAKER_00Well, yeah, uh, I I don't know actually. So I think so. We're building the system of that builds other systems. Does it work like a software factory like we see today? I don't think so. Uh, but I think we're trying to figure it out. What I don't like in many approaches that I see right now is they're kind of a little bit of throwaway ATI. So basically, like let's create agents to create more Asians to create more Asians and so on, or let's create Asians, let's make agents write skills to write skills, and like all of that craziness. And what I see a lot is like it degrades so badly. Like the output is just complete garbage, and there's a lot of it, and there's a bunch of noise.
SPEAKER_00So I don't know how people are not um I'm not drowning in garbage in those. Maybe, maybe I'm just not like doing it right. I haven't tried to be fair. Yeah, I've tried a little bit of this approach, like I I tried like the Clot Flow, right? And it in the problem that I tried many other like so I have like a default problem that give agents like, hey, new model, try this, and I see the result. And it's nice because I have a constant that I compare things against. And the Clot Flow, in my experience, did the worst job of them all.
SPEAKER_04Ah, really?
SPEAKER_00Yes. Uh at the time, maybe I wasn't holding it right. But um, but so so I see the problem of like if you if you have the building blocks that you're relying on that are not actually doing the thing you think they should be doing, why why do you think the whole system works? How?
SPEAKER_04So there's a level of autonomy here, I guess, which is the key, whereby uh do you leave the factory to kind of like build and almost maintain itself? Or do you allow the factory to to kind of make suggestions uh whereby everything or the most important pieces need to be validated and verified by a human to say, yeah, this is actually a good move, or this is the direction I wanted to go in. So there's probably you know levels of self-building factory versus uh I guess you architecting the factory to build code.
SPEAKER_00I have no idea actually. So I I'm really curious about this. I I overheard somebody talking about uh their factory from Gas Town, I think. Um Gas City. Oh, Steve, yeah. Gas City. Yeah. No, no, no, there wasn't somebody from Gas City. And it was interesting. So I I absolutely jumped into the conversation and asked, like, uh, hey, tell me in practice how you're actually working, like how you how do you deal with this, how you deal with that. It was very interesting because actually a lot of things he said was kind of matching my experience with refactoring. So I have my refactoring process that I just point at things and and they go in a good direction for me.
SPEAKER_00And so a lot of that made sense. So I'm super curious about factories. What I'm trying to do myself, and I think I I will believe in factories that I build myself, well, while it actually, like with my standards. But um, what I'm trying to do is I'm trying to get to the building block that I can trust. So this is where my refactoring process comes in. Can I have a refactoring process? I can put uh point at a piece of code, several files perhaps, and it will just do it, I go away, it goes, does a good job, I come back to good code. Yeah. And I have that now.
SPEAKER_01Right.
SPEAKER_00Uh so that's nice. And I'm trying to kind of can I do this for extracting the knowledge? So right now, if I ask AI to save something into a file, it oftentimes does a very bad job at it, like too much noise or over like focuses on something that's not important in a broader context and all that. How do I get it to extract knowledge? So I'm playing a lot of that stuff. So kind of trying to build those building blocks, and then maybe I can construct them into something automated that I can actually trust. But I need such a level of verification here, and I want to automate as much stuff, needs to be code as possible here, I think.
SPEAKER_00So that's where I'm at. Yeah, I'm not there. I'm dissatisfied with my process. It works pretty well with what I'm doing, but I'm constantly dissatisfied and I'm looking for more. That's why I'm the commentary.
SPEAKER_04I think that's the key though, right? It's it's about knowing you're dissatisfied where you want to improve and that over time. It's a it's a gradual. I I think people who think they can just switch into a software factory uh they're living a dream and they're and they're they're um It's not a good dream, I think. Yeah, it's a nightmare, sorry. They're living a nightmare, but they're but they're they're dreaming about how fast they can they can grow. It's about and and I think in order to gain that trust over time as well, it's about gradual increments and improvements.
SPEAKER_00That's what that's why Dex Horty's talk yesterday was so interesting because he was like, hey, we tried this factory thing for half a year, it doesn't freaking work. Now we're going back to this. And I want to listen to those people. Yeah, right.
SPEAKER_04Yeah, the people who have tried, the people who have got experiences in the world.
SPEAKER_00Exactly.
SPEAKER_04Talking of factories, there's also have you had a a guy called Harry Kane?
SPEAKER_00Say it again. Harry Kane? Say the whole thing.
SPEAKER_04Harry Kane. Patrick Dubois in the background is laughing right now. But Harry Kane is a goal factory, and he's just put England 2-1 up against Congo DR.
SPEAKER_00So I think we'll like him very much.
SPEAKER_04We like him very much. He's a he's an amazing person. I can't believe I'm I'm so excited about that. I thought this was gonna be a five-nil walkover, but you know, everyone's everyone's a potential potential banana skin.
unknownOkay.
SPEAKER_04Let's uh let's let's talk about your tooling and workflows. So um terminals, IDs, how do you build it? No, no, no, no.
SPEAKER_00No, so so so like I started very on it very early on with Asians, and like basically I was a month late to the party. So now like November when this started, I started in December. And I was in Windsurf, and immediately I was unhappy because I was uh I'm in love with IntelliJ, or I used to be in love with IntelliJ. I I refacturing tools are unprecedented, like there's just nothing compares in the industry, right? And I can do crazy things with this. Um and uh then I have this Windsurf thing with Agent, where Agent lives, right?
SPEAKER_00So now I have to switch between this and this constantly. And I'm like, no, I just want to be in the project. Yeah, I just want to talk to why why am I this locked into this thing? So like as soon as I a squad code became like a real thing, which to me happened in May uh last year, I switched and never looked back. So console only, pretty much. I just barely look at files sometimes and I use IDE basically as a text editor.
SPEAKER_04D did do you file do you feel more disconnected from the code as a result, or do you still kind of look back at it? So you have to it's a trust, it's a trust thing as well, right? Because you step away.
SPEAKER_00It's trust because it's it depends on the level of risk, right? Yeah. So what are you dealing with? I'm right now not in the the the you know the gory and the bad. Uh I'm I'm actually very much greenfieldy. A lot of prototyping, a lot of other things. If I was in the brown like brown brown field, I would behave very differently, I think. How would I behave? I don't know. I think I would automate the hack out of that. I would make sure my test suit is something really, really good that is super fast, right? Um and this is where like the skills come in, like what I actually like my value is there, right?
SPEAKER_00Provide the like teach the agent to do the test suit that is just like this. And then uh kind of probably that then I can actually go a little bit higher level, maybe. But still I would be much closer to the code. Right now I don't need to do that, so I don't.
SPEAKER_04Awesome. Let's we're coming close to to wrapping up. We have uh we have about um I'd say five minutes of injury time to to to before the final whistle. So before the final whistle, let's let's wrap up by asking, I guess, when we talk about where you are experimenting next, yeah. What are the things that you know when you leave a conference you're gonna be super excited to to try to play with and explore?
SPEAKER_00I'm I'm I'm super interested in so many things, and there's only one me, unfortunately.
SPEAKER_04But yeah, I think you need to use more agents to explore and be your brain, see.
SPEAKER_00Yeah, probably. Yeah, so yeah, yeah, I'm definitely uh I'm the bottleneck, I'm feeling it like acutely. But yeah, um one of the things I really want to understand is how people doing loops and how am I doing, like how do I do loops better? Because I'm doing loops a lot, but like I'm I think I'm doing it differently than people are doing. So I'm very curious to just be exposed to a lot of like different ways of doing things and trying them out. Uh, another thing that's been on the radar radar for a long time is like this event sourcing and uh event modeling. So I I took a class from Martin Dilger a little while back, and I unfortunately didn't didn't uh didn't have time to actually like try it out, but it had a promise of something that was um his process is pretty great.
SPEAKER_00So like it um it feels like it solves a problem of keeping up in the complexity management all at once. So it's very interesting. There's some trade-offs on top of that, but basically what he's doing is he he's doing uh event modeling, which means he thinks through the process and he writes like uh this little like diagrams of what's happening, and then each one is a s like there's slices, so basically this piece of behavior does that, and then he generates the code for them. The slices are completely isolated, which means there's a lot of duplicated code, but it doesn't matter because it never overlaps, not ever, right?
SPEAKER_00And then what does it matter that you created like two objects a person if it doesn't overlap ever? Um I'm not sure how it works in practice. I'm a little bit worried about some things, plus like event events, so he does events uh sourcing, which is basically don't don't don't collapse your state, right? You have events for everything, and then you can unwrap and you can do so many things there. So then then you have views um basically for every everything that UI needs, which is kind of constructed uh aggregate aggregation views, and and also very many of those.
SPEAKER_00So this like very many of those, and which looks like call duplication worries me, but I want to try it and see because he seems to be very happy and I think he's he's solved basically from what he says, it feels like a lot of that solves a lot of problems, so I'm gonna try it out as well.
SPEAKER_04Sounds good and great, great things for our audience to uh to listen to as well and try try out. Love it. This has been wonderful. Uh thank you very much. I'm so glad we managed to do this in person. Thank you so much for sharing all your insights and and and your ways of working with our audience. Um been a pleasure chatting, very much appreciated.
SPEAKER_00Thanks so much. I'm so happy that England won.
SPEAKER_04I know, full time is almost uh upon us all. Uh so that's great. Thank you so much.
SPEAKER_00Fantastic, thank you. Wonderful.
SPEAKER_04I hope you enjoyed that conversation. Really lovely insights. And um tune in to the next episode soon. Bye for now. The AI Native Dev is brought to you by TESL, the package manager for skills and context. Your hosts are Guy Pajani and me, Simon Maple. Our producer is Tom Dowler. The AI Native Dev is not just a podcast, it's a community. And we host monthly meetups at the TESL offices in central London. Visit TESL.io forward slash community to learn more, and I hope to see you there.
Developers are about to write skills instead of code, and Microsoft's Daniel Meppiel argues the software factory that runs them needs versioning, a bill of materials and signed receipts before enterprises will trust it....
Paul Stack helped build Terraform. Now he says he'll never write another line of infrastructure as code, because an agent harness runs his team's production infrastructure instead. Watch all the AI DevCon NYC 2026...
Automated software delivery is moving faster than the people who used to approve every release. LaunchDarkly co-founder and CEO Edith Harbaugh tells Guy Podjarny why rollback should be autonomous, why one-shot coding...
Agentic coding has outgrown the text box. Maggie Appleton, staff research engineer at GitHub Next, argues that walls of CLI output and AGENTS.md files serve coding agents far better than the humans steering them. She...
AI attackers now move at machine speed, while most security teams still move at meeting speed. Former Robinhood CSO Caleb Sima has seen an AI attacker up close, and he argues the only way to keep up is to take humans...