Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow artwork

Every Repo Is a Software Factory Now | Don Syme, GitHub

Aug 18, 20261 hr 5 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Turning every repo into a software factory sounds like marketing until you see the machinery underneath. Don Syme, Principal Researcher at GitHub, breaks down continuous AI and GitHub Agentic Workflows, now in public preview. The short version: the agents are the easy part, the quality gates and guardrails are the work. Watch Don's AI DevCon Talk here: https://youtu.be/kbvqRWY-bUs What we cover: - What continuous AI is, and why it sits beside CI/CD rather than inside it - Why bounding the context is what stops automated agents going off the rails - Whether the repo is really the right unit for a software factory, and where monorepos break it - One workflow or an agent zoo? Cost control, scheduling and model exams - Quality gates, human review, and equipping the reviewer instead of drowning them Chapters: 00:00:00 - Introduction 00:03:52 - What is continuous AI? 00:07:33 - From single player to multiplayer automation 00:09:42 - Bounding the context so agents don't go off the rails 00:12:57 - Software factories, loops and harnesses 00:18:44 - Inside GitHub Agentic Workflows 00:25:19 - Why the repo became the unit of production 00:37:11 - What belongs in the repo, and what doesn't 00:45:07 - One workflow or an agent zoo? Cost control in the factory 00:52:01 - Quality gates, human review and equipping the reviewer 🌐 Tessl: https://tessl.io 🔔 Subscribe for weekly episodes on AI-native development If you're building a software factory of your own, tell us in the comments where your bottleneck actually sits.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_01I mean, chief or not, it's the nature of the factory. If it produces rubbish, you throw it away, right? I do understand, of course, that human review is a bottleneck. When you're creating a factory, you know, a lot of the work is on the quality gates. You know, there's just no point creating a pull request for the human to review when that thing is low quality. My job as a factory creator is to deliver high-quality pull requests where the reviewer is equipped. And I love that word equipped. My job is to equip the reviewer with all the information they need.

SPEAKER_00The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding. Join your host, Guy Fajani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today. This is the AI Native Dev. Yesterday, you were writing code. Today, you're prompting agents. Tomorrow, you'll be running a software factory. Agentic coding is moving fast enough to give anyone whitlash.

SPEAKER_00This June, AI DevCon London sold out. Hundreds of developers, engineering leads, and VPs packed in to hear speakers from Anthropic, OpenAI, Netflix, and GitHub cut through the hype and get to what actually works. This November, we're bringing it back to New York. Bigger, bolder and full of people actually building the agentic future. Three days of real talk, real tools, real answers, whether you're directing the agent, building the loops themselves, leading the teams, or setting the strategy.

SPEAKER_00Our early bird pricing ends on September 30th, so make sure you lock in your place before prices increase. As a podcast listener, you can get 15% off your ticket using promo code pod50. We have so much planned, and we can't wait to have you join us in New York.

SPEAKER_02Hello everyone, welcome back to the AI Native Dev. Today we're going to talk about continuous AI and talk about uh repos and you know what is the sort of the container for factories or whatever we call them on that front. Uh and to dig into all of that, we have with us Don Syme, uh, who is a long-term kind of industry contributor when it comes to programming languages and how we develop, big open source contributor, worked a lot on F sharp, on various other projects you would have known. Uh, and today a principal researcher at GitHub working on you know many things, AI development, but specifically also uh kind of continuous AI and agentic workflows, uh, which we'll cover more.

SPEAKER_02So, Don, thanks for coming onto the show. Uh, I should also point out you've given uh an amazing talk at our latest AI Native DevCon, and you're a big part here of the London community of uh of uh of development as a whole, but also specifically now as we uh transform into AI. So love having you as part of the community.

SPEAKER_01Guy, it's a pleasure to be here. And uh it's it's been a pleasure to get to know you over the last couple of years as you've uh created TESIL and what a wonderful community here in London and also worldwide AI native dev community. I've used your resources a lot, I've learned a lot from uh what you've done. And you know, really kudos to uh to kind of create that to take that community ethos that was so strong in the developer world and continue it through the really big shifts that have been happening in the developer world.

SPEAKER_01And yeah, I love what you're doing.

unknownCool.

SPEAKER_02I appreciate that. And uh, you know, it's all about gathering brilliant people and uh giving them a stage to share their learnings. So uh precisely what we're doing here. So uh we're gonna talk a lot about uh you know how do you sort of systemize, if you will, software development in the sort of new era. Uh and uh you use the term you kind of individually and also at GitHub use the term continuous AI when you think about uh that type of development. So maybe for starters, just define for us a little bit what is continuous AI.

SPEAKER_01Yeah, so the the thinking is that there's been a bit of a a blind spot in the uh in the AI industry, a really big focus on the kind of use of AI, basically coming from the chat modality, the individual productivity modality. Of course, we see that with Chat GPT uh just exploding in it in its initial uh and then all the kind of productivity enhancements that come around the chat modality. But we think there's kind of a a missing point of view, which is about uh collaboration and which is about automation and which is about situated automation.

SPEAKER_01And we were thinking about this a lot last year at GitHub, and we did some kind of thinking about thinking around about it, then we kind of looked, well, you know, we already do a lot of situated automation in what in CICD. We have continuous integration, we have continuous deployment. And then there are this whole range of activities that people want to do in this continuous modality. And the continuous modality is just so central to how we work in the in the software industry that change is happening in this kind of context, usually the repository, but there can be other kinds of contexts, the organization or multi-repo.

SPEAKER_01And that means continuous means you're kind of continually folding in that kind of sense of change and eventing and difference and sort of uh incrementality that's kind of happening and always re-establishing this crucial invariance like uh the software actually works, you know, in kind of continuous integration, or the software is correctly deployed uh in continuous deployment. And a lot of these things are very they're they're they're central uh to how we how software collaboration and deployment works.

SPEAKER_01But equally we want something new, which is also these subjective activities like continuous documentation. Now, writing documentation is a kind of is a is is a subjective activity. It's not the sort of thing we those CICD kind of concerns are about like determinism. We want everything to be exactly right and double check absolutely everything.

SPEAKER_02Repetition and sort of insistence that it always happens, like a lack of less creativity and more consistency.

SPEAKER_01Yeah, that's right. And but now we have this whole new bucket of things, like we want to continuously improve the performance of our software, we want to uh continuously uh fix bugs in our we want to continually triage bugs or have every issue labelled correctly within cut some kind of schema. We want to continuously look for duplicates in our in our in our bugs, and of course you can kind of do this as a these things each as a one-off, but in the end, you're gonna have to put them on some kind of operational permanent kind of basis in the context of change.

SPEAKER_01And it's so easy for people to kind of blank on that and kind of say, Oh, look, I can I can find the duplicate bugs in my repo. But you've got to make that continuous, right? You gotta you gotta get that on an operational, permanently running kind of basis for that to really be a sustained activity. So we looked at this kind of collection of things and we said, well, what what really characterizes this is they're different to CI, they're different to C D, but they're also similar. Okay, and it's that con that that and that said, well, let's partner that CI, CD, and continuous AI.

SPEAKER_01And the the the key things is continuous but also kind of subjective and implemented with kind of AI, and that's and that's where we land with that term. When we put that out there, and it kind of seemed to resonate with people, it kind of captured something that other people weren't talking about, and that that that's that's our conversation.

SPEAKER_02It's sort of uh so I like let me kind of echo back a little bit and the things that resonate with me. So it starts from the sort of the single player to multiplayer move on it, um, and then I guess naturally over there weaves into into automation, right? Because to one extent, like we talk about continuous, but I think what happens with AI is that it has this jagged edge, and you know, sometimes it works amazing and sometimes not. And we have this kind of uh hard to uh determine points in time in which we want a human to intervene and apply their judgment, which is really a blocker for scaling.

SPEAKER_02And in that sense, it's similar to uh continuous integration because before continuous integration was non-continuous integration, like you know, someone manually running it and so running the sequences.

SPEAKER_01There was a lot of homebrew uh continuous uh uh automation as well, sort of gauntlet systems or something like your your PR would run the gauntlet of your organization, and and that kind of all came together into things like GitHub Actions and other CI CD kind of systems. And uh we see plenty of homebrew for people kind of automating uh parts of AI kind of workflows as well. And it's just like, yeah, you know, we we've we should consolidate that into systematic ways of thinking about it and systematic ways of actually kind of doing it in practice as well.

SPEAKER_01Performing.

SPEAKER_02So so so it has these maybe like two different routes, uh one one going from single player to multiplayer and the collaboration element to it, the other is the automation, uh, and sort of do this repeatedly, maybe starts with sort of homebrew automation and uh and doing it and then consolidating those into the sort of infrastructure that is CICD um usage style, but extended and expanded into uh the type of actions that AI can now do are uh naturally less deterministic uh in uh in nature, and so that really leans into not just the automation but also the definition of how would you track this activity, what is a failed build, you know, versus a successful one.

SPEAKER_02How do you continue with that? Does that so all of those kind of bundle into that term of continuous AI?

SPEAKER_01Yeah, yeah, absolutely. And there's another key aspect here, which is about bounding the context. I think we're gonna kind of talk uh bounding the context of automation and bounding the possibilities of what happens in the automation. People, um, conversations about some kind of automated I often go AI go off the rails unless you're actually situated inside actually some bounded kind of context. But we're gonna do this in the repository with these particular outcomes, uh, like it can create a single pull request, it can create a single issue, uh, that whatever the the job of the automation is on each run, it gets these particular kinds of inputs and doesn't get anything else.

SPEAKER_01It has a firewall around it and so on. So to have these kind of bounded, easy to understand, simple, uh kind of simplified view of what are we, what automation are we actually doing here? And because people are already really familiar with CI CD automation, yeah, a lot of people use GitHub Actions or some other kind of CID systems, all sorts of people can successfully kind of use those. It's really a natural starting point for uh for saying, hey, let's automate, let's like let's do this together uh in the in the because we're all doing it already, and now let's kind of automate some AI-driven subjective uh kind of elements of this as well.

SPEAKER_01And so yeah, situating it there does make a lot of sense. It's also counterintuitive to a lot of people, I gotta say. A lot of people listening on this might say, CI CD, I don't want AI anywhere near that, right? That's like that's my determinism, and there's a reason for that, which is it the one of my friends kind of says, it's the place where the grown-ups are, right? You know, it's the place that really establishes that your system actually works as it is. And a lot of people have plenty of problems with their CI CD systems. It's not like it's it's a it it's it's so important uh that's a major focus for team activity, is just kind of get the invariants established there, get get the green ticks and get everything kind of checked there.

SPEAKER_01They use it for everything because it's so useful. And the thought of adding kind of more subjective activities into that, like uh labeling or issue triage or bug fixing, that that that can be disturbing for people. And it's one reason that we don't say we're putting AI into your CI. No, no, no, no, no, no, we don't do that at all. It's CICD, they stay as they are. And continuous AI is a separate bucket of things. You can do none of it if you like, you can do a lot of it if you like. You can dial that as you like, and that's really, really important that we're not marking with the core ethos of what CICD does and how crucial that is in the industry.

SPEAKER_01We want people to double down on that because that's where we really establish software equality in the industry.

SPEAKER_02I think so so I I relate to that. I I think uh like on the context of of context, you know, of like defining context and uh defining it. Um I I do think that there's a lot of analogies to the world of CI CD in that environments are actually a core element of CI CD as well, right? They had us move up from works on my machine to something that is defined. Continuous deployment is about the environment of the production. And so defining the context uh is just just this probably, I'm saying it in quotes because it might uh um it it's it might actually get fairly complex, uh uh just a new type of environment variable for it.

SPEAKER_02But I it it's a good uh it's a good uh tee-up though to a question I had, which is, you know, you use the term continuous AI. I feel like there's a lot of analogies in the continuous AI narrative to the world of sort of factories and factory lines. In my uh in my own kind of A native DevCon uh um talk, I talked about sort of the software stack, and in one of those paths, I was a little bit stuck with saying, okay, you have uh models on top of which you have tools, on top of which you have skills, on top of which you have harnesses, they all kind of compose down.

SPEAKER_02And then the harness, it was hard to know what's above a harness. Uh and you know, I wrote there a factory line. The term loop became popular like the next week or something, so maybe that's the term on it. Now you have this continuous AI. But I guess I guess do you feel those are just different words, or is like a system harness or a factory line or even a loop? A loop has a little bit of that optimization piece to it. Uh the the same as continuous AI, I guess how much how much is just a different lens to arrive at the same destination versus you're defining something different?

SPEAKER_01I think I some of those conversations like are orthogonal, because the shift that I'm most interested in is this one from the kind of individual productivity point of view to something which is more continuous and kind of team and kind of integrated into some particular kind of context. And so um a lot of the things that are kind of getting developed, like the notion of kind of loop engineering, you can apply that all in a kind of continuous AI setting. The ledger of record might be different because we typically use the repository and the issues pull requests as the actual kind of ledger of record of what happens and maybe some augmenting kind of things as part of that.

SPEAKER_01Uh whereas maybe if you're kind of loop engineering on your local dev machine or something like that, you might be using a different kind of ledger and uh uh so on. Uh but the concepts can actually flow very freely between the two, and I really like that. It's like it's you know, there's a lot of conceptual kind of uh competition in that kind of like what comes next kind of space in the individual productivity space. And I kind of like saying, hey, that's not what we're doing. We're we're over here, and yeah, you do all of that and bring it over, like bring over all the good ideas and and rethink them how they kind of work in the like uh have what what the loops kind of mean.

SPEAKER_01It might mean lots of steps of a loop, like you get to see you uh you set the uh continuous AI um uh setup. Like we have goal chasers, for example, continuous goal chasers, which once a day try and make progress towards some particular defined goal. And each day you get a pull request towards that goal, and you kind of integrate those, but it maybe it won't work on the next pull request until you've done the previous one. So it's kind of a it's not really a loop, is that a loop or is it not a loop? It's kind of a loop because it's kind of you know repeating those pull requests towards a goal, but it's it's a cooperative loop that's actually quite slow in its kind of action.

SPEAKER_01That might be what an enterprise exactly wants to kind of to kind of chip away at the kind of all the big backlog of accessibility bugs or something like that that uh that a product has. You know, that that those kind of loops are actually really satisfying for teams where they have like this cadence of like daily improvement towards some kind of goal uh that is measurable. You can see you're making progress, and managers can kind of understand what's kind of going on. Whereas, you know, if it's in the kind of loop engineering world where the individual developer is like, yeah, I fixed a thousand kind of accessibility bugs, and here's a thousand pull requests for the team to review.

SPEAKER_01Let's you all have to spend or drop everything you're doing and kind of review those, you know, that that's kind of a little bit like uh a little bit disruptive to how teams work, kind of things. So we kind of get to shape things a bit differently in uh in in when you when you slow things down a bit and kind of move in a different kind of pace.

SPEAKER_02I feel like I I relate to the the distinction between the single player and the multiplayer. I think that is very important. Uh there are still sort of two things here. One is the the the pipeline, uh so the continuous AI element of the pipeline. This is a uh uh an aspiration to get to a point in which you can get a a recurring or like a repeatable motion of getting a type of input to come into the pipeline and process it and get a certain caliber output out of it in a repeatable fashion from the maybe it's even two other aspects.

SPEAKER_02What I would what I would say is a loop, which is more look at how that was done again and again uh for it and improve it. So so a self-improving pipeline. So because you can apply a loop to CI CD. I mean, in fact, we do it, you know, and and we do it for ourselves, we do it with the Tesla agent for others, which is uh fix flaky tests. So flaky tests are an example of uh of something that isn't necessarily a pipeline in its own right, maybe it is, but it is about identifying failures in the CI CD and fixing them. Uh but I think that's the loop.

SPEAKER_02And then I think uh debatable a little bit on the term factory, uh, where there is a proactive initiation of uh of a of a th of a of something that is like an event-driven, uh, you know, a bug was reported. That's probably not its own term. It's probably like either we're talking about two things, the pipeline and the loop is actually the initiator. So do you do you agree with that sort of separation? We have the pipeline and the uh uh and and maybe the loop and the initiation of it.

SPEAKER_02Uh and and and how does that relate to the GitHub agentic workflows, like the sort of the core capability that is from GitHub? I mean, what is the scope of that product? Is it both of them? Is it more? Is it others? Maybe tell us a little bit about the the kind of the product side, you know?

SPEAKER_01Yeah, yeah, because we've talked kind of conceptually about kind of continuous AI. And I'm very excited to say that GitHub agentic workflows, which is our uh implementation of our kind of continuous AI system for uh for GitHub Actions, uh, is now in public preview. And um, you know, it's gonna be part of the it is part of the GitHub product lineup. And so, you know, get up there and kind of use that. So, what is GitHub Agentic Workflows? Uh we think of it as sort of intelligent automation for GitHub at the at the high level. Uh and uh that concretely means you're running the coding agents you know and love, the ones you're very familiar with, you know, your Claude codes or your um copilot CLIs or your Gemini CLIs and the like.

SPEAKER_01And you're running those uh with strong guardrails in GitHub actions uh on uh so that's on an automated kind of basis. And all of those elements are kind of crucial to us. First of all, it is intelligent automation, continuous AI, like we've kind of been talking about. You get a choice of coding agent. We're not tying you into kind of co-pilot uh, although absolutely that's kind of the default one. It's a wonderful, wonderful coding agent. And uh you get the choices of models and you bring your own key and lots of kind of things about that.

SPEAKER_01It's all kind of open source as well. You can kind of look at it, look at it, look at it all, how it works. And this with strong guardrails is absolutely key. And I'm kind of we're gonna be talking more about security and guardrails later on, I think. But you know, the the better the guardrails are, the faster you can run with automation. It's like train tracks, you know. The the stronger the train tracks, the the faster you can run. This is so crucial, and so many people are kind of missing this in that uh in the automated modality, in the continuous AI modality, you've got to be able to sleep at night.

SPEAKER_01You've got to be able to have those things running when you're not there and responding to things and trusting those kind of things and have making sure the inf that uh making sure all the kind of integrity and information flows are kind of right and reviewable uh to check you're not kind of leaking information.

SPEAKER_02I had uh Ori from Sara on the podcast here, and he pointed out says like if you if you allow the agent to do something and you run it enough times, it it's almost inevitable that it will eventually do that thing. So you have to do that. That's right.

SPEAKER_01You need to you gotta set the guardrails, you've gotta set the constraints really strongly. And uh information flow integrity, uh and you you y you you do need to think about what you're doing. You you you shouldn't make your private repositories kind of accessible uh to agents running in public repositories. That's you know, that's just asking for trouble. You've got to know your basics of agentic security, uh an agentic uh uh kind of information flow that people bringing in issues. If you're gonna triage those issues, you've got to think twice about triaging issues from from third parties.

SPEAKER_01And uh and we do that mostly through forcing you to opt-in uh when you're due certain actions, like the default is to kind of ignore issues coming from untrusted contributors, for example. You've got to opt in if you write an agentic workflow that's gonna process that kind of information. Uh so that's what GitHub agentic workflows are intelligent automation for GitHub, uh running the coding agents you know and love uh with strong guardrails in GitHub actions. Uh and it's a starting point. We're gonna see a lot of different implementations of coding. Continuous AI, we're going to see we have a thing called GitHub automations as well, from uh which are more for individual automate automations for individual productivity.

SPEAKER_01There's gonna be a lot of different kinds of automated uh kind of systems. There's a whole spectrum of kind of things. Uh you know, this is how we're cracking open the space to say, hey, you know, that DevOps mentality of CICD, bring it along to the party.

SPEAKER_02And it sounds like uh uh GitHub genetic workflows are, in my analogy before, maybe more the pipeline than the loop. So they are the means of interacting it. They are an event-driven system, you know, an issue comes along or something, you might sort of trigger that. Uh, but they uh they are not, they don't, it doesn't implicitly drive you to create the loop or not. It's sort of your choice. Now you can create another time-driven uh schedule that will now do this on a daily basis. And so you can do that with a programmable sort of uh system uh on it, but it is about an agentic pipeline.

SPEAKER_01Yeah, and you kind of uh think of it as one step of the loop as well, like uh to taking the issue and creating the pull request. Uh now taking the goal and doing one step of that, uh taking the backlog and doing a small part of that and and implementing you know one, five, ten kind of fixes in one, five, ten PRs. It doesn't have to be one issue to one pull request. A single run might create multiple pull requests, it might process multiple issues, it might do other kinds of work, it might work off uh backlog as a source of input. Uh it um it might also be a planning step, it might sort of decompose that kind of um, it might be a research step that's actually very common where it says, hey, here every day we're gonna do some research and how we can improve performance in this repository.

SPEAKER_01We're not gonna do the work, we're gonna do a lot of investigation of that. We're gonna write a report on like what are your top five opportunities for improving performance? And then the the uh human, the the maintainer or the contributors can say, let's break that down and actually take that to the next step and actually go and implement those, uh, each of those kind of uh opportunities. They're actually just do numbers one and three. So there's kind of like it's a it allows you to create a lot of different interesting kind of uh back and forth assistive kind of flows.

SPEAKER_01Uh fairly they're they're not they're not super fast uh kind of flows. It's like CICD that you know, because you're you really are doing depth research, it that may involve building the product, right? That may involve uh taking screenshots and doing research on the product. And and that's where these things are can be very different to sort of other kinds of automation you see floating around the industry because we do have the expectation that a lot of the a lot of the time these steps can do real software engineering. They can build the product, they can run, they can take, they they can they can be working off production telemetry data, for example.

SPEAKER_01They can be kind of getting the latest like 10 failures off your site and and then say, let's prioritize those and write your research report on how's your site reliability going, you know. Uh surfacing all this kind of subjective information into useful, synth uh useful reports and actionable kind of work to take things forward.

SPEAKER_02So what so uh that makes a lot of sense to me, and agents are amazingly powerful and you can run them, and and cloud agents have also become you know much more sort of useful as we sort of run those. Right. So this is sort of situated cloud agents. So so that's uh that's kind of where I wanted, so I wanted to maybe shift and talk a little bit about the sort of the repo as the unit. So as I think about GitHub Agentic workflows, which granted is sort of a fairly new product on it, so I'm sure it'll expand uh many things. But one of the things that I I feel if I was to explain why is it different, how is it different than any other cloud agent, uh, you know, it it feels like a a lot of the core of it is just that sort of repo centricity.

SPEAKER_02Uh and uh you know, one of the one of the sentences kind of in the uh in the uh in the web pages there talks about turning every repo into a software factory. Um and so i uh I guess it feels to me like GitHub agetic workflows are effectively uh cloud agents that are kind of repo native. So they're they get defined in the repo, they execute with an expectation that all of their activity and all of that, which other agents can potentially also do, uh, but uh but it's just like a core element to it.

SPEAKER_02Is that is that right? Is there something else that's sort of fundamentally different?

SPEAKER_01That's right, and that's absolutely uh a key core design decision about where we started with the CI C D kind of analogies, which are also highly repocentric kind of uh flows and automations. Uh and um now and there there's a reason why in that it there's just so much becomes simpler when you answer that question of what is the bound of operation, where is this running, where is the definition, let's pair that kind of definition.

SPEAKER_01There are pros and cons to that in this, and in and in the DevOps community, of course, that went back and forth for kind of many years. Do the definitions of the automation sit sit with the in in the repo or or or not? Right. Yeah, and there's a whole world of GitHub and such things. And on the on the terminology like the repo as a as a factory, I actually kind of prefer to kind of think of repo as a site of production, a site where things happen, a site where value is kind of produced. And that might turn out to be a factory, but we produce value in all sorts of other kinds of ways.

SPEAKER_01You know, a film studio is not a is not a factory, but it's also a place where value is hopefully kind of produced uh as well. So they're all sort uh and the anal the factory analogy is kind of most useful when there's kind of repeating tasks that are the same, that are on the pipelines kind of coming through. But I think we should think quite broadly about uh since AI has such incredible breadth of use, we should look for other analogies and other kinds of inspiration that uh use a factory sometimes, that's great, but also use you know, you know, people you do use the words like studios or productions or workbench and all sorts of other kinds of things that uh that that kind of make most sense.

SPEAKER_01So um there's another crucial dynamic that uh um in the context of the repo and the context of CI CD, and I want to kind of bring this out, which is that when an organization adopts CI CD and gives adopts GitHub or some other similar platform, what they're really doing is saying to the developers in that organization, here's a place called the repo, where we as the the organization give you the permission to automate within that bound.

SPEAKER_01Okay, you the developers know best what automation you need for your repositories. You know best uh what CICD you need. Therefore, we're going to give you the permission to and you can create these things called workflows which claim some of those capabilities, they claim the ability to create issues.

SPEAKER_02It's like a security boundary and uh and a decision authority boundary.

SPEAKER_01Yeah, and but we give it to you. You go for it. Do what you need to do. Okay? Okay, maybe we have some organizational guidelines, maybe they're reusable actions and so on. We kind of tame that a little bit to make sure we're kind of keeping a platform centricity in our organization. But this is an incredibly powerful decision because suddenly uh developers have access to a limited uh form of cloud resource. They have access to compute, they have access to network, and they have access to limited kinds of storage for kind of caching and whatever else goes on. And that allows developers to be incredibly productive.

SPEAKER_01And this is why CICD is so successful, okay, because there is a boundary context where they can developers are empowered to claim resources. They do not have to go and ask the cloud team in the company for a new machine, they do not have to go ask for a budget. The budget is somehow it's all ultimately measured, but uh, this is absolutely crucial organizational dynamic for why CICD is so powerful and so successful. And we believe that continuous AI or that AI automation uh in bounded in contexts will go the same way.

SPEAKER_01There will be a cost center where those this stuff does cost money, of course, like CI C D there will be a cost center where that kind of flows up to. There can be accounting controls, there's visibility of where this goes, there's an authorization context that within that context we let you use AI according to we still have guidelines and and and and policies and so on, but that allows incredibly creative aspects. And I think this really matters to me in terms of the kind of the jobs discussion in the industry, because there's evidence that uh if you empower people close to the coalface in a sense to use AI, not the of course, not an actual coalface, yeah, metaphorical coalface.

SPEAKER_01Yeah, because AI doesn't have non-real coal faces, I get that. Uh but uh then they find amazing productive uses for that, uh for for the for these technologies. And so I'm I love uh GitHub workflows because it is an empowering technology that allows the the maintainers of the repos and the people close up close to the software to work out how best to use AI. And you can have incredible conversations about this with people as you start to unwind, you know, talking to someone working on children software for children, uh, and they say, look, the thing I struggle with is is is that my the the language used is is getting by by you know is getting written for 15-year-olds, but I need it I need it uh suitable for for 10 year olds.

SPEAKER_01And of course the AI can make those kind of judgments, right? The AI can read all your documentation strings and all your text and kind of simplify the language down and just automate that and do that continuously day in, day out. It's that's like incredibly productive.

SPEAKER_02I think uh so I I agree, but I want to challenge it on sort of the two sides of it. So I think what we're describing here is we're saying uh a repo is uh a highly convenient unit. You know, it is uh it is a a uh a space, you know, a production site, you know, in which you can say I consolidate a lot of my data on the repo. Uh and because of that, I can define my automation over here, my code itself sits over there, or my kind of other assets, potentially context can sort of sit over there uh and such. And I think there's a lot of merit to it.

SPEAKER_02I think also the rise of GitHub Actions, which you know functionally have a lot in common with other sort of CI systems, which you know are successful in their own right, but the sort of the rise of GitHub Actions is probably because of that sort of convenience of sort of being being built into the repo. So a lot of value there. However, uh there are sort of two other counterfacts to it. One is uh the actions that happen cross-repos. So we we talk about CI CD as though it's one thing, but actually C D oftentimes is not just sort of one repo. Uh and so uh sometimes those things become a little bit tricky and they become another repo.

SPEAKER_02So suddenly you have those uh those paths. And maybe even more uh that one is maybe more of a side comment, uh, and we can get back to it after when we talk about factories as a whole. The other one though is like in the world of AI, there's this there's been kind of this uh resurgence of the monorepo. It never went away, but I think it's it's fairly clear now that a lot of the agents, a lot of the models have been trained to work in the context of a single repo, but oftentimes they need information that comes from these other repos.

SPEAKER_02And so the easiest solution to that uh is to consolidate everything into the uh single repo. But that starts breaking this kind of clarity of it's in the same spot, because the monorepos can be beastly. I mean, they can be enormous uh and they're no longer one thing. So using a repo now as a unit of access, a unit of security, a unit of uh uh empowerment of who's allowed to do what starts really kind of uh fraying at the edges, especially in what we're seeing.

SPEAKER_02Like we're doing it here at TESL, which is uh field teams have access to sort of uh repos and do that because again, it's so convenient in terms of uh agent access to pull them in, but you're kind of losing a lot of that identity of a repo as a project.

SPEAKER_01Yeah, I think there are you've hit the nail on the head, there are absolutely real tensions here. There's kind of a sweet spot somewhere in the middle where I like for some kinds of projects where just everything works really, really well in an individual repo, and we we kind of recognize that when we walk up to a repo that's just the right size and it's perfectly set up for deployment, and everything is working really, really smoothly. And then there are these other tensions that are pulling things in various directions. I think at at GitHub we have work to do on allowing making sure that the repo is not the only notion of context that actually kind of works effectively, that you could have multi- uh a collection of repos as a kind of collaborative context, like friend repos or whatever kind of features you might kind of add to that.

SPEAKER_01And uh I I would I you know that's one of the things that get discussed at at GitHub for sure. Uh and and then there's the the the question of the organization as a kind of context as a whole, so you could but of course organizations can be absolutely enormous, uh, and thousands and thousands of repos and um it's a huge question to kind of scale there, and even then some you have enterprises which have multiple organizations and so on, which which which adds a new kind of uh question, and and there are auth kind of questions um in all of this kind of work.

SPEAKER_01Um and you're right, the coalescing effect into the monorefo that was a trend that uh was before AI. It is uh that it's it we see it's in some projects I'm working on. I'm saying, hey, look, that we we should really consolidate these AI, it's gonna work a lot better if we if we bring it together. Bring it together, like it's just no point having these separate. Um so yeah, the tensions you say are absolutely real, uh, and you always see dynamics where people pull things apart and then they put them back together and they pull things apart. And I um I don't have a lot to kind of I I I think there's absolutely work we can do at GitHub to to tackle to give more effective solutions to stop you being sort of forced to bring everything into one repo, at least give you more kind of choice and control about where that's where how that works.

SPEAKER_01And um there would also be the question of like auth within kind of a monorepo. I'm not really an expert on that kind of thing. I'm yeah, but I get I get that that's controls.

SPEAKER_00Hey everyone, hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development. Whether that's talking about the latest tools, the most efficient workflows, or defining best practices. But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favour and hit that subscribe button. It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.

SPEAKER_00Alright, back to the episode.

SPEAKER_02So I guess maybe concretely when we talk about agentic workflows, what type of automations would you say are uh sort of suitable to run within the context of a repo versus which ones are outside? I mean you gave, for instance, the example of a repo pulling in production data example and running in it. Clearly the production data is not naturally in the repo, you know, for for I think many good reasons. Um and that system is elsewhere. Like, no, that's so so that's one example. Is like should that really be a repo level automation versus something that's outside?

SPEAKER_02Like, how would you how would you suggest people start thinking about what what automation should be done in repo versus which automations uh I guess maybe you technically can do them in the repo, but the there's there's no uh even shouldn't.

SPEAKER_01I I mean we we we have on the GitHub agent workflows site some really interesting design patterns in this kind of space. And uh it's one of the things I think the team are doing really well. It's just kind of exploring the design patterns. Um so there are absolutely cases where we kind of uh run a whole zoo of agents in a single repo, and we've written that up as uh Pelly's Agent Factory, Pelly's Agent Zoo. Uh and like we've got we've documented like 20, 30, 40 agents there that are just really, really interesting kind of repeated things that you want kind of happening in repositories, some really very unexpected ones.

SPEAKER_01A lot of them are about code improvement, uh, a lot of them are about like um performance. There are ones about styling, uh, for example, things that are maybe a bit really subjective and are really bit uh unexpected, like uh um GitHub genetic workflows uh includes a uh uh a GitHub CLI extension, G H A W, and it has terminal output, and we want that well styled. And it kind of so the workflow runs in the in the repositor in the repository in this case and kind of runs the with the output and checks it runs the tool and checks its outputs and say, hey, you could kind of make this look better, uh, organize it better.

SPEAKER_01And uh and and that's actually really, really nice. And there's some a lot of them have got like 70, 80 percent kind of PR merge rates. You can start to kind of track the causality of like of the repeating kind of action and get kind of statistics on it.

SPEAKER_02Can I pause on that a sec on it? So so I love those, and you know, there are many, and I highly recommend people go check out their very, very useful automations on it. But that's a good example, like the styling one, which is you know, the definition of that style, um, I mean, it it could very well be something that you want to apply to multiple repos, is it not?

SPEAKER_01Oh yeah, absolutely.

SPEAKER_02So we would kind of pattern to think about for like where would you then store is that just a reusable action or reusable reusable skills.

SPEAKER_01I mean, absolutely. So you import skill libraries then. Right. So of course that's again where we a lot of the things that are kind of emerging in the individual productivity kind of world, like whole libraries of skills, and a Tesla has kind of done work on these and um yeah, bring them in to your agentic workflows and you can do that directly and just reference the markdown files and and and and or or check the skills into the repo, is is another thing if you want to go down.

SPEAKER_02So it's an interesting boundary there of so the execution might be defined in it and the skill that you sort of pull in. But if you want to create, for instance, shared access to system or such, you might do that through skills. The skills might sort of span repos.

SPEAKER_01Yeah.

SPEAKER_02And then there are uh I guess MCPs kind of serve as a similar type of uh uh shared access. So if you want to access that sort of gets managed in different ways. That's right. So I guess all of those the tools, the context, and all those are so to bring in production telemetry. Uh they're all a part of the continuous AI narrative. Uh they're just not uh because the continuous AI is broader than the agentic workflows.

SPEAKER_01Uh yeah, yeah, that's right. So the uh GitHub agentic workflows can bring in um uh MCPs, of course, and a lot of those kind of things uh flow flow flow flow across. Um and you were uh asking like when to run things in repo and when to run things out. I mean, GitHub isn't a very open platform. You can obviously kind of run automations outside thing outside as well. A lot of tools integrate with GitHub, and that's absolutely great.

SPEAKER_01I mean, I I think our uh point of view is to give you know there there are there are for instance um data integrity retention kind of policy kind of reasons why you might sort of convenient because you've got to copy the repo outside the GitHub platform.

SPEAKER_02You've already dealt with this, you've already approved it within your org.

SPEAKER_01Yeah, there may be legal or whatever reasons, uh privacy reasons, whatever, you can and can't kind of take all that issue and pull request metadata and kind of take it out and then put it back in. And uh so it's it's it's obviously a kind of we want a credible, set uh powerful default kind of option to have in GitHub for these kind of workloads. Uh, we do, of course, recognize you know, lots of kind of depth specialty security agents or penetration testing or all sorts of other kind of things that people do that that will have an out logically happen outside.

SPEAKER_01Yeah, absolutely. We do see some people kind of doing a mix where they their actual focus uh uh an Israeli startup called HUD is one. Uh uh Mai was talking at AI native DevCon, where their focus is on producing production information and they kind of need a way to kind of surface reports about the what they're finding about bugs and problems and performance results in through production probes. And for them, GitHub genetic workflows is just an a really easy way to do that kind of reporting, a kind of regular reporting into the repository kind of context.

SPEAKER_01And so, in that kind of sense, the the main intelligence, the main yeah, the the the real kind of systems value add is actually over in the production probing that kind of gets done, and maybe some AI processing that happens there. And GitHub genetic workflow is just kind of just a secondary role where it's kind of about surface using some intelligence to kind of surface information that's there. I think we'll see a a lot of that kind of thing. Right. Using it as a kind of uh auxiliary utility in other kind of flows to actually get that information regularly delivered into the repository.

SPEAKER_02Yeah, it sounds like there's sort of three tiers over there that you've got information that specifically relates to the the code or other content that you know your repo holds. So those are the sweet spots, like fine, you automate it, you kind of you bring the compute to the data, if you will, yeah uh where it is. You have another sort of pair of things in which a repo might be a convenient way for you to run recurrence on it. And so the repo is is just sort of uh you yeah, and you see that pattern often, which is you create another repo, like the whole world of GitOps. And things like that are worlds in which the repo is basically just it's purely a product, it's an automation kind of holder on it.

SPEAKER_01Yeah, that's kind of side repo pattern where you have an automation running in the side repo and the target repo is actually some kind of monitoring or something like that. And in that case, you use that much.

SPEAKER_02And you get the benefit of uh the kind of enterprise uh whatever legals and security and such approvals. If you're already a GitHub user, it's kind of convenient. Then you might have a third bucket, uh, which are cases in which uh you need other sort of optimizations and specializations, whatever bursting to a lot of compute or things like that, in which you might want sort of slightly different systems.

SPEAKER_01You might be applying some proprietary AI knowledge tools. To do the into your repository, and that's kind of going to live off to the side. The side repo one is actually really useful for controlling the um uh authorization about about changing the automation as well, like who can write the automation, who can run that, who's the factory runner in a sense. And uh so to have that side repo targeting a kind of control, uh a monorepo is um a pretty pretty effective pattern.

SPEAKER_02So should people uh I want to talk a little bit about doing it right and maybe get into philosophy, but maybe one more tactical question. Um when people do the automation, there are sort of these uh two again kind of push almost like monorepo, sort of small repo style debate, which is should you have one workflow that does multiple things uh in it? Um, because there's still a bunch of things you need to configure pro workflow and all that. There's a bunch of convenience to it, to it. Or should you have uh I think I guess that's how I interpret the agent zoo to be, which is you know, I've got a hundred uh different agentic workflows that are sort of running on it.

SPEAKER_02I guess what's your what's your sort of bias over there? What's your guidance?

SPEAKER_01My my particular bias is more towards one uh workflow, which kind of can do multiple kinds of things, one point of cost control. Uh like if you have a um so so so one of sorry, I should say my collaborator's bias, uh Pelly de Hallio and is um he is more to have lots and an agent zoo. That's why we call it Pelly's agent zoo. And part of that's just exploring space, like regular, you know, we don't, Pelly's amazing guy, and we don't want to kind of constrain what he does, and you know, we just like just go for it, just create as many workflows and and let's let's see what we let's see what comes out.

SPEAKER_01Let's learn, learn, learn. That's that part of what we've been doing here, and uh um but when it comes to actually putting automation, say, into I'm a repository maintainer for open source uh technologies. Uh and I I don't, you know, you can create learn agent zoo, but that's that's also a maintenance burden, right? You know, that's you've got to maintain that zoo, you've got to attend to that. And my attention like span is limited, right? I don't want this to be sucking all my my my time.

SPEAKER_01So I prefer to have kind of one point of attention, one point of uh kind of um and I also I maintain multiple repositories. So I'm gonna have to look at that across multiple repositories. Actually, I've made myself a little IDE to kind of help with that, to kind of attend to multiple automations running across so not across multiple, but in multiple automations in different repositories. And um so I I so one example is a a workflow I've put together called Repo Assist, which is kind of a sort of broad spectrum kind of repository assistant.

SPEAKER_01It like treat it labels issues for you, it it does first response to issues, so it sort of researches issues and responds to them. It it can choose to implement a PR to kind of fix those issues, it won't merge the PR. So my regular daily working with repo assist is to kind of look, well, what PRs did it create last night? You know, let's take a look at that.

SPEAKER_02But why why do you why do you prefer that, for instance? Like for instance, you you named there like labeling, researching, and uh and opening a PR. You could have just as well had three different workflows.

SPEAKER_01Yeah, so this is in sort of a supervisor orchestrator pattern, and you can have workflows which you know uh uh effectively choose to uh delegate up to more powerful models is uh interesting kind of pattern, uh or weaker models for kind of things that's important for cost control. Uh so why do I prefer it? I think just conceptual simplicity, just um uh I I I just it makes my life simpler and better than I think. Think about the one entity that you uh just just just to have that.

SPEAKER_01I I cost control really does matter in this kind of thing, complex overall complexity. I just find having five workflows considerably more complex than kind of having one. Uh how does how does cost manifest here? Oh, so these workflows uh are very often done in a scheduled kind of way. So uh the you may not you with issue triage, you have a choice. You um triage each issue as it kind of comes in and and do depth research, or you can do it in batch, or you can say each day we're gonna see what new issues have come in and we're going to kind of catch up in a batch and actually maybe do kind of a batch of them in a in a single run, uh, which can actually kind of give interesting kind of results as well as it kind of collects information across.

SPEAKER_01Uh there there are things in between that uh uh you could imagine having workflows which kind of have a service uh policy, which says, you know, we're going to triage all issues within half an hour uh and uh and do depth research on on them within half an hour. Um that's kind of our policy, so then that sort of dictates when uh how things kind of match. Yep. Uh so now if you is repo assist is a scheduled workflow, so you choose the cadence that it kind of is triggered at and you hardwire that into the workflow and you have to go and modify the workflow to change that as things currently stand.

SPEAKER_01It's uh and that's important because you dial up, you can dial the automation up, you can dial it back down. If your uh um if activity is very high in the repo, you kind of tend to dial it up, and if if nothing is happening, well probably you dial it down. The software is being maintained and there aren't very many issues that remain to be fixed. Each of those cases I can roughly estimate how much it's going to cost. Like we have an hour, you can see in the histories the average cost per kind of run.

SPEAKER_01Uh it might be in the cents, it might be in the a dollar or something, depending on the models you're using. And so, you know, you've got a bound, a pretty good bound there. And there are, I should add, there are actually very strong cost control features in GitHub agentic workflows. I think it's absolutely essential. It's the thing that's missing from the harness discussion. I think harnesses should have cost controls and kind of budgeting as an absolutely crucial kind of thing. And again, it's the kind of thing that becomes natural in the automation world, but it's a little bit uh unnatural in individual productivity.

SPEAKER_02Yeah, it's also hard to know uh when you do the action again and again, then you can kind of introduce uh an assessment, an eval of whether you can run into a kind of metric.

SPEAKER_01You can absolutely you can uh we we've we've added features that it will do exactly the same work at exactly the same time with multiple models. It'll only choose the it will choose, say, the first one as its result, but you can use turn that on for a day to get like uh a sort of examination testing, uh is what I call it, where you kind of exact you're doing the models, uh a room full of models is doing exactly the same work at exactly the same time, setting them an exam. And that really matters when you're using a shared ledger, because if you write anything to that ledger that hints at the answer to that, like a closed issue or something like that, those models will go find it.

SPEAKER_01It's like, oh look, I saw another model kind of did it using this closed pull request or something like that. No, you've got to test those models at exactly the same time. So we've got a feature for exactly that, uh, which you can kind of turn on for a day, and then you kind of get results about like, hey, which models kind of work work better. Work better on the in the exams, yeah.

SPEAKER_02So um maybe let me Vera us. We started talking about costs, which I I would sort of qualify a little bit as like doing it right. So you're sort of running it. And maybe I want to touch on on two other topics. You know, one is maybe more around autonomy uh and you know, sort of the risk of regressions, and you know, as you as you rely more and more on these sort of continuous AIs uh and improvement. And then the second is is uh is is safety. Safety and and autonomy are related a little bit. It's like what is it that it's allowed to do? So I guess I'll ask the sort of the broader question is as I as I lean in, this is sort of like beyond product, right?

SPEAKER_02But as as one leans into continuous AI and makes changes to the docs, to uh you know production deployments, fixes, um how uh uh how do you know, especially when you think about loops and you think about systems that improve themselves over time, right? Gets better at better on it. How do you uh how do you sort of see or what do you see as a best practice for avoiding regressions here? Do you do you expect every one of these agentic workflows to generally default to want to have an eval?

SPEAKER_02If not, like when should they sort of have evals to sort of know that they're they're not changing? You know, when do you even define what's correct and how do you even define beyond vibes what correct behavior is?

SPEAKER_01Just to clarify, we're talking correctness for the software or correctness for the agentic kind of automation?

SPEAKER_02So let me kind of refer. So we're talking about automation. We talked a lot about how do you sort of systemize the uh the automation of an AI workflow. Uh but what we didn't talk about that much is around uh the fact that those workflows are non-deterministic. And so you run them enough times uh and they might make mistakes, they might kind of cause problems. Uh and so I guess what I'm asking around regressions is what when we think about CIs, we have some indications. We know that you know tests will sort of run.

SPEAKER_02The tests eventually could become kind of useless, but if they fail, we see it, we sort of start optimizing them. And again, it's also like humans have created them. I think in the world of AI, you give much more sort of vague instruction that there's a lot, many more degrees of freedom, and especially if you have a self-optimizing workflow that modifies itself and and improves. Uh how do you how do you know or like what should you have in place to keep degrading?

SPEAKER_01To be uh to be clear that in this world the human is always in the loop. Okay, the at the at the point of the pull request in particular, or the issue, so an issue gets created or a pull request gets created and uh no pull request gets merged without the human actually approving that. And that um will um and it's also the case that there are limitations on what pull requests can sort of self-modify the the automation, okay.

SPEAKER_01That GitHub Actions has some built-in kind of limitations about uh actions cre creating pull requests which themselves modify the actions. And th those limitations are in there for good reasons for um you don't want sort of malicious actors or somehow kind of marking with your kind of CICD kind of pipelines. And some of that thinking does carry over and we so you can have self-improvement kind of loops, you can have like then they can write new skills or they can update new skills and so on, which are kind of separate to that kind of system.

SPEAKER_01But I do want to kind of say that there are some important kind of controls about how much of that thinking we kind of bring into this world, and some of that is inherited from GitHub Actions.

SPEAKER_02But if you uh I just want to dive a little into like no pull request gets merged automatically, but that's not a sort of that's an untenable sort of destination, right? Like as you sort of produce and you make more changes, can humans really review? Like, is there a is there a path for scaling this that doesn't require pull requests to be auto-merged?

SPEAKER_01So uh that is in some ways a separate uh discussion. I mean in the sense that for Git hypergenic workflows, we have a point of view on that, which is that the human is in the loop. Okay, and the human is loop uh de in the loop, definitely at the point of the of the modification of the ledger. Oh not not the full ledger, the some of the ledger can be modified, the the issues can be written, but the actual code, and it's always been that distinction. The code is kind of a kind of the golden truth of uh of of what's going on.

SPEAKER_01Uh and we I do understand, of course, that human review is a bottleneck. Uh so of course you can have a genic workflows and other kinds of AI uh genic automation and all sorts of other algorithm algorithmic automation which uh do review, review, review. And when you're when you're creating a factory, you know, a lot of the work is on the quality gates. You know, there's just no point disturbing uh creating a pull request for the human to review when that thing is low quality. You just got to create as many high-quality quality gates uh as possible and automate those to reduce the amount of tension you're sucking from the humans involved.

SPEAKER_01Uh ultimately, I think you do need uh, you know, on a I mean, some products you could almost like auto-merge kind of things. Uh you can of course use kind of stacked pull requests, or you could just take one long-running pull request. You can push to a pull request branch in an automated way to kind of create a larger and larger and larger feature. So that's that is okay, and that is how some loops kind of work. And of course, we have workflows which you know get the CI green, for example. So look at all the pull requests, find the ones that are red, work out what's going wrong, make the extra pull pushes.

SPEAKER_01Uh no, not propose, actually make the extra push onto the branch because you're only pushing to a branch. So we do allow automated pushes to pull request uh branches uh for, and you can usually constrain that down to be pull requests that have been created by the automation itself. Okay, so you're really kind of carving down to a to a subset. Um so where the industry and different projects will land, uh, I mean there's no one answer to that, but I do get that is there is more and more pressure towards sort of kind of automated, kind of full construction of software and automated absolutely everything.

SPEAKER_01And uh we don't go in that direction. We're being responsible. You know, yeah, there there are you know there are guidelines about how and uh an absolute human in the loop is absolutely a core guideline of what we're doing here. Of the path.

SPEAKER_02I mean, I I I think uh I I absolutely understand uh the sort of the driver for it and it's uh safer, but I'm also very conscious of the fact that I guess when I think about a factory, I think about a higher level of control plane, uh, and really everything that is in the current control plane that is includes sort of the modifications to the code, should sort of uh we should aspire to make it agentic. And so maybe it is the next batch of questions around uh saying, okay, you have kind of continuous actions that are sort of being done in a collaborative fashion on this sort of shared ledger that is the repo.

SPEAKER_02Uh and uh and you you propose those, and so you've created the the automation uh of uh of these activities uh at some point, and I think that point for many organizations is now, they're they're at a point at which they already have more changes they can accept. It at SNC oftentimes, actually very much sort of today, it's just like hey, the problem is not finding issues, like I have a whole world of issues that I already know of vulnerabilities, the problem is fixing it, hence the sort of the uh you know kind of a uh uh focus from inception on on fixing them.

SPEAKER_02Similarly, over here is like the the the if you've if you just got it to the finish line but you're not able to get it over the finish line and merged, uh then um then you just sort of create a review fatigue. And you know, to be to be blunt, like people are not necessarily well equipped anyway at that point in time. They haven't written a lot of things.

SPEAKER_01Oh, but that's and and this that's exactly they're not really able to make exactly review, right? So some of my work is applying uh GitHub agentic workflows to factories inside GitHub. Okay. Uh so we have, for example, a factory for CI performance improvement. Okay, now GitHub's a big product, and we and CI takes quite a while to run. And uh we want to improve that, the the the CI times, okay. We want to make incremental steady sort of imp improvement by finding the long-running tests and kind of working out maybe the matrix is a bit big or kind of setups, it can be become a fixer or whatever.

SPEAKER_01And um that that uh my job as a factory creator is to deliver high-quality pull requests where the reviewer is equipped, and I love that word equipped. My job is to equip the reviewer with all the information they need to understand why this is being done, what are the risks involved, what are the trade-offs involved, is this a minimal thing, as many quality gate checks on this thing as possible.

SPEAKER_01Of course, in this context, you get the CI CD quality gates uh kind of uh as a as a number one thing. Now, in this in this particular example, we're we're trying to improve CI perf. And actually, in this setting, you've got the evidence right there in front of you. You've got a run that was done with the zero changes or the latest run on master or whatever, on main, and then you've got the run that's done here, and you see, look, the test took 120 seconds there, and it took 15 seconds there. That we can bank that kind of improvement. We've got the evidence. So you s you you you you give the evidence to the to the reviewer, you say, you know, I found it, and maybe you didn't find the evidence.

SPEAKER_01You say this thing should be closed, or auto kind of close this kind of get rid of the trash off the off the production line because this was a this is a dodgy one, throw it away. It's a lot of throwing stuff away in this kind of world, much more than before. Yeah, you get you you get you get trash.

SPEAKER_02The work has been cheap, it's easier to throw it away because it was just a eye that sort of ran along. Cheap in relative.

SPEAKER_01Yeah, or it's just in the nature. I mean, cheap or not, it's the nature of the factory. If it produces rubbish, you throw it away, right? You know, the can has got a crush on it, you know, throw it away. On our next offsite for our uh our our team, I'm uh I want to go visit at some real factories with some real uh maybe maybe film studios or something like that. How do you actually how do people actually make stuff? How do they organize this? How do they tell what's good and what's not? How do they, you know, how many uh uh you know let's let's let's let's think about this differently. Yeah.

SPEAKER_02Um I think uh I have a whole world more of questions on it, but I'm afraid we're sort of like uh a little bit uh a little bit uh into some more of these things. Yeah. A little bit out of time over here on it. So uh I love uh first of all, Don, thanks for sort of coming on. I love the I I really, really relate to the sort of the term continuous AI, and I think uh you know there's there's there's kind of a world to define around it. There's still very much around definitions. Uh and nice and nice and world. I love that I love that.

SPEAKER_01It's always been my my my my joy of working in the in the tech industry.

SPEAKER_02So so super fun, very much like you know, powerful, kind of new, repo-centered, uh uh kind of cloud, uh, cloud agent sort of platform uh in the agentic workflows, and sounds like a lot over there that we didn't even sort of scratch the surface over here. And then some sort of forward-looking questions on uh the agentic workflows will kind of get you to the PR, what happens beyond, how does it relate to the loops, and uh you know, lots and lots to still explore.

SPEAKER_01Exciting times. And uh yeah, we call it the uh agentic repository automation revolution, is I think the title I used for the talk. And I I do think it's gonna be at the center of a lot of what practical teams are doing, and especially in enterprise, but also open source context. And you know, it's a great time to be a GitHub and this kind of work. Thanks for having me on.

SPEAKER_02Yeah, thanks for joining us. Thanks everybody for listening in. I do very much encourage you to go watch Don's uh talk, which is on uh on our YouTube channel on the AI Native Dev channels. We'll put it in the show notes. Uh it's a really excellent talk. It actually has a lot more content uh than what we discussed over here. Uh it's very practical advice. And I hope you tune in for the next one.

SPEAKER_00The AI Native Dev is brought to you by Tesla, the package manager for skills and context. Your hosts are Guy Pajani and me, Simon Maple. Our producer is Tom Dowler. The AI Native Dev is not just a podcast, it's a community. And we host monthly meetups at the Tesla offices in central London. Visit Tesla.io forward community to learn more at hostile dev.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Your Software Factory Needs a Bill of Materials | Daniel Meppiel

    Nov 19, 20261 hr 1 minAudio

    Developers are about to write skills instead of code, and Microsoft's Daniel Meppiel argues the software factory that runs them needs versioning, a bill of materials and signed receipts before enterprises will trust it....

    Credit the people on this episode

  2. Paul Stack: I'll Never Write Infrastructure as Code Again

    Nov 17, 20261 hr 2 minAudio

    Paul Stack helped build Terraform. Now he says he'll never write another line of infrastructure as code, because an agent harness runs his team's production infrastructure instead. Watch all the AI DevCon NYC 2026...

    Credit the people on this episode

  3. Edith Harbaugh: Code Is Cheap. The Last Mile Isn't.

    Nov 10, 202649 minAudio

    Automated software delivery is moving faster than the people who used to approve every release. LaunchDarkly co-founder and CEO Edith Harbaugh tells Guy Podjarny why rollback should be autonomous, why one-shot coding...

    Credit the people on this episode

  4. Maggie Appleton: Agentic Coding Has Outgrown the Text Box

    Nov 3, 202652 minAudio

    Agentic coding has outgrown the text box. Maggie Appleton, staff research engineer at GitHub Next, argues that walls of CLI output and AGENTS.md files serve coding agents far better than the humans steering them. She...

    Credit the people on this episode

  5. Caleb Sima: AI Attackers Move at Machine Speed

    Oct 27, 202655 minAudio

    AI attackers now move at machine speed, while most security teams still move at meeting speed. Former Robinhood CSO Caleb Sima has seen an AI attacker up close, and he argues the only way to keep up is to take humans...

    Credit the people on this episode