Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Pop Goes the Stack

Pop Goes the Stack artwork

Shift left into runtime: Vibe coding and AI guardrails

Jan 6, 202621 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Coding pipelines are evolving and AI agents are taking the wheel. In this episode of Pop Goes the Stack, F5's Joel Moses teams up with Buu Lam to dive into “vibe coding” - where tools like Claude Code and GitHub Copilot plan, build, and optimize apps faster than humans can debate lint rules. But is faster better? While agentic AI unlocks game-changing efficiency, it also introduces new risks: API keys hardcoded into apps, runaway GitHub actions, and a stark need for guardrails like sandboxing, runtime tripwires, and logging. As we embrace smarter pipelines, how do we stay in control? Join us as we explore the promises and pitfalls of shifting left with AI agents - and why the era of “self-improving code” is both exciting and terrifying. The machines are coding. Are you ready to debug?

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show.

00:00:05:06 - 00:00:21:20 Joel Moses Hi everybody and welcome to Pop Goes the Stack, where we take emerging tech flip it upside down and see what falls out, besides our own fragile sense of control. I'm Joel Moses filling in as your lead host today for Lori who can't be here. I'm here to talk about what shift left means when the agent is writing code

00:00:21:20 - 00:00:48:16 Joel Moses faster than we can argue about Linting rules. We're diving into the new reality where guardrails start at intent and policies kick in before a single token hits the repo, and your delivery pipeline might have to babysit an overenthusiastic vibe coding model with control of your keyboard. So buckle up. This one gets spicy. With me today is Buu Lam, Director of Community Evangelism, or as I like to call him sometimes, Mr. DevCentral.

00:00:48:18 - 00:00:49:20 Joel Moses Welcome, Buu.

00:00:49:23 - 00:00:58:21 Buu Lam Thanks for having me. Excited to be here and excited to listen to one of your intros live. It is always a treat to listen to those. So I feel honored.

00:00:58:24 - 00:01:16:00 Joel Moses Oh, it's good to have you here. You know, I thought we would begin by talking about a little bit about your experiences with using AI directed coding vibe coding as it's come to be called. Can you describe for us some of the things that you've done for the community work that you've been doing?

00:01:16:02 - 00:01:39:29 Buu Lam Yeah. Vibe coding for me has been a bit of a journey. I would say, like, well, I don't know, maybe other folks have gone into things like Lovable, you know, type in some sort of prompt and be wowed that some sort of app just appeared based on a couple sentences. So I've gone in between that, of just, hey, what can I do from kind of a one shot

00:01:39:29 - 00:02:02:07 Buu Lam and here's an application. I also went through, and I recommend this for a lot of people, me not having a development background, I actually went through a crash course or a really quick course on Python. And through that I was like, oh, now I know everything about programing, and so started to play a little bit more with vibe coding.

00:02:02:07 - 00:02:27:02 Buu Lam And I went from, hey, I'm just going to put in a couple prompts and just create something to actually peeling it back and being like thinking about, okay, now that I know how these pieces come together, can I be a little bit more thoughtful about how an application comes together and actually vibe code pieces of an application? And so, we're fortunate to have GitHub copilot at F5

00:02:27:07 - 00:03:01:00 Joel Moses Yeah.

Buu Lam and so was playing with that. And more recently I got on the Claude code train and I don't know if I'm getting off because having AI from the CLI has been fantastic. So again, I've gone from really testing how Claude code, how far can I get with a single prompt to actually starting to have a conversation with Claude to plan an application first, and weighing out all the decisions and having it ask me questions about what I want to do with this.

00:03:01:03 - 00:03:29:22 Buu Lam And then coming up with a few spec documents. And when I actually go to start building up the application, we plan out a feature list and we actually go through and vibe code a feature list one by one. And it's really allowed me to narrow down certain use cases for applications. And I still don't think I'm necessarily the best vibe coder at this point, but I actually start to see like, there's a range of vibe coding out there.

00:03:29:24 - 00:03:39:04 Joel Moses Yeah, there is. Now, how would you rank vibe coding as an asset for you? Tell me about what it was able to help you accomplish, for example.

00:03:39:07 - 00:04:06:19 Buu Lam Yeah. So I have an example of, you know, a lot of the things that people see from me out in public on social media is these videos that I do. And sometimes people will ask me, "hey, how did you be at an event and have that video that you recorded come out that evening or the next morning?" And the answer to that up until a couple weeks ago was, oh, I actually just stayed up all night to do that by hand.

00:04:06:22 - 00:04:29:11 Buu Lam Now, you know, using Claude, I actually went through and took all of the processes that I have that have actually been processes that are as fast as possible that I can do. First, principles: let's get rid of all the things that we don't need and do that process manually if, quote unquote, manually, if you will.

00:04:29:13 - 00:04:50:18 Buu Lam And so knowing all of that, I actually worked with Claude to say, "Okay, this is what I'm trying to do here. These are all the things that I have to think about. What could an application look like?" And we went through and made some adjustments and I started to build that. And so that pipeline of media: taking the content in, doing the transcriptions of the content.

00:04:50:21 - 00:05:11:11 Buu Lam Those are to me are really important because I do deal with a lot of international folks and I want, although it's English, I want the English to be right so that auto translation can work off of a caption file. So I do transcripts for folks and I have OpenAI Whisper integrated into this application to do that.

00:05:11:13 - 00:05:36:23 Buu Lam OpenAI Whisper has it's, understand it has different size models and different variations within those sizes as well. But you can, you can take the approach of fine tuning of Whisper or you can take the approach of actually just adding a vocabulary file to Whisper as well. So in our industry we've got a lot of acronyms, we've got a lot of terms,

00:05:36:26 - 00:05:58:12 Buu Lam and we've got as I travel internationally a lot of accents as well. So I can actually build a vocabulary file. Something that I can export, import as well, so it's transportable between projects. So I can do those transcriptions super easily. This app monitors a folder for MP4 files and then I can cut my social copy.

00:05:58:15 - 00:06:17:02 Buu Lam I can actually do edits on the fly within there. So it'll allow me to just break apart a video and be able to crop out different parts and just sends it off to FFmpeg and spits that out back to me. I have stats tracking in there as well. Another really great part of this is LinkedIn

00:06:17:03 - 00:06:41:10 Buu Lam is not very open with its API for folks. It protects the social network. But you can build, you can have Claude build extensions in Chrome that will do things that don't need the API to get the information that you need, at least in much fewer clicks as well. And so I built that into it.

Joel Moses Interesting.

Buu Lam You know, little add ons to make everything go faster.

00:06:41:11 - 00:07:02:25 Buu Lam So, at the net of it is I took a business process that I see value in, collapsed that business process into an application, kind of proved that that saves me time. I think it'll save my team time as well. And then we can look at, okay, what does that look like if it's actually a production system and not just the little script that I'm running on my laptop.

00:07:03:01 - 00:07:04:09 Joel Moses Right.

00:07:04:11 - 00:07:26:04 Joel Moses Well, it sounds good and, effectively, the net value to you is that you get to sleep at night while

Buu Lam That's helpful.

Joel Moses all of this post-processing occurs, which is, which is wonderful. Now, a lot of this is essentially describe, what you're describing to me is a managed pipeline. So, in one side goes the raw video and out the other side comes a finished package,

00:07:26:04 - 00:07:44:24 Joel Moses and all the way down to applying it to the various social networks that you need to apply it to. So, you're managing a pipeline and you talked about moving it to production. So let's roll back a little bit and talk about what you've done. What you've effectively done is you've created a structure that vibe codes,

00:07:44:24 - 00:08:14:08 Joel Moses and you have, you use directed input to an AI system to help you plan out what elements that you could bring to bear for that pipeline. So, your experiments, for example, with Claude code, you're developing effectively a development plan, a product plan so to speak, and then you're executing on that. Now, there of course is a lot of interest in having that whole solution managed as part of an ongoing code maintenance pipeline.

00:08:14:08 - 00:08:39:20 Joel Moses So you have the AI in the background continuously working to improve the pipeline that you've helped that you've developed. And so a lot of this is beginning to tie itself to agentic AI. So not just using vibe coding as a tool to assist the human in coding, but also applying vibe coding structures to automatically create enhancements to the pipeline when it notices problems.

00:08:39:21 - 00:09:03:07 Joel Moses Now, it's the I guess, you know, there's there was a lot of talk in the development circles about shifting left, about doing things with coding API first. And developing a shift left approach to your use of agentic AI in your pipeline is also becoming a popular thing. Now, what are some of the risks that you can see in that?

00:09:03:07 - 00:09:16:18 Joel Moses Like describe for me some things where maybe the vibe coding system that you used maybe took a wrong turn. Did you have any of those occur?

00:09:16:20 - 00:09:48:11 Buu Lam I've seen it take a wrong turn in terms of creating the thing that I asked it to create, and not in that it did something wrong, but I wasn't descriptive enough. And I think that's kind of where the sharp edges are, because I could see that it's not asking me about package versions. So there, you know, you would probably need a Claude skill to go out and do a scan to see if hey, the packages that I'm pulling down is there anything that I should be concerned about here?

00:09:48:13 - 00:10:21:16 Buu Lam There's the tests that Claude will do as it's built, as it's testing the features that I asked it to build. And I can see it correcting itself, but within there there's no security validations as part of it as well. So you probably need to build

Joel Moses That's a good point.

Buu Lam skills around that, as well. It's also just up in building the Docker, in my case, building up a Docker container after it's done doing local testing and packaging it for me, because I want to be able to just spin it up with a Docker compose.

00:10:21:18 - 00:10:43:05 Buu Lam And so from that respect it's also pulling in from the requirements file to see, you know, what other things do I need to build up for this environment. But all of those things along the way, those are all the things I think we talk about shifting left. And those are things that, you know, if you haven't adopted all of that already, you probably need to think about that just in your own little environment there.

00:10:43:07 - 00:11:09:21 Joel Moses Yeah. Now, what other things do you think you should think about if you, say you want to enhance the pipeline that you created and you want to use agentic AI to its fullest extent and have the code effectively self improve, to notice new library versions, as you mentioned, and incorporate those automatically without having to go back and run through your prompt sets again in order to generate new code.

00:11:09:23 - 00:11:14:18 Joel Moses What are some of the things that you think you might want to watch for if you do that?

00:11:14:21 - 00:11:50:28 Buu Lam Yeah, I've seen the wonderful folks at JFrog actually gave me a demo of this before, but having global or at least corporate wide policies on the artifacts they're brought in and there's, I think there is security implications to that, but there's also compatibility implications to that as well. You know if you, you might be creating something that is more client server and you're creating the client side of things and you're creating something that could potentially pose impose something on the server side or vice versa,

00:11:51:00 - 00:12:23:07 Buu Lam I think there's implications to that as well. But Claude can just go out and pull anything it wants at the end of the day into your stuff. So, if you're not paying close attention to what actually comes in and then, in addition to that, like things like static code analysis that you would normally, hopefully, normally people would do as well or dynamic analysis or dynamic security testing on your application, that's not built into the into that pipeline as well.

00:12:23:07 - 00:12:45:09 Buu Lam So, you know, translating, all of that and, you know, I had Claude build some GitHub actions for me to do a rebuild of a thing that I was doing as well. But within there also know it rebuilds once a day and within that there isn't--hey, this is not a production application by the way--there's

Joel Moses Right.

00:12:45:11 - 00:13:05:10 Buu Lam no security checks in those GitHub actions that it's building.

Joel Moses Right.

Buu Lam And I'll give you an example here. I was building something that needed an API key. That API key has constraints on it. But Claude asked me to put, hardcode the API key into the code. And like,

Joel Moses Yeah.

Buu Lam hold up Claude. I don't want to do that.

00:13:05:10 - 00:13:08:19 Buu Lam And we worked together to, you know, do secrets management.

Joel Moses To resolve that,

00:13:08:21 - 00:13:30:01 Joel Moses yeah. So effectively what you're telling me is, first of all, you need a place to evaluate as part of the planning loop. So putting evals in your planning loop, it's a bit like having a personal trainer that slaps the donut out of your hand, before you commit to your repo, right. So having a place to evaluate what's about to be applied.

00:13:30:03 - 00:13:47:26 Joel Moses I think also maybe using sandboxing by default. So, you know an agent, if you ask it to do things you want to have a box in which it can draw with crayons. It's like a toddler with a box of crayons. They'll still try to color the dog, but at least they'll ask permission first, right?

00:13:47:29 - 00:14:16:24 Joel Moses So I think that's another mechanism that can be brought to bear there as well. Another thing is, you know, these services and you used a GitHub action as an example, never incorporate as part of their security model any secrets or keys. But even beyond that, there are elements where you might want to incorporate a check for, what I would call, over utilization.

00:14:16:25 - 00:14:41:26 Joel Moses So instead of instantiating 50 different GitHub actions, try to figure out ways that you can reduce the instantiation to one, because that has a direct correlation to cost. So runtime tripwires, I guess. Maybe, maybe watching to see if the agent is intently spawning infinite threads or using four actions where one will suffice.

00:14:41:29 - 00:14:53:15 Joel Moses So, these are all elements that you're going to have to catch or write into your prompts that guide the actions of your agent. Any other things you can think of?

00:14:53:18 - 00:15:21:19 Buu Lam You know, just on that note too, that last point there is that my early attempts at vibe coding where I didn't really know what I was doing, I was passing full context back and forth, and I was using a GitHub plugin, or sorry a VS Code plugin, that was showing me the cost for each back and forth. And the number kept going up and up

Joel Moses Up and up.

Buu Lam and up because it kept adding more stuff and something broke and copilot was like, 'oh, I'll just add in some debugging.'

00:15:21:21 - 00:15:49:23 Buu Lam Well, it added debugging everywhere. So that number just kept

Joel Moses Going up and up and up.

Buu Lam So I can see for GitHub actions, you're paying for usage on that one as well. So seeing that go up and up is yeah, that's a scary thought for letting things get away from you. I've also seen more recently Docker in your sandbox environment there as you're building, if you're using Docker they have, seems like, policy-driven

00:15:49:25 - 00:16:06:18 Buu Lam MCP or they, policy-defined MCP servers, that they make available. And also MCP gateway for egress control or and MCP server as well. So that might be a scenario there where, you know, people incorporate that into their sandbox development; they can see what's going on.

00:16:06:20 - 00:16:24:11 Joel Moses Yeah, it's a good idea. The other thing that I can think of is that, you know, if you're going to have a system that conducts these actions, every action should be logged. Make sure that you can trace it back. I mean, you talked about a GitHub action that needs to use an API key.

00:16:24:12 - 00:16:49:09 Joel Moses Well, if you give an agent the access to use that key and then expect it to stay within some confines that you set, you need to know when it goes outside of those confines. And you can't do that unless you're logging the action sets that it takes. You know, if an agent spins up 300 GPU instances, you're probably going to want to know exactly what conditions that was done under.

00:16:49:11 - 00:16:57:06 Buu Lam Yeah. Yeah. Hopefully you have quotas set on your cloud accounts and whatnot that'll hopefully trip off some of that stuff.

00:16:57:09 - 00:17:27:11 Joel Moses Yeah. That's true. Now, let's reverse course. We've talked about the dangers in using agentic AI and some of the things that you need to use that you might need to think about if you're going to design an agent based coding system into your pipeline. Which, but there's a lot of interest in doing so because, you know, I have been in development for a long time and I've sat and watched two engineers debate a single lint rule for 45 minutes, whereas, you know, 45 minutes

00:17:27:11 - 00:17:53:00 Joel Moses an agent can write, test, ship, document and benchmark a feature. So, yeah. I mean, in the same time that it would take to for the single lint rule to be debated an engineer shipped no features and an agent shipped one. So clearly there's some value to be had here. Any other benefits to incorporating agentic coding?

00:17:53:03 - 00:18:13:27 Buu Lam Yeah, I would say, you know, all of those things--your wish list of things that I wish I had--if this just existed, I could do my job so much more better or faster or whatever it might be that benefit getting that proof of concept out the door and showing that value, and then that becomes a justification.

00:18:13:27 - 00:18:22:28 Buu Lam But, you know, being able to validate something super quick and pursue business I think is super valuable for businesses right now.

00:18:23:01 - 00:18:37:09 Joel Moses Yeah, I agree. We're ticking down to the close to time and so we always like to finish out Pop Goes the Stack with some lessons learned. What did you learn today about the possibility of an agent coding for you?

00:18:37:12 - 00:19:05:02 Buu Lam I would say that even though you can go out to those single short platforms to build something, just taking a little bit of time to learn, and again I'm no expert, but taking a little bit of time to learn a little bit of software development principles is going to help out in a lot of ways. And actually, I would increase my knowledge now just because I know I can work with Claude better when I know more about that.

00:19:05:02 - 00:19:19:12 Buu Lam So I think vibe coding is not a hands off situation. It's very much actually the more knowledge you have and can apply to vibe coding, just it's jet fuel to really smart people.

Joel Moses Yeah.

Buu Lam So, hope to become a really smart person.

00:19:19:15 - 00:19:40:13 Joel Moses Yeah. I think I learned that when you're talking what you should inject into your pipeline, you can't forget to do the things that you would do with an inexperienced coder. You have to know who asked the system to do what and when. So making sure that you record transactions, that every action is logged, and that you can trace it back.

00:19:40:14 - 00:20:02:12 Joel Moses That's important and as you embed agentic AI into these systems, you need to ask it to do that for you. Runtime trip wires, having some sort of outside system that mediates and looks at the elements of cost, I think that's important as well. Sandboxing by default, making sure that you are very explicit about what the agent can do and what it should never do.

00:20:02:14 - 00:20:09:00 Joel Moses And then putting evaluations in your planning loop for your product plan. These are all important things to accomplish. But

00:20:09:02 - 00:20:13:15 Buu Lam It sounds like you've created vibe coding regulations on the fly, Joel.

00:20:13:17 - 00:20:34:03 Joel Moses Well, you know, I'm trying. I'm absolutely trying. Unfortunately, that's all the time we have for today. That's, if you enjoyed learning how to keep your agents from YOLO shipping code at 3 a.m., go ahead and hit like and subscribe. We'll be back with more ways to stay one step ahead of the machine and preferably upstream of their pull requests.

00:20:34:06 - 00:20:38:03 Joel Moses For now, that's all. Thanks for listening to Pop Goes the Stack.

Open the transcript file

Hosts and Guests

Hosts

  • Joel Moses Host Feed-stated

Guests

  • Buu Lam Guest Feed-stated
  • Tabitha R.R. Powell Producer Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Agent ecosystems: Identity, control, and the shadow AI problem

    Nov 17, 202623 minAudio

    AI agents are no longer “features,” they’re becoming enterprise citizens with identities, permissions, runtimes, and a very real need for governance. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses...

    Credit the people on this episode

  2. Vibe coding is manufacturing tech debt at machine speed

    Nov 10, 202622 minAudio

    Vibe coding is moving fast, but is it moving software forward or just piling up a new kind of debt. In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses are joined by Narayan Veeramani to dig into...

    Credit the people on this episode

  3. AI governance: Shadow AI, real access, real risk

    Nov 3, 202621 minAudio

    AI outputs may be polished, but in this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and returning guest Ken Arora focus on what happens when confidence meets real-world access. The discussion moves...

    Credit the people on this episode

  4. AI broke your app? It’s probably DNS (Again)

    Oct 27, 202622 minAudio

    AI may be rewriting workflows and reshaping infrastructure, but this episode of Pop Goes the Stack argues that the future still depends on an old, unglamorous constant: DNS. F5's Lori MacVittie and Joel Moses are joined...

    Credit the people on this episode

  5. Agentic fingerprints: Detect, debug, and control AI traffic

    Oct 20, 202620 minAudio

    AI is everywhere, but success metrics and security controls haven’t quite caught up. In this episode of Pop Goes the Stack, hosts Lori MacVittie and Joel Moses are joined by Lee Ennis to unpack “agentic fingerprints,”...

    Credit the people on this episode