Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Down the Security Rabbithole Podcast (DtSR)

Down the Security Rabbithole Podcast (DtSR) artwork

DtSR Episode 727 - Phil Brass - When AI Supports Pentesting

Oct 13, 202629 minAudio
ShareXLinkedInFacebookWhatsAppEmail

TL;DR: Phile has been penetration testing for a long time. The difference now is that he's having a much easier time with it thanks to AI. It's an interesting world, and he's here to explain the offensive side of it. Guest: Phil Brass Description Zero-days are not a once-in-a-career event anymore. From B-Sides Atlanta, we sit down with veteran penetration tester Phil Brass to unpack what happens when large language models make vulnerability research dramatically faster and cheaper, not just for defenders, but for attackers too. We get specific about where AI penetration testing actually moves the needle: LLM-assisted security research that can turn a normal two-week internal pen test into a hunt for brand-new bugs in the commercial software your environment depends on. Phil walks through the mindset shift from “scan the network and review configs” to “download the exact products the client runs and start looking for flaws,” including lessons learned from targeting VMware infrastructure and finding critical issues in vCenter, the control plane that can unlock everything downstream. That power comes with consequences. We talk responsible disclosure, vendor response, and the uncomfortable reality that patches can be reverse-engineered quickly, leading to real-world exploitation soon after fixes land. On the defense side, the guidance is refreshingly direct: tighten patch cycles, take threat intelligence seriously, and implement real micro-segmentation so one foothold does not expose every dependency on your network. We also dig into third-party risk management and why asking suppliers about LLM-assisted testing may become a new baseline. We close with the human side of AI in security: validating LLM output, dealing with hallucinations, avoiding intellectual laziness, and making judgment calls that protect production systems. If you found this useful, subscribe, share it with a teammate, and leave a review so more builders and defenders can find the show. Have something to say? Let's hear it. Support the show >>> Please consider clicking the link above to support the show! -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= YouTube home: https://youtube.com/playlist?list=PLyo0dkKRvfVtWXjRxNISrhme1MgBj3C2U&si=scHDiTuLXSEQ9qHq LinkedIn Page: https://www.linkedin.com/company/down-the-security-rabbithole-podcast/ X/Twitter: https://twitter.com/dtsr_podcast

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00They say they say we should have known better than two. Let's go deep down, deep down into this rabbit hole. And uh let's think. Hi, this is Nikolai, and you're listening to My Rabbit Rafael, co-host James Curity, and featuring Mr. Jim Tillers on the one and only Young Security Rabbit Hole podcast. Again, listen up and enjoy.

SPEAKER_03Hanging out with Phil Brass here. I'm recording live this week from B-Sides Atlanta. Uh, today, I don't know, it's what is today, October 3rd? So you guys are getting a whole slew of episodes recording in a single day released over the next four or five weeks. But I've already done Mehul Avankar, he's been on the show. We talked about cyber defense in the age of AI. Phil's about to give us a little bit of LLMs and pen testing flavor. Uh still got Wes Widener coming on, talking about open weight models, Xavier Ash, story time. I can't wait for that one. And uh Matt Coons is gonna do agentic detection engineering.

SPEAKER_03You guys know what this show is all about. So welcome to it. All right, sitting down, Mr. Phil Brass, the one, the only. Uh I'm gonna let him do his own intro because Phil's done some things.

SPEAKER_02Yeah, yeah. So, you know, I moved here uh to Atlanta in 1998 to join Internet Security Systems. At the time, I was a software engineer on Internet Scanner. You'll never guess what that product did. It scammed the internet. It definitely crashed a lot of things. In fact, if we did storytime, we could probably talk about a lot of high-value assets that got destroyed in the early days of testing networks. I moved into consulting in 2002 as a penetration tester, application security guy, hung out there at IBM, and then eventually moved to Acuvant, worked with a bunch of great people over there.

SPEAKER_02They got bought, turned into Optive. I survived that merger and persisted for a while, moved up into management, left as kind of director of AppSec, consulting like a 60-person program, went over to Direct Defense, had a great time there, made a bunch of new friends, and also stole a bunch of old friends from Opti. Rescued, you might say. And then recently I am over at a Traders Partners where I'm actually billing mostly. I don't do a ton of management, people management and stuff.

SPEAKER_02I'm doing a lot of pen testing, a lot of application security work. So I love it. I love it too.

SPEAKER_03We got to work together briefly. It was fun. It did. It was we we have that we were part of the Optive Survivors Club? Exactly. That's right. So listen, uh there's a lot of like I think every every interview we're doing this today has something to do with AI. And uh pen testing is is no exception. For those of you that caught last week's episode or the last one I did with Mahulur Vankar, he talked about how basically vulnerability scanning is is you know the costs and the are are the only limitation on finding bones.

SPEAKER_03We on this show, Phil, I don't know if you caught the last couple episodes. If you scroll back far enough, we had Brian Martin, we've had Anton, we've had Antel, we've had it actually all started with uh with our snake talking about how like this whole patching process is BS, and then we like went from there. Look, the the the long and short of it is I think AI is fundamentally changing how we find and do stuff with vulnerabilities. Yes. Tell me what you you give me your like high-level perspective what AI is doing for pen testing, and then we'll go from there.

SPEAKER_02Yeah, yeah. So I think there is a, you know, I saw a survey recently. It was like, which one of these things is AI gonna be most helpful for, right? Code reviews, internal pen tests, red team, external pen tests, and almost everyone said code reviews, right? And I think that's good. I think it can absolutely help with code reviews. I think the reason people don't want to use AI on their internal pen tests in particular is the clients don't want to expose their sensitive data, right? The internal network is not a test network, it's prod.

SPEAKER_02And so if the AI goes rogue and takes down your Kubernetes cluster, you might be kind of out of luck, right? The client will probably fire you, you might get sued. And a lot of the automation that we do in pen testing doesn't need an AI, right? It's a pretty simple runbook style automation. If you get responder, it does this, and then you move on to that and you pivot over here. Easy stuff, right? But what I found is that actually LLN can do something really useful for you, and that's they can do your security research for you, and they can find vulnerabilities.

SPEAKER_02And in particular, they can find zero days in the infrastructure that your clients are using. So, if, say, for example, the example that we gave in our talk today, right? We're doing a pen test, an internal pen test, against a big financial organization's private cloud. It's all VMware based. So first thing we did is we downloaded the same versions of VMware that they were using and started looking for bugs. And we were using Cloud to do that because VMware is not open source. So hold on.

SPEAKER_03Sorry, I I'm gonna interrupt you a couple times. Go ahead. I've stream a thought. Is that where we start with now? Because it used to I would think that we would set AI loose on like their existing config, because oftentimes, right? It's it's even if there are no open bugs, the configurations are the weakness. More often than not, has that changed?

SPEAKER_02Not not in every case, but in this case of a big financial and a hardened environment, like we did our first two days' worth of work, you know, we did our scanning, we looked at open services, and you know, there wasn't anything there. There were no vulnerabilities, like it's not our first time around going around with this client. And uh we had previously had to turn in a pen test with almost no findings, and we didn't like that. So I was like, all right, we're gonna do this differently. We're gonna start with a hunt for you know end days and zero days in the products that they're using, not in their network necessarily, but in the actual products.

SPEAKER_02Right. Okay. And the other thing I want to say is, and I can use VMware as an example because we've done disclosure and they patched it and it's all fixed. But in the like three of my last four pen tests, we've done the same approach with targeting, you know, third-party software and finding critical vulnerabilities. Okay. It's a really, really useful approach. People don't think about it, but your LLM can be basically a little zero-day factory for you while you're working on the client's config and talking to their network and all that stuff.

SPEAKER_02You you say, hey, I'm I've noticed this CMS on the client site. Go analyze that CMS and look for vulnerabilities.

SPEAKER_03I don't think that's a way I think that's novel, right?

SPEAKER_02That that's not how testers operated before. It hasn't been like that, but I think it's new, and it's because the the models have gotten so good at reverse engineering and vulnerability finding, like tearing apart, remember VMware ESXi in particular isn't a known operating system. It's not based on Linux. It's like it's kind of based on BSD, but like there's no source, no one's ever seen it. But the LLMs were tearing it apart like mad. I mean, they had no problem being like, oh, this thing and this ring buffer, and we're gonna overflow this.

SPEAKER_02And they were designing all kinds of things. Really? Yeah. Now I will make this point. We never got anything useful working against ESXi. Okay.

SPEAKER_03But I was gonna say, you're about to scare the hell out of people.

SPEAKER_02No, what we got what we got instead was two critical vulnerabilities against VMware vCenter, which is like the Active Directory server of VMware, right? It can controls everything. And so with those that's somehow worse. Yeah, because we could start at the top, we could then take over all the ESXi servers. Like we were able to spin up a Kali Linux VM on the client's equipment, you know, because we were administered. Oh, it was fun. It was such a blast. And every time this works, it's such a rush that you get this, you get this new toy, and you show the client, and then you go do disclosure.

SPEAKER_03Do you feel like this? I think I know the answer to this question, but do you feel like this gives you an unfair advantage over what you had before and previously? Like, is this is this really I think the answer is yes.

SPEAKER_02No, I'm I am like the the the power that I have now as a penetration tester and the power that attackers have as attackers is that they are going to be about a hundred times better at vulnerability research. And this is different from Mahoul, by the way. I saw his talk and it was amazing. And he's very focused on known CVEs, right? Yeah, yeah, yeah. And this is different. This is zero days. You can't be ready for these, yeah, because you don't know what they are. But attackers can bring them in like this anecdotal, right? Three of my last four, I've been able to find them, but that's pretty compelling.

SPEAKER_02If if 75% of the time you can bring a zero day to an engagement, yeah, the world of pen testing is going to be so much more fun. That's interesting.

SPEAKER_03Um but you're also probably at I mean, at some point you have to you're having to follow the all the disclosure. Oh, we totally that's that is the follow-on process, right?

SPEAKER_02Right. That's that's you have to work with the the vendors. Yeah, so we worked with Broadcom, they were very, very responsive. They got their patch out in two weeks after we notified them, which is really fast for commercial software, and they were just very responsive in general, I feel like. It was great to work with them. And then within another two days, the patch had been reverse-engineered by attackers and attacks were happening across the world. Oops. Yep. I mean, that's going to happen. I think if we think about this new model of pen testing, and really this new model of adversarial behavior, right, where an adversary can come with zero days, the question has to be what are you going to do about it?

SPEAKER_02What can you do? At least one thing is you have to be right on top of your CTI. Okay, so tell me, tell me about that. Your threat intelligence comes and says, hey, there's a new vulnerability in vCenter, you have to either patch immediately or isolate or find some way of making sure that it's not exploited because attackers are going to be right behind that, if not a day in front of it. Yeah. Yeah, fair enough. You have to be super strong with your micro-segmentation of your network. Like this flat networks are not going to fly anymore because if an attacker gets one foothold, I don't want them to be able to see every single piece of software on your network, because I guarantee at least one of them has a critical escalation.

SPEAKER_02Phil, are we about to say go do the basics? Yes. That's what we're talking about. The basics work. I love it. But I think what I'm trying to say is you have to be more rigorous with them than you used to be, right? You really have to be able to, like I said, like actual micro-segmentation, like making sure that each box on the network can only see the necessary other things that it needs, that's hard work. Figuring out that policy takes a long time. Implementing it is hard. People will complain, but you have to do it.

SPEAKER_02Stay on your stuff right now. Staying on top of the every threat that comes over your threat intelligence platform, that's going to take more people. What I'm saying is, it's going to take more people, it's going to take more work, right? You need to, when you're doing your third-party risk management, right? You're asking your partners who are selling you software, hey, guy or gal, did you do your standard security practices on this? Now you have to add a question. You have to say, did you get like a full LLM assisted penetration test on this appliance or this piece of data?

SPEAKER_02Is that the new requirement? I think it's the new requirement. Because like the vulnerabilities we found in vCenter, neither one of them would have been detected by dynamic analysis, by static analysis, or by software composition analysis. They're all little vulnerabilities. That's why they were there. They were there because they fit in the cracks.

SPEAKER_03They're in between things.

SPEAKER_02Yeah, they're little in between. They're like a config file that just didn't have like host name validation.

SPEAKER_03But that's like a I feel like that's all that's a lot of what we see is is the really juicy ones are the bugs that they don't, it's not quite they're harder to find. It's not seen by run you see it only like at runtime. Like you only see it when you've loaded that module and put that bad buffer, it filled that buffer, you go, oh crap, right?

SPEAKER_02Like that's they're harder to find. And the static analysis tools will find a lot of useful stuff, but they won't find everything. And so if you're making software, you need to add this type of assessment into your process. Yeah. And if you if you're an organization and you got some critical dependencies on third-party software, you might not want to wait for them to do an LLM assisted pen test of their stuff. You might just take it upon yourself to do it yourself. Because if you can get it, if you can find those vulnerabilities before the attackers, you've won.

SPEAKER_02You can take them to the vendors and say, you have to fix this. I'm gonna disclose in 90 days, right? Like, or you can have a cybersecurity partner of any flavor do it, right? It's not an ad for mine. I'm just saying, like, you gotta get them to fix it, but you should be looking for this stuff because your environment suffers if it gets exploited.

SPEAKER_03Has this how how has not has this, but of course it has. How has this changed the nature of your engagements?

SPEAKER_02So it's made them a lot more successful.

SPEAKER_03That's for sure. Awesome.

SPEAKER_02Yeah. Here's the interesting thing about it. You think to yourself, the LLM is a 100 times better security researcher than Phil, a hundred times more productive, hundred times faster, right? So it can do things in 30 minutes that would take me a couple of weeks. Yeah. And this is why it's useful now on a penetration testing engagement. So I've got two weeks to do this test, and normally not going to spend those two weeks reverse engineering one piece of software. That's fair. But if I can spend 30 minutes or an hour reversing one piece of software and do that for four or five pieces of software and end up with a tool that gives me root in your environment, I'm gonna do it.

SPEAKER_02And so is the attacker. So it changed them in that sense. We think about does it make the pen test shorter, right? Is it do you are you more productive? And the answer is some parts of the test become shorter, but because I have so many more findings that I have to write up, other parts of the test take a lot longer.

SPEAKER_03So you that that I was gonna ask you that next. So does how does it how does it how does the nature of the work change?

SPEAKER_02And so now you spend some non-significant amount of time validating what the LLM said, that they lie wholeheartedly and frequently, making sure that it the thing it said is a zero-day works and it works in the client environment, and then making sure I get my write-ups done. And and if it produces, you know, I ran a little LLM sweep against the client I'm working on next week, getting ahead of the game, right? Downloaded their software, started the thing running. I've got a hundred findings from like yesterday afternoon to this morning.

SPEAKER_02And now I've got a hundred things that I have to start working on on Monday before anything else.

SPEAKER_03What's your what's the uh success rate? I'm afraid to ask this. What's what's the actual success rate? Like the the the because there's the finding of bugs, yeah, and we know that that rate is going to be high, but then there's the creating a reliable, durable, repeatable exploit that then gets you something you want. Because it's one thing to blow up a machine, right? Like oh sure. Listen, I I I've written some I've messed with, I'll say I have a I wouldn't say written, because that that that's giving me way too much credit, but I I've messed with exploits that like somebody says, hey, this is almost working and I'll try it, and like, oh, blew it up, oh, blew it up, oh, blew it up.

SPEAKER_03Yep, but that's not I mean, look, I can blow up your machines, okay.

SPEAKER_02But you know, it's a denial of service. Yeah. Um, so in particular, as we look at binary exploitation, which not everything has to be a binary exploitation, and we look at race conditions, those are areas where proper alignment of memory can be very difficult to get, right? If you're doing heap grooming to make the heap look a certain way so that your heap overflow works and your functions get called, that's got a high chance of failure. But the LLMs know a lot of techniques to make it work. They know how to send a bunch of data in to align the heap the way they need it.

SPEAKER_02But they also like for race conditions, right? Like I'm just recently working on a very popular web server, and and Daybreak was like, oh, I've got a race condition here. It's totally gonna let me get remote code execution. And walking through it, and uh at the end of it, like it could never quite get it to work. I'm like, how long was our window of opportunity? It's like, oh, we had six instructions to get these other 20 things done. I'm like, so there's no way this is ever gonna work, right? Like somehow that process has to get swapped out in the middle of those six instructions, and my other process has to be able to run what would feel like indefinitely at machine speed.

SPEAKER_02So it's absolutely possible for them to make claims that you can't actually implement in the real world or to produce vulnerabilities that don't quite work. I had a great auth bypass that bypasses client certificate protections, and it works about one in every 250 HTTP requests. That's not bad though. I'll take that. Yeah, I can just get how many HTTP requests a proxy or machine sees it. I just keep sending them until I get in and then I'm in. Like, no, no, no complaints, no notes.

SPEAKER_02That's worthwhile. So there's different types of exploits and they have different types of success and failure, I would say. And things like binary exploitations and rate conditions are prone to not being that successful consistently. And some of them are prone to crashing the process and being unrecoverable. And so, yeah, you don't necessarily want those.

SPEAKER_03So that's what I was gonna ask next is kind of like where I'm going is what's the downside? Is there is there any downside to doing this? Is there any any pitfalls, any, any challenges, anything that that you're like, hey, you know what?

SPEAKER_02This is great, but I would say the clients don't always like it because they're like, what's my what's your recommendation? The recommendation is work with the vendor and get this fixed. And that's not a quick recommendation, right? That could take months. No, yeah, you're right. We don't know. So, like, I think that's kind of a downside of it. If it takes too much focus away from the rest of your penetration testing, you've got to do a whole web app, right? And you spend all your time in Claude instead.

SPEAKER_02I think there's downsides on the knowledge side. Like, one of the vulnerabilities in vCenter, you know, like I had a working proof of concept. I had no idea how it worked. It's a really intricate, bizarre cryptography failure that just lets you log in as an administrator without a password. It was great. You can read the advisory if you want. But in order to write that advisory in a human-sounding voice and not Claude voice, I had to go back and read everything Claude had written and walk through the cryptography and be like, ah, now I see the problem.

SPEAKER_02So you it is very easy to rely on it to the extent that you are never gonna understand the vulnerabilities you're using and the exploits you're using. And we think about that a lot in the world of AI. We worry about our are our kids, is our kids learning, right? Or whatever the famous quote was.

SPEAKER_03This is uh I I was kind of I was hesitating on whether I want to go here or not, but what the hell? Let's go here. This is what this podcast does.

SPEAKER_01Yeah.

SPEAKER_03Try this again. Does this have the propensity to make pen testers? Not less in like does it less discerning, lazy maybe, lazier? Because I as a pen tester, I know I was lazy. I think laziness is a virtue. Well, that's that's what I'm saying. Like that that I think that was a good thing.

SPEAKER_02Yes, I it intellectually lazy is a bad thing. And I think it absolutely has the capacity to make you so intellectually lazy that you don't understand the tools you're using and what they're capable of. That's a real danger. That's the problem. Don't really want that. How do you avoid that? Well, you know, you you make people like my boss is like, yeah, you can't submit that advisory. You didn't write it. Oh. Yeah. Okay. And he was right. I shouldn't have submitted it.

SPEAKER_02I didn't submit it until I'd written it myself. You've got to be able to describe your findings. And like if I got on a call with a client and they're like, What did you do? And I'm like, I ran this tool, and they're like, What does the tool do? And I'd say stuff, Claude, then I Yeah they can just hire Claude.

SPEAKER_03They don't need you then.

SPEAKER_02Don't necessarily need me, right? Exactly. Even though in this case, like they need me to operate the tool safely and do things like that. But it's still like if I don't understand it and I can't explain it, then I'm failing as a consultant. I'm giving them very little useful information.

SPEAKER_03Does having AI in the pen test cycle change anything fundamentally about the way you pen test? Or does it simply give you superhuman powers?

SPEAKER_02It it can out. I mean, there are people who are doing pen tests without humans at all. Right? I just think there's a lot of risk in that. I think that the rogue AI risk, and it's not like rogue in a malicious sense, but just rogue in the sense that it might make a bad decision and drop your cluster or delete your file share. Yeah. Or think that this is a critical vulnerability. I need to patch this right now on some machine that can't have that kind of downtime. That's always a danger. Right? I worry about that. So, but it can, it can absolutely change it completely into a 100% machine driven operation.

SPEAKER_02And I think we do see people selling that type of service and people implementing that internally. And then we see a lot of services that are human. A loop. I'm thinking of services like Terra and companies like that that are doing application testing and code review, and allegedly, you know, there are people making reviewing what the thing wants to do when it's testing the application. Those are great in particular because it's almost always testing in a test environment. I don't really care if the AI goes rogue in a test environment. I worry about AIs going rogue in production. Yeah. And so that's why I feel like internal pen tests in particular, and to some extent, external pen tests.

SPEAKER_02The nice thing about external pen tests is whatever I'm going to do, some other actual bad person has probably already tried it.

SPEAKER_03Yeah, that would that's been, I mean, but that's been the argument for a long time. Is like if you're not if you're not pen testing or you know red teaming your environment, somebody else is. Someone already almost guaranteed. Yeah, you just don't get the reports. Right. Except now it's, I don't know if it's somebody, something, right?

SPEAKER_02Right, exactly. Something.

SPEAKER_03Listen, is is this is adding AI into this loop creating an efficiency for you?

SPEAKER_02Yes. 100%. It is speeding up, it is, it is making it possible to do research in a penetration testing timeframe that would not have been possible before. Okay. And by doing that vulnerability research, we can bring those vulnerabilities to bear during the penetration test. And that's absolutely adding value to our clients because it makes us, it keeps us close to the attackers they want us to emulate so that they can understand how they fare against that level of attacker. So we're bringing that kind of advanced attacker capability when we bring AI.

SPEAKER_02Okay.

SPEAKER_03Are you able to I guess this is a a career type of question, but does that mean that does this allow you to hire and bring in let we'll say more novice people into a into a pen testing environment and let them and and does AI in the in the pen testing loop help them learn more? And I guess that you know we've already talked about the danger of like being lazy and letting the tool do the work, but like does is this a learning tool as well?

SPEAKER_02Well, first of all, I think it can absolutely be a learning tool. I think there's a talk going on in about 10 minutes where there's a guy who I met at OWASP Atlanta in Braden who used AI to teach him most of AppSec in like a month or two to the point where he passed an interview, he's got a job now, he's an AppSec guy. Right? Wow, there you go. And it was a hard interview, right? But he learned all the stuff with L with the LLMs helping guide his learning journey, basically, and really helping drive that knowledge into his brain.

SPEAKER_02One way to do it. Yeah, uh hey, I mean, if it worked and got him a job in this market, like let's go. Right. I'm like, maybe I need to go to that talk and see if I can relearn some of the things I've been forgetting. I'm not getting any yellow. Yeah. So it it it LLM can absolutely be a tool to help you learn, I'm convinced. What we need to keep in mind is that good pen testing and advanced pen testing requires a lot of judgment calls about what is smart to do and what is not smart to do, what is in scope and what is not in scope.

SPEAKER_02And it's not always written down in language that an AI would understand or written down at all what what might be acceptable risk and not acceptable risk. And so for now at least, I think those risk decisions are best made by people with at least some experience in IT environments who understand this is a prod environment and nothing can go down, is an important thing, right?

SPEAKER_03That that that brings me to a so you you okay. I'm glad you brought that you brought this up because I wanted to because we always I say we, the royal we had there was a there was a belief that the best pen testers were former practitioners, IT people, right, that first knew how everything worked and then could step back and say, now I know where it's more most likely to break and what it means if it breaks right here. And the worst pen testers were always like the ones that simply started there and and only looked at like through the, as they say, through the paper towel tube.

SPEAKER_03Yeah.

SPEAKER_02Yeah, yeah. No, I absolutely agree. And I think you know, the great thing about being a former administrator, right, is that when you do get administrative access, you understand exactly what's within your power. Yeah, that's fair. Right? And you know all of the tools. You're like, oh, I'm a Cisco admin. Now I can construct a tunnel between my computer and this Cisco router, and now I can scan the internal network from the external network, from the internet, right? Over this tunnel I've just created. Like you know all of the tricks and all of the things you can do. And you know whether you should do them.

SPEAKER_02I think that's the that's the part I'm not convinced uh AIs are quite ready for yet. But like an AI could absolutely set up a tunnel over Cisco, it would do it in like, you know, 10 times faster than me. I would have to pull out like an old book. And the AI already knows it, it's part of its training data how to configure Cisco's. It knows everything. So, in that sense, I think, you know, like the need for skilled operators, it's more about the need for people who can make good judgment calls about IT impact.

SPEAKER_02Because pen testing can always have an IT impact, and you need to be able to make good judgment calls about what degree of impact would be acceptable to your clients and what degree would not be acceptable in order to prove your point about access.

SPEAKER_03Okay. So, all right, as we run out of time here, what do you want to leave people with? You give us the you got two minutes. Yeah. What do you want to leave us with?

SPEAKER_02I mean, I think the pitch is this almost every piece of software that you rely on has critical zero day vulnerabilities in it right now. Maybe at least three-quarters of it, based on my, like I said, my simple data from my own pen test. So you need to start thinking about the world where attackers show up with zero days, and you need to start thinking about whether you should get more aggressive in your testing and use Cementel to test your own products and test the products that you rely on critically.

SPEAKER_02That's what I would say is this the change that is happening is AIs can bring a lot more adversarial attention. Attackers are going to use that, and you need to start using that too. Or your consultants need to start using that. And there, and dare I say, do the basics better. That's a key element of defense, is much tighter patch cycles, much better network segmentation, just staying on top of everything, much better third-party risk management, all of it. Yes. Do the basics with a degree of rigor that is an order of magnitude better than what you're doing right now.

SPEAKER_02Okay, well, good luck with that. Um I mean, it's gonna increase your budget by an order of magnitude. What could possibly go wrong? Right.

SPEAKER_03Yeah, I can just imagine going ask for an order of magnitude, more budget.

SPEAKER_02Yeah, that million-dollar budget is gonna need to be 10 million next year because AI. Yeah, right, because AI. Awesome.

SPEAKER_03All right, Phil, it's been a pleasure. Thanks for being on the show, folks. Thank you, Rafael. Thank you so much for listening. Coming up next, uh, Wes Widener, we're gonna talk about well, I'll let him introduce it, but it's gonna be a fun conversation. Thanks for listening so far. This is Rafa, I am live at B-Sides Atlanta. If you're not here and you're in the Atlanta area or generally in the Southeast, why not are you here? Uh, this is a great one. Thanks for uh thanks for having me, everybody. And all right, on to the next one. Bill, thanks, folks. Thank you so much. We'll talk to you later. Bye-bye.

SPEAKER_01This is Bella. Thanks for listening. Don't forget to leave my dad a review and share this with your friends. Bye.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. DtSR Episode 732 - Anton Chuvakin: Is AI the New Land War in Asia

    Nov 17, 202651 minAudio

    TL;DR: AI asymmetry is real. While AI is clearly helping attackers find weaknesses across the enterprise attack surface, is it helping defenders just as much? Where is the benefit of AI felt the most in cybersecurity?...

    Credit the people on this episode

  2. DtSR Episode 731 - Anton Chuvakin AI Laughs At Your Firewall Policy

    Nov 10, 202634 minAudio

    TL;DR: AI doesn't care about your firewall policy. We dig into the uncomfortable question many security teams are quietly asking: Are firewalls still relevant in the age of AI, agent swarms, and automation? Guest: Anton...

    Credit the people on this episode

  3. DtSR Episode 730 - Matt Coons Agentic Detection Engineering

    Nov 3, 202633 minAudio

    TL;DR: Stop vibe-coding your detections! This week's pod features Matt Coons at GitLab as we talk about detection engineering, but with AI agents. Guest Matt Coons Description Your detection backlog is endless, your...

    Credit the people on this episode

  4. DtSR Episode 729 - Xavier Ashe 1 A.M. at Sprint: The Night Yahoo Went Down

    Oct 27, 202631 minAudio

    TL;DR: Xavier Ashe was there when that first DDoS attack by MafiaBoy went down. He's got stories, and we're going to tell them. Recorded live at BSides Atlanta in 2026. Guest Xavier Ashe Description: A DDoS attack used...

    Credit the people on this episode

  5. DtSR Episode 728 - Wes Widner - Unlocking Open-Weight Models

    Oct 20, 202633 minAudio

    TL;DR: This week's pod is a conversation about AI and "Open-Weight Models", and if you aren't intimately familiar with what these are this is an episode for you. Wes has a lot of insight, and this is a discussion rather...

    Credit the people on this episode