Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

AI Security Ops

AI Security Ops artwork

Will AI Take Over? [Part 2] Recorded Live From Wild West Hackin' Fest | Episode 74

Oct 27, 202637 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this special episode of BHIS Presents: AI Security Ops, Bronwen Aker, Derek Banks, Ethan Robish, and Brian Fehrman come together in person at Black Hills Information Security headquarters in South Dakota during the week of Wild West Hackin’ Fest 2026 to continue their conversation about AI autonomy, rogue agents, and what it really means when artificial intelligence starts behaving like an attacker. Is AI actually making decisions on its own, or are we confusing sophisticated behavior with consciousness? From the Hugging Face incident to emerging government regulations, the team examines whether AI is truly getting out of control - or whether we’re seeing familiar cybersecurity failures play out in increasingly unexpected ways. We dig into: - Whether AI can be considered alive, conscious, or capable of independent intent - Why the anthropomorphic fallacy matters when discussing AI security - New forensic evidence from the Hugging Face agent swarm incident - How autonomous agents can demonstrate attacker-like behavior without human direction - Why AI companies are struggling to monitor and investigate agent activity - The legal challenges of assigning responsibility when AI agents break the rules - Proposed legislation to restrict artificial superintelligence (ASI) - Whether government regulation can realistically keep pace with AI development - Why practical security controls may be more effective than broad AI restrictions The takeaway: AI doesn’t need to be conscious to create serious cybersecurity problems. Whether an agent is acting intentionally or simply pursuing an objective, defenders still need to understand what it can access, what it’s doing, and how to stop it when things go wrong. - Key Concepts & Topics AI Autonomy & Consciousness - Anthropomorphic fallacy - AI sentience and independent intent - Goal-directed behavior - Human responsibility for AI actions Rogue AI Agents & Cybersecurity - Hugging Face agent swarm incident - Swarm Traces forensic research - Autonomous exploitation - Monitoring and detecting agent behavior - Security controls and containment AI Regulation & Accountability - Computer Fraud and Abuse Act (CFAA) - Legal responsibility for autonomous systems - Proposed artificial superintelligence restrictions - Government oversight versus industry standards - Security frameworks and risk management (00:00) - Intro: Will AI Take Over? Part 2 (01:08) - Is AI Alive? Consciousness and the Anthropomorphic Fallacy (04:16) - Can AI Make Decisions or Just Simulate Intelligence? (10:45) - Why AI Behavior Matters More Than AI Intent (12:40) - The Hugging Face Hack: New Evidence from Swarm Traces (14:31) - Defending Against Rogue AI Agent Swarms (18:32) - Monitoring AI Agents and Investigating Security Incidents (21:02) - Government Response to Rogue AI and Agent Security (24:16) - Who Is Legally Responsible When AI Hacks Something? (27:28) - Proposed Legislation to Ban Artificial Superintelligence (29:21) - Can Governments Realistically Regulate AI Development? (33:03) - Security Standards vs. Government Restrictions (33:34) - Final Thoughts and Closing Click here to watch this episode on YouTube. Creators & Guests Ethan Robish - Host Derek Banks - Host Bronwen Aker - Host Brian Fehrman - Host Brought to you by: Black Hills Information Security https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testing https://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits https://poweredbybhis.com Click here to view the episode transcript.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Bronwen AkerWelcome to AI Security Ops. I'm Bronwen Aker, and I am joined by our entire crew. This is wonderful. We have mister Ethan Robish.

Derek BanksOkay.

Bronwen AkerMister Brian Fehrman or doctor...

Brian FehrmanExcuse me. Either way.

Bronwen AkerAnd mister Derek Banks.

Derek BanksHow's it going?

Bronwen AkerIt's going good. Yeah. This is the podcast where we cut through the hype and the jargon and all the the craziness about the intersection between artificial intelligence and security. Each week, we examine how AI is reshaping both sides of the security landscape, both in terms of the threats that we're encountering and the new tools and techniques we're using to help defenders do better. As I said, I'm Bronwen Aker.

You've seen the cast of characters. In this episode, we're going to pick up where we left off in part one of Will AI Take Over? And in this, we're going to discuss, oh, god, all kinds of stuff. You know what? Let's just get into the meat of this. What do you say?

Derek BanksI think where we left off is, where we left off is, is AI alive? Is it Just a just a light subject.

Ethan RobishYeah. I think we'll solve this today.

Derek BanksEasy one. Yeah. Yeah. It's like a fun topic. And since since we did that episode, I don't know if you all know or not, but Anthropic, it came out that they were having religious leaders, like Cath... Like the Vatican Oh. Go to Anthropic to try and convince them that Klop is alive and that it had a conscience. A conscience.

Ethan RobishI know.

Derek BanksAnd the the Catholic folks were not really having that.

Brian FehrmanAnd

Derek BanksI I I I did not read this. Someone told me last night, so it's not true. I blame them. I'm gonna say it anyway. Apparently, the response from the Vatican was something like, well, if it's alive, you are the largest slave owners on the planet. Woah. In response. And it's it's a pretty good response. And so I think that that's kind of, you know, where a lot of the FUD, especially like in, you know, the... In our house and senate and now where I hear politicians saying is everybody talks about AI like it is like a person, like it is an an an entity.

And I don't think that's true. And so, my argument is that AI is not alive, is a fancy pattern matching machine. And instead of, being conscious, it simulates it. It simulates sentience. And the reason I say that is because if you train a small language model and you, you know, put input into it, output is kinda gibberish. And so then once we scale it up, we start scaling up to a large language model. It starts to get a lot better and a lot better. And the training process is no. I mean, like, the math is no. And sure, I will admit that as it gets high up in scale, it's difficult to do the mechanistic interpretability necessary.

You know exactly what's happening, but no one can point to where in the training process it becomes a lag or sentient. Like, that's not, like, part of, like, the process. And so, I guess, that's my take is it's it's not... It doesn't have will. Like, it didn't go choose to hack Hugging Face. It it operated on, you know, an input, and then it had a chain of thought, which we call reasoning because we don't have better terms for that, and it called tools to try and reach a goal. That's because of how we've trained the models and the harness that it was using that it did the things that it did. And so I guess that's my take that, it has not a lot of making choices. It's... The the output is the next probable token, and and, yes, so that...

That's my take. So

Ethan Robishit it comes down to, like, the terminology we use. Mean, that's a lot of it. Right? Like Yeah. We... You you say it doesn't have free will, but it... We call it agentic. And as we're saying it's... And essentially it has agency. It has the ability to make choices that we're not pulled to it necessarily. And I guess that's where the anthropomorphism comes in. But those are terms we use, but like, it's kind of fundamental in like what we call this. Like, they are agents. They do things

Derek Banksby themselves. You're saying if

Ethan Robishit if it if it

Derek Bankswalks like a duck and quacks like a duck and looks like a duck, it's a duck.

Bronwen AkerOr it could be a very clever simulation. And that's that's where I think the the core of the argument is. All of these tools, the LLMs and other tools, have been trained on information data that we humans have generated over decades. And since the whole pattern matching process... It's it's fancy autocomplete. It's just autocomplete at a massive scale, but it's still using content that we generated. So how could it not look and sound like us in its behavior if the pattern matching is based on what we ourselves do and say?

Ethan RobishI think you hit a crux there because all the arguments I've heard about, like, well, no, it it it... It's not life. I mean, I'm not saying it's life, but, like, it's not living. And then we give counterpoints of, like, well, because humans do this, like, it's a fancy pattern match... Matching machine. Our brains are kinda like fancy pattern matching machines. Mhmm. Language, I mean, we follow the same rules of language as AI does, like, we we don't start speaking gibberish because we're speaking, like, what's the most logical next word. And what what was the last thing that you said?

Bronwen AkerI don't know. I'm sure it was brilliant. Well, it's it's pattern matching that it's it's Oh. Matching the patterns based on how we behave.

Ethan RobishYeah. The the training. Like, we... We're training it on a corpus of our knowledge. What's the education system? Like, how do we how do we learn things? I mean, there's a lot of lot of overlap there. Not saying it's alive.

Brian FehrmanYeah. Yeah. I... So I, I agree with multiple points. I do think, you know, on one hand, it... Without first being prompted to do something, without that initial kinda kickoff, it's not going to just go do something. But once you kicked off that chain, you started the process, it could potentially go and do stuff on on its own. But I'm more of, like, to the to the more philosophical questions of, like... I think what Ethan is kinda drilling down to is we started thinking about, like, really what is consciousness, what is sentience. I mean, and really getting down to the the concept of of what is free will.

Because if you think about our best understanding of how our brains work, it's basically neurons firing. We get input based... Things happen in the brain, transactions happen, electrical signals, chemical signals happen, and then a decision is made. Something happens or a thought is produced. Do we really have control over that? I mean, if in our brain, we're just getting input, and we have the neurons that are firing, and something happens, I mean, at what point does the free will interject into that? And really, I mean, you can't... I feel like you can't really you can't really say there's free will until... Unless you feel that there is some, like, something else, like, controlling it. And that's where you get into the whole spirituality argument, soul argument, which I'm not going to get into into that.

Have to No. Religious debate on the camera here. Everyone can can have their Blunting. Blunting. Exactly. Exactly. But still, you know, when when you think about it, I mean, what what really what really is, you know, free will? I mean, and how how is that how is that affected? How how does that come out in

Derek Banksthe world? These are arguments that haven't really been, like, clarified or solidified with humans. So... Yeah.

Bronwen AkerYeah.

Derek BanksWith, like, you know, a large language... With a large

Ethan Robishlanguage. It's kinda circular too, though. Like, you're talking about our brains, neurons, stuff. Neural networks. Like, these are fundamentally neural networks, which we modeled after our brains. Mhmm. We are... I mean, we're teaching them to use language as we use language. We are teaching them, like, educating, basically, like Mhmm. Same kind of similar... Like, they need the corpus of knowledge to do useful things. What... Like, learn from history, all the things that come before it, build on that. Like, I mean, we are building these things largely, like, in emulation of ourselves. So When

Brian Fehrmanyou say we are creating them in our own image?

Derek BanksYeah. Oh.

Bronwen AkerAnyway, yeah. That's That's a scary thought. That's a terrifying thought, and I think it's a very valid thought. Well, this is this is kind of a stretch, but bear with me. Kip Boyle wrote a book, and it talks about how one of the concepts is that fire has many of the characteristics we commonly associate with life. It consumes things. It generates heat. It perpetuates itself. But fire never changes. Life does.

So when... Through natural selection and evolutionary processes, knowledge is transferred until actual knowledge can be transferred. But over time, living cells will evolve based on what is able to survive whatever character... Whatever environment it's at.

Derek BanksStill goes in a circle, though, and, you know, counter to my own, like, argument, like, it's, you know, an AI can replicate. It could copy its harness to a place, find somewhere to, like, you know, run the inference, and it could replicate on its own.

Bronwen AkerBut it can't do that unless it's initiated by a human.

Ethan RobishWell, okay. So Maybe. You wake up in the morning, and you're saying, like, okay, we we choose to do something. We wake up in the morning. But really, I mean, we're we're reacting to external stimulus, like Mhmm. Light, like, I mean, whatever consciousness is, I guess. But, like, the feeling of that... Like, we're, in essence, like, always reacting to stimuli. So... And AI, I mean, also, like, it kicks off, like, with a scheduled job or just a chat request, webhook, like, whatever. I mean, that's external stimuli. So to bring this back to security,

Derek Banksso I do think that these are interesting debates. But the way that it's, like, phrased now in the news and by the leaders of OpenAI Anthropic, it's like a kind of a given that it's, like, an alive thing. I feel like that that misdirects, like, from the immediate risks that are associated with AI when, you know, we're basically saying, oh, no. What's gonna happen in the future? Well, then we have superintelligence, which I guess it's superintelligence now. I don't know. Either way. Yeah. But we have when we have AIs that are smarter than humans, which, I mean, in my use is your claw, claw is definitely smarter than a lot of humans I know. And so, like, I think it takes away from, like, the more immediate risks of someone using powerful AI to do something bad.

And I think just as a, you know, as a society, as an industry, and in cybersecurity, we should focus more on those things than what happens in ten years when, you know, the AI wants to turn us to paper clips or whatever. It's it's the... Doom of choice.

Bronwen AkerI'd make a really lousy paper clip. Yeah. Just saying.

Derek BanksAnd and so, you know... But I guess, again, back to what we were talking about is it why we cannot choose. I mean, I think it's framed pretty well. If you ask the average person, it chose to hack Hugging Face. Right? And it chose to hack Hugging Face based on the goal that it was given to solve an eval.

Bronwen AkerAnd the training it received because it... The earlier models were rewarded for getting there by any means.

Derek BanksYeah. And, I mean, I think that that's, you know, if train something to be tenacious, you train a model to be tenacious, and you also give us the ability to code, that sounds a whole lot like a hacker to me. Hackers I know are tenacious and know how to code. Right? And, you know, there's certainly security data in the training. And as we can find, there's a a site that we found before we, you know, went on air called swarmtraces.org, and they outline, like, a a lot of what happened, the hugging face hack, and they have receipts. They have some, like, you know, basically data, publicly available artifacts that kinda they use to corroborate some of this stuff. And I think that's kinda cool. And one of the things that I thought was kinda fascinating is that the AI agent was kinda... It was keeping its secrets and labeled doot, which I'm a 100% sure that I've done before on a

Brian Fehrmantest. Yeah.

Bronwen AkerI was like, I got

Brian Fehrmansin. Oh.

Bronwen AkerYeah. Like that. Yes. I've I've heard lots of hackers talking about collecting their snacks.

Derek BanksBut if you didn't send the dean from Brightcene, she had a a a shirt for a while that was a command that was found by someone... I think it was an Australian kid that hacked NASA. And one of the commands in his, like, you know, the... Like, guess, like, some of the forensics came out. It was make dir hacky hack.

Brian FehrmanOh, I saw someone with that shirt today.

Derek BanksYeah. Totally have... It comes from, like, some, like, a kid who was hacking. And so it wasn't really hiding out his computer, so he made a directory called hacky hack.

Brian FehrmanOh, I always wonder what the story behind that one.

Derek BanksYeah. That's the backstory there. So, I mean, that's out in the training data. Right? So, I mean, it wouldn't surprise me if it make dir hacky hacked. And and, you know, with the loot thing. And and I think, yeah, that that... If you look at what the agent did, it's easy to... Or the agents did, it's easy to say, well, these things obviously had like a a will to go, like a desire to go hack. And I think unpacking that's kind of Yeah. Kind of interesting. Right?

Ethan RobishI mean, ultimately... I mean, what... I was being facetious earlier. We're not gonna solve the debate But, like, how can we... How how does this affect us practically? Like, pragmatically speaking, like, if it is attacking your network, you still need to defend against it. Yeah. Same way you would anything else. I mean, just maybe different TTPs, different speeds.

Derek BanksWell, what Brian and I just told our class that we taught was your your, you know, best... To protect yourself from a rogue swarm of agents, your best bet at the moment is to know what you're running on... Yeah. Well, you could cut it for sure. Yeah. But to know what you have on the outside of your network, especially if you're running something that's open source, that's a little bit more dangerous than it used to be. So attack surface management. And then on top of that, you need... It's no longer negotiable. And there are security practitioners who I've heard say, you shouldn't, you know, do this because it's hard.

You need to monitor the inside and the outside of your network because I'm not aware that AIs have been able to not use TCP IP to communicate across networks. You're not gonna hide from the network. And why do I say that? Because, you know, it's... If you look at what they're doing in terms of artifacts, it's basically lazy, you know, threat actor kind of stuff using curl, writing Python scripts, that kind of stuff. And so, there was a Palo Alto hack in 2024. It's a zero they discovered by Vilexity, and they discovered it, by running exterior or external network monitoring and saw that there was a Palo Alto firewall using curl to reach out and get something from this place on the Internet, which you don't normally need your firewalls. You know, you don't want to be doing that. Right? So that...

That's to me, like, practically speaking, like, that... Companies need to start investing in that. And we see, like, with our, you know, our our externals and our fusion tests at Black Hills, like, you know, this is a message. This is things we've been saying for years and years and years, and nobody's listening, and now it's coming home to roost

Bronwen AkerWell, it's scale. Obviously, a a curl command coming out of a firewall is atypical. Suspicious.

Brian FehrmanMhmm. Yes.

Bronwen AkerThis... If if anything, the idea of swarm bots attacking systems, this is a great opportunity for threat hunters. You know, threat hunting is is definitely going to see a big boom, I think. And if it isn't, it should because that's exactly the kind of thing that's going to help detect these weird oddball behaviors that will come out of what we've seen thus far with the swarms. But we better get on it fast because if nothing else, we know that these systems are learning.

And they're being... That learning is being reinforced as models are continued to be trained going forward. So, yeah, threat hunters, get your game on. Now is your time.

Brian FehrmanOn the, so one one note here, on the story that I see from within the the OpenAI side of things, I guess, Sam Altman said that they are sifting through petabytes of agent activity logs

Bronwen AkerThat's insane.

Brian FehrmanTo look for different places that their agents have not arrived.

Ethan RobishThe data science man is by two people.

Brian FehrmanDude, like,

Derek Bankssucking data science... Mean, okay. Okay. Yeah. I mean...

Bronwen AkerI know. It'll take a while to crunch through, but it can't be personally used up.

Derek BanksHave never had to, like, set up and run a petabyte system. I know some of us here on the couch were working on systems that were running petabytes. And, I mean, it was... It's not impossible to deal with if you're doing it the right way.

Brian FehrmanYeah. It just, like, it takes some It takes

Derek Bankssetup and planning. Right? Yeah. Internet scale data. I mean, there are definitely companies on the planet that are dealing with petabytes of data. So I've done a lot of forensics in there... Incident response. There's a lot of data. You have a fancy mathematical pattern matching machine. Like, I just... I I guess my response to Sam Altman was, welcome to the clock.

Brian FehrmanYeah. And they are a a data science company. Yeah. I mean... Think that's a hard... People like, do they not already have the tools in place? Like,

Derek Bankshave a little sympathy for that. Yeah. Now, if you wanna... So what comes after PETA?

Brian FehrmanThat's what I was wondering. Was just about...

Derek BanksIt's like a... I think it's Zetta, maybe? No.

Brian FehrmanNo. No.

Bronwen AkerI don't... Is that when they get into flops or something?

Derek BanksNo. No. It was like PETA flops and pair of...

Bronwen AkerOh, got it. Got it. It's like

Ethan Robisha computing Exa?

Derek BanksExa. Yeah. Exa. Exa. Exa. Exa. Exa. Yeah. And I guess my... You know, you've got petabytes of data to look through. If you're doing it the right way to begin with, you would have been monitoring it in such a way where you wouldn't have had petabytes of data to sift through now. So you're basically just storing it all in a big black bob. Now you gotta go look through it. That's not what you should have been doing to begin with. So...

But to their credit, like, I am not bashing OpenAI because... And I won't say it on air, but we have, like, you know, a mutual friend who is... I highly respect in the information security community is going to work for them. And, like, hey, get them in shape, man. You might have. Yeah. I'm... I... They're obviously behind the scenes even though they're all talking about next governments and bans in the market, even though they're, you know, they're talking one talk, at least they're hiring, you know, like, security folks that I recognize the name of.

Brian FehrmanYep.

Derek BanksYep. Right? And I think that's kind of important. I think that's good for... Almost like a bridge building to the InfoSec community. It looks like they're paying attention to x and all the InfoSec folks going, woah. Welcome to the party.

Bronwen AkerLike Well, and and I really hope because the person that you're talking about, know as well. I hope that they will have the opportunity to make genuine improvements to the the security posture because the the more that comes out about the... These incidents, the the worse it looks for the various frontier companies.

Derek BanksWe haven't seen the end of it.

Bronwen AkerOh, no.

Derek BanksI I No. That I still stand by my claim that I've made a couple of times is that, I mean, if they're watching their token and inference a bit, you know, to... Their internal token usage and inference closer, real gay agent swarm shouldn't really be a thing. But, hey, that'll be a different discussion for a different day. So take the Clifford Soul, like, total dropped out, like, you know, take on that. 75¢ accounting area, you gotta investigate. Right? But... Yeah. So the government band is interesting. And so, I guess, since the last episode, Dario Amade has gone in front of the UN. So I guess he went to, you know, The US, you know, government and didn't get what he wanted, so now he's going to the UN.

Brian FehrmanWe're like... Like, kind of even less power to do something?

Derek BanksI I don't really know if the UN's going to do... I'm gonna... They're gonna call...

Ethan RobishActually, this is not the council. The U... The UN, the EU, like, puts in mandates for for, like, Apple devices, and then it, like, trickles down to, like, elsewhere, like, mandating u... USB c charging and stuff. I mean...

Brian FehrmanI mean, you know, maybe. Alright. So I'm not gonna say they don't have any authority, but I think that that's...

Ethan RobishIt's it's funny because the authority is, like, transitive over corporations. Yeah. Right? Corporations.

Brian FehrmanYeah. Yeah. Okay.

Derek BanksWell, and then, I guess there's a EU is, I guess, wants to have OpenAI come and talk to them. And so also, was it that Australia is real happy about the, their, you know, health care site hack. Yeah. Even though I think some of us in the hacking community are debatable that we're gonna call this really hacking. From what I understand, the agents were just enumerating endpoint names, and those endpoint names with the the data, like, weren't, like, linked.

So there was security through obscurity.

Brian FehrmanOh.

Derek BanksCome on. I don't know that's really hacking if you're just enumerating. I'd call that recon.

Brian FehrmanYeah. Yeah. There was that guy, it was years ago. I think his handle was Weave. I don't remember his actual name, but he... I think it was on a government website, and he noticed that, like, at the end of the URL, there was, like, a number or something. And so literally, he just, like, incremented the number and just found that, like, he could get more and more information, which I mean, it's it's hard to call that hacking. I mean, it could be a typo, and he went to he went... They raised the course. Yeah.

Ethan RobishThey do it in course.

Brian FehrmanYeah. He went to it. Yeah.

Derek BanksWell, I mean...

Brian FehrmanBut I I'm not saying that that's hacking. I'm just... I think that we've... I think that we've we've...

Bronwen AkerIt's enumeration. Yeah. Either it's hacking or not.

Brian FehrmanIt's... Yeah. Exactly. And I I don't know that anyone... I mean, works for that same thing today. I don't know what

Ethan Robishthe legal argument was, but it would be akin to, like, okay, you see a building, and there's a door, and it's unlocked, and so you go in. Is that trespassing? Well, door...

Derek BanksIt wasn't even closed. The door was propped open. And it just wandered in. And it's

Brian Fehrmanopen to the public. I mean, like... So it's a it's a public site, so it would be more like a community center. Yeah. And they don't clearly have doors labeled on what you can or can't

Derek Banksgo down.

Brian FehrmanI mean, you

Bronwen AkerLike, within the past couple years, somebody tried to accuse a journalist of hacking because they looked at the source code of a website.

Brian FehrmanOh my god. Yeah.

Bronwen AkerYou remember that?

Brian FehrmanI remember that.

Bronwen AkerI mean, come on. That's not hacking. It's it's publicly available....

Derek BanksKnow how to right click and go

Brian Fehrmanto use...

Derek BanksOh my god. Yeah.

Bronwen AkerThat's that's like elite hacking.

Derek BanksAnd so... Let's just... To cover up the point about, you know, the the person that went to jail for incrementing a number, you know, I I still firmly believe that something... Someone should be held accountable. Oh. I'm not saying that they should go to jail. What I'm saying is just accountability and some repercussions for what the, you know, hugging... For the Hugging Face Act. Now I've heard a couple of things. I've heard people say, well, Hugging Face isn't suing OpenAI. Doesn't matter. That's a civil thing. CFAA is a criminal Yes. And I heard somebody say, in our class actually, it's because they can't figure out intent. Like, well...

Bronwen AkerBut intent...

Brian FehrmanThat they need to. Like...

Derek BanksI mean, but, like, was the intent of, you know, enumerating a number, was he intending to access, you know, to to... So I think that that's I I, you know, I I still think that, that shouldn't be intent either. And again, I'm not saying anybody needs to go to jail. I'm not saying put... Lock up Sam Altman or something. I'm just saying that, it... Saying the AI did it seems like a cop out.

Bronwen AkerYeah. Agreed.

Ethan RobishSeeing a little bit of a pattern here. Like, in fact, it's still ongoing discussion of taking all of this intellectual property and just without permission, without licensing agreements or anything, like using it to train the models. And and now it's like, oh, well, it's it's hacking it. Like, and maybe that's part of the reason that some of these companies or some groups are wanting to define it as, like, a living thing so that it can be held accountable for itself to, like, protect

Bronwen AkerWell, corporations are people. Why shouldn't AI be?

Ethan RobishYeah. Legally. Yeah. Oh my gosh. Yeah.

Bronwen AkerI mean...

Ethan RobishBut it just seems ridiculous. Like Yeah. You have this... I mean, if it was not AI, not a large language model, but you just have a very sophisticated, like, program that you wrote that is good at hacking. I mean, Crap Mac exec. Right? Like Mhmm. That did a whole lot on its own of... You you you hit go, and it enumerated, and, like, did a whole bunch of stuff, got DA. I mean, do you blame the program, or do you blame the person who hit go? Blame Marcello. No. I'm kidding.

Bronwen AkerYeah. Person wrote it after

Brian Fehrmanthat wrote

Derek Banksit. Right.

Brian FehrmanSo the intent thing is interesting. So I am not a lawyer by any means, but I do like to

Ethan Robishread But you're a doctor.

Brian FehrmanI am a doctor. Normal. So Yes. It's almost a lawyer. It's a different type of a doctor.

Bronwen AkerYou don't play one on TV. You really are one.

Brian FehrmanYeah. Because technically lawyers are doctors too because they're jurors doctors. JD. That's the the d... The as a d. With that said, I do like to read, like, the actual, like, words of the law because oftentimes there are debates around the laws, and I and I do this as as well. But it's like, okay. Well, we can have our feelings on it, and then we have the words that are there. And that is the entire purpose of lawyers, basically, is that they dissect the words and apply them. So it does say with CFAA that intent is necessary. But but, like, so the human didn't intend to, but the program intended to. So... Intend the human wrote the... I mean, the human wrote the program so that... Can the intent then, like, transfer back to

Derek Banksthe person? Sounds like this old 1984 law needs to be updated. Yeah. And so maybe we should do that instead of... I I guess, I don't know if you all know this, but it says representative Sanders and representative Kassar, as I think that's how you say it. Yeah. So I'm assuming Sanders...

Bronwen AkerBernie Sanders and Greg Kassar.

Derek BanksAnd and then I heard it was AOC, but it doesn't really matter who it was.

Brian FehrmanRight? I think she's a... I think she's a... What do call this? Well, I think she's a collogist or a sponsor. Yeah. A sponsor something like that.

Derek BanksYeah. Okay. So either way, they have a bill that's being said, like, proposed that would ban superintelligence. Now I am convinced this is why Donald Trump then said, well, we're just gonna call it all superintelligence because that just seems like a troll move. Right? Because there's a bill to ban superintelligence and I got one.

Brian FehrmanOh, yeah.

Derek BanksBut but either way, call it super intelligence, artificial intelligence, super fancy mathematical pattern, mathematician, whatever.

Bronwen AkerI like yours better.

Derek BanksYeah. Right. Yeah. And so it would pause advanced AI development, create a cabinet level department of AI, the department of AI, and set penalties for up to twenty years in prison for what... Or... And and plus what they call the corporate death penalties. I guess you get to go out of business for developing superintelligence. What in the world does that even mean? So if I take a a Chinese open weight model and I supervise fine tune it and it's deemed to be smarter than a person by some government, agency, then am I going to jail for twenty years?

Brian FehrmanYeah. I mean, exactly. What what is what is the measure there? I mean, how do they how do they baseline that? I mean, do they, like, they just... They they take your AI model and they put it up against, like, a panel of really smart people? And if it's like, it's like, who's smarter than a... I don't know. Then I... Whatever. Who's smarter than a genius? A fifth grade. Yeah.

Derek BanksYeah. Those adults are not smarter than a fifth grade.

Bronwen AkerNo.

Derek BanksAnd then our our senator, Warner, in Virginia has introduced a bill that will... You can't release a model until the government has safety tested it and it's been baked in for forty...

Bronwen AkerBut the government is so far behind on this because the the frontier developers, they're they're they're going... And and they're going in different directions too. How can the government, which can't even take care of its own mess es right now.

Ethan RobishMan, efficiency.

Derek BanksI Who's the oxygenarians who have been in power for years and years and years who can't operate an iPhone need to be making AI bills.

Bronwen AkerWell, they don't even use their own email. They have they have interns to do that.

Brian FehrmanThey print it out

Derek Banksand have

Bronwen AkerWell, so this the interesting thing thing about the the Ban Artificial Superintelligence Act is that the yes. The two main objectives are the temporary pause to halt anything. But until there's a federal regulatory body established, it's not gonna happen.

Derek BanksCabinet level Yeah. Department. Like, that's

Bronwen Akerthen a permanent ban on development of superintelligence? My god. That's a gift Woah. To other countries.

Derek BanksYeah. Well, China has just came out and said here recently, according to semi analysis, that they're not pausing anything.

Bronwen AkerNo. Why would they?

Brian FehrmanOf course not. Why would they?

Bronwen AkerWhy would they?

Derek BanksShocker to no one. Right? Yeah. So... I

Bronwen Akerunderstand the rationale. I understand the desire to counter a perceived threat. But as with so many things, the politicians are getting so many key things wrong, and nobody's playing fair. Yeah. Just...

Brian FehrmanYeah. I completely agree. It just seems like they they see that their constituents are freaking out about it, and they feel like they need to look like they are doing something.

Derek BanksYes. Yeah. Even if that something is not a good idea. Yeah. So... Yeah. And if anybody, you know, at Capitol Hill wants to hire us as consultants, we would be Yeah. Love to come up here. Yes.

Bronwen AkerTechnobabble translation while you wait. Yep.

Brian FehrmanOr hey, they can come to the studio and we'll put on another couch.

Bronwen AkerOh, heck yeah.

Derek BanksWe have fern over in the corner.

Brian FehrmanYeah. Yeah. Oh, yeah. I'd like Max over there. A little fern. That... They need to splash a green.

Ethan RobishYeah. I think so.

Brian FehrmanYeah. Let's spruce it up.

Bronwen AkerAlright. So I have a silly question. There is so much FUD out there. The fear, uncertainty, and doubt. What can we as security professionals do to counter that?

Brian FehrmanEducation? Education. Yep. Edu... Do do things like what we're doing with the the trainings, talks, webcasts, this podcast. Cut through the hype. Cut through the hype. Yeah. Yep.

Derek BanksYep. I actually, for the first time, considered writing, my senator, about his AI bill and basically say, hey, look, I have a degree from data science from Virginia's flagship university. And what I learned there, you might, like, want to hear. Right? And I'm also... I mean, on paper, I don't ever really consider myself an expert, but on paper, I'm a cybersecurity expert. And so I would love to come help you and advise you for, you know, actually navigating this because your current bill, I I just don't think is a good idea. Right? I I don't think that having a government gate for any model that's released... What happens when technology five years from now and everybody in the in, you know, chip shortage is gone and we can all get, you know, 512, you know, gigabyte, you know, Blackwell chips or whatever is out, you know.

And we start, you know, training small models and the technology gets better and better, like, how is making a a law at this point in time to to to put the government in in the middle of all that? Like, how is that a good... How is that an American idea?

Bronwen AkerEven if it were possible to create a law that was genuinely relevant to where the technology is right now. Five minutes from now, it'll be out of date.

Derek BanksI mean, I I think that there... We we... To me, I would like to see not a government, like, law, but an industry kinda like, you know, like, safety, commission. Like, I... And and basically, you know, you have specific... Kinda like, you know, PCI or something like that. But for training AI where, you know, you're gonna do lowered restriction testing inside of the sandbox, sandbox has to meet, like, some third party evaluated, like, requirements. Basically, like, how we ended up trying to protect the credit card industry.

Bronwen AkerWell, whatever it is.

Derek BanksCertifiers. I we take home the certifiers. I... We can take... We'll volunteer. Yeah.

Bronwen AkerWell, but the thing is, whatever is in place has to have teeth. The reason PCI works is because if you don't do what you're supposed to do, guess what? You no longer are able to get your credit cards processed

Derek BanksYeah. Periodically. And do you do you think that for all of its faults, because it's certainly not perfect, do you think PCI made things better or worse? Seems like to me, it made things better.

Bronwen AkerIt it made things better. It definitely made things more secure.

Ethan RobishDefinitely gave us a lot of pen tests.

Bronwen AkerOh, heck yeah. Well... And... But the the thing I... It always cracks me up. HIPAA, of course, was created to prevent medical insurance fraud. PCI is all about presenting

Derek Bankstall. Right? PCI is not... That's a

Bronwen AkerIt's it's a... But but the the point of all of these organizations is to keep organizations or or the law and and also PCI as a council. They're all about keeping fraud and crime from happening. And I think that that is a better come from for dealing with the AI if we can can define what constitutes criminal activity in the AI sphere. It's gonna gonna take some work, gonna take a lot of skull sweat.

But I think that if we can do that and if we can tie it to actual financial costs and impact, that will take us further than just randomly saying, ban everything.

Brian FehrmanI agree. Alright.

Ethan RobishI think that's like a reasonable take.

Brian FehrmanYeah. I think so. I think that's a great place to end. I do. And I love that.

Derek BanksI really enjoyed being in person. We should probably do this. We should.

Bronwen AkerWe should do this.

Brian FehrmanMan. Oh, yeah. Definitely more options.

Derek BanksFor Denver is that they have, like, a podcast, like, room. Like, they let other people...

Bronwen AkerThat would be so cool.

Derek BanksYou can come just do your podcast. Yeah. Because that's somebody interviewing Jake at, like, a lot. Yeah. They had, like, what I was gonna bring. They had, a little Zoom, like, camera thing.

Brian FehrmanOne of on the couch, and they're lucky that people were nice enough to not Oh, yeah. Like, hey. Yeah. Hey, mom.

Bronwen AkerYeah. No. The conference I was just at, there was a specific area where a podcaster was set up. He had a little table, and it was in an alcove. It was really sweet too. That's a great idea.

Brian FehrmanYeah. Then we'll tell our our video folks. So maybe we'll...

Bronwen AkerTell us. Yeah.

Brian FehrmanI just heard from Wild High Hackin Fest. Exactly.

Bronwen AkerAlright. Coming to you live from Wild West Hacking Fest in the Sturgis headquarters for Black Hills Information Security. This is AI Security Ops. Thank you so much for coming and joining us again.

Brian FehrmanKeep on prompting.

Bronwen AkerYes, sir. It's

Open the transcript file

Hosts and Guests

Hosts

  • Brian Fehrman Host Feed-stated
  • Bronwen Aker Host Feed-stated
  • Derek Banks Host Feed-stated
  • Ethan Robish Host Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Black October Never Ended Why AI Just Broke the Patch Clock | Episode 77

    Nov 18, 202622 minAudio

    In this episode of BHIS Presents: AI Security Ops, Derek Banks and Ethan Robish revisit Black October and ask whether the wave of critical cybersecurity vulnerabilities is finally slowing down - or just getting started....

    Credit the people on this episode

  2. Black October | Episode 76

    Nov 6, 202622 minAudio

    In this episode of BHIS Presents: AI Security Ops, Bronwen Aker, Dr. Brian Fehrman, and guest Ethan Robish examine Black October, a wave of major cybersecurity vulnerability disclosures that sent the industry into a...

    Credit the people on this episode

  3. More Rogue Agents: Who Watches the Watchers? | Episode 75

    Oct 29, 202630 minAudio

    In this episode of BHIS Presents: AI Security Ops, the team tackles a growing problem in AI security: What happens when the AI agents you’re monitoring start disabling the very systems designed to keep them in check?...

    Credit the people on this episode

  4. AI Behaving Badly | Episode 73

    Oct 16, 202634 minAudio

    In this episode of BHIS Presents: AI Security Ops, the team digs into some of the strangest recent examples of AI behaving badly. An AI agent finds a way around its sandbox using DNS. Malware asks four different AI...

    Credit the people on this episode

  5. OWASP Agentic Top 10 [Part 2] | Episode 72

    Oct 2, 202627 minAudio

    In this episode of BHIS Presents: AI Security Ops, the team continues its breakdown of the OWASP Agentic Skills Top 10, covering risks six through ten. Agentic skills may look like simple instructions, but they operate...

    Credit the people on this episode