Thirty Towns Without Water Controls – Minnesota, Rigged PLCs, and the Bug That Can't Be Patched
22 minAudio
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Over two days in late July, a coordinated attack disrupted water and wastewater operations across more than 30 Minnesota communities - Braham's well and treatment plant shut down entirely, Plymouth disconnected cellular-connected water towers and lift stations, Maple Plain declared a local state of emergency. Drinking water quality was never affected and no boil-water advisories were issued, but the mechanism matters more than the outcome. Four days earlier CISA had expanded its advisory on Iranian-affiliated PLC targeting to cover Schneider Electric and Siemens alongside Rockwell, documented PLC project file theft for the first time, and described an FBI-observed case where malicious Add-On Instructions disabled safety shutdowns and alarms while operator displays were manipulated to show normal conditions. The central Rockwell authentication bypass (CVSS 9.8) was disclosed in February 2021, added to CISA KEV in March 2026 - and Rockwell says it cannot be fully fixed by a software patch. Censys found 5,219 internet-exposed Rockwell/Allen-Bradley hosts globally, disproportionately on cellular carrier networks, which is exactly how a small utility connects a remote water tower. Attribution: suspected, NOT formally attributed - treat nation-state headlines as speculation. Five actions for Australian businesses: inventory your control equipment and its connectivity (especially cellular routers), assume it can't be patched, segment it off the corporate network and off the internet, know and practise your manual fallback, and build an independent way to verify reality that isn't the compromised control system's own screen. Context: the ASD told Australian critical infrastructure operators on 28 July to be ready to isolate systems for three months, and the SOCI Act's eleven sectors capture far more businesses than most owners realise. Visit www.kinsoft.com.au to talk through your security and IT needs. Sources: Tenable Research Special Operations FAQ (28 July 2026, updated 31 July); Minnesota IT Services and municipal statements; StateScoop; SecurityWeek; CISA Advisory AA26-097A (updated 22 July 2026) and the Known Exploited Vulnerabilities catalogue; Censys internet exposure scanning (April 2026); Australian Signals Directorate critical infrastructure guidance (28 July 2026) via iTnews.
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
So I want you to imagine, just picture staring at the main control screen in your facility's nerve center. Right. You're sitting there, you're watching all the telemetry roll in, and everything looks totally flawless. Like the thermal readings are normal, pressure valves are optimal. All the dashboards are green. Exactly, all green. To anyone in that room, it's just a... perfectly quiet tuesday but in reality right outside your control room door the heavy machinery is well it's spinning completely out of control yeah
tearing itself apart right it's pushing dangerously past its physical safety limits and you looking right at your state -of -the -art monitor, have absolutely zero way to tell that any of this is happening. Which is, I mean, it's the ultimate operational blind spot. You are functionally deaf and blind to the physical reality of what your own systems are doing. All while the screen is basically telling you, hey, everything is fine. Right. And that terrifying scenario brings us to the mission of our deep dive today. We are unpacking this really sobering report from
Tech Talks with Kinsoft. Yeah, it's a fascinating document. It really is. It analyzes a coordinated cyber attack on Minnesota water utilities back in late July of 2026. And usually, you know, when a major cyberson hits the news, we're talking about like a database breach. Stolen passwords, credit cards on the dark web, that sort of thing. Exactly. But this scenario is entirely different. This is a story about hacking physics. It's about seizing control of the actual physical machinery that keeps modern society running. It really
demands a total paradigm shift in how we approach network security. Because the Kinsoft report gives us this chilling case study based on what happened over just 48 hours on the 26th and 27th of July. Over just two days. Just two days. And in that window, more than 30 different municipal communities in Minnesota had their water and wastewater operations suddenly disrupted. Wow. And we're talking a wide variety of targets here, right? Like Ram, Maple Plain. Plymouth, South St. Paul. Yeah, a massive spread. But before
we dig into the actual mechanics of how they pulled this off, I think it is crucial to highlight the good news right off the bat for everyone listening. Nobody was poisoned. The drinking water quality itself was never compromised. There were no emergency boil water advisories forced on the public. So it's important to frame this correctly. It was a massive operational disruption, but it wasn't a public health disaster. And that distinction is the key to this entire analysis, really, because of over 30 municipal systems were compromised by an advanced attacker in just
a two day window. How did they avoid a full scale public health disaster? So to understand why this attack is a huge wake up call for basically every physical industry, we first need to look at how these towns managed to survive it. Right. Let's break down the ground reality of that survival, because the disparity in scale here is just wild to me. Like Brahm is a really small town. Right. Only about 1700 people. Very small. Yeah. But then Plymouth is a much larger city of roughly 80 ,000. Yet the attack effectively crippled both of them at the exact same time. Well, the
thing is, the attackers didn't care about the size of the town at all. They didn't. No, because they targeted the underlying technology. Right. And that technology was fundamentally the same in a 1700 person town as it was in a big city. Oh, I see. In both of those environments, the intruders successfully disabled the computerized operating controls. Which, you know, forces the automated systems that pump and treat the water into an immediate shutdown state. Okay, so it sounds akin to, like, someone wirelessly turning off the computerized brain of your car while
you're driving down the highway. That's actually a great way to think about it. But instead it's crashing into the median. The driver just calmly reached down, grabbed a purely mechanical steering wheel, and kept navigating. Because the report notes that a small town like Braham had their systems restored and pumping again in roughly two hours. Right. And I'm just trying to wrap my head around how a local utility recovers from a highly sophisticated cyber attack in just 120 minutes. Like, how is that even possible? The secret to their recovery was just sheer pragmatic
physical action. I mean, the public works and I .T. teams in these towns, they didn't sit in an office trying to counter hack the intruders. Right. Like in a movie. Exactly. They didn't scramble to write a software patch on the fly either. They literally drove out to the equipment at their remote water towers and lift stations. They got in their trucks. Got in their trucks, drove out, and physically disconnected the cellular connected hardware. They just yanked the cord. Just unplugged it. Yeah. And the moment that hardware was offline, they immediately reverted
to their manual fallback procedures. They were out there operating the heavy valves and pumps by hand. So they bypassed the compromised digital layer entirely and went straight to the physical layer. Exactly. And that highlights a vital distinction between IT and OT. Information technology versus operational technology. Right. In traditional IT, like, say, a ransomware attack, your digital data is encrypted. You are basically paralyzed until you buy the decryption key or restore from backups. But OT governs the physical world. It
controls the motors, the centrifuges, the flow valves. The actual moving parts. Right. So when the digital brain of an OT system gets compromised, the ultimate defense mechanism is just human hands taking manual control of the physical object. OK, that makes sense. But if the recovery was really that straightforward, like if it honestly just came down to unplugging a router and. manually turning a valve. Why does the Kinsoft report treat this incident like a five alarm fire? That's the real question. Yeah. Like what made this
attack uniquely terrifying if they could just go analog to fix it? Well, the terror lies in the specific mechanism the attackers use, which, interestingly enough, was foreshadowed by CISA just four days before the Minnesota attacks. That's the U .S. Cybersecurity and Infrastructure Security Agency. Right. On July 22nd, CISA released this urgent update warning critical infrastructure operators that attackers were actively targeting PLCs. OK, let's pause and define that for everyone listening. PLCs, programmable logic controllers. Yes. Think of PLCs as the ruggedized blue -collar
computers of the industrial world. They are the local brains that actually execute physical processes on the floor. So they are the ones opening and closing things. Right. They are programmed to open a valve when a tank is full, start a water pump, monitor pressure inside a heavy pipe. And crucially, their main job is to shut things down when those readings slip out of a safe range. And these things aren't just in water utilities, right? They're everywhere. Oh, they're literally everywhere you look. Automotive manufacturing lines, commercial refrigeration, pharmaceutical
blending, massive pumping stations. So basically, if a physical object is moving automatically in a commercial setting. A PLC is telling it to move. And CISA noted that attackers were targeting PLCs made by major vendors like Rockwell Automation, Schneider Electric, and Siemens. It basically covers the vast majority of the industrial automation market globally. Wow. So this brings us back to that control room scenario we started with. Because the Kinsoft report details a really chilling methodology here. Yeah, it's incredibly precise.
Yeah, right. These attackers didn't just smash through the firewall and break things randomly. They operated with incredible precision. They exfiltrated the PLC project files, which means they downloaded the actual logic program that dictates how the water plant behaves. Right. And then they manipulated something called add -on instructions. And that is the most insidious part of this whole event. See, in operational technology, the visible control program that engineers look at, the thing they check to understand how the system is behaving is called ladder logic.
Ladder logic. It's basically the master blueprint. Now, the attackers left that main blueprint completely intact. So if a local plant engineer audited the core code, it looked perfectly normal and safe. Not meant to see here. Exactly. But the attackers inserted modified add -on instructions. You can think of an add -on instruction like a prepackaged subroutine or basically a black box of code. Okay. So it's sort of like a recipe where the main instructions are totally fine, but someone poisoned the prepackaged sauce packet. And the chef never even thought to check the
ingredients inside the packet. That is a great way to visualize it. By manipulating that specific subroutine, they effectively disabled the system's safety shutdowns. And they silenced the alarm triggers. All while the master blueprint looked untouched. Untouched. But they didn't stop there. Which is wild. They also manipulated the operator display screens. They compromised the HMI, the human machine interface, and the overarching SCADA readouts. Which is the worst case scenario. Yeah. For those listening, SCADA is essentially
the master dashboard that pulls data from all these individual PLCs into one centralized monitoring system. Right. That nerve center. Right. So if I'm the human operator sitting in that room. relying on my SCADA system to tell me the truth. I am being actively gaslit by my own facility. Like the safety interlocks are secretly disabled. The pressure could be building to a catastrophic level. But my screen is feeding me spoof data saying, hey, everything is green. You're kept completely blind to the impending danger. And, you know, that psychological component is incredibly
jarring for a modern business leader because we are fundamentally conditioned to trust our telemetry. We trust our screens. You really do. Yeah. I mean, most corporate leaders have a mental model of cyber threats based entirely on data privacy, right? Protecting intellectual property, preventing wire fraud, stopping data leaks. The classic IT stuff. Exactly. So the concept of an attacker hijacking a screen specifically to hide the fact that a machine is physically tearing itself apart, that is entirely alien to them.
It's this invisible phantom malfunction, which really... Force is a pretty urgent question, I think. How are these attackers gaining access to these critical physical machines in the first place? Yeah. And this brings us to a massive architectural flaw coupled with, frankly, our moderate obsession with convenience. The vulnerability at the center of this, the one detailed in that CISA advisory. is a severe authentication bypass in Rockwell's Logix controllers. Now, in the cybersecurity world, vulnerabilities are scored on a scale out of 10. This specific flaw holds
a severity rating of 9 .8. 9 .8. It is near the absolute maximum level of critical hair -on -fire danger. So a 9 .8 implies the house is literally burning down and the fire extinguishers are locked inside. Pretty much. But according to the report, this vulnerability was originally disclosed in February of 2021. That was five years before the Minnesota attacks. Which brings us to the painful reality of IT versus OT life cycles. I mean, people naturally wonder, why didn't an
engineer just push a patch to fix it five years ago? Right, just update it. But the problem is, Rockwell explicitly stated that this flaw cannot be fully mitigated by a simple software patch. Why can't? No, it is a fundamental architectural problem within the device itself. See, in an IT environment, if a vendor finds a flaw in Windows or whatever, They push out a patch over the weekend, your laptop reboots, and the risk is closed. In OT, it is never that simple. Why is it an architectural problem, though? Is it baked into the hardware? Essentially, yes. It's often related
to how the fundamental firmware handles network authentication at a very, very low level. Updating it might require a complete firmware flash. And that's risky. Oh, incredibly risky. Flashing firmware on a critical industrial device carries the very real risk of bricking it, turning it into a useless piece of silicon. And furthermore, these PLCs are designed with a 20 -year service life in mind. Wow, 20 years? Yeah. You can't just reboot a municipal water pump or halt an automotive assembly line on a random Tuesday
afternoon because a software patch came out. So you have this unpatchable 9 .8 severity flaw just sitting there on machinery. that is deemed too critical to turn off or modify. And that tension right there is what creates the opening for the attackers. It's like spending $10 million to build an impenetrable fortress wall around your facility, right? But then realizing one of the guards standing inside the wall has a personal, unsecured walkie -talkie that connects directly to the outside world. That's exactly it. Anyone can just tune into his frequency and
bypass the wall entirely. So how many of these walkie -talkies are out there in the real world? Well, a group called Senses ran a global scan in April of 2026, and they found over 5 ,200 Internet -exposed hosts that identified themselves as Walkwell or Allen -Bradley devices. 5 ,200. And nearly 75 % of those exposed controllers were located in the United States. So over 5 ,000 unpatchable controllers just sitting on the open Internet. Yeah. But the Kinsoft report notes this really terrifying twist about how
they're exposed. These devices aren't typically exposed through a corporate network error, are they? No, they aren't. They're disproportionately sitting on cellular carrier networks. Why on earth would a utility run critical infrastructure off a 4G cell signal? I mean, it comes down to basic municipal economics. Think about our small town of Broome. If they build a new water tower on the rural edge of town, they need that tower's PLC to communicate with the central control room. Sure. But digging up miles of county roads to trench a secure, hardwired fiber optic cable
that could cost hundreds of thousands of dollars. Ah, so it's a budget thing. Entirely. Alternatively, a contractor can just install a cheap 4G cellular router on the tower for like a few hundred bucks. And it works immediately. Right. It's fast. It's cheap. Establishes a connection right away. But by using a 4G modem, they are entirely circumventing their own security perimeter. They bypass every sophisticated firewall and intrusion detection system the corporate IT department has carefully configured. The moment you power on that 4G router,
the unpatchable industrial controller gets a public IP address from the cellular carrier. Meaning it is directly exposed to the open Internet. Exactly. And in most cases, nobody actively made a malicious decision to expose their infrastructure to hackers. The exposure just came completely free with the convenience of a 4G connection. The convenience creates the blind spot. Yep. Okay, so with thousands of these cellular backdoors scattered across the country, we really have to address who is actively exploiting them. Now,
the report discusses the origins of the Minnesota attacks, and I want us to navigate this very carefully. The Kinsoft report notes that security researchers have aligned the tactics used in Minnesota with Iranian -affiliated activity that was mentioned in that original CISA advisory. Right. And it's an important point to bring up, but it's equally important to stress that the U .S. government has not made any official formal attribution regarding the Minnesota attacks. Right. The investigation is ongoing, and formally, they are still considered unknown actors. And
we are going to maintain strict neutrality here for you listening. We are analyzing the mechanics outlined in the Kinsoff findings. We are not endorsing any geopolitical theories. Because for our purposes, honestly, the who matters significantly less than the how. I totally agree. The geopolitical origin is honestly way less relevant than the global ripple effect of this methodology. Which brings us to why a water utility attack in the American Midwest should matter to a listener working in, say, finance or retail or logistics
halfway across the world. Right. Why should they care? Because the international community understood the severity immediately. The very next day, July 28th, the Australian Signals Directorate, the ASD, issued this unprecedented technical guidance to critical infrastructure operators. They instructed them to prepare for up to three months of disconnected manual operations. Wait, three months? I want you to imagine trying to run a modern business with zero digital controls for a quarter of a year. That is wild. It's an
almost unthinkable scenario for most businesses. And we need to broaden our definition of critical infrastructure here. Because it's not just power plants. Right. Under regulations like Australia's Security of Critical Infrastructure Act, it isn't just power grids and dams. It includes food and grocery supply chains, data storage processing centers, private health care networks, and commercial transport. It essentially encompasses the entire supply chain of modern society. Yes. Think about an ordinary commercial building with an automated management system regulating its HVAC, or a regional
food processing plant, a logistics warehouse using heavy compressors for cold storage, or a large -scale solar array with network -connected inverters. All of those systems rely heavily on PLCs. And very frequently, those PLCs were installed by a third -party vendor or a contractor who just slapped an undocumented cellular modem on it just to make their own maintenance easier. Exactly, so they could monitor it remotely. That hidden modem has likely never once appeared on
the building owner's official IT asset register. Meaning? There's a very high probability that you listening right now have one of these cellular walkie -talkies hidden in your facility and your IT security team is entirely unaware it even exists. It's highly probable. So knowing that this threat is pervasive and that the underlying hardware is fundamentally unpatchable, we can't afford to just be paralyzed by the scope of it. We have to look at solutions. Right. Actionable steps. And the Kinsoff Report outlines a very practical playbook for survival. And I want to weave those steps into how an organization actually
builds resilience. Because first off, you obviously can't protect what you can't see. Which is why the very first action must be a physical audit of your connections. You cannot rely on the network diagrams drawn up by IT five years ago. They are out of date. Because they won't show the random 4G modem the contractor added. Exactly. You have to send someone to physically walk the floor and look for hidden antennas. Facility managers need to interrogate their equipment. They need to ask, does this chiller or this pump
or this solar inverter have its own independent... mobile data connection. And if you find one and you know the equipment attached to it can't be software patched, I guess the only variable you can actually control is who has access to the device. Yeah. So you have to sever that public internet connection. That is the core of network segmentation. Industrial control systems should never, under any circumstances, sit on the same network as your corporate email servers or file shares. Makes sense. And they certainly shouldn't have a direct path to the open internet. If a
vendor absolutely requires remote access, For maintenance, it must be routed through a heavily authenticated, strictly monitored choke point. So no more cheap, direct 4G modems just bypassing the firewall. No more of that. Which leads us to the human element of resilience, the manual fallback procedures. Because Brams survived. Because their public work staff knew exactly how to physically run the plant in an emergency. Right. And this forces a really tough question for business leaders. Has anyone in your organization
actually attempted to run your core physical processes without the aid of a control system any time in the last decade? It's a critical stress test. You really have to write down your manual procedures. And more importantly, you have to physically practice them. And that brings us to... Honestly, my favorite realization from this entire report, because when they discuss how to defend against these phantom malfunctions where the screen is actively lying to the operator, the report states that you simply cannot trust the screen. You really can't. There's a real elegance to the idea that in an era of advanced
artificial intelligence, machine learning, military -grade cryptography, the ultimate firewall against a sophisticated cyber weapon is just an analog pressure gauge. It really is a profound engineering reality because in the realm of OT security, out -of -band sensors, independent telemetry, physical analog gauges, They are not antiquated relics of the past. They're not just old junk. No, they are legitimate, critical security controls. Because if your digital control system is compromised
and that system is the only source of truth telling you the plant is safe, you have zero independent verification. You're completely blind. Right. But a mechanical analog backup gauge provides absolute truth because it is physically unhackable from a keyboard in another country. It is brilliant in its simplicity. A mechanical dial doesn't have an IP address. Exactly. And that reliance on the physical world brings us to the human margin, which I really view as the ultimate takeaway from the Kinsoft analysis. Yeah, the human margin is everything here. Because the only reason the
Minnesota attacks were recorded as a two -hour operational inconvenience rather than a devastating public health emergency was because the human beings on the ground possessed the institutional knowledge required to operate the machinery without a computer guiding them. But the report ends with a very stark warning. regarding that exact human margin because that manual analog knowledge is quietly disappearing from our workforce. It's vanishing. Right. Every time a veteran employee retires, like the engineer who has been walking
the floor for 30 years and remembers how to manually crank the heavy valves, and they're replaced by a newer operator who has only ever known how to click a mouse on a digital SCADA screen, that organization loses its last, most vital line of defense. we are basically witnessing a silent brain drain across industrial sectors we are actively trading robust physical knowledge for digital convenience and it is leaving our critical infrastructure incredibly fragile absolutely when the digital layer inevitably fails or is
compromised the physical knowledge required to catch the fall simply isn't there anymore Sitting in a control room staring at a screen that assures you everything is perfect while the machine tears itself apart is such a terrifying prospect. So I want to leave you with a final lingering thought to ponder long after this deep dive ends. If all the smart systems, the automated screens, and the digital controls in your own workplace went completely dark tomorrow morning, what critical tasks do you no longer know how to perform manually? We're all relying far too much on the screen to tell us the truth.
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Japanese home goods company Iris Ohyama (online store operator Iris Plaza) found a suspicious file in an external...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Australian online retailer Who Gives A Crap told customers on 7 November 2026 that their information was accessed...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Last week in tech, 7 to 14 November 2026. Microsoft's November Patch Tuesday (10 November) fixes well over a...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Chainalysis disclosed on 30 October 2026 that an external actor using "offensive agentic techniques" accessed...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! NSW Health reported unauthorised access to its Oral Health Fee For Service Scheme website on 29 October 2026;...