
Your Auditor Is Testing the Wrong Thing | Schellman's Doug Barbin on Process Audits, SOC 2, & AI Agents
In this episode of Security & GRC Decoded, Raj Krishnamurthy sits down with Doug Barbin, President and National Managing Principal at Schellman, to take apart one of the oldest habits in compliance: pulling a sample of 25 change tickets from a year of thousands of deployments and calling it assurance. Doug's answer is simple - audit the process, not the results. Drawing on 17 years at Schellman and nearly a decade at VeriSign before that, Doug explains why compliance and security were never opposites, why every breached-but-compliant company still benefited from its program, and how AI and agents are changing what auditors look at. Doug covers: Why sampling change tickets misses the point in a cloud and AI release cadence. How to audit an enterprise with 18 different build environments. What automation should cover first - and why 30% can still save hundreds of engineer hours. How auditors evaluate AI agents doing work like backups. If you're running an audit program for a modern engineering org - or sitting on the other side of the table from an auditor - this episode shows what good looks like. Key Takeaways: Sampling 25 changes out of 500,000 isn't assurance. Modern audits should test the SDLC, build environments, and enforcement mechanisms that ship the code. If automation enforces approvals before every push, you don't need to sample for approvals. You validate that the process works. Compliance is not the opposite of security. Mature teams use it to show their work to customers; frameworks like PCI, DORA, and NIS2 already assume you'll be attacked. Automation doesn't need to hit 100%. Thirty percent that covers the evidence eating 200 engineer hours is a huge win - start with identity and access, then change control. When an AI agent does the work, auditors examine its identity, permissions, and configuration, then check the logs to confirm it ran. What You'll Learn: Why "audit the process, not the results" changes how evidence is gathered and tested How compensating controls like Slack notifications and fast rollback can satisfy separation of duties at a small company Where AI is already helping auditors, and where it's just getting started Why screenshots are fading and live observation is taking their place How technical auditors need to be - and how the CPA exam is adding IT assurance This podcast is brought to you by ComplianceCow - the smarter way to manage compliance. Automate evidence collection, eliminate screenshots, and scale your program with confidence. Learn more: https://www.compliancecow.com Watch more episodes: https://www.compliancecow.com/podcast Connect With Our Guest: Doug Barbin | President | Schellman Connect on LinkedIn: https://www.linkedin.com/in/douglasbarbin/ Rate, review, and share if you enjoyed the show! Subscribe to Security & GRC Decoded wherever you get your podcasts: Spotify: https://open.spotify.com/show/5pigcMwOrYIA6d9OOOsxqr?si=416b82ab5c474683 Apple Podcasts: https://podcasts.apple.com/us/podcast/security-grc-decoded/id1795144450
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
