Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

DevOps Academy Podcast

DevOps Academy Podcast artwork

DevOps Academy Deep Dive S1E14: FinOps Meets DevOps — Should Engineers Be Accountable for the Cloud Bill?

Sep 7, 202623 minAudioSeason 1, Episode 14
ShareXLinkedInFacebookWhatsAppEmail

Cloud infrastructure is easier than ever to provision, scale, and automate. But with that flexibility comes a growing challenge: who is responsible for the cost? In this episode of DevOps Academy Deep Dive, we explore where FinOps and DevOps intersect, and why cloud economics is becoming an increasingly important part of modern engineering. The conversation looks at how engineers can balance performance, reliability, and cost without turning FinOps into a simple cost-cutting exercise. We discuss unit economics, Kubernetes cost allocation, Infrastructure as Code, platform engineering, AI workloads, observability spending, and the role of automation in identifying cloud waste. 🔑 Key Takeaways Why cloud cost is becoming an engineering concern The difference between cost reduction and cost efficiency How unit economics can make cloud spending more meaningful Why unused capacity is not always waste How overprovisioning, idle resources, storage, and telemetry increase costs Why Kubernetes can make cost attribution more difficult How Infrastructure as Code can bring cost awareness into code review 🎧 Tune in to explore why the future of DevOps is not just about shipping faster and staying reliable, but also about understanding whether infrastructure spending makes sense for the value it creates. 💬 Should DevOps engineers be responsible for the cloud bill, or should cost remain a finance concern?

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Imagine handing a junior developer a limitless corporate credit card. Oh, that sounds like a recipe for disaster. Right, just dropping them off at a massive hardware store and then acting completely shocked when they come back having built a sprawling 10 -bedroom mansion instead of the little garden shed you thought they were building. Yeah, but you gave them the credit card. Exactly. You gave them the card. You gave them the tools. But you never gave them a blueprint or a budget. And no one checked in on them. Right. And in the modern tech world, that exact scenario

is basically playing out every single day, whether you are prepping for a budget meeting, looking to optimize your own projects, or just curious why cloud bills suddenly look like phone numbers, you are in the right place. It really is the defining financial crisis of the cloud era. I mean we spent a decade optimizing for speed and we succeeded so well that we just completely bypassed financial gravity. Totally. So today we are going deep into a stack of notes and research from the DevOps Academy on this this massive

collision between DevOps the methodology of moving fast and Fine ops the practice of managing cloud costs, which is a super tense collision right now It is and the burning culture we're exploring today is should engineers actually be held accountable for the cloud bill Okay, let's unpack this because the core irony driving this crisis is fascinating to me Cloud computing was supposed to make everything incredibly easy. Right. That was the grand promise. You know, you no longer had to wait six months for a physical server to be shipped to a data center. Oh, man, the old days of literally racking

and cabling servers. Exactly. Waiting for the IT team to plug things in. The cloud meant you had limitless scale just available at your fingertips. You want to provision 50 servers. You can do it in minutes. You want to scale globally. Just takes a few lines of code. But that exact frictionless convenience has accidentally created a monster. Because there is virtually zero friction. into spending money now. Right. So we end up with over -provisioned resources, redundant databases, dev environments left running when nobody is using them? Just absolute staggering cloud bills.

What's fascinating here is how fundamentally the cloud broke the traditional corporate procurement cycle. How so? Like, mechanically? Well, for decades, the standard dynamic was strictly siloed, right? Yeah. Finance controls the money, and engineering builds the product. If engineering needed a new database, They submitted a formal capital expenditure request. A capex form. Right. Finance reviewed it. They approved the budget. And then the purchase was made. A very slow, highly visible paper trail. Super slow. But in the cloud, that separation of duties completely

collapses. You cannot have finance simply hand engineers a massive invoice at the end of a 30 -day billing cycle and say, you know, make this number smaller. Which happens a lot, I'm guessing. All the time. But that approach lacks all necessary context because in a cloud environment, engineers are essentially acting as procurement officers on a daily, sometimes hourly basis. Because every architectural decision they make is fundamentally a purchasing decision. Exactly. I mean, they are deciding which compute instances to use. They're choosing how long to retain system logs.

Oh, log retention is a huge one. Huge. And how frequently to transfer massive amounts of data between global regions, like data transfer alone can cost hundreds of thousands of dollars if the architecture is inefficient. Wow. So I see the disconnect here. Finance only has part of the picture. They look at the ledger, they see the cloud bill jumped by 25 % this month, and they just see red. Right. They panic. But only engineering can tell you the actual why behind that jump. Yeah. And this is why FinOps is emerging not as a new accounting tool or some tedious

spreadsheet exercise, but as a critical operational discipline. It requires engineering. finance, product, leadership to continuously collaborate. So they have to actually talk to each other. Yes. Because a doubling cloud bill is not inherently a disaster. Let's follow that logic for a second. If you are listening to this and looking at a cloud bill that just skyrocketed, your first instinct is probably to freak out. Naturally. But the source material emphasizes that a higher bill could actually mean you were succeeding. Right. You have to think about the underlying

business metrics. If your cloud costs doubled, but your active customer base grew tenfold. Oh, right. And your transaction volume went through the roof, that is a massive celebrated win. You are scaling beautifully. The architecture is actually handling the load. Exactly. However, if your cloud bill doubled and your revenue or user base stayed perfectly flat, Well, then you have a severe architectural problem. So the dollar amount in a vacuum is basically meaningless. Totally meaningless. Fine Ops fundamentally asks,

are we getting appropriate business value from what we are spending? It's entirely about ROI, return on investment, which forces a really interesting philosophical battle. If the goal isn't just to arbitrarily slash the budget, how do you determine what is actually waste? That is the million dollar question, literally. Because one person's waste is another person's safety net. If a Site Reliability Engineer, an SRE, whose entire job is to keep the system from crashing, wants extra capacity sitting idle just in case. Right. As a buffer.

Right. But then Phenops looks at their exact same idle capacity and sees it as just burning cache. Who wins that argument? That tension is arguably the most important dynamic in modern cloud architecture today. Because unused capacity is not automatically waste, right? It is very often the price of resilience. The cost of doing business safely. Exactly. The DevOps Academy notes highlight a brilliant example of this. Imagine a workload that typically uses 60 % of its available computing capacity. Okay, 60%.

From a ruthless utilization perspective, a finance auditor might look at that and conclude that you are wasting 40 percent of the resources you are paying for. I can hear the CFO now, you know, saying, we have a 40 percent inefficiency. Let's cut our provision resources to match the 60 percent we actually use and save a fortune. Right. It seems so logical on a spreadsheet. But I would argue that cutting that buffer is like flying a plane without a parachute just to save a little weight. That's a perfect analogy, because that 40 percent buffer is doing Vital, invisible work.

Mechanically, when a sudden unexpected traffic spike hits your application, say you have a viral marketing moment. Which everyone wants, right. Right. Your system needs to process that sudden load immediately. If you have removed that 40 % buffer, your CPU utilization instantly pegs to 100%. And when CPU hits 100%, the system doesn't just gracefully slow down a little bit. Oh, no. It creates a catastrophic backlog. Incoming requests start queuing up. The system starts dropping

network connections. Sounds like a nightmare. Background jobs fail, and eventually the entire service just crashes. Now the common counter -argument you'll hear is, well, why not just rely on cloud autoscaling? Right, like, let the system automatically add more servers when traffic spikes. That's what the cloud is for. But autoscaling isn't instantaneous, is it? No, it's not. And that is the critical flaw in relying purely on auto -scaling for sudden spikes. Booting up new virtual machines, installing the necessary software images, attaching them to a load balancer, all

of that takes time. Like a few minutes, right? Often several minutes. And during a severe traffic spike, a two -minute delay in adding capacity means two minutes of your application being completely unresponsive to customers. Wow. Two minutes on the internet is an eternity. It really is. That 40 % idle capacity is your shock absorber. It handles the immediate impact, while the auto -scaling mechanisms have time to spin up those additional resources in the background. OK, so if you force an arbitrary financial reduction in resources without deeply understanding the mechanics of the workload, you are basically

actively engineering your own outage. You really are. The cheapest infrastructure is rarely the best infrastructure. Yeah, it almost never is. We've spent decades teaching engineers not to prematurely optimize their code because it leads to fragile systems, right? The exact same principle applies to cloud infrastructure. You need empirical evidence of waste before you make a gut. OK, so since we can't just take a machete to our safety buffers, where is the actual waste hiding?

Because the money is definitely leaking out somewhere. Oh, it's definitely leaking. There are a few usual suspects that plague almost every organization. Idle resources are the most obvious, but they take a lot of different forms. Like what? Well, we're talking about virtual machines that developers spun up for a quick test and then completely forgot about. Old storage disks that are still generating monthly charges despite being detached from any active server. Oh, ghost storage. Yeah. unused load balancers, and massive development environments that are left running 24 -7 including

overnight and throughout the entire weekend when no developers are even logged in. It's like leaving every light, television, and air conditioning unit running in a giant house while you go on vacation for a month. Exactly. But the notes point to a more insidious form of waste. which is observability waste. Oh, that's interesting. Yeah, if we connect this to the bigger picture, observability tools are meant to help teams detect incidents and troubleshoot bugs. But the modern trend is to just collect absolutely everything. The source material specifically calls out high

cardinality metrics. Now, for anyone who isn't deep in the DevOps weeds, my understanding is that a low cardinality metric is something simple, like tracking login success versus login failure. Right, there are only two categories there. But a high cardinality metric is when you attach highly unique identifiers to that data, like login success for user ID 8 ,462 ,911. You've hit on the exact mechanism of the problem. When you start indexing by highly unique user IDs or IP addresses or individual session tokens,

you aren't just creating a few data buckets. You're creating millions. Millions, sometimes billions of unique time series data points. And the software platforms that ingest and store all these metrics, they charge you based on the volume and complexity of the data you send them. Every single data point costs money. It's like installing 500 incredibly expensive 4K security cameras in an office building. You are paying top dollar for the hardware. You're paying for the massive climate controlled server rooms to store the high definition video. But nobody ever

actually sits down and watches the footage. That is exactly what happens. Teams have to critically evaluate their telemetry. They need to ask which data actually supports compliance requirements or active incident response and which data is just extensive noise. Because more data is not automatically better observability. No, it's just a higher bill. And speaking of expensive noise, we have to talk about the absolute blender that is Kubernetes. I look at Kubernetes and it seems like an incredible technical achievement that creates an absolute nightmare for finance.

Oh, it is. Kubernetes is brilliant from an engineering standpoint, honestly. It creates an abstraction layer over your infrastructure. So instead of worrying about individual servers, developers just tell Kubernetes, hey, I need to run this containerized application. And then Kubernetes just handles it. Right. It automatically finds the available CPU and memory across a massive pool of shared servers. But because it's a shared pool, the cost attribution completely disappears. Yes. You end up with dozens of different microservices managed by completely different product teams

all dynamically sharing the exact same underlying cluster of machines. Wow. And they are scaling up and down on the fly, constantly trading resources back and forth. So at the end of the month... The cloud providers billing API just sends finance a single invoice that says this massive cluster of servers cost $150 ,000. Yep. And finance treats the engineering department and asks, okay, who spent this? And an entire room of engineers just stares at each other because nobody actually knows. He literally can't know without foundational

metadata. It's impossible. The cloud providers billing system only sees the raw servers. It has absolutely no idea which specific internal application was running on them. at 2 .03 p .m. on a Tuesday. So how do you solve that mechanically? I assume it comes down to tagging, but how does the metadata actually bridge the gap between that Kubernetes abstraction layer and the cloud provider's billing system? It requires really deep instrumentation. Engineers have to apply strict labels and tags at the namespace or pod

level within Kubernetes. So they're tagging workloads by specific cost centers or teams or product features. Okay, that makes sense. But then you have to use a specialized phenom software that ingests those internal Kubernetes metrics and maps them directly against the cloud provider's hourly billing API. Without that complex mapping, you are flying entirely blind. Which is a terrifying place to be for a business. So if the core problem is that nobody knows who owns what after the money is already spent, how do we fix this before

the code even deploys? Well, the industry calls this shifting cost left. It basically means bringing financial visibility directly into the daily workflow of the developers. And a major way to do this is by injecting cost estimates directly into pull requests. Here's where it gets really interesting. In a typical CI -CD workflow, continuous integration and continuous deployment, an engineer writes code, they submit it as a pull request, and automated tests run to make sure it doesn't break anything before it deploys. Exactly. Standard practice. What you were saying is we add a financial

test to that automated process. A bot reviews the code and says, hey, your estimated monthly cost before this change was $1 ,200, and it's going to jump to $2 ,700 if you deploy this. Yes, it provides immediate Get contextual visibility. When an engineer sees that number in real time, it changes their behavior. They might rethink it. Right. They might realize they accidentally requested far too much memory for a really simple background task, and they can fix it before the code ever merges. But let me push back on this

a bit. The entire point of CI -CD pipelines is velocity, right? The goal is to deploy small iterative changes rapidly, sometimes dozens of times a day. If you start throwing financial roadblocks into that automated pipeline, doesn't it grind the entire engineering culture to a halt? That's the risk, for sure. Like, would you ever actually block a deployment because of that estimated cost? You have to design those guardrails with immense care. Usually, this relies on policy as code. For instance, using a tool like Terraform or Open Policy Agent, you don't

just alert the developer. You literally write a hard -coded rule that evaluates the infrastructure they're trying to spin up. Oh, like writing a script that says if the instance type equals massive expensive GPU and the environment equals development, reject the code. Precisely. You block, obviously, inappropriate behavior in low -stakes environments automatically. But you must ensure Those same rigid rules do not apply to your production environment during a crisis. Right, because if things are breaking, you just need it fixed. Exactly. If an engineer is trying

to deploy a massive database scale up at 3 .0 AM to mitigate a live customer outage, a budget rule cannot block that deployment. Context is everything. So how do you enforce governance on a daily basis without making developers feel like they're constantly navigating this financial minefield? This is where platform engineering becomes your absolute best tool. the most effective governance is entirely invisible. You don't want developers manually calculating infrastructure economics every time they build something. Right, they have enough to worry about. Instead, a centralized

platform team provides pre -approved, t -shirt -sized defaults. Like offering a small, medium, or large deployment template. Exactly. The developer just selects the medium profile for their application. Behind the scenes, the platform team has already optimized that template. They've set the appropriate resource limits, the auto -scaling boundaries, and the cost tracking tags. You make the financially responsible path the absolute easiest path for the developer to take. That shifts the burden entirely. I love that. Now... We cannot talk

about excluding compute costs today without addressing the absolute biggest driver of them in the modern tech landscape. Oh, here we go. The entire industry is racing to integrate artificial intelligence. Yeah. And this raises an important question about the dual nature of AI in this ecosystem, because on one side of the coin, AI is actually being positioned as a powerful FinOps tool. How so? Machine learning models are exceptionally good at analyzing vast amounts of utilization data and finding anomalies that humans miss. An AI

agent could analyze your network logs and automatically alert you, hey, Service A's data transfer cost spiked 18 % immediately after the 4 .0 PM deployment. That's super useful. Or it could safely handle the auto remediation of those weekend development environments we mentioned earlier, right? Like an AI bot notices a cluster hasn't received any developer traffic for six hours on a Friday night, and it just automatically shuts the whole thing down until Monday morning. It can do that, but auto remediation requires severe guardrails. I would imagine so. If you allow an AI agent

to actively modify your infrastructure to save money, the risk profile changes drastically. Give me an example. Say the AI aggressively shuts down a database shard to save ten thousand dollars, but inadvertently causes a hundred thousand dollar customer facing data outage. That optimization was a massive failure. Reliability always supersedes cost reduction. And what about the cost of the AI itself? Because as I look at these notes, AI isn't just a management tool. It is rapidly becoming the core of the phenom's problem. Absolutely.

AI is the new frontier of the cost crisis because it makes it dangerously easy to consume unbelievable amounts of compute power. An engineer can integrate a complex large language model into a basic application with three lines of API code. But the mechanical reality of what happens next is incredibly expensive. Break that down for us. Where is the money actually going when a developer hooks up an AI model? Well, first you have model inference. The AI industry talks a lot about the massive cost of training models, but inference is the compute

required every single time a user hits submit on a prompt. Generating those answers in real time. Right, and that requires incredibly expensive GPU instances running constantly. And the notes also highlight the data storage side of AI, specifically vector databases. Now, my understanding is that these are specialized databases that allow AI to search for concepts by similarity or context, rather than just matching exact keywords. That's correct. But because of how complex that math is, they have to run heavily in active memory,

like in RAM, rather than just sitting on cheap, slow hard drives. And that memory footprint is astoundingly expensive to maintain at scale. And when you combine that with autonomous AI agents that might get stuck in a logic loop, endlessly querying a model and generating millions of tokens per minute, the costs just compound exponentially. Without strict visibility, AI spending scales at a terrifying speed. It is fundamentally altering the baseline cost of running software. So to survive this new world of untraceable

Kubernetes clusters and completely unpredictable AI compute scaling, engineering teams clearly need a fundamental cultural shift in how they measure their own success. We established earlier that just looking at the total cloud bill at the end of the month is a broken strategy. Highly broken. I think of it like judging a car's efficiency. Looking at the total bill is like getting furious at how much money you spent on gas for the entire year. If you drove cross -country three times, your gas bill is going to be massive. Right. Of course it is. The metric you actually need to care about is your miles per gallon. That

is the exact shift to unit economics. Engineers must begin to treat cost as a core system characteristic, exactly like they treat latency, security, or reliability. It's just another metric. Yes. When you build a system, you naturally ask, is it fast enough and is it secure? Now, the engineering culture must shift to automatically ask, is this architecture economically sustainable? But if you walk up to a team of software engineers and say, hey, your code is too expensive, they are

going to get defensive. They are going to feel like you are turning them into accountants. Which is why you must never use fine ops as a mechanism to punish engineers. Handing down a rigid executive mandate, like cut the cloud bill by 15 % this quarter, is a massive anti -pattern. Don't do that. Definitely not. If you force that, Engineers will simply cut their safety buffers, which directly threatens your SLA, your service level agreement, which is the fundamental uptime promise you make to your customers. Right. They will sacrifice resilience just to hit a financial target. Instead,

you target the unit metrics. You measure the cost per transaction, the cost per API request, or the cost per active user. A health goal looks like this. We need to improve our cost per transaction by 15 % this quarter. without degrading our reliability SLA. Because if a single customer transaction costs you 10 cents in cloud compute, but the business only generates two cents of profit on that transaction, your business model is fundamentally broken at the engineering level. Exactly. But by framing it as a unit economics challenge,

you give the engineering team a complex, interesting optimization puzzle to solve. You align the financial reality of the business with engineering excellence. Yeah. Because the ultimate goal of Phoenix is not to make your infrastructure cheap, it is to make it economically efficient. That's a great distinction. Engineers are not accountants, but they absolutely do own the technical decisions that generate the bill. Going forward... engineering economics is no longer in the specialty, it is a permanent foundational aspect of software quality. Which means for you, the listener, whether you

are building the next massive AI platform, managing a complex Kubernetes environment, or just trying to keep a legacy web application running efficiently, it is time to look at your own projects. Are you just reacting to the total bill at the end of the month, or do you actually know your unit economics? Do you know what one single transaction costs your architecture. That's the real test. It really is. And as we wrap up this deep dive, I want to leave you with a thought to explore on your own. We have talked a lot today about how AI will optimize our infrastructure and how

AI itself is driving up costs. But think about the logical conclusion of where this is heading. Because it's wild. What happens when we reach a point where autonomous AI agents are dynamically negotiating their own compute pricing in real time, like bidding across different cloud providers just to execute a single task for the lowest possible cost? A global AI stock market for compute. Exactly. Will future engineers have to design systems that are not just economically sustainable, but are actively trading on a global autonomous compute stock market milliseconds before a function

even runs? It is a wild frontier out there. Keep questioning the why behind the tech you build, make sure you know exactly what you are putting on that limitless corporate credit card, and we will catch you on the next deep dive.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. DevOps Academy S1E22: Platform Engineering 2.0 — Are Internal Developer Platforms Becoming AI Platforms?

    Nov 16, 202628 minAudio

    Internal Developer Platforms were built to simplify software development. But as engineering teams begin building AI-powered applications, are traditional developer platforms ready for what comes next? From integrating...

    Credit the people on this episode

  2. DevOps Academy S1E21: Shadow AI Is Becoming the New Shadow IT — Do You Know What Agents Are Running in Your Organization?

    Nov 9, 202623 minAudio

    Your engineering team may be using more AI tools than you realize. But do you know which ones have access to your code, credentials, and production systems? From AI coding assistants and locally running models to...

    Credit the people on this episode

  3. DevOps Academy S1E20: MCP in Production — Are AI Agents Creating a New Infrastructure Security Boundary?

    Nov 2, 202622 minAudio

    AI agents are moving beyond generating answers. They're connecting to real infrastructure, accessing internal systems, and interacting with production environments. But who controls what they're allowed to do? The Model...

    Credit the people on this episode

  4. DevOps Academy Deep Dive S1E19: AI Workloads Fail Differently — Is Traditional SRE Enough for Production AI?

    Oct 26, 202623 minAudio

    What happens when your AI application is running perfectly, but its answers are wrong? Traditional Site Reliability Engineering focuses on availability, latency, error rates, and infrastructure performance. But...

    Credit the people on this episode

  5. DevOps Academy Deep Dive S1E18: AI Is Making Developers Faster — Is Infrastructure Becoming the New Bottleneck?

    Oct 19, 202623 minAudio

    AI coding assistants are helping developers write code, generate tests, and build applications faster than ever. But is the rest of the software delivery process keeping up? In this episode of DevOps Academy Deep Dive,...

    Credit the people on this episode