Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Quality during Design

Quality during Design artwork

Define Kill Criteria to Avoid Zombie Projects

Dec 11, 202513 minAudio
ShareXLinkedInFacebookWhatsAppEmail

When pursuing aggressive benchmarks, engineers must employ portfolio thinking, running multiple design projects simultaneously. But choosing winners requires a decisive way to eliminate projects that are not feasible to continue innovating, often referred to as a "project killer". In this episode, we analyze Tesla's battery development as a case study. We delve into their use of five clear-cut constraint categories that define failure conditions upfront: the Economic filter, Performance filter, Scalability filter, Resource filter, and System filter. We discuss the challenges engineers face in letting go of projects due to the sunk cost fallacy, where prior investments irrationally influence future choices, leading to the creation of "zombie projects". Learn why defining explicit kill criteria before development begins is a vital, often overlooked exercise that saves resources and ensures rational decision-making. Blog for this episode: https://deeneyenterprises.com/qdd/podcast/define-kill-criteria-to-avoid-zombie-projects/ Episode with Dianna's review of Annie Duke's "Quit": Exploring Product Development and AI Through Literature: Insights from 'Loonshots', 'AI 2041', 'Quit', and "How Big Things Get Done' (QDD Book Cast) - Deeney Enterprises Send us a message Work with Dianna I provide independent quality engineering, product-assurance, and technical-risk support when product or process decisions need experienced outside scrutiny. My work includes: Quality-system and product-development method reviews Technical evidence and standards-based rationale Independent risk and readiness reviews Project-based and fractional quality advisory Start with a 20-minute conversation: https://calendly.com/dianna-deeney/discovery-call Interested in being a guest? If you have a practical perspective on quality, reliability, risk, or product development, email Dianna at dianna@qualityduringdesign.com. Pierce the Design Fog My book explains how cross-functional teams can align around customer needs, clarify design inputs, and make better product-development decisions. Read more: https://piercethedesignfog.com The book was an NIEA Finalist and was reviewed by PDMA. BookLife Reviews described it as: “Great for fans of Tony Fadell’s Build, David H. DeWolf and Jessica S. Hall’s The Product Mindset.” Free resources for subscribers Subscribe to the Quality during Design digest for monthly practical highlights, plus access to the Strategic Quality Integration Checklist ...

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00Welcome back to Quality During Design. Today we're looking at Tesla's aggressive battery targets. To manage the risk of pursuing multiple battery chemistries simultaneously, a necessary element of portfolio thinking, Tesla developed a framework to identify winners and eliminate losers. As engineers, the hardest thing we do is kill a project, often falling victim to the sunk cost policy, where prior investments of our resources influence our future choices, risking the creation of a zombie project.

SPEAKER_00Tesla proactively defined failure conditions up front using five constraint categories, including economic, performance, and system filters, to measure project viability. Join me as we explore why defining these explicit kill criteria before development begins is the most valuable and often overlooked exercise in product engineering. Welcome to Quality During Design, the place to use quality thinking to create products others love, for less time, less money, and a lot less headache.

SPEAKER_00I'm your host, Diana Deaney. I'm a senior quality engineer with over 20 years in manufacturing and product development and author of Pierce the Design Fog. I help design engineers apply quality and reliability thinking throughout product development, from early concepts through technical execution. Each episode gives you frameworks and tools you can use. Want a little more? Join the Substack for monthly guides, templates, and QA where I help you apply these to your specific projects. Visit qualityderingdesign.com.

SPEAKER_00Let's dive in. Welcome back. I've been reading up on Tesla's battery development program and looking at it from a case study point of view, starting with their 2020 battery day. All sorts of versions of this are on YouTube that you can go and view it yourself. In 2020, they announced some pretty aggressive Go criteria, like 56% reduction in battery cost per kilowatt hour, and a 54% increase in vehicle range.

SPEAKER_00To meet these kind of targets, they started to self-develop their own battery, the 4680 cell, a custom design, custom manufacturing. They not only had these types of benchmarks that they wanted to meet, but they also had to keep producing and meeting other targets for their existing product line. So they started running multiple battery chemistries simultaneously. And we can all sort of guess why. When you're in development, you're not sure about the sure thing.

SPEAKER_00You don't want to put all your eggs in one basket, so to speak. You don't want to focus all of your resources onto one idea. In case it fails, you have a backup plan. That's the portfolio thinking of this case study, right? You run parallel options, and then throughout you systematically choose the winners. So the company has these targets they want to achieve, and they're doing design and innovation to be able to meet that. And those are a good criteria to have because they are benchmarks, your goal, what you're trying to achieve.

SPEAKER_00And the closer you can get to that, the better. But there's a whole lot of other things going on too. We also need a project kill criteria. If we're developing multiple things at once to be able to reach the same end goal, you need to be able to decide when to stop pursuing some of the other options. Not only where to focus with what's performing the best, but what's not going to be feasible to continue to innovate. Some of the constraint categories that I'm seeing that Tesla used were cost, this is the economic filter.

SPEAKER_00For example, this is the 56% target they announced on their battery day. So if the innovation raises the pack price or fails to hit the cost curve, it's rejected regardless of density. And speaking of density, that's their performance filter. They wanted to significantly improve the volumetric and gravimetric density over existing cells. So if the innovation path that they're developing only offers marginal improvements or compromises safety for density, it's rejected.

SPEAKER_00So those are two constraint categories or kill criteria for this portfolio management, economics and performance. But I'm seeing that they had maybe three other constraint categories. One was scalability, which is related to manufacturability. Whatever they're developing must have been compatible with rapid, high volume production. And if the process required exotic materials, complex wet processing, or it can't be implemented on the dry battery electrode line, they abandoned it.

SPEAKER_00The next to last category was a resource filter related to the supply chain. They wanted to utilize the globally abundant and environmentally stable materials. For example, they wanted to move away from the high cobalt materials. So if their development relies on the scarce, geopolitically sensitive, or environmentally damaging materials, they rejected it. Finally, the last constraint category that I saw was the system filter.

SPEAKER_00They wanted to make sure that whatever they were developing integrated seamlessly into the vehicle structure, the cell to pack or the cell to vehicle. And if the design couldn't function as a structural element, the technology was rejected as it compromises the overall vehicle efficiency and cost. So if you think about this, this is Tesla's battery innovation development kill criteria, economics, manufacturability, performance, resources, and the system filters.

SPEAKER_00And I think they're really well-defined and clear-cut criteria against which to measure whatever it is that they're developing, whatever it is they're managing. So think about your current projects that you're working on. You definitely have Go criteria. Those are the user needs, requirements, and other performance requirements that you're trying to meet. What is your kill criteria? Do you have any? You might have some, but you don't know what it is.

SPEAKER_00How can you find out? And how can you use that to help you make decisions? When we're developing kill criteria, the project kill criteria, we're actually defining the failure conditions up front. Doesn't that make things clearer? You see this in non-engineering and product development communities also, talking a lot about New Year's resolutions right now. Sometimes people will give themselves a cutoff time.

SPEAKER_00You know, I'm going to start this new business. I'm going to give myself I'm going to give myself two years to have this many customers. And if I don't, I'm going to pivot and do something different. The hardest thing to do when you've invested so much time and energy, research and resources into a project is to kill it. You're invested in it and you want to see it succeed. You want to see it through to the end. I think many of us engineers think that way.

SPEAKER_00It may have something to do with how we were trained as engineers to find the problem and then keep digging or evolving or innovating until you find the solution. We might find a solution, but it may not be feasible against all the other criteria because it's not just performance, is it? It's also cost and manufacturability and what the customer wants, what the business needs. I sometimes refer to some of these other aspects of product design and engineering as our internal customers.

SPEAKER_00Yes, we need to meet some outward external goals for our customers or whoever's going to be buying our product, but we also have a responsibility to engineer for many other people and divisions and things. So kill criteria is something that we want to be able to define for our projects. Project manager or some of the people that are deciding what projects are pursued in the business probably have this kill criteria.

SPEAKER_00Maybe you don't know what it is. It might be worth finding out. If there is no kill criteria and you're just going to keep developing until you get it, your project might turn into a zombie project where it never really gets done, it never sees the light of day, and it either just fizzles out or eventually somebody pulls the plug. There can be sunk cost fallacies in our engineering projects, also, where we have an emotional attachment to our design or our project.

SPEAKER_00The sunk cost fallacy is a cognitive bias where we continue investing in a decision based on our prior investments. So we've already put so much time, energy, and effort into whatever we're developing that it doesn't make sense for us to not continue developing it. Whereas really we're eliminating or not factoring in the future benefits or future non-benefits. If we continue investing in this innovation or this development, is it going to pay out in the end, really?

SPEAKER_00Or is it better worth our time and resources to be working on something different? When we're being rational about our decision making, these costs that we've already incurred in our project should not influence our future choices. They're irretrievable. During product development, we may be working under our sunk cost fallacy. It's something we need to be aware of and recognize and pivot our own decision making if we need to when we recognize it.

SPEAKER_00So in addition to the success thresholds of our projects, which we need, we should also define and get alignment on the kill criteria of our projects. And this is an often overlooked exercise. In her book, Quit: The Power to Know When to Walk Away, author Annie Duke shows us the value of knowing when to quit. She offers a lot of examples from different points of view, different industries. She even looks at the sport of boxing.

SPEAKER_00She also gets into some of the psychological and cognitive biases that we have when it comes to quitting. And she promotes defining kill criteria in the beginning, quit criteria. I recommend reading it. It will help you identify when people are ignoring the quit signals or avoid setting criteria in the first place. I reviewed this book in a previous podcast episode. I'll link to it in the show notes. As engineers, we're also used to running tests with acceptance criteria, especially if they're late-stage designed tests.

SPEAKER_00Our product has to perform within these windows or it's a failure. Defining kill criteria for projects is sort of like that. You're defining criteria for your project up front before you get into it. And that'll help you define when to quit so that your project doesn't become a zombie project. Define your kill criteria before you start testing, and for your projects, even before you start developing.

SPEAKER_00In the next episode, I will talk about the expected value calculation that helps end the debate of whether to keep going or not. Join me on Substack for the Ask Me Anything post. This month it's called Choose It: How to Decide Between Viable Options when you'll never have 100% confidence. And then later in the month, I'll be publishing another post, including some of the information that I've discovered about Tesla's battery development and what we can derive from that and how we can use it for ourselves.

SPEAKER_00Find me on Substack at quality during design. And if you're just finding me on this podcast, please subscribe to your favorite podcast provider. This has been a production of Deanny Enterprises. Thanks for listening.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Why We Jump: Fear

    Nov 12, 202613 minAudio

    Fear is one reason teams jump from an idea to a solution before they understand the problem. It does not always look like panic. It can look like adding another inspection, building a feature that can be placed on a...

    Credit the people on this episode

  2. Customer Discovery Across Product Stages, with Ashley Phillips

    Oct 22, 202640 minAudio

    Ashley Phillips joins Dianna to discuss how customer discovery changes across startups, hypergrowth teams, and established products. Customers may surprise a team with new ways to use a product, while the environment...

    Credit the people on this episode

  3. Two Ways We Jump: Speed and Certainty

    Oct 8, 202610 minAudio

    In this solo episode of Quality during Design, Dianna looks at two reasons product development teams jump from a partial understanding of a problem to a committed solution: speed and certainty. Fast prototyping,...

    Credit the people on this episode

  4. When Building Gets Cheap, Decisions Matter More, with Ashley Roach

    Sep 24, 202637 minAudio

    Fast prototyping changes the constraint. When software, AI tools, and other rapid-prototyping methods make it inexpensive to turn an idea into something testable, the challenge is no longer only Can we build it? The...

    Credit the people on this episode

  5. Why Do We Jump? About the Moment Before We Commit

    Sep 10, 20267 minAudio

    When a project goes off track, we usually look back and say, “We didn’t define the problem, and we jumped.” That sounds like a process breakdown, as if one good checklist or gate approval would have prevented it. But...

    Credit the people on this episode