Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Quality during Design

Quality during Design artwork

Stop Risk Theater, Start Real Decisions

Oct 23, 202514 minAudio

We break down why risk analyses often become checkbox theater and replace them with a simple, practical impact vs likelihood matrix that guides action. From quick wins to high-stakes unknowns, we show how to calibrate effort, buy the right learning, and move with confidence. Join the Substack for monthly guides, templates, and QA where I help you apply these to your specific projects. Send us a message Work with Dianna I provide independent quality engineering, product-assurance, and technical-risk support when product or process decisions need experienced outside scrutiny. My work includes: Quality-system and product-development method reviews Technical evidence and standards-based rationale Independent risk and readiness reviews Project-based and fractional quality advisory Start with a 20-minute conversation: https://calendly.com/dianna-deeney/discovery-call Interested in being a guest? If you have a practical perspective on quality, reliability, risk, or product development, email Dianna at dianna@qualityduringdesign.com. Pierce the Design Fog My book explains how cross-functional teams can align around customer needs, clarify design inputs, and make better product-development decisions. Read more: https://piercethedesignfog.com The book was an NIEA Finalist and was reviewed by PDMA. BookLife Reviews described it as: “Great for fans of Tony Fadell’s Build, David H. DeWolf and Jessica S. Hall’s The Product Mindset.” Free resources for subscribers Subscribe to the Quality during Design digest for monthly practical highlights, plus access to the Strategic Quality Integration Checklist ...

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00Once you know how to do risk-based thinking, it really is a game changer. However, there are some pitfalls. We talked about some of them in the last episode, about picking the wrong tool. When we're applying risk-based thinking and we're reaching for risk analysis tools to help us, if we're focused on the wrong thing, we can get into risk theater. Let's talk more about that after the brief introduction. Welcome to Quality During Design, the place to use quality thinking to create products others love, for less time, less money, and a lot less headache.

SPEAKER_00I'm your host, Diana Deaney. I'm a senior quality engineer with over 20 years in manufacturing and product development and author of Pierce the Design Fog. I help design engineers apply quality and reliability thinking throughout product development, from early concepts through technical execution. Each episode gives you frameworks and tools you can use. Want a little more? Join the Substack for monthly guides, templates, and QA where I help you apply these to your specific projects. Visit qualityderingdesign.com.

SPEAKER_00Let's dive in. I've heard of this complaint quite frequently that people are doing FMEAs and they're a little bit confused about what's going on and how to use it, or they're getting results that don't line up with the decisions that they've already made, and now you have a conflict. And it can feel frustrating because you're doing these risk analyses that are supposed to help you implement risk-based thinking, but all they're really doing is being a checkbox, something that you have to do before you can move on to the next stage of your development.

SPEAKER_00And it's not helping you make decisions or drive changes. It's just something that you need to get done and put on file. And then when you do put it in file, you never really look at it again. That's what I call the risk theater. You're going through the motions, but we're not really using the information to be able to make decisions. In the last episode, we talked about how to know when to reach for what tool. And this episode, I want to talk more about making the risk-based decision. So not necessarily looking for potential risks, but you know what the risk is, or that you know that you have a problem, you have a risk, and you need to assess it to be able to help you make a decision.

SPEAKER_00What I want to introduce you to today is an impact versus likelihood matrix. So it's a two by two matrix. On the x-axis is the impact if we're wrong, if we make the wrong decision. Y-axis is our likelihood or confidence in making this decision. Generally, a high likelihood means that we're confident that our current design or belief is correct. We have some initial evidence or a strong past precedent.

SPEAKER_00A low likelihood means we're not confident in the current belief or design decision. We have conflicting data, no data, or a high degree of technical uncertainty. So you're mapping out how risky is it and how confident am I in the decision that I'm making. And why do this? Because wherever you land on this matrix gives you more of a clue towards next steps. And even before that, just being able to name the problem and the impacts to your project with your team, because they have a different viewpoint and understanding of the problem too.

SPEAKER_00So work with your team and really define the impact that this decision is going to have on your project. And then you're naming out loud, you're putting a pin in how confident you are heading into this decision. Let's take a closer look at the quadrants of this matrix. And you can see how applying risk-based thinking can help us decide what to do next. If this decision that you're looking at is low risk to the project, and you're really confident in the decision that you're making, then those are probably easy wins.

SPEAKER_00Those things you want to delegate and move fast to implement. So say with our product, we have an ancillary cleaning kit that our customers are asking for. Our customers are actually asking for it. We know what special solutions and swipes we need to include in the kit and how to best clean our device. And really, it's a simple ask. So it's a low impact if we get it wrong. And we're 90% confident that customers want this.

SPEAKER_00That's why we may want to consider moving fast on these things. You know, if we get it wrong, it's still a low impact. Um, that doesn't really change. But on the other hand, we're really not that confident that we're going to be designing a cleaning kit in a good way. Maybe it's going to have chintzy things in it, or the wrong stuff, or maybe we're totally missing the mark on what our customers want. In that case, um, that might be a resource sinkhole. We really need to evaluate if we really want to invest time or money to investigate this.

SPEAKER_00It has low impact to the performance of our product, and we're not sure about the decision. So we don't want to over-engineer it. We either want to ship the simplest possible version or de-scope it entirely to eliminate waste. It might be worth doing a few customer touch points on it to learn a little bit more about this cleaning kit, or maybe we don't want to include it at all. This is the most complex quadrant. Since the impact is low, you shouldn't spend a huge resources to increase confidence.

SPEAKER_00Either make the decision simple enough that the uncertainty is trivial, or just live with the uncertainty because the cost of investigation isn't worth the low payoff. With our cleaning kit example, we covered half of the matrix, having to do with a low impact if we're wrong. Usually these decisions aren't giving us a lot of heartburn. They're things that we might spend a little too much time on if we don't understand the risk, which is why it's good to consider them and map them out with our risk-based thinking tools.

SPEAKER_00The things that do give us heartburn are on the other side of the matrix. And they have to do with high impact if we make the wrong decision. Those are the things that we think about and keep keeps us up at night, and that can really paralyze us from moving forward with our project if we are having a hard time making a decision. Let's consider that our product has a user interface to it. And right now we're shipping product that has a keyboard and screen interface.

SPEAKER_00So the user is using a keyboard to type in information into our product for whatever reason. Now we want to modernize it and use a touch screen. We're going to get rid of the separate keyboard and screen and instead have a touch screen user interface. Our uncertainty point in the decision that we're facing is how we get the users to enter in information with this changed interface. Luckily, we have an expert on our team who is an expert on UX and user interface, and we've had a lot of customer input and we've done a lot of research on what works.

SPEAKER_00There are not only academic studies, but there are also studies that we performed ourselves with mock-up interfaces with our customers. This is a big change in the way that we work with our customers. So it is still a high impact if we make the wrong decision, but we're feeling really confident in the decision based on all of the previous work that we've done. So a high impact and high likelihood confidence is a calculated certainty.

SPEAKER_00With these, we want to proceed with implementing our ideas, but we also want to immediately focus on minimizing any of the downsides to it. We want to have a plan and define how we're going to monitor, test it, and build in fail-safes. So really these calculated certainties are the exciting part of product design. The last quadrant in our two by two matrix perhaps is the most interesting.

SPEAKER_00This is the classic high-risk quadrant. There's a high potential for failure combined with a high uncertainty, and all of that demands an investigation to reduce the risk before we proceed. These are the critical unknowns that give us heartburn until late in the night or really make us question whether or not we're able to move forward if we're ready. We want to prioritize data gathering to shift the likelihood to high. The information that we use to assess the impact and likelihood of our project problem is going to guide us into what kind of information that we need and how much of it is going to help us to make a decision.

SPEAKER_00Here's a situation. And it was never the intention to continue to use that in the final design. The intention was to eventually translate that or change the design to be able to use a plastic injection molded part. But it was a big unknown with the performance between the 3D printed part and the injection molded part.

SPEAKER_00The impact to the project in its current state was high. There's timeline concerns because if the mold isn't made just right, then there could be an eight-week delay to revise it. There's cost. Injection mold tooling is really expensive. It could be around$45,000. And they may also need to complete revalidation, which adds more thousands of dollars and more weeks to the project schedule. Being able to state what the unknown is, in this case, will an injection molded part perform as well as our 3D printed prototype in use conditions?

SPEAKER_00That's our problem and the question that we're grappling with. And we spell out the impact if it's wrong. We have timeline, costs, and other revalidation and testing risks if we make the wrong decision. So those are our high impact. Now we want to get more clarity around the likelihood that we'll make the right decision. We get these variables out in the open for our team to be able to explore and discuss. Material properties differ significantly, so we're not quite sure about that.

SPEAKER_00There's no stress testing on the molded version yet because we don't have one in hand. And we know that geometry changes like ribs and draft angles can alter load path when exposed to stress. Because of all these reasons, we're not feeling very good about our design decisions. We have a low confidence that our product is going to perform adequately if we move forward with what we know today. If we find ourselves in this critical unknown quadrant, this exercise is going to give us information against which we can decide what to do next.

SPEAKER_00What kind of testing or modeling or research do we need to increase the likelihood of success in this decision? You can use this matrix in a couple of ways. You can use this in your cubicle or your office by yourself to start gauging how you feel about this design decision. And you can use it to help you identify proposals you make to management, project management or your engineering management on what to do about it or how to make it better.

SPEAKER_00You're essentially explaining the risk of this decision and making recommendations of what you can do about it. Or you could use this matrix with your team, with your project team. If you're the only one that knows about this problem or is concerned about it or it's keeping you up at night, if you're the only one concerned about it, uh that could be a team management problem because your whole team should know about this high impact risk that you're dealing with.

SPEAKER_00And there may be aspects of it that you're not aware of, which would either make the risk less heavy, or there may be additional risks or problems that are related to it or associated with it that you hadn't even thought of yet. So another great reason to use this tool is to talk out these problems with your greater team so that you have a smoother project execution. This impact likelihood matrix we covered today, it's not in Pierce the Design Fog, the book.

SPEAKER_00The book covers front-end risk analysis for concept evaluation. This two by two matrix is for back-end technical execution. Different stage, different tool. If you're working on concept development, grab the book. If you're past that and facing technical validation decisions, this Substack series might be for you. If you're facing these types of decisions where it's later stage in the product development and you need risk-based methods to do it and ways to move forward, then consider joining us on Substack.

SPEAKER_00It'll be a three-month series here in October, November, and December of 2025. On Substack, there will be deep dive posts, QA, and live chats on these topics, so consider subscribing. This has been a production of Deanny Enterprises. Thanks for listening.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Why We Jump: Fear

    Nov 12, 202613 minAudio

    Fear is one reason teams jump from an idea to a solution before they understand the problem. It does not always look like panic. It can look like adding another inspection, building a feature that can be placed on a...

    Credit the people on this episode

  2. Customer Discovery Across Product Stages, with Ashley Phillips

    Oct 22, 202640 minAudio

    Ashley Phillips joins Dianna to discuss how customer discovery changes across startups, hypergrowth teams, and established products. Customers may surprise a team with new ways to use a product, while the environment...

    Credit the people on this episode

  3. Two Ways We Jump: Speed and Certainty

    Oct 8, 202610 minAudio

    In this solo episode of Quality during Design, Dianna looks at two reasons product development teams jump from a partial understanding of a problem to a committed solution: speed and certainty. Fast prototyping,...

    Credit the people on this episode

  4. When Building Gets Cheap, Decisions Matter More, with Ashley Roach

    Sep 24, 202637 minAudio

    Fast prototyping changes the constraint. When software, AI tools, and other rapid-prototyping methods make it inexpensive to turn an idea into something testable, the challenge is no longer only Can we build it? The...

    Credit the people on this episode

  5. Why Do We Jump? About the Moment Before We Commit

    Sep 10, 20267 minAudio

    When a project goes off track, we usually look back and say, “We didn’t define the problem, and we jumped.” That sounds like a process breakdown, as if one good checklist or gate approval would have prevented it. But...

    Credit the people on this episode