Damien welcomes Business Value Analyst Steven Fine to the pod and discusses the evolution of measuring success in contact centers, focusing on integrating AI and automation technologies to improve customer experience and operational efficiency. Their chat emphasizes the importance of marrying sentiment analysis with Customer Satisfaction (CSAT) metrics for a more accurate understanding of customer satisfaction, as well as diving into innovative means of measuring success, such as looking at the entire customer journey and combining bot performance with overall conversation sentiments. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Stevenwe're getting better with AI, being able to recognize if people are happy, sad with their experience. And when you're able to marry that with a C SAT and make sure there's a bindment between the two, then we're going to get a much better picture of how. An individual person feels about their experience and then being able to track that satisfaction over a longer period of time
DamienHey folks, Damian here and welcome to another episode of the Deep Learning with Poly AI podcast. I'm joined by a brand new guest today. The mythical entity that is Stephen Fine. He is a master of all things business value and analysis therein. Stephen, welcome to the show.
StevenThanks so much for having me, Damon, really appreciate it and happy to finally be here.
DamienYeah, seriously. I know you were waiting a long time. We got all of your letters. So thank you for sending those requesting to be on the show. Very important. But no, in, in all seriousness really glad to have you on. You are the eyes and ears of our client base with regards to, driving value from having this really awesome thing that we've been putting together over the past bunch of years. And so I thought, what better topic to discuss with you than taking a look at how contact centers historically measured success?
DamienWhy it was so dry and stale and boring and with the advent of technologies like conversational AI in automation efforts, being powered by those technologies, where you see the future of measuring success, so again, thanks for joining us today. Tell us a little bit about what you do, Polly.
StevenI'm a senior business value analyst. So essentially what's what that title means is I work with clients, pre sales and post sales pre sales. We help with building business cases and making sure we're able to actually quantify the impact that we're likely to have with our clients. And then working post sales on actually making sure that we bring that value to life and we're tracking our performance over time, because fundamentally, our goal here is to drive business value and I want to make sure we can measure that.
DamienIt would be pretty terrible if our goal wasn't to drive business value. I'm not sure how long we'd be in the business, but no that's excellent. Obviously there's a lot of like low hanging fruit when it comes to how you can measure success through automation. It's very it's. A popular concept that goes back to the dawn of technology, right? How do we do things better, faster, cheaper? So why don't we talk a little bit about historical state and those legacy KPIs that existed within the contact center. If you're running a contact center and you're being tasked to be better, what are you measuring for let's just say in like legacy contact center environments?
StevenSo I think if we look at the past of contact centers, It's really focused around like operational efficiency metrics. So I'm going to throw it. It's probably the industry with the most little acronyms because everyone loves an acronym. It's really focused. It's focused around your average handle time. So your HT yeah how quickly can you get people off the phone to be as efficient as possible? How long are they waiting in queues? First contact resolution, a lot of the stuff like first contact resolution is great, but a lot of the stuff is always really focused around that, like operational efficiency.
StevenHow can I have like as little work and as little time with customers as possible? And I think What we've seen as we've moved away from seeing just the contact center as a cost center is we're starting to realize that the time you spend with your customers and be that on the phone channel, which is a key kind of parts of the customer journey is you want to make that time that you spend with them, not just as quick as possible, but you want to make it as meaningful as possible.
StevenAll right. So like first contact resolution does really talk to that. And I think it's critical that we don't lose that metric, but average handle time, it's not, the goal is not always to drive down average handle time. And I think like when you misalign metrics and we're basically trying to get people off the phone as quickly as possible is what and agents being judged on that is. They drop calls quicker. They don't always get to the answer as quick as possible and they pop it off to the next person because then they're incentivized to get people off the phone. I think that's a misalignment of incentives from like just being a cost center, which is operational efficiency.
StevenAs we start moving to this is a revenue and experience center. We want to make sure our customers have the best experience and that doesn't always mean the shortest time.
DamienRight. And it's this double edged sword too, because. The fact that there's no like silver bullet metric that all, contact centers across industries are going for one of the common criticisms. So you have a contact center. It's you're getting stretched for resources. The call times are really long. People are waiting on hold for a very long time. Your call abandonment rate is really high. So you look into the potential of, okay let's look at outsourcing some functionality. When you go to an outsourcer their concern is meeting some SLA is which.
DamienAre basically tied to that call handling rate and the average handling time, right? They're not going to give a great experience if they know that the clock is on as soon as they handle the call, because you basically outsource for possibly cost containment savings. And again, expanding some resources. They're concerned about hitting their SLAs, which doesn't take the customer into consideration and maybe a off brand experience. So there's this constant muddling of what's the most important. It seems like it's a nightmare to prioritize how you even define success from a quantitative standpoint.
StevenYeah, Getting me started on outsourcing is a whole other discussion. It's a whole nother discussion. Cause I know we've seen a ton of clients, they go down that path and It's not always all that it was promised because the quality, you do it for a cost efficiency, but then they spend twice the time on the phone because they're not trained up and they keep attracting so different kind of story around, around that. And I think, one of the challenges, and I, frankly, I don't think anyone's solved this challenge is trying to balance the efficiency metrics and the, performance metrics from a, first contact resolution and CSAT NPS perspective is trying to balance those two kind of sides of the scale, because you can't ignore the efficiency.
StevenLike efficiency matters, but you want to ignore customer experience, right? So, so you to really get a, it's a, it's trying to get a more holistic view of all of these metrics, because each of them, as you push one up, there's probably a different, a negative effect on something else. If you're just trying to get people off the phone as quickly as possible, it drives down your first contact resolution. If your first contact, if you're just shooting for first contact resolution, You're doing everything to make sure your agents stay and they will stay on the phone to make sure it's resolved.
StevenSo that drives up your, you'll drive up your CSAP, but then your agents are on the phone for ages and you're not staffed for that. So it's really a really hard balance of trying to marry the customer experience And the efficiency,
Damienyeah the better, faster, cheaper pick to write, but the other interesting thing is that if the concept were that first call resolution, if that were really like a guiding force for a lot of contact centers out there, then they would have the senior, most experienced agents handling the calls on the front end, but it doesn't work that way. And it's largely due to bandwidth issues, right? So you have someone who's a more junior agents or that tier one frontline. Because they're there to, depending on the phone, the context, let's just say customer support, right?
DamienThey're there to answer those glance or ask questions. They're there to maybe do some real quick updates. Something that could probably found on a website call resolution could be high. But if they get an actual pressing issue that requires escalation and it's gotta go to a few other individuals. If that first call resolution were so prominent, you would think you would have more senior people answering the phone every time it rings.
StevenYeah. The dream is, and I think that this is truthfully where contact centers will go in the future, is, The dream is to have really specialized agents answering the kind of hardest questions, and that can deal with a general group of complex stuff. And all the baseline stuff is dealt with, frankly, with automation. And I think as we were definitely not there yet, but I think as we go into the future and we see the progress with what we're seeing around generative and knowledge bases, what we will see is that like baseline 40, 50 percent of work is that some, a lot of it quite repetitive quite repetitive, not overly complex work that's going to be completely automated away.
StevenAnd we're seeing, in, in some of our clients where you hit those, and I'm just thinking like retail where you're taking away. 40 percent of calls and that's just because those are the simple calls. Those were your tier one agents, but what you're frankly not going to get with automation is you're not going to get those calls that take 15 minutes automated away, ones that like really take human empathy and human empathy and a lot of multitasking across different people and getting approvals from different places.
StevenAnd. And what we're going to take away the first bit, but that complexity, we're going to have higher skilled agents that are going to get paid more. I think they get paid because it's going to, it's going to be actually become a career that like people want to stay in because there's actually, it's, it can become quite meaningful, like helping customers at a high level, so reasonably complex problems. And I think that when that shift happens and we're seeing it happen, I think it's going to be amazing for contact centers as a whole.
DamienDoes that also neutralize even the concept of measuring for first co resolution? Will it be expected? Almost with this new organization structure, knowing that 50 percent of those calls are mundane and are going to be handled appropriately through automation. The others are going to senior agents who are able to, have that have that knowledge set and have the skills to be able to handle almost any other call that an automated agent couldn't handle. Is, does that mean that ultimately it's like uptime for cloud based services, right?
DamienIt's count the nines. You're just going to say every call should be handled on the first time on the first contact.
StevenI, I think it's everyone's dream that the first call gets handled on first contact. And I think like this actually goes back to measurements to measuring the performance of, voice assistants or bots. Like they, and this is where I believe it's going to go is. It's really important that you, that the companies are measuring, the first call resolution within a box. And that a lot of the time, what we've seen in clients that we've gone into, and when they've had, call it these old touch tone IVRs, they get super excited about telling us that their containment rates are incredible.
StevenThey're sitting at 50%. And when we dive in and actually listen to those calls, they're It's containment, but it's not resolution.
DamienLet's go into it. So containment contain, how do you define containment?
StevenSo I think it's a really interesting one. Cause the industry, it would be lovely if we had one way that everyone defines it. But the truth is that everyone picks their own little way of defining it. I'll tell you the way that I see it, I see multiple kind of levels of it that you should work through. Like you do have this level of, call it raw containment, right? This level of containment where it's just, here's the number of calls that come in. And this is the number of calls that don't get through to an agent, right? That's your overall containment.
StevenThen you have your level two, which is assisted containment, where it's a voice assistant or chatbots, whatever channel you're on for this is, it actually answers some questions. And at the end of that the human drops off, right? So reasonable assumption that they've got what they want. Alright, so now that's level two level three then goes, okay, well now we assume that we've answered their question and so we've done something and now how many of them don't call back within a certain timeframe.
StevenAnd the reason that I think it's really important to consider that is if people are just instantly calling back, like fundamentally. You have not likely not resolved their query and they're calling back again. And I think that's the proxy that, that often we try to use to measure resolution, right? Cause it's very hard to just measure it on a mass scale with, the millions of calls that we get in is we use that as a proxy to measure kind of resolution.
DamienSo, tearing up, containment, we got your call, we answered it. Handled, we did a good job on getting those answers, but you called back. Resolved. We answered your questions. You don't need to call back. And that's that's the gold standard.
Steven100%. I think I always say the best customer experience on a call is one where they don't come in, where they don't actually call the best customer experience
Damienis when you leave us alone. Thank you very much.
StevenYeah, exactly. Well, I think it's like a, it's a broad, it brings it back to like broadly a contact center problem is most of the time when people are calling into a contact center. It's because there's something's broken down There's been some sort of break in the chain. Otherwise, they don't call all right unless they need to get something that they just don't know most of the time It's there's the problem that they want to get dealt, have dealt with.
DamienYeah, and the emotionality going into it, it could range, right? It could be, okay where's my package? I need to call and check my package. I don't have the tracking number, so I need to call in. It takes a little while, but you tell me the answer. Okay, great. I'm not calling in furious because I bought, like I bought some sort of, Stroller for my infant and the wheels flew off and I'm furious and I need to understand why this is the case. And if there's a recall, if my child is in jeopardy, then that's a much more severe call, but to your point, like there's, you're not usually calling in just to say, Hey, love your product, love your service.
DamienJust wanted to tell somebody, and so tapping into the emotionality of the moment, does skew for that. Is that historically why? Maybe when it comes to having to have that trade off that you were talking about earlier about efficiency and like the experience, why historically experience was secondary, but consumers, it seems, especially the end user, they're getting to be a little more shrewd and loyalty. Isn't what it once was. So I think Gardner has a statistic that's 62 percent of people will abandon a brand if they have one bad experience with that brand and that bad experience, to your point, could be when they're calling in because they have an issue with something.
DamienSo are we seeing this sort of surge, this sort of surge in the importance of CSAT? Or customer satisfaction for you non in the nose. Yeah.
StevenAnother analogy, another acronym. Yeah, I think this has been like a wave that's been coming for ages. I think the importance of CTAP is. Not going anywhere. I think, there's debates on whether it is the perfect measure for kind of customer satisfaction. It's like one of those things that you want to marry with a couple of other things. And I think this is actually a nice, you spoke about emotion in that. And I think what that leads me to think and talk about is when you're able to, and this is where we're going is when you're able to marry that Kind of static CSATs of, are you happy with the experience that you went through today on a scale of one to five?
StevenWhen you're able to marry that with sentiment analysis, I think that's when you start being able to get a more accurate Picture of actually how that person felt during that. And what we're seeing, around sentiments analysis and being able to understand tone, it's definitely not perfect yet, but we're getting better with AI, being able to recognize if people are happy, sad with their experience. And when you're able to marry that with a C SAT and make sure there's a bindment between the two, then we're going to get a much better picture of how.
StevenAn individual person feels about their experience and then being able to track that satisfaction over a longer period of time through that customer relationship management platforms and CPD. So like data platforms I think when we're able to start really capturing that CSAT and sentiment for that person over time, we'll have a much better picture of actually how people feel in the contact center. A little bit of a rant that I've gone off and totally mischievous. No, but not answered your question at all.
StevenIt's
DamienYou did. And I think you bring up a really interesting point about marrying sentiment analysis to CSAT fidelity, right? Because if I'm on the phone and I'm like if it's a transcription based model, that's taking a look at sentiment based on the words that I use. And I was like, Oh man, that was totally bad ass. It's like both of those words, it may strike as a negative sentiment. It may set off some red flags, but really I was positive. And I think with the advancements of natural language, understanding and language models that are being more savvy in, in artificial intelligence applications, I Like you're able to surmise not only the intent of the call, but actually the actual true sentiment of of the caller, because there's nothing worse than, someone who had a, who had a, maybe a neutral of a positive experience that gets false flagged as negative.
DamienAnd so having those sort of mutual validation points, I think is going to be essential in the future.
Stevenit's so funny. We were talking about this the other day within the company, in the mobility space, and one of the things that I'm based out of London and one of the things that sentiments analysis really struggles with is British sarcasm, like when they end an unhappy call and you ask, so how was your experience the next day? Yeah, that was just great. this is like not exactly,
Damienit was not in pay. Morgan Freeman comes background. It was not in fact great.
StevenNo, exactly. And I think one of the things that. We're definitely focused on, and just a lot of plug is really around and moving it away from just like humans and moving it to the bot world is one of the things that has always been missed in the bot world is really talking about engagements that we did. All the talk is around containment. It's an efficiency. One of the things that we don't talk about enough is engagement. All right. So how are people actually engaging with the voice assistance?
StevenBecause really what that means is they're building trust and. Not have, they're having a decent experience, right? They're getting likely to get what they want done. And that's driven by engagement.
DamienWhen you say engagement, are you talking about essentially would one sign of engagement be they're using more words as opposed to that typical, I don't trust you bot. So I'm going to give everything in one word responses and think that I'm going down a decision tree based on your limited capabilities, right? Billing representative or whatever that's a telltale sign of a lack of trust whereas people who actually have like realistic Dialogue with with chatbots or with virtual assistants or virtual agents that they're, that's an indicator of trust that they think that this automation will be able to assist them.
StevenYes. It's a 100%. And I think it starts at the top when the first, the call first comes in. So your first proxy for engagements will be where after the bot speaks, how many words are, is the person speaking, right? That's your first kind of term of engagements. Then there's this critical points where the person, the customer speaks after they speak. Now it's an opportunity where the voice assistant actually needs to prove that it's able to do its job, right? So you've got that first bit of engagement. Now it's about building trust. And so now they've got to answer the voice assistant needs to answer with the right information to prove that it's able to do this job that the customer came in to get done.
StevenAnd then when they aren't bought answers correctly, that's your first link. Of trust built. So now you're on the trust path, you've got more engagements, and then what you really want is can you have a multi-term conversation, right? Where the, there's a two and fro making sure that they get through to that resolution point and that number of turns, another measure of engagements, and then one that I believe is there.
DamienWhen you say number of turns, that is. That's jargon for a back and forth between the caller and the automated assistant, right?
StevenExactly. Just like when you think about when you have a conversation with an agent is you don't, the customer doesn't just speak and then the agent says one thing and then the conversation's done. Usually there's a multiple back and forth between the agents and customer, and that's what you want to try replicates when correct in the world of voice assistants. And then the one thing I wanted to add is the one thing that I always love, and I think it's basically a measure of. Did the person get what they want done when they, when we have customers at the end of it, say, thank you.
StevenThat's just like my absolute favorite moments when they, at the end of the conversation, they're like, Oh, thank you so much. I'm like, Oh,
Damienwe know we've done our job. It's and it's 10 times better when they've acknowledged during the call that they know they're talking to a bot and they still say, thank you. That, that is the icing on a pretty delicious cake. Speaking of customers and with your role, both Pre and post sale, obviously establishing how they're going to find business value and then how they actually measure it. Have you heard any stories from the street or What you would consider to be pretty innovative approaches to measuring success coming from any customers and let's not name them but just like Maybe a metric that historically hadn't been measured before.
DamienYou seem to be touching on a few things, especially when it comes to sentiment and overall, like trust markers, but are there any other signs of success that people are, our customers are measuring for that you think is a true sign of the times?
StevenSo I think in this example, what I just like to do is separate like performance of bot and performance of contact center. And the reason I say that is. Separate the bot's performance is fundamentally there to serve the whole contact center as a whole, right? It's not they're not isolated. They're not isolated things to look at you have to look at them together So I think really where there's what we're seeing Kind across a couple of our clients the retail and FinServ industries, specifically marrying the conversation with the bot the portion of the conversation with the voice assistant and the portion of the conversation when it's transferred to an agent for call it an escalation in this example, something that's not in scope.
StevenFor the voice assistant, or it's just fundamentally not right. Marrying those conversations together and looking at sentiments on the overall conversation and CSAT on the overall conversation, not just an isolated portion between the two. When you're able to get CSAT at the end of that conversation and marry that with the sentiments that we, across the board. The whole conversation, you get a pretty good indicator of whether there is a overall uplifting customer experience.
DamienAnd it makes total sense too, because at the end of the day, an automated solution a voice assistant, a chat bot is a tool. Right. It's a tool in the entire customer experience and to separate it or to try and demarcate that as something other than a cohesive single stream of a customer experience is a little bit archaic. So that totally makes sense to me that it's looking at the entire experience, right? Because even though these voice assistants may sound really human, like they're not.
DamienIt's a system, it's a technology, it's a tool. And so it should be treated just like any other tool along that customer experience, that support call and married in for one holistic approach. So what I'm hearing from you is that merging of these emotional aspects of satisfaction along with operational efficiencies, there's no separation now between man and machine. It's basically all one thing.
StevenYeah, I think we're definitely moving in that direction. It takes quite a lot of kind of customer education. I bet. But we're definitely moving in the, in, we're definitely moving in that direction where you're going to, you're going to see, contact centers treating their virtual assistants, just like they look at their agents. I, and I think that's really where you'd like to get to, obviously slight difference when the boss is answering, a million calls compared to that a human can answer makes things slightly different.
StevenBut I think what you will see is you start looking at it as an integrated, one whole integrated system that every, they both need to be judged on together. Not as separate entities.
DamienThe bot also will never eat the last bagel in the break room. So that's something you don't have to worry about as well. Steven, I really appreciate your time. This to me is really fascinating because the concept of just measuring success, I love to see how that evolves along with the technology and the capabilities that we have. Within this particular space, but in general and what AI is bringing to to like setting a new standard for measuring success. And it really is more customer centric, which I think is excellent down the road.
DamienAppreciate your time today. Hope to see your lovely face at another episode here really soon. And for those of you who are faithful watchers and listeners, we will talk with you next week. Take care, everyone.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...