Damien welcomes Michael Chen, Vice President of Alliances at PolyAI, to chat working with big tech platforms to deploy conversational AI solutions for millions of users. The episode highlights the difficulties in transitioning these solutions from sandbox environments to large-scale production, the inherent challenge of probabilistic vs deterministic software (when it comes to testing), and gives a glimpse into some of the fun stuff we're doing with AWS. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
MichaelIt's really easy for you to test a solution that you've built yourself right to understand how you speak, right? but then when you try and put that into production into how, millions of customers might choose to engage that's a whole different challenge, right? For whatever reason, human mind is limited in our ability to imagine how other people might interact, the language that other people might use. And that is a huge challenge getting conversational solutions out of sandbox.
DamienHello, folks, and welcome to another episode of the Deep Learning with Polly AI podcast, the only podcast that not only delivers breaking news in conversational AI and contact center automation, but it might also send one lucky listener a bouquet of flowers this week. Probably not though. I'm very, very excited to welcome a new guest to the podcast, but an old friend at poly AI vice president of alliances, Michael Chen. Hey, Michael, how are you?
MichaelHey, Damien. Thanks for having me. I've been watching with Envy the last, many, many weeks that you've had this on. And so I'm excited for my shot here.
Damienshoot your shot. That's all I have to say, because there's no guarantee we're bringing you back on. So you better impress not only myself, but our fair viewers and listeners at home. Michael's been with the company a number of years. Michael, why don't you give us a little bit of a breakdown of what you do at Poly.
MichaelWe look at how can we Partner with other organizations that are already, key leaders, key advisors to the customers and enterprises that we work with. And make sure that they're aware of what we do, they're aware of the new innovations that we're bringing to the markets. And we go jointly to market other co sell motions co marketing where part of the conversation around transforming customer experience with language models, with generative AI.
DamienNow, obviously, being in that role you've worked with a number of, external agencies like consultancies, cloud providers, other tech companies to formulate that nice little interconnected network of functionality that just adds value to the end customer. What is the general consensus among those firms with regards to the potential generative AI? I feel like we always talk about, we drink our own Kool Aid, right? We're out to market because we believe in what we're selling. we talk to our end users and obviously if they're interested, they're having deeper conversations as to what we can do or how they can bring generative AI, conversational AI in house to help with automation, with operational excellence, competitive advantage.
DamienWhat are these integrators who, maybe it's a piece of like a bigger hold for them. What are they? What are they saying with regards to the potential there?
MichaelIt's slightly different depending on segments, right? So, I I view my world in terms of segments. So we have technology alliances. So like you mentioned cloud providers, maybe CRM systems, systems of record, right? Where our, our AI will be expected to pull data in from, or send data to, in order to be part of a workflow, right? And it's only by being part of a workflow or it's mostly, I should say, by being part of a workflow that we can deliver a true value business value because, the AI is able to actually update and information so that a human agent doesn't have to do it behind the scenes, right?
MichaelIn a more human style, right? So we have these type of type of technology alliances. We also have like you mentioned, consulting from systems integrators. They're more advisory pieces. Now, these companies know A lot. And they've built incredible businesses. Deeply understanding enterprises really deeply understanding a particular enterprise, a particular sector, the challenges of that sector, right? Where they just have a lot of that expertise to add to us. And we bring the technology around language models on journey to AI. Those partners are layering on like, Hey, this is how you could fit into not just this specific.
MichaelWorkflow for this industry, but into this particular client, because each client, each enterprise does things slightly differently, even if they're operating in similar spaces or in a particular or in the same sector. Right now, in terms of conversations around generative around language models, I had a really interesting stat that I heard the other day, I think it was from quantum black, like one of the sort of expert firms out there, they mentioned that only about 11 percent of.
DamienIs that AI McKinsey?
MichaelYeah. I think, I think that's the analytics arm, right? I think of McKinsey. And they mentioned that they saw only about 11 percent of AI applications within an enterprise made it out of sandbox and into production. And within that subset, they're seeing generative with generative only about three to 5% Of generative applications made it out of the sandbox environment. And
DamienAre those in house developments are those in house projects?
MichaelYeah. Yeah. In house.
DamienWhy is it so low?
MichaelYeah, it's so that that's a big part of the conversation right now. It's amongst businesses amongst IP leaders. How like why, troubleshooting, what's the root cause? How can we improve that percentage? I think in our world, right at least the world that we understand right in customer experience particularly in contact center, what we've seen is it's really easy for you to test a solution that you've built yourself right to understand how you speak, right? But then when you try and put that into production into how, millions of customers might choose to engage You that's a whole different challenge, right?
MichaelLike we for whatever reason, human mind is, is limited in our ability to imagine how other people might interact, the language that other people might use. And that is a huge challenge getting conversational solutions out of sandbox. They might perform well in a closed test environment. But they falter when they're put into production. It's especially difficult in voice, right? You might have a sandbox chat bot that works really well, very clearly you're typing to it, the spellings largely. Okay. The language model has a clear signal of what you're requesting.
MichaelNow you add to that speech recognition, you add to that speech recognition errors, you add to that, a telephony line that might have background noise that might have static. That is. There's a whole order of magnitude in complexity in a production environment that is not as evident in sandbox unless you've had that experience before.
DamienYeah, we're talking about environment. It's tough to do environmental recreation on one end of a voice line when you're fielding like potentially millions of calls. You can't adequately sandbox millions of test environments for all those little nuances of what could negatively affect those various layers of things like natural language understanding and processing. So that makes a ton of sense. And I especially appreciate The difficulties within voice, right? When computing environment at an end user terminal is one thing, but whoever's calling in, whatever's going on behind them and the strength of connection to, to even their own unique voice itself.
DamienIt's really, really difficult to to recreate. So that's interesting. It's pretty fascinating. It's so that's, those are obvious with voice, but why is generative deployments? Relegated. Let's even talk about outside of our sweet spot and voice. Why is, why are other things sitting in the sandbox for so long? What is it? Is it a little bit of maybe like. executive board hype. Oh, we're hearing about generative AI. Let's throw a couple dollars toward it. And then, whoa, this is scary. This thing is spitting out hallucinations. I don't even know what that word means, but I read about it in the Wall Street Journal.
DamienLike what's happening there.
MichaelYeah. Yeah. It's it's definitely a bit of that, it's a there's a difference between using chat, UBT for your own personal, like, Oh, I wonder if it can, I wonder if it can like help me summarize this document or this email. Sure. Versus then going, how do I put this into like a software development pipeline? How do I. Measure whether a change that I've made to my prompt has improved or actually regressed the performance of that solution. Right now it's interesting because in these, I always think of these natural language software is different to traditional software that enterprises might be used to, right?
MichaelSo there's, enterprises have a lot of experience with ERP systems, with CRM implementations, where there's a lot of customization that goes on, right? They go, Hey, I want the dropdown menu to be. Over here rather than over there. I think for my workflow, I need these set of options in the drop down menu versus, ones that come out of the box. But it's all still very deterministic, right? When you build that software, when you customize that software, when I press the button. The first option in the dropdown menu, the software does the same thing every time. It's very predictable. Every single time.
MichaelWhen it's regressed, right? You go, Oh, okay. When I press that button, it's meant to make this API call. It did not make that API call. Therefore there was a regression, right? In performance. And we need to relook at that change before we push it into production. Now, With natural language software, there is just an inherent level of probability now that it is not been evident in previous software, so the deterministic software. Now it's, I asked the model this and it provided me a certain question. I might ask it the same exact question again, and it might provide just a slightly different response, right?
MichaelNow, how do you determine whether, and some of those responses might be subjective, right? Or those answers, and it becomes. Harder and harder to evaluate how to figure out when something has improved or performance is improving. What does continuous improvement look like? And as a result, we're hearing and I was part of discussions recently with software development teams where, yeah, they were racking their brains to go. Hey, I don't. Yeah, we haven't quite figured out how to test for regressions, right? And, and it's a new, it's a new paradigm that is uncomfortable to the enterprise.
MichaelNow they're having to learn a bit of this from, and yeah, we're in the, Again, Mason technology, right? It was early days in technology.
DamienAnd that's the big question is that often, when it comes to these deployments is you're talking about, I'm on testing protocols in the sandbox, but the technology is so new, like even, even X, we don't even have all 100 percent of the testing protocols down to ensure that every deployment is going to be flawless. With this great potential comes great responsibility to Spider Man or tech. And that's what makes it so exciting, but it also makes it terrifying for big business leaders when they're talking about rolling this out at scale. So that makes a ton of sense. when it comes to the integrators and whatnot, like, let's, let's like look at big Seacast providers because we partner with a number of them, right?
DamienSo they have a predictable. Most often they have an entrenched relationship with end user organizations. It's old school tech, very much in that traditional software environment where you can easily test regressions, et cetera. And now they're like, Oh, we're going to shake things up a bit. We know it's going to deliver value to our end user, to our clients. And we know that there's a tremendous amount of business value in this partnership with someone like Poly AI. But they are bringing in this like risky edge tech, what do we, how did they go about or what's the conversation between you and them when it comes to your go to market message as to why these things work well together?
MichaelYeah, I think that's, to your point, right? Like it's scary. It's early days. Everyone's figuring out how to do best practices are emerging, right? All the time. I guess one of the benefits for us is This is all we do, right? So day in day out, we're just deploying these solutions. We have, we actually see the performance and, we're implementing testing and improving testing based on an accumulation of experience rather than just a single sort of point solution. And so that's one of the areas where, Hey, they're going, this is a specialist provider, right? This actually, the challenge of providing.
MichaelHiding more automation over the voice channel given the complexities around language models around journey to AI around speech recognition. I think the market is going. Actually, I want a really specialist partner. I want someone that has not just the people that have done it before, but actually has technology that has been Optimized for this particular use case, right? Because of the complexity, right? And that's one lens and that's one part of our conversations, right? With with a lot of our alliance partners and these consulting firm systems integrators is like, Hey, we are, this is all we do, right?
MichaelThis is where we chosen to focus. We're not going off to AGI, right? Yeah, but leave that to the open eyes of the world to feel the AGI. We're here, we're here for customer experience. We're here for enterprise use cases. And then the other part to your point of a contact center environment is a complex tech stack. There are so many solutions involved in there. There's Workforce management. There is, the telephony systems. I think there, there's an opportunity there to think about what is the role of of AI, right? What is the role of generative now? Is that more an agent assist?
MichaelIs it more in, in automating the conversations themselves? And where a. Best in class provider. We're just a part of that conversations that, Hey, this is what you could do, right? If it makes sense for your business and we're here to help.
DamienSo it's interesting that you say that because obviously, as we take a look at what those either systems integrators or consultancies and implementation partners that come into play aside from the actual solution providers, like let's take a look at where we could be. A function that's offered. i've talked with a number of guests in the past about like how we have a love hate Relationship with bpos in the world, right and I wouldn't say hate that's brutal but it is seen as a bit of a conflicting relationship because for every function that you automate or for every subset of calls that you automate.
DamienThat's less seats that go toward, like, let's say a BP or a shared service contract. So we do have those relationships and they work really well because at the end of the day, it's about driving value to the end user. What, what happens when we're taking a look at the fact that even like figureheads, like The CEO over at Tata consultancy a couple of weeks ago, put out a press release or spoke at a conference and said, the future of our revenues lies in consulting on generative AI projects and basically said our existing model of business process outsourcing is going the way of the Dodo.
DamienLike where does that leave the conversations? How does that make conversations either more or less awkward for you when it comes to those types of partnerships and alliances?
MichaelI actually think there. Isn't the tension right that people might think in those conversations, right? And the reason I say that is like we all operate in and I have a tremendous amount Of empathy for a contact center or a customer experience leader right now, right? It is a difficult job to start right whether you're a bpo or whether you're doing it for an in house call center, right? It is a difficult job, right? It is one that people have a high turnover rate, right? Like 30 40 percent is pretty normal.
MichaelYeah, right And part of it is Because one, consumers are willing to pay a certain price, right? And it'd be great if we were all willing to pay a lot more for our products and therefore the service could increase, but that's not like reality, right? Things get pushed down over time. And contact center leaders are having to do and being asked to do more and more and more with less and less and less. And that affects not just the contact center leader, but I imagine BPR outsources as well. Like they have to also do more, they also have to manage all of this retention.
MichaelAnd one, one of the threats of conversation I hear in the industry is we need to create more sustainable career paths for a customer service or a customer support agent. And I think the way to do that, the starting point of doing that is getting them into more value added troubleshooting, value added tasks. And that means, we joke internally Damien about like, Hey, we just want to reset passwords. That's like the stuff we want to do, right? But that is, that is, but that is part, that is one thread of the challenge in this industry is that is not interesting work, right?
MichaelBeing asked to clock in, clock out every day and help people reset passwords, right? Now, if AI can take that away, we can resolve that. Then, we can help, whether it's an enterprise, like their in house support teams, or whether it's a BPO processor elevate into higher value work creates. More meaningful careers for these roles. I think that's part of the solution because I think that the trends are that there are just fewer and fewer people that want to do customer support jobs.
MichaelBut the complexity of products keeps increasing or the complexity of our world keeps increasing and there's more support inquiries.
DamienThe concept of an entry level agent, I think historically has come with that, for every 10 calls you take, which requires some level of empathy and critical thinking and problem solving, you're getting 90 calls that are reset my password or why isn't this working or where's my order or something like that. And so that's contributed to why that sort of a career path is so like, Non sauna or not sought after. It's just it's pretty mundane and people don't want mundane work. But fortunately, like back in the day, the idea of going to a business processing or to go into a BPO was that you just needed to create some instant cost savings.
DamienAnd sometimes that came at the expense of things like maintaining or being on brand or maintaining an utmost customer experience. And now those tides are shifting a little bit. Contact center is seen as a value center, not a cost center. And so because of that, there's more investments to be made. If the investments benefit the end user, then customer. And so I think that entire dynamic and that paradigm shift, it's definitely contributed to, to be more like, Hey, not only are you answering your calls at like, negotiating per minute costs or how many seats you got, but you got to bring an on brand experience. And that's not something that historically BPS have been known for.
DamienAnd you
Michaelcan be
Damiena bit
Michaelmore creative maybe in that. Space, right? Everyone wants to get creative work, right? That's more high value work. You can, if your training agents and if your contact center agents have more time to spend and actually troubleshoot, get to know the customer more. Guess what? They're actually in a better position to have an advisory conversation and go, Hey, actually, you have this pain point. Have you considered this other product? And that's how I think you start to Turn the experience right into not just the cost center. It's not just something that you have to provide, right?
MichaelBut it is actually a valuable interaction to have with the customer. You get to learn a bit more about them. And you get to have a few more data points about what they care about and what they struggle with.
DamienAmazing. What are some of the more recent, let's do a little self backpack here. What are some of the more recent partnerships or alliances that we've struck up that you think are really exciting and you can't wait to see how it fleshes out over the course of the next six to 12 months?
MichaelYeah, great question. I'll shout out some of our friends at AWS. They're a big sort of one of our big cloud providing partners there. Yeah, I heard about some exciting, yeah, yeah. I think, I think, I think started by, by a little known person named Jeff Bezos, I believe. Yeah, we got some really exciting things cooking this year. We're doing a lot already with their contacts in the platform and it's on connect. And there, there's more to come. Watch this space in that part of the year.
DamienThat's very cool. All right. Cool. Michael, I really appreciate you coming on the show. This was great. I can't wait to have you back. I think you've passed the, you passed the guest test. So you will definitely be getting an invite to come back sooner rather than later. So I appreciate all the insights that you've had. And for those of you listening at home, I will see you next week. Take care of everyone.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...