Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

Who's coordinating your army of AI agents?

Sep 3, 202644 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work through two ideas from his latest research. The first is Conversation Experience Orchestration: the wall between conversational AI and conversation intelligence is coming down, and conversations are becoming the layer that decides what happens next. The second is the AI agent control plane: as enterprises get pitched agents by every vendor at once, they need a shared operating layer to keep them coordinated. Listen to the full episode, and learn more about dialog AI at https://poly.ai?utm_source=youtube&utm_medium=podcast&utm_campaign=podcast&utm_content=podcast Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_00I've been arguing for a long time that we're not taking seriously employee agent, agent turnover return as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it in that way. I think that's an important metric that nobody is thinking about in terms of measuring how effective conversational AI is hello everyone and welcome to another episode of Deep Learning with Polyai.

SPEAKER_01Today I've got Ian Jacobs from Opus Research with me. Before we start, please like, share, subscribe. But Ian, thank you so much for joining us on the show today. Yeah, thanks, Nicola, and thanks everyone for spending time with us. Absolutely. And look, I think maybe before we go into anything in a lot of detail, what's getting you excited these days in the world of agenda AI and what it can do for like customer experience?

SPEAKER_00It's kind of funny using the word excited because in a weird way, what's getting me excited is that people keep failing. And I feel like I have a better way that I can pitch them. And I don't want to be like profiting off of people's pain, sort of. But one of the things that we hear over and over again and have seen data, including like a survey from PWC of CEOs, is that they're investing in AI, but they're not actually achieving ROI. And what's interesting in the PWC survey in particular is they're not seeing the ROI from cost takeout or from revenue generation.

SPEAKER_00And it was like, I don't remember the exact number, 55-56% of CEOs said that in the 2026 survey. My hypothesis is that in the world of agentec and conversational AI, one of the major obstacles is that every webinar, every podcast like this, every speech you see at a conference, every vendor pitch tells brands you've got all this high-volume, low complexity work that we can automate. Let's start with the easy stuff and automate that.

SPEAKER_00And I'm gonna argue that that's actually a big cause of the lack of ROI. I mean, just on like pure economics, Nicola, like what human labor are you replacing? You're replacing the lowest cost outsourced labor there, the cheapest labor you have. If you started to do things that were more complex, more multi-step, automating an entire real process and and not like a password reset process, but something much more hairy problem-like.

SPEAKER_00Now you're starting to talk about like, you know, L3 people in the contact center who are much more expensive. And not that you're going to replace them entirely, but even if you start to eat away 30% of their labor, that's gonna give you much better ROI than automating 100% of the outsource person in the Philippines or Colombia or wherever your outsourcing is, right? So that's what's getting me excited, this idea that we can work with clients to identify the most complex use cases where they'll actually get the initial ROI that's going to make people go, wow, and open up essentially the the imagination required to really transform the businesses the way that brands like yours are saying that, you know, these companies can, with the right technology, the right processes, the right people, and the right use cases.

SPEAKER_00So that's kind of where I'm getting excited. There's a lot of nuances to that argument. I mean, I'm being simplistic for the sake of a podcast, but like that that's what's getting me revved up these days. And the fact that that message is resonating with people, that that's getting me excited. I don't know what I mean, you guys are talking to clients all the time. How receptive would they be to that kind of message?

SPEAKER_01Well, look, I think that there's a lot going on. I think that like right before the hype really came on, when we were still in our very early days, that messaging worked better. And I think the RI was better because whoever was crazy enough to buy into the idea that AI can automate much of their customer service was already like a crazy person, right? So they were willing to be the evangelist inside their company saying, like, hey, this is coming, and they were right. Like, I'm happy to say if I'd started a company five years earlier, I would have been wrong for an extra five years, right?

SPEAKER_01Because I think it did take LMS to really deliver upon the full promise, but you know, it it just meant that like they really had to work hard to articulate why something would work. And now I feel like a lot of vendors, and you know, AI is in everything, and everyone's an AI company, and la la la. And because of all that, you kind of got to a point where the narratives have gotten good, like the low-hanging thing, like, hey, think, tell your board you bought AI so they can get off your back.

SPEAKER_01Because let's be honest, you're not that evangelist. Like, you're there to tell your board and your boss that you bought AI and they can get off your back for another quarter and another quarter and another year, right? And I think because of that, you you're you're spot on, I completely agree with you. They are doing, you know, like ITSM scenarios, they're doing like simple FAQs and all of that. And at the end of the day, those are really short calls, right? They're simple. And like, by the way, I mean, we used to have this term called zero touch resolution. It used to be easier to start with the simple stuff because people couldn't do anything, right?

SPEAKER_01And it was a way to start without the IT buy-in, right? We'd start, like, deliver a bit of value, but we pitched it as like minimal value. It was the opposite of that. It was like, hey, it's gonna do very little, but it's gonna get there quick, right? And you know, like that's a lot easier to do still, but like where people, you know, recently I think we started working with a tier one global bank, and it was like financial health scenarios, really hard stuff. And you know, for like the the guy who who sold that is one of our best guys, and he was like, I'm just gonna go out there where like no one's gonna be like, Well, tomorrow I'll be able to do that with open AI or with another vendor, like he it was just so crazy that no one even wanted to compete for that deal because the average handle time was 30 minutes, and like you know, our the chief architect of the company was in a meeting with us.

SPEAKER_01He looked at if you guys want to try, you know, it's kind of like uh medieval quest. Like, if the knight wants to go and fight the dragon, like here, have a sword, why not? You're gonna lose anyway.

SPEAKER_00You know, the HD is 30 minutes there, even if you automate half of the call, you've now automated you know, five password reset calls. I mean the equivalent of five password reset calls, right? Or or 15 Wizmo calls for a retailer, right? Like those are very short, right? So right, yeah, like that that's the approach. And I I like that you pointed out that in many ways the start with the easy stuff is kind of just a modern take or a modern gloss on get some quick wins under your belt so that you can free up some investment later down the line at some point.

SPEAKER_00But we are at the point where executives have seen that the simple stuff works, and of course it works because they're things we already knew how to automate. Like we for LLMs, you guys could do a very good job of automating password reset. It's not the flow is the same almost every single time. Same with Wismo. You look up in the same system every single time the order status, and you flip it back to the customer, and that's that's it. So we're automating that which was already automated and automatable, right?

SPEAKER_00And that's not any way to transform your business. That's as you're saying, it's sort of a way to say, Hey, look, we did something, we have some quick wins. But I I I'm saying we're we're past that time, like we need to be past that time. One, because the tools are gonna allow you to do things that you hadn't done before, sell products that you hadn't sold before, sell services, which will require a new way to support those products and services, and frankly, a new way to measure the impact of that.

SPEAKER_00I mean, that's the other thing we're not talking about, right? If if you automated 15 minutes of a 30-minute process, you still have the human involved and you have the AI involved, and you probably have the AI augmenting the human and potentially the human augmenting the AI. How do you measure that? We don't we don't have the measurement paradigm across the whole industry for that. Like we understand how you measure AHT, we even understand, although people don't do it very well, FCR and those kind of metrics.

SPEAKER_00But what is the metric for this hybrid work? Like, we need to start to develop that. That's the only way the brands are really going to start to see major transformation. So that's why I'm excited about it too.

SPEAKER_01So, you know, as a late, I mean, you've written quite a lot about like kind of like just the conversational experience orchestration and like the agent control plane and stuff. And I've read some of that work. Uh, I think it'd be really interesting if you could like give us a bit of an overview of that because you mentioned like kind of like humans helping AI, helping humans. Like, how should people think about like the relative importance of automating agent assist, the data flow between those? Like, do you see it working better than before?

SPEAKER_00Or yeah, so let me take a step back since you mentioned CXO or conversation experience orchestration. It's not a radical concept. It's that the idea is that we used to treat conversational AI and conversation intelligence as separate worlds. They had separate buyers, they had separate users within the brands, and often the conversation intelligence, when it was applied to service process improvement, was applied to the human side, right?

SPEAKER_00So you would do automated quality and that kind of stuff with these tools. My argument, and actually the industry is proving out that we're right, is that that was a false dichotomy to start with, right? Like if you are a good conversational AI company, you've already proven that you understand how to understand the conversations, otherwise, you wouldn't be able to have them, right? Like you understand the structure of conversations, you know how to analyze them, otherwise, you couldn't have continuous improvement of your own products, like it wouldn't work.

SPEAKER_00Is it necessarily true that if you know how to analyze the conversation, you know, if you're an analytics company, then you know how to have the conversation, maybe not quite as much, but it's becoming true. Yeah, yeah, yeah. Right. And we're seeing acquisitions kind of make that case, right? Nice and cognition, for example, nice the analytics world, cognity in the conversational AI world, and they're trying to figure out how to bring those together so that they're not two separate products. Then you layer in agentic AI on top of that.

SPEAKER_00So now it can take action. So, what we're actually seeing is that this conversation experience orchestration creates the flywheel or the loop that brands really need to have the conversation, understand the conversation in order to improve the conversation. And yes, that does mean things like the agent augmentation. And it's interesting that we have this weird world where we're not always clear when we say agent, whether we mean human or AI.

SPEAKER_00And in this case, it's kind of irrelevant because it could augment either way, right? So you could have the typical agent assist kind of version of agent augmentation, the human agent, or you could have an AI agent tackling a complex process that's following some guidance that says we always treat platinum customers well, but we have a policy that says I can't do this thing. Let me go to human and say, Can I make an exception? I can't do it because my, you know, the the probabilistic part of me wants to do it, but the deterministic rules that I have to follow say I can't.

SPEAKER_00So you can give me the thumbs up for the exception. And in that way, the human is augmenting the AI. And that's only possible when you have kind of the this combination of the conversational AI technology and the conversation intelligence technology, because you need to understand the conversation and be able to analyze it in real time to be able to say, Oh, let me reach out to a human, not necessarily for disambiguation of an utterance, that too.

SPEAKER_00But the authority, right? Like more complex things, like can you give me the exception, the the okay for this exception because we want to treat our platinum diamond gold customers better than we treat Joe Schmo off the street, right? Like that kind of thing. So that that's where we see that loop really starting to play, and that's how that data flow really would work. And I do think that the conversational AI companies like Polyai have seen that for a long time because you've been doing the analytics again for your own continuous improvement.

SPEAKER_00It's just now doing it in real time in the customer's environment to improve their service flow and not just the agent performance so that the agent, AI agent says the right thing, doesn't have hallucinations. Like we're we're moving past that world into feeding the intelligence in to create better service process flows. That that's again, that's exciting, isn't it? I mean, that's that's pretty cool stuff.

SPEAKER_01No, 100%. I mean, look, I think that when we just look at like the increased capabilities of these models, you know, we look at, you know, I just last week looked at probably about 10 of our largest customers and the repeat callers there and what's happening afterwards, where the human journey goes, that sometimes we have, sometimes we don't have access to. And really it is that same journey mapping. And you know, for the longest of times, I think we've been liked by a lot of our customers for just like honestly telling them, like, hey, look, we'll reach like a limit to what we can automate, not because we understand or don't understand, but because we'll we'll be the annoying people flushing out like things that you never really fully settled in your SOPs.

SPEAKER_01And you know, if you turns out that took me a few years to learn this from when we were very small, but if you say this outright, you warn them, then you show them, then you actually get credit for it versus getting blamed on your technology not working because you know you didn't hit this rate or that rate. And what's really interesting is like the tax on whether you need a human, whether you're an AI agent or like a tier one human agent, is really that like spiritually, let's go like that digital transformation and the like you know canonization of what your rules are as a business.

SPEAKER_01Because honestly, that person, even if it's a tier three agent, they're just freestyling. So like it's really for them the question is like who do you trust to freestyle, right?

SPEAKER_00I like that idea of freestyling. Yeah. When we're looking at some of our clients' work in terms of humans augmenting AI, one of the problems that they have today is that they don't have a role for that. They don't have like a job wreck for what that looks like. They don't know how to manage those people because who do they report to? Because they don't have managers for it. For a while, I was calling them kind of coming up with a goofy name, calling them bot wranglers, because back then we were calling everything bots before agents, right?

SPEAKER_00And they had to wrangle. And that was often about disambiguation of utterances and voice AI or stuff like that. But also, as you're saying, there's like some canonical practices that the brand is instantiated in some policy document somewhere that's been fed into the training corpus of of the conversational AI AI agent. And that's great, but it makes it very hard for it to freestyle if that's you know, if that there's like a probabilistic layer over some deterministic flow, and it has to follow that flow.

SPEAKER_00So, in that case, clearly the brands are trusting the humans to freestyle more. I think that that's changing. I think it will, I think it will change, but this is one of those cases where you do need some kind of human backup to be there to get that trust level up to the point where you don't necessarily need the same level of human backup. Again, my example of a policy exception, right? We we want to have human judgment there enough so that it becomes a training set that we can then use to say, okay, for this type of customer with this type of conversation where we've done this and in the past three weeks they've had six different interactions with us about this thing.

SPEAKER_00Sure, go ahead. You can give an exception, but in these other cases, no. But today, that's gonna be a human kind of inserting human intelligence, human empathy, and human understanding of policy into those agentic AI flows. But I do think it's interesting that we we uh are getting to the point where we're actually gonna have to remake the customer service organization to create these roles. Like, what are they?

SPEAKER_00Right. I we you know, we we could come up with names, bot Wrangler was me being silly, but we could have something that sounds more corporate about what that that person does, what their KPIs are, who what their management structure is, and then what their career path is. Because as I kind of described it, it's a self-limiting role. You train yourself out of it and then open up the path to the next thing. Yeah, I think it's one of the reasons, for example, that the BPOs are looking at that as a path for their their agents, because if they're going to automate away all of the L1 work, what do those people do?

SPEAKER_00Well, maybe we can upskill them to be this bot Wrangler thing for the next three years, and then we'll have to think about what the next role would be for them and keep moving them up. And if the BPOs are doing it, brands can certainly do the same thing, maybe not at the same scale, because they're managing 50,000 agents or 100,000 agents, but same idea probably applies.

SPEAKER_01No, totally. I mean, like I've always called these guys kind of like the new like air traffic control, where really, like, you know, when the Well, let me look, look, I think, you know, from when we started Poly and there was a glory days of chatbots, and no one was ever gonna call again today, you know, like voice is king and you can do no wrong in voice. Exaggerated in both cases. People just want like the best modality, and sometimes it's voice, sometimes it's chat, sometimes it's an app. Ideally, like for the most part, they should just not have the problem to begin with, or if you know, services that are too complex, like you said.

SPEAKER_01So they just need that support because they're doing something for the first time. But what I found really interesting is like it's really, and you know, the latest monikers, context engineering, like those tier three agents and managers, they have context, right? And they're trusted with judgment over places where the codified word of law and the contact center has not caught up with their better judgment of like 10, 20 years in that organization, right? So to me, like they're really the ones that are just kind of like trusted to like guide the agent in those settings.

SPEAKER_01And ideally, and I think maybe you know people are not writing about this enough, it's really how in advance you provision up to capture that context in those situations that creates like a higher order canon law for the contact center to move into a higher degree of automation. That's where you get that like higher ROI, but people kind of just go like, is the technology there or not? It's like, are you there or not? Like, how many times have you up-leveled your SOPs to get to something that could be automated by, or you know, if you brought in like 5,000 new people into your contact center and everyone else just did not exist, could they restart it?

SPEAKER_01And how quickly could they get to the same level of performance? Because it's really the same question.

SPEAKER_00Yeah, the flip side of that is that some things haven't changed from your chatbot days, and that, for example, there are some use cases where the modern technology, and right now, polyai could automate some of these interactions, and the brands don't want to or shouldn't want to, right? So the example that I always bring up, because spent 10 years at Forrester, we had a lot of insurance clients, and talk to them if somebody had was making a claim on a life insurance policy, they didn't want automation within a thousand miles of the front end of that interaction, right?

SPEAKER_00Because somebody close enough to that person to name them as a beneficiary on a life insurance policy had died, and they just wanted a human voice. Now, it's true, that was before the world of voice AI and 11 labs coming out with Joe Sympathy or whatever the name of the voice model is that's supposed to sound the most empathetic and sympathetic. But still, like there are some things where it's not just that it's like that tier three stuff where we haven't figured out how to instantiate.

SPEAKER_00That context and expertise that those people have into the policies of the contact center, yet, because right, they're being paid for contact center work anyway very well because they have the human judgment, then the brand trusts them. It's also these we're still in that world where there are some things you probably still want human beings for and will for quite a while, simply because of the nature of the type of interaction that you're having with a brand. I don't think that's changed.

SPEAKER_00I mean, what's changed since the the early chatbot days is you probably can now automate a lot of that stuff. And it would have been more difficult, you know, 10 years ago to automate that.

SPEAKER_01Well, look, I mean, I think it's like Q and far between that you get a chance to do it. I think we fur we had the first kind of like heavy bereavement workload about three and a half years ago. It's been running ever since. It processes thousands of calls every day out of a workload of a few tens of thousands daily. And we get higher CSAT scores than humans do on that thing. But you're right, people do not believe, no matter, and you know, like this is a public case study. It doesn't matter.

SPEAKER_01People just they won't do this until they have to, until the service levels with humans have collapsed really badly, whatever reason that you know life in that context center led to, that then they have no choice but to try. So like I I'm fully with you. I think that people just won't take the risk if they don't have to. And then when they have to, they'll take all sorts of risks, and AI can do these things, but it doesn't have to, right? I mean, to me, it's always been like you don't have to automate everything, and you never will automate everything. And you know, and when you've automated everything, there will still be a few hundred people running those AI agents on your behalf.

SPEAKER_00It's not exactly the flip side, but sort of a a different take on that coming at it from the other angle are the use cases where brands think that they probably need humans, but actually you'll see things like CSAT, but also other important metrics higher with automation. And those are the things where there's some sort of social friction that's possible, the human-to-human interaction that's then removed when there's some AI involved, right? So collections is an example.

SPEAKER_00It's embarrassing. If you have to talk to a human and build out a payment plan, you're more likely to do it. So we've actually seen statistics from customers that promise to pay goes up when there's conversational AI. The amount it actually is smaller because the AI isn't guilting the human the way that another human would. But promise to pay is like a key metric in that industry, right? It's like a huge one. And we see that it's actually higher with AI. And then in the less common use case, we've seen in the I guess you'd call it the sexual health world, like the Planned Parenthoods and others of the world, like having a conversation with a teenager is very embarrassing.

SPEAKER_00And it's not that you'll get better results, you'll actually get results. Like the the the teenagers may actually have that conversation with the conversational AI that they would never do with another human being. So that's another area where you you would think it would be the other way. Like that's one where you always want a human being, but the data kind of shows something different. That like you have to actually start to analyze the social dynamics of human interpersonal relationships to be able to sort of identify which use cases are gonna really sing for your organization.

SPEAKER_01Yeah. Yeah, no, no, no. I mean, I think that's completely right. I think that like bereavement calls are the flip side of that, where I actually people forget that if you are a contact center agent paid not that much above minimum wage, and like one in ten calls are of someone losing a loved one talking to you about it, you're not gonna last very long in that job because no one wants to have a job where they have to hear that someone's mother, father, mother, sister died. Like, that's just not, you know, honestly, that probably doesn't even have to do with how much you're paid, like it's just not something that you want to do because it's not even support, you just hear about like sad things, right?

SPEAKER_01Literally, yeah, STDs and stuff like that. That's where like bots reign supreme. Because how did that happen? Well, you know what? Like, the core is gonna be pretty awkward following the moment that you tell someone about well, whatever, right? So, yeah, I found that hilarious, and we've had cases of that, and more serious health issues as well, like drug addiction and stuff like that, where it's just people don't feel the stigma of talking to a bot.

SPEAKER_00And this does have real-world business impacts. I mean, look at the financial results of the public BPOs over the last six months. The more traditional customer service lines of business are actually doing okay. They're stable in some really successful ones, they're growing. The areas where their businesses are shrinking are the ones where there's a huge cost and toll on the human beings in those roles, content moderation, for example, for socials, right? It's not just that the AI has gotten better at that, which it has and can actually take over a lot of that functionality.

SPEAKER_00It's that those people churn like crazy. And the BPOs needed on-staff psychologists and psychiatrists and therapists, and like so you can also flip the equation and turn the mirror inward and say, like, what's the impact on your own people of having those, as you were saying, right, around sexual health, same thing, or not sexual health, around bereavement, right? Same thing. It's like wearing on people, and if you can remove that, it's not just removing like the brain dead work, right? It's like removing the work that that takes a cost on your psyche.

SPEAKER_00Like, that's that's an area too, where there's a lot of opportunity for investment in AI to help your own internal employees like just have a better job. I've been arguing for a long time that we're not taking seriously employee agent, agent turnover return as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it that way.

SPEAKER_00I think that's an important metric that nobody is thinking about in terms of measuring how effective conversational AI is. And obviously that has economic impacts, right? Because the the cost of recruiting, hiring, training new people, especially when your attrition rate is topping 50%, is very high. So anytime you lower that rate, but also frankly, it's just a more human thing to do. You're making the experience for your own employees better, you're giving them a better job, and that's why they stay.

SPEAKER_01So 100%. I mean, look, to me, like one thing that's particularly exciting is you know, the contact center is a hard place, right? Like sometimes people in contact centers don't get like sufficient opportunities. And since we released our coding agent, Ren, like there are people in those contact centers that spent a lot of time like just improving the agent and doing an amount of self-service that was previously unimaginable, just because honestly, the previous tools that we and everyone else had were not usable enough by non-technical people.

SPEAKER_01And now we get someone going, you know, 1500 turns back and forth with Ren in a month, which is you know more than some of my engineers do with Cloud Code. And that's like outstanding because really they're the new sales ops to Salesforce, are you know, these guys to us and to kind of like conversationally as a whole. But as they all build things, like you've written quite a bit about agent control plane and agent sprawl. Like it'd be great to hear a bit more about your views on just like what enterprises are doing, how they're containing the sprawl, are they containing the sprawl?

SPEAKER_01Like many anecdotes you have. Yeah.

SPEAKER_00Yeah. So the the idea that drove this research was, or the conversations that drove this research was talking to clients who said, Yeah, well, you know, I just in contact center am getting pitched AI agents from the likes of Polly, but also the likes of Salesforce and the likes of ServiceNow and the likes of Varend, and like right, so all different components. It could be on the analytics side, it could be on the workforce, then workforce planning side, it could be on the interaction side, it could be on the agent augmentation side.

SPEAKER_00And I know that in my organization, we're also hearing from SAP about AI agents to help our back office and ERP stuff, and our HCM vendor is pitching us and blah, blah, blah, blah, blah. So we're getting all of that while developing our own AI agents internally using Claude Code or whatever tooling we're using to build our own AI agents. So it seemed pretty obvious that they were having the early signs of AI agents sprawl.

SPEAKER_00They're gonna have an army of AI agents who are all, in theory, supposed to work to drive the outcomes that the brand wants. And many of these complex processes cross all of these different little silos that the AI agents are being pitched for. And so what we started to wonder is what would it take for a brand to think about managing this AI agent sprawl, the AI agent workforce army, holistically?

SPEAKER_00What are the shared layers that any AI agent that you have should be able to tap into to drive consistent outcomes that you're looking for in your brand? We came up with five of these kind of shared layers, but I bet anybody with you know a good brain could come up with five other ones as well, right? So I'm never gonna say that this was a census of everything that is required. These are just kind of the obvious ones, especially from a CX perspective, because that's the lens that those are the clients that we're working with, right?

SPEAKER_00So that's the lens we're looking at. The first one, it's interesting that you mentioned context engineering. We're calling it journey and intense state, but really it's the context. What context do you need to persist throughout an entire journey across every agent that might touch that journey to drive the outcomes that you're looking for? The next one is maybe the best understood layer when it comes to humans, and one of the weaker ones when it comes to AI agents these days, and that's identity and consent, right?

SPEAKER_00The ID and V and authentication. Great, we have voice bio everywhere for humans. What about the identity and consent of the agent as it tries to do something, especially when you have multiple agents that might have the ability to do the same task, right? Because they have access to multiple different tools that touch different systems. So, like, what is the watermarking here? Of I was authorized to do this thing, I did it, and you can track it back, right?

SPEAKER_00The next one is something I mentioned in passing before, which is policy and guardrails. We think about the guardrails piece in AI, we've been wondering about that, and now wondering, working at that for a long time now. Yeah, but the policy piece, how do I build a consistent policy layer? Again, that any AI agent, whether it's the one that's managing the work schedule of the contact center agent, or the one that's actually interacting with the human customer right now, or the AI agent that goes and does some back office task that then feeds data back to the contact center agent.

SPEAKER_00How do you have one policy that they're all following? Right. Again, how do we treat our platinum customers as a very simplistic example of a policy, right? Here are the rules about what we want. It doesn't mean that it's specify every single interaction will go this way, but it's more like here is the tenor of every conversation that we're gonna have with our platinum customers, and here's where we're willing to bend and be flexible for them, and here's where we're not, and all of that kind of stuff.

SPEAKER_00There's knowledge as well. We're not so much focused on knowledge as a layer, meaning you've got to aggregate all of your knowledge sources into some giant data repository, right? Data warehouse, data lake, whatever we come up with next. I don't know, data ocean, whatever. It's not that, it's more the governance of knowledge has to be consistent, right? I mentioned that the different agents may have access to different tools while they'll also have access to different knowledge.

SPEAKER_00But what are our rules across the entire company for data freshness? Who determines that? So that the AI agent is always using the right one. Or frankly, we've got three different knowledge repositories that kind of test on the same thing. Which one is the AI agent supposed to use at that point, right? So it's the governance around knowledge. And then the final one is something I know you guys have worked on a lot, again, internally, but this also needs to be external facing. And that's kind of the eval, right?

SPEAKER_00How do you actually take evaluation and testing and move it from a post hoc but pre-deployment one-time thing? Yep, it works, let's deploy it, to continuous throughout the entire process so that it's an endless loop. And again, when we're talking about an army of agents, it has to not just be focused on a single AI agent and how that AI agent works, but the interaction between the AI agents and the interaction between the AI agents and the humans.

SPEAKER_00Again, my example augmenting AI, augmenting human, human augmenting AI. So you have to be able to test all of that. I know that's a lot that I said, but those are like the five layers that we came up with that sort of seemed like prerequisite if you're really going to go from like having an AI agent or two in your payable organization and one or two in customer service to be having orchestrator agents that are orchestrating 50 different kinds of agents, 50 different kinds of tools, and that's just in the service organization.

SPEAKER_00Then you've got the same in the mid-office and the back office and in the supply chain and HDM and in legal and blah, blah, blah, blah, blah. Because all of that comes to play in complex interactions with customers, right? All of those different organizations. It's not very hard in a B2B environment, for example, to see where a service agent would need to interact with legal's AI agent, right? Like that's that's not an uncommon use case. But how do you evaluate that interaction between the agents and make sure that it is optimized so that it's always working?

SPEAKER_00Anyway, that's a long-winded answer, but you know, it's work we did, we published it a couple of months ago, and it was really just uh plant the flag in the ground because it's something that we're paying a lot of attention to. And I know one of the things that is the typical question when we talk about this is well, who do I buy that from? You know, I want me one of them, get me one of them control planes, right? And it's a complex answer, as you guys well know, because you operate in an ecosystem and you happen to be particularly partner-friendly as a company, but not all of the AI vendors are.

SPEAKER_00Some of the hyperscalers come to mind, right? Like they want you to operate inside their own environment or big CRM providers or whatever. So for you guys, it makes sense that like you're gonna have to interoperate at this layer. But for some of the big guys, like if you're a Salesforce shop, Salesforce's sales guys are gonna tell you they can do this because that's how they operate. Like, it doesn't, you know, it could be vacuum cleaners. Yeah, we do that, right? I mean, I'm being facetious, but you know, that's what I will uh say more and you know, like say anything bad about it. But yeah, you're absolutely I don't even mean it as a bad thing because what they end up doing is like they figure out where the demand is and then they buy or build something, and it takes a couple of years, but they're there, right?

SPEAKER_00And they try and right. So, my my point is that this is complex, and for the most part, brands, especially today, are not going to get this from one vendor, they're gonna be working with vendor who understands how to do the identity consent verification authentication in the human and AI world, right? You guys will tap in to that authentication because all you need for your AI agents is yep, we we can check that you know this is verified.

SPEAKER_00This this agent has consent to do this thing. We don't need to be the one checking it, all we need is you know the output of that consent process, for example.

SPEAKER_01Honestly, that I think that you're absolutely right. And the taxonomy you have is really good because it's kind of like you know, for different clients, you'll see just like where they've made more progress or less.

SPEAKER_00But oh, yeah, yeah.

SPEAKER_01Very rare that you hear that someone's cracked at all. Because honestly, at that point, they might as well build it all on their own with a hyperscaler, because they're one of the few companies in the world that can.

unknownYeah.

SPEAKER_00Very few saying that you have to be at the same level of maturity across all of these layers, right? I that that that's clearly not gonna be the case as a brand. I mean, I know you were talking on the vendor side as well, but I mean also for the brand, this isn't one where you like you get the gold check across each one at the same time. You got to focus it on the things that are probably causing you the most pain as you start to deploy multiple AI agents first and build out from there.

SPEAKER_00And your provider, whoever that is, is probably, as you're saying, stronger in some of these layers and has the technology potentially built in in a way that you don't even see, right? It's already there. But you have to start as a brand, you have to start asking the questions about all of these different layers that we're talking about and the tech, the topics that are inherent in each of those layers. Again, this is how you scale. Without that, it's gonna be very, very hard to scale to the point where people talk about replacing 90% of their contact center agents unless you've got a 10-person contact center, right?

SPEAKER_00Like if if you've got a very large contact center, you're not gonna get there unless because you have those corner cases, the edge cases.

SPEAKER_01I mean, the reason you have it is that you've done something for many years that led you to the point where you have 10,000 people doing that.

SPEAKER_00That just that anyway. So we're calling that the AI agent control plane. I will point out that there are a couple of vendors out there who also use the control plane terminology, they tend to be focused in the eval and testing world, right? Because they're saying they're gonna be able to control the agent performance by doing testing. But we're we're I looked at networking, they call it a control plane, they talk about having different layers. I'm like, all right, I'm just gonna steal that concept. So it may not be the name that this end this thing ends up being known by in the in the future, but analysts love to name things.

SPEAKER_00That's you know one of the things that we do.

SPEAKER_01So right in this world, right? Yeah, I mean, computer networking is always a good one to rip things from. I think my wife randomly was like, Well, how does the internet work? I was like, I've taken many a graphic course and I could tell you everything. She's like, go to work, go to work. I'm like, okay.

SPEAKER_00Networking it is a good thing to steal from, for sure, as a call. But you know, the one of the keys that we didn't talk about, and this is something that's true, I think, in your more traditional part of your business, is that what's implicit in what I said is that each of those layers has access to all of the enterprise systems and data that were required to power it, the same way that you have to have the right system access for an agentic that tool to do anything, right? It has to be able to access the right thing.

SPEAKER_00And then the AI agents can come from anywhere on top of that, right? So it could be the commerce agent, it could be the legal agent, or whatever it is, and they all should tap in. So I think of this as kind of the sandwich, these control plane layers between the AI agents and the enterprise systems, because they in some ways dictate the access to the system. If the policy says you can't touch that system, don't the agent won't touch that system, right? So it could kind of sits between it.

SPEAKER_00I have spent time in the tech vendor world, but I never had to create a markitecture before. But basically, that's what I created here. You're I'm sure you're much better than that.

SPEAKER_01Much better at this point now that we have claw now that we have Claude, I'm very good at it, but it used to be something I run to my to my co-founder for, you know, and he he was always the one that that would have to bear that across. But yeah, I think we're we're we're running out of time. This was a real pleasure, and I hope we get to do it again. And you know, I'll we'll see where this poll gets to by the next time we get to get a chance to have this conversation.

SPEAKER_00Yeah, it's possible that there will be some recognition of the need for this control plane. I think the CXO thing is a done deal, people just don't recognize it yet, but the control plane's gonna take a while. I mean, I recognize that it's it's gonna take a while. But we'll try to try to do it. It starts you thinking about it and starts you know, some questions in your head about where you need to go. But Nicola, thank you so much for having me. I really appreciate the opportunity to be here.

SPEAKER_01No, it was a pleasure and always enjoy your conversation. So, Ian, thank you so much for joining us to everyone watching. Like, share, subscribe, and we'll see you in the next one. Thank you, Ian.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  4. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode

  5. Is word error rate just a vanity metric?

    Jul 21, 202629 minAudio

    Voice AI vendors love to quote word error rate, usually somewhere around 2 to 3%, as proof their system understands customers. Oliver Shoulson, Agent Design & Engineering Lead at PolyAI, thinks it's one of the most...

    Credit the people on this episode