Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

04: The Future of Conversational AI (Pt. 1) with Yan Zhang, COO of PolyAI

Oct 27, 202321 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Ooh, a new look! Michelle (SVP Marketing), Nikola (CEO), and Yan (COO) discuss the changing landscape of conversational AI, the impact of generative AI, control and expectations, and the potential for brands to sculpt their voice assistants to enhance customer experiences. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Yanit's a really good shot for the, for this notion of control. I think that what enterprises will come to realize is that the conversational agents. Automated conversational agents doesn't matter whether it's in the chat channel or the voice channel is going to be a significant feature of those enterprises brands,

MichelleHi everyone, and welcome back to Deep Learning with Polly A. I. podcast. Today I have not just our co founder and CEO, Nicola, who you've all seen a lot of, but today I'm also joined by Yen, our COO. So today we're going to be talking about the future of conversational AI for both builders and for buyers. And Poly AI is I think pretty qualified to talk about this. Seeing as though they've been at the forefront of conversational AI, delivering. Enterprise grade, customer led voice assistance for over six years and it's pretty safe to say that the landscape is changing.

MichelleIt's changing fast, and the rate of acceleration rate of change is really speeding up. And so I think one of the best indicators that the market's changing and changing very quickly is the stratification. So just being able to see these really tiny, conversational upstarts joining the arena. Seeing a lot of consolidation up at the top with Seacast providers and with, other types of solutions that are starting to add conversational AI into their lineups. But perhaps the most dramatic shift that we've seen is one that sort of is very close to home for a lot of the startups and companies that sort of use analysts.

MichelleTo understand the space and also to make sense of it. And Gartner has been the first of the analyst companies basically to discontinue their conversational AI for enterprise category. So to discontinue their magic quadrant. And yeah, I really want to talk to both of you about what this means for the space, how people should be thinking about it. Yeah, what does this mean?

YanSo I think that the way that conversational AI products are being built and delivered has changed significantly. It will continue to change for the next 12, 24, no matter, God knows how long, but it also means that a lot of the a lot of the circles that we've drawn around what a platform means, right? And the functions of a platform are becoming more ambiguous, right? Those lines are becoming more ambiguous. It's going to encompass new functions drop some old ones.

YanAnd I think everything's in flux and it's a very exciting time.

NikolaI think it's interesting, right? What Jan said about platforms is exactly right. People buy this technology, they buy it in a wider context center, and they prioritize different things. I think with generative coming in, it's really. Turned the capabilities around where a lot of people can do a lot of things build a lot of different things There's a lot of excitement out there. I think you know, if I were an analyst I wouldn't want to put down definite predictions or you know That's a topography of a space here because it is changing really fast.

NikolaSo I think I don't have an opinion about the Gartner discontinuing that stuff, but I think it's really hard for a buyer these days to understand what it is they're buying, what they're optimizing. I think it's really important to look at the incentives of the different parties they're working with and the different products that they're selling them if you're both a builder or a more traditional business side buyer.

MichelleLet's talk about, so you just touched on the fact that generative AI. Has been one of the big accelerators of the change in the space of conversational AI. And I think that, Gartner said the same thing when they told us personally why they discontinued the MQ. It was really like that was cited as one of the big reasons is because generative AI was changing the landscape so rapidly that they wanted to press pause. And again, like to your point, they didn't really want to make predictions about something that is, changing right under our feet. So maybe let's talk about that. A.

MichelleI. Part of the conversational high piece. Let's talk about the generative landscape. What do we think that impact is on conversationally? I like what's happening

Nikolalook, I think that it's easier than ever to build a really impressive. Demo, right? If you're a builder inside a company and you think, Hey, I would love to introduce some more sophisticated form of automation for you to show something around the company and get a buy in for a project, it's very easy because now more than ever, you can build a demo flow quickly especially in text. And when you do that, it's maybe we should go in and just build this stuff ourselves. And I think what it's like really then behind the logic for Gartner and others is.

NikolaAre we going to see a consolidation of the space where it all flows back to the larger platforms that play around here? So I think Seacast and Cloud above all or are we going to see a few champions emerge that are going to be the snowflakes of being really cloud first companies delivering a ton of value for people who want to do this? And of course, I think the two sides of the debate are. Are people going to build all this? Is it just, a technology, a platform that people inside every large enterprise are going to leverage on their own, or is there space in the middle for a snowflake or two of this?

NikolaWho will bring again, a platform, but in a way that's easier to consume. Towards those companies that stand to benefit from this massive increase in the capability of virtual assistance.

Michellego ahead. Yeah,

Yansure. What I was going to say in terms of, how generative AI has up the game is that traditionally for the past, let's say, six or seven years where I came into, conversational agents and how they functions that we were really solving for figuring out what the customer is saying, right? It's intent based, right? So that, and, whether it's in the text channel or in the voice channel what people say is a very complicated input, right? It's not necessarily, symbolic, or just a keen letters and numbers.

YanIt's it's everyday speech. And so you have to leverage AI to figure out what people are saying. And we call that an intent based model. So in generative AI or in generative powered conversational AI, we're sticking with that, basic capability, but we're adding reasoning to it. And we're adding much, much more sophisticated reasoning. So a good example would be calling a hotel and saying Hey I'm a little bit hungry. Is there or is there a bar, or where I can grab a snack?

YanAnd then, I use probably too many words to ask that question, right? Because that's just how people speak. But, conversational AI would be able to figure out what I meant. Then generative AI would be able to answer questions like so my flight lands at 7 30 and, Let's say there isn't a too long of a line in immigration how, when do you think I can reasonably be at the hotel? So that's actually taking several different pieces of information, applying common sense to that particular question, and giving the passenger or the guest an estimation of when they would get to the hotel.

NikolaYeah. Yeah. And to add I'm in Vegas right now, which is actually the highest concentration of poly clients on the planet. Going to Reno in two hours, hopefully, actually there's no immigration because it's domestic flight, but Hey recent question from a client, someone called reception and said, Hey, I'd like some mayo. And it's not within the purview of, reception to do that or the voice assistant, right? And what do you say to that? You're like there's a Safeway nearby or something like that. Or like, Hey, look, there's a bar and a restaurant. It's a tough thing to answer.

NikolaEven if you're a human, cause what exactly do you do? Do you bring a jar of mayo that you procure somehow? It's really interesting, but with generative. That question can be answered in all sorts of good ways because just the objective, intellect, if we call it that of the model now is to really understand the context of it and the context of what it's supposed to do and then to improvise far better.

MichelleYeah, I think like anybody calling down, to the front desk asking for mayo and getting like a human being on the other end is expecting that the person knows the nearby. Safeway is CVS, whatever, where they can go and they can buy that. And now we're saying, generative can help answer those types of questions. And so I just have another question for both of you here, which is just that I think a lot of people's experience with chat GPT which is really like the first time at an unmasked, like a person could interact with generative AI.

MichelleIt has. Raised the bar or raised the expectations for the types of reasoning that we think that bots should be able to do if you're getting a bot at the other end of the call in Vegas, maybe the expectation is that it's going to point you to how to get Mayo, even if that's not really in the purview of the bot. What do we think about this? And what do we think about what this means for, Conversational AI as a category do we all need to raise up to this level?

NikolaI think that's a thing that comes to our mind first, cause like we do so much of this. I think the expectations are still forming. I don't think people expect all this. I definitely don't think the consumer expects it. By and large, the consumer in the U. S. is still just screaming for an operator. When you hear voice assistant on the other end of the line. It's a bit different in Europe where there have been fewer of these historically. But, I think as we build more and more things for in the builder's mindset and people planning it, it's a brave new world. So maybe, to some that people that I've spoken with, that's a given, of course it should work, right?

NikolaAnd there is nothing that is natural or obvious about that conclusion. So I think a lot of people are jumping there and then I think it really opens the space of. The most show off you think that you can do with generative versus like controllability, robustness, and then another set of buyers is really thinking much more about that. I think those guys are almost a bit paralyzed when it comes to reaping the benefits of generative because they've been, they've heard horror stories about how it can just say just about anything, and then on the other, the builder is really excited and ready to jump on build whatever it is that they think they can build and then, where those two intersect Will be like what we get in the end yeah,

YanI think in terms of expectations is really interesting right because it's I think we're making a leap of logic of comparing similar experiences that maybe the consumer It's not really intuitive to the consumer because I think that The, Alexa has been around for a long time, but people still have very low expectations of enterprise voice assistance. And we obviously review a lot of calls and there's so many calls that we take where we greet the customer with, how can I help the customer immediately says, I want to speak to a person and we're like, okay that's totally fine, but before we, pass you on to a person then, what can you tell us exactly why you're calling so we can get you to the right person?

YanThey're like like I want to track this order, but I don't have my order number and we're like, okay That's okay so what day did you buy and so on and so forth and we carry that conversation to its conclusion? And we surprise the person right and which is really interesting is that I think nikola This is something that you said a couple years ago. We're trying to Figure out what our mission statement is. And one of the parts of what we want to do is to be the first eureka moment for many people, right? To say, Hey, this is not your your old like voice automation experience, right?

YanThis is something new. It's capable of doing much more than what you were used to. And I think there's going to be another kind of level of eureka experiences. With generative models where people are not just looking up their accounts or okay, what are my repayment options? And expecting, not really expecting a clever answer, but when they get that clever answer, it's going to blow someone's mind. It's going to, at that point, give them new expectations for all the experiences that they're going to have over that channel.

MichelleYeah, that makes a lot of sense. I think I guess follow up question there. It's like on that way to a Eureka moment, like on the way to where we've had this, this complete up leveling of what people are used to there's that degree of control that's going down. As a sophistication is going up, your degree of control is going down. Because these highly controlled IVRs are like routing systems that just get the person to the right. Other person on the other end of the call, like those are high degrees of control. And so just curious to know your thoughts on that.

MichelleAs we approach these Eureka moments, what are we letting go of?

NikolaYeah, I think There is a way to mix both things. And I think it really depends on where you're using them, what you're using them for, which subsegment of the market it is. I think those that play with it more are going to have some like really positive surprises, a lot of good Eureka moments, but also like brand damage potentially. So you need to basically create a much more modular way of using generative so that you have like guardrails around just like where you're really like letting it take a whack at the problem and do really well. And where you want to limit its ability to improvise.

NikolaIt's like having different players in your team. Sure. If basketball game and you're losing, seven points and there's 30 seconds left. You're probably best off shooting some three pointers, right? If you just need to control the game and you're, you have the lead, then, I think the, you don't necessarily need to do the same kind of stuff. You just follow a very different rule book and that may not have to be generated. That's just, a transactional flow where you use things that are really good at parsing input and, you don't expect a high degree of variability there.

NikolaThere are other implications here about cost. And latency and all of that, and I think that we'll talk about it more. And there's a lot to unpack there. Cause where the world will be in two years is not where the world is now. At the moment, if you were to go full generative, you'd end up with systems that often, cost more than humans, maybe substantially. it's happening, but there's a bit more work to be done.

YanYeah. it's a really good shot for the, for this notion of control. I think that what enterprises will come to realize is that the conversational agents. Automated conversational agents doesn't matter whether it's in the chat channel or the voice channel is going to be a significant feature of those enterprises brands, right? In the same way that their app is, that their website is, that their even product is, right? And it's going to be especially true. For brands that interact with customers a lot, like banks, airlines, hotels, right?

YanWho answers the phone and what kind of conversation you have with those brands is going to be a huge part of that experience. So when we're thinking about when we're talking about control we're thinking about it from almost we're thinking about a kind of, a prohibitive type of control. You want to put guardrails against the bad things and and make sure that, only the good things are within the boundaries of the conversational agent is talking about, but then there's a positive side of control where really you're giving the enterprise a way to sculpt a brand identity using this new medium.

YanAnd all of a sudden you can imagine how. A premium hotel brand or a particular, low cost airline is going to want an avatar that lives in the phone that lives in the app that is conversational, that is able to be far more functional than what an IVR or what a chatbot is going to be doing, it's going to be incredibly, upscale to to help you in all sorts of. Of things as important as the app, right? For the brand, if not more. I think that's really interesting.

YanAnd that really is probably one of the reasons why Gardner wisely probably suspended or paused what an enterprise conversational AI platform can is right. So when they restarted, it could be something that's much, much more important to the enterprise, right? Where the enterprise would spend a much, much larger part of their annual tech spending on, on, on this experience.

NikolaYeah. It's happening already because if you look at the more traditional conversational AI platforms, they really had an it audience in mind, but the it audience is not the one sculpting the brand, owning the brand controlling the narrative with generative AI. The control is not actually. It can be prohibitive, but it's not intended to be, it's intended and it will democratize access. It will create a whole new job category effectively of, a dialogue engineer, the same persona that is both layering down the flow of the intents, etc, which you will have to do increasingly less of.

NikolaAnd then really making sure that, you're on the right way, you're saying the right things, you're getting information from the right place. It's really exciting.

MichelleYeah, I agree. Do you have any advice, for the, for, Yen, you said something about, the brand being able to sculpt this new frontier for them the same way that they were able to sculpt from an app perspective or website perspective. But, if you see those things as, as fairly contained things where the idea flows from marketing or brand or experience leaders, over to builders within the business, do you see the same flow? For a voice assistant for the voice channel for a brand. And how did what advice would you give to builders? I guess inheriting that. Message of what to sculpt and how to sculpt

Yanit. Yeah. So I think it's a really good question. I think that a lot of the vocabulary probably that, that will exist to talk about this doesn't really exist yet, what we do is when we create conversational agents is that, it exists on a graph and we sometimes talk about nodes and transitions. We don't need to go into what those things are. But I do feel that, those will be supplanted by LLMs and what they can do. But I think those conversations are still going to be modular, right? Having sense of control is to define a part of the conversation.

YanWhat happens in that part of the conversation, what's supposed to happen, and then to be able to then. Go to this next part or this other part of the conversation, right? And so I do think that there is going to be an emergent architecture of how that's going to be done. I do think there'll be a convergence right in, in, in the market about how we build and talk about these things.

MichelleYeah, and it sounds like what you describe is the cloud winner, I think, of this space, the person who can get that right.

NikolaYeah. There's a lot to mix just in between who is actually doing that work, right? And I think the real question is just will it be so easy to build everything with one platform owned by a Google or someone like that? Or is there, a space to like really. Have a snowflake like thing in the middle where, in theory there are competitive products there, but these deliver value a lot more quickly. Yeah. And cater. I think maybe more in the case of our space, they cater more to that builder because who that builder is not, there is no definite answer, right?

NikolaThere is it, and then there is like customer experience and the more they converge. the faster these companies will iterate and building a better thing. And I think like any IT or any tech initiative or building any product, right? It's really should be iterative. And I think the company that can help deliver value fast and motivate more investment in this. For the buyer is going to do better. And that's how we think about it because the alternative is that people go back to the drawing board and they are building their own LLMs.

NikolaThey are doing a lot of this like basic foundational work that, they're making some betts now that they might not have to make, they're making some investments that they probably won't have to do because the next version of GPT will just be like way better at that. And then, like you could, the four will get a lot cheaper and faster and they might not have to tune their model with their data. This is really important for people to think about because the number of times I've heard prospects in recent weeks tell me, and of course we have our own LLM look at it.

NikolaAnd I'm like, like not everyone needs a private jet, right? In fact, the vast majority of people don't, it's a bit of a. It's a big spend.

MichelleYeah. Yeah. Speaking of spend. So I know we are, we're currently at time. And we touched on a lot of super interesting things. So this like concept of control and how it's changing in the platform evolution of where we are. And so just the. Just tectonic shifts that are happening within conversational AI. But I think the private jet analogy is a good one, especially as we start talking about generative as we start talking about building these voice assistance and the economics of voice, which I think is going to be our next topic. So thank you so much yen and Nicola for joining me.

MichelleI'm really looking forward to the next one.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode