022: AI News Recap! Sora, LAMs and the Elephant in the Machine
25 minAudio
Michelle and Nikola discuss the impact of recent AI advancements on customer experience, including the developments in generative AI, multimodal explorations, and new AI products. They cover topics such as the potential for AI memory, personalization, and consumer data privacy, as well as the evolving consumer expectations and trust in AI technology. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
MichelleFirst party data back in the day used to mean, well, you gave me your name, number. Income email address. Now it's going to mean I know your kid's name. I know what kind of school you want them to get. I know your aspirations. I know what you dream of. I know what you need therapy about. Hello everyone, and welcome to another Deep Learning with Polly AI. I'm joined here by Nikola and I'm Michelle.
MichelleAnd today we're going through three pieces of pretty interesting AI news that just basically hit. Digital newsstands. And I think that they might have a pretty interesting impact on CX. So we're just gonna pontificate for a bit. Maybe talk a little bit about how this might impact customer experience. How these kind of major progressions in technology where they seem to be going. And just to spoil it. Two out of three of these pieces of A. I. News are by Open A. I. So one of those is Sora which is their text to video model that they just released.
MichelleI don't think it's open to the public yet, but they released it in
Nikolabeta. I think they've let it like the red teamers are working on it to see What risks it has, and how they need to vet it.
MichelleThat's something I love about OpenAI. Just a sidebar for a second. The other one was, is memory. So just the fact that ChatGPT will have memory. But OpenAI, if you ever read their sort of like product release notes, and like the marketing materials that they put around product releases they always anticipate. Every bit of like fear and criticism that they might get. And they do it so elegantly. Like I just loved on the SORA page, they had a section that was like, we know that this is going to be problematic and this is how we're talking about it.
MichelleThis is how we're thinking about it. It's like, they know their audience. They know safety needs to be one of their big headlines. And I appreciate that. The Sora landing page actually has a video that was created of a guy running backwards on a treadmill. So the front of the treadmill where you're supposed to be facing is here and the guy's running in the opposite direction. Fantastic. Anyway second piece of news memory. And then the third piece is this release. The are one rabbit release from rabbit dot tech. Who just put out like an actual physical thing that looks a little bit like a piece of gaming hardware.
MichelleBut they are also calling what they've released or what powers it a large action model. So this idea that you're taking. Action like real world action outside of this digital thing. And what's cool is it's actually embodied in a real world. Thing. It's not a website that you go to, to interface with the AI. It's an actual thing that you can carry around. So that's the news. And I guess what we'll be talking about today is how this, how we might see these trends, how we might see these leaps forward, make their way into customer experience or make their way into our world, which is a contact center.
MichelleAny initial thoughts just hearing that,
NikolaI think that if we go to our own slightly smaller universe of customer experience, I think that a lot of people I've had conversations with distinctly remember the CIO of the French bank who having never really deployed a voice assistant said, Hey, maybe we could pilot this on our website with an animated avatar, the people will talk to. And I was like, maybe, but in reality, that sounds like a science project, right? And I think we're waiting for the world of AR VR that stalled for a while, but seems to be picking up again to really propel humanity into an era where we have better hardware for it, frankly.
NikolaCause if you're already talking on the phone, you don't really need to see. An avatar for an assistant. It's interesting. But I think like it's probably best left and we'll probably make more progress in the likes of gaming and other, places where maybe education. We played around with this in the very early days of poly before we were firmly set on customer service and, it's interesting, right? And there's a lot that the agent itself the ability to talk, hold these conversations, have memories, well, it's relevant, right?
NikolaBut I think you have to figure out where that experience has a place.
MichelleYeah. But I think going back to the point you made earlier the AR VR kind of like developments that have happened. I think this is a really interesting point in history from a tech perspective where you have two, I wouldn't call them bubbles, but maybe some people would, but you have these sort of two big. Movements in tech happening at the same time, and one of them is generative AI, obviously, the, one of the main or like most common themes on this podcast, then you also have spatial computing.
MichelleYou also have AR, VR, maybe that the hardware isn't there yet. But the idea that these things come together somehow to help better support, a customer that bank with the avatar, they might not have been onto the right thing just yet, but the idea that these kind of. Technologies help support a multimodal kind of support future. It's, I think, super, super interesting. We're starting to see a little bit of those worlds colliding. Yeah, there
Nikolawas a bit of the pull towards it with consumer voice assistance, right? Where we thought that it would be important for the brand to have Alexa skills, Google actions, and then it didn't happen yet, but the way that we build voice assistants to be like part of the brand, you can totally see how, I remember right. Pfizer bank like big in central Europe has this animated chat bot with very I think it's a logo is yellow and black it's very well animated, like it embodies them. I think other companies have done it to Bank of America with Erica. So obviously like you're actually going to the world of VR and you have an actual person that is like your.
NikolaBrand representative. Yep. There, there's opportunity in that. You could probably imagine that celebrities will be paid a lot of money. 100%. Like a scaled sales rep for, I dunno, old state or something like that. Well, you
Michelleknow, progressive the comp. Insurance Company, progressive Flow their avatar, like the company mascot. I speaking at a conference a few months ago and we just showcased the product, talked a lot about it. It was really like a fact finding. Panel for people that learn about technology like ours and the amount of times I got the question. Do you think that you could ever sample a voice? If you're selling for a company like progressive that has an avatar already, they have a mascot already. That's associated with the brand. Could you scale that? Basically? Could you make that work?
MichelleAcross in this case, it wasn't multimodal. It was really just chat and voice. But there's a world in which maybe it's flow. That's actually helping you select, your insurance, walking you through that process and putting an overlay, over your, your experience on the website. Absolutely.
NikolaI think that's probably not that far off. I think you have a lot of companies that make these avatars. You can sing them a speech. So it's probably right around the corner, right? Coming back to Luxora, it's going to be interesting. Just, I haven't seen the pricing model for it, but obviously it's a lot more compute intensive and, I don't know about you, but I've wasted or wasted hours, designing like things with Dolly and stuff. And, um, I'd love for it to work a lot faster. The main reason I haven't done way more with it is that it's slow, right?
NikolaAnd I've seen like people that I spend time with kind of start something and they're like waiting. And then they give up. And it's like the AI nerd in the room that stays playing with it. But the voice is going to be like, Oh, no, sorry. The video is going to be like a whole lot more demanding. So, we might have to wait a few years until that's actually. Look easily usable at scale.
MichelleYeah. Yeah. Maybe maybe, but it is interesting. I saw like somebody screenshotted a message from about nine years ago, I think on Reddit and they said, Oh, text a video. Somebody said this world is coming. This like this world in which you see somebody do a prompt that's just text. And then a video comes out. This is definitely coming and they got flamed like the amount of comments underneath that was like, this is so far off. It's not going to happen in our lifetime, buddy. It was like the most condescending kind of response. And it was so many people just piling on this poor individual.
MichelleAnd then that resurfaced, as like one of those things, it's just yeah. Who's laughing now? Like this is, this
Nikolais law is merciless, right? I think that once you, you crossed like the, the belief and the capabilities of our lungs to generate this much stuff, like hell, if they can do a photo, it'll definitely be able to do video. I think the whole aspect just to capturing physics, it's modeling the world in the right way and stuff. Yeah. It's probably going to have some very interesting glitches the classical dolly generator thing where, you know, An extra arm find its way into the photo and, it's coming out of your armpit. That's like deeply disturbing. Like the video, we're probably going to see like much worse phenomena, okay, it'll be ironed out like quickly.
MichelleBut the fact that we did go from Dolly just a few years ago to Sora, I think that is, that's telling you a lot about the rate of change here. And the pace of development, I think it's really cool. So, so when it going back to multimodality and support, like one of the things that I remember Like having run this in, and that's like a very technical environment where people were struggling with parts of their website or parts of their code was glitching their entire site. And so we literally have to log in and sometimes do a desktop takeover actually get into their computer and say look, this is the problem.
MichelleWe can't help you. We can't touch your code because it's not, I worked for a company like GoDaddy that did tech support for. Web hosting businesses. So businesses that were hosting on our servers. And I remember that being like a legal problem, like we weren't allowed to really go in and change anything, but we could show them we could do, we could remote into their computer and we could show them like, this is where you put a semicolon in the code and that's why it's not working. We basically broke the whole thing. And so we could show them stuff like that. And I wonder if like this idea that, from our last podcast, where we talked about, the call center being potentially fully automated, maybe even as soon as five years from now probably more like 10.
MichelleBut let's say a world like that, you start to see. And you start seeing like the input part of that, like being able to take information about, or take in information about what you're seeing and put that back, push that back out and generate something that's supporting whatever you're trying to solve. All of these things seem, feel like they're coming together in a really interesting way.
NikolaSomething, the rabbit announcement is the part of this calling the universe of that, right? Where you have a controller that. Hopefully be able to do all that for you.
MichelleYeah. Yeah. What's the promise of the rabbit? You know, it takes in your requests and then it let's say you have, I don't know, like a delivery issue with a food delivery or something like that. Is it one of those things that's actually solving all sorts of problems just by being connected to a bunch of different APIs or a bunch of different, function calls for different types of businesses. Like really, do you remember what the promise of it was?
NikolaSo I'm looking at it now and I think that it's like it's there to use your apps for you. And I think maybe to abstract away some of the work you'd have to do. So you could think of it, I think, partially as a bit of an API controller, right? For all of those things.
MichelleThat's cool. That's very cool. There was a company, I don't know what happened to it, but like one of those sort of they were in stealth. They had a very big, very overhyped beta. Did you ever hear of this company called magic? Yeah. Get magic that you could text it and would act like your concierge. It's Hey, I need a reservation at this restaurant and I need to have the world's best car, at the same time, because I'm picking up the, I don't know. I'm going on like the date of my life. And so I need all these things to happen at the same time. And it was supposed to be able to go out into the real world. The
Nikolapromise was that as people do it, that over time we create like the right ontologies and things to learn how to do it with machines. And maybe that's okay. Like when you think about training LLMs, like a lot of it was done with like humans doing stuff repeatedly, like labeling it with positive or negative signals as well. Yeah. So maybe, but I, yeah, I don't know if that delivered just yet. I don't think it did, but I've always thought, and like from the time that I was at Siri onwards, like I've thought about this why Siri? So dissatisfied with it, right? And you hear a lot of like vitriolic stuff Oh, it doesn't work for this or that.
NikolaBut should be told for the things that, look, it needs to do. It actually works. You just know how to do something. So it's a bit of an interface issue. But a lot of that appeal. And I almost think that the word assistant didn't help. Because you imagine this entity that knows everything about you, that has context for your life, that it's basically a clone of you. And I think we did the whole thing, a bit of a disservice by framing it almost as a simple, task in the pursuit of AI, AGI, whatever, right? Where we were like, Hey, we just need to build that.
NikolaAnd then later we can I don't know, go solve cancer or deal with more complicated problems. Whereas the truth is, if you were to have anything like, the ideal end state of Siri, which kind of sounds like magic. And I guess Rabbit is working to support their future. I like what they're doing. Cause that's like a step that you need and like a way to collect data and create the interface, which is actually useful. But I think that it's almost an approximation of you and the judgment of quality. For that assistant is really much more about did it do what you would have done and then probably have even more inflated expectations because at least, it can go and look at every airplane, like every way that you could get from city A to B and, should know that you maybe don't like early flights, but you don't mind layovers or your mind layovers.
NikolaAnd it's just a really difficult thing to get right. I know whenever someone else is doing planning for travel and stuff that I get annoyed, and I would have probably been done a worse job myself, but it's very easy. It's hard to know, like whether it actually did a good job. But that memory, like knowing your preferences and all of that, and that's coming to the open AI thread around memory. Like it needs to know a lot about you, right? What, what you would have done in certain situations and collecting those kinds of subtle preferences is difficult. And even if you have an EA, and you work with them, I think that it takes a long time to actually like.
NikolaDelegate things super confidently because they don't know priority of different meetings. They don't know the importance of this person versus that they don't know that you'd maybe happily skip this one meeting to go have a beer with this person when they're in town and maybe you would never do that otherwise, right? And it's very highly contextual, probably depends on your mood on a given day, right? And that just then becomes damned if you do, damned if you don't, right?
MichelleWhat's interesting, you mentioned an EA, so an executive assistant, and what's interesting about this and why I think Siri was a necessary stepping stone on the way to AGI and the way to what Rabbit's paving the way for is that people don't have real world context on average for an assistant. They're not used to dictating text messages to somebody. They're not used to like having something that understands the flight, the best flights that they like to take or, what they're willing to do on a day to day basis or from a calendar perspective. So, And probably the same for your for that bank that wanted that avatar.
MichellePeople aren't used to seeing the support representative when they're making a call or they're chatting with someone. So that's not a real world context that adds value in that scenario. You have to teach people to want that. Fortunately, in my opinion, that, that was never taught. I wouldn't like to see a digital avatar while I'm chatting with somebody that's, that doesn't add value to me, but a Siri will. And does, and has continued to add more, in my opinion as it's progressed. And I think probably the same is true for what Rabbit's doing. If we think of, they've called it R1, presumably there's an R2 coming at some point, but the idea that they're teaching us how to Interface with something.
MichelleThey're teaching us about human and machine interaction that we did not know before, that we didn't have context for in the past, whereas you have experience with E. A. S. And you want them already. Already. You have the expectation. They should be able to think like me in this moment in this context and then bring me a response. I would have made whereas I think the average person doesn't, which is why the consumer perspective of open a I and rabbit. Yeah. Are so interesting. It's so interesting to watch for as a business building for businesses.
NikolaAbsolutely. I you're exactly right. And, like for us, a voice assistant for an enterprise doing a task at scale can do really well and you can optimize it and create a really good thing. But yeah, on that personal consumer level, I think like the basic version, Siri was for some things, it's as you said, who actually has someone typing out their text messages? And then you judge it against ideal performance, it misspells something and then you're very frustrated. And honestly, like you probably just shouldn't do that anyway, because it's not the best, use of it, but when you mentioned magic, that's exactly the promise.
NikolaI think it's just we're very different people. And you're spot on the people have different expectations of what they would like to do. And then the. becomes can it get you in the loop in a way that still makes the experience a lot better for you? I don't know, kayak and not Skyscanner and one works better and you're like, Oh, but it didn't do BA and I have BA like loyalty status and just the whole thing collapses, even though it might've done like a really good job. Yeah. It
Michelledoesn't know me. It doesn't have my context. Which is what makes this memory thing so interesting. I like coming from like a customer data background, the thing that. Is interesting about this or like super, super interesting is opening. I like one of the fastest growing, if not the fastest growing consumer products ever. It now will be a huge treasure trove of first party data straight from not even the customer's mouth. It's almost first party data back in the day used to mean, well, you gave me your name, number.
MichelleIncome email address. Now it's going to mean I know your kid's name. I know what kind of school you want them to get. I know your aspirations. I know what you dream of. I know what you need therapy about. Like I know everything about you and
Nikolaall I see is color science. Yeah, I think it's like advertising goals, right? Just that pictorial representation of michelle. And the best thing is there used to be a lot of talk like differential privacy and stuff like that, like this doesn't have to have your name tagged against it. As long as you live in the part of the vector space that says, advertise this to, to, to Nicole and this shell, and it's a very highly dimensional vector space. It can capture things where we have similar preferences and different ones. And it's going to be like a really valuable thing for advertisers.
NikolaAnd maybe we're just cynics here because really what it's about is getting you closer to that, like perfect assistant. Can have evidence. But yeah, someone has
Michelleto make money off of it. Exactly. And selling a bunch of stuff, paying for all the data footprint that it's generating.
NikolaYeah,
MichelleI think I know, but I do think it's really interesting. Like the idea that in order to provide and it's, and this is like an issue that I think most, yeah. Anybody who's thinking about customer experience is grappling with anyway, which is in order to provide a great experience in order to predict anything about what you want and what you, what we think you should do next, you need a tremendous amount of context. And the way it was done before is you tell us a bit of information and we'd extrapolate on top of you, what, what we think you want and who you, who we think you are.
MichelleBased off of profiles that are really just in aggregate. It's not the promise I think of a purely, like perfectly personalized, perfect understanding, perfect context about an individual. I'm not sure. No. I'll
Nikolagive you a counterexample, right? So, I'm in a hotel room. I came here, relatively new TV. So what do I do? I log in on YouTube. Because that's basically my personal feed of things that can keep me occupied for 10 hours. It's there, right? I don't want it to know that I'm, like, a Serbian man living in the UK in my 30s because I don't want to watch another basketball clip about Nikola Jokic. I don't, right? But, if you show me everything about, I don't know the Russo Ukraine war and other things that I've spent time watching I want to see that in my feed, right?
NikolaBecause that's something that I'm following very actively, right? Yeah.
MichelleKnows that based off of your behavior which makes
Nikolabrowsing history. I think if someone deleted my YouTube browsing history, so my recommendations get worse. I would feel separation from a body part at this point. So I'm on the side of have my data, if you're going to give me better products, if you're going to use Google maps without a history and they're just selfish because they're not contributing to our hive mind.
MichelleLove it. I love this. So that's the message to take away from all of this guys. Like everybody needs to use all of these.
NikolaDisclaimer, PolyAI will not use your data in this way. And we're actually extremely data friendly and we do not store any of this data. But I hope what PolyAI does for the benefit of future consumer products.
MichelleYeah. And the nice thing is For the most part, this idea that your chat experience is now going to have memory will not cost you more as a user. It will give you a much better experience. You can see already how it would you just carry forward conversations or thoughts or ideas that you're having. There's a million ways where we can see this kind of get better over time, but there's a reason why I think. Things like a consumer level, AI products are so important, especially when you're thinking about them and how they change the culture, how they change the perception of AI from the consumer first so that enterprises can use them to drive, right?
NikolaI think that, Our small part in doing this, I'm just. As an example, we launched with a bank in a country where this might've been the first voice assistant. And initially, we saw like changes in behavior that we don't typically see like in the English speaking world, where initially there were first like some silences and then like later, like there's dropped and the engagement rose and rose as people that had their second, third interact interaction with the voice assistant. And that's just like reformatting consumer behavior. Yep. And we see much higher initial success rates actually in markets where people haven't had previous voice of your experience, like I remember listening to some Polish schools where like people going immediate life story, it's not like your average American where one in five people scream for the operator immediately.
NikolaAnd does just look, a different starting point.
MichelleYeah, it's interesting. Like we've broken the trust. I think a lot in the U S around, data and the promise that it was going to make your life better around IVRs and automation, the promise that you were going to get to your answer faster because we sacrificed, I think, for a very long time, the quality of that interaction in favor of the speed. and the operational efficiency. And I think that now that we're starting to flip and there are companies like us that think about it in the reverse, which is like, how can you maintain that, that standard? The trust should theoretically go up, but just thinking, just imagining, which is something that you know, that, that example that you gave, you don't have to imagine anymore in a world in which that trust was never broken.
MichelleYou can probably build it up a lot faster by having these great interactions where people are like, Oh, this works. Like this is not what I was expecting.
NikolaYeah. And I think, it's it's. It's prevalent with technology, like around the world, right? Places that never had dial up and started with ADSL, have had much better internet for a while. And then like other places got fiber first because they were fed up with like weaker previous generation. So yeah I think interesting equally, I think there's a large part of the planet while the excitement of chat GPT is huge, especially like in California and the UK and these costs I think there are like huge segments of society that are not actually like using it in their daily life. And starting to my wife, she was at work and just testing it because she has access to it.
NikolaShe's summarize daily news. It's oh, I don't have this information that she's it doesn't work. And I'm like, it's but it can do so many other things, but it doesn't work. I think we forget that, right? Like you asked something obvious. It's an omni powerful AI. What are today's headlines? Oh, I'm sorry. I'm trading on data. Like as far as consumer goes. It doesn't work, right? So I think we, as technologists and stuff, we get overexcited and we project. But I think, there is no calibration from user feedback is important.
MichelleThere is no doubt. There's no doubt. I think just, even though that Reddit user that talked about the fact that text to video was around the corner, I think equally we could have been humbled by the idea that, Yeah, sure. Maybe it is, but it doesn't work, and that being the end of the conversation. I think that's, we've all been humbled by thinking, oh, this is just right around the corner or
Nikoladriving above all.
MichelleAwesome. Well, looking forward to the next conversation. We talked about what else is around the corner. What other impacts we're going to see happening from AI to CX. It's always great talking with you, Nicola and talk soon.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...