Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

042: Breaking Bots - Navigating AI deepfakes during election season and beyond

Aug 2, 202418 minAudio
ShareXLinkedInFacebookWhatsAppEmail

From political campaigns inundating platforms with AI-generated content to the intricacies of voice AI and deepfake technologies, Damien and Nikola offer practical tips on distinguishing real from fake. Learn how pronunciation differences, odd question prompts, and latency can help you identify AI bots posing as human callers. This episode provides valuable insights into the evolving landscape of AI in politics and commercial telemarketing. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Nikoladesign wise, if you can't interrupt it, then you're like, oh, okay, this is automated, right? And when you build automated systems, it actually, for the most part, that we can see in our millions of interactions, it makes sense to not allow barge in for most actual pieces of things you're going to say. Because if you allow easy interruptions, it can really take the conversation.

DamienHello friends. And welcome to another episode of the deep learning podcast with PolyAI. I'm your host, Damien. I'm here with Nikola. We are actually in the same room, just a couple cubes apart. Nikola, I'm in London. Thanks.

NikolaGood, good, good, good. Yeah. I don't think we figured out how to set up one room. So we still use, good to have you here.

DamienYeah, definitely great to be here. And it's really, really exciting to get away from the states because now we are in the thick of a political battle of effort proportions. As you may have heard, we're fast approaching the 2024 presidential election. It's a hot one, a lot of spice, a lot of spice in this race.

NikolaYeah, I flew in from Chicago yesterday. What? Let's be reunited here. But yeah, it's nowhere near as exciting. We have a new prime minister here. And self proclaimed centrist without radical beliefs who wants to unify the nation. We'll see how he does. But we're out of the throes of politics for now.

DamienYeah, for now and enjoy that love. Meanwhile, Back in the States, we are getting inundated with messaging on all platforms, campaigns are in full force, and of course, that leads to the possibilities of pop ups here and there, showcasing how videos are being deep faked with some candidates saying things that they never said, et cetera, et cetera. It's putting a whole new lens on the application of really progressive artificial intelligence. It's video models on an omni channel front.

DamienBut part of that is, it includes getting those calls from campaigns and PACs that are essentially trying to convince you to go for their candidate. Here's the issue though, it's now becoming indistinguishable from playing sound clips of things that candidates may have said, but actually didn't say. And so I thought a good conversation topic for us to chat about today is, As a leading mind in voice AI, conversational AI, and obviously what we do on the commercial front, just giving folks some tips as to how they, if they suspect that they're receiving a call and it's coming from a potential.

DamienVoice AI bot, how they can go about breaking it to see what's legit and what isn't and filtering through the deep fakes. A lot of states have already put some legislation in motion that apply really severe penalties to any organization that's caught deep faking video. It's also applying to voice. So from an omni channel front, In your opinion, why? It's obviously easy. It's accessible. What are some other reasons why this is running so rampant?

NikolaListen, when there's technology, people use it all in all sorts of ways, just because they're interested in trying it out, right? I think in America, it's always like the land of advertising, and at the forefront of it, it's just another form of ads, right? Like, why not, right? I was always surprised with how much more effective telemarketing was in the U. S. relative to Europe, right? And Despite the numbers of calls that people are still willing to stay on the line, right? I would have thought people just hang up immediately like for pretty much any number. You don't know that's what I do

DamienThat's definitely what I do. A lot of people don't though, especially older people who are voters

NikolaYeah, that's I think the crux of it, right? I think like as much as we have Committed our lives to creating technology that is virtually indistinguishable from humans because we think that, for us, the mantra was always like, don't trick people into thinking you're a human, help them get the change, right? Make it smooth list, make it. And like that pursuit is fun. But when it comes to these kinds of activities, I think if you want it to be really nefarious, it's really the cocktail of, virtually completely human coupled with.

NikolaWhere things get really, really dodgy, right? And the demographic you mentioned, right? Older people who might not be at the forefront of what's going on, they just don't even think to think about whether it's real or not, right? And I think that's at the core of like how it can be abused, Because if you don't think it can be fake, right? And then you, Hear it and the voice sounds really real, like park that whole, like doubt suspicion, you park it away. You don't think about it. And the truth is then when you engage the system will probably respond.

NikolaOkay. Yeah. And then you tell us, right? It really depends on the design of the system around things you can and cannot do. I think we can probably list a few things that you can do. To see. Okay. If it's human or not, like to me, the most obvious one let's take like pronunciation, think of like Birmingham, Alabama versus Birmingham in the UK. People say it differently, even though it's spelled the same. And I think if you spoke with a human and you said, Hey, look, I've been getting a lot of robocallers.

NikolaI hope you don't mind. Would you mind saying Birmingham for me? And if it's a prompted LLM, it might. Be smart enough to go, and say Birmingham, but do you go no, no, no. Can you say it the way Americans do Birmingham? And if they're using speech recognition, they're translating that into, That thing, the model is not conditioned to do that, not yet. And this is where I think like you have a modality discrepancy between text in text out LLM powered by speech recognition that doesn't encode your stress. How to say we won't get there.

NikolaI'm sure we will. But we're not there yet. Similarly, we could play Hey, my last name is Mrkšić. Can you repeat that? And of course you can't. No one can. Maybe a few Serbs and Croats. Right. But you're going to like, Struggle and like how it struggles because you'll get a mis transcription, right? And it will say some other word and it's clearly not that a human knows that it's not that To a human it's obviously a weird slavic name, right? Okay Oh, i'm just gonna have to butcher it up a little so I think that's the simplest failsafe, right?

NikolaBecause the model will just get very confused the other thing obviously Seems

Damienlike it Yeah. Yeah. Latency, just one second, but it seems like one of the reasons that that might, aside from the dynamics and the actual like text to speech and speech to text and a loss of like the stress capabilities is that it's also a pretty odd question to ask. So if you get a response from an LM, that's essentially just okay, I'll try again. And they're not at all put off by the fact that you're inquiring about it. That to me. Is another indicator, right? Ask a weird question. See how the LLM responds.

DamienIf it responds as though what you asked was perfectly normal, then chances are it's automated.

NikolaYeah. Yeah. But I think, like once people are doing the obvious things there, part of the problem says, if you're asked a weird question, that wants to, and the safeguards or whatever LLM is being used fail or don't exist then, that's something we can work against. Hey, why don't you ask me that? Do you think I'm a bot? Oh, okay. Fine. Your test fails, right? Maybe. The other thing, obviously, there's a certain clunkiness to voice interactions still, right? Where, you have a design choice, for the most part, between allowing easy interruptions.

NikolaAnd if you say anything to the background, say you were talking to me on the phone and I'm with someone in a car and I go yeah. You hear that and you stop talking it's obvious that, the reaction is something or whatever, you'll get that you'll model out that they're probably saying something to someone, you caught them on the phone, you understand that again, that ability to park that in your mind, just continue talking the way a human would, right? Versus being interrupted. That's one way. Now, if you can't interrupt it, In design wise, if you can't interrupt it, then you're like, oh, okay, this is automated, right?

NikolaAnd when you build automated systems, it actually, for the most part, that we can see in our millions of interactions, it makes sense to not allow barge in for most actual pieces of things you're going to say. Because if you allow easy interruptions, it can really take the conversation. But a human would, that's one way in which we're still better than like the best machines running out there.

DamienSo is that two ways? So obviously there's the element of if it doesn't allow you to interrupt, then that's one. But what about it interrupting you? Or if you're having a conversation with that person in the background and it thinks that it's still receiving, interrupting you. Input. Is that a current like trigger? Would you say that's not, or that there are mechanisms that get beyond that? If you get confused, it's Hey, sorry, what are you saying?

NikolaWhat do you mean with that? I guess that's confusing. So I don't know if that's like an easy to apply test. But you could definitely just think about if you can't interrupt it to save your life, then it's probably, and how you interrupt and how it reacts to interruptions. Again, like a thing. So it really, I think it depends. The first thing we talked about, it depends on whether you're okay. Like potentially embarrassing yourself by starting to say random things, right? they're like, I have no problem doing

Damienthat. Anyone who's a fan of the podcast knows that. Yeah, yeah. Why

Nikolaare you asking me the, what is the capital Rwanda? Like what? Or if it knows an answer to a particularly weird question that's also a bit suspicious, yeah, very

Damiensuspicious. That's a really good point.

NikolaYeah. It's yeah, I think like you can have a lot of fun with it. It's really cause it's very powerful, but it's also quite crude and where it's been placed. So it's apt to ridicule it a little and mess around. And like you said, if it complies or just doesn't get weirded out in a way that a human would. Tough. You've been called out, right? And like the barrier will get tougher and tougher, right?

DamienWhat, now you, we talked a little bit about this with regards to barge in and being able to detect when there's more input coming in and like active listening as far as bots go. But that also lends to inappropriate latency, which is another indicator. What, what are we talking about? It's humans converse and they talk like there's a real fine science between knowing when to shut off the listening ears and. Turning on and you and I, we banter, we talk over each other a lot, bots really can't do that yet, but it is getting more and more difficult. What are some latency flags that people should be looking for when they're having these conversations and are suspicious?

NikolaThis is where I think we're not actually. Like that, it's not one that can be used reliably because you can't configure these systems to have very low latency to be very sensitive. I think you were saying something else yep. And that encourages you to keep talking. That's not something that we've built into these systems just yet, right? To have a stream where we model in your feedback while we're talking. The other thing of course is if they're taking very long to respond I think you're going to have, I think for us that is probably what gives away a lot of our systems, right? People see that there's a guy, but it works.

NikolaThey just interact with it. And that's the implicit signal that it's not human and that's okay. You can tweak it the other way to be hypersensitive and reactive, which will lower latency. And if you do if you're in a quiet room demoing, like Siri. If you saw the opening on demo day where they were demoing low latency with a guy to plug this phone with an ethernet cable yeah, okay. You've cut, well done. You've cut down one roundtrip time, right? You can demo those things, but they really break down in production. If you're in a car driving and like you hit a bump, you hear a sound and things stop talking, like you know, but now we're talking about things that I think we're pretty far from needing to detect that.

NikolaMargin and they continue talking for another five, six, seven seconds. There may be people who do that on the phone, but

Damiennot a lot. Not when it's an inbound call of someone trying to convince them of something, yeah.

NikolaYeah. And, but, like there are situations where, for example it's an offshore contact center running through VoIP. That's not configured particularly well. Like you must've been on phone calls where you've had latency with someone, right? Sure. Video calls. It happens all the time with I think with Google meets, right? Like I think this week alone, I had a few one on ones where I'm like, okay, my video, your video, let's turn it off. Yeah. And there's a two second lag, so we learn to not interrupt and we talk as if we're sending emails. Oh, wow. That can happen. And if that could happen with a contact center, especially, complex routes through like different seacasts environments and handed off to a third party contact center that again, routes it through its own telephony, it can latency can add up with humans as well.

NikolaSo I don't know if that's completely fail

Damiensafe. But then there's what happens after that initial latency bump, how it's identified on both sides of the call, et cetera. And you could trust your gut to say, oh, this is a human reacting to latency issues along the telephony stream, as opposed to this is something that's been programmed by an l by within an LLM within a bot to respond in this awkward way that most humans would if there were connection issues.

NikolaYeah, that's right. I think it's a it's a brand new world. There will be more and more of it. Sure. If you don't suspect it then, you can get tricked very, very easily, right? I think we have a number of calls where people really, start off thinking that it's a human. They go, oh, are you a robot? We also have calls where people start off, they hear it's automated, but then like they hear a few things that sound really like incredible and they go oh, i'm, sorry I thought I was speaking. We're human. I've been really rude. I don't know if I ever played you those but It's interesting, right?

NikolaAnd

DamienWhat about asynchronous stuff? What about pre recorded content that it's, that eschews any ability to break a bot, right? Because there's no engagement, there's no interaction, there are no conversational turns. Are there some audio cues that right now you can try and fine tune your ear to differentiate, or is it becoming more and more impossible?

NikolaI think it's getting borderline impossible. I think you, you can probably do forensic analysis of it where you would find patterns that are generated that like an expert could identify, right? Like they do with you, when people would figure out that a photo has been Photoshopped because some pixels look weird. But that's not something you can do with the naked ear. I, we had a deep fake of my with 10 samples. Yeah. I got three out of 10. It took me a long time to figure out why it was, why am I so bad at detecting my own voice, right?

NikolaAnd the truth was the clone was tweaked from like a US English model, right? Despite living in London, I never really picked up any shreds of it. Of a UK accent. And the model, the generator model spoke like American English with a more American accent than me. It was me like five years in the future when I, tweak my pronunciation further, away from my like Serbian base. And I was like, why am I getting it wrong? And we did it a few times and I kept preferring, I preferred the generator version to my own recordings.

NikolaAnd the truth is it just had less of an accent, right? Accent versus accent, right? Like the versus just roughly that, it just basically made me a better speaker of English. That sounds

Damienlike it could be a tool to benefit a lot of candidates as opposed to using it for

Nikolavarious reasons. There are companies doing this again, like latency is a bit of an issue. I find it to be a little racist, right? I think accents are interesting, right? I think a lot of the applications of this are offshore contact centers with speakers of, a different form of English. I think a lot of them are in India, right? The demo is typically someone speaking Indian English and being converted into a generic American, right? And I don't care much for that. But it's it's interesting, it can definitely be done.

DamienThat is really interesting. But again, going back to the affectation of accents, like very subtly, but just making maybe a person, a personality, a voice, a little more compelling and making it more targeted to the person who's listening, like thinking of instead of, using a pre recording of a voice or a clone voice of your companion. Of your opponent and having them say something nefarious you clone your own voice and then make it a little softer, a little more regionally appropriate for your accent to whatever phone bank you're calling into. That's interesting.

NikolaYeah. You could, you could, I don't know if it's needed, if our lives would be any worse with or without it, but Hey,

DamienBut we also have a hard time identifying our voice, right? So do you think your, do you think your wife would have gotten a better score on hearing your 10 clone samples?

NikolaI'll try. I'll let you know in the next one. I don't know. I don't know. I really, like the other thing, of course, that we did in testing, we did it over the phone because we really care for the phone, but we do implement things around over the course that basically low band filter that you apply. Makes the voice poorer, right? The signal is weaker. It's not like this microphone. It's the phone, right? Going through, like we noticed, for example, degradations of performance. If you call through you know how applications have that, like global calling thing where you can pay Viber or other apps to call any number in the world.

NikolaBut it goes through voice over IP, right? So voice over IP, that part uses a different filter to like national telephony. And these days, even the telcos are probably putting your voice through VoIP, especially if you're calling something abroad. That's able to often, get much lower rates. And while like roaming rates are free. Yeah. They've basically gotten to the point where. A U. S. to U. K. call is not really going through an expensive telephony network where you have to pay that much, right? So it's only really their choice if they want to tax you for doing that, right?

NikolaAnd that's why, and then what's interesting also is that national telephony they don't use that. I think there are two or three different kinds of these filters. So if you apply them all you lose further clarity and accuracy. So the more you do it you're used to a phone line being, weaker and the signal not being crisp. And you know that feeling when there are two good camera setups and you hear someone really well, you see them all and you're like, wow, this is Better than usual, right? I can really hear you, right? If I used a microphone the way you always tell me to for a podcast my voice would not sound better so like telephony is just and this is a good thing for fakes.

NikolaI'm sure it's a good thing But like for them to seem more real it's harder to tell because you're already like Inflicting a lot of like loss onto the signal. So it's hard to tell

DamienBut maybe that is an indicator itself if it sounds too clean If it sounds probably at a higher level, then you could be dealing with a pre recording or a synthesized voice. That's incredible Hey, look, I know we're running out of time here This has been really, really fascinating. I hope that this offers some tips for folks who are getting those barrages of calls for political and other reasons. Hopefully it's not too bad, but as the technology gets better, of course, there are going to be better mousetraps. Appreciate your time, Nikola. It's been a lot of fun and we'll talk to you next week.

NikolaYeah. Look, we're just don't pick up your phone until the fifth of November.

DamienAlways the best advice. All right. Take care, man.

NikolaCheers. Bye.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode