Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

019: How practically safeguarding generative AI works IRL

Feb 1, 202427 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Kylie Whitehead and Meghan Berton discuss the importance of implementing guardrails to limit a system to its intended scope and ensure the generated outputs are safe. They also touch on topics like user input filtering, hybrid generative AI solutions, hallucination management, and preventing offensive or biased responses. These and highlighting challenges and opportunities of using generative AI in voice assistants and the importance of continuously improving and refining the technology - in today's episode! Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Meghanit's a science developing the guardrails, but also an art because we don't want to Lose that inherent beauty of generative responses where we can encourage creativity to help better support customers.

KylieHello, welcome to Deep Learning with PolyAI. I'm Kylie Whitehead and I'm joined today by Megan Burton. Megan's a product manager at PolyAI and she focuses specifically on generative AI and machine learning. And we've been talking a lot about generative AI guardrails on the podcast lately. We've had a few questions about practical applications. So I've asked Megan to join me today to break down what Gen AI guardrails actually are and how enterprises can implement them. So Megan, thank you for joining me today. Perhaps you could start by telling us in really simple terms, what people actually mean.

KylieWhen they say generative AI guardrails.

MeghanYeah, of course. Thanks for inviting me. Kylie. Great to be here. I would consider generative AI guardrails is just thinking about how we want to limit a system to its intended scope. So, what that looks like is building measures or controls. To guide the system to certain outcomes and make sure we address anything that's unintended. So you could consider 2 pillars potentially for generative AI safety guardrails to take place throughout the different layers of the tech stack and that would be thinking about filtering user inputs, particularly malicious inputs and then controlling and making sure the generative, generated outputs are safe as well.

MeghanAnd then beyond that, at PolyEye goes into designing the entire system. How do we ensure for data security and protection? How are we making sure the voice assistants are instruction following and even more beyond that?

KylieIt's the way people say. Generative AI guardrails. It just, it makes it sound, it makes it sound simple. It makes it sound like putting the bumpers up at the bowling alley, right? Like it's just something that you just do like that. And I think an example might be that if you're putting together a generative bot, maybe you think you can just say, don't ever swear. And then that'll be it. It will all just work. But that's not true, right? People seem to be getting that wrong.

MeghanNo, and I think we've seen that in a previous podcast. We talked about some of the mishaps of organizations implementing these systems, and there weren't necessarily the best or rigorously tested guardrails in place. There's actually a lot more that goes into it than some simple prompting. So you're absolutely correct. So you

Kyliespoke about So let's start there because anybody who's worked in a call center or has launched a voice assistant in a call center will know that people phone up and they say all kinds of things, not all of them nice or helpful or some of them even malicious. So. What kinds of things should enterprises be thinking about putting in place to handle different kinds of malicious or unhelpful

Meghanuser inputs? Yeah, absolutely. So you could think about user input filters, and I think one of the benefits why we consider hybrid generative AI and conversational AI solutions is so we can monitor for those inputs and then decide how we might want to handle how we generate or whether we produce a predefined output. User inputs. We might want to the inputs are toxic, abusive, harmful. And in those scenarios, we don't even want to pass the user input to generative AI.

MeghanWe just want to block those inputs and back to a predefined response. So that's one of the ways in which we might want to handle some user input. There are other cases as well that we might want to build in other particular scenarios. They're a particular client is concerned about competitor filtering and competitor mentions, or in certain cases, maybe a use case for hotels might be emergency detection escalation. We want to make sure that those user queries are put down the right path and we can handle them with absolute certainty.

KylieSo when you're deciding, so you can switch in the middle of the conversation, depending on what the user says, you can say, yes, handle this with generative or no, handle it the sort of traditional way.

MeghanI think it goes into understanding. Conversational AI is not new. This, you've been at Polly for such a long time working on amazing solutions. In certain cases, there's benefits of conversational A. I. And we might want to use that solution. In certain cases, we might want to use generative A. I. To answer users question because of its respective strengths. And I think that's the beauty of our managed services not to sell our managed services, but just saying that we can seamlessly switch between these two technologies.

MeghanAnd I think that also goes to speak. Generative AI is not necessarily new for Poly as well. We were exploring how to incorporate generative AI into these solutions before the hype cycle, so it's cool to be able to switch between the both. It's like working on two absolutely similar technologies and systems at the same time.

KylieYeah, I guess you get to have all the benefits of generative, which that it, the bot can sound much more natural, right, because it can make up words on the fly. It's not restricted to. Sentences that it has been told to reply with. But then sometimes you just, you need that safety and you're like, Nope. Say exactly what I tell you to say in this situation. Go down that route.

MeghanYeah, exactly. It's it's thinking about what's my particular use case? How much control do I wanna have with the system, and am I willing to. Relinquish a little bit of the predictability so that I can produce, for example, with generative AI response. That would be really contextual and personalized for the user. And so it falls into it's a science developing the guardrails, but also an art because we don't want to Lose that inherent beauty of generative responses where we can encourage creativity to help better support customers.

KylieAnd it's so different for different businesses, I suppose. If you were say you were a trip, for example, you might want to. Give all the access to all of the reviews. You might want the bot to be able to say things that if you were a specific hotel, you'd be like, Nope, don't go near that. Don't say that. And so I guess I guess you work with clients to make sure that the solution fits what they're looking for, what makes sense in their business.

MeghanYeah, exactly. I could think of maybe, for example at Vox, which was a great event where we invited some of our clients to talk about their experiences. And we also talked about some new products we're releasing. I did a demo of a generative AI agent and we called a hotel agent that was able to on the fly, answer a contextual question about whether I could bring a steak knife to the restaurant, which of course it's just, yeah. In certain scenarios, predefined intent based flows, they take a lot of time to build and monitor.

MeghanAnd of course, businesses are dynamic. So it's unreasonable to think that your business is not going to grow and change and adapt to its customers and market needs, and the AI system doesn't have to adapt as well. And so the generative AI solutions allow you to really quickly on the fly, help address those customer needs. And it makes you, it gives you the ability to not need to anticipate all the different reasons why a caller might call. Because I'm sure you've probably heard some crazy examples of callers calling some of our voice assistants that are quite ridiculous,

Kyliebut yeah, exactly. Yeah, everyone has different reasons for calling. In fact, that's, often the reason people pick up the phone in the first place is because the thing they want to know. Isn't an exact match for what you have in the FAQs on your website, right? It is something a bit more nuanced or they can't quite decipher the answer from that from those like written instructions. So they call up and yeah, they can say absolutely anything at all. And to be able to intelligently look at the rules and then And pick out the relevant parts is what makes generative so exciting because it's how we work as people.

MeghanI think one of the coolest things that Polly brings is expertise in what the systems were like before generative AI. So having that really good knowledge about when is it best to use generative AI, when's it best to use an intent based system and flow. What's your perspective on generative AI safety guardrails from the marketing standpoint?

KylieI think it's, I think there's so much mystery out there still. I think we've seen chat GPT and we've seen what chat GPT can do. And we think, oh, it's ready. Like I just need to give it my FAQs and it's ready. And and I think that's what's happening is that enterprises have take, cause the thing is you can build something that is pretty good and works pretty well, very quickly and very easily. But pretty good and pretty well, it's like not good enough to represent these brands that have, been decades in the making.

KylieThey've had so much put into them to build these customer relationships. And then they're coming out and saying, yes, we're useless or try our competitors. They're better. And I think that's happening because people don't know enough. About well, like a what's behind the technology and how it actually works because it is quite it does happen in a black box, right? This, all this, the models, it's all just happening somewhere else. You know that how it's. How it's happening, what's coming out of it. But also then knowing what to do about it. And that's what was, what I really wanted to talk about today and getting into the guardrails.

KylieBecause to me, it just sounds like people are going, just slap the guardrails on. You're like, okay, I just like press the guardrails. It sounds easy.

MeghanJust toggle them all on. Nothing bad will ever happen.

KylieWhen, in a couple of years of you working on what you're working on, then in a couple of years, that's far time in, but I think it will be a case of toggling it on. I think you'll get there. I think you'll crack

Meghanthat. I hope so. Well, the wonderful engineers will be the ones that help us get there, but to your point, I like the idea of it being a black box. And I, when I think of generative AI guardrails, cause we have our managed service clients and then we have Jupiter, which is our voice AI platform where. Anyone is able to quickly log on self serve SAS model. You can build a generative AI voice assistant. And I think of what are we doing behind the scenes to build in these guardrails, to ensure the system safe and act with our clients needs in mind.

MeghanAnd then the really. Is how can we surface those guardrails to other people so they can develop an understanding? It's no longer a black box. We can demystify it, but then they're also able to quickly enable those guardrails without having to dive in deep. Like some, sometimes industry domain knowledge, it's so time consuming to figure out how things work and why it works. And it's. So frustrating and everyone has better things to do than dive deep into some random solution. So that's what I think will be most exciting.

MeghanHow can we enable other people to self serve with those card rails too? Yeah.

KylieAnd there's so much talk of rag, right? So you retrieval augmented generation. So you have this knowledge base. And I think when I first heard about this, I thought, Oh, the knowledge base. Great. That's your FAQ section from your website. That's your knowledge, but it also, it needs to include what not to do, right. What the bot shouldn't be able to say or do. really need to understand when you're building that knowledge base out, you need to be able to understand everything that can go wrong. In order to make it not go wrong. Right. And I think this will become a homonetized, I think this knowledge base of, here are all the things you don't do will become a product in its own.

KylieRight. People will be able to take that off the shelf, plug in, the specific information about your business. And then you have something that is very robust and very safe, but also very knowledgeable and open.

MeghanYeah, exactly. I think to touch on, you mentioned RAG, which is retrieval, augmented generation and the knowledge base. Maybe we could dive into that just a little bit. Yes. So you could. Use like we talked about the example of guardrails and you just telling the bot. Don't do this. Don't do that. So you're just trying to prompt it. So rag or retrieval augmented generation goes to a step beyond that to introduce more levels of control and additional performance and things like that. So you're able to. Build out your knowledge base of FAQs as you mentioned or other information within the knowledge base and a retriever can pull that and send it to the large language model to generate a response.

MeghanAnd so that's a really wonderful tool to leverage, because for really complex use cases, it makes it more scalable to build a voice assistant, so you're not limited by the token limits. And then you're also able to better structure and organize your information to produce the best outcomes. And then. Beyond that, there's so much testing that's done to iterate on the knowledge base and improve.

KylieYeah I think because That's like I said with everyone saying, you know thinking that generative is possible, is real, is being deployed in the real world, but you only have to look at the name RAG, right? Retrieval Augmented Generation. So, okay, we're not quite at generative yet, we're between retrieval and generative, and where we are at the moment is somewhere in the middle, because fully generative, not quite safe, fully retrieval based, a little bit rigid, a little bit stiff. Somewhere in the middle is where we need to be now.

MeghanYeah. I think one of the cool things is that I might challenge you and say, is pure generative, not necessarily is pure generative intrinsically unsafe, or are we able to put in guardrails and produce really high outcomes and have really reliable outcomes? So I actually think more and more as the technology is advancing is what And building these generative AI systems. Do we have that confidence to say we should release pure generative AI systems?

MeghanAnd potentially there's certain cases where we might want to go down the hybrid route for additional control based on the customer's use cases,

KylieI think you're right. I, like you said, I've been at Poly AI for a really long time. And so I've been here for five years and the last year, it feels like more has happened in the last year than the four years. Before that, like all mushed together, like everything is changing. The quality of voices is changing. Like voice synthesis is changing. Generative is changing as much as possible. Everything is moving so, so quickly now, which I think makes it really difficult for people who are just entering this space, a lot of contact center leaders, a lot of CX leaders being asked specifically.

KylieLook at generative, look into generative, there's a lot of hype around it. We want it. Look at the technology versus think about the problems that you have and look for solutions. So people are coming into this space now and instead of saying, I will reduce call volume by automating calls. They're saying, My boss wants me to look at generative what is this?

MeghanIt's just, as I mentioned, the hype cycle with generative AI, it's also just such a big buzzword of wanting to include it and not understanding. I think there's this gap of where we are today and understanding. The limitations of where we are today and then thinking about where we're going in the future and how we can bridge that gap. And I think what's making it really hard to keep up is the increasing rate of change that we're seeing advancements come into play and then be implemented middle world solutions.

MeghanSo it's. It's tricky. I wouldn't envy being a customer and understanding how to incorporate generative AI in my solution. It's really tricky. Yeah,

Kylieit is tricky, but I think it, the point, the focus on what we're trying to achieve, not fully AI. Remains the same with generative is helping to push what we do forward. But it's not completely changing what we do, which is really focusing on the customer experience, right? Building bots that people can enjoy having just a natural conversation with. You don't have to say, credit card, credit cards. You don't have to talk in a really technology led way. Can you can be yourself and generative is making that possible.

KylieSo I think, if I'm talking to a prospect a customer, my advice is to really think about the end user experience, think about how you're going to get people to engage and generative is just another tool in your toolkit to make that happen. But there's a lot of other voice technologies around generative, right? Generative. Opens loads of like a whole can of worms when it comes to deploying that then over the phone because you have all the spoken language understanding things to put

Meghanaround that. Yeah, I think it's a good way to look at it in terms of improving the end user customer experience and improving. I think customer service could be completely revolutionized with this technology and improved. I think one of the other cool things we're seeing with clients is not just talking about cost savings of implementing a voice automation solution, but is there the opportunity to have revenue uplift as well? And can these assistants help you with upsells and other ways to grow and expand your business, which is it's a cool mindset, mind shift change as well.

KylieYeah. Yeah. I think we're moving into this place where instead of just automate these calls, it will make the call center cheaper. It is enable customers to have access to these like personal assistants that are going to help them make smarter shopping decisions or just purchasing decisions and help them get. Product. So you're creating value for them. So they're creating value for you. So the whole thing just feels a bit more collaborative.

MeghanYeah, exactly. I agree with you completely. How, what else do you think? We talked a little bit about the black box of guardrails and trying to mystify it for our enterprise customers and clients. What else do you think would be valuable to touch on in terms of. And as much as I, I love to keep talking about value and, but I could go off on a tangent all day. So what would be, how do we, I know the topic of this podcast is guardrails.

KylieYes, I, yeah, I know we could talk about this all day. We could just go on and on. I think I think one of the other interesting words that comes up a lot is hallucination. And I think we all know that generative models can just make stuff up. And I, that's probably, that's being a bit flippant, so maybe you could tell us a little bit about why. They make stuff up and how you can prevent them from doing that while still getting that like freedom and openness and all the things that really offers.

MeghanYeah, absolutely. I think one of the interesting things, if we pause for a second, take a step back and think about what a generative AI model is actually doing, it's. built using a large language model. So the large language model, it's a statistical model, and it's essentially trying to predict what the next word is. And they're pre trained on a huge corpus of text data depends. Which text data it is depends on the large language model, but inherently the large language model, it's not necessarily hallucinating if it doesn't respond to your client specific use case because it's pulling from its pre training data.

MeghanSo I think this is where it comes in thinking about how we can leverage this retriever technology and how we can curate and build a really exceptional knowledge base to limit the risk of hallucinations because a model may not. Necessarily be hallucinating because it's pulling from its pre training data, but it's hallucinating in the specific use for the client. So 1 of the ways we can look at hallucination management from a continuous monitoring standpoint is almost.

MeghanI would think of it as a citation. So whenever the model generates a response, can it tell us where it took this information from in the knowledge base to generate that response? Really quickly, we can use analytics and understand, okay, in this subset of calls, how many might be, and there might be some false positives as well, but which can we quickly pinpoint as potentially having hallucinated in terms of not pulling information from the knowledge base.

MeghanSo that's one of the things with hallucination management, but it's definitely one of the biggest topics in gen AI safety and a focus for us on building guardrails. Yeah,

KylieYou, God, so many interesting things that I don't know where to start. But I think one thing is the term hallucination, right? We can't help when we look at AI systems, we can't help, but personify, we think they're working in the same way that we're working like that. They're either making it up or they know the answer and they don't know the answer. They're making it up, but you're right. What's actually happening is they're just trying to fill in the next one. The next word, the next few words in the sentence, which is, so it's not really, they're not hallucinating, they're just guessing wrong.

MeghanExactly. I can bring it back to when we talked about the demo I did at Vox with the generative AI voice assistant. You could call that a hallucination in a sense too, when I asked, can I bring my steak knife and it countered and said, no, we have the best steak knives at the restaurant. Does the restaurant have the best steak knives? I didn't tell the assistant that the restaurant has the best steak knife. So it's something it's inferred that's likely the case, but not necessarily. So I think it's. Building these systems.

MeghanIf there are certain cases in which you can anticipate, there might be a hallucination because of a lack of information. So I want to specify it because we don't want the agent to misinterpret the user's query. Then building that in is really important. And. One of the things with GenAI safety guardrails, we're considering how do we build the best user interface on Jupiter, which is our self serve platform, to build your own knowledge base, to help prompt you to put in that information to achieve the best outcomes from your voice assistant.

MeghanYeah.

KylieYeah, amazing. So how do you, so you you can take steps to prevent it from hallucinating or guessing incorrectly the next thing that should say, how do you prevent it from saying something offensive? I think you touched on competitor, competitors earlier. So how do you stop it from, it could say, it could start swearing. It could just say something really offensive and wrong. It could say, Oh, this competitor is really good. How do you stop it from doing that? Yeah,

Meghanso I, there, the certain large language models we use have moderation endpoints and have content filters and controls built in. So things that sway really far to one extreme of really abusive toxic content, all those things. Are protected from the best practices of these large language model providers, which is great. And then we have the poly AI guardrails. So user input filters, we build in that are going to identify what the user's query was and say, we don't really want to pass this to large language model.

MeghanWant to generate a response for this. We simply want to block it and not respond. So that's how we would be doing it as well. Did that answer your question? Yeah,

Kylieyeah, it did. And I think that's really exciting and it opens a lot of doors in terms of what we do as well. I think I remember an example years and years ago, we were building a bot for a restaurant. And we found during testing that you could say something to it and it would make an inference that you just didn't really expect. So I'm going to give the probably, I'm going to try and think of the least offensive example I can give, and I'm sorry to anybody that this offends. But you might say. My friend is French and it would say, Oh, our corkage policy is X, Y, Z, because the model understood that French was most commonly associated with wine.

KylieSo then it just starts telling you something that it thinks is relevant. And I think actually, the way that we're able to build up guardrails with generative gives us much more control over how we prevent things like that from happening, over how we make sure that the system is less biased, because we can be more specific in telling it, don't do that. Yeah,

Meghanexactly. And I think to your point, it's an art and a science. When we talked about in the beginning, just adding in no, please don't do this, like prompting and iterating on that prompt, rigorously testing, identifying those edge cases for the client specific use case. Those are all things that are super important to do. And so you can develop such a beautiful, exceptional prompt. And over time you have to continue to iterate. To make sure you're addressing sort of the changing user queries you're seeing.

KylieYeah. And I think even, I didn't realize this, but Arcadius, our lead dialogue designer was saying the other day, that even the way that the prompt, the order you put the words in or where you put the subject, where you put the object impact, how effective the prompt is. So, and that's, there's no sense in that, right? You just have to, to trial and error.

MeghanYeah. No, that's what it's exciting that there we have our best practices today, but it's a moving goalpost. So, as we uncover new information, we're identifying certain limitations and strengths of these models. We're establishing new best practices. So I think that's what's. What's really exciting about being at Poly is that everyone across functionally and all the different teams is working to help with that and make sure these models become even better and our voice assistants leveraging generative AI or hybrid are even better and more safe.

MeghanOkay,

Kyliegreat. Thank you, Megan. I think after our conversation today, it feels that there is so much that can be done to make generative AI powered bots safe and brand worthy, right, that can really, you can put them out there and they can represent your brand. But it takes a lot of work. It takes the guardrails. You need to know what you're doing. You probably don't just want to use something off the shelf at this point. There's a lot more work to do. That's going to change really quickly. possible today, it's going to be easier.

KylieTomorrow it's going to be easier the day after and so on and so forth. And I think that the opportunity it can bring to brands in terms of building something really personalized, really really helpful and valuable to customers is going to be. be really powerful in the future. Thanks to the work of people like yourself, Megan and the teams that you're working with every day. So thank you so much for talking to me today. You can see other episodes of the PolyA podcast on YouTube or wherever you listen to your podcasts. and until next time, we'll see you soon.

KylieBye.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode