Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

033: PolyAI's Series C Raise & GPT-4o

May 9, 202424 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this episode of the 'Deep Learning with PolyAI' podcast, Nikola and Damien discuss PolyAI's recent $50 million Series C funding round led by NVIDIA's NVentures, alongside other notable investors. They explore the company's growth, innovative engineering, and the future of voice AI technology. The conversation also touches upon the recent developments in OpenAI's GPT 4.0, including its ability to handle voice interactions and the technical challenges of latency and interruptions in automated systems. The episode wraps up with a discussion on future product announcements and the broader implications of voice AI in customer service and consumer applications. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

NikolaThe low latency you see there, it's not just about how fast your model is. It's really about how you're tuning and priming your system to respond to periods of silence. Are you catching your breath? Do I need to jump in and say something? They did a really cool trick where they. Stop and listen for a second. If you continue, they will roll it back. That's cool. That's like true, like hackery innovation, which is why like open access, great company, right? It's not just the fundamental models. There's like just a mountain of really good engineering in there.

DamienHello, and welcome to another episode of the Deep Learning with Poly AI podcast, the only podcast who at this very moment is about to announce the significant Series C raise, thanks to the hard work and efforts of an entire team spearheaded by the CEO and co founder. My guest today. Hey Nicola.

NikolaHi Damien. How are you today?

DamienOh, no big deal. How about you?

NikolaPretty good. I'm pretty elated today. You've been busy coordinating, fundraise announcements, PR news wires and so thank you for your hard work.

DamienOf course, of course, let's just dive right in. This is a massive day for us here at Poly AI. So normally we like to talk about some pretty high level trends and we'll get to that a little bit later in the episode, but now we're going to do a little self backpack because we just moments ago announced the closing of a successful series C round to the tune of 50 million raised Nikola, this was your baby. It has been. Tell us about a little bit, how we got to this massive, massive milestone.

DamienAnd what the last year had for you.

NikolaYeah, absolutely. Well, look, we just announced our 50 million series C with NVIDIA Ventures, with Heda Sophia Zendesk participation, including follow on investments from Kozlo Ventures from Point72, Georgian, Sands Capital, Passion Capital. And we're really excited. Like the last year has been incredible, right? Like the adoption of our product by large enterprises. And not so much enterprises has been really, really good. I think that, throughout this podcast, we've been telling people about how excited we are about everything that's happening in AI and, about how it will impact people's relationships with brands, with technology, and, it's it is a big moment.

NikolaFor this technology and for us as a company, so, just over the last two years, we've grown our revenue by just over 10 times, right? And we've closed some incredible logos. We've built some incredible systems and this funding round is. Just, part of that growth and giving us really ammunition to do even more to invest in the product, to invest in our teams and scaling the whole go to market. Yeah, it's been like the last two great years since our series B and we only see good things to come.

DamienWe yeah, it had been a while. So let's let's add another letter to the Ray's round roster. I know that in the past, especially in past episodes, when we've taken a look at the market in general, we've noticed that there have been some, Short lived, thankfully, trends of companies that have tried to capitalize on open source LLMs and put AI to their business use and investors have been savvy and have seen through their short sightedness and that market has fizzled out. Of course, that also results in some headlines that showcase a lot of those thin wrapper companies as, sometimes seeing down rounds, if not just losing funding entirely.

DamienWhat do you think really contributed to PolyAI success in a time that sees an increase in the number of devalued startup companies?

NikolaYeah. That's a good question, right? I think that the era of kind of like You know, Q4 21 was probably the best time in history. Maybe some of those early dot com era quarters, but not at this scale, right? Q4 21 saw the highest multiples ever, and everything was getting funded at ludicrous valuations. And then really what a lot of investors will tell you is that it just has gone back to levels that are pretty normal. Okay. I think we're still in a pretty decent market right now. Now, yeah for people who raised at the peak of the market and then had to raise again, maybe they hadn't grown into the previous valuation.

NikolaBut, for us we've grown a lot. We're in a hot space. And I think one thing that was really interesting is in the past people would look at the future of automated customer service and they would say, Hey, maybe just, maybe it will all go digital, only old people want to use the phone and everyone else is just going to chat or something like that. Those questions are gone, right? Like the number of investors that came to us after maybe knowing us for a few years and said, Hey, like you told us that voice is the thing we see increased capabilities of.

NikolaAll the models coming out. We see at the end of the day, they see our product and they see our case studies and they're like, well, what you said was right. And then, the market, if you tell people what you're going to do, maybe they don't think you will, but then once you do, like you buy a lot of confidence. So this was a very hotly contested around then. I think. All the usual things that are maybe too boring for most viewers of this podcast Really matter. It's like how does your revenue grow? What does your net revenue retention look like?

DamienWho are your leading

Nikolacustomers? How much are your customers paying you? Are they paying you more and more? Are they getting value out of your product? How do they feel about your product? I think that a lot of our A lot of our customers got, north of 20, 30 reference code requests where they were asked to, talk to someone for half an hour about how they feel about poly AI. So it's always a really interesting experience because we get to learn a lot about how people feel. And it's always a really good time to reflect because, venture capitalists are smart people. So if you spend a few weeks on the road talking to all of them, then they're just testing your hypotheses.

NikolaThey're asking you interesting questions. Some you might have obvious answers to others get you thinking. So, I think contrary to a lot of people, I love fundraising. I love those conversations. Maybe I should have been a banker. I don't know, but yeah I've had a good time. We've had a great outcome and we're excited to just go, get back to work.

DamienSo a quick snapshot, you touched a little bit on how we're looking to expand our efforts and our go to market and that's what this money's going towards. Like, where do you see, is this going to be the year of some pretty significant product announcements coming from our house?

NikolaYeah, I think that you'll see everything from like products to deployments to everything else. It's no secret that we've been the first to deploy fully generative voice assistance with large enterprises and, we're spearheading our own frameworks around safety and controllability that allow people to mix the best of what is generative with, Like a level of comfort around predictability and enterprise outcomes. We're not talking about hybrid systems. We're talking about fully generative ones, but we are talking about methodologies that allow the user, someone in the contact center and the broader customer experience team to really be in the driver's seat of something as powerful as a modern LLM.

NikolaDoing a lot of the hard work empowering their voice assistance. So, that platform is now being used to deploy most of the new clients that are working with us. And we see that as the future. And at some point that will also be something that people will be able to use on their own. That's just a matter of like prioritization on our end, but we're pretty excited about it. And there are people externally already using it and with great outcomes.

DamienYeah, you said in the release, and I love this, you were confident in the next five years, Poly AI is going to be powering the voice of over half of automated customer service calls globally. And I think that that is an outstanding goal. Can't wait to hit that with you.

NikolaAbsolutely. Look, The bigger question there is how many of these phone calls will be there, right? There are quite a few already, but far fewer than there could be, right? It used to be a toy for the very, very big companies, think like 5, 10, 000 agents plus, and then you do a bit of cost cutting by implementing like a slightly IVR, right? What we're doing is we're, not to turn the podcast into a sales pitch, we're turning this voice channel voice AI channel into A phenomenal way for all customers to engage with the brands that they do business with buy products from need help from Or make appointments at so, you know as that becomes a reality more and more companies will be deploying this So we expect a huge surge in interest For poly and for all of our competitors.

NikolaYeah, the industry in

Damiengeneral, I think is going to get a, is going to get a big surge of interest and it has been over the past 18 months or so. Like you said, we gambled on voice. A lot of companies do. There's the idea of digital say gamble.

NikolaI've done nothing else my whole life. So I didn't really have

Damiena choice. Yeah, but yeah, so to that point, speaking of voice and let's talk about some recent voice developments in the market OpenAI had a pretty significant bolstering to their GPT 4 product with GPT 4 0 for Omni. I just found that out before the podcast. I had no idea what the lowercase was for. But Omni and the fact that they're taking a really multi modal developmental approach to generative AI. Are you familiar with the ins and outs of GPT 4 0?

DamienAnd tell me what your key takeaways are from what you've seen from this latest iteration.

NikolaYeah. Look, I, other than the, like increases in speed and costs, which are phenomenal and will drive like global AI to be even better. And that arms race between the LLM providers is fascinating. And that's all great. I think the whole pivot of a demo of course, we're all waiting for GPT 5, that's like the announcement that we really can't wait for. But. What they showed in those demos was really interesting. I think that just like the broad ability to both use like video. And audio mix it in was really, really good. I think the demos were done well, but revealed just enough about the shortcuts and well, not really shortcuts as much as design decisions that allow you to do a really good demo.

NikolaFor example, to get quite technical the low latency you see there it's not just about how fast your model is. It's really about how you're tuning and priming your system to respond to periods of silence. I, how do I know? How do you know that? As I'm blabbering now, how do you know that I've stopped talking? I'm not just

Damienwaiting to say something new. Am I catching your breath? Sure.

NikolaAre you catching your breath? Do I need to jump in and say something? Because we could run the conversation like that. Something tells me you and I could, with our kind of like creative speech. It probably wouldn't be very enjoyable for anyone out there. But that's what they've done. Where the moment you're not talking, it like stops, starts saying, and like they did a really cool trick where they. Stop and listen for a second. If you continue, they will roll it back. That's cool. That's like true, like hackery innovation, which is why like open access, great company, right? It's not just the fundamental models. There's like just a mountain of really good engineering in there.

NikolaSo yeah, the demo was good. The whole gimmick with breathing with like working on the equation, flipping languages the voice qualities I think. Among the best of the new providers that we see in terms of text to speech. I think the whole package demos quite well, and it's really exciting. If you really think about. A consumer voice assistant Alexa used to have 14, 000 people, right? I remember interviewing a guy who ran a team of 20 that was just focused on alarms.

NikolaAnd okay. Didn't need the team of that size back then either. But it tells you something about the level of effort needed to make progress. And when Sam Altman talks about their talent density, Like he's not messing around, right? Like I think OpenAI by last, count that I did has 770 employees, right? That's pretty serious, Because if like a small subset of those can build a consumer voices system that not only rivals, but seriously exceeds the performance of the more traditional ones, think like Apple, Google Amazon, right?

NikolaPretty cool. I was just looking at that and going you go amazing. Great. Will people use it? Like the form factor of a voice assistant, do you really want to use it? How do you use it? That's a tough question. That's a UX question, right? It's not so much as capabilities. It's also if someone made you pay a dollar a year to use Siri. Would you? Probably not for most people based on like the work we've done. And then would you pay for open AI just to have this or as other tools and stuff? Like those are commercial questions, like they're boring.

NikolaI think that just from a product perspective though, like how this will get integrated into the different products, let's wait and see. But it's exciting and they're doing a lot of good work.

DamienAnd that's actually begs an interesting like ancillary question. So how it gets embedded into products. I think the one true approach to a competitor that would go against sort of someone who doesn't plan to leverage existing LLMs that are just so incredibly powerful. Is to do something that's a little more scaled down and put it on device. So taking the cloud out of the Internet of things and making devices self sufficient and we've seen some come to market that have claimed to be able to do that, but have really not won over anybody.

DamienThe humane pen, the rabbit are 1, both kind of consumer products that flopped

NikolaI

Damiendon't think I

Nikoladon't think I've ever flopped at all. I think it's another attempt at, and disclaimer part of the same portfolio as us. But I think that they're making progress. They're early and they've sold a lot of devices, right? I don't think they're on device. I think that they're still cloud enabled. But I could be wrong about that. Maybe

Damienthe question is, do you think that's the necessary angle to go? Because there's not going to be a competitive property out there. That's going to

Nikolado a bit of a like history tension, right? A lot of these systems got really exciting for in car scenarios, right? Where people essentially, especially pre COVID like. How many Americans commute for two, three hours a day is just shocking. Oh, just America, very prevalent in America and Americans happen to have the most iPhones. So this was very relevant for Siri. And those in car scenarios then like in the world before 4g 5g, et cetera. Or at least really high quality, like internet everywhere.

NikolaThe question was like, can you do it on device? And also if you can do it on device, maybe it can be more responsive, right? Is that important? Is there a use for it? For perhaps, right? Obviously for the benefit of all software, if you're on cloud and you can push out updates and it works that way. To this day, I think I've had my car for two years now. I think it should be able to update automatically. I still haven't really figured out how or if I

Damienremember the old school, when you would have a, when you would have an old school GPS and you would have to download the Garmin maps every time that they released, it was just, it was so taxing. And that was at a time where that was the expectation is that you had to manually update. So,

Nikolaso, I think like the whole piece of like on device, Perhaps like for localized scenarios and stuff. We'll see. I don't think it matters. I think the connected one is just fine and it will be better. That's how we should track progress. Fair enough.

DamienAll right, going back to 4 0 and you had mentioned a little bit about this when it comes to the nuances of latency, like being predictive on whether or not this is just a pause to take a breath, or if you're done making your statement, should we start logging in 10, et cetera, et cetera. One of the things that I think has come from some of the chatter, some of the consumer chatter after seeing some of those demos for 4 0 is, The ability to interrupt a voice assistant, which I think in, in, from a lay perspective, you would think, Oh, that should be a pretty easy thing. Why is barging or interrupting a voice assistant so incredibly difficult?

DamienAnd why, if you do believe this, has what 4. 0 demonstrated very impressive?

NikolaWell, two pieces to it, right? Like in the setup that OpenAI has, there's an LLM running, producing tokens, and it needs to produce them fast enough so you can start responding. And even if You completely abstract the way, like when you start talking, and how fast the process is. So that speed increase is like hardcore, really good engineering. Well done OpenAI, right? The problem of bargin and doing it well is hard. Doing bargin and detecting, and just like being able to like, say, Hey, person's talking, let me stop. That's not that hard, right?

NikolaThat can be done with very simple technology. It can be done fast. The only issue is, In this gap that I just made, should you have started talking? It's really difficult to design the right interaction so that the communication is seamless. And what I mean by that is we've done a lot of experiments with them without margin. We turn it on and off selectively, and it's a lot more arts than science. If I'm saying something, you ask me a question and I respond with a yes. And then maybe I'm saying something, maybe you can hear that bird there.

NikolaLike whether you can hear that bird, whether that means I'm talking, can you distinguish between humans? That's the difficult part. So like making sure that you're not interrupting me. So you can trade time. You can always be ready to respond even while I'm talking, right? You can interrupt me. So knowing how to do that in a way that. The other side will be comfortable with that's the not million dollar problem, but a whole lot more The other thing that's really interesting in the form factor that they have right now is that you can see That bubble and you're talking you can see that it's processing or not They have pretty decent haptics around like the vibrations of what's going on So, they didn't talk about it much but when you try it out it's actually really an interesting problem because you're teaching, you're building a whole new modality of how you're going to use this when maybe like vibrations being a subtle feedback, I'm still thinking, still thinking, or, like I'm thinking rather than I'm waiting for you to stop talking because you can also click to interrupt.

NikolaAnd that's that's pretty taxing on you. To have to do that. But then again, it works pretty well. Whereas like whenever we also, when you're

Damientalking about proper applications right now, it would be a lot of hands free scenarios. So the idea of needing to be dexterous in your engagement with this sort of a system, it seems a little far fetched from consumer use.

NikolaAbsolutely. So you might've noticed, I think that when one of the guys demoing like close their laptop. The thing stopped,

Damienright? Yes. And I

Nikolawas like, oh my God, someone tuned that margin, right? And cool, right? It's what we do live and breathe. So, it's like watching a sports game and just I see what you did there. But when you do it in say, our applications, let's hear walking in the street and there are voices around you. You'd be interrupting it nonstop. And then at some point, maybe another model will go Oh, this is not working. Like the user's trying to interrupt me. So often you might just, then after a while say, look, I'm going to stop listening for interruptions because they make for great demos more often than not in like at least automated phone conversations, it can get pretty difficult to do it.

NikolaWell, And it should be more trouble than it's worth. And it delivers,

Damienas you said, the perception to the caller, like in a customer service application, they just think this is latency. They think that this is a lazy system or a poor system that isn't performing that well, because it might be waiting, thinking that bird that it heard in the background was offering some sort of interruption. So I could definitely see how it does more harm than good. It's really interesting stuff. Where have you seen, you said we've dabbled a little bit with that. Where have you seen it really work fairly well?

NikolaI think the real trick here is to know where you are in the conversation and like with everything context is, Like the most important thing. So if you ask me a very simple yes or no question where there isn't really a way that I can hijack the conversation. If you hear, yeah, I did. Okay, great. Well, if you did, then we, like I'll continue your application in this way, or I'll send you this link. That's great. That's what a human would do. And there you can interrupt or you can just, it's all about streaming, right? And the ability to just hear what you said, but again, to be patient enough, because if you said.

NikolaYeah. And I actually, booked online already, like maybe then you need to do something else, right? Sure. Overall it's really more trouble than it's worth when you just look at a containment and customer satisfaction. So we tend to steer clear of it and we allow it maybe five seconds before the end. Of our system talking and that's where design then really matters because you should never and if you Use the system, you'll see how you'll be initially impressed like the opening i1 but then you'll be like, oh my god I can't listen to this Do something talking for this long, cause on the phone, we didn't go into those monologues.

NikolaAnd it might just be, my ADHD, but it's if you go into those like rants and you're reading like half an A4 page, like that's not why you use the voice assistant. And that just goes back to that, like product question. Like, why are you doing this? So then it's again, like just a matter of yeah, you probably want to be able to interrupt it. But if you're going to use it holding your phone, you might as well click, right?

DamienYeah, for consumer use. We're at a stage right now where the idea of interacting with some sort of automated system, whether it be a a home consumer smart device or calling in for an automated customer support experience, like our current culture, where it is a part of the culture is. This thing needs me to speak as little as possible and follow some sort of key of commands, and we all expect the first time that we ask Siri to play a certain song that it's going to mishear the comment or the quote, and then it's going to do something else, same for Alexa, same for Google Home.

DamienBut we, that, it would be a far, it would be a real far timeline, I think, for us to get to a point where folks are sitting here like ranting to some sort of automated service agent and giving too much information to where the complexities of that conversation. So I think that right now we're just, the issue is getting people toward that natural language, that interacting with automated agents or devices at a level that they would talk to somebody else and not feel like they have to give a tome of information, but also not talk in short blips and beeps.

NikolaYeah, Will people talk to these things for a long time? I think some people do, right? They're again, it's like as companions and stuff but yeah, you're right. There's so much to do and, they're taking a new crack at a pretty big problem that many have already worked on quite a bit. It's not just about their technology. It's also about like where it lives and where the product market fit for that specific system actually sits. Like the reason we went into enterprise applications is that for customer service, it's pretty unambiguous that someone's got to pick up your phone call.

NikolaAnd it's also clear many years in that, like those phone calls are going to keep coming, so that's why we chose to work on it, but look like a general Jarvis like voice assistant. Look, it's on everyone's radar. We all care about it. So, I hope OpenAI continues to work on this stuff and that as a result, we all get a bit closer to it a bit faster.

DamienCool. Well, thanks again for today's episode. And again, congratulations on the big series C announcement. That's fantastic. 50 million bucks in the bank. Love it.

NikolaIt's great. Yeah. Well, thank you. And the whole team, it's really, we couldn't have done it without everyone at Poly AI. We're thrilled and we are really excited about the products, our customers and everything else that's coming with with this money and just the enthusiasm that comes along. Amazing.

DamienAll right, Nicola, we'll talk to you again next week.

NikolaSee you Damien.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode