Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

07: Reaction to OpenAI DevDay 2023

Nov 10, 202318 minAudio
ShareXLinkedInFacebookWhatsAppEmail

PolyAI's Michelle and Nikola unpack announcements from this week's DevDay, touching on GPT-4 Turbo, the impact of consistent model output, the marketization of custom GPTs, and the intriguing possibilities of new text-to-speech capabilities. Additionally, we touch on the importance of reproducibility and security considerations in enterprise-level AI applications. 00:51 Recap of DevDay and its Impact 02:50 What it Means for AI Companies 08:03 The Future of AI and Cloud Companies 11:54 Exploring the New TTS Capabilities 14:50 Implications for Small and Large Businesses 19:26 Safety and Security in AI Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Nikolathe question becomes do you build a rich set of cloud companies that benefit from this? Can, cloud players managed to capture enough of this growth and turn it into their enterprise value or will it all accrue to the cloud? Will foundation model companies become the new cloud provider for a new category of things or will they be subsumed by cloud?

MichelleHello everyone and welcome to Deep Learning with Polly AI. The only podcast that willfully ignores the fact that two real humans are conversing with each other about the virtue of automated phone calls. Today we'll be talking a little bit about what we saw coming out of Dev Day. Huge day. I think it rocked quite a lot of AI companies certainly people in our space people outside of our space. And I think it. It really injected a lot of excitement into the AI universe. So I'm here with Nicola the CEO and co founder of Poly AI, also a PhD holder from Cambridge in machine learning and natural language understanding.

MichelleAll right let's get into it. So I think before we start talking about Dev Day, we should probably do just a really quick recap of what came out of it. And I'd say from, maybe not from biggest to smallest, but I'll start with the biggest one. GPT 4 turbo and the reduced costs that were associated with it. A huge announcement. I think massive impact for developers. We'll talk more about that. Also reproducibility. So consistent model output. The GPT marketplace, this idea that you have an app store for GPTs in themselves, it's really exciting stuff. And then also some new TTS capabilities, stuff that has absolutely no impact or interest to a company like ours and some copyright shielding things.

MichelleSo talking about, open AI, basically being able to protect. It's users from copyright infringement and issues. All right. All right. Nicola, anything that you'd add to that before we start talking before we get into it? No, I

Nikolathink, I think it's actually really interesting. Just like in terms of opening and their ability to ship, or continuously ship and just surprise people at how much work they're doing. And, some of them says the talent density they've got. It's really impressive, right? I think they're leaving a lot of, like other foundational model companies into their, behind them. And it's really impressive. It's really cool. I think the scope of their vision is slowly evolving. There was a bit of an Apple esque attitude to all of it, which is like weird when mixed together with a dev day targeted at developers. So I think there's a lot of like language for mixed audiences there, but it's good.

NikolaI think it like reflects the scope of their ambition.

MichelleYeah. Awesome. I actually, I'm wondering how many employees. OpenAI actually has right now but we just saw from one of our friends a VC researcher that just put out this report. And one of the sections was literally OpenAI reveals itself as the next big tech giant. And I think we're seeing that to be the case. So with respect to Dev Day, who do you think is losing sleep over these announcements? Who do you think's really worried right now?

NikolaWell, I think you have to go through like things one by one to see what what it means for different groups, right? I think like the obvious gauntlet say to 11 labs with making the next kind of like generational text to speech that much cheaper That's definitely there, and I think that the quality, despite their excitement, they're like, I don't think the six voices there at least are as good as some of the stuff that 11 lives has. But then, I think the real exciting bit is really GPT 4 turbo. You can put 300 pages of content into a prompt, right?

NikolaSo I think for a lot of people building all sorts of different things the needs, the need for a database just like disappears because 300 pages is. A lot of content, right? I don't think I could repeat 300 pages of anything back to you. A few years to learn it by heart.

MichelleYeah, it is incredible. I think another piece of this too, is all of the companies that were building things that were really more feature esque than company esque on top of OpenAI, on top of ChachiPT. So what we talk about is thin wrapper companies versus thick wrapper companies. OpenAI basically lipped them in the face and dared them to go bigger or die, basically would you agree

Nikolawith that? really hard for people that are, like, really close to that thing. There's a whole mountain of companies that have raised money to make it a bit more enterprise ready, a bit more reproducible. They're talking about things that... Frankly, I've gotten a little bit boring, right? Yeah, we know that deep learning models are a bit of a black box, right? That you don't have fine grained controls. They're probabilistic models. That's okay. I think what we're seeing here is that these guys are moving at a speed where a lot of that low hanging fruit that maybe someone was hoping to build a little bit of the motor around won't get to be there because the water level.

NikolaHas risen and reached that moat and flooded that and kept going. I think the other thing they're really doing well is continuing to give, the market a firm belief that they will be that supplier. Of this kind of technology because the cost reductions are just fantastic, right? And if it continues at this scale, then, all talk of, different models that you have to train, make them smaller, more efficient. Why? And I think that, when cloud started a lot of people felt that, um, they might at scale still benefit from having their own data center and they're all pretty much wrong.

NikolaYeah, maybe for some security, if you absolutely need, if you want the onus to be on you. To guarantee that security and stuff. Maybe you still want to do it, right? Maybe you're a intelligence agency or something of that kind, but if you're not, chances are that, the leading player in the field is going to have more research resources than you to make the whole thing safe, scalable more stable, et cetera. I was at another event here in San Francisco yesterday, and someone mentioned the figure of 4 percent of added GDP growth.

NikolaTo the US economy over a period of the next 10 years that they expect to see from generative AI. And that's higher than the comparable figure for cloud was two and a half to 3%. And, I think that just the stuff that can be built on top of this. Is really fascinating. I was listening to another to another example yesterday and someone managed to reproduce angry birds by having an interaction, 500 back and forths LGBT, kind of starting from a conversation around the describing it all the way to getting the code that.

NikolaRuns Angry Birds, right? So I think it just shows you the acceleration and the ability to innovate and to do things that were previously unthinkable. So I think really, that whole developer angle and what it does for them. The other part of, just who's being disrupted. If we think about companies that were like, Hey, explainability. Reproducibility, right? So the addition of the seed parameter and the release of log probabilities. So like knowing the, how likely something that I produced is That's really good because it allows you to do things that behave similarly every time.

NikolaAnd that's a big part of enterprise readiness. It's a minimum precondition. It's not the only thing, but it's really important. I think that it's all really interesting, including like the marketplace idea and all of that.

MichelleWell, you just did mentioned an interesting stat, which was comparing kind of the cloud growth to the AI growth, like what GPT is going to add or generative is going to add. And I'm curious about what you think the perspective is of the cloud companies on that? Like, how are they? Are they shaken up by these types of announcements and the lowered cost and the things that sort of make them less

Nikolaadoptable? Yeah. Okay. Obviously Satya Nadella is counting dollars in the background. Cause they moved and they bet on the right horse. I think we continue to see Google lag, the question becomes do you build a rich set of cloud companies that benefit from this? Can, cloud players think of, your kind of like market favorites, think of a data dog, a Palantir Atlassian, all of these companies will they? Managed to capture enough of this growth and turn it into their enterprise value or will it all accrue to the cloud?

NikolaWill foundation model companies become the new cloud provider for a new category of things or will they be subsumed by cloud? And there again, we go back to the one player behind all of them and Avidia who will definitely profit. 'cause they're oh yeah. nOw the question there, just who gets to keep the margins and control the distribution of the entire thing, what role open source has to play, I think open source second player, and I think the excitement and the whole community around everything happening on that front is huge.

NikolaAnd it should be because I don't think a clear number two exists. alL of the people losing sleep, I think it's probably all. Foundational model companies, but opening I hear, cause yeah, sure. We've got, we've got someone who really looks like they're going to be a winner for many years to come.

MichelleYeah, that's for sure. And when you say open source, you're not talking Lama, right? No,

NikolaNo. All of that stuff. Just like that's a model and like we'll have more things and this will evolve, right? And I think that everyone's expectation is that that's really what you need. And, some people will tell you that, like the internet, which ended up running mostly in open source stuff many things built with generative AI may end up running on these things as well, like cutting out the end supply, who knows, I think it's getting very interesting because. Of all of these applications, not all require the same level of computing. You have to think forward to a world where these models have much greater capacity, right?

NikolaAnd as we increase compute and improve the training and do everything else to to increase their capabilities if we just look at like the raw. Let's say intellect of these things, like you don't need it. Like you don't even need GPT four for everything. I think a lot of people are excited about GPT 3. 5 turbo and the increase, the further decreases in costs there. So it will be a trade off. And I think for a lot of those things, people may want to run it on their own and cloud providers will probably want them to run their own stuff so that. More of the value, again, accrues.

NikolaYeah. But you know, you still see OpenAI talking about their custom models and really freely offering their professional services to the highest bidder to help them. I see that, yeah. Like it's ambitious. I think they're shooting at developers, they're shooting at end applications, they're shooting at creators online, they're shooting towards like large enterprises that have their own committed. Instances and they're offering to do work there just to increase their stickiness because we think about that, Google is doing the exact same thing, trying to stay the number two player.

MichelleAbsolutely. All right. So let's talk about voice for a bit. Since that's something, obviously poly AI has a big stake in and a lot of experience in a few podcasts ago we had Yen, our COO on, and he talked a little bit about how oftentimes new technology. Exists and comes in on the fringes of things. Like the small mom and pop shops, might be able to go full force at adopting something that looks like, uh, an open AI built voice assistant. What do you think about this? Does this?

MichelleA new announcement. Does this make it more likely that these small businesses are building for themselves? Does it make them more likely that they're going to be pushed into companies that are able to offer this as like an actual service? What do we think this does? Yeah. I

Nikolamean, like what a small company is an interesting question, right? So if you think about mom and pop shop, like restaurants, stuff like that, are they going to go and build stuff here? Well, it's definitely become a lot easier for them to do it, but no, they won't because it's still like very complex technical challenge that requires integration with their booking systems, with their telephony stack, still quite a bit of work, right? Does it make it easier for people all around to build things? With LLMs and push out like MVPs of their voice assistants. Absolutely. Is everyone going to be offering this and are we going to have a cacophony of different, the same six voices.

NikolaYeah. I think the one thing that made poly, I really take off that I didn't expect when we started was the fact that we crafted a new voice for every client. It just from a Texan voice here, a Scottish voice there. We got to a very large set of clients using very different things that I couldn't have imagined before, right? Different tones, emotions, sentiments. Speed, style, slang people care about this. And I think if you care about user experience, the voices that we heard from OpenAI are not even scraping the surface.

NikolaAre they functional? Can they serve like a wide audience? Yeah, but so could any text to speech model before. So I don't think they're really rocking the voice world. Whisper is a nuclear. Model, it's like big, it's accuracy is okay, but it's not the kind of shift that LLMs offer in terms of everything that people are doing with them. And when it comes to text to speech, I think that, we're seeing movement, but I think like for those looking to just play with text to speech, I think 11 labs is still superior in terms of quality. It's not cost.

NikolaSo I think it'll be an interesting kind of like micro conflict to see play out between those. Yeah. And they're not the only ones. There are many other players.

MichelleSure. Sure. And then for large businesses. So at the other end of like the mom and pop shop, the people who are willing and do have teams and resources that can be deployed to building these types of things. They just got a lot of access to do this with assistance API. Like what what can they do, with those six voices? What can they do that would still be impressive? Does this affect any enterprise sellers of voice assistance? Yeah. So I think

Nikolalike when you look at the majority of excitement there, I think that it really isn't aimed so much at voice and, when they talk about multimodality, they're going through the same kind of like traps. Many of the Google demos had, I don't know how many times we're going to see. Someone also randomly speaking to a voice assistant, but where would be good? Where would be a good place to have pizza in my neighborhood, right? That just comes back to like technologists talking about problems they'd like to work on rather than the problems that people really have, right? I think a few other things that they announced are just more about, democratizing the ability to do a lot of things with LLMs the function calls got more advanced and stuff like that.

NikolaYeah, I don't know. I think there's a lot to unpack. And we'll see how people use it. I think that we'll see an acceleration for sure. More than anything, I think we'll see companies emerge to cover small use cases. With these models, but then at the end of the day, they still have to be companies, right? They have to have quality pack product that has a standalone future, right? And either they'll manage to do it and they will, I'm sure in many different markets. And then others, you'll just have like the cloud and the opening, I rising towards the application layer and consuming everything.

NikolaBut I think that, in terms of like raw capabilities here. The biggest thing we see is that controllability and reliability aspect, the seed function, which kind of allows you to get the same behavior every time. And maybe just for the audience that is not familiar with the terminology, the seed parameter in a random function is set so that the first number you generate is always the same number. And then the random sequence follows a certain pattern. So here one big frustration for people using LLMs was that the output was so probabilistic and that they couldn't do that.

NikolaI think this will have a big impact on people, wanting to get. Somewhat predictable behavior. Yeah.

MichelleDoes predictable behavior mean that it's also interrogable? Like that you could actually look at it and see what's happening? No. Okay. So in terms of,

Nikolaso log probabilities, tell you more about the nature of the output. So like the thing that was generated, like how likely it was. And um, log probability, cause it's a sum of the probabilities of all generated things rather than a product. And Yeah, I think that has a bit of an impact on just like knowing how confident to be like, whether you want to believe the output or maybe you want to run it again. Cause at the end of the day, you just got one like sample out of the distribution, right? So if you've got something that's quite unlikely, you may want to run it again and get something that's a bit more of a likely output sequence.

NikolaBut sometimes you want that unlikely one because you're looking at it for creativity, for something interesting, right? But it will give people, a tool to use in trying to get, to build a better application in the end, but in terms of just like interpretability and like why, like ground rules, facts around why it says something where it's basing that information from I don't think that anything announced this time has, they're working on it and they're constantly trying to make it better. And I think that those that use open AI LLMs often have seen this in practice, right?

NikolaLike the model is evolving and getting better. But maybe one thing that is interesting is like the whole kind of like JSON mode where you will get like output in JSON format always. One thing that a lot of people playing with GPT have found is that it's just sometimes a bit disobedient around the, Hey, I want the structure of an output. It just gives you something else. And like a very firm Boolean control over it must be a JSON, which is like this data format that you can use to pass information around. It's a standard that's useful. I think that just like addresses like something that clearly, you know, to a PM at opening, I was a feature request that had.

MichelleYeah totally. So I didn't see a whole lot around their safety announcements, but did they make any changes that would make them make it a little bit more secure for the enterprise? Did they address that? Well,

NikolaI think that they're looking at usage. They've made some announcements around rate limits. But that's really not to do with safety. I think this really isn't a long term, well, safety is different from just like reliability and stuff. I think that, that whole piece around custom models for large enterprises and stuff. That's something that, we know that OpenAI has already been doing with a lot of companies. It's not very public. But it's there and it's happening and they're signaling to the world that they would like to do more of it. But you know, with 500 people, you can only do so much.

NikolaAnd I think that I don't know anyone who's not impressed with just how much they're putting out there right now.

MichelleFor sure. For sure. Well, I believe we are at time. But this is, this was, it, Probably the talk of the town, if Slack was a town for Poly AI, when it came out, it was just, it's great to have a recap. It's great to chat through it and I'm sure it'll continue to come up in future podcasts.

NikolaYeah. I think this was definitely something this week. I think for next week we've got our our customer conference. So we're excited about that. And announcing some of the stuff that is leveraging a lot of this kind of material and other advances in deep learning that that we've been pushing internally.

MichelleThanks for being on with me, Nicola and looking forward to the next one. Until next time.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode