Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Deep Learning with PolyAI

Deep Learning with PolyAI artwork

038: Decoding McKinsey/QuantumBlack's State of AI

Jun 27, 202421 minAudio

Guest host Steven Fine and series regular Nikola Mrkšić discuss the current landscape of AI adoption in businesses, drawing on McKinsey's recent The State of AI report. They explore topics such as the rapid increase in generative AI usage, the challenges of integrating AI into existing business workflows, and the debate over building versus buying AI solutions. They also touch on the future of AI, its impact on job markets, and the what's hype and what's reality in the world of autonomous AI agents. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Nikolawhen it comes to an LLM, I think it's a pretty nuclear investment, so it's pretty hard for them to kind of like go and say, Hey, I'm gonna take all our data and do this. Having said that, I think that a lot of large orgs are getting approached by. Foundational companies, cloud providers, et cetera, to talk about licensing their data and being part almost like of a federated learning group where they contribute a lot, forget preferred terms in terms of usage, access, SLAs, these models. I think it's like falling in place where it should be.

StevenHello and welcome to another episode of deep learning the poly AI podcast. I obviously did such a good job as a guest that I've now been invited to guest host this session. I'm Steven Fine senior Business Value Analyst at Poly ai, and with me as usual, I got Nicola Mech.

NikolaGood to have you on that seat.

StevenYeah and I reckon this is probably the only podcast that has a Serbian and South African. Both working at Poly AI, and that doesn't take a lot because there's only two of us.

NikolaWe can find a few more. We'll do one or more next time.

StevenYeah. So I'm really excited about this episode. So today we will be touching on the report, the state of AI by McKinsey and a lot of what they're talking about and what they're saying. Yeah nothing gets, me going quite like analysis from McKinsey.

NikolaWhen you joined PolyA, did you think that in the cards is a podcast where you talk about McKinsey reports?

StevenNo, and look, I never thought I'd make it onto a podcast. This is like something I'm gonna tell my parents about. I'm actually gonna make my mom watch this. It's gonna be so proud of you.

NikolaThat's what it takes to sell software.

StevenOkay. So to kick it off with today, I think there's a really interesting insight right at the top of the reports. And I think it speaks to a lot of what's out there is They stated that 65 percent of respondents within the survey say that their organizations are using genuine generative AI regularly. Now that's nearly twice as much as was reported last year, which is a pretty hefty bump. And I guess my question to you is. Does this figure surprise you?

NikolaWhat does using Gen AI mean? I think that it's not surprising that people are falling over themselves to report that they're adopting new technologies. And, I think quite a few times I've like thought of Gen AI as the new cloud. And I think big organizations are really like following the same pattern where Quite a few companies fail to do cloud transformation early enough, and then suffered from having their competitors become more agile, have lower, hosting costs and just being able to use newer technology. So I think people are doing their best to push the adoption of these things.

NikolaI think in the, in a previous episode, we talked about just. How are individuals at those companies using it? I think there are a lot of people who are power users of generative AI, but by no means do I think that 70 percent of the people in those companies are making good use. What do you think?

StevenI think it's, I reckon half the people don't even know. Whether they're using generative or not I think there's, I was at a conference last week and basically every single company had slapped on generative AI to what they're doing, whether they're actually doing generative AI or not. So I think there's definitely like an uptick in AI and a lot more interest. But I can't get behind 70 percent of people are saying that they're using it in their day to day.

NikolaYeah. I think that's a bit of a, that's a bit aggressive, right. But I think it might be making, it might be impactful to 70 percent of the people, although even that seems a bit rich, right?

StevenMaybe it's just 70 percent of people are using chat GPT to write nicer emails to their boss.

NikolaThat's something, I don't know. I don't know. I think maybe we're just maybe we're just too skeptical. I think, to those companies that use like Microsoft office at large, if it's properly enabled and it works, I think, people might be using like copilot and similar things. So it should fit the bill. And if the adoption is really that impressive, then I who knows?

StevenLet's assume that this adoption is true. What do you think stopping it from not being even higher? What's stopping more people from adopting this kind of technology?

NikolaTime, ease of access. Like whether it's well integrated in the workflow, like people don't change workflows all that much. We all tend to do relatively repetitive things. As a good friend of mine likes to say what do you do at work? Like you open your computer, like what do you do? And fences are like, you do the same thing every day. And then you go from there. So I think it's just the tools you use, if it's in your inbox, if it's in your PowerPoint, Excel, et cetera, like that covers a lot of people's workflows. Now that I think about it, it could be 70%. I'm just skeptical that all these orgs have rolled it out and enabled people at scale that fast, because we do a lot of this and we don't see, we don't see such fast adoption.

StevenYeah. I think you make a good point with the Microsoft, like if it's integrated in with what you're doing, it becomes super, super easy and accessible to do it. At the barrier to actually doing this stuff doesn't. But it's not a limiting factor anymore. So here's a, here's an interesting one that I actually think is a very good call out and I know from the sales side, we've seen a lot of is what, one of the things they spoke about is that with all this generative stuff and all the hype it's put a much kind of broader focus on AI capabilities as a whole, not just generative.

StevenSo I know that they said that, adoption, you Like around AI is hovered around 50 percent at organizations. And now this is suddenly bumped up to 72%. So I guess it's this idea that, there's all this hype around generative, but then that also opens up this opportunity towards just general AI. Do you see this, do you see this playing out in like in a business context that like the hype of generative is playing out in other facets of AI?

NikolaAbsolutely. I think that we've seen that with our buyers where the moment kind of thing late 22, through 23, like the interest in AI in general, before people could really tell you what generative means and genuinely, like how many people Can tell you what generative actually means, right? And what is a generative model versus not a lot. Are we talking about like an LLM or, Gmail's had like predictive sentence completion for a while, right? Like where does a language model stop and an LLM begin?

NikolaSo I think from that perspective the one thing that really helped was just the prominence of chat GPT as a tool, the capabilities, the really, became in a consumer form, very easily accessible to a lot of people. What that meant then is that through that hype and as the AI bandwagon, I'm saying, yeah. But truly I think a lot of companies then just said okay what I often say to people is Hey, in 50 years, I think you believe that we'll have Jarvis like voice assistants.

NikolaSo then the real question then becomes As you see an acceleration of progress in AI, does it become five years or 10 or two, And I think because so many people got exposed and got encouraged by senior leadership in these companies to really make a bet on AI, it meant that they just started looking at all sorts of AI products, right? Now, the really good thing is that AI companies making heavy use of generative models because they're so incredibly good. And through that, I think general adoption of AI driven products is rising.

NikolaI think that people much like they learned how to use search engines are also learning how to use generative models in their work, that it's. more than anything best used as an assistive tool. I think, our use case is a rare one where we actually really don't use it as an assistive tool, but rather we provide a full kind of like end to end solution. But I think that's a bit of a harder Olympic discipline than using it to to help you in your work because then you can still check whether what you think it did make sense or not. Yeah, and I've always, I always look at this as in the last year, it's like a it's a shift from a push from the bottom, like the bottom of orgs that are like we've got these cool solutions that we're now, we've got to push up to suddenly you've got like executive leadership who are pushing down this agenda of AI and if execs have it, it means that there's budget in there and when there's like a year budget cycle that, a year ago it pushed in.

NikolaNow we're a year later. So budgets started to be allocated and now there's a huge uptick in spending and that's at the same time as well, I think from the broader AI perspective is I think a lot of the companies that are, like you've said, are already in the workflows, like you're already using their solutions is they've now integrated AI into their products, not always generative, which then fundamentally means as that happens, more people are essentially jumping on the AI bandwagon. And I guess this is it's an interesting kind of follow on from that is, you looking at like products, one of the things that McKinsey spoke about was people, leaning to off the shelf products.

NikolaSo not building these things like these AI products themselves. So I guess I'll go on like why do you think this is the case? Like the old. Both by a verse by debates and the pros that I want to see, I want to obviously we're a buy, we're not a build. So what's the pro of building yourself and then what's the pro of just buying off the shelf? Yeah, I think when you just look at conversationally, I would always found the certain buyers are more prone to like just embarking upon building up a team and trying to build it on their own.

NikolaAnd what was interesting always with voice assistants, which have boxes, it's a lot of fun to build these things, right? So if you can I think that any tech team given a choice between building an ERP system or voice assistant, if it's really all the same, then, most people would prefer to build something that, they can pull out a phone and a pub and show their friends what they've been building. Now when it comes to LLMs and a lot of these models. We're talking about extreme investments, right? You can burn a few million, like it's nothing.

NikolaAnd I think there are plenty of companies, especially in like the highly regulated environments, which are cash heavy. Think like very large banks, et cetera. I think there are LLMs being at least fine tuned if not trained from scratch. But that's an expensive sport, right? That stops being, I think it's harder for, and I think there's enough is understood when you look at just how foundational companies, they're raising hundreds of millions to even be in the game and you've already see consolidation, right? And. Quite a few of them falling off.

NikolaRecently like stability, inflection, et cetera. So you look at all that and you're like, Hey maybe I can't spend. A few hundred million on this, no matter that I would like to do it. So I think it's a bit harder to do in this case. Also like the skill sets, the knowledge and just, the confidence of doing it, you'd have to be pretty confident that you can do something and that it makes sense and to present an ROI on, a team of 10 doing something, no matter that that team doesn't finish what you intended in three months, that it becomes a team of a hundred building.

NikolaSomething for a very long time. Building extra competencies, not delivering like that would be the case. Typically say when it comes to building a voice assistant, right? We know plenty of clients that tried, some of them came back. Others I hope I will. They will, but when it comes to an LLM, I think it's a pretty nuclear investment, so it's pretty hard for them to kind of like go and say, Hey, I'm gonna take all our data and do this. Having said that, I think that a lot of large orgs are getting approached by. Foundational companies, cloud providers, et cetera, to talk about licensing their data and being part almost like of a federated learning group where they contribute a lot, forget preferred terms in terms of usage, access, SLAs, these models.

NikolaI think it's like falling in place where it should be. I think probably about six months ago, I'd heard a few banking CIOs told me that they were definitely going to train their LLM. I think it's subsided a bit and I think that's a good thing. It's a rational thing. So I think, it's, It just makes sense that it would be a built thing because it's not an easy thing to build. It's an expensive one, right? The space race is not really for, a Serbia or even a South Africa to pursue, right? It's it's for the big boys. It's like

Steventhe whole GDP of South Africa. But I guess it's like the idea of training these models with the big boys. I know that like Microsoft. Launched or tried delayed, they launched, they like retail recall, sorry. Product where it's just taking screenshots of your screen the whole time, and now with open AI and Apple it, they literally could have all the data in the world to train these models, which will be pretty hard to beat. So I guess like risk, if we still keep keeping the business context and of risk.

StevenAll right. Inaccuracy of these models. And I know that McKinsey kind of called that out inaccuracy of the outputs that these generative models are producing as being the major risk. Is this a valid any path that's actually ever solving this? And how should companies be thinking about this?

NikolaI think it is absolutely a risk. I think there's nothing exaggerated about it. I think, like the difficulty mostly at the moment with the current paradigms is in like, how are you feeding the models? And the complexity moves from building a system, which curates specific information to specific inputs. It moves into like, how are you doing rack, right? Like, how are you selecting the supporting data that the model then uses to inform the specific answer. And the truth is as much as, interacting with chat GPT is elegant, right? And you go back and forth.

NikolaOnce you start building a business application, a lot of the logic that is custom, Goes into how you do rag, right? And also do you have access to the right bits of information? Are you perhaps providing contradictory pieces of information? And as you build and scale up any complex system that uses an LLM to drive the whole application as you should, it's a phenomenal tool, right? It gets really hard to know why it might have produced an incorrect answer, right? What prompted it? What didn't?

NikolaAnd then as you build safety layers that override behavior or control for it very, very strictly, and you will do that in a lot of enterprise settings, then, you will slowly ship away at the most expressive thing that anyone can do, which is, reason overall that data in perhaps some creative ways, right? So as you do all that, you'll probably. You've bent the system into a different shape and then you're making it perform an action that, you might have inadvertently completely restructured an input that the customer has provided into something else.

NikolaAnd it could just fall on its face and do nothing if you've over constrained it, but it could also be creatively confused. Or use information that is just perhaps correct locally, wherever you pulled it from, but when put together makes for an answer that is like patentling correct, right? And it would be quite difficult, I think like we do a lot of that and attributing just the source information clearly that's where deep learning gets you, right? Because these models have always been incredibly good black boxes and the trade off between their maximum expressive power.

NikolaAnd the interpretability like that remains like the main point of contention for using a lot.

StevenIt's that explainability part that becomes really key. Cause you can also, if you're able to explain it, you can work backwards to solve the kind of where it's gone wrong. Which is, I think will be a huge kind of parts part of where this goes, but it also one of the things that, That keeps coming up is if foundationally, and I'll just speak to like voice assistants, like if you're using generative AI for this stuff, like you have to have a knowledge base that is correct. Like you can have all the great tech in the world, but if foundationally the knowledge base is horrible, it's really hard to get this technology to do exactly.

StevenWhat you wanted to do consistently which I do think is like a major challenge of this.

NikolaIt gets boring, right? Like you're back to square one of have you done digital transformation right away? Do you have everything in the background? Is it all aligned? Do you have the right APIs? How do you use them? How do you call them? Where's the information pulled from and that's it, right? So as much as we want to fast track a lot of it it will be the bolder enterprises which can afford to make some mistakes. They'll probably see some more impressive outcomes, but, they'll also see some some risk, right?

StevenThere was a great article by one of the partners at BCG, just like around Can we make sure that we get this AI stuff right? And he wrote very much about you can't use technology and AI to fix a broken process. If processes are broken, and if your knowledge bases are, Horrible. Can't use technology to fix that. So it's like this, there's like part of it is right. Workflows and it's beyond, it's a little bit beyond technology. Like technology is not the solution to everything. You don't have good foundations. It doesn't really matter what the technology is there to do.

NikolaYeah. Look, I think that, um, these transformation projects, starting from guys like BCG and McKinsey all the way into, larger consultants. I'm not trying to be a cynic and say that they have, A lot of things to sell different work, but they don't, right. Like they've got to say

Steventhe stuff they've got to sell it. They're selling their services.

NikolaThey are selling their services, but I used to be a skeptic, right. And I thought that these were just really smart people who, insert themselves into these processes. But, the more we work. The large enterprises, the more we figured out that these guys are completely indispensable, all the way from like setting the strategy of doing these things to actually getting them done. And often they're really the carriers of continuity in those organizations and they're able to put together a patchwork of technology. Some may be like end of life. Some may be things that other partners don't know how to integrate with and do yeah, you've got to figure out like.

Nikolaall of those things so that you can then reliably feed them to a neural network that can do some of that logic. And look, you'll have to build Less of the piping. But I think it may take a bit of time before we trust GPT 4 to write APIs and go into these systems. I think, it's also, let's be honest, it's hard to predict, right? If we can create, an almost we would almost have to be conscious if you see what I mean, if you could create a level script that, throws around these databases, interacts, learns, figure out the shape and format of everything and, like cross references things, validates yeah, like it could maybe, that's the pipe dream that people have GPD seven.

NikolaWe'll go in, look at your database, figure out that you don't need this part, you need this part, we'll encode this one, and know whether it's right or wrong, and, ask you a few questions, delineate and be done. The truth is, There's the data entry job at the end of it, probably, which is still going to have to be done manually. And yeah, so you'll have to clean house to make sure that these things can run well, because, if the cloud was put everything in one place, think of what snowflake does and all those companies are creating data lakes and all that it's you need that stuff.

NikolaTo make the, to make use of, big data right now when it comes to a reasoning thing that can go on top of it. Kind of like, I don't think like a 5th generation bomber or fighter, taking off from a runway. I'm no expert in that, but I imagine that a very bad runway is probably worse for a very fast fighter. Plane then for, a very old kind of don't know what the older planes were called, but I can imagine that a greater speed requires greater precision and accuracy. So I think, you'll have to do a lot of building before you can make maximal use of these things and especially rely on it in an enterprise setting.

StevenI think what the idea of where this could go in the future is incredibly exciting. And I think that's the whole promise of these autonomous AI agents that people are talking about at the moment and is a huge craze. So the last thing that, McKinsey spoke about was that a lot of people and companies have reduced their worry about AI now taking jobs, like that's not something that they're planning for. Do you think that they should be planning for AI to take jobs? Or that they should just keep going as is.

NikolaListen, I think AI will accelerate aspects of certain jobs, right? And that would mean that you can do things more quickly. If your job is highly repetitive, then yeah. I think you can do a part of your job and maybe impact the headcount. I don't think that's the main bit, right? I think that we don't see a lot of our clients thinking that way. I think Demographic pyramids look the way they look in most of, Western Europe, America, et cetera. People are struggling to fill jobs, right? And that's, I think, the theme for our generation. So I think we've got to just figure out how to come up with extra resources, more than fearing for AI.

NikolaWill a moment come where maybe we see a flip and realize that, You and I both have been completely replaced by technology. I don't know. I find that pretty hard to imagine, but if it does, we can pick up the old Marxist literature and see what we'll do with it. With all our free time, right?

StevenYeah, all our free time. We will self actualize. That's the, we'll self actualize. It's happening.

NikolaYeah, I think so. I'm looking forward to it.

StevenNicola, thank you so much for the time. I really appreciate it.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. What if your contact center became a product team?

    Nov 17, 202635 minAudio

    Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...

    Credit the people on this episode

  2. What happens if your AI agents get 1% better every day?

    Sep 10, 202639 minAudio

    An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...

    Credit the people on this episode

  3. Who's coordinating your army of AI agents?

    Sep 3, 202644 minAudio

    Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...

    Credit the people on this episode

  4. Why should CX leaders care about MCP?

    Aug 20, 202636 minAudio

    When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...

    Credit the people on this episode

  5. Can AI really hear a call the way a person does?

    Aug 6, 202622 minAudio

    PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...

    Credit the people on this episode