Host Damien and PolyAI CEO Nikola Mrkšić discuss the latest in an increasingly vague spate of regulatory actions: this time, the AI Framework Convention treaty from the Council of Europe. The treaty, aiming to balance AI advancements with human rights and democracy, is broken down for its practicality, technological neutrality, and the challenges of achieving global parity. The duo explores the tricky dynamics of AI regulation, especially in contrasting legal systems and geopolitical contexts, highlighting the importance of including major players like China. They debate the 'significant impact' clause, implications for small vs. large AI developers, and touch on potential loopholes for national security exceptions. Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
NikolaIf they're going to have parity and truly global standards, they have to get China on board. The fact that the UK, you and the U S agree. They agree on most things are important to like your mind, the other life, right? So I think to go a bit further than that really matters
DamienHello everybody. And welcome back to another episode of deep learning with PolyAI. I'm your host, Damien. And as with me, as usual is CEO and co founder of PolyAI. Nikola Mrkšić. Nikola, welcome.
NikolaDamien. Good to see you. Great to see
Damienyou as well. By the way, deep learning with poly a the only podcast that has this incredible view behind Nikola Nikola. What's with the new background?
NikolaOh, I moved recently and this is the reason I chose this place. I think that having a good view, I grew up on the 11th floor of a communist skyrise in Belgrade and the view you could see like half of central Serbia. So ever since I've been trying to replicate that experience and getting there.
DamienYeah. Chasing that literal high. It's a gorgeous view. Enjoying the backdrop. Mine. It's still the same, but some lovely work I'll discover the artist and give you a background on them in the next episode. I'm sure. But thank you for joining. Today I wanted to talk about some recent news that came in the headlines coming out of the council of Europe and the next step. And I think Globalizing some sort of approach as to how we can regulate and legislate this massive monster that is AI in general. Which is funny when you think about how local municipalities, two large global entities and governing structures are tackling this sort of thing.
DamienBut they just released a treaty. It's actually called the council of Europe framework convention on artificial intelligence and human rights, democracy, and the rule of law. I'd like to see how they decide to shorten that, or at least give it an acronym because right now, SEFICA, it just doesn't work. What this particular 12 page legally binding framework for the participating nations and municipalities that essentially tells the world. Okay, fine. AI, but people and local laws first, and as opposed to seemingly similar legislation, like both the EU and UK is AI act.
DamienAnd some of the stuff that are popping up in the States here, state by state, this one is mostly a playbook to help establish those sort of sweeping regulatory actions. And what's notable about this one, and I'd like to get your opinion on this, is that it's technologically neutral. It's not saying you need to do this with particular systems. It's saying, Hey, if you're going to regulate, keep these keep these tenants in mind. It's already got the founding set of nations has includes heavy hitters, like Angora, San Marino, even the Holy see the Pope's getting in on the action as well as European union, UK, the States.
NikolaI've historically been involved in a lot of important legislation.
DamienReally? Okay.
NikolaIf you just think of like from the inquisition onwards, I'd say legislation
Damienthat goes above the the state capitol,
NikolaThey've been pretty active. Yeah.
DamienYeah, it's it's cool. They got to stay in the zeitgeist. But anyway, as a whole, what I thought I would do is I took a look at the 12 page doc, I pulled out some of the more interesting elements to it that I thought I would get your opinion on as a practitioner, as a researcher in AI, and just also someone who's just very interested in the idea of the global politic. So I wanted to go through and get your feedback on that. So let's start off with. The purpose of why this framework was put together the purpose of the convention in general was to establish this framework and they're guiding like, or their guidelines their guiding forces were to pay respect and to.
DamienTo keep in mind the concept of human rights, democracy, and the rule of law as it's included in the title of the doc. But also acknowledging that we need this global approach to AI governance. So first question to you, Nikola, just in general, in practice, how can entities like this ensure that a global AI framework is going to be effective, first of all, and adaptable? To this, to the plethora of different like cultural and legal context and precedents that already exist worldwide.
DamienIs it even feasible to have a framework that every country and municipality can adopt?
NikolaThat's a really good question. I think on quite a few levels, right? The legal part is interesting because even the countries that are signatories of this do not have the same legal systems or even like legal frameworks, right? From like law to Roman law. I think really what it is, it's just if you're a Lord of the Rings fan and, Diane speak and speak, and I think Mary or Pippin ask have you made a decision? Are you going to war? And they respond with we have only just finished saying hello. You think about the fact that they needed to sign a document that is there to say that things we do should uphold the rule of law and human rights.
NikolaI'm really happy that I guess post Brexit, my taxpayer money doesn't go to these guys, but since the UK is a signatory and the U S is as well, I guess we both funded their great work. Okay, cool. But if we put skepticism aside, I think that the only real way you could go about creating a global governance framework for AI, for those things that will end up mattering for. Human race for people, for economies, for companies is something like this, right? It has to be aligned at least within the context of this new cold war that we're in this side, our side, right?
NikolaSo you look at that and it's the obvious thing that's missing is China, right? Had they managed to do that, we could talk about. A UN style thing, right? Is it UN or is it a league of nations? That's and will it fail like the league of nations? I don't know. But they've started, and I guess that's a good thing. All the things they talk about are somewhat like non controversial, right? There are ways in which I think the real burden is on what product and how. So like when you look at the EU act, it goes a lot further and it's a lot more opinionated about, modalities, things, et cetera, that systems maybe have to do for example, disclose that it's an AI system versus a human one.
NikolaSo I think there's really nothing particularly obviously controversial and what we've seen here. I've not read it word by word, but it sounds like it's just really. Maybe these companies, my company, sorry, countries, assigning stakeholders will be part of the conversation going forward.
DamienAnd that's what it strikes me as well. One, and it's in its briefness, the fact that it's a 12 page guiding document or document that is supposed to be like universally adopted. When you say people first and you say local laws first, then it's pretty easy to just go and make some some suggestions thereby afterwards. But it is still a legally binding document. It does include some things like, mandatory disclosure of AI systems being involved and a very interesting element about the ability to push back against output that's created by AI systems. So maintaining human autonomy which leads into.
DamienIs this a thinly veiled, essentially like guiding document that says people should always be at the center. And while there's some more concrete terms in some regional legislation, like in California right now, they have this state bill that's in the docket 10 47, that basically mandates a kill switch. This doesn't go so far as to say that, but it does say we need to maintain control at all times. So is this sort of just a touchy feely framework that says people should always still be at the center of these systems, no matter how autonomous they may get.
NikolaI don't know. I think it's, it depends on how it plays out, how it's enforced. Like these things can have pretty massive repercussions, right? I think that in the U S you still don't really have speed cameras that can autonomously issue a fine for speed. In the UK, it's get them in the mail daily, not daily, Are you too fast,
Damientoo furious over here, nickel? What's going on?
NikolaI've never had, I know I have zero points on my image.
DamienOh, well done.
NikolaAt least as far as the speed cameras are concerned, I have never exceeded the speed limit, but the real implication of that is in California, like you will not get. Something in the mail saying you owe X dollars because you were speeding or because you went through a street, which was. I don't know, monitors don't go through or wasn't the UK will happen all the time. So if you say when you go through airports in the US when they take your photo to go through TSA pre, like a very clear sign saying, if you don't want to be photographed, tell us. And then I'm guessing the, I've never done it, but the police officer has to take a look and make a judgment themselves.
NikolaThis sounds like one of those things, like it's an inbuilt, maybe, Blocker to a wide scale deployment of these things. And, I think that just makes for consumer and adequate products later on. Again, it depends on the situation, right? And I think in the limit of it, it could prevent something very nefarious, right? I think the other tenant here is like the anti discrimination thing. And that's something that's been talked about in the context of machine learning systems long before the Geniax, where you train the system on data. The system is.
NikolaNot, making a clearly delineated decision based on a number of parameters, right? where it says like I have decided to set Nikola and Damien aside because the bags under their eyes are like Substantial and they've not shaved in a week, right? It goes. Let's just set them aside, right? If you can't explain it, then you know, you can literally look at different groups and how they're treated and you know You can basically then retroactively see if there's a bias in that system against a particular group.
NikolaThat's obviously a bad thing. And there's always going to be this tension between models that make decisions on latent parameters that we don't fully understand. But they do a better job, right? As humans, we call that intuition,
Damienright?
NikolaIf I trust your gut feeling around which podcast episode is going to be more popular, or, you trust my gut instinct around what kind of models kind of work for a certain application. It doesn't matter, or my gut instinct around which pub is better. What kind of fight that I see in the street, it's you're basing that on experiential evidence of me being good at that, or like maybe on the fact that I've been to a lot of pubs, you can't really, I can't tell you why A is better than B maybe, but I have a strong intuition. Yeah. It's that
DamienMalcolm Gladwell concept of intuition. But as we've discussed in previous episodes, like if, as we're fighting towards or, moving towards AGI you Will intuition be replicated?
NikolaIt has been replicated. No. I think at its base level, any statistical system that takes a large number of parameters and doesn't create a rule based system that says, if this parameter is this, and this one is this, and this one is this, I will do this. Sometimes you can disentangle it, but sometimes those criteria become quite nonsensical, because they are statistical, and they capture some other latent factor that you cannot name. Okay. Now that latent factor happens to be race and the system is biased against. That's a system that is by itself discriminatory, right? Sure.
NikolaOkay. Maybe it's actually really good at figuring out which, potential suspects or something you want to interview. But then it also ends up false tagging a certain group of people of a certain race while they're not. Where a certain kind of clothing item, then you're going to be like, Oh damn, right? What do I do with this?
DamienSo de biasing to this framework or what this framework is suggesting and, to, to a point of anti discriminatory like completeness is, that seems like a monumental task when and it seems like it's anti progressive.
NikolaIt's actually the thing that keeps popping up pops up in our daily work. And it's this, right? Like we have very different criteria for technology than we do for humans. A human, a police officer at an airport telling anyone, Hey, you've been selected for random screen. I'd probably say that when you weren't selected for random screen, right? And that way, that legal responsibility just dissipates, disappears, right? Whereas if an AI system does it, you can track exactly who did, what did, when did, right? Like you're probably logging and putting a lot more scrutiny on that system.
NikolaI'm not saying that the system cannot be biased. But so can humans, right? Yeah, very
Damientrue. In
Nikolathat situation, when we put those humans back in the center, what we're really saying is we're going to trust intuition, maybe we're going to trust our biases. A lot of our biases are based on experience and they're actually true, right? And if you go that way, then you're just saying in those situations, I'm going to label it as a human instance of something, rather than a human. Machine and then you don't have a system that discriminates, but maybe your police officers still do it. And that's like a controversial topic. I don't many people even want to talk about.
NikolaBut
DamienNo, but it's actually, it is actually addressed in this text. And again, a brief 12 pages dedicated to it. But a very notable excerpt from it states that these guidelines don't necessarily apply to things like local law enforcement, as a matter of fact, it promotes local laws first and taking prioritization, but then also exonerates defense systems, military, anything that's there to protect the citizenry is exempt from these guidelines. I think that was an interesting little piece that came out of those 12 pages.
NikolaYeah. Like it's quite a rebellion, isn't it? Cause what is that national security and what is. I think there's like certain exceptions for certain private corporations, vested interests, and it all gets diluted in the whole legislative process. Okay, whatever. Business as usual, right? If you consider all that, it just goes back to just like overall posturing and pretending to know what you're doing, right? Big government and, there you go. You become more libertarian, right?
DamienIt's the get out of jail free card for the fact that they know that's going to likely be the first, highlight example of the public sector who adopted these, these guidelines or this framework all of a sudden instantly violates. And it's usually coming from for national security measures or something to that effect.
NikolaYeah, that's the computer vision example I had is it can be taken to an extreme, which is what kind of battlefield, right? And obviously I think that all governments are quite keen to have autonomous security drones. And you can see from like recent conflicts that they're being used with devastating effects. So if that's examined, then you begin to wonder, like what is the impact of all this? Cause a lot of innovation tends to happen in the defense industry because they often get that, get out of jail free card. Of course, from DARPA for speech recognition. So the CIA could, listen to Russians at scale.
NikolaI'm probably the other way around. Like all the way to, today, like it's yeah, I don't know. It's probably means that we're spending a lot more money in companies like, others,
Damienbut at least we're doing it for the sake of human rights and and prioritizing local laws. I am curious about this other little excerpt from the document itself. So according to the convention and this treaty, which covers both public and private sectors involved in the development deployment and use of AI systems, it clarifies that the convention applies to AI systems that have a quote Significant impact on individuals without real explanation behind what a significant impact is.
DamienWhy is this phrase in there? Why is it so nebulous? What is your interpretation of significant impact on individuals?
NikolaI think it's probably a lot of deliberately debate, right? Like in that if we think about our context like some of our calls have a significant impact and that it may have to do with your, gas, electricity, internet being restored or not, is that significant? It is, if you're sitting working from home and you're really in need of internet, right? But yeah, I think it's Look, legislation should not go too far, right? The world changes, technology evolves. I think it's fine to leave it up for measure. And then, you'll have a case of something. And in the end people will a verdict one way or the other will define it for a specific industry or a specific use case, right?
NikolaSo I think, and again for common law countries, Makes sense, right? I think you could almost trace Brexit to the fact that, like incompatible legal systems and the one major common law countries out of the Roman law block, right? Now, of course, when you look at the EU Act, what's really interesting is that it goes a lot further. And it tries to prescribe in certain situations and certain applications, what is okay and what is not right. And that's Roman law where you have to legislate for it and you have to define what is like right and wrong.
NikolaWhereas I think the beauty of common law is that you can just say this is bad. And that's up to you. And the high level thing is do the right thing and then you leave it for a judge in a highly followed case to make a verdict on like how much in their own were you, did you make a significant impact or not? Or is someone just after you because they're looking to get like a juicy settlement, right? That's the downside of terms and low American legal systems where anyone can sue you for anything. And you need someone to opine a bit more heavily or maybe even a jury before, like The court just dismisses it, whereas like in Roman law, you're able to like, make that judgment call a lot more quickly.
DamienBecause you have some conscript in place that says this is against the law, as opposed to what we're talking about here with this where we're not giving legal advice on deep learning with poly AIA.
NikolaNo, of course. I've never actually commented on matters to this extent, right? But it's really, when you think about Roman law, like European systems, it tends to be a lot more prescriptive around this is good, this is bad, in this situation, this, in this situation, you must do this. And then it's a bit onerous. It can be a bit too legislative, right? Sure. But then equally, you know that you've done something right because it's stipulated as such, whereas in common law, you'll have a verdict that changes the course of future legal history and what is right or wrong within a legal framework, right?
NikolaSure.
DamienYeah, it makes sense. And I don't know maybe to dumb it down. It just, it allows, it permits for the establishment of precedent in real time for something like this based on use case. But I think it's interesting when you were talking about sort of businesses or organizations that have significant impact, that's not what this framework is really, to me, that's not what I interpret from this, it is, does the AI application, does the system itself have significant impact? And I wonder if this is maybe. Counter productive towards involving and making sure that smaller technology companies I say technology companies, it is technology neutral, but smaller AI developers are having less of a voice because they don't fit that qualifier for significant impact, which I think is bad practice.
DamienAs a whole, especially in established nations, what do you think, do you agree with that take, or do you think that this is, is it putting too much power in the bigger developer of AI systems hands when it comes to lobbying and impacting local legislation?
NikolaI don't know, if you think about significant impact and like applications as such, this in theory should make it a bit more difficult for larger products, greater footprints to. At AI without any concern that it might be impactful, right? And in that case, Microsoft adding features to their products, it probably becomes a bit more difficult for them to experiment than for a new company where they might be able to claim that it does not yet have significant impact. Again, we don't know who the arbiter of that is now, on the other hand, with any legislation, regulation, compliance, it benefits larger players, right?
NikolaYeah. If one company has invested a lot in. Streamlining and showing that they are compatible with the country's legislation rules, mandates for a specific use case and undercut. And the other company may maybe actually follows all that. And it's like completely. Doing the right thing, but it's not able to easily prove it. One has a market competitive advantage over the other, and that is just work with us and you won't be in violation of this thing.
NikolaThat's long been a tool that hyperscalers and large tech companies use against smaller ones. I think, every series ABC founder knows the work involved in getting through. Like all the compliance certificates and big shout out to Toby and our team and the whole team has gone through years of things and, changes to our systems processes and trying to create changes that won't hamstring us and cause us to be really slow to comply with all that. And, we've been at it for years. So when a new entrant is trying to do something purposeful and have an impact on certain things, it can be really difficult until they build out that sophistication and make the investment in demonstrating that.
NikolaThey're complying with all these
Damienthings. Yeah. Shout out to Toby heads of RIT has all the accreditations for security and compliance. Thank you so much, Toby. One of the last things I wanted to touch on is going back to what you just mentioned with regards to size of organizations and how this would affect them, maybe giving a little bit of a competitive edge for the smaller, more agile companies that are less. Significantly impactful from that terminology. But let's talk about interoperability on a global geopolitical perspective. This thing is supposed to be an overarching framework. But we've talked about both in this podcast and I think ad nauseum as we've grown as a company and reach different areas of the globe, how difficult it is to do localized business because of prohibitive existing commerce legislation.
DamienI'm looking at you, France. Really tough. But what are, do you think are some of the key challenges and opportunities for this level of international cooperation in the field of developing AI initiatives and does a framework like this make that effort easier? We have to see it
Nikolain practice. I think that for now, any extra legislation, any of these things does not make life easier, right? Okay. They make. A lot of people's lives harder. And then we do this enough times that we comply. And then we have a competitive advantage, right? But I don't see either of the two as a particularly good thing, I think for like true concerns that many of the thinkers in this space have comes down to like global parity and use of these systems in like the new cold war. So until they can find something like And, I don't think it's reached that stage yet, but it very well might you need something like a nuclear arms treaties around, rockets range where they're placed.
NikolaSimilarly, I think, when China catches up in terms of the chips and the ability to just autonomously do what the West does. If they're going to have parity and truly global standards, they have to get China on board. The fact that the UK, you and the U S agree. They agree on most things are important to like your mind, the other life, right? So I think to go a bit further than that really matters. And so far as global governance and the development of AI as it pertains to human life, really. Because if the Chinese.
NikolaBuild products that incorporate AI to, maybe violate one human rights privacy, for instance. But they get vastly superior, systems for doing God knows what will emerge here. I'm not sure that there aren't many people in the West that would not subscribe to that product. I had this discussion with someone about opting into Google maps. And it's very easy to be like a privacy zealot. And say, don't track me. Don't do anything. But I don't know about you, but I love knowing which road is busy and which road is not what I'm driving. And that actually depends on me being willing to disclose my private data, which I do.
NikolaHappily. And especially I think, what I found coming from, maybe a lower trust society and behind the Iron Curtain, coming into the West, which I think is amazing for the high trust society that it has, where, things are assumed to be good. And, people collaborate and share things. It's funny that the more autocratic regimes are coming from behind and like using the fact that they're just willing to legislate differently and behave differently with these large companies. Resulting in them building superior products. That's the talk, right? I don't, I know you watch a lot of YouTube, but most.
NikolaI don't think YouTube shorts have caught up with TikTok just yet, right? And I hope they do, right? But it it will be interesting to see. And then, as we think of AGI developing, there will be certain kinds of data sets and things available in China that may not be available in the West, right? And then, we may very well quickly change
DamienWhat's going to get what's going to be means
NikolaIn this piece of legislation.
DamienSo what's going to get an autocratic nation, an isolated nation, at least when it comes to like governmentally playing nice countries like China on board with something like this. Is it focusing it more on the private sector where it's a little bit looser and more difficult to enforce, or is it making it completely out of the private sector and keeping it in the public? Which again,
NikolaI think it has to be like when it comes to overall, like AI governance, I think it has to be over the private sector. Cause that's where most things happen. I think that this has happened before, right? If you speak to people who, have an informed opinion about like intellectual property, they will tell you that the willingness of Chinese companies for the public sector and private companies to really like uphold global IP legislation. Has increased massively as they started producing a lot of their own IP and were at risk of others than taking their intellectual property and capitalizing on it, right?
NikolaSo I think with AI, like we're not in a world where one side is so far ahead of the other. It is at the moment ahead, but not by I think an unreachable difference. I think that means basically that in the future you need some kind of regulation. If you care about even having regulation, the alternative, of course, is just like hands in the air and who cares?
DamienSo have something, but nothing too heavy, nothing too overreaching, but if you don't have something, then we recognize that you are intentionally pursuing like developmental chaos.
NikolaI don't know. I think it's really hard to even have an opinion on this yet. Between, that's what
Damienmakes it a fun topic to discuss. Because
Nikolaclean, like meta releasing models left and right, just undermining this whole I think there are a lot of people with a heavy interest in saying our models are so far ahead. And we're, use our services, pay us a lot of money for using these models because nothing will catch up, so I think what Zuck and Yalicun are doing by pushing these models out is, creative destruction. It's basically saying look, we're not number one. We're not number two. To hell with everything. Go on
Damiena bender.
NikolaSo really wild night out, but like big fan, and I think what they're doing, the. You know how, emotional discussion can get when people put it in a geopolitical frame and, immediately you'll find a more likely than not a grandstanding American or a Brit saying that releasing this model is to give up our way of life, democracy, rights, et cetera. And then it's ha. So I'm going to Facebook releasing a model that like, here you go. And the truth is it's just a global parody and it's if all the countries had nukes wouldn't matter Right because some idiot does theoretically, right?
NikolaSo I don't know it gets interesting, right? It's a different version of the future of the world I think that like this thing feels quite slow and European to me.
DamienOkay.
NikolaIt's like France mandating that bread can only be what like wheat salt and water. Okay, cool. But what if what if Damien finds this killer ingredient and he adds it and bread becomes 20 times better, right? We should let you do that.
DamienI'm just putting Gabba and everything these days.
NikolaYou already, said bad things about France. So I'm just going to dump on them. Let's dedicate it to the point where one of us gets banned in the country.
DamienThat's how you make headlines, baby. Cool. All right. This was a enlightening chat. I love it. I can't wait to see what happens with this particular treaty and ones like it as it makes more ripples through the global footprint, but good talking with you as always, Nikola, maybe next week we'll get back to general enterprise practice. Maybe we won't, who knows, but it's always a pleasure.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...