The Deep Learning pod is back, and this week Nikola and Michelle discuss effective altruism, its influence on technology builders, and its potential role in the recent OpenAI drama. Is it a guise for-profit entities to deflect scrutiny? Should it be the guiding force for AI regulation, or at minimum, responsible deployment? What's Peter Singer's shallow pond argument? These questions and more answered here! ------------ 00:13 Understanding Effective Altruism and its Impact on AI 00:50 The Role of Effective Altruism in Tech Industry 01:40 The Dilemma of AI Regulation 02:13 The Philosophical Perspective of Effective Altruism 03:15 The Shallow Pond Analogy and its Relevance 03:57 The Data-Driven Aspect of Effective Altruism 05:59 The Cynical View of Effective Altruism 07:33 The Role of Tech Leaders in the New Age 08:37 The Potential Dangers of AI 09:26 The Intersection of AI and Capitalism 09:50 The Impact of AI on the Economy 10:53 The Ethical Dilemmas in AI Development 13:41 The Future of AI Regulation 16:07 The Role of Users in AI Regulation 17:31 The Potential Misuse of AI 19:21 The Importance of Good Governance in AI 20:48 The Challenges of AI Regulation 23:05 The Role of Individuals in AI Regulation Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
MichelleBecause they're already thinking ahead. They're already thinking especially banks that you mentioned, like they've been heavily regulated. So they, they know what's coming around the corner and they know that they already need to start thinking about these types of things now. So I guess the question is it enough? To let your users help regulate you when you're building really fast. If your user is not educated in self protection the way that an enterprise might be.
MichelleHello and welcome back to Deep Learning with Poly AI. Today, Nicole and I are going to be talking about effective altruism and AI examined through the lens of the chaos over at OpenAI. For anyone that isn't familiar, effective altruism is defined as a research principle or an intellectual field that focuses on how we might do the most good For the most important problems empirically. But it's also become this philosophy that's guided many of the most prominent builders of technology today.
MichelleAnd it may have been behind some of that tension between Sam Altman and his board that led to his eventual firing. And bringing back and the board being fired. So if you think like a builder who uses effective altruism as a guiding architectural principle then the core question is probably something like who better to protect the fate of humanity from technology. Then those that are building it and those that actually understand it but for those that are a little bit more cynical, effective altruism might be more like a lullaby. A for profit company would sing to the masses to put you to sleep about safety and self regulation leaving the company without any lucid scrutiny, so basically, there's this idea that because I'm smart, because I understand this technology, I must be smart everywhere.
MichelleWho better than me to save the world? But the recent events, I think, at OpenAI beg questions about nonprofits and for profits coexisting, AI regulation. The scale of Sam's from Bankman Freed to Altman and then the role of startups to really continue to push barriers and perhaps the role of governments and smart people to stop them from going too far. So let's talk about it. Nicola, had you heard of effective altruism? before you founded a tech company?
NikolaA little bit. I think I think the way you described it is fairly accurate, but cynical. I think to me it's like in the long thread of philosophical Protestant thought really, where you go like, how can you do the most good? Should you do the most good? It's like... Predestination and that stuff where, you know, should we behave in a certain way? Does that show your virtues? All the way, like following the soap opera and, like looking more towards the anthropic types and the whole kind of we are the Oppenheimer of this area.
NikolaAnd if we don't behave responsibly, we may unleash upon the world something that will change it forever in a way that we don't like. And I think that's very fair here. I don't think it's really about AGI per se killing people, but more like. You've unleashed a tool that rendered the economy completely ineffective. It could give unbelievable weapons to bad actors or just actors that are not you. So I think that's all perfectly valid. I know a little bit about it from the shallow pond thing.
NikolaI think it's worthwhile if we go through that a little bit just to set the scene for people. I think it's a perfectly valid like philosophy. I think, how you choose to act upon it is really we'll show whether it was a good thing for the world or not.
MichelleYeah. So what's the shell upon thing? Maybe we can talk a little bit about that for anyone who's not familiar. Yeah.
NikolaOkay. So I, I think and I, it's been a while, but I think the main idea is basically if you see a drowning child, and like you're wearing a nice suit, like if no one knows, like you should say that child, that's just like universal morality, yeah. A good thing to do. And then If no one knows that, like you say, you're wearing a really expensive suit and you're going to the job interview of your life and you don't want to ruin the suit and you don't do it. If people know or don't know am I doing it justice you should do it regardless or like another one was just in the grand scheme of you are say wealthy, right?
NikolaAnd you're looking to make the world a better place. It's really about then also being data driven around how you make those decisions. So not just doing like the first thing that comes across your mind that is good, but you quantify it and make sure that. You do the most good possible. So for example, a dollar goes a lot further if you're fighting, like malaria, then if you are solving, I don't know, Christmas presents in the UK, right? Cause it's one fuzzy feeling, but doesn't like actually save anyone's life.
NikolaIt just makes them feel better about themselves. I think the the other interesting one was I think this was Peter Singer one of the philosophers who talked about it, who, I don't know if he was the one who donated the kidney, because I think the theory was, hey I could get I could donate one of my kidneys, and the risk is like one in 40, 000 people who get a kidney extracted. It leads to complications and like they die or something with their life, right? Goes badly wrong, right? So the argument then, I think it's a really interesting one, probably the most effective one here is hey, do you if you do not choose to donate your kidney right now, If there is someone who needs a kidney, like you are valuing your life at 40, 000 times the life of that person, right?
NikolaI think that's a good example of how like the data driven aspect of effective altruism governs that stuff. Now the cynical part where you said I'm like judge, jury and executioner, like what that side of the coin is, it's almost like Leninist vanguardism, right? A few people in the world will get all the money. And, maybe these companies that are implementing AI will have all this value accrued to them, right? So if a few of these ultra rich people that become wealthy through doing this and maybe, through not just creating value, but moving value from others to themselves, right?
NikolaAre inspired by this ideology, then maybe they can justify the chase. Yeah. To pass this possible outcome, because as you said they have good intentions, they have a mathematical, and these are smart people, right? Yeah. If they have this ideology, which is also like mathematical about I will do better with this money that I'm mostly going to give away anyway. I think it's fair to say that most of the people in this industrial revolution, and maybe even in prior ones, but maybe that's like a different podcast, which we should get some time for it wasn't just the money that they're after making the world a better place, they're after having an impact, they're after a legacy, they're after a legacy, it's really good to find them.
NikolaIf they donate it all away and channel it to the right places, then even if they've... Destroyed 90 percent of the world's jobs, maybe. It's okay because they'll do it. Yeah. Yeah. Q and D we're done.
MichelleIt's interesting. You started off your response with the Protestant sort of angle. And I think with effective altruism, with businesses, with governments, you almost need to talk about it through the lens of religion and like being able to separate church and state, this idea that Religious people like religious ideology has been going down quite a bit. And it's been replaced. I forget who said this. There's a really great professor basically said, tech leaders have become like the new Jesus in this country, they've become this new kind of, oh, not cult like, but maybe it is called But they've become, they've replaced this idea.
MichelleYou're
Nikolanot feeling
Michellethe AGI? I'm not feeling the AGI. But yes, exactly. You've got this sort of religious effigy burning, thing with Susquever and OpenAI, and I think that's all speaking to this idea that tech companies and like the people that are innovating the most are changing the world. The most are have become almost like religious figures and they need to be guided by these moral principles in the same way that you might have the 10 commandments. You have this idea of effective altruism and all that comes with it. But, who are the people that are best set up to do, to do the most good for the most important problems?
MichelleGenerally, if you're not cynical, it would be your government. And probably the best way to support that the best form of effective altruism is paying your taxes. Which is something that I think a lot of these very large companies don't do. Let's not even talk about that. I think this idea, though, that you do need to decouple the culture.
NikolaI think that the whole premise is that because they're better at allocating those resources. Yeah. They shouldn't even pay taxes because they will allocate those resources. Yes. Like it's very convenient in addition. Yes. Sounded okay.
MichelleYeah. And if you think about it, if you extend that all the way to ai, now we're also the best to regulate ourselves, the people who are building it. We're the best to say, we think that this is likely to happen. And in order to protect the people from this, we've got, universal based income. We've got a world, Vinod Koles talked a lot about this. I heard him on a podcast a few weeks ago where he said, AI is gonna be the gift to the masses. Not no longer having to work on silly mundane things anymore. Like we'll be able to spend our free time, playing guitar, playing with our kids, doing things that are much more meaningful and that's a really positive way to see those types of things, but who's responsible for making.
MichelleThat, those rules,
Nikolayeah, full disclaimer to the audience, there's one of our investors and far from, me calling him a Marxist with this, cause I think he couldn't be further from that. The idea is not new, like Marxism, but once we, have more productive means of. pRoduction, right? And we distribute it effectively, we'll create a world of infinite. I'm translating from Serbian here from old books that used to be mandatory reading in schools. Not for my generation, but just before. Like you would have a world of abundance and then no one would have to work and we'd all just discuss philosophy, right?
NikolaThe other pointer, I think to the audience, if you're again, one of more cynical worldview is young Yanis Varoufakis the legendary Greek minister of finance, he talks about techno feudalism, and how um, we've gone rather, I think this guy is. I don't know where he stands on the political spectrum, but he'd probably be closer to a communist than to a capitalist. He has gone full circle where at this point he is longing for good old capitalism because he thinks that in this accumulation of power through mostly cloud providers and large tech companies he thinks that we're moving towards a more feudal Kind of like society where, all surfs, right?
NikolaBecause, and, I think a former CRO of ours, Rich Neeson jokingly once explained to me, like these days you're selling software who's your cloud provider? What are the integrations? Who do we need to partner with? Who needs a cut? Who's your telephony provider, right? I think, it used to be the case that. You could benefit from the software, we'll implement it, we'll bring it to you on a CD, you'll pay for it, we'll go have dinner, celebrate, and, call us next year when you need something else. These days really like they are getting deeper and deeper into the value chain of everything.
NikolaYeah. And to whom the value of AI accrues, there's, there are many good posts, but I think this was an interesting one where they like just analyze will it accrue to the cloud? Will it accrue to the one and only NVIDIA? Can't stay in the application layer. Again, I'm steering, I'm going away from effective altruism, but in terms of who should govern it. I don't really know, but like what Vinod will tell you since you mentioned him is that, like you need a few companies that are ahead of it and that it shouldn't be shared publicly so that there's at least somebody that kind of keeps your side of the world right in this kind of like protocol war that we're in ahead of like adversaries, right?
NikolaSo there's a bit of that like capitalist thing where it's just make as much progress as possible with relatively sane people guiding it, right? So if it's a Manhattan project, make sure that, there is some governance structure. I think we saw the one for the modern nuclear era. We've seen the structure shake tremendously. I think it looks better now, but then again I don't really have an answer. What do
Michelleyou think? Yeah, I don't have an answer either. We were talking about this before, but like that, governing mission of OpenAI, the nonprofit of ensuring that AI is broadly beneficial for humanity, like really who's responsible for that. I don't think I have a good answer either. I think I agree with what you're saying, which is that as long as you have. Good actors, sensitive actors that are building toward this at a pace that satisfies like their the pace of change that would probably happen anyway, if a bad actor was also building AI but that then regulation comes later and my, my feeling on this is almost like you, we talked a lot about capitalism and I read a great quote, which said that capitalism was the first uncontrolled artificial intelligence.
MichelleSo yes, capitalism didn't come out of a business but once it took over as a form of economy, economic government, you saw it do things that I think were unexpected. And you saw it snowball in some areas and have impacts on people. And, we talk about wealth inequality all the time in this country, like as a function of capitalism for better and for worse, it came with a lot. And there's a lot, of course that I don't know about, to be able to talk to the nuance of what it means to have capitalism be an artificial intelligence, but it then got this, these layers of control.
MichelleJust even look at the interest rate and what the feds done in the last year. It's got these layers of control on top of it. And the government catching up, trying to look ahead. Trying to put regulations in place to keep it from doing the most damage to the most people. And so I think I agree with that. I think, just to talk a little bit about the non profit for profit, issue as well. It's really hard. To ask a business leader I think it would be really hard for us to ask you, for example, as the leader of this business to split in half and spend half of your time thinking about innovating as fast as humanly possible.
MichelleSo that, so that we're a successful company at the same time that we're asking you to keep it as safe as possible. That's a very tough ask. I think people balance it with things like effective altruism, but. Yeah,
NikolaIf you listen to, to Elon talk about regulators, he's I like regulators. I agree with 97 percent of the things that like they're imposing and that's just way of life and like how it should be done. And I think would AI, again cause we love the cynical view. You could say that like regulation is there just to help like the people who are not ahead. Slow down. Yeah. I know, slow down OpenAI so GCP can catch up, right? And so that Google can have a competitive product, right? I think no one was rubbing their hands more while the OpenAI thing was unfolding than they were, right?
NikolaPeople obsess over Microsoft, but Microsoft already has the golden goose, right? Oh, yeah. I think Tatiana Della was just like trying to keep any... Of the many status quo's that were possible there to keep progress going and keeping that good thing going, right? Yeah. I think like one thing, like you're right, like capitalism as an AI or really it's just Darwinian, it's evolution. So now that like the genie's out of the box, right? Like people will do it, right? People everywhere will do it and they need it. They need more resources where like the layers of commoditization will stop where they might not be.
NikolaLike physical ROI on a bigger model that does all these things like we're interpolating and we're excited and maybe some people are a bit afraid. So like you imagine an AI that can create pathogens that are can exterminate entire continents or stuff like that may not be around the corner, right? Like AGI being around the corner I mean I was sitting, with Sean and Andy last time my co founders, right and I generated an image of a piglet waving the serbian flag being chased by a horde that had the Taiwanese attitude on it While the sky was raining sushi and pork scratchings, and it looked amazing, which probably invented, right?
NikolaAnd I'm like, am I API? Because like I can't draw to save my life. I can't draw, right? With this thing, I was sitting in a restaurant and I generated this thing in a few iterations, right? So it's okay, like we might be there already. We don't know what it means, how effective it is. Will it like disrupt us? So I think like a lot of this like philosophical like ideology That's probably it goes into politics over time, and then it finds one of those governments to reign us in if we become too powerful, or, if the world doesn't look the way that, most people like it to.
NikolaBut it's really interesting. I'm with you, right? I think the government needs to catch up and then do the right thing. I was listening to John Major interview, and he was, like the former British PM. And he was explaining how he had to raise interest rates and inflation was down for years pretty much all the way up until now from the 90s in the UK. And it's yeah that probably lost him election, right? But yeah, that's a good thing. And, if there's like a similar thing with AI where We want to avoid feudalism with Microsoft and Google ruling everything because those means of production are now the most valuable ones.
NikolaI don't know, but this is like a thing that we've seen in history. Like how is Sam Waltman different from like a Dale Carnegie, or someone like that.
MichelleYeah. Yeah. Yeah. Hopefully. Actually, yeah, that's a good question. I don't think Carnegie was a, was an effective altruist by any means, whereas Sam Altman was like he, he was, all the profit in the world. Give me every dollar I can possibly squeeze out of this company, but I'll give it to charity. That was, and that was the name under which he operated and he defrauded and, hurt people. But. But yeah, it was under this sort of umbrella of effective altruism. So I wonder about that sometimes. It's it's sometimes, maybe on the, in that sort of scale of Sam's, discussion, like maybe for Sam Bankman Freed, it was a way for him to hide in plain sight.
MichelleIt was a way for him to be successful without anybody. Being too concerned about him being, whereas, you have a Bernie Madoff and that's gonna raise a lot of eyebrows and Concern a lot of people because it's just in the name of money. But for him he had this effective altruism like Coat of arms on top. Whereas Sam Altman, I think he's seen, I'm not sure maybe he's not, but at least from what I can tell, and the media sentiment about him, he's like good guy AI, right? Like he was the friendly face of AI.
NikolaI think he comes across extremely genuine, and I think he's like really good. I'm happy it's Sam Wolfman at the head of that company and not like pretty much anyone else. I think the guy has high integrity. He, when it comes to just the pure conflicts of interest and stuff, the guy's I think a billionaire a few times over. So hopefully, being sent a billionaire, how do you say, I don't know I don't think that being the richest man in the world is a motivator for him. So I think like it's not about it's not about staffing decision. Yeah. Yeah.
MichelleYeah. But yeah it's very possible too, that this idea of effective altruism, which I think is at the heart of what we're talking about too. It's just like good people at the helm, innovating. empathetically and who are sensitive to the impact that what they're building is going to have on people. Like maybe that is the best form of regulation we have now for AI. And I actually believe that this is true for a lot of companies directly in our space. We are selling to enterprise companies with a lot at stake. If you get what we do wrong, it has a massive brand impact.
MichelleIt has a massive share price impact, but forget about that. It has downstream impacts on real people. And so getting it right is extremely important. And our customers are helping us regulate in that sense, because if something's wrong, they're going to let us know, we're going to feel that pressure and we're going to, we're going to course correct.
NikolaFor sure. You look at and they're often ahead of regulation, right? So I'll tell you that most of our banking clients prefer to actually communicate that the voice assistant is a voice assistant before the conversation starts to the best of my knowledge. do quite a lot of work on it. That's not legally mandated in the U. S., in the UK, in the EU, but people go further because like they imagine what good regulation should look like in this case. So I think like people will over try to do the right thing and like it's the brand impact, but also just they think of their consumers and how they would want to be treated as consumers.
NikolaYep. The world's full of good people. I think it's just it's hard to know what to do in some of these situations.
MichelleYeah. Yeah. But then for a company who's not whose user, is a little more passive or they're not they're not able to see the damage that might happen if the product is bad. So let's pretend for a second that OpenAI is a bad company. And I don't think it is just for the record. But let's pretend that it, you use chat GPT and it became one of the fastest growing like user bases ever. I think it is, I think threads maybe usurped it a little bit, get in their speed to get to a hundred million users, but anyway but let's say that chat GPT, the purpose of it was really to scrape your data and I don't know, somehow defraud you.
MichelleGet your social security number and yeah, open a bunch of credit cards in your name. Let's just pretend like that was the purpose. People wouldn't know that the same way that a buyer of enterprise technology. Because they're already thinking ahead. They're already thinking especially banks that you mentioned, like they've been heavily regulated. So they, they know what's coming around the corner and they know that they already need to start thinking about these types of things now. So I guess the question is it enough? To let your users help regulate you when you're building really fast.
MichelleIf your user is not educated in self protection the way that an enterprise might be.
NikolaDefinitely not, right? I think when you look at every Ponzi scheme in the history of like banking and credits and get rich fast schemes people have, more sophisticated actors with these products have been defrauding people over and over again. I think yeah, you have to trust OpenAI that they're not using your stuff for training and stuff. And they're pretty explicit around what they are using and what they're not. I think there are good ways of, as you expose this for use in other applications, which is where I think it really gets problematic. Because chain of things that gets forgotten around, like, where data goes.
NikolaCan't be problematic. And I think today on Twitter, I saw a paper, I think it was from Google finally, where they've prompted I think GPT 4 or 3. 5 into just saying the word open forever, right? And then after some time, it starts reproducing some of its training data verbatim, right? People's content, stuff like that. Hilarious stuff, right? And like these are neural networks with a lot of parameters overfitting. Has always been like the notion of just remember what you've seen, right?
NikolaWe love LLMs cause they mix and match things. There are all these kind of like effects like where you effectively, gotten things that work better than, they're able to reason in ways that we didn't expect. But yeah, I don't know. It's it's always been possible to recreate. And for example, if you use like a smart keyboard on your mobile phone, my name is Nikola Murešić and my postcode is, I bet you that I think it's SwiftKey. It will tell you what my postcode is and then my phone number. And I love that, like when I'm like, Hey, email me at, I want that to autocomplete, right?
NikolaLike it's just been trained on my data. So anything trained on stuff like this could be used potentially in a bad way and sanitizing and just where I don't think there's malice here as much as it's just like really hard to reliably remove that from all training data. And especially when you're training with this much data. Things slip in and out and like the veracity of what is produced and like the cleaning, safe burning. There's just so much work to do. And that's like where are we? Is it about to, produce pathogens to kill continents?
NikolaOr is it just, maybe accidentally going to leak someone's credit card details as well? Yeah. Like for us, it's the end of the world if we do that and that's why we design in a way that will never allow for this to happen, right? But if chat GPT can do this, is it really the end of the world? Probably not It'll just get fixed, right?
MichelleYeah. Who knows? Who knows? Fortunately, we haven't seen that. Open. That seems like that's one of the worst cases so far of, it exposing sort of its weaknesses rather than actually hurting a user. But yeah I do wonder about one thing with effective altruism, something that you said, which is just that auto complete, which is a very convenient feature on your phone and none of us are personally educated well enough to know. Okay. How could somebody scrape an auto?
MichelleComplete, thing. Could somebody take this somehow? I know you've got key loggers and stuff like that, but are there vulnerabilities that I don't know about that I can protect myself from? Effective altruism. And I think a lot of the doomer ism and like all of this fear, it puts a lot of pressure on individuals and that's. That's been something that I think has been proven to be ineffective for a really long time. Climate change is one example, but yes, we recycle and we do, we make individual choices that might reduce your, our carbon footprint, but really it's not individuals that are doing the most damage.
MichelleIt's not individuals that should be responsible for protection. And so even thinking like, how can I personally, donate, the right organ for the risk benefit, or this ROI analysis, that's been done by these effective altruist mathematicians somewhere. It's really more about how can we set up systematic ways to, to protect humanity. And I think that's for me, like the big. The biggest question for AI regulation is really who's responsible. It can't be Sam Altman, right? It can't be Nikola Murtish.
MichelleIt can't be individual people. It's gotta, it's gotta be something bigger.
NikolaYeah. And the UN safety council is a very effective thing that agrees on everything. You
Michelleknow, consensus all day, all the time. Awesome. I think we are at time. But this has been super, super interesting. I think
Nikolatalking about... I think people find it hard to accuse us of just talking about, enterprise readiness and do well by our kind of like lead generation. So looking to just like also provide a bit of lighthearted entertainment and just like general information to our audience.
MichelleExactly. Exactly. We aim to please. Awesome. So we will meet next week and who knows what we'll be talking about. Maybe religion and tech, maybe changing the world through AI. Nicola.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...