Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Big Small Talk

Big Small Talk artwork

Do We Need To Pop The AI Bubble? With Cory Doctorow

Aug 12, 202654 minAudio
  • Apple Podcasts
ShareXLinkedInFacebookWhatsAppEmail

Everyone has an opinion on AI. Some say it's the inevitable future. Others think it'll take our jobs, fuel misinformation, and hand even more power to Big Tech and billionaires. This week, Sarah sits down with bestselling author, technologist and journalist Cory Doctorow to unpack whether AI really is the future or just a wildly overhyped tech bubble. They also dive into Cory's theory of the "reverse centaur", why bosses and billionaires are so obsessed with AI, why he believes it's one of the dumbest financial investments of our time, and where Australia fits into the global AI race. Cory Doctorow appears at Sydney's Festival of Dangerous Ideas from 22-23 August and Now or Never on 25 August at Melbourne's The Capitol, presented by The Wheeler Centre. Cory's books mentioned in this episode: Enshittifcation: Why Everything Suddenly Got Worse and What to Do About It The Reverse Centaur's Guide to Life After AI: How to Think About Artificial Intelligence Before It's Too Late

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

A lot of these AI weirdos, they're effectively religious maniacs. They believe that they are about to make God, and God is going to solve all the problems AI is creating. Hi, I'm Sarah and this is Small Talk. It's not quite a news headline, but it is what the girls are talking about. And I am so excited to have Corey Dockertreau in today. I don't want to put any sort of pressure on you, but we have been waiting and waiting and waiting to do an episode on AI.

And I've been honestly a little nervous to do it because every time we mention it on the podcast, we get so many messages from people. saying if they hate it or that they love it or that it helps them or that they're terrified. And I'm excited to speak to you because I think you cover so much ground. You're a prolific writer, a science fiction author, a blogger, an incredible thinker. And so you come at this at both a creative and obviously a super intellectual perspective, but you're also a technologist. And I feel like you're someone who understands both the promise of AI and the reasons of why people are so worried.

What's your thoughts on everyone having such strong opinions on this topic? Yeah. Well, it's funny, you know, I've been an activist with the Electronic Frontier Foundation for 25 years and working on the core digital rights questions about whether you're going to be spied on or manipulated or whether evidence is going to be gathered against you to be used by authoritarian governments. And all of those issues are infinitely more important than AI. And I wrote this book because I got so sick of people demanding that I talk to them about AI. It turned out to be a terrible miscalculation because now everyone wants to know.

to talk about the book, but I'm happy to do it. AI is fine. It is a normal technology. The thing that's abnormal is that AI is backed by a giant investment bubble that has, like, real identifiable reasons for its existence. They're not good reasons, but we know what they are. It's not like the Earth got hit by a financial meteor. And because of that bubble, we are destroying the climate. We're firing workers who do jobs that matter and replacing them with bad chatbots that do those jobs badly. And eventually the industry is going to run into money because they are the money losingest thing the human race has ever done.

And then most of those chatbots are going to be switched off. And so we're going to have a really catastrophic outcome when we – lose both the labor that those workers did and then the labor that the chatbots did badly. And at the same time, you know, a third of the American stock market is now seven AI companies, six of which are losing money to the seventh one. When the bubble breaks, when people, you know, on the institutional investment side stop giving them literally hundreds of billions of dollars to piss up a wall, then we're going to vaporize a third of the

deal with this the way we've dealt with every other financial crisis this century. It will be chased very quickly by brutal austerity, which will drive people into the arms of fascists. So those are all reasons to worry about AI. But the fact that we have like a new technique for statistical inference that does some interesting things sometimes, but also some stupid things sometimes, that's like that almost the least important thing about AI, right? All that other stuff, that epiphenomenal stuff, that's the stuff we need to be worrying about. There's so much in this book that you've written, The Reverse Centaur's Guide to Life After AI.

And I think that's such an interesting focus because your view seemed to be that this is this sort of overhyped bubble that's sort of driven by this desire to just replace the workforce. Is that sort of what you have landed on? I think there's three questions in that question, right? The first one is, is it overhyped? The second one is, is it a bubble? And the third one is, is it about replacing workers? So maybe I'll answer all three of those. There's the whole episode. We've got the perfect layout. Yeah, yeah. So it's overhyped.

It's overhyped in the sense that... You know, these statistical inference techniques are very powerful. They produce things that under normal circumstances, we call a plugin. Plugins are fine. You know, my first word processor, because I'm an old weird nerd, was a program that I bought in a magazine at a news agent that was printed in the pages of the magazine I typed into my home computer. And in the years since, you know, 45 or so years since, like I've seen lots of new word processor features come and go. A lot of them were OK. Some of them were awful and some of them made writers, in my judgment, worse at

writing. And now there's a new plugin that generates plausible word salads. And like, I guess there's some applications for that. And that's fine. But like, we don't need to bet the world economy on it. And the argument is that if we just keep like throwing money at this and throw money at teaching words to the word guessing program, that it will wake up. And it will become God or something and maybe turn us all into paperclips. And if it doesn't, then it'll be able to do all of our jobs and then no one will ever have to work again. And this cites what the lawyers call facts, not in evidence.

So it's very true that AI started off this current bubble, this current wave of technology, started off about 10 years ago when some researchers who worked in machine learning said, hey, we have these machine learning techniques. We know how they work. What if we try it a little differently? What if we try minor variation in how we get it? Maybe we'll get some better results. And they didn't just get better results. They got much better results. That's always really exciting, right? There's a piece of low-hanging fruit. You just change the way you interact with this algorithm a little, and it gets a lot better. It's cool. And it turned out it wasn't just one piece of low-hanging fruit.

It was in like a low-hanging fruit orchard because they could just do more of it, and it would get better. But the thesis that... Getting better at guessing words and pixels and other things that we do with the statistical inference technique eventually results in consciousness. It is incoherent, and it's incoherent in the same way that believing that breeding horses to run faster and faster will eventually lead to one of your prized mares giving birth to a locomotive, right? Because it implies that we're just good word-guessing programs.

Or here's like maybe a concrete way of thinking about it. Sometimes AI boosters will say things like, you can predict what your spouse is going to say, and the AI can predict what your spouse is going to say. Therefore, the AI understands your spouse the way you do. And... Here's the flaw in that argument. It is true that I and you and everyone you know sometimes repeat ourselves. And the autocomplete on your phone is pretty good at guessing what you're going to say next. But while I can predict what my wife is going to say sometimes, when she says something that is like not what I ever would have predicted because she's upset or happy or something major is going on in her life –

I can hazard a pretty good guess at what's going on because I understand her. I don't just have a table of how often one word follows another when she talks. I have a theory about her that is what we call understanding that results from my consciousness. And it's true that if you get, like, better and better at guessing what word she's going to say, you will get better and better at guessing what word she's going to say. But that will not improve your understanding of her. That just improves the fidelity with which you can guess her words.

And if you can't understand the difference between being able to guess words and being able to understand why those words are being said – then yeah, you might like believe that teaching words for the word guessing program will make God. But that's because you're very bad at understanding how people work, not because you're good at understanding what consciousness or understanding is. Yeah, it does seem that the whole human element of this is why everyone's so upset, but it's also why we're seeing these billionaires or people that are owning offices so excited to get the human element out.

And I know you've spoken about this before, about how maybe bosses are overexcited and it is at its core an ego thing. to be able to feel like they only need them for their business. And these billionaires can say that they're these geniuses that are behind this. It seems it's all human, ironically, all very human nature. You're kind of getting to the next two parts of this. So like, is this a bubble? And then is the bubble driven by the desire to fire workers?

And so I'm actually going to take those in order because I think it is driven by the desire to fire workers. But I think that that story becomes a lot crisper when you understand that there's an irrational amount of money being committed to this and that even if you could fire a bunch of workers, it would still be a bubble. So can we start with just like, is this a bubble? Yes. Can we please? Okay, so let's talk about the expenditures to date because profit, this is weird that we have to explain this, but apparently there's a lot of very rich people who don't understand it.

Profit is the money you spend deducted from the money you make. And if you spend more money than you make, then that is called a loss and not a profit. So the money spent here, well, when I wrote the book about a year ago, the total global expenditure was in the $600 to $700 billion range. Today, it's at least a trillion dollars more. And that's just in a year. And Gartner, who are not always the most reliable people in the world, they are the people who said we'd all be spending seven hours a day in the metaverse by now.

But Gartner says it's going to be 2.5 trillion in total by the end of this year. So whether you think it's 1.7, 1.8, 1.4, 2.5 trillion, it's in the trillions of dollars. And then we have how much money they're making. They're making $50 billion a year. $50 billion sounds like a lot of money unless you have to spend a trillion dollars to get it. If you had a trillion dollars and you wanted $50 billion back, I could do that for you in my sleep. I put $950 billion under the mattress and then I give you back the other $50 billion.

I'm such a good-natured slob. I'd give you $60 billion and call it a deal. So now the question is, can they turn $50 billion annual revenue into a trillion dollars annual spending? And the answer is no. And here's why. So there's a thing when you're doing accounting or finance called capital depreciation, right? That's like how often you have to replace an asset. We know about this, right? You buy a car, you drive it off the lot. It's worth half as much, right? And so eventually it goes to zero and you have to replace it. So how often do they have to replace all of these hard assets, data centers, GPUs, and so on?

Well, they claim, like when they depreciate them on their tax returns, on their books, they claim that these are five-year assets. They actually, in practice, replace them every two to three years. It kind of doesn't matter, though, whether it's two years, five years, three years. Because if you're making $50 billion a year, but you're spending a trillion dollars a year – and you have to spend another trillion dollars every X years, it doesn't matter whether that's five years or three. If your assets only last three years, and you made $150 billion from them, or they lasted five years and made $250 billion from them, you still need to spend another trillion dollars because you've worn them out.

And it's hard to overstate how much these get worn out. A lot of people get really angry about data centers, and I think rightly so. And they worry about things like the noise, the carbon consumption, and so on. But there's another thing to worry about, which is that this industry might not ever finish work on the data center. You might end up with just a giant concrete slab the size of like seven football pitches in a place where you used to have a farm that's been expropriated through eminent domain. But even if they do finish it.

One of the ways that the big chip manufacturer, NAI, this company called NVIDIA, is keeping its revenues going. They're the one company that all the other companies lose money to. The way that they keep that revenue going is that they make a much better chip every year. And the way they're squeezing all that performance out of each year's chip is by throwing away the idea that this year's chip should be backwards compatible with last year's chip. And so they're like making chips that have totally different energy consumption and heat dissipation networking requirements.

And that means that from year to year, it's often the case that like if you want to put this year's chips into last year's data center, it's cheaper to just scrape the data center down to the foundation slab and build a new one. than it is to change the cooling, the energy distribution, and the networking within the data center because the chips are so different. Okay, so you have trillion dollars expenditure, $50 billion revenue. You have an amortization schedule, depreciation that runs between two and five years. And then you have seven companies that are 35% of the S&P 500, the most important index of the US stock exchange.

And as I said, these seven companies are just passing around the same $100 billion IOU really fast and pretending it's in all of their bank accounts at once. And like, look, in my other life, I'm a science fiction writer, write techno thrillers about forensic accounting and accounting fraud. And I have to say, in my professional opinion, calling that wheeze accounting fraud does violence to the noble accounting fraud. Like, it's so stupid, right? So this is where we're at with this industry. They're spending money that is far in excess of their revenue.

Their revenue is not growing anything like their expenditures are. They're having to recur those expenditures on a really frequent basis. And the core finances of the companies that dominate the industry and the economy as a whole – right? Those core finances are so bad. Their balance sheets are so ugly that it's a wonder they don't burst into flame. So that's why I say it's a bubble, right? Yeah. What I find shocking when I listen to this, and it's on an ethical level, which seems obvious at the moment, but even now just hearing the financials of it, this idea that this AI bubble could burst or will burst, and that there's this very, very, very slim chance, if not impossible, of utopia that we could find with this.

And it doesn't seem to matter, though, if it fails or not. And I think this is the thinking behind so many people that are pioneering it or running it or in charge of it, It sort of seems this like frame of mind that, well, someone's going to do it, so I might as well be me. Like, might as well have my name on it. Do you agree with that? Why are people running towards this? Yeah, so this is taking us nicely into the third part, which is like, what's this about? Why is all this capital being raised? And I think we can split... at least when we're talking about the billionaires who are kind of the, they're like the first money in.

Now there's a lot of banks and stuff that are putting money in, which is really scary because like banks are structurally important to the economy as we all discovered in 2008. But, you know, this started with just a, just a gigantic amount of capital allocation by like Silicon Valley billionaires, other kinds of billionaires and like Emiratis, right? So what are they betting on? Well, I think that there's a lot of billionaires for whom the idea that workers aren't really real and nothing they do matters very much and they could be easily replaced with some software, that that's like a very easy thing to believe because I think a lot of billionaires are basically just solipsis.

They don't really think other people are real. Look, how could you inflict as much pain as you needed to inflict to like make billions of dollars if you thought other people's suffering was as important as yours, if you thought they were as real as you? You know, how could you go to Epstein Island if you thought the children being raped there were as important or as real as your own children? Right. These are people who already call the people who disagree with them NPCs, non-player characters. That's Elon Musk's favorite insult. And, you know, when you look at their metaphysics, I don't know if you remember – back when Sam Bankman Freed was going to prison, we all learned about his weird idea of effective altruism, which started off as this idea that like, if you're going to try and eliminate malaria, you should figure out whether it's more effective to like buy mosquito nets or distribute chloroquine.

Right. And, and like, that's perfectly reasonable, but it turned into this thing where they're trying to maximize the utility of all good that they do in the world, which cashes out to this extremely bizarre idea. And I'm not making this up. Hmm. That you should earn to give, which is to say you should do the highest paid job you can find no matter how many people you hurt, provided that you then commit that money that you've made to doing good in the world. But the way you do the most good in the world is not by helping the 8 billion people who are alive today because there's only 8 billion of them.

Whereas in 10,000 years, because of AI, there'll be 10 to the 53 artificial people like in the deconstructed bones of Venus, which we'll have turned into a giant computer. And so you should commit all of the resources that you get from harming 8 billion living humans to infinitesimally improving the lives of 10 to the 53 artificial humans on Venus in 10,000 years.

It's a very solipsistic way of looking at the universe. And so I think a lot of those people, they're just like, yeah, let's do it. But I think there's another group of people who are just cynics. You know, look, you don't have to believe that like peptides and supplements will make young men into love gods to think that Andrew Tate can sell a lot of peptides and supplements on his podcast. And you might very well invest in Andrew Tate's peptides and supplements business without believing that they work. Right. You don't have to believe that putting a jade egg in your vagina is good for you to think that Gwyneth Paltrow can sell a lot of vaginal jade eggs.

Right. So there are a lot of people, I think, on the capital side of this who are going, look, bosses. are total suckers for anyone who shows up and says, how would you like to replace your mouthy workers with pliable software? How would you like to never have another ego-shattering confrontation with someone who knows how to do things, who you have to convince to do those things to make your ideas real, but when you tell them about your ideas, they insist that they're illegal, immoral, will kill people, and will have you all in prison.

What if you could just have an idea, and then the software would shit it out as a product? And in the same way that there are a lot of sad young men who will buy whatever peptide Andrew Tate is selling, there are a lot of bosses who will buy any automation tool that is claimed to be a replacement for workers. And so, you know, maybe some of them believe that the AI can do your job, but I think a lot of them believe that an AI salesman can convince your boss to buy the algorithm, even if the AI can't do your job. Yeah, this is the ego part of it.

What that leads to is, you know, when most of our output and products then eventually become down to a few people that just want something that's going to agree with everything that they say, you know, we sort of do get, in result, the inshittification of everything. Sure. Obviously, a word that you've coined and massively popularized through your book. How does that relate to all of this? Like, how does that now play into the output? In shitification, it's this idea that when companies are not disciplined, right, by competitors, by their own workforce, by regulators, or by what economists call new market entrants, like so you make a product and you deliberately make it worse.

And rather than someone competing directly with you, someone makes a product that just fixes the thing you broke and eventually your customers become theirs. So like think here about like... The inkjet printer cartel, four companies make all the inkjets in the world. They have raised and raised the price of ink to the point where it's $10,000 a gallon. It's the most expensive fluid a civilian can buy without a permit. It'd be cheaper to print your grocery list with the semen of a Kentucky Derby winning stallion. And under normal circumstances, you'd think, OK, well, someone's going to enter the market and sell you cheaper ink, right?

Except here's what actually happened. The U.S. Trade Representative, when they negotiated the U.S.-Australia Free Trade Agreement in 2003, got Australia to agree to what's called an anti-circumvention law, where it's just illegal to change how the inkjet printer works. And so, you know, you can go to Australia, sell your inkjet printers, pick the pockets of every Australian who needs to print a grocery list, and no Australian business will be able to enter the market with this complementary good that makes the printers better by making them accept any colored water, because Australia has made its courts available to American companies to destroy Australian companies that protect Australians from those American companies.

So you take away all the things that might go wrong if the worst people's worst ideas are implemented. And those ideas become the best money-making ideas. They don't have to worry about a competitor. They don't have to worry about a regulator. They don't have to worry about their workers. And they don't have to worry about new market entrants. And so why wouldn't they make things worse? You go to a physicist and you say, like, can you explain something abstract about the way the universe works to you? And they'll say something like, well, let us first imagine a simplified model where you have a perfectly spherical cow of uniform density moving on a frictionless surface.

Right. And the business school professor version of that is like, let us imagine the ideal firm. It pays nothing for its inputs. It charges infinity for its outputs. And yes, that is like the academic publishing industry. But everyone else has to accept some discipline. If you pay nothing for your inputs, your workers won't show up for work and your suppliers won't ship you supplies. If you charge infinity, customers won't pay for them. And so you accept some discipline. You work for suppliers, market competition, regulators, and so on all make you accept a lower margin than infinity.

And the more of those constraints you take away, the worse things get. So that's in shitification. So how does it affect the EII sector? Well, yeah. Like in shitification, as I talk about it mostly, is a matter of what happens when a company that would be profitable anyways, who you rely on, gets scared of running out of growth headroom, right? You know, if you're Google and you have a 90% search market share, you're not going to keep growing your market share because like everyone uses you. They get scared that the market is going to start treating them as a mature firm.

They're going to sell their stock because it's overvalued because they're not growing anymore, that the net worth of the executives running those companies are going to decline. And so they decide to start seeking growth through extraction rather than like signing up new customers. So they're not going to get more customers. They're going to get more money from each customer by making the product worse. So, you know, Google did this in 2019. They just turned off a bunch of things that made it better at guessing what the correct search results should be. So that you'd have to search two or three times to get your results. So they could show you two or three like tranches of ads, right?

So that's normal in shitification. The thing is that AI is not a profitable business, right? AI is like the money losingest thing we've ever done. AI is the human race perfecting the money furnace where you just have to shovel $100 bills in it all day long. And no matter how fast you shovel them in, it's always in danger of going out and you have to shovel faster. Yeah. And so on the one hand, that means that like the AI companies are probably going to incentivize more, right, because they're like going to try and figure out how to get more hundred dollar bills to shovel into the money furnace.

But on the other hand, like one of the frustrating things about incentivification is Google could just be an extremely profitable search engine without deliberately making themselves suck. Whereas like AI could never sustain itself with these big frontier models that are so subsidized it's as though the company bosses are selling you $100 bills for $1 apiece. Right? That like they're – bound to collapse. And because AI is very opaque, you know, we hear the industry say that their products are defective with the way they describe it as they say, oh, we have hallucinations.

And so like, how could you tell, right, if you search the AI and you say, like, tell me what the best air conditioner is. And instead of giving you a good air conditioner, it gives you an air conditioner where they get a big commission. How could you tell whether it hallucinated? Right. So like maybe what they've done is they've gone in and they've said, hey, I do that one thing that you're extremely good at, which is multivariate analysis. So look at all of our customers and figure out which ones are paying the least attention. Like maybe parents with small children making online purchases before eight in the morning, like just click whatever the first link is, because they're the ones who are like trying to buy something they've just run out of.

So if you think it's a parent with a small child before eight in the morning, give them the shittiest deal. Right. Whereas if it's someone who's like paying close attention, then give them a better deal. And because everyone's working in their own app, they can't see what deal the other person is getting. It's hard to know. So, you know, there's lots of opportunities to and shitify. Right. Effectively undetectively. But the end result is very different because you don't end up with. with a company that maybe it collapses and we replace it with one that's sustainable or whatever. AI is as a industry that like builds giant data centers to run frontier models is not only like the walking dead, it can never be made sustainable.

AI is like a bunch of tools that you run locally on your computer that aren't, you know, like burning up entire like coal faces to make child pornography on Grok. that's like a normal business and it'll probably be around for a long time and it'll make some people some money. It's just not going to be like the future of the human race. What I'm hearing as well is like, you know, we have all of these reasons to understand why AI is so damaging on that level.

But then also we haven't even touched on like the security breaches, privacy, impact on environment, propaganda, child pornography, deepfakes. And yet knowing all of that and knowing like what you're saying is that we know that this is not economically viable. Why is this not enough to stop investors? Like, is the only way to stop AI, like, does it not seem to matter what all the ethics are, is the only way to really stop this if we can prove it's not economically viable? Well, I don't think we have to prove it. We've proven it.

Well, no, in the sense that if something's not economically viable, it just stops on its own. There's this law in finance called Stein's Law, right? And it goes anything that can't go on forever eventually stops. So I'm confident enough that AI is going to go away like on its own when the bubble bursts out. That that's not the thing that I worry about so much. I do worry that we don't know how long it's going to go and how much damage it's going to do before it bursts, because in addition to anything that can't go on forever, eventually stop Stein's law.

There's also John Kenneth Galbraith's famous saying that the market can remain irrational longer than you can remain solvent. Yeah. And the problem is that as long as the bubble runs, it does a lot of harm. And you mentioned some of the harms, right? The harms are things like deepfake pornography, the replacement of skilled workers with incompetent chatbots, the risk of, you know, the economy having an even bigger hole blown in it. And then there's the like risk of the bubble of every bubble. So. The thing you have to understand about bubbles, whether it's the housing bubble or the bubble that we're living through now or the cryptocurrency bubble or the metaverse bubble, is that the point of them...

is for insiders to lie about an investment opportunity so that normal people take the money that they're saving for retirement because we don't get company pensions with a guaranteed benefit anymore. We all have to go into the market and make bets on stocks. And if we bet wrong, we starve to death or we're homeless when we get old. So these people who are saving money so they don't end up starving and homeless when they get old, give the money to the lying insiders. The lying insiders trouser it and then run away before the investment collapses. So the longer this goes on, the more of that we're going to get.

So this is the reason to shut down the bubble, irrespective of the environmental questions, which are very real, and the deepfake porn, which is very real, and so on. Those are all reasons to, like, be doing something about this right now. And, you know, the AI companies are starting to look for exits. Right. Right. So a lot of the news that we've heard in the last couple of months, including one of the things you mentioned, which is the security story about the AI that escaped containment and the AI that's so good at finding bugs, you're not allowed to look at it or see how good it is at finding bugs and so on.

Those are all kind of best understood as overtures to a flotation to an IPO. which is the most efficient way of allowing insiders to offload their holdings onto everyday investors who are being lied to, who are just trying not to be homeless and starving in their old age. And the first salvo here was Elon Musk with SpaceX. So SpaceX is a moderately profitable government contracting rocketry company into which Elon Musk stuffed rock, which is, again, like just this money furnace that burns hundreds of billions of dollars, as well as Twitter, a company he paid way too much for and then destroyed the business case for.

And all of the insiders who gave him money cashed out on the SpaceX IPO on the flotation. And the way that they were able to do that is that Musk did a sweetheart deal with the people who run the Nasdaq exchange, which is the most important high tech stock exchange in the world. It's the one Bernie Madoff used to run. And Nasdaq gave him a waiver. for inclusion in what's called the index. So if you have a retirement fund, you probably invest in what are called index funds. This is what everyone's told to buy.

So what you're doing is you're buying a little bit of all of the top firms in a given stock exchange, whether that's the FTSE or the NASDAQ or the Dow or whatever. And so you buy an index tracker and the people who manage that fund, so say it's the NASDAQ 100, they're looking at the top 100 companies on the And if the top one is worth 5% of the exchange, 5% of the fund is made up of shares of that company. And if the second one is worth 3%, then 3% is made up of that company.

And every day, constantly, they're rebalancing that so that the portfolio looks exactly like the top 100%. Normally, you can't get into the top 100, even if you would qualify otherwise based on your valuation, unless you've already been on the exchange for a couple of quarters and you've been profitable and growing through that period. Because a lot of times, companies go public and then they tank because it turns out that once the market understands them, they get really bad. That's actually what's happened to SpaceX. It's lost all of its IPO pop and more.

But by including it in the index on the day of flotation… It meant that every fund manager in the world who was managing a NASDAQ 100 fund had to go out and buy SpaceX shares at any price. And Elon Musk got another waiver from the regulator that allowed his insiders to unload their shares on the day of flotation. Normally, you're locked out for like 90 days or 180 days or sometimes longer. So that if you're an insider who knows that it's really just a pile of shit, you can't just flog your shares the day it goes public. You have to hold on to them.

Oh, God. And in this case, what happened was Musk sold almost no shares of SpaceX from the company. So these fund managers who had to buy SpaceX at any price could only buy them from insiders. And so all those Silicon Valley billionaires and Emiratis were made whole and then some from the everyday savings of everyday savers who had just done the prudent thing and bought an index. This is why it matters that even though this is a bubble and even though... eventually it's going to collapse. We want it to collapse as fast as it can, because every day that goes by, someone else is going to be, you know, using their life savings to make Dario Amadei or Sam Altman or one of these other creeps, you know, Larry Ellison, hole on their catastrophic AI bets.

It's just the fact that, you know, billionaires throughout this are just going to be so out of touch and continue to exist a thousand miles away from the consequences because they're protected from their wealth. But what you talk about and what you've interrogated in your work as well is that really what that means is that the ultimate loser here is the ordinary people who will fill the economic sting and lose their jobs. Yeah, well, and again, what we do every time there's a financial crisis is austerity. I don't have to tell Australians about this, but I'm living in the UK these days.

You can hear by my accent. I'm a British citizen. I was naturalized. I'm from Canada originally. We're cold Australians. But I live in the UK. There was just a big study about what happens to people's voting habits when our health service, the NHS, the National Health Service, shuts down their doctor's office. And what they found is that the likelihood that you will vote for a fascist party goes way up when your doctor's office closes. Because when your doctor's office closes, that is catastrophic for your life. And then when a fascist comes around and says the reason you have lost the bedrock necessities for a good existence is that an undeserving other has taken them.

And then you see the rise of fascism, which is itself very destabilizing. And, you know, a lot of these AI weirdos, they're effectively religious maniacs. They believe that they are about to make God and God is going to solve all the problems AI is creating, right? You know, you have Elon Musk, who, you know, styled himself as an environmentalist for many years. That was the whole Tesla pitch, who says now... It's OK to emit unlimited amounts of carbon because AI will solve the climate emergency.

And when medical regulators and officials in the United States government at the National Institutes of Health were begging with Doge, the Elon Musk operatives who were shutting down big parts of the American government after Trump was elected and saying, please don't shut down these long running cancer studies. Because we need them. Once you shut them down, we can't revive them. They've been going for years. They're critical. These teenagers who work for Musk laughed and said, we don't need cancer research because once the AI comes, it's going to solve cancer.

And so it's not just that they're like, oh, yeah, you know, New Zealand will sell us a bunker and a passport to live in when everything goes to shit. And then we can emerge, you know, with a thumb drive full of Bitcoin and an AR-15 and live as warlords as seen in that important documentary about Australian life, Mad Max. They also... some of them think that they're making God and God is going to solve all the problems they've created. And also like, it's so they've created the problems that are so intense.

Now it's the only way out is through. We're just going to have to keep trying to make God. And if we bet wrong, God help us all, so to speak. It is believing that it's God in a way, but then also believing in the efficiencies of it. And I think there's an interesting part of this is like even in the context of you had a cancer diagnosis, you're now cancer free. But AI, you know, was used to help in the medical fields of that. But then what's sort of an interesting thing? element is then looking at even who's benefiting from that in the medical world.

Like the problem is now is AI is being used to cut resources in order to pay for the AI, to pay for the technology so that if anything does go wrong, they can blame the remaining human or doctor. So it's sort of now being used to replace rather than augment like it could. Yeah, you're getting at one of the – at this idea from the title of the book, the reverse centaur. Yes. I can't believe we haven't even explained that yet. Sorry. Yeah, yeah. I'm dead. So, no, no, not at all. So, this isn't the first time that – let me – a sidebar here.

Like, the funny words are fun. I like saying them. But, like, the point is to engage with the serious policy discussion. If you go away without knowing what the funny word means, it's fine. As long as you understand the policy stuff, fine. But let's talk about the funny word here for a sec. So in the history of automation and labor, when we study that, what we see is that when automation is driven by labor, it's typically in service to improving quality. And when automation is driven by capital, it's typically in service to increasing throughput, so making more.

And that's because, you know, if you're the boss and you bought a subscription or you bought an asset and the asset's only going to last for so many years and you want to get the maximum return on that, then you're going to use it as much as you can. Right. And and if you're part of a cartel or a monopoly or duopoly, you're going to use it as much as you can. You can probably make inferior products at a lower price. You fire workers who would make the product better. You replace them with machines that make more of the product that's worse. And then you foist it on the public because they've got nowhere else to go because you're part of a cartel or a monopoly or a duopoly.

And so that's the normal trajectory of automation. And it shows you that like automation is not good or bad. It's like the contingencies of how it's adopted that matters. My other life, I'm a science fiction writer. I always say the point of science fiction is not to explore what the gadget does, but who it does it for and who it does it to. Right. What are the social arrangements? And that's the crux here. So a centaur is someone assisted by automation, whether you're using a spell checker or a bicycle. You know, you are a person. You are the human head on the horse's body. You have the intellect, the discernment, the judgment.

And then the machine is like the body. It's stronger than you. It has more endurance than you. It's faster than you. But it doesn't tell you what to do. It takes the orders. And a reverse centaur is the reverse. It's when the horse's head is on the human body, when the machine is directing the person. When the person is effectively conscripted to be a kind of peripheral to the machine, there's some part of the job the machine can't do. And so you're being used by the machine. But you don't just get used by the machine because, remember, you're slower, weaker, and have less endurance than the machine. So when the machine uses you, it uses you as hard as you can go for as long as you can go, which means you get used up.

by the machine. So Amazon's American warehouses are the most automated warehouses the world has ever seen. They also have tripled the injury rate of other warehouses in America. And that's not a coincidence. It is the consequence. Because when you spend seven figures automating a warehouse, you run it as fast as you can, right? You use that automation. And the human is the slowest part of it. And when a human is asked to work at the absolute limit of their endurance and capability, they eventually make a mistake. And when they make that mistake, if you're in a warehouse, you might get impaled on a forklift.

And so that's the centaur reverse centaur story and applied to radiology. It's the difference between, hey, you've got 10 radiologists here. They cost you $3 million a year and they do 100 x-rays a piece per day. Why don't you spend another million dollars? Yeah. and get some software that sits on their shoulder, and a couple of times a day taps them and says, take another look at that x-ray, I think maybe you missed something, which is a centaur arrangement. And as someone who is now, according to my oncologist, what I am is considered disease-free.

So I had cancer, I was treated, I'm now diseased, I'm not, I haven't beat cancer, but I'm considered disease-free from cancer. I think that sounds like a dandy arrangement, but I think it's far more likely, and I think the investor story is that the salesperson walks into the hospital administrator's office and says, fire nine-tenths of your radiologists, save $2.7 million out of the $3 million you spend on their wages, split it between my boss, Sam Altman, and you, and take the remaining radiologists and have them mark the AI's homework and turn them into the accountability sink.

So when someone dies, you blame the radiologist whose signature is at the bottom of the x-ray. Radiologists today actually have a lot of market power because they're scarce. But the point of AI is to erode that market power. And when it does, radiologists will flip from the first arrangement, which is the dominant one today, where radiologists primarily direct the adoption of automation in their workplace to one in which their bosses do at the expense of quality, which in this case means at the expense of people's lives, in order to erode their wage bills and trouser the savings.

And I suppose this is why we're hoping, even though it feels like it's such a catastrophic outcome, whichever way, that this AI bubble does burst. Because the longer this goes on, the more that happens and the more people lose their jobs and more people start. And I think this is something we're seeing in our inbox a lot of people. I guess, feeling guilty or feeling that they really need to defend that they now have to use AI in their everyday or that they have become a reverse centaur in a way. Otherwise, you don't keep up and you can't keep up with the amount of output.

It's this like ethical debate that everyone's having with themselves right now. Sure. So I think one way to like put – because ethical debates, they're important, but they often generate more heat than light. So let's put a little – let's shed a little light on the debate by talking a little more about the economics of this. So the web 30 years ago, 20, 28 years ago was a money-losing proposition. And a lot of AI boosters will say, yeah – AI is a money-losing proposition. The web is a money-losing proposition. But the web became really important, and so will AI.

And the difference is something called unit economics. So in economics speak and finance speak, unit economics is what happens to your profit and loss when you sell another unit, right? When someone like... you get a new customer who makes a use of your product or your service. And the web had great unit economics. Even though it was losing money, every new customer made the businesses more profitable as a sector. And every new use by a new customer increased that profitability. And every generation of the web was more profitable than the previous one.

Unlike AI, right? AI is, they have terrible unit economics. When an AI company acquires you as a customer, they immediately lose money. And every time you use AI, they lose more money. And that's true whether or not you're paying for a subscription. If you buy the highest tier of subscription, the $200 a month tier that some of these companies are selling, it allows you to use $4,000 worth of tokens a month. So they really are selling $100 bills at $1 apiece. So this is the unit economics, and every generation of it is less profitable than the last.

And then here's the kicker, and it goes back to your inbox full of people who are anxious that their bosses are making them use AI. If you go back and you read the business press from the late 90s and early 2000s, the Harvard Business Review or just the trade press, you will find article after article. hand-wringing about how CEOs are going to cope with the workers who are entering the workforce who will demand that they be allowed to use the web while they're on the job as part of their job. And today, if you pick up those same publications, you will find article after article about how bosses are trying to force their workers who do not want to use AI to...

to adopt it. And I think that like, if you want to understand the difference between a thing that loses money at first and then makes money later, and a thing that loses money at first and then loses a lot more money, you could do worse than to look at these two factors, unit economics, and the driver for adoption. When workers don't want to use it, then basically the minute the boss starts cracking the whip, the use declines. It is fascinating that this is this, you know, we keep getting told the train's on the tracks, there's nothing we can do about it.

But I've never seen or will never see anything like this where everyone is saying we don't want this. And yet here we are. What do you think is going to happen next? when this $1.4 trillion bubble finally pops, if that was to happen in the next, this year? I think that in terms of the technology, it'll just keep going in the sense that we're going to have data centers at 10 cents on the dollar and we're going to have a lot of...

skilled workers who know how to do things with those data centers and those GPUs. You know, we're going to lose the foundation models, these frontier models that just light money on fire at the rate of knots. But there are all these really interesting open source models that many of which were actually released by the AI companies like Meta as a kind of demo, but which turn out to be like far more powerful than anyone thought and have a lot of room for improvement. So remember, this is like a stock swindle. And when you're doing a stock swindle, you are hoping to con naive investors into parting with their savings.

And so since those investors don't really understand the technology, they need some other rule of thumb to assess the investment opportunity that you're presenting. And broadly, the thesis is money out will be a function of money in. Like Google wouldn't spend $100 billion on AI unless they thought there was $2 trillion coming. It's basically the investment equivalent of the idea that if a pile of shit is big enough, it always has a pony underneath it somewhere. Right. So in that circumstance, you don't really want the stuff you do to be efficient. The more money you use up, the more people think that you've got a really cool idea and a giant return on the investment coming at the other end and the more willing they are to part with their money to make you whole and let you exit.

And so what actually happens with these open source models is that anytime people kind of look at them and go like, yeah, I think I can make them run a little better on older, less powerful hardware, they get these huge returns again, just like we were getting 10 years ago with the first – AI computer science breakthroughs, like we saw this very spectacularly. There was a Chinese hedge fund that had a little like Skunk Works, an R&D lab. And they said, here's $6 million. Go tinker with these open source models and see what you can do with them. And they took a model that Meta had published.

And with just a little bit of work, they made it so performant that they were able to found a new company called DeepSeek. And DeepSeek ran so well on old hardware that in one day the market sold off $600 billion American dollars worth of NVIDIA shares. It was the largest decapitalization of any firm in the history of markets by a gigantic margin. So... I think that we'll see like lots of people finding new ways to do cool things with this technique that is itself fine and interesting.

Using a lot of hardware that will be available cheap that you won't run as hot or as intensely as we do now and won't have the same kind of power and cooling demands. And that... It'll be fine. It'll just be a bunch of plugins. And the corollary of this is that, like, if you're the Australian government and saying, oh, we need an AI strategy, like, your AI strategy should be to wait for the crash and buy distressed assets at 10 cents on the dollar. Right. Buying at the top of the market is nuts. So I lived in San Francisco during the dot-com crash.

And all through the dot-com years, all my friends who worked for big companies had these great chairs. They were called Steelcase Leap Chairs, and they were like $1,500 each. And I really coveted one of these chairs. And I wasn't going to spend $1,500 on a chair. So one day after the dot-com crash, after the market falls apart, I'm walking down the street. And like one of those 1929 Wall Street images of a billionaire selling apples off the bus. of his Rolls Royce. There's a dot-com CEO in front of the office he's just lost the lease on with like a fleet of steel case leap chairs still in the plastic for $25 each, right?

Buying AI services, expertise, or hardware now is like buying a steel case leap chair for $1,500 two months before the crash. It's nuts. It is financially imprudent. It's bananas. Just wait and get it on the cheap. Look, if you want a digital strategy for your country, I got one, right? If Donald Trump got angry at you tomorrow and said, all right, we're switching off all the chatbots in Australia, nothing bad would happen. Every company would keep running.

Every ministry would keep running. Maybe some kids would have to do their own homework. Who cares? Right. Meanwhile, Donald Trump has been ordering companies like Microsoft to take away the online services to government officials who make him angry, like the judge at the International Criminal Court who swore at the arrest warrant for Benjamin Netanyahu and the judge in Brazil who sentenced Jair Bolsonaro to prison for trying to stage a coup and take over the government. And if he did that to Australia, if he said, right, Microsoft, turn off Office 365 tomorrow, you basically cease to exist as a country.

Every important firm, every government ministry goes away. If he does it to John Deere, do you remember those Russian soldiers stole John Deere tractors from Ukraine and took them to Chechnya? And John Deere was able to send a kill signal that just permanently immobilized those tractors. He could permanently immobilize all the tractors in Australia. Yeah. Right. He could take away all your IT. What you need to be doing is what the European Union is doing. You need to be building your own infrastructure, which is data centers, not AI data centers, just normal data centers with normal energy needs. You need normal computers running on them.

You need open source software, which the European Union is working on. And you should be collaborating with them on because it doesn't make any sense for Australia to have one stack and Europe to have another. It's like Australia having its own maths for calculating the load stresses of steel joists or whatever. Right. Like everyone just uses the same infrastructure. Right. So, you know, the same foundation. So you improve the software and then you repeal this law that you passed as part of the free trade agreement with the United States in 2003 that makes it illegal to reverse engineer and modify American tech exports so that you can have –

tools that take all your data out of Microsoft Office or that lets you access Facebook without logging in using their app, which spies on you. Instead, you use a different app that kind of autopilots itself to Facebook, pretends to be you and grabs everything there and lets you read it in another app that maybe does the same thing to Twitter and Blue Sky and Mastodon. So it's all in one place. So you're not living in this weird, like 1980s world where you have to know which network someone is on to talk to them. It's like, like, Imagine if like to speak to someone on a Vodafone SIM, you had to have the same SIM in your phone.

It's bananas. So like those are all opportunities you have. You could even be making the tools to jailbreak printers and selling them all over the world. So no one has to spend $10,000 a gallon in ink. And this would not just give you digital sovereignty. It would give you a giant high tech business turning America's trillions into Australians billions. You could be a disensitification nation. Right. And that's like your actual digital strategy. Right. And if you do all of that, well, then like when the AI bubble bursts, you're going to be well situated because you have a lot of technologists and a lot of expertise and whatnot to actually figure out what to salvage from this otherwise ridiculous speculative bubble.

What the core technology that's worth preserving is and to preserve it and integrate it into your other tools. That's a valid futuristic strategy for you. Wow. I mean, the sort of future foolproofing that that plan makes sense and requires is just obviously not the direction, unfortunately, our government is going when, you know, reading the news even this week about them proposing copyright laws that would force AI companies to pay creators using their work to train AI, which sounds amazing. But then we sort of go, well, we've actually missed a much bigger point here that it's not about copyright laws.

Yeah. Yeah. And look, we made copyright laws broader and covering more works and so on for like 45 years. And the media sector is more profitable than it's ever been. And the share of income going to creators who actually make the copyrighted works is lower than it's ever been. Because giving someone more rights when they're bargaining with five publishers and four studios and three labels and two ad tech companies and one company that controls all the e-books and audio books is like giving your bully kid extra lunch money. It doesn't matter how much lunch money you give them. The bullies just take it, right? It's not like, you know, give them enough money and the bullies will run a national campaign saying, think of the hungry artists.

Give them more lunch money. And so, you know, like we tried this, right? We gave – we had the news bargaining code. It was supposed to help the media. Yeah. So rather than preventing big tech from stealing from Australia's media companies by monopolizing ads and monopolizing payments and monopolizing social media, we're like, go ahead and steal as much money as you'd like, but give some of it back to the media so that they become dependent on the companies that they're supposed to be covering. And then surprise, surprise, the majority have ended up in Rupert Murdoch's pocket anyways.

Right. Like doing the same thing over and over again and expecting a different outcome is not actually the definition of insanity, but it's not smart. Right. If we want to help artists like bargain with AI companies, don't give them rights that the media companies will expropriate. Yeah. You know, when the Hollywood writers went on strike, it wasn't because Sam Altman wanted a chatbot in the writer's room. It was because the owners of Warner and Disney wanted chatbots in the writer's room. They're the ones suing for more copyright. They're the ones who will take our copyrights as a condition of us working with them.

And they control access to our audience and they will then sell access to the to the A.I. companies to create the chatbots they're going to use to try and replace us. You know, when you brief for more copyright, you're basically just briefing for the media companies getting a bigger share when you are made a meal of by the media companies and the tech companies. And you're not trying to take yourself off the menu. And if you want to be off the menu, look at how they won the writer's strike. which is that the writers uniquely among American workers have something called a sectoral bargaining right, where all the freelancers are still in a union that bargains with all the employers.

That was outlawed in 1947, but the Hollywood guilds were grandfathered in. And as a result, all these people who were not actually employees, they're freelancers and would otherwise have no bargaining power, were able to form a union against all the employers, whether or not they were, quote unquote, a union shop. And they won that strike. Mm-hmm. And like the idea that we would use more copyright, the thing our bosses want, instead of more labor rights, the thing every other worker would back us on and that our bosses hate, tells us that like we've really not learned anything. Yeah.

Thank you so much for all your time and for helping us zoom out. I think you're right. It's one of those topics that everyone is finding a very hot button topic at the moment. But you're right. You're right in the perspective that we need to be able to zoom out and realize there's other things at hand at the moment in a much bigger picture. Yeah. So thank you for that. And thank you so much for your time. Oh, well, thank you. Of course, you'll be at the Festival of Dangerous Ideas. And that is really exciting to have you there for that. Yeah. Thank you.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Should I Quit My Job?

    Nov 18, 202651 minAudio

    There’s a lot no one tells you when you start working. Luckily, this week Sarah is joined by someone who has made all the mistakes so you don’t have to: Sophie Hirst (aka Soph Workbaby). With a career spanning Nylon,...

    • Apple Podcasts

    Credit the people on this episode

  2. James Bond, Chinese Spy & Daddy Issues

    Nov 17, 20261 hr 2 minAudio

    We are coming to you with some breaking news: Robert Pattinson has been named as the next James Bond, and Sarah has thoughts. Is he the reinvention 007 needs, or a safe studio pick? Then, Justine unpacks how one of the...

    • Apple Podcasts

    Credit the people on this episode

  3. Blame It on Saturn Return

    Nov 11, 202637 minAudio

    If you're in your late twenties and feel like your entire life is being turned upside down, you might just be in your Saturn Return. Breakups, career changes, friendship shifts and existential crises. According to...

    • Apple Podcasts

    Credit the people on this episode

  4. Democrats Win, Russell Brand Jailed & Taylor and Karlie Are Back?

    Nov 10, 20261 hr 1 minAudio

    Well well well. It’s a new week and a whole new world that we’ve woken up to. First up, Hannah explains what happened in the U.S. mid-terms. The Democrats won BIG and we want to know why. What does this mean for the...

    • Apple Podcasts

    Credit the people on this episode

  5. The Feminine Urge to Become a Witch

    Nov 4, 202639 minAudio

    From WitchTok and tarot cards to manifestation and etsy witches - it seems women are embracing their witchy side more than ever. But whats the darker history of witches? For centuries, women accused of witchcraft were...

    • Apple Podcasts

    Credit the people on this episode