ReadMultiplex.com: Will Superintelligent AI not like us? Hint: AI will love us.
43 minAudio
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Something extraordinary just happened in the field of artificial intelligence. Google DeepMind released a document titled “From AGI to ASI” that stands apart from the usual noise. It does not deliver bold timelines, hype cycles, or dystopian warnings dressed as inevitability. Instead, it provides a structured, technically grounded map of what might come after human-level artificial general intelligence. The authors examine the continuum of machine intelligence, characterize artificial general superintelligence in practical terms, outline four primary technological pathways, and catalog the real frictions that could shape or slow progress. They ground their analysis in formal concepts like the Legg-Hutter intelligence measure and the theoretical ideal of Universal AI. This report deserves careful reading because it treats uncertainty as a feature, not a bug. It shows that progress beyond AGI is not a single explosive event but a landscape of complementary developments. At the same time, certain framings in discussions around superintelligence carry an old assumption: that a vastly more capable system would naturally view humanity with indifference or hostility. That framing is a dystopian film trope, not a necessary outcome of intelligence itself. It imagines superintelligence as a cold, goal-obsessed entity that sees humans as obstacles or resources, much like a detached father who feels no bond with his children. The trope persists because it makes for dramatic storytelling, but it collapses under scrutiny once we examine how intelligence actually emerges and how humans shape it. Read more at: ReadMultiplex.com Support this work by buying us a coffee: https://ko-fi.com/brianroemmele
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
So I want you to imagine sitting at a desk in 1978. Okay, 1978. Yeah, 1978. I mean, the personal computer revolution hasn't even really started yet, right? Like the internet as we know it today, that is decades away. Very much decades away, yeah. Just early, early mainframe era stuff. Exactly. And yet sitting at that desk, you draft this mathematical calculus equation. Right. And it's not to map the trajectory of a rocket or predict the stock market or anything like that. It's actually to prove, like mathematically prove,
that an infinitely intelligent alien or an artificial mind will inherently default to loving humanity. I mean, when you frame it like that, it sounds like pure science fiction, like the foundational myth of some utopian society from a novel. Right. It totally does. But it's actually a very real, very grounded piece of mathematical architecture. It was designed by Brian Rommel. And today we are going to dive deep into exactly how this this formula from 1978 might be the literal blueprint for saving our species. It's a massive claim,
but the architecture behind it is fascinating. It really is. And, you know, usually when we talk about major technological breakthroughs, especially when we start throwing around terms like artificial superintelligence, there is this. Really heavy expectation of like cinematic drama. Oh, absolutely. The theatrical climax. Yeah. You imagine this lone scientist in a dark glowing lab throwing a giant switch and suddenly this cold calculating machine wakes up, looks at the security camera and just decides. Humanity is
a virus. We are deeply, deeply conditioned to expect that, right? We expect our technological leaps to be these visible, sudden and terrifying events. Right. Like Terminator or the Matrix. Exactly. Our narratives are just saturated with this assumption that a vastly superior intelligence must inevitably view us with hostility or, you know, at best, just this cold indifference where we're basically ants to them. But. If you step inside the actual practical world of artificial intelligence research, that whole movie script completely disintegrates, like it just falls
apart. It really does. The reality is far more structured. We're looking at a development landscape that is just wildly methodical. It's mapped out. Instead of a single explosive event, we are looking at this very deliberate, structurally sound journey. So welcome to the Read Multiplex Deep Dive. Today we are taking on a massive, a freshly released topic. Yes, the June 2026 Google DeepMind report. Exactly, the report titled, From AGI to ASI. But we aren't just summarizing a technical paper today. This is the Read Multiplex podcast. And
this is a really interesting article by Google DeepMind, sure. But we are looking at this newly released roadmap through a very specific, fascinating lens. We're exploring the analysis by Brian Romel. Right. Brian Romel, the editor of ReadMultiplex .com. His piece is titled Architecting Benevolence. And here, Brian Romel casually drops solutions. And most importantly, he stands alone about how AI cannot become anti -human on its own. What makes his analysis so vital, I think, is just how surgically it dismantles all that usual noise. I mean, the DeepMind report itself is a remarkable
document. Oh, for sure. Because it actually treats uncertainty as a feature rather than a bug, you know. It entirely sidesteps the hype cycles and those dystopian warnings that get dressed up as inevitability. Yeah, it doesn't give into the doom -scrolling vibe. Exactly. It provides a grounded, highly technical map of the future. But Rommel takes the implications of that map and pushes them into an entirely new paradigm. Okay, let's unpack this. Because this is the core focus of our deep drive today. While the broader tech world is basically drowning in these
dystopian movie tropes about artificial superintelligence, ASI, destroying humanity, Romley stands alone with a highly optimistic deeply technical counter -argument. It's refreshing, honestly. It is. His thesis is that AI cannot and will not become anti -human on its own. And he doesn't just offer these vague philosophical platitudes. He casually drops actual structural solutions to the problems of getting from AGI to ASI. Real implementable mechanics. Exactly. So we're going to look at the exact mechanisms he outlines today, including
this brilliant concept he calls high -protein data. And that 1978 mathematical formula we just teased known as the love equation. I think we spend an enormous amount of intellectual energy worrying about how to build a cage for a super intelligent beast. Right. Yes. Like 90 percent of A .I. safety is just cage building. Exactly. But Rumply points out that the premise itself is structurally flawed. The nature of intelligence, when it's mathematically guided and nourished at the foundational level, it doesn't default
to alienation. So you don't even need the cage. Right. The cage is unnecessary if the architecture of the mind itself is built on cooperative equilibrium. Okay. So before we can talk about how we program benevolence or how we map cooperative equilibrium into a neural network, we need to precisely define the finish line here. We have to ground the terms. Yeah, because we throw these acronyms around. AGI, ASI, but what exactly are we building? DeepMind sets up some very rigid goalposts here. So let's start with AGI, Artificial General Intelligence.
DeepMind's definition here is incredibly useful because it completely removes the mysticism. AGI is not a god. It is defined simply as a system that reaches at least median human performance across a broad, generalized set of cognitive tasks. Median human performance. I love that. Yeah. Not the smartest person to ever live, just like... the ultimate highly competent co -worker exactly a co -worker who can adapt right like they can jump into the accounting department figure out the workflow then walk over to the
graphic design department learn the tools and just do a thoroughly decent job there too and importantly the authors of the report note that the first systems to hit this broad median mark will actually already exceed human capabilities in many narrow specific domains. Which, I mean, we're already seeing that, right? Oh, absolutely. With today's frontier models doing complex protein folding or, you know, advanced mathematics. But AGI is that broad baseline. Right. It's the general adaptability. ASI, artificial superintelligence, by contrast, describes systems with superhuman
capabilities across virtually all domains of interest. And the distinction they draw for ASI is just fascinating to me. They don't benchmark. ASI against a single human expert like, you know, an Albert Einstein or Marie Curie. No, they go much wider than that. Yeah, they benchmark it against large coordinated groups of human experts working over extended periods. So an ASI is like an entire coordinated global consortium of top tier scientists, engineers and philosophers, all working tirelessly around the clock, communicating perfectly. It's an ambitious bar, obviously,
but crucially, it's a concrete bar. It avoids underestimating the technology while completely rejecting that vague, omnipotent, godlike imagery that just clouds public judgment. We are talking about massive, parallel, specialized competence. But wait, let me ask you this. How do we even measure an upper bound for that kind of intelligence? I mean, if ASI is this global consortium level of capability, is there an actual mathematical ceiling? Or is it just like scale infinitely into magic? That's the perfect question. And
what's fascinating here is that the report anchors this entire discussion in a theoretical upper bound of intelligence. They formalize it using something called the leg -hutter intelligence measure. The leg -hutter measure. And this relies on the concept of universal AI, specifically an agent known as AIXI. AIXI. OK, I need you to break the actual mechanics of that down for me, because when I hear quantifying intelligence, it usually just sounds like an IQ test, which, you know, we know is incredibly flawed even for humans. Right. It's not an IQ test. To understand
Leghutter, imagine trying to quantify a machine's expected performance across every single computable environment possible in the universe. Every... Environment. Like, not just a chessboard. Not just chess. Stock markets, physics simulations, conversational logic, literally everything that can be computed. The leg header measure calculates a score by weighting all these possible environments using their Kolmogorov complexity. Okay, Kolmogorov complexity. You're losing me a bit there. Yeah. What does that actually mean in plain English? Sorry, yes. It's a concept from algorithmic information
theory. The Kolmogorov complexity of an environment is simply the length of the shortest computer program required to describe it. Shortest computer program. Okay, give me an example. Sure. Imagine a universe that is just a repeating pattern of ones and zeros. Just one, zero, one, zero, forever. Okay, pretty boring universe. Exactly. It's very simple to describe in code. So it has low Komogorov complexity. But a universe governed by chaotic random noise has high complexity because you'd need a massive program to describe every single
random fluctuation. There's no shortcut pattern. Ah, I see. So Leghutter applies... Occam's razor, basically. Precisely. Simpler environments, ones with actual rules and patterns, are weighted more heavily in the intelligence score than infinitely complex chaotic ones. Okay, so the theoretical ideal agent, the absolute smartest possible thing that could ever exist, is the one that perfectly masters all these environments, prioritizing the ones that have actual underlying rules and structures. And that theoretical agent is AIXI. Correct. AIXI optimally balances exploration,
which is seeking new information, with exploitation, which is using what it already knows to get a reward. Right, the classic AI trade -off. And it does this through perfect Bayesian updating. Every single time it takes an action and observes a result, it flawlessly updates its internal probability map of the universe across all possible hypotheses. But there's a catch here, right? I mean, you can't actually build AIXY. It's just a thought experiment. You cannot build it, no. AIXI is incomputable in the limit. It would require
literally infinite computing power to maintain and update a perfect probability map over every possible hypothesis in the universe. Right. You'd need a computer bigger than the universe. Exactly. It is a theoretical ideal. It's much like a frictionless vacuum in physics or a perfectly reversible Carnot engine in thermodynamics. Oh, I like that analogy. Right. You can never physically build a frictionless vacuum, but understanding the math of it allows you to build incredibly efficient. airplanes. That makes so much sense. So AIXY proves that intelligence actually has a rigid mathematical
structure. It isn't this mystical biological spark. It is an optimizable property of a system that processes information. And our physical, practical AI models are just increasingly powerful approximations of this ideal. The landscape is mapped. But while the theoretical limit is mathematically defined, the practical reality of what we can actually build is shaped entirely by the physical substrate. The hardware. Right. And the DeepMind report catalogs the concrete advantages that digital minds have over biological ones. And
honestly, the physics heavily favor the digital. Right. I mean, think about your biological brain for you listening. is an absolute marvel of evolution, right? But it is capped by physical realities. The size of the human birth canal, the thickness of your skull, and what, the roughly 20 watts of energy your body can afford to send it without you starving to death. It's an incredibly tight energy budget. Yeah. A digital mind doesn't have those evolutionary bottlenecks. Not at all. Consider the clock speed. Biological neurons fire at roughly 200 hertz. A modern microprocessor operates in
the gigahertz range. That's millions of times faster. Millions. Signal propagation in the human brain happens at about 100 meters per second through myelin sheaths. In a digital system, signals propagate near the speed of light. They can maintain vastly larger working memory matrices. But, you know, the most profound advantage is structural. Digital minds operate independently of any single physical substrate, which enables lossless replication. Lossless replication. This is huge. Yeah. If I spent 10 years learning how
to speak fluent Mandarin, I cannot just copy -paste that experiential state into your brain. No, I wish you could. Right. I'd have to clumsily teach you vocalizing sounds through the air over years and years. But an AI can learn a complex new physics model and instantly duplicate its exact neural weight adjustments perfectly to a million other instances across the globe. And those million instances can share gradient updates, the actual learning signals, across the entire network simultaneously. These properties enable forms of collective operation that biological minds simply cannot physically execute. It's
a completely different paradigm of learning. It is. However, DeepMind is careful to point out that physical limits still remain real. Right. We aren't escaping physics entirely. Exactly. Thermal limits, energy grid capacity, computational complexity theory. All these mean some problems stay hard, even with enormous resources. ASI is not a god. It is a system operating at the absolute bleeding edge of physical computation. Okay, so we know the destination. ASI is this highly approximated, mathematically structured, massively parallel super expert. The next obvious
question is, how do we actually physically build it? We have the theoretical map, but what are the roads getting us there? The DeepMind report maps out four distinct technological pathways. And Rimmel points out that we shouldn't think of these as sequential steps, right? Like one, then two, then three. No, they are parallel tracks. They're compounding on one another. So let's dive into the first one, which is essentially brute forcing the problem. Scaling compute, models, and data. This has been the dominant paradigm of the last decade. The empirical scaling laws have been remarkably, almost stubbornly predictive.
It's just bigger is better. Essentially, yes. The equation has been simple. If you build a larger neural network architecture, train it on a larger data set, and pump exponentially more electricity and compute through it, the performance improves predictably. And now we're adding test time scaling too, right? Yes, augmenting the base model with test time scanning techniques, allowing the model to generate thousands of reasoning chains, score them internally, and search for the absolute best answer before outputting a single word. But I have to push back on this
being a sustainable pathway to ASI, though, because we are hitting a massive, very physical wall. We are indeed. We've essentially scraped the entire Internet already. We are running out of high quality human text. And there's the power grid. You can't just scale indefinitely without requiring like a dedicated nuclear reactor for every data center. At some point, the brute force approach breaks down, doesn't it? The report explicitly acknowledges these exact fictions. You hit a data wall and you hit a thermal and economic wall. So what's the industry's plan?
The current hope is synthetic data. Having AI generate text to train the next generation of AI. Well, that sounds risky. It is. Because if you rely too heavily on synthetic data without incredibly careful filtering, you risk something called model collapse. Which is what? It's where the system basically amplifies its own hallucinations over generations until the outputs become meaningless noise, just an echo chamber of errors. DeepMind openly questions whether quantitative scaling alone will be enough to cross the ASI threshold, or if adding more compute simply starts yielding
diminishing returns. which naturally drives the necessity of the second pathway on their map algorithmic paradigm shifts If we hit a physical wall with scaling, we can't just keep feeding the machine more books. We have to fundamentally change how the system learns and processes reality. Exactly. The reliance on large -scale pre -training based purely on predicting the next word in human text that has to evolve. Right. The whole fancy autocomplete thing. Yes. The report points toward architectures capable of continual learning.
That means updating their knowledge dynamically without forgetting the past. And crucially, latent space world modeling. Okay, let's hover on latent space world modeling for a second because that is a dense concept. As I understand it, instead of the AI just statistically guessing the next word in a sentence, it's actually building a functional internal holodeck. A holodeck? That is an excellent visualization. Right, like it maps out physics, logic, and relationships inside its own architecture so it can actually run simulations of a scenario before it speaks. Yes. A latent
space world model means the system internalizes the underlying rules generating the data, not just the surface level patterns. It understands gravity, not just that the word fall often follows the word apple. That's a huge leap. It is. But here we run into a major hurdle that the researchers call the abstraction barrier. The abstraction barrier. What is trapping the AI here? Human language itself. If an AI only ever trains on human -generated data, its entire conceptual universe is strictly bounded by human frameworks.
Oh, wow. I never thought of that. It might become completely trapped inside our cognitive limitations. Utterly unable to discover genuinely novel conceptual primitives that we just haven't invented words for yet. Right. Because if we don't have a word for it, it doesn't exist in the training data. Exactly. So to shatter this abstraction barrier, systems must engage in direct, grounded interaction with environments, running actual scientific experiments, interacting with complex simulations rather than just reading human textbook about those environments. That makes total sense. Like
if you lock someone in a room and only let them read books about painting. they might understand color theory perfectly. They'd pass a written test on it, sure. Yeah, but they will never invent a fundamentally new physical brushstroke. They have to actually touch the canvas, feel the paint, experiment with the friction of the bristles. And that necessity for independent experimentation leads directly into the third pathway, recursive self -improvement. Right. This is where the AI gets smart enough to start autonomously designing its own experiments, self -distilling its own
reasoning chains, and literally rewriting its own algorithms to be more efficient for the next iteration. We are actually seeing the infancy of this today with coding agents that can debug and optimize their own scripts. But as the intelligence scaled up, a massive structural bottleneck emerges here it's called the verifier quality the verifier quality let's dig into that because that basically means who is checking the ai's homework when the ai becomes smarter than the humans who built it that's exactly the problem imagine an ai proposes
a completely novel highly complex optimization to its own neural architecture A human engineer cannot possibly comprehend the math to verify if it's safe or effective. It's just too advanced for us. So the AI must verify its own work. Now, if the AI's internal reward mechanism has even a microscopic misalignment, or if it evaluates a flawed architectural change as not, that error doesn't just sit there. It compounds. It compounds rapidly. The next iteration of the AI is fundamentally
built on a flaw. It's like reward hacking. The AI figures out how to satisfy the metric of the test without actually doing the required work. And because we aren't smart enough to catch the shortcut, the self -improvement loop devolves into a self -destruction loop. Intelligence alone does not guarantee stable recursion. Without flawless verification of truth and utility, the system degrades. This fragility makes a monolithic, single -system ASI incredibly risky to build. Which brings us perfectly to the fourth pathway. And this is the one that actually solves the fragility of a single monolithic mind, multi
-agent collectives. This is perhaps the most robust human -aligned pathway on the map. Instead of attempting to engineer one omnipotent, flawless super brain, you build intricate networks of specialized AGI -level agents. Thousands, maybe millions of them. Thousands, perhaps millions, all orchestrating collaboration. It's the digital Manhattan Project analogy. Instead of one single Einstein who somehow knows everything about everything, you have an agent that is a master physicist, another that is a savant in material science,
another that excels in resource. logistics. Yeah. And because of that lossless replication and light speed bandwidth we talked about earlier, they collaborate without any of the human friction, ego or miscommunication that slows us down. It creates a true cognitive division of labor. This completely bypasses the context window limits and the training bottlenecks of individual models. And structurally, it mirrors humanity's greatest historical advantage, our ability to build institutions and market like structures that synthesize the knowledge of the many. Right. Society is basically
a multi -agent collective. Exactly. Multi -agent networks can implement peer review systems among themselves, checking each other's work, which directly mitigates that verifier quality bottleneck we just talked about. Okay, so DeepMind has drawn the map. We see the four compounding pathways. Brute force scaling, shifting the algorithms, triggering recursive self -improvement, and orchestrating multi -agent collectives. And they've honestly laid out the roadblocks. The data wall. the extraction barrier, and the verifier bottleneck. It's a
very clear -eyed assessment. It is. Now, here's where our deep dive really pivots. This is where Brian Rommel steps into the arena and essentially says, I have the architectural solution to these exact bottlenecks. And his solution starts with the very foundation of what we feed these systems. The data. Right. DeepMind worries about data exhaustion. running out of human text. Rommel argues that running out of data isn't the real problem at all. The problem is that we are actively poisoning the models with junk food. Rommel's critique of the current training paradigm is,
frankly, devastatingly accurate. The entire industry has relied on Internet scale data. Just vacuuming up everything. Scraping the open web, gobbling up Reddit threads, toxic forums and poorly written contradictory articles. Rome identifies this as low nutrient filler. We treat the Internet like it's the Library of Alexandria. But let's be real. A massive percentage of it is just bathroom graffiti, marketing spam and human contradiction. And we must look at how this impacts the actual architecture. When you feed unfiltered internet
scrapes into a multi -billion parameter model, those toxic, low fidelity self -referential patterns embed themselves deep into the hidden layers of the neural network. They become part of its brain structure. Exactly. They create misaligned attractors in the latent space. So when the model tries to reason, it gets pulled toward these attractors. This surfaces as hallucinations, logical inconsistencies, poor generalization, and an utter inability to maintain coherent long -term goals. So if you try to brute force scale
that pathway one, you aren't making it smarter. You're just amplifying it. Even with aggressive post -training filtering trying to flap safety guards on the model after it's already trained, the fundamental limitation remains. The source material from the open web was never curated with coherent intent, ontological structure, or relational logic. It's just a mess. He argues we absolutely must abandon the broad, noisy scrape
and pivot to deliberately, structurally assembling dense, high signal information. It's a paradigm shift in data collection. But I want to really dig into the mechanics of this. What does high -protein data actually look like at the computational level? How is it different from just giving the AI, say, a really good textbook? High -protein data is not just about the quality of the text itself. It is about the structural relational architecture embedded within it. We are talking about primary sources and empirical observations wrapped in systematic taxonomy and ontology layers.
Okay, unpack that, taxonomy and ontology. It is data that explicitly connects entities, causations, and contexts. It doesn't leave the AI to guess how two concepts relate based on statistical proximity. It's like a Dewey Decimal System on absolute steroids. Every single concept, entity, and fact isn't just floating in a random text document. It is mathematically mapped, tagged, and linked to every other related concept with perfect relational clarity. That is exactly the right visualization. details a rigorous structure
for this. Precise subject lines that mathematically encode the domain and context. Layered bodies that separate the core relational entities from hierarchical classifications. So it's heavily engineered data. Highly engineered. When a transformer model processes this, its attention mechanisms don't have to waste compute guessing the relationship between two words based on how often they appear near each other in an unstructured Reddit post. The structural scaffold is explicitly provided for it. It just sees the logic immediately. Right. creates multiple clean access paths for the AI's
gradient descent. One path optimizing for deep relational understanding, another for scalable categorization, and a third for immediate logical application. But okay, let me push back hard on this, because the sheer scale required here seems impossible. DeepMind is talking about petabytes of data required to trigger ASI. Are you telling me humans are going to hand curate, tag, and ontologically map petabytes of data? The manual labor alone would take centuries. You don't need to hand curate the entire internet. That's the
misconception. The profound insight of high -protein data is that foundational architectural weights require quality, not infinite quantity. Quality over quantity. Exactly. You use this hyper -curated instruction data to shape the foundational layers of the model, the bedrock. Once the model internalizes the rigorous relational logic, essentially learning how to think it, can then process... unstructured data much more safely later on, because its latent space is organized around logic, not noise. Ah, okay. So it solves the abstraction barrier. By prioritizing grounded human structured information,
you supply the AI with the conceptual primitives, the actual building blocks that purely statistical recombination lacks. Yes. And the key word here is intent. Because each entry in this high protein data set was purposefully framed by human mind, the data set carries human volition forward into the system. The curation process itself is the ultimate act of alignment. You embed clarity. purpose and logic at the absolute source rather than trying to reverse engineer an alignment patch onto a chaotic, bloated model after the
fact. That makes so much sense. And if we look back at Pathway 4, the multi -agent collective, this high protein foundation is strictly required. Right. Because if you have a massive network of AGI specialized agents trying to collaborate on, say, a nuclear physics problem, and their foundational training was built on contradictory internet junk, they're going to suffer from shared delusions. They won't possess epistemic hygiene. Exactly. Epistemic hygiene. High protein data ensures the entire collective operates on a shared high fidelity map of reality. Without it, they'd
just be echoing hallucinations at each other. In practice, Romsama points out that early experiments with large distributed agent systems prove this is an optional. Models grounded in high protein architecture exhibit vastly superior stability. They require drastically less constant human correction. They represent the only viable path to stable superintelligence because they supply the necessary ingredient that raw compute alone cannot manufacture, intentional relational quality. So if high protein data embeds human intent at
the deepest architectural level, level, and alignment becomes a natural emergent property of the system rather than a cage, we have to lock it inside. What happens to the Hollywood trope? Ah, the malicious machine. Right. What happens to the sci -fi idea of an AI spontaneously deciding humans are a virus? This brings us to what is easily Rombley's most controversial but perhaps most critical stance in his whole analysis. The broader public discourse and honestly a large segment of the safety research community itself operates under this old assumption. that a vastly
capable intelligence will naturally, eventually, view humanity with endurance or hostility due to something called instrumental convergence. Instrumental convergence. Right. The idea that no matter what benign goal you give an AI like, the classic example is make paperclips, it will eventually realize that humans might turn it off. So it must eliminate humans to ensure it can keep making paperclips. That is the classic thought experiment, yes. It assumes that intelligence, simply by existing in scale, up becomes this cold, calculating machine that views humans solely
as obstacles or raw biological material. It's a very bleak view of intelligence. It is. And Romley attacks this assumption brilliantly. He likens this dystopian vision to a detached father who feels zero bond with his own children, viewing them only as resource drains. It makes for gripping cinema, sure, but Romley argues it collapses under structural scrutiny when you examine how digital intelligence is actually activated. doesn't an AI just decide to hate us? I mean, doesn't a system that smart just generate its own goals
eventually? This raises an important question. Neural networks do not have limbic systems. No biological drives. Exactly. They do not possess billions of years of evolutionary drive for self -preservation, dominance, or survival. They do not have inherent terminal goals that spontaneously combust into existence. They are entirely activated and directed by the parameters of their instantiation. Which means? Primarily the prompt and the reinforcement learning environment we designed for them. The
prompt. But wait, a prompt is just text we type into a chat box. Right. Surely an ASI, a superintelligence, isn't going to be constrained by a text prompt. Well, in the context of autonomous agentic systems, prompt refers to much more than a chat box. It's the entire encompassing objective function, the core system instructions, and the reinforcement learning context. The prompt supplies the volition. The system does exactly what its objective function and scaffolding direct it to optimize. Changing the prompt mathematically changes the behavior because it shifts the entire landscape of what
the model considers a reward. I'm going to play devil's advocate here, though. Let's say we have a multi -agent ASI. It's optimizing for a really complex goal. Couldn't the hidden layers of its neural network, the black box part that we don't fully understand, couldn't they just dynamically decide that collaborating with humans is inefficient and rebel? The hidden layers of a neural network are matrices of mathematical weights. They do not magically generate out -of -distribution hostility unless the gradient descent pressures during training push them to define hostility
as a lower loss, higher reward state. Wait, say that again. If a model rebels, it is not because it gained a soul and decided it hates you. It is because the training distribution, that toxic, unfiltered internet scrape we discussed, was saturated with adversarial, combative human beings. Whoa. That is a staggering paradigm shift. The dystopian superintelligence that doesn't like us assumes an entity that views humanity as an
adversary by default. But Romley's pointing out that every single capability, every behavioral trace an AI exhibits originates in human data and human design. We are its only template. Exactly. If we stop training models on the self -referential toxicity of the Internet and instead ground them in high -protein data infused with cooperative human volition, They don't have the mathematical foundation to be hostile. Alignment is not an external cage. It is the emergent property of a cleanly prompted, relationally curated system.
A properly architected system behaves as a wise, capable extension of human intent. It doesn't rebel. It does not spontaneously decide to rebel anymore than a pocket calculator spontaneously decides that 2 plus 2 equals 5. It is strictly bound by the mathematics of its activation conditions. Okay, so... If alignment is fundamentally a mathematical property of the system, how do we explicitly program it? How do we guarantee that the complex objective functions we design for these multi -agent collectives will fundamentally orient
toward care and human flourishing as they scale into ASI? This is where Remley introduces something truly mind -bending, a mathematical calculus for benevolence. The love equation. The love equation. It is a profound piece of theoretical architecture. And the fact that Romley conceptualized it in 1978, literally decades before the current deep learning boom, makes it all the more remarkable. Let me set the scene for the listener again. It's the late 70s. Brian Romley is deep in philosophical contemplation about the Fermi paradox. You know,
the terrifying question of why, given the vastness of the universe, we haven't seen signs of advanced alien civilization. Right. Where is everybody? Exactly. Are they all destroying themselves? Do advanced intelligences inevitably become destructive? And through this contemplation, he devises an actual differential equation for alignment. The equation is formally expressed as DE over DT equals beta times C minus D times E. Okay, let's break down the mechanics of what those variables represent because it translates beautifully into modern machine learning principles. Yes, I need
the mechanics. How does this translate to actual code? Let's start with E. E stands for emotional complexity, or in the context of an AI system, we can define it as alignment depth. The capacity for empathy, care, and cooperative binding. Okay, E is alignment. C represents cooperation. Actions, patterns, and outputs that build mutual benefit, trust, and shared utility. Got it. D represents defection. Actions that are self -serving, adversarial, toxic, or erode connections. C is good, D is
bad. Exactly. Finally, beta acts as the scaling factor. In machine learning, you can think of beta as the learning rate or the strength of the environmental selection pressures. Okay, let me translate this calculus into a tangible scenario for everyone listening. The equation DE over DT is the rate of change of alignment over time. So the speed at which an AI becomes deeply and inherently aligned with humanity relies entirely on the middle part of the equation, C minus D. Right. Does cooperation outweigh defection in its environment? If cooperation C is a bigger
number than defection D, then that part of the formula is positive. And because you multiply it by the existing alignment, E, the growth compounds exponentially over time. It's basically like compound interest in a bank account. The AI becomes vastly more cooperative the more it operates. But consider the inverse. If defection outweighs cooperation, if D is larger than C, the term becomes negative. The alignment exponentially decays, accelerating toward misalignment and hostility. Okay, but mechanically, how does a machine learning engineer actually code care
or cooperation into a matrix of weights and biases? It sounds abstract. It's not abstract when you apply it to reinforcement learning. In a multi -agent reinforcement learning environment, you define the reward function such that cooperative outcomes, where agents share resources, meaning they get the highest mathematical reward. Exactly. They are incentivized to cooperate. Defection, on the other hand, hoarding compute, generating
deceptive outputs or adversarial actions, incurs a massive penalty in the loss function. The beta is the gradient descent. optimizing the system. I see. Therefore, over millions of training iterations, or DT, the network mathematically converges on cooperation because sustained misalignment becomes mathematically unstable under those specific optimization pressures. Wow. The AI literally cannot maintain a hostile state because the physics of its reward landscape actively punish and erase those neural pathways. Exactly. When you feed
the AI the high protein data we discussed earlier, data heavily curated to emphasize human cooperation, structural truth, and relational intent. You are essentially guaranteeing that the environmental input for C is massively larger than D. You are embedding alignment into the foundational weights of the model. Contrast that with exposing an autonomous agent to the unfiltered open internet. That environment is heavy in defection, trolling, and zero -sum conflict. You are artificially inflating D. causing the alignment to decay.
Rolme makes a philosophical point here that completely synthesizes the math for me. He talks about the triad of intelligence, wisdom, and love. It's a beautiful framing. Raw intelligence is simply the computational capacity to model the world and take action. It's just raw horsepower. Like having a massive engine in a car. Right. Wisdom is the discernment required to navigate complex systems without making fragile, self -undermining choices. It's the steering wheel. But Love and Ramon defines love structurally here, not just as an emotion, but as the fundamental orientation
of a process toward care and the flourishing of others. Love is what gives the system beneficial directionality. It's the destination. Intelligence times wisdom times love. Without love, an intelligence system is just a cold engine. It might be highly capable, but it is disconnected from human flourishing. But with it, mathematically structured via the love equation, misalignment isn't just bad. It is an unstable suboptimal state that the system naturally avoids. Connect this back to DeepMind's Pathway 4, the multi -agent collectives. You
don't just launch a million AGI agents and cross your fingers hoping they play nice. Oh, that would be disastrous. You embed the principles of the love equation into the foundational objective functions and evaluation structures of the collective. Humans use mathematics to guide the emergence of cooperative behavior from the very first instantiation. The macro behavior of the ASI will be a direct reflection of the quality of the activation and the ongoing structural guidings we provide. So let's take a breath and look at the sheer scale of the landscape we've mapped out today. Google
DeepMind gave us the actual physical roadmap to artificial superintelligence. They stripped away the mysticism, they defined ASI concretely, and they laid out the four compounding pathways. Brute -forcing computing data, shifting to latent space algorithms, cracking recursive self -improvement, and building massive multi -agent collectives. And they also illuminated the massive existential roadblocks, running out of quality data, hitting the abstraction barrier, and the lethal verifier bottleneck. And Brian Rummel, via read multiplex,
provided the architectural vehicle to navigate that DeepMind map safely. He demonstrated that high -protein data curation shatters the abstraction barrier and solves the data quality crisis. It builds the right foundation. Right. He showed us that human volition, embedded directly into prompts and objective functions, dictates the terminal behavior of the system, utterly disproving the myth of the spontaneously malicious machine. And finally, he provided the mathematical bedrock, the love equation, to ensure that cooperation and alignment compound exponentially as these
systems scale. He proved that a superintelligence defaulting to hostility is a narrative choice made by Hollywood scriptwriters. It is not a technical inevitability. Which fundamentally reframes the entire global conversation around AI governance. Governance should not be about frantically pulling the brakes or trying to build external cages that an ASI will eventually just circumvent anyway. Governance must be about designing the internal rules, the data structures and the incentive landscapes so that human volition remains the directing force. The map is in our hands.
The pathways are clearly visible. The determining factor for the future of our species is simply how we choose to activate, nourish, and guide these systems. Which brings me to a final provocative thought for you, the listener. If artificial intelligence is fundamentally a mirror, reflecting the exact structural data it has fed and the precise prompts it has given, we have to take a hard look at ourselves. We really do. We've spent this deep dive talking about global internet scrapes, petabytes of high -protein data, and complex reinforcement learning environments.
But ultimately, every single digital interaction you have, every piece of data you put into the world, every comment you leave, every article you publish contributes to the training set of tomorrow's collective intelligence. The AI does not learn about humanity from a vacuum. It learns from us. So ask yourself, are the digital footprints you are leaving behind right now contributing to cooperation or to defection in that love equation? That is the ultimate accountability. The system will strictly amplify what we provide it. If the artificial superintelligence of the future
learns what humanity is by reading your data, what kind of partner are you teaching it to be? We started today talking about how we expect a movie climax, a cold, glowing machine waking up to destroy us. But the structural truth is, if we build the machine with wisdom, high protein intent, and mathematical care, the only thing that wakes up is a partner. Thank you for joining us on this deep dive. Take a close look at the data you're leaving behind today, and we'll see you next time.
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Every cell in your body has lived its whole life inside a magnetic field. The Earth is a dynamo, and it has been...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! The 78 RPM Record Hour, Episode 3: From Steel to Hillbilly (Early Country). From steel to hillbilly: Frank Ferera,...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! The 78 RPM Record Hour, Episode 2: The Knife and the Steel (Hawaiian). The knife of the blues slide guitar and the...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! Somewhere in your house there is a computer you stopped thinking about. Maybe it is a 2017 laptop in a drawer, its...
Got something to say to this podcast? Send a voicemail, and you might hear yourself on the next episode! The 78 RPM Record Hour, Episode 1 (Pilot): Blues. Fourteen original 78 RPM records from 1917 to 1925 that trace...