What happens if your AI agents get 1% better every day?
39 minAudio
An agent that gets better every day compounds into a very different agent within months. That's the idea behind Wren, PolyAI's new release: an agent whose job is to make your other agents better. Some of what it finds is a small refinement, some of it is a big fix. Either way, Wren watches every real customer conversation, learns from what happens, and keeps improving your customer-facing agents. Nikola Mrkšić sits down with Arkadiusz Kwapiszewski, Head of Agent OS, to get into what that changes, how Wren is putting agent-building in the hands of people who could never touch it before, and why 95% of PolyAI's own production changes now ship through it. Listen to the full episode, and learn more about PolyAI at https://poly.ai?utm_source=youtube&utm_medium=podcast&utm_campaign=podcast&utm_content=podcast Follow PolyAI on LinkedIn Watch this and other episodes of the Deep Learning pod on YouTube
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_01You tell okay, build me an agent based on those specs, based on those conversations, and it will configure everything on the platform for you. You don't have to do it manually. You have that option. Everything is readable. You can go and like edit everything you want, but RAN really is the main entry point. And it it handles everything for you. And that everything gets better over time. It watches all the interactions that are happening and makes the system better. Adapt the system, right? It's an adaptive loop where your agent gets like 1% better every day.
SPEAKER_01And that obviously compounds like that's that LinkedIn meme, right? If you get 1% better every day, you end up being quick math.
SPEAKER_00Hi everyone, and welcome to another episode of Deep Learning with Polyai. Today with me, I've got Arkadiusz Kapushewski, who runs our agent OS team, and we're here to talk about Run, our coding agent, and so much more. But Arkadius, I thought we could maybe start with just like a bit of your background and how you ended up doing this stuff.
SPEAKER_01Yeah, so I started Poly four years ago, which feels like a lifetime in this industry. I was originally part of the deployment team, so was working on actually configuring and deploying dozens of agents to production, designing them, integrating them, making sure that everything went smoothly for the users. At peak time, how many reports did you have? I think it was about 40. So it was the whole agent design team, which we still obviously have, like really brilliant people, lots of linguists, lots of uh people with computer, sorry, human-computer interaction backgrounds.
SPEAKER_01So yeah, they're still doing fabulously. But now I moved to the project team to actually work on sort of automating some of those tasks that we were doing, making them work at scale so they're not just confined to human experts, but all that knowledge, all that expertise is now available like at scale to all of our customers at the click of a button.
SPEAKER_00Totally. I mean, I think that like there's a lot of talk of FDE. I think a lot of our competitors who previously didn't believe in forward deployed work are now rushing to talk about it, to hire people. I think we as always sorry, two years ahead. I think like where Palantir got it right with building Foundry after many kind of like forward deployed deployment, is like people forget the second part is for the platform to become something that doesn't require hugely you know motivated, intelligent people, problem solving ad hoc for every client individually.
SPEAKER_00Like it shouldn't be that way forever. If it's like that forever, then you're just a consulting business.
SPEAKER_01Absolutely. And I was always like a believer in this deployment product feedback loop, right? Because all that expertise from the front line, you don't have to bring, you have to bring it back, you have to platformize it so it then is available at scale, right? Like doing like reinventing the wheel, doing things individually on each deployment that just doesn't scale. We've definitely been guilty of that too, right?
SPEAKER_00I think that, well, you and I are both very motivated for less of that to happen. So okay, tell us about Run.
SPEAKER_01So Run is our coding agent, but it's so much more than just a coding agent, right? So it has it has basically two ways of interacting with it. You can think of it as your platform assistant, helper, or really the main interface to the platform, right? You bring come to a platform, Agent Studio, you say what you want, like you know, tell me about the trends for the last week, build me a new flow for making bookings, RAN will do it for you. But Ren isn't just sort of passive waiting for you to ask a question, it also proactively reviews all of your calls, reviews all the interactions, and brings the recommendations so that every morning you open the platform and you see like a list of lists of things that can be better, right?
SPEAKER_01Here's a here's a list of suggestions that that you can accept to make your agent like a couple of percentage points.
SPEAKER_00I mean, one of our customers just posted yesterday saying that booking conversions went up 30%, but not in my wildest dreams. No. Did I think as you said, I thought it was gonna be like a lot of people. I loved watching that.
SPEAKER_01Yeah, and I he just he just went through the whole list, engaged with each recommendation, thought about it, gave a bit of feedback, pushed to production, and and just saw the stats go up, and then was so excited he posted about it on social media.
SPEAKER_00I think like, you know, as I think about like the role of AI and who gets to implement it in an organization. I was with a very large customer yesterday talking about this, and you know, big dog, top guy in the room, started asking questions and then had a few really sharp observations about this like application that was designed by our pre-sales team. And his insight was like viscerally accurate. I was like, well, no offense, but there are five people in this room from your team who did not communicate that to our team, and that's because they don't have your context, right?
SPEAKER_00So I think similarly to how you had all that context and were able to start building this. I feel like making good quality use of time by those people who are empowered to make those decisions and just agree to them is where the organization gets like. I'm not saying it becomes a one-person organization, but the improvements don't go through like five committees and debates and risk and governance, and just someone who's fully equipped to understand what they mean can just say yes.
SPEAKER_01Now it is it is true that sort of like organizing human-to-human collaboration is kind of the bottleneck, right? Because passing all of that context, making sure that the person who's executing understands the vision, all the context, it's it doesn't really scale. Whereas with AI, the this the person who has all the vision who knows what to do, like can just say say that and and watch the results happen.
SPEAKER_00Totally. Well, you you know, you promised some religious references to me in the in the run-up to this one, but I think one of my main corny jokes in the company now is that I refer to my co-founder Sean as the Holy Spirit, and that I think at this point he will only really meaningfully engage with you if your agent can fully outline your worldview around something, and then he will like mix it with his and we'll take the next step in that it's easier than than talking about it because good talkers tend to dominate conversations.
SPEAKER_01No, absolutely, and and to go back, I think we should talk a bit more about like REN and what it is, right? Because it is really sort of the next evolution of what a coding agent can be, especially for conversational AI platform. It really is the first time where we can generally call a platform like fully agentic, which is very exciting. So, you know, until a year ago, the platform allowed you to build agents, and those agents were interacting with your customers, right? So you had agents for different use cases, you had hotel booking agent, a concierge, right?
SPEAKER_01Or maybe an agent, customer service agent that allowed you to check on your order. Lots of different use cases, but but it was like a single agent talking to a single customer at the time, handling millions of conversations, right? Then we have sort of the first iteration of Ren, really, which was a coding agent, right? So again, here are the religious references that I promised. It was like a day God of deism, right? It builds a system for you and backs off, like leaves you leaves it running. So you can come to it, and this is all like testable, like a platform is open, so you can go and play with this.
SPEAKER_01You tell, okay, build me an agent based on those specs, based on those conversations, and it will configure everything on the platform for you. You don't have to do it manually. You have that option, everything is readable. You can go and like edit everything you want, but RAN really is the main entry point, and it it handles everything for you. So it sets up the system, the system then interacts with individual users, but it doesn't it doesn't follow up on those interactions. Whereas now what RAN is doing, it's like an intervention is gut, right?
SPEAKER_01It's it watches, it does it, it still cares about the world it's created, about the system it's created, it watches everything that's happening, and it basically performs miracles in order to make sure that you know history sort of tends towards improvement theologically and that everything gets better over time. It watches all the interactions that are happening and makes the system better. Adapt system, right? It's it's an adaptive loop where your agent gets like 1% better every day, and that obviously compounds like that's there's that LinkedIn meme, right?
SPEAKER_01If you get 1% better every day, you end up being quick math several times better over the year. And that's basically that's basically what's happening, right? Performing these little miracles of like finding finding interactions that can be improved, suggesting better design solutions to certain problems, finding knowledge gaps for you, implementing the changes for you. So all you need to do is click like approve, and then your agent and agent gets better, right? And we will we'll also do auto deployments in the future.
SPEAKER_01So you know you won't you won't even need to be there to reap the the rewards.
SPEAKER_00I mean, really, you are just as a team working with Ren. Your job is to provide the right context and maybe direct the plumbing of that data in a slightly better way. I think one really interesting thing is the project context feature in there where it's kind of like you look at the recommendations one day and you go like we're over obsessing over like, I don't know, handoff routes or something like that. Instead, like I'd really like to see more ideas around how to keep people in cancel subscription flow by whatever, or or you know, recently I've just seen some really cool stuff with like agent memory then being used and targeted by Ren to figure out what we could do and kind of like loading up your information, figuring out how to talk to you and then running tests about it.
SPEAKER_00But it's just all this stuff that was so hard to coordinate, implement, execute, and then evaluate, that just runs in the background and compounds. It's literally miraculous.
SPEAKER_01Yeah. Yeah. I mean, I love I love the fact that there's no latency cost, right? Because the user, whenever you have like your standard coding assistant, you ask it to do something big, you have to wait for like half an hour for it to complete with run. Now everything's kind of ready for you, right? So it really is as simple as kind of swipe left, swipe right.
SPEAKER_00Yeah, I mean, it's like when you have an idea, you should absolutely go for it. But the truth is, and I think this is where like the magic of what we do is maybe hardest to explain, but most important is there are people who live in cloud code, like you do, I do, although more and more we live in Iran as well. But do you know how many messages you send back and forth with Cloud Code in a month?
SPEAKER_01Well, I was on Path Leave last month, so you send more than usual. I send more than you should. No, I think it'll be easily in the thousands, right? Like I I I I think about it a lot, like how how quickly we like adapt to this, because like my job was completely different like a year ago. Whereas now now it's all several clause code terminals running in. How many?
SPEAKER_00How many do you actually use?
SPEAKER_01I think about like I I limited myself. I found myself like, you know, the the human attention now is the bottleneck, right? So I have a role where it's like okay, three or four things at once. So I can actually actually actually follow what's going on. Yeah, I have five.
SPEAKER_00I renamed them every few days just to kind of keep a coherent thread of the things I'm working on. Nice.
SPEAKER_01Now everybody will have these personal strategies, but the reality of work has changed a lot. Uh and I think this is also like an interesting thing, right? Because we're we're trying to kind of take that experience that we're like frankly like addicted to now, right? It it is like I love this new way of working, of of having an idea and being able to like delegate it at speed and and just sort of be you know in charge of a team of agents, and but but it's still not like super accessible, like there's only you know these these most devoted people in tech companies who are really deep into this.
SPEAKER_00Yeah, yeah, I think the world is like a few, maybe 0.1, maybe 1%, probably not even that, or like heavy addicts. And then the rest of the world is just like, oh cool, I can write an email.
SPEAKER_01Yeah, exactly. And and I think what I see as part of our mission with RAN is to make that experience like more accessible, right? To all these people managing custom like call centers. So they also get they also feel the magic, but we make it sort of tailored to them, and we you know the things that that we find, we experience as friction, like the overwhelm, the the slop, right? The making sure that validating all the changes manually, like we want to build it into the harness, into the workflow, so it it it feels even more magical, right?
SPEAKER_01It doesn't require much skill and they can get hooked in the same way on an experience that is like even better and just tailored exactly to their needs and to their workflow.
SPEAKER_00Yeah, I mean one thing I think about a lot is people talk about slop, and I think a lot of people who are not bought into the new world kind of use it as a disparaging term. It's true that it's tiring to kind of like work through these things. It gets like you're holding a lot of, you know, our context windows are limited to it, right? But I think what what is wonderful about the REN and the way that it works in live deployments now is that the customer calls and the data actually ground and work as like a regularization function for the swap. So like it's actually not like the swap is guided very heavily by enough like forces and input that you could never provide on your own, trying to contain it.
SPEAKER_00It's just too hard. You can't see enough calls, you can't think about enough things, you can't remind it about enough things. Whereas there's this flow of things coming in constantly. Yeah. And like it's reflected in the in the tickets it creates.
SPEAKER_01It's that takes us back to basically this whole idea that you know part of our role is to make uh to give RAN all the data that it needs, all the constraints, and then surface some of that to the user, right? So so we can build trust incrementally, right? They see a recommendation, RAN suggests a change, and you know, we link relevant calls so they can see at a glance, they can like just understand intuitively. Okay, I can see the issue, I see it like a one-sentence description, I see how it affects real colours, right? These are these are not made up, these are real calls, you can listen to them.
SPEAKER_01I see that RAN actually already made the change. It used simulation convert uh simulated testing, right, to actually replicate the issue and then and calls it there for posterity, so the same problem doesn't happen again. Exactly, right? So so it shows you like how it's simulated the same conversation after the change and and how it's now working way better, right? So you have the confidence that that the change is real, and then you know, once you once you approve it, it keeps monitoring the calls. So then, you know, shortly after you get like an okay, like I can see that these this type of calls is now improved.
SPEAKER_01You can see the stats going up. So that's exactly the constraints you're talking about. It's it's not slop, it's just intelligence, right? Within within that workflow of of improving your your call center assistant.
SPEAKER_00Totally. I mean, it's superhuman in a way that's just unbelievable. So I was on a bit of a customer tour over the past week and just demoing things around, and you know, I think you and I are deep in this rabbit hole with probably your team and a few other people. And I just like would show people something new, like, hey, what's coming next? And the whole meeting we get hijacked and overrun by an hour. Because people start seeing these like recommendations, and then they're like, wait, wait, what is that? Click on that, and then like these esoteric things that someone might have mentioned to them once or twice, and then they were forgotten because you know they could never really quantify the impact of that change.
SPEAKER_00They're like, wait, wait, wait, hold on. Click on that, click on that. Two of them enabled the post-call kind of like tracking because they're like, I want human data in here as well, because like I just want you to surface these complex, like, phenomena happening in a call when the agent AI agent does this and then a human does this. Why did that happen? Try this instead, tee up this piece of information for the human. People just get so creative, and I think to me, like the Eureka moment, and I mean you remember kind of like how we went from having like a studio assistant somewhere in the corner behind the feature flag for internals to like this is now the homepage, and that's it.
SPEAKER_00Like, I think that was a big moment because it's really a gateway drug to having a lot of fun, right? And that as I look at the data by our customers now, there are about 10 people who have 2,000 messages back and forth with Ren every month. Now that's addiction of this.
SPEAKER_01Yeah, and we have had that feedback, right? They they love it, they trust it, they uh they see it as coworker and they have a lot of fun because it shouldn't feel like work, right? Like with AI at its best, you just like turn ideas into reality. It doesn't feel like work, there is no friction. You're just you know, you you just see your vision happening in real time. Yeah, so Ren, why run? So we went on that name like back and forth for for a while, but Ren definitely had the most internal support.
SPEAKER_01This is a this was a name we could imagine ourselves saying like every day. We could use it as a noun, we could use it as a verb, right? Can you rent that change? Can you ask Ren to do it? It's a proper name. So the original inspiration is Christopher Wren, uh the polymath physicist, astronomer, architect who rebuilt London after after the fire of London. Also has some incredible buildings in Cambridge, so there's that local connection.
SPEAKER_00So, I mean St. Paul's Cathedral where he's buried as like one of his many other kind of like London ones. I I spent at least like three years of like heavy exam terms in the Rand Library in Cambridge, and he wasn't a Cambridge man, he he went to Oxford, but uh he designed the Rand Library A Trinity, and like it just had this like well I guess anglophiliac feel that Polly eyes very much about. And as you said, he was really a polymath.
SPEAKER_01Yeah, polymath. So that's that's the sort of historical connection, and obviously when it comes to branding, we sort of also it's cute, right? I mean it's the name of a bird, like there's no hiding that fact. And and we have we have that bird icon in the platform, which again also helps, I think, to have like a little mascot, right? Uh we we we see that with a lot of coding assistance.
SPEAKER_00I mean, we've had many, we've had many a bird, you know. I think the previous version of our LLM was was Raven, the speech recognizer, Owl, TTS Macau, right? I think like code names for our initial kind of like vertical agents, the kind of like non-interventionist ones were things like Finch, and wow, I'm forgetting now. We might have had a Starling and a few others, but yeah.
SPEAKER_01What I kind of like about the the bird metaphor as well in this case is that you know you basically do get notifications, right? Like you get the notification chime when you come to the platform, it tells you, oh, here are some things you can do. And that also is kind of like being woken up by bird song, right? It's supposed to be positive things, it's not alerts, it's just you know, you get like that positive association of like a nice sound to come to. And that's also like this first experience of coming to the platform, it's also something I wanted to talk about, right?
SPEAKER_01Because in the past, I think most people just looked at the conversations review page, right? So so most of our external users, call center managers, they they look at analytics, they look at the conversation review page, that's sort of the entry point. They review some calls and then sort of but you know, how many calls can you really review?
SPEAKER_00No more than 50 a day without less than a lot of things.
SPEAKER_01These are busy people, right?
SPEAKER_00Like it's not even just that. I think there's just a limit. Anyone who's worked on, I mean you know this, but anyone who's worked on dialogue, like 50 calls, and whoever can do more is like superhuman.
SPEAKER_01Yeah, you have to be it gets boring. It gets boring, and and then I think the problem with that is that then you look at you look at the performance of your agent through this like super narrow lens. Okay, there's a sample of 50 calls out of 3,000 that happened that day. You see one example of friction, and then you sort of like, you know, because humans are fallible like that, right? It's anecdotal evidence, but you're like, well, I want to fix that one example of friction, and then it turns out that it literally happened only once and it will make no difference to your stats, right?
SPEAKER_01Whereas now the the sort of the homepage, as you said, is REN, right? You come to the platform and REN is your gateway into the platform. You see the recommendations, and the recommendations are based on like thousands and thousands of calls. Like AI can see patterns in data that no human ever was, right? Like we had all that data before, we've had it for years, but it's only now that we can make it properly like actionable and we can make it understandable to humans, and I think that's that's the beauty of it, right?
SPEAKER_01Is that AI can sometimes lead to overwhelm, but also it is like the cure to overwhelm. Where suddenly suddenly you can make sense of the.
SPEAKER_01So that I think that to me is what would feel super magical as well.
SPEAKER_00Yeah. I mean, like, I think it's just really interesting, the the whole kind of like I mean, maybe for the general audience, right? People used to kind of like listen to a sample of two percent of the calls. Sometimes they would listen to calls by the best agents and the worst agents, and they would have like these like very brittle algorithms that. Would be like, oh, when people speak Turkish, they say nice words like happy and thank you. And when they speak with Nicola, they're frustrated because they say things like, you know, unbelievable or whatever, right?
SPEAKER_00And that was just like not great. Because the best thing they could do is like performance manage me out because I'm a worse agent than you, if they had enough agents to begin with, right? And now I think like with this, it's just possible to ask anyone of these questions and get an answer. And I also really like the kind of like drift of any individual deployment where through the context, they're able to guide, like, okay, like for the next few weeks, we're gonna work on optimizing this and this and that. So I'd really like you to dig into the potential things that can be done here or there. And then I think, you know, there are people who will just deploy, there are people who will test, and I think that that's how do you see the future of that?
SPEAKER_01Well, so I definitely want to make it more autonomous, right? So there is there are some really, really interesting decisions, design decisions to be made in terms of when to engage a human in this AI loop and and and when not to. When you think about what Ren Ren is, like our customers in the past sometimes they watched, watched, looked at a conversation they they they didn't like anticipate when the agent did something cra clever they they hadn't realized, and they they would ask us like, is the agent like self-learning?
SPEAKER_01Is it learning from the interactions? And and you know, again, until Ren, the answer was was no. And now now it's it's yes. It is learning, it is changing. Every interaction Ren has changes it a little bit and it optimizes Ren for the actual production calls. So we want to make that optimization process better and faster, right? So your agent is really the perfect assistant for your color base. If that color base changes for whatever reason, I mean, like, you know, we have that uh drift, right?
SPEAKER_01The agent will change with it, right? People start asking things about asking about things you hadn't anticipated, the relationship with your brand changes, like we can catch it, we can optimize for that. In terms of like when to engage a human in that process, right? Because it's that loop can be like fully autonomous, right? You you detect friction, you repair friction. That could happen without human supervision. But obviously, there will always be moments where you need to engage a human to talk about like business roles, right?
SPEAKER_01If there is a let's say there's the user is asking a question, we don't have any answer to. Well, RAN might be able to find the answer. It might be able to find the answer on your website, it might be able to find an answer in some of the documents that you uploaded to the platform. If it can't, it absolutely should ask you and will ask you. It won't it won't make it up, right? Sometimes you just we just have to defer to you to help us improve your agent and fill in the content. There is like a really interesting distinction as well between self-healing and self-improvement, which is guiding like a lot of the design decisions that we're making.
SPEAKER_01Self-healing in a way is easier because it's all about fixing friction, and friction is easy to detect, right? There's some frustration on the call, some gap, a bug or something, and and and run goes in and resolves that so everything goes smoothly in the future. Self-improvement is very different and it's it's very much KPI-based. Like you have some kind of North Star metric, so you want to optimize your booking rate, because obviously this is how you get your return on investment by agents making bookings autonomously.
SPEAKER_01You could have an agent that that has no defects, it's performing exactly as designed, but it was just designed not in the most optimal way, right? So suddenly it's not about fixing defects, it's about finding better ways to engage your color base to represent your brand in order to get those metrics to go up. Or maybe finding ways to you know integrate your agent with with your systems better to make it more than you can use.
SPEAKER_00I think that like what I found is often the difference between our deals that are like mid-six figures and deals that get to look mid-seven figures, is the level of sponsorship you have. But really what that means is you move up from just the contact center and into like the wider remit of a chief commercial officer, CMO, someone typically more in charge of revenue as well as just the bottom line. And what's interesting then is like the metrics they look at are different. And then if you can connect your agents, in this case your fully autonomous contact center, to those objectives, then for instance, the value of cross-selling when people can't book a restaurant here.
SPEAKER_00Like, no, I don't have that time. I've got half an hour later, I've got something that's four blocks down, same time. That's like really valuable. And then you experiment with it and you book and then you break through the ceiling where previously, you know, you were trying to you can't really do much. If it's not available, it's just not available, right? And in theory, that some of those things were always possible, but that takes connecting it to the wider org, right? So that's where that human contact center would have been a cross-selling one. Now, again, the superhumanity of our agents is that they've got time, right?
SPEAKER_00So they're not comp on how many calls did you take, why is your average handling time too long? Well, if it's longer because I convinced you to show up at a different location, I'm a hero. But the metric typically would just show that Nicola is low and he should chat. And well, then I'm not really incentivized to do it, so I'm not really gonna do it, right? And then I think when you go up through the org, then like the design of like, should we incentivize people and AI agents to do that or not?
SPEAKER_00And was the propensity of that thing to succeed? Like, it's just impossible to quantify with humans that I think the you know that iterating at like the speed of dialogue where you just kind of have this idea, you don't even have to code it up in the way that you would with Cloud Code or Ren, right? You put it in the context, and Ren comes back to you the next day with like five suggestions. It's gonna have a whole team of analysts that went in, came up with these proposals, then a whole IT team that implemented it, then a whole team of, you know, like continuous optimization people running it and evaluating it.
SPEAKER_00But you just come and be like, oh, conversion's up 0.8% tick, accept, done. And like I can't we're not there just yet, but at the rate that this is all going, I don't think we're more than three months away from someone just going Uber auto mode, like accept it all and and go.
SPEAKER_01Yeah, I think I think all the decisions that can be made autonomously should be made autonomously, and then and then we we need to delegate to clients when when we just need to like integrate with the systems better, or where we need to do something like unusual in terms of like something out of the box in order to really move the metrics in a more dramatic way, like what you said, right? About about across restaurants, right? So this is like you know, you asked like what's the future. I think I think improving the self-improvement part of it, to make sure that we're not just picking the like low-hanging fruit, but we really think strategically, we really think out the box about like how can you really like change the design, the flow, the conversational flow, the setup of the agent to engage, engage people and you know, convert more users and find these solutions.
SPEAKER_01And and we have a system now, if we can, if we like we're getting more and more users to our platform, and we we're getting all those decision makers as well, right? So we now don't have to necessarily go through these loops of human approval. People, people who have the full business context can interact with Ren directly to make those decisions, and and they will know what's best for their business and for their brand, and then they can make it happen and we can be aligned with these top-level metrics, right?
SPEAKER_00Yeah, you know, I think that people always get like completely petrified when the CEO has a pull request, right? And I've done it a few times. It gets people to just re-implement it in the right way because the risk is too high. But where I think it's really interesting.
SPEAKER_01I mean, that's kind of what we want, right? The the CEO has the has the context and and they have the decision power.
SPEAKER_00And yeah, and I've done like that with a number of people. I'm like, okay, you think that's a good idea? Share screen, dad, yeah, cool. My microphone, I just say it, hit, and they're like, oh, so when can we expect to see that thing? I was like, have five minutes, and I think that like at that point, people are just like their minds are blown because it's like, oh wow, this is not like a transformation project where there's an invoice, and then they're like, I think the dopamine hit goes, I did this, right? And then it's like, oh my god, let me do another thing and another thing, and as you said, you know, one percent better every day, and like also like they have the context, right?
SPEAKER_00So I think that what do you think the future is of like all these roles? And like, because you know, we make it seem like there's just one person at the top, like, you know, pulling all the strings, but I don't think that's really true.
SPEAKER_01No, I mean it definitely is easier to be like one person at the top again with what we talked about with with AI like allowing you to actually make sense of all the data, have better overview, right? But I I think you still need people to like go deep and and have all the context of specific areas. Uh I do think that this flattening of roles is very real. I mean, I've experienced it myself where I mean my role is like part engineer, part product manager.
SPEAKER_01And that just means you can carry the context and the expertise you have through through the entire workflow and just make things happen. Again, it's all about yeah, having having that context in your head. I think I think I think this is almost the most valuable thing about workers now is just having all the context and knowing what the direction is, knowing what needs to happen, knowing what needs to be built, and then you have all the tools at your disposal to make that vision into a reality, right?
SPEAKER_00And I think really organizationally, the only thing you have to do is find the people willing to, you know, have their skulls collapse under the pressure of the ever-expanding context window and empower them. Because they'll just if you empower them to make those decisions, like they'll iterate and they'll get there way faster than they can explain. Because I feel like the iteration has become a lot cheaper than the preceding debate. And the debate used to have to happen because it was like, you know, are Mike, I can Peter working on this for a month, or are they working on this?
SPEAKER_00And you know, we've got five sessions each, we're gonna have ten sessions each, and you know, I think and and that's that's that's super true.
SPEAKER_01Where you can just they make things happen, and again with Ren, it's it's so cheap as well. And a big part of it is experiments as well. So, you know, we talked about self-improvement. You don't necessarily always know what will be like the best for your users, right? Like sometimes the way you phrase a question, like how many questions you ask, it it makes an oh like which voice your users will respond to, right? Like you can't know a priori like how that will impact your metrics. So so you basically do have to do experiments and Rand being able to conduct these experiments for you, saying, okay, like I have a hypothesis.
SPEAKER_01I think this user journey can be like cut in half and just you know, just make the questions like less verbose, make it more snappy. I think this will really help users engage. How about we test it out? Like we can launch it for like 1% of calls, 5% of calls, whatever you want, right? We'll watch the metrics, we'll we'll see how we we'll review the calls, see exactly what's happening. It's like so the friction is so low, right? For you, like the cost of that experiment is so low, but then you don't have to debate it anymore, right?
SPEAKER_00You then that's the it cuts through the debate. I remember one of our largest customers, who shall not be named, had a few voice options and a very senior stakeholder one with one of them. I think the other two had like 98% of the votes between our team and theirs. And like for years I had to kind of like prod and like and with like experiments, it's just well, everyone contributes their idea and then you let the data decide. And honestly, by the time they throw in their idea, I feel like a lot of the egos are satisfied, and then like you know, the data will vote and you kind of forget about it, and you know, like you feel good if you if your idea works, but like really if it didn't, you just compete harder to add more ideas to it, and then that's part of that optimization loop, right?
SPEAKER_01Like, I mean, this is this is what we're doing as a company, right? Like, we're iterating on a platform faster, we're pushing to production, and we're letting like we're putting all the new like UI, all the features in front of users to validate them, right? Because we want to we want real feedback, we don't want to like theorize this, and we want our clients to be able to do the same, right? Like make changes, see how real people react, validate that against data, and then make it I mean that's it scientifically, right?
SPEAKER_00Yeah, I mean I think that a lot of people kind of like try to de-risk this for companies going, well, learn from how your humans talk to this platform. And like the only thing I've been able to come up with as an analogy is, you know, we see Waymo's around London now, they've been around the Bay Area for a long time. That took time because people behave differently around Waymo's than they do around other drivers. So Waymo's have to like collect this data and gradually adapt. And humans over time change their behavior around them too, right? I think initially they're a bit aggressive, a bit afraid, and then later it's just kind of comfortable because they know it won't do anything unexpected.
SPEAKER_00In fact, in many ways it's maybe in all ways it's a better driver. So I feel like the levels of autonomy there are just like increasing. And I think the fact that we have so many advanced workflows just allow you to go like, all right, can we now do the highway? Can we maybe try the highway at 120 miles an hour? Because we all know it can be done, right? It's just a matter of like, you know, we need to trust it to be right. And I feel like in a fully agentic world, you can do that a lot more easily. And I think we've crossed a chasm of where we now have enough data and the abilities to put it back in the hands of our customers to do it because they're really always the ones with the context, right?
SPEAKER_01Absolutely. And and again, cut the middleman, like they have the context. I have Ren. They have the context, they have Ren, and they can just make things happen.
SPEAKER_00No, I am super excited about Ren and everything that we're doing with your team. I think that our customers should just expect this thing to change twice a week going forward. And I think that's like one of the greatest permissions that we gave ourselves here. So yeah, I just I'm thankful to our customers for being so excited about it as well.
SPEAKER_01Yeah, I mean it's it's like the privilege of my life really working on this, and it's also just so much fun. And again, as I said before, like my mission is to just share that fun with our clients so they can also like experience, experience this, these feedback loops, experience the growth, experience like immediate results with like zero friction and just like love using the platform. Like we want the platform to be to be part of people's routine, right? They come to it every morning, they click on these recommendations, they watch the stats go up, and they just you know, it's such a good feeling, such a every like hit of dopamine in the morning with your coffee.
SPEAKER_00I look at it every day and like I can't believe how fast it happened, right? I thought it would be just a gradual uptick, and then I'm like, oh wow, like person from a telco in the Middle East, and then someone in a software company in America, and then in a European bank have all done more with REN than I've done with Cloud Code. I'm starting to feel inadequate.
SPEAKER_01I mean, I mean, easily, like I think 95% of pushes to production and I'd done through REN and it will just go up. There'll be more, and and we'll see better and better results for customers and for everybody calling customer service, right? We are we have a mission to fix customer service, and it actually is looking really realistic.
SPEAKER_00I think we finally have lightsabers, right? I think that we we're equipped for the fight and so our customers, so I guess we'll report back in a little bit.
SPEAKER_01And again, tackling this problem in in two ways, right? We have the harness, like we we allow you to improve the harness, and we have the model layer as well with dialogue reason one. So really, I mean those two improvements are happen at the same time, but we we see them both just pushing all the stats up.
SPEAKER_00Yeah, I mean, yeah, dialogue reason one, I think will have a whole different push these weeks. But you know, it's the world's fastest reasoning model. It's the only one that's smart enough to do these tasks well while being, you know, both quick, reliable, not hallucinating. And you know, the more people build complex use cases with run and get them into production, the more data we have to make the algorithm. One, two, three, I don't know what. It's feedback loop. Yeah, feedback loop loop all the way down.
SPEAKER_00Loops loops yet again. Yeah, awesome. Well Khadush, thank you so much for today. And we'll check back in like three months and see how much further we we got in. Thank you. Thank you. It's been a pleasure. Thank you. Thank you all as always. Like, share, subscribe, and we'll see you in the next one.
Three years ago, Simplyhealth's PolyAI deployment was only resolving simple FAQs over the phone. Today, they run voice, in-app, email, chat, and WhatsApp conversations on the platform. Nikola Mrkšić talks with Paul...
Companies are pouring money into AI, but not everyone is getting the returns. Ian Jacobs, VP and Lead Analyst at Opus Research, thinks he knows why. In this episode, your host Nikola Mrkšić sits down with Ian to work...
When PolyAI built its Builder MCP, one client helped shape it: Vixxo Facility Solutions, one of the largest facilities providers in the US. In this episode, guest host Damian Sasso talks with Vixxo's CTO Derek Neighbors...
PolyAI just launched Dialog-RSN-1, its first audio-native model, and your host Nikola Mrkšić sat down with its builder, Matt Henderson, to unpack why it’s a game-changer for building voice agents. Most voice AI either...
Voice AI vendors love to quote word error rate, usually somewhere around 2 to 3%, as proof their system understands customers. Oliver Shoulson, Agent Design & Engineering Lead at PolyAI, thinks it's one of the most...