The story is that AI isn’t paying off, that the ROI just isn’t there, that the org charts and the earnings calls are still waiting for the productivity spike. Harsh Savergaonkar and I are pretty extreme vibe coders. Since January, I personally push at least 1K commits a month… Stop right there, I already know what you’re thinking. I know, I know… lol. Commits do not mean a thousand units of careful thought and productive work. Blah, blah, blah. Yes, I agree. The metric isn't, by itself, a great measure of valuable output. But it is a directional indicator. A good enough proxy. The fact is, though, that whatever you think of commits, companies are creating valuable products and services today with AI that were impossible just 6 months or a year ago. A couple of examples at my company: * Rocketboard: Like every other company, I created a custom project management tool that kicks ass over existing tools, and we used it to replace ClickUp. Saved over $5K a year. * Synthetical Research: We will soon be launching our internal market research product for synthetic consumer research and fake app store testing that we have been using internally. This enables synthetic consumer testing for pennies on the dollar and app store testing at dramatically lower costs than current vendors. And by the way, I’m not the only one pushing lots of commits/PRs; I’m not saying I’m super unique or special; many others have also publicly shown similar output using AI. In fact, plenty of others are dramatically outproducing me… I’m not exceptional at all. But take that as a directional indication that AI-driven output could change how builders abstract development and generate increasingly valuable, well-considered outputs. And output, with the recent shift to orchestration, increasingly more competent models, and the chief of staff operating paradigm, only increases valuable output. Harsh started years before me and got me started back in December. We’re also pretty good at what we do. In fact, we won first place at an AI hackathon sponsored by Peak XV (India’s most prestigious venture firm) and Apps for Bharat (a famous Indian startup). All that to say, we know a thing or two about working with AI… from experience. And from using it better than the average vibe coder. We are operators who have been doing this at a relatively extreme level. And we’re certainly not the same loud mouths who opine on things they have only ever dabbled in, if that. No executive hand-wavey nonsense here. Today, we explain the paradox of AI ROI. Why do we hear about or personally see some examples of extreme productivity by some AI vibe coders, but when we look at teams, companies, and our economy, the story seems to be that AI ROI is just not there? My claim: the productivity and ROI are here. The problem with AI ROI isn’t AI; it’s the people using AI and the nature of our organizations. Check out our full discussion here: Based on our discussion, the answer came in three parts. None of them is the model. It’s not about the ability of AI to drive productivity; instead, all three are organizational and about people. Part 1: It’s a skill issue Start with the uncomfortable one. Most people cannot do this. My rough estimate, and it's mine, not a study: at least 95 of 100 people cannot use AI to build software to production quality. Take the number as directional, not precise. The point is the gap is wide, and most people are on the wrong side of it. There’s a ladder people don’t see. Level one is a prototype. Moving from prototype to something generally usable is at least a 10X jump in complexity, time, and effort. From usable to production is at minimum another 10X, maybe 40X or 100X increase. So the world fills up with prototypes and brittle software that breaks all the time, and everyone concludes AI doesn’t work. Harsh put the skill point more cleanly: “Everybody has access to TikTok and Instagram; everyone has a phone. If they were given the same task, like record a 20-second video, only a few people will come up with something that’s fun to watch.” His conclusion: “Tools don’t matter; it’s that your skills matter.” The skill breaks down into four things. Subtraction. The model over-builds by default, so half the job is telling it what not to do. Ask for a sign-in page, and it will bolt on forgot-password, social login, ten generic things you never asked for, all of them sloppy. Harsh’s rule: “You need to be able to tell it: these are the four things we need, these are the four things we don’t need.” And his frame for finding those four: “you need to act like a QA tester for your own software.” Attack it from every kind of user behavior, and name the cases you’re not designing for. Discernment. Even frontier models make logic mistakes and lack-of-context mistakes all the time. They fill missing context with an assumption and then build on it. From misdiagnosis to relying on training data patterns that aren't appropriate for your situation, you must safeguard your deliverables from the mistakes the model you employ will undoubtedly make at some point. My test: if you’re not catching two or three major problems a day in what the model hands you, your judgment is missing, and you’re the one being driven. Harsh goes further than reviewing output. He reads the model’s reasoning: “I literally expand its thinking, and I read through it.” That’s how he caught it, mid-task, adding IAM security roles to a multiplayer feature with zero users. He stopped it: “Bro, we don’t have any users using multiplayer. Just make it as simple as you can. Let’s get 10 people using it. Then let’s start adding security.” The architecture got much simpler and the whole thing shipped in a day. Best practices. Vibe coding is probabilistic, so you engineer both sides of the probability. On the input side: your context setup, your prompt scaffolding, what files you inject, what’s in your Claude.md or agents file, and most importantly, adversarially reviewing plans or architecture by decorrelated models from fresh context. You need to understand what the LLM has in context, because it will confidently fill any gap based on that context. Harsh runs an adversarial room on the input side: “It becomes like a group project: I have Codex, Claude Code, and three of my own personalities talking and coming to a conclusion. People don’t go to that depth to build something. They just expect that they’ll type something and it’s done.” On the output side, you do the same thing: evals, adversarial reviews by multiple de-correlated LLMs. And you need deterministic gates. Harsh: “If you don’t have those gates for input and output, there’s no way you can say the LLM produced slop.” No test, no verdict. “You didn’t give it parameters, so how the hell is it supposed to know?” The progression I tell people: prompts, then reviews and evals, then loops, then agentic orchestration. Naive usage. Most people never get past step one of that progression. I’ve interviewed a lot of tech leads in gaming who tell me “I’ve been using AI for a year and a half, I’m an expert.” I say, show me your GitHub repo. There’s nothing. I ask basic questions about injecting context or prompt scaffolding, and they just don’t know. They type “make this for me” and expect it to work. That’s not skill, that’s hope. More expert usage would have a deep understanding of the tools you use and a framework for experimentation. Part 2: Defending the status quo In the games industry and at my own studio, I’ve observed that the employees who have the most difficulty leveraging AI are engineers. Why? First, the correction to the popular take. The best vibe coders are not non-engineers. They’re elite engineers. The absolute best are probably the S-tier guys at Anthropic and the other frontier model companies; I’ve met some of them, and their output is ridiculous. Harsh’s version: people from Anthropic and OpenAI will tell you “I haven’t seen a line of code in the last six months, and everything was shipped using AI.” Sk… Get full access to GameMakers at www.gamemakers.com/subscribe
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_00Hello and welcome everybody. I'm here with Harsh. And Harsh, you and I, back in December of last year, you're the one who introduced me to Vibe Coding. And from that moment, you've been Vibe Coding for a while. We worked together. We entered a hackathon sponsored by Hack4Broth and Peak15. We won first place. Since then, it's been six months. We haven't really caught up. A lot's happened since that initial period when I just started learning about Vibe Coding to today.
SPEAKER_00And so I thought we'd talk about a lot of the lessons that we've learned from Vibe Coding and why there seems to be a lot of confusion around the Vibe Coding space. And so the specific point I want to like, one of the main points that people are really Confused about is the paradox of AI, ROI. So anecdotally, we've seen specific people, including ourselves, who are dramatically more productive.
SPEAKER_00We can build things that were impossible before. But then when you look at a team level, when you look at organizations, when you look at the economy, It's like, where is the ROI? And so there's kind of this paradox of, like, we're hearing about people who are dramatically more productive, but we're not seeing the results. And so I wanted to talk to you about that. Like, why do you think there is this paradox? I have my own hypothesis, but I'd like to hear from you. What do you think?
SPEAKER_01Yeah, so I think there's multiple reasons or multiple things that are happening at once. One of the first things that I feel is happening is engineers are still figuring out how to best use AI and they still haven't reached a point Where they trust AI enough. Because what I've observed is when you have like access-free stock models, what engineers do is they hand off a task that they would do instead. And then they hand off the next task, then the next task. So they still are involved.
SPEAKER_01So you're talking about human in the loop. Yeah, human in the loop sort of angle. So let's say the engineer has gone for let's say lunch or they have just stepped out for the day, then the work is stopping. So in a way, even though AI is the one doing it, it's still going at a human speed. And I feel for it to actually get to a speed which is more than human, the human needs to first trust the AI that it will actually do something.
SPEAKER_01Then second, You can set up systems so that it can actually do a lot of things and then the human in the loop becomes only something that you observe and then you basically come back to a bunch of things done, you review them well and then post reviewing you can then ship all of the things that you know are right. So that's like one part of it. What do you think?
SPEAKER_00So I'm going to give you a less charitable interpretation of what's happening and what I think is that if you look at bytecoding at least today and things could change in three months to six months. But I still feel that Vibe Coding is a very high skill endeavor. And this is something you taught me. So back in December, we were competing against so many other teams in this hackathon. But you actually knew how to leverage AI in better ways that you taught me. And not a lot of people knew how to actually leverage AI.
SPEAKER_00And so... I think it is a skill issue. And I think it's also... And so the simple way that I think about it is that if you were to take 100 people, how many of those 100 are actually 10x, 40x, 100x vibe coders? And I think there are very few who can do it well. Because there's also... The nuance here is I believe there's a gap. And that gap is... So level one is like you kind of are able to make a prototype.
SPEAKER_00Then from the prototype, a lot of people think they're done. But then it's like from prototype to something that's usable and useful to you. And so that you can use in your own applications. And I think the step from prototype to having something that's usable is a minimum of a 10x... Jump in terms of complexity, time, effort, all of that stuff is a 10x jump. And then from usability to production is at minimum another 10x jump.
SPEAKER_00Maybe 40x, maybe 100x, but it's significantly more. And so what you're left with is a lot of people that have prototypes or Or kind of usable, but very brittle software that just breaks all the time. And we see that today, like look at OpenClaw, right? With all the contributors that OpenClaw has, with all the time investment and energy they put, that thing breaks all the time.
SPEAKER_01I don't even think it's useful. The app that they launched, it was such a shitty UI. I mean, I was surprised. Like how the hell?
SPEAKER_00You know, and compare OpenClaw to Hermes Agent. Like I use Hermes Agent. Why? Because it's just more stable. And so this gap, and so going back to like out of 100 people, I think at least 95 out of 100 are not able to actually leverage AI in a good way.
SPEAKER_01So this is my theory, right? I agree, I'm really agree. So I think skill issue is a real thing. And I mean, please complete what you're thinking.
SPEAKER_00No, no, no.
SPEAKER_01Go ahead. Yes, I mean, skill issues definitely is a real thing. And I mean, a very simple example that I keep thinking of is everybody has access to TikTok and Instagram, everyone. And if they were given the same task, like record a 20 second video about something, then only a few people will come up with something that's fun to watch. Everybody else will just miss the mark or they just make something really boring. So I think you can see that tools don't matter. It's that your skills matter. And I feel that for leveraging an AI, I imagine it like harnessing a Pokemon.
SPEAKER_01Like, do you know, when you are a low level, you're a Like you basically as a trainer are low level in a game like Emerald or any of those original Pokemon games. But you have like a 70 level plus Groudon or like one of these Pokemon. They stop listening to you. And then you have to figure out a way to harness them. And I think it's a Similar thing with LLMs because people just look at them as chatbots, but if you're able to harness them well enough and play that mind game with them, then you're able to get insane amount of outputs.
SPEAKER_01So an example that I can give of this mind game theory. Is whenever I use an LLM to wipe code, I act like I am four different personalities with the LLM. So in a single chat, I'll talk to it about how what we are doing is amazing. In the same chat, I'll talk a little bit about how what we are doing. I try to simulate that but with an LLM.
SPEAKER_01So it basically becomes like a group project. I have Codex, Cloud Code, and three of my own personalities talking and coming to a conclusion. So that's just one example. Like people don't go to that depth to build something. They just expect that they'll type something. Right.
SPEAKER_00And that's one of the concepts that I think gets a lot of people stuck, which is essentially, and this is something I've talked about quite often over the last year, but like the fact that when you're vibe coding, It's a probabilistic output. And so all the things that exist in terms of from the harness to all the processes that you put around this. So you give a prompt to an LLM, some kind of output comes out.
SPEAKER_00But all the things in terms of the inputs as far as how you set up your context, what you tell the LLM, your prompt scaffolding, All the things on the input side, how you structure your prompts, things like that, helps optimize what comes out. But then on the output side, then you have to do a bunch of evals, reviews, and things of that nature. And the thing that you're talking about is on the input side, you're having multiple LLMs discuss how you plan stuff. But you can do the same thing on the output side where you have multiple LLMs review
SPEAKER_00the code and things like that. Agreed. And I think this process and how you optimize the inputs and the outputs, all of this is this issue around skill, right?
SPEAKER_01Yeah. And you also, like, what you sent over in that, there was this one sentence that if there is no test or there's no deterministic test, then how do we say that the LLM failed or, like, Right, right. So I think this is what it basically is. If you don't have those gates for input-output, then there's no way you can say that LLM produced slop. It's open-ended right now. You didn't give it parameters. So how the hell is it supposed to...
SPEAKER_00And so there are a lot of these skill-based things. What Harsh is talking about is I had put together a presentation that talks about all the things that you need, like a good vibe coder to get from prototype to something usable to production, like all the kind of skills and things you need to do, as well as how you increase your outputs in terms of not just like, because so many people I talk to talk about prompting.
SPEAKER_01Yeah.
SPEAKER_00Vibe coding is so much more than prompting, right? It's then you go from prompts, then you go to reviews and evals, and you go to loops, and then you go to, like, agentic, right? So, okay, so I thought the next topic we could talk about is, like, one of the other things that I found to be strange is how back in March during GDC, I talked to a number of game studios who were game studios who were Seeing
SPEAKER_00that they were AI Native. And it was easier for them because a lot of them were working on web-based games or working in Godot, for example. But one of the things that they had told me is that their highest output employee from a Vibe Coding perspective was never their engineers. It was always a product manager. And even at my company, I would say the kind of employee from a discipline perspective that has had the most difficulty Learning how to leverage AI is not the product manager.
SPEAKER_00It's not even the artist. We even have some artists who are out producing engineers. It's been the engineer. And so I wanted to ask you, what have you seen? Like what are the kinds of disciplines or what are the kinds of people who have been more successful at leveraging AI versus those who have not been able to? And would you agree with me that in the general case, engineers for some reason are having a lot of problems?
SPEAKER_01Oh, I completely agree. I think one of the problems is that engineers have been for so long, been into like the depths of the code that they're writing. So I think even now they are trying to, uh, Still manually do a lot of things in between or they're trying to check between every step or they're trying to, you know, go across every small edit or every small diff. And I think it for a lot of cases is important to obviously check what's happening. But in a way that's very different.
SPEAKER_01So it's no longer about, you know, checking each line after every edit. It's more about running tests. It's more about like having the right plan in place and so on.
SPEAKER_00Yeah, but to some degree what you're saying, kind of how I'm interpreting what you're saying and kind of what I'm also seeing is that it seems like engineers are used to a certain way of learning. And I think a general human characteristic is like when you're used to something, you're like, oh, this is how I code. And then you're like, okay, here's a new way of coding. And then there's this defensiveness. And then there's also this laziness of learning a new way.
SPEAKER_01Yeah.
SPEAKER_00But if you're a PM or an artist and you don't have that legacy sort of workflow or ways of doing things, then everything's new for you. So you're just like, well, I'm going to just try to do it the better way.
SPEAKER_01No, I agree. It's one of those principles where if you are never told no, then you never think that it's not possible. So for a lot of developers, a lot of things... They feel are not possible or they have been sort of reading about it on Stack Overflow or getting bashed by their seniors that hey this is just not possible and now it suddenly is but they have heard the no so many times that they kind of are still skeptical. But for non-developers they have never heard that this is not possible or they have never heard a no. So they from the get-go they are happy to try it out and then it just works for them.
SPEAKER_00I think that's one of the reasons why But I would also say there's a little bit of a nuance here too, right? Because the guys who are the absolute best at Vibe Coding are probably like the super engineers, right? Like the guys at Anthropic, and I've met some of them, and their output is like ridiculous.
SPEAKER_01Exactly. I mean, one of the things that people say that it's not possible to code something amazing using AI, but then all of these people from Anthropic and OpenAI tell you that I haven't seen a line of code in the last six months and everything was shipped using AI. That's when you know that if they can ship such good software without looking at the lines of code, then it has to be a skill issue if you can't.
SPEAKER_00Right. And just for our audience too, because I think that people still don't understand the magnitude of the impact.
SPEAKER_01Yeah.
SPEAKER_00Is that, so for example, At my game studio, before we restructured, because we, like, once I saw this, it definitely became an imperative to restructure our... Oh,
SPEAKER_01I remember, you started wipe coding so much, like in December you weren't, and then after that, anytime I meet you or see you, you're doing something like...
SPEAKER_00Yeah, yeah, like January through March of this year, I was barely sleeping.
SPEAKER_01How many comments did you say you were doing every month?
SPEAKER_00So, since January, I've done over, from January through June... Yeah. I've done over 6,300 commits. So over 1,000 a month. Now here's the crazy thing. Our engineers average 20 a month.
SPEAKER_01That is crazy.
SPEAKER_00So we're talking about the magnitude difference is 20 versus 1,000. And it's not just me at our company. We have a product manager who's going crazy.
SPEAKER_01And literally one of the sort of...
SPEAKER_00Philosophies or mantras that we have at our company now is that at this point right now, and this is just becoming more possible with things like goal and loop and things like that, is that the output to the point that you're making about people, about human in the loop, is that the output should actually not be constrained by the person, but by your token budget. Yeah, that makes sense. So how many math accounts do you have?
SPEAKER_01Oh, I mean, I actually only have two right now. I actually had two Cloud Max accounts. I now switched one of them to Codex Max. I just wanted to try it out for a bit. So I've been trying it out
SPEAKER_00for the last few
SPEAKER_01days. And I also have Grok Max, the $200 plan, but that's just because I have the X Premium.
SPEAKER_00I definitely have to try Grok. I've been hearing a lot of good things about it.
SPEAKER_01The latest model is decently good. You can give it good tasks and it can actually do it.
SPEAKER_00And this kind of would lead to maybe one of the other topics that we can talk about, which is like for some of the people in the audience who are trying to understand how do you get more productive? How do you actually get to production level code? Like what have you seen in terms of some of the main problems or issues? What are some of the The complexities or the problems that prevent people from getting from prototype to usability to production. What have you seen?
SPEAKER_01So I think one of the main things that I've seen, like be it friends, colleagues, is that they approach it very much as a what What needs to be done problem statement. So basically, if you want a website to do X, they will tell AI that, okay, do X, or they will set up loops or whatever, evals, etc. It does X. But the point is that when you're building something with AI, you also have to tell that don't do Y and don't do Z. So I think that's a very big thing that people don't really... Understand or it doesn't feel that important, but it's very important.
SPEAKER_01So I'll give an example. So let's say you are building a sign in page and you tell any LLM to build a sign in page for you and you have evals and everything set up. But at the same time, you know for sure you don't want sign in with Facebook on your sign in flow. Or when you sign in, you want to ensure that a user's user ID is set at two different places on your database and one specific like cron job needs to be kicked off, etc. So you will tell it to do three, four of these things. But then it will try to also do more things which are generically done by people in
SPEAKER_01a sign-in flow. And then you will have a website that does 10 things and all of them in a sloppy way. But you need to be able to tell it that, okay, these are the four things we need. These are the four things we don't need. Or maybe login was not the best example. But like imagine there's a screen where you want to make sure that whenever you hover on a button, it should pulsate a little bit.
SPEAKER_00Yeah.
SPEAKER_01But whenever you hover on the second and third button, it shouldn't pulse it. So that's a detail you need to tell because if you tell it that okay, make the first button XYZ, it will probably then consider it as a memory and then it will use that for the next buttons you create as well. So I'm actually going to think a little bit more about a better example for this. But the whole point is that you need to act like a QA tester for your own Software that you're building and you need to attack it from all types of user behavior. Then you need to think that a user might do XYZ on this page,
SPEAKER_01another user might do ABC and you need to consider all of that and tell the LLM these are the things not to be done or these are the cases for which you're not designing.
SPEAKER_00Yeah, so some of the problems that I've seen There are a few problems, but I would say the very first problem, and this is especially the case, again, not for the S-tier engineers, but for the general engineer, is just the problem that we talked about before, which is they're used to a certain way of working, so there's almost this laziness. I've interviewed a lot of people in India, a lot of tech leads in gaming, a lot of other people who are like, oh yeah, I've been using AI for a year and a half.
SPEAKER_00I'm an expert. And I'm like, really? And I would ask them, okay, show me your GitHub repo.
SPEAKER_01And there's like nothing.
SPEAKER_00And I'm like, okay. So I'm like, okay, so what's the main harness that you use? You know, what's the main tool? And they'll say like Cloud Code. Then I would ask them some basic questions. Things like, what are the various ways of injecting context? How do you modify your status line? Like a bunch of like basic questions in terms of the usability of the tool. And they didn't know anything. Like how do you set up your prompt scaffolding? All these things, they just didn't know. And so like the first problem that I see is that The way that a lot
SPEAKER_00of people approach Vibe Coding is completely naively. Oh, I just say, make this for me.
SPEAKER_01Yeah. And expect it to work.
SPEAKER_00Yeah. And they don't understand, okay, for the LLM to work properly, you have to have a view in terms of what the LLM knows.
SPEAKER_01Yeah.
SPEAKER_00What context does it have? What's in your cloud.md or agents file? How do you structure your prompt appropriately? What files do you inject? All those kind of things like they don't even know or think about and they don't know how to use the tool. So that's one major problem that I see. And then I think the other major problem I see is something that you were alluding to. Now you were talking more to the point on taste, right? How do you make the user experience in a way that would be appealing to users?
SPEAKER_00I think that's one issue. But I think the other issue is something, I call it a little bit more around discernment. Okay. But for example, I believe that even the most current frontier models make a lot of logic mistakes.
SPEAKER_01Yeah.
SPEAKER_00Or mistakes based upon the lack of context.
SPEAKER_01Yeah.
SPEAKER_00And so let's say you're trying to figure out what kind of backend to use and You're asking for a recommendation from an LM and it keeps like, and this is something that just happened last week. It kept saying, okay, railway would be the best sort of backend to use for this, for these reasons. And then I'm like, well, what about this? What about this? And then it would be like, oh yeah, I didn't know about this. Oh yeah, I made a mistake. You're right. And so like, in my experience, working in AI live coding,
SPEAKER_00if you're not catching like two or three major problems every day, There is a level of discernment that's missing. And so I think if you're going to do vibe coding well, you have to have two things. One is I think you have to have kind of like high IQ in the sense that there's always some level of logic jumps. There's always some level of things where the LLM just gets it wrong. Even Fable today.
SPEAKER_00And this problem might get solved later. But if you don't have that attention to detail in that discernment to try to find where those logic jumps or problems are, you're not going to wind up with the best output.
SPEAKER_01I think it's not even about how smart the model is. Imagine the smartest engineer out there. If you go and ask that engineer that, hey, I want to do X, Y, Z, and that engineer will probably say, okay, use Python, for example. That's the best language for you to use. There's no reason for you to use PHP or like C++. But then you give them additional context that my software needs to run on a very specific kind of a hardware. And I need to ensure that my compile time needs to be insanely low. XYZ and then that engineer will be like, oh, that makes sense. I think that's why you should use C++.
SPEAKER_01I was wrong. You should not use Python. So that tells you that if you give the entire context, then the LLM will, you know, work for you. And that's how you catch mistakes.
SPEAKER_00Right, because the LLM is essentially, it's looking at patterns of things that it's learned from its training.
SPEAKER_01Yeah, exactly.
SPEAKER_00And similarly, like a couple months ago, I was working on a Godot project. Okay. And I noticed that it was telling me to use GDScript and C Sharp, but then I'm like, Why? Why use C Sharp? So if you have a performance hot path, why wouldn't you use C++? And it's like, oh, oops, yeah, actually, you should use C++. So there is this thing about how even today to the point that you're saying the LLMs may not have that full context. And the LLMs do tend to bias towards certain things, right?
SPEAKER_00Another thing that a lot of the models tend to bias towards is understanding that, for example, if you're just building... A new application, new software, Greenfield. But all of the LLMs will always bias towards legacy infrastructure. Okay, well, now we need to build all this stuff for legacy. And I'm like, no, we don't have any users. Stop putting all this legacy bullshit in. And even you put that in your cloud.md file. But even then...
SPEAKER_01It will ignore it.
SPEAKER_00You don't ignore it. And then you're like, no, stop. And so this level of discernment where you have that attention to detail, you're not just fully
SPEAKER_01like, you know. Yeah, exactly. I mean, I like to say that, like, you need to see what it's thinking. I literally expand its thinking and I read through it. Yes. So the architecture became so simpler and I was able to get it done in like one day.
SPEAKER_00Yes, so these are like, and for the people in the audience who start vibe coding, is you start vibe coding, and this is why it's so important to have the experience, because these are the things that Harsh and I can think about right now off the top of our heads, right? Yeah. But I would say that there's so many other small things that if you don't know, Just based upon experience, what to do, you're not going to be able to make that jump from prototype to usability
SPEAKER_01to production.
SPEAKER_00Okay, so then maybe moving on to the next topic. So what in your mind are the kinds of people, whether it's disciplines or the kinds of skills that you need in order to be successful in VibeCoding? What do you think?
SPEAKER_01So I feel at least as the way things stand today, models keep changing a lot. I think one thing is important for you as a human is to have a large context window yourself. Because the thing is that when you are wipe coding, even a small feature, it might touch multiple parts of your project. Previously, you had developers touching small parts of projects and then figuring out merging them and all of those things. But now it's not required. So you end up telling a whole feature and it Touches multiple parts of your product. So you need to understand that what all it might have touched and you then need
SPEAKER_01to test all of those other things as well. For example, if you're like changing something in your auth and your product is already out there, but you're not testing your login flow after you made a change in your auth somewhere else, then it might end up breaking because the LLM is over correcting or optimizing for the current task you give it. And it sometimes doesn't care if something else is breaking. So, I think that's one of the core skills.
SPEAKER_01Because now that building is very easy, you need to build something that actually matters and that actually has a reason to exist. Like I have gone through this problem for a while, so I can talk a lot about it maybe after I hear your thoughts about this.
SPEAKER_00And maybe we can separate kind of two things that are two kind of related topics here. But I kind of think that there's one aspect of it, which is, When we think about AI vibe coding and how that impacts roles and the kinds of characteristics or things that allow you to be successful. And then the second is this notion about, well, then how do you actually become successful using vibe coding? And they're kind of two separate things. But speaking on the first point,
SPEAKER_00the way that I've kind of thought about it and what I tell people at our company today is that I look for these four things. So Arnie talked about High IQ, high agency. The third is attention to detail because to the point that we had already talked about, you have to find all these small little problems. You can't just, I mean, there's kind of like this conflict, right? How do you take yourself out of the loop? But then for some very important things, you have to make the right decisions. You have to watch what it's doing. And then fourth is having a technical foundation.
SPEAKER_00And so I do think that At least today where AI, LLM frontier models are today, you still have to understand technology. You have to understand whether it's like infrastructure, database. You have to understand the various kinds of technologies because you have to help and work with the LLM to make the right technology decisions. Right. And so those are the four things that I generally look for. And I feel like if you have those four things,
SPEAKER_00you can be very successful when it comes to ad coding. Would you agree?
SPEAKER_01Yeah, I completely agree. I mean, like we were discussing point two and three is something that I have a lot of opinions on as well. yeah, please. Yeah, so point two, like the agency part, like we discussed, like one of the biggest things is that most people aren't incentivized to get more output.
SPEAKER_00Right.
SPEAKER_01And that is why they are not getting that output. That is why overall it looks as if the output hasn't supercharged because of AI. Right. But if you look at incentivized teams, small teams, solo founders, so all of these people who are super incentivized to reach the end of the line, they are actually showing that how Crazy good level of output you can get using AI. And the last point that you said, the attention to detail. I think that makes so much of a difference. Like the smallest of the smallest things. If you watch the movie, The Intern, in which Anne Hathaway visits one of her warehouses where she packs one of
SPEAKER_01the boxes herself. And they have that moment where as the CEO founder, she makes a very small change. It's not that big of a deal, but it just shows how much tiny attention to detail People who care about it have. And I think that makes a lot of difference. So when you get a parcel and if it's all messed up inside, it won't feel as good if it's nicely laid out. It's the same stuff.
SPEAKER_00And one of the other ways that you can think about high agency is like how much you care. So like that you give a shit. You really care that you're actually going to be doing things well versus, and I know a lot of people to the point that you're making, if you're at a company, you're an engineer, you have a certain amount of output. Your incentive is just to show that output or a little bit more, and then you're just going to check out. So if you don't have that incentive or if you don't have that agency, if you don't care enough, your output's fundamentally going to be limited.
SPEAKER_01You are limiting it yourself. That's the whole point here. And I think a lot of people get it wrong that give me more incentive, I'll give more output. I think it's I'll give more output that will make you give me more incentive. I think most people don't understand that. And that is why I think they never push AI to the level that it can actually go.
SPEAKER_00Right. And then... Right. And so another topic I wanted to talk about is because the other thing that also kind of gets conflated is going all the way back to the original question, where's the ROI?
SPEAKER_01Yeah.
SPEAKER_00Is kind of like this question around, so if you do the things that we talked about, it answers the question of can build.
SPEAKER_01Yeah.
SPEAKER_00So you now can build whatever you want. Yeah. And for me, I feel like if it's web, There is nothing that I can build. I can build anything.
SPEAKER_01Exactly, yeah. Right?
SPEAKER_00But then that doesn't answer the question to the point that you were kind of alluding to before, which is what to build.
SPEAKER_01Yeah.
SPEAKER_00And that's where things like taste and discernment and things like that come into play.
SPEAKER_01Yeah.
SPEAKER_00And so maybe you could talk a little bit about that as well, because I do think that that's the other thing, is that even at my company, when I see other people building stuff, it's like, okay, they're building a bunch of crap. Does this thing that you're working on, does it help the company? Does it help you get to profitability? Does it help you really help achieve your goals or lead to something that's going to make a difference? So what do you think?
SPEAKER_01I think that there's types of things that you build. One is crap, just don't build that. But the second type is something that you're building So very early on in your building journey, you need to figure out what Are you trying to build? And if you don't, then you will end up wasting a lot of time. So I have built a lot of things in the last two years. Like I've built probably over 50 projects and all of them in a very good quality
SPEAKER_01because AI can just let you do things so fast now. But a lot of times I approached it in a very... Pre-AI way where the idea was that okay, this is hard to build, it will be a meaningful project. But now that it's not hard to build, but the dopamine still is the same because our brains probably haven't understood it yet that how easy things are completely. So you still get the dopamine and then you end up shipping things that were hard.
SPEAKER_01Right. Look amazing. And then it's just something that you will never launch. Or even if you do launch, nobody will care about it. Right. So I think if you're able to answer that question very quickly, that is this something I'm building for myself? Is it just a project? Then it will actually get to a point where you can define ROI. Right. And obviously, there's a lot of things that people can just build for the sake of it, which doesn't really make a lot of sense. So, yeah.
SPEAKER_00And I think this is the issue that We're the human part, right? The AI, again, at least today, cannot answer this question, which is, in a world where you can build anything, what should you build? And then the other nuance there is, even if you're building the right thing, are you building it in a good way? Because you could have, and the two apps that I see everybody building, like every company It's building their own HR recruitment tool,
SPEAKER_00and they're all building their own project management
SPEAKER_01tool.
SPEAKER_00But they're not all the same. So to the point that you made about taste, it's like you have to now figure out not only what you should build, but in terms of the details of it, how it's built, the what and the how.
SPEAKER_01Yeah, and for what, like what probably we're trying to solve with it.
SPEAKER_00Those become the more fundamental and important questions in the role that humans play versus can I build it, which is increasingly more and more the models are just helping you get there.
SPEAKER_01Exactly. That makes sense. Okay. And the taste part, also important. And the example I always give is YouTube exists for so long now, but not everybody's creating content. And a lot of people are creating content, but it's still not good. So only a few people have mastered how to create content that keeps viewers hooked. Similarly, software has become like content. Everyone can make a video, but if the video is good or bad, it completely depends on how tasteful you are, how creative you are.
SPEAKER_01And I think that's something that you also need to understand very quickly about yourself if you have that person.
SPEAKER_00So there's two other topics I thought we could talk about before we wrap up this conversation. And the first of which is something that I think is a little bit of a sensitive topic. So we're here in Silicon Valley, right? San Francisco, which is very close and you're seeing San Francisco, but like one of the main things in San Francisco, which is like the center of the universe of AI, is that it's kind of become very sensitive and almost politicized when you talk about this topic around AI's impact on the economy and to job loss.
SPEAKER_01Yeah.
SPEAKER_00If you now say something along the lines of you believe that there's going to be job loss, people get really sensitive about that. And they're like, no, no, Jevons paradox, Jevons paradox, demand increases, blah, blah, blah, blah. And so I wanted to get your take because I actually don't think that just adopting the kind of Silicon Valley narrative on this and just saying, oh, no, it's going to expand jobs. Yeah. I don't think that's the right mindset or even mentality to have.
SPEAKER_00But I don't know if you disagree with me or not.
SPEAKER_01I agree with what you said. So, I mean, again, I'll go back to the YouTube example. So before YouTube or Twitch existed, there were only these large production houses which were able to create movies or content. But now every kid in their bedroom with their friends can just create content and start posting on YouTube. So you suddenly have these pockets or like cells of creative groups that can now create and post. So similarly, because of this, AI has democratized that piece in humanity, where it's more and more possible for you and your friends to just pick something up
SPEAKER_01and build something. And I think the type of work is changing a lot, like rather than 20 organizations employing 10,000 people, it will probably probably be like 1000 organizations employing that many people, and each of them trying to solve different things in the world or trying to build different things in the world, just like how content has become Very broad in terms of what all kinds of content can exist. So I think that's how jobs will change. Like one company won't have a density of people. It will be multiple companies having a smaller density of people.
SPEAKER_00I think YouTube is an interesting analog because I think what we've seen with YouTube is that YouTube has put the power of the creation of content Into the hands of many creators throughout
SPEAKER_01the world.
SPEAKER_00But still, there's only one Mr. Beast. That is true. And even Mr. Beast would say that building a viral video is a skill issue. He's famous for saying he could do it again anytime. And he continuously gets hit after hit after hit. But also what we've seen is that if you look at the... Success of people on YouTube, there is a concentration amongst a small group of creators. Not everyone gets the same reach. And I think that's also what's going to happen with AI, right?
SPEAKER_00So what YouTube has done is they've been able to scale reach and kind of like that distribution aspect of it. And what AI is doing is it's scaling the production part of it, the building part of it. But still, I think that in a world where one person can now make 100x of applications, of services, of things like that, the Silicon Valley religion now is that, oh, no, no, no, Jevons paradox, building becomes easy, so that's going to expand demand.
SPEAKER_00But where I push back is that that may be true in certain industries, but there are certain industries where that's not true. And the reason it's not true is that, for example, I come from the games industry. In gaming, there's a finite set of attention, right? And so whether you're spending time playing games or watching YouTube videos or movies or how you spend your time, Just because the output increases doesn't mean you have an infinite amount of demand.
SPEAKER_00Demand is not going to be elastic. You're not going to be like, oh, there's more content. I'm just going to not sleep and just spend more time on that content, right? And so that's where I think that there's this danger with this Silicon Valley religion around you cannot talk about job loss. You cannot talk about the negative impacts of AI because I do think there are going to be Negative impacts that come from AI for some people. But I also think that the thing that too many people are anchoring on is the downside.
SPEAKER_00Because I would say that if you look at every technology transition in history, what has always happened is that there are a set of winners and there's a set of losers. And too many people are focusing on The bad part of it, the losers. But there are gonna be winners. Now that you can build anything, new opportunities do arise. And so focus on that and try to like, it's just the nature of what's happening. It's the nature of the world where there are gonna be winners,
SPEAKER_00but there's gonna be a smaller number of winners in certain industries. So you want to try to be in that group of winners rather than in what I believe is gonna be a larger group of losers.
SPEAKER_01I think that there's two nuances to what you said. So the first part is, I'll again go back to the YouTube example for a second. So even though Mr. Beast or like only a few creators hold the power, it's not always about being the creator. But instead, each of these creators have their own media houses now. And they employ 20-30 people each. So it's not always about being yourself as a creator, but working at one of these creative agencies. So after YouTube democratized content, there suddenly existed 1000 more agencies,
SPEAKER_01and each agency employed 3040 more people. So even though people felt that larger production houses will now cut off employees, they were automatically reaccommodated by these smaller houses. So similarly, in AI, even though there will be a few winners, they will end up re employing or like Bringing more of these people to work with them who might not be able to win in a solo way, I guess.
SPEAKER_00But I would agree with you. But then I guess where I would push back is still, okay, so let's, you know, we talked about project management. I think that these companies, you take a company like Jira, like Atlassian with Jira or Monday or ClickUp, like Monday employs over a thousand people. Yeah. I vibe-coded a project management tool, which I think is dramatically better, and at least for us, is so much more powerful, capable, and does everything that we need. And that's replaced a product that has a company with over a thousand people.
SPEAKER_00So I do think that there will be some compression. But there will also be new job categories that open and things like that. But I don't think what you're saying will necessarily cover all the jobs. Yeah, maybe not.
SPEAKER_01Because I imagine that, let's say, for example, you end up making this tool that you built for project management into a company. Then you'll end up hiring 5, 10 of these people. Oh, will I? But if you don't even make a company out of this.
SPEAKER_00But let's say, and by the way, I am open sourcing my tool. But I think what that's true. That is true. That's true.
SPEAKER_01Yeah, that makes sense. Obviously, if you make it into a company, you want to hire salespeople, etc. But then that's a different game entirely. Right. I agree. I mean, definitely, it's going to be very interesting to see how people are reaccommodated throughout this wave. But I believe that it's not completely negative, like people keep shouting. And I feel that it's going to be a very different kind of a world where agency and all these things will be rewarded much, much more. And I feel that we are very early in AI right now. Like the only thing that's changed is software.
SPEAKER_01But soon, entire food industry, agriculture industry, things will change so much that the kind of opportunities that open up over time, I feel would still be very different but still exist. Because there's just so much that we've not even imagined. Because imagine tomorrow if Fable 5 is so powerful and Fable 9 comes out. And Fable 9 is probably going to be so good that it can literally tell us how to create a new source of energy, for example. And if that happens, then suddenly you're going to need a thousand new people to work on extracting
SPEAKER_01that source of energy. So I feel that if AI is able to keep going in the hockey stick growth, then so many new things that humans need to do will pop up.
SPEAKER_00But I guess like to the point that you're making right now, one of the key open questions, I haven't been able to figure this out, but a key open question for me is like, How long is the opportunity window open? And so another thing that I tell a lot of folks at my company is like, I tell them, you have to push hard right now. Because there was a blog post by this venture capital firm called NFX. And I was reading it and they talked about looking back
SPEAKER_00at various technology revolutions in the window of opportunity for certain things like social networking. And they said that window was 11 months. And so the open question I have is like, how much more opportunity are we going to have where Is it 11 months to learn AI and then the people who learn how to use it are producing so much that they're going to crowd out everybody else? Or are we going to have a sustained period of like a golden age of opportunity where
SPEAKER_00for the next 10 years you can still continue to like use AI and come up with a bunch of new ideas and the economic advantages or benefits accrue In a way that's more spread out versus like a power law distribution where only a few people capture all the winnings and it's the early people who win. That's the open question. No, it makes sense.
SPEAKER_01I mean, the only thing that I can imagine is that maybe the types of industries will keep changing rapidly. But again, I get it. It's not something we can predict right now because it's going to be
SPEAKER_00very confusing.
SPEAKER_01Yeah. So like for me, and again, this is just my current perspective, but the thing that I tell a lot of employees in my company is like, I think this is the last opportunity for people to be able to work hard.
SPEAKER_00So just push hard now, because it may be that if you learn all the things you need to learn now, and you're able to get to a point where you're at a point of leverage, right? Like Google search. There's a certain point where you just have a dominating position in the market, and so you kind of control the opportunity. So then are we at a point in time where now is the time to work? There might be an 11-month window where And then after that, in two years from now, you're either working three or four days a week or zero.
SPEAKER_00That's the thing. Again, I don't know the answer.
SPEAKER_01I feel there's enough people in this world who can't sit idle. So no matter how things get easy, we'll find a way to keep working. I feel that.
SPEAKER_00But if you look back through time, through history, there have always been times of massive... Economic disruption, civil unrest, revolutions. And so, I don't know.
SPEAKER_01Can humans sit?
SPEAKER_00Maybe that's coming. Maybe that's coming. You know what I mean? Because I will say this. I do think, and now we're kind of going a little bit far afield of just AFI coding, but I will say that I believe that in society today, there are too many people who own capital, who are wealthy.
SPEAKER_01Yeah.
SPEAKER_00And they take it for granted and they don't understand that if you are like somebody who lives in the Midwest, you've got your job, you're providing for your family by being a truck driver or you've got some kind of job, but then you get displaced. If you can't feed your family, I think it's extremely serious. I think you will become violent.
SPEAKER_00I think there is definitely a potential for AI to... There's a path. And again, I don't believe in being able to predict various scenarios. I generally try to assign a probability. I believe that outcomes are more of a probability distribution. Okay, so I know we've been talking for a long while. So one last question, which is like, okay, so now we're in a world where you can... Almost build anything. And so in that kind of world, then how does that impact the world, business, economics?
SPEAKER_00One simple way of thinking about it is if you can build anything, then production and development, as those costs trend to zero, Then distribution, those costs will increase. But how do you view how the world will change given AI? What are the things that people should be thinking about as we move into this world where, again, AI is increasingly just enabling people to build anything they want.
SPEAKER_01So, I mean, to begin with, I still am of the camp that product is more powerful than distribution. And I strongly believe that if you have a great product, distribution will happen. And right now, at least, AI can't understand what humans want. It doesn't understand Scalability of a product, it can't understand marrying all of these concepts together. So you still need to be the one to figure out what to build and you need to be the one figuring out what needs to be built. So for example, like Steve Jobs made the iPhone.
SPEAKER_01But back then, if somebody was using AI to make the Blackberry better, they would have kept going in the same direction by cutting a few things, adding a few things. They wouldn't have imagined a completely new parallel route or be it like the shift from a carriage to a car. It's such a parallel shift that the creativity and taste of a human needs to come into picture where you need to figure out A, what does a person want? B, can they pay for it? And C, can this actually work for a group of people, not just like one or two people?
SPEAKER_01And can that actually turn into an income or company and so on? So I feel that at least for this window of time until AI can't do that part, this remains the most important thing. And we cover all of it and name it taste like as one thing over time. So you need to figure out what People want, because people can't tell you what they want. They think they know what they want. But most of the times, consumers don't know what they want. And how to build it really well, that people really understand how to use it,
SPEAKER_01you have amazing user experience, it's simple to use, everybody can use it, and so on. So I think that's where your focus goes into picture. And if you do that well, then you will win, I guess.
SPEAKER_00Right, and I agree with you. A very simplistic way of thinking about the world of AI is in terms of product development and distribution, as we're alluding to. To the point you're making, you're saying knowing what to build as well as how you build the product because that's where taste comes in, right? Because in a world of a billion different project management software, what are the specific ways that you build it that will be more compelling to users?
SPEAKER_01Exactly.
SPEAKER_00So that's one thing, right? But then as the cost of development goes down, I think the distribution cost increases. And so then you're shifting to how do you actually gain distribution? Because I think those costs are going to go up. Is it by owning a community? Is it by working with influencers? And I do think that the value of influencers, of people who own audiences, That starts to increase, the value that starts to increase. And that's kind of like a simplistic view of viewing where the world is going with AI.
SPEAKER_00But I think the more general principle that I think people should understand is more around what do the constraints become? For example, to the point of product development, What are the things that AI is weak in? What becomes more valuable? It's the things that the AI and LLMs are bad at. And so those things, how do you become extremely good at those things so you can bridge the gap? Because ultimately, when you're working with AI, it's a partnership.
SPEAKER_00It's human plus LLM together. And then in terms of distribution, what are the areas where that become hard? Without going into the specifics of that, and I have some points of view on that, but for the purposes of this talk, I would say that's the general principle that I think that people should be thinking about is when development goes to zero, then what becomes valuable? In terms of what to build, what are the LLMs bad at?
SPEAKER_00What are the skills that you can double down on in terms of what we already talked about such that you can make the right thing?
SPEAKER_01Yeah. So I mean, I think I have a good example for this. So imagine a world where a match three games don't exist. Okay. And you have LLMs, you have AI to code whatever you want, but match three games just don't exist yet. So until you prompt your way to figuring out that this could be a game mechanic, LLMs won't tell you that. So the ability to figure out that this could be a game or this game design could be addictive. I think that becomes such an important mode because LLMs can write amazing code, but if your game is not fun, no one will play it.
SPEAKER_01Or similarly, if your app is not solving the problem or if your app is not addictive enough, it just won't work in the industry. That's what people need to figure out that in game dev, for example, coding no longer is the important piece, but the pieces, how do you design it? Well, how do you make sure the mechanic is great, and so on. So that's where people need to sort of move on to the step. And the reason why I'm taking gaming as an example, it's easy to explain, right, but the same applies for any SAS or like any other software that you're building, right?
SPEAKER_00Okay, so I think we've covered all the topics today. Maybe the one thing that we can end on is for anyone watching who's interested in VibeCoding, What would be your last bit of advice for them? What should they do next week if you were to give them advice?
SPEAKER_01So I think the one advice I can give and I give to everybody is just ask a powerful LLM if it can do something for you, whatever you have in your mind. Because most of the times you will just be surprised that it will do it for you. Like so many people, they might just think that how the hell can it do this? Or how the hell can it do that? Just ask it to do. Either it will just do it or it will give you three steps. To help it do it. So for example, like a very, very random example, like my mom, she writes a blog, and she has like a bunch of images that she converts to WebP every single time
SPEAKER_01and uploads it. And it's obviously something that Cloud Code can very easily do. But for somebody who hasn't been coding for a long time, she didn't even think that it's possible that an LLM can convert images for her and give it to her in a batch. But she one day decided to ask Cloud Code, can you do it? It's like, yeah, I can do it. Or I made a promo video for my app using Cloud Code for the first time. I didn't even imagine that Cloud can make a video. It did it. So I think the main advice is whatever problems you have, the hardest thing you can think of, just ask an LLM. It's 90% probability that it will just do it.
SPEAKER_00And it changes all the time. I remember somebody had posted, hey, check out this This video that was made by an AI alum, I was like, oh man, I didn't even know you could do that. So to the point that you're making, there's so many capabilities that AI can do, and people don't know what they don't know.
SPEAKER_01Yeah, exactly, exactly. So you need to accept the truth that you don't know what you don't know, and keep probing until you figure out what all can be done and what all can't be done. Like I've been using LLMs to wipe code for over a year now, but I'm still surprised every single day that how the hell did you do that?
SPEAKER_00Right. Yeah, and I think my own advice would just be for the audience. Like we are, in my opinion, we are living in an extremely like significant and important time in human history, right? And I actually do believe that AI is the most significant and impactful technology in all of human history. So my advice would just be like, if you have not tried AI Invest now.
SPEAKER_00Again, I just assign a probability distribution to any potential outcome. But there are some potentially very dystopian and very negative scenarios that could happen in the next year or two. So I think that a lot of people may regret it unless you just carve out two to three months right now to do whatever you can. Just start building. Just start learning. Because you don't want to be a year from now thinking, I wish I would have done that.
SPEAKER_00And then it's too late. And then you lose your job. You can't feed your family. And you're part of this civil war.
SPEAKER_01I hope it never comes to that.
SPEAKER_00Right. Okay. All right. Well, folks, there you have it. Harsh and I have been vibe coding. I would say they're probably not. I mean, I'm sure there are a lot of people out there vibe coding. Yeah. You and I in particular have been doing it a lot. So these are our perspectives from a couple of guys who have done a lot and who have launched code that's being used by other people. And so for whatever it's worth, I think that's what we think. And so for the audience, I hope that was helpful for you.
SPEAKER_00And we will catch you next time. Thanks, Arsh.
I like to think about game development as product velocity, and velocity is a vector. It has a magnitude, which is how fast you build, and a direction, which is whether you’re building the right thing. AI is pushing the...
Kalie Moore ran the prompts across ChatGPT, Gemini, and Perplexity. What came back turns game discoverability into a PR problem, and she gave me the playbook she uses to fix it. When I ask people in games what PR is...
In 2002, Electronic Arts banned ICQ - the office chat app - and put a single internet terminal in the center of the building. A programmer named Steve Ganem responded with what he calls “civil disobedience”: he built...
Here’s the top of the Fortnite metaverse, as of June 20, 2026, during a Saturday night at around 8 pm EDT: (Source: fortnite.gg) #1. Battle Royale: 549.3k concurrent players. #2. A battle royale variant: 97.7k #3....
Here’s a contradiction worth sitting with. Earlier this year, Epic Games - the studio that spent five years and north of $100 million breaking the platforms’ 30% tax - quietly cut the value of V-Bucks by 20% to, in its...