Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Code & Curiosity by Maximilian Schwarzmüller

Code & Curiosity by Maximilian Schwarzmüller artwork

Devin - The World's First AI Software Engineer

Mar 17, 202411 minAudio

Devin - the world's first AI software engineer - was announced. What does that mean for software developers? And how does it actually work? Let's take a look!

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

So it happened. The world's first AI software engineer called Devin was introduced by Cognition AI, a company that, well, builds AI programs as it seems. For example, Devin, this AI software engineer. And that of course means that now the end is near for us software developers and engineers, right? Well, maybe not yet. And I'll share why I'm impressed or

interested by and in Devin, but why I don't feel like it'll replace you or me anytime soon. And I'll explain. But first things first, what exactly is Devin? What is this thing that was introduced? Well, it's a program that in the end uses AI models under the hood to solve problems that are given to the program. And with problems, I don't mean, hey, give me the code for storing a file on

AWSS free with Node.js. No, I mean, bigger problems, which then would require multiple steps to solve them and which would require logical thinking to solve them. I mean, these kinds of problems, like, hey, build me an online webshop website where users can authenticate, add products to their cart, and then proceed to checkout. I mean, problems like this. That's what Devin is meant to solve.

These kinds of problems. And that, of course, sounds pretty impressive or intimidating, depending on where you come from, right? And indeed, Cognition AI, the company that built Devin, so that built this AI software engineer, they shared a video, an announcement video, and actually they shared multiple detailed videos as well, where they show what Devin can do. And they, for example, show that it was given the task of analyzing the

performance of different AI models. And then Devin went ahead and wrote a program that would do that. And in that announcement video, they also showed which tools Devin is able to use at the moment to solve problems. Since it's meant to be an AI software engineer, it has access to a terminal where it can run commands to, for example, create a new project or run a project. It has an IDE, of course, to write code. And Devin has access to a browser to

research, to take a look at official documentation pages, for example. And of course, also, if it's building a website, to test that website. And then Devin goes ahead and in the end, uses AI models or AI APIs behind the scenes to solve a problem step by step. So it's issuing multiple requests to those AI models. The first request might be, hey, how do I create a new web project? And then with the response given, it

would execute the command that was given to it. And then it would continue asking the AI models behind the scenes. How do I use this framework or which framework should I use for this? What's the official documentation page? It would then maybe go to that page and pass the results it finds there back into the AI model to understand the results and deduct the next steps and so on. That's essentially how it works. And in their announcement post or in a

video shared there, Cognition AI, for example, also showed that Devin was able to use these tools to, for example, solve jobs on Upwork. And in the announcement post on Twitter, for example, they also shared that they also used Devin to solve open GitHub issues. And there it was able to solve just a bit below 14% of all the issues it was given without assistance, which could be

considered a low number. And I'll get back to that, but which is actually quite impressive, especially if you compare it to the numbers of similar projects or programs, which are far lower, as they also mentioned in their post. I think they mentioned that comparable programs only achieved around 2% without assistance compared to the 14% of Devin, so way less, and around 5% with assistance. So clearly Devin is pretty impressive

regarding that, and it has some pretty interesting capabilities. So that's all great. But still, I'm not shaking in fear. I'm super interested in experiments like this. And I do believe that as developers, as software engineers, we should embrace technology like this. And I did already talk about that in another podcast episode. So I think we should really be open to

that. And we should use these tools to make our job easier and to become more efficient. And I would gladly take an AI software engineer as a coworker and outsource tasks I don't want to do to that AI software engineer. But when it comes to Devin or other programs like Devin that might come up in the future, replacing me or you, I'm

not too convinced that this will happen anytime soon. And the reason why I'm not afraid is that, of course, Devin does some impressive stuff here. But it's still far off the numbers, the numbers of issues solved, for example, that would be achieved by a human software engineer. And in addition, there also isn't too much information available regarding the exact issues it did solve. We don't know how well it would, for

example, be able to build entire software projects that are maybe a bit more complex. So that's all unknown. And you would guess that if Devin would be able to build super complex applications, they would share an example of that as well, right? Now, you could say, it's just a matter of time, because these AI models will improve, right? And you would be right to some extent. But if we think about other areas where AI is used, think about self-driving cars,

think about speech recognition. And if you then consider when we first heard about technologies like this, and when we first started using such technologies, which is pretty much 10 years ago already, and how far we've come, then you would probably agree that, yes, there are impressive demos there and impressive use cases. And there are some valid use cases where AI might be used in our day-to lives. But you would probably also agree that

AI hasn't really brought us self-driving cars yet, no matter if Elon Musk keeps repeating that it will be available for everyone soon. If we think about voice recognition and AI assistants that talk to us and that really understand us, also not too convincing in my opinion. And therefore, I feel like we got some interesting numbers here. We have a very interesting demo here.

And Devin and similar tools might absolutely become helpful tools very soon. I mean, we have helpful tools like GitHub Copilot already, which also become better and better over time. So don't get me wrong, definitely, they will play an important role. They will do some of the work we need to do. But when it comes to replacing software developers or engineers altogether, I think we still have a very long way to go.

Because, for example, if we stick to the 14% of GitHub issues that's an interesting number, especially compared to the way lower numbers of comparable programs and models. But of course, it's still far away from what a human would be able to achieve here. Maybe it would take a human longer, but a human, a software engineer who knows what they're doing, would be able to solve all those issues essentially, and not just 14% if they wanted to.

So that is an impressive number, but it's still far away from what we need for such a program to actually replace software engineers and developers. I also already mentioned in that other episode, I already mentioned earlier, that of course, the main thing developers do in the end is solve problems. Yes, we write code to solve problems, and that is a skill we have. But if you're honest, the problem solving

part and coming up with solutions, and then writing code, refining and improving code, that's the main thing we do. And that's the thing an AI will need to do really well in order to replace software engineers or developers. And to me, it does not seem as if Devin or other AI models or programs we have out there are very close to replace

human developers when it comes to that. Now, I'm fully aware that there are a lot of people out there who will absolutely tell you the opposite. And for them, it's very clear that AI will replace essentially all of the high skilled work humans do nowadays in the next five years or so. And of course, obviously, no one knows what will happen in the future, though they know at least if you read what some

people are writing there. But anyways, obviously, no one knows. But for the reasons I mentioned, and if we take a look at other AI developments, like self driving cars and so on, I still am pretty confident and I feel pretty confident about being a software developer and about staying a software developer. But of course, I'm also interested in hearing your opinion. What did I forget here? What do you think? Where am I wrong?

Or where do you agree? Definitely share your opinion. And well, I see and hear you in the next episode, hopefully.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. We will have to accept AI-generated content

    Oct 15, 202612 minAudio

    I think we'll have to get used to (and accept) AI generated content. Not to slop though.

    • Apple Podcasts

    Credit the people on this episode

  2. Is software dead?

    Oct 13, 202612 minAudio

    There's a lot of talk about software being dead. Unsurprisingly, it's complicated.

    • Apple Podcasts

    Credit the people on this episode

  3. GPT 6 Astra - Good Beyond 3D?

    Sep 9, 20268 minAudio

    GPT 6 Astra was released. It's a good model, I have to say. Especially for 3D work it seems to be a real game changer. But due to all the hype, you may be disappointed...

    • Apple Podcasts

    Credit the people on this episode

  4. You may ignore Omarchy but not its impact

    Aug 26, 20268 minAudio

    Omarchy got a lot of hype and visibility recently. But even if you don't care or don't use it, you shouldn't ignore it.

    • Apple Podcasts

    Credit the people on this episode

  5. SvelteKit 3, Solid 2 - Do they matter anymore?

    Aug 18, 202614 minAudio

    We're not going back to the JS framework wars. BUT new frameworks and framework versions are important. For a different reason now!

    • Apple Podcasts

    Credit the people on this episode