Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The Pragmatic Engineer

The Pragmatic Engineer artwork

Uber’s AI infra, from the inside

Oct 14, 20261 hr 33 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Brought to You By: • turbopuffer - a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable • Linear - the product development system for teams and agents • WorkOS - everything you need to make your app enterprise-ready. - What do developer tools look like inside of Uber, today? They are very different from what you would see at most places. For example, can you guess what service a system called DevPods, Code Inbox, Slate or Wisdom would be providing to engineers? Today’s episode is a deepdive into the day-to-day tools of devs at Uber, our tour guide being on of Uber’s longest-tenured engineers: Uday Kiran Medisetty. Uday is one of the few Distinguished Engineers at the company, and leads much of the work on agentic AI, today. With Uday, we talk through how Uber’s developer infrastructure prepared it for AI agents. We look at the systems Uber built before generative AI and the new tools that connect agents to its code, internal services, and engineering knowledge. Uday explains how engineers use background agents, why tools like Slate make development (and bugfixing!) more efficient, and how code review is changing. We also cover Uber’s approach to token costs and how it measures productivity as the engineer’s role changes. Timestamps 00:00 Intro 02:48 Uday’s early days at Uber 05:40 Uber’s move to microservices 07:55 Uber’s engineering team, then and now 09:05 Developer tooling before AI 16:03 DevPods 19:28 Building for AI agents 29:27 MCP gateway 34:37 Agent Skills repo 38:46 The context graph and Cortana 47:54 PRD and ERD tool 50:45 Minion 54:46 Testing and code review with Bazel, Slate, and Capri 1:12:26 Edge Gateway, Shepherd, and Wisdom 1:21:33 The changing role of engineer and measuring productivity 1:30:40 Uday’s favorite new products - The Pragmatic Engineer deepdives relevant for this episode: • How Uber uses AI for development: inside look • Scaling Uber with Thuan Pham (Uber's first CTO) • Inside Uber's move to the Cloud • The Platform and Program Split at Uber • Hypergrowth startups: Uber and CloudKitchens with Charles-Axel Dein • How Uber Built its Observability Platform • How Uber is Measuring Engineering Productivity • Developer Experience at Uber with Gautam Korlam - Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_01What kind of internal tools do software engineers at Uber use to build software today? We get an exclusive, insider, and visuals-heavy look from one of Uber's longest-tenured engineers, Udaykira Mendicetti. Today, Uday leads much of the work on agentic AI at Uber. In this conversation, we get into internal developer systems Uber has built and used pre-AI like UOwn, M3DB, Michelangelo, Cadence, Morpheus, and others. AI Foundational Systems is a company built like its MCP Gateway, AI Guard, and AI FX. Recent systems used by nearly all engineers like Minion,

SPEAKER_01CodeInbox, Slate, Shepard, Wisdom, and others. If you want to understand what AI native development looks inside of a modern tech company with thousands of engineers like Uber, and why it's much easier to retrofit AI on top of BattleTest's developer Infra, this episode is for you. This episode is presented by TurboPuffer, Vector, and Full-Text Search built on object storage. It's fast, cheap, and extremely scalable. Today's episode is about the close to two dozen internal systems that Uber has built to help engineers ship with AI agents. One of the takeaways will be that production-ready code is far more complicated

SPEAKER_01than just using an AI coding harness. The harness is the easy part. What's hard is all the things around them. This is where I need to mention our season sponsor, WorkOS. One piece that is always a hard problem with agents is permissions. Do you want an agent to have standing access to everything it might touch? Or do you just approve every single prompt until you just stop reading the prompts? Worker was built airlock for this. Airlock is an authorization layer that sits between your agents and the tools they call, and it judges every single request against the agent's intent and your rules.

SPEAKER_01You write the rules in plain English, agents never see your credentials, and every verdict is logged. It works with coding agents like Cloud Code and Codex. Workhorse Airlock is how agents do real work without over-permissioning. Check it out at workhorse.com slash airlock. Uday, welcome to the podcast.

SPEAKER_00Thank you so much for having me. I've been following your work. It's amazing to see the kind of impact you've had and all of the great content you publish for all of the engineers. So I'm super honored to be here and talk to you today about what you're doing.

SPEAKER_01Well, we worked together well before The Pragmatic Engineer was born back at Uber. Not together together, but I knew of your work as well. When did you join Uber exactly?

SPEAKER_00Yeah, so I joined Uber in May 2015. In fact, like the two engineers who referred me, they are still here. So I joined in our real-time API team. It was during the period where we were trying to break out the Break our monoliths into microservices and I was working on our API Gateway for all of our mobile apps and like I worked on changing our mobile protocol. Since then I worked on many different business platforms from our Fulfillment, Uber for Business, on our earner onboarding, marketplace pricing and the last one year has been all agentic AI both

SPEAKER_00for our users, customer support, end users and so on. So yeah it's been like a roller coaster last one year on working on this.

SPEAKER_01Yeah. And for those people who didn't work there with us, can we recall a little bit what real-time API or RT APIs, we used to call it inside, what it was and why it was a big deal and why it had so much load?

SPEAKER_00So when I joined, We had just two services at Uber. It was called Dispatch written in Node.js and API written in Python. Dispatch used to handle all of the rider and driver traffic at that point in time. And the protocol was pretty simple at that point. It was like a four second ping protocol we used to call that would give you the entire state of the world. And when the trip is completed, we would hand off to API. It would handle all of the post trip processing, payments, receipts and everything. RT API was born at that point in time to abstract all of the internal services

SPEAKER_00and mobile and also changing the protocol from simple four-second protocol to back like where we have both REST APIs and we also have server to client. We used to call it push protocol. So we would, because so much of our app is very dynamic, everything keeps changing every few seconds. So we needed a way to efficiently update the state of the app across both riders and drivers as and when we get newer Updates as in when things change in physical world. So it used to be at that point, like one of the highest QPS services.

SPEAKER_00And over the years, we kind of because all of our stack was just no JSON Python. So that was the natural choice at that point. But in the subsequent years, we have converted that into Go and it's still one of our highest QPS services at Uber.

SPEAKER_01And I think for people listening, they're going to be surprised when they hear like, oh, there was only two services in 2015 or around that time at Uber, because of course, Uber was known for the microservices boom. And Tuan was on this podcast. He talked a little bit about this. But would you mind just recalling how those microservices came to be? And from your perspective, you know, like what it meant to suddenly have so many services?

SPEAKER_00Yeah, I mean, I think there's like so many blogs and like so many people talk about that. So I think like when I joined, right, like we were, as I said, like we were just two services and there and you also think going back in those days, the kind of hyper growth. We were constantly expanding to new countries, we were constantly expanding our product portfolio across mobility and later we started Uber Eats and that also was expanding. We needed a mechanism for teams to really build all kinds of functionality very quickly to match up to the scale at which

SPEAKER_00the business was growing. So we have invested a lot in all kinds of service infrastructure making it easy for teams to build services, metrics, tracing. Monitoring, observability, deployment, the whole bunch of stuff around that. And there's not much off-the-shelf software for these things at that point at the scale and rate of change that we had. And there's a lot of stuff that we had to build in those days internally. That created a boom cycle of a lot of, we went from like two services to order of thousands of services.

SPEAKER_00Then around 2018, 2019 was when we standardized on our languages. We picked Go and Java as our primary languages. We modernized a lot of our tech stack. We spent a lot of time in thinking about the layers of our architecture. How do we platformize? And over the years, in fact, when I worked on one of our fulfillment platform, We actually reduced some of the services to a few services. Even now, we are kind of in the similar ballpark of number of services. Even though our business is 10 times larger, we are doing many more products.

SPEAKER_00And this whole bunch of platformization and years of investment that has gone into it, like we are still running one single global tech stack across all the countries we operate in. There's many, many platforms that Serve all of the mobility verticals and all of the delivery verticals using one shared building block. And I think that kind of allows us to build more mobility verticals, more delivery verticals, because we have platformized a lot over the years.

SPEAKER_01Would you mind telling us when you joined, how big was Uber's engineering team and how did it go over time? Because I don't think people from the outside appreciate just how big that growth was.

SPEAKER_00So when I joined I was around the first 600 engineers and in fact the whole of marketplace team at that point used to be like maybe 50, 60 engineers and over the next few years As we've scaled our business, as we scaled our traffic, we grew to thousands of engineers and around 2019-2020 we started also expanding across different sites. Now we have 12 global tech sites across the world in almost Every continent we operate in.

SPEAKER_00In fact, that helps us build products for those hyper-local markets. And some of those products are then rolled out globally across all the markets where we could roll out that kind of functionality. So I think that gives us a lot of ability to tap into talent globally and also build nuanced local products. And that also helps us build one global tech stack.

SPEAKER_01And speaking of building products, tech stacks, You still work on the platform team, right?

SPEAKER_00Yeah, so at least until my first 10 years, I worked on different business platform teams. And in the last one year, I'm more spending my time on a developer platform, on an agentic AI for engineers, agentic AI for the rest of the company. And in the process, I've also worked on some user-facing AI initiatives as well.

SPEAKER_01Can you take us back to 2015 on how... Uber's internal tooling changed because again, I was there and I was really surprised to see just how much custom tooling we had. We talked about RT API, but there were things like Morpheus Edge Gateway and other things. Would you mind talking through like back in 2015, 2016, if an engineer joined Uber, what kind of tools did they use and how did those evolve? I'm talking obviously the main ones that most engineers back under mobile engineers touched at Uber.

SPEAKER_00Yeah, absolutely. Like in those days, given the way in which we were scaling, there's not many off-the-shelf products that we could tap into. So I think we talked about RTPI and Edge Gateway. Some of the other stuff that we had to build was a system called YouOwn that will enable anyone in the company to identify who owns a specific asset.

SPEAKER_01Like it was like services, right? And you could look at during on-call, if there's an outage, you could figure out which team owns it, who's on-call for that kind of stuff, right?

SPEAKER_00Yeah, absolutely. And as the engineering size grew, we also had to build like end to end, like our engineering design docs, product design docs, getting all of those into one single place. We had to work on getting all of our incident manager into We are like really obsessed about how we handle our incidents and the learnings from that. So we build a lot of really good tooling around that. I think there was some of the more complex ones was our metrics, like even back in 2015, 2016, we used to probably We have around like 500 million metrics emitted per second with

SPEAKER_00the cardinality and there's no off the shelf metric system that we could use and that gave rise to M3DB which was open sourced later and there's even companies formed around that. And that enabled us to build a handle kind of metrics volume that we had. We also had to build our own end-to-end machine learning platform because we used to have hundreds of ML models. And if you take the whole lifecycle from creating the features,

SPEAKER_00building the models, monitoring, serving.

SPEAKER_01This is Michelangelo, right?

SPEAKER_00Michelangelo and like we recently just few couple of months ago, we open sourced our deep learning and part of the Michelangelo infrastructure. So there's a lot of like really good solid foundational building blocks there. The other interesting one is also Cadence, which is If you think about some of the workflows, right, like they span across multiple days, whether it's a driver onboarding, whether it could be some CRM campaign, you could see a lot of engineers doing their own resiliency stuff, their own state management, and that gave rise to Cadence,

SPEAKER_00which is a durable workflow engine, and later even that got open sourced.

SPEAKER_01I think not many people will know Cadence, but a lot of people will know Temporal, right? Which was, as I understand, either inspired by Cadence or a lot of the ideas came from there. I'm not even sure it was the open source version of Cadence, but it had a lot of, like it's kind of the external version of whatever Cadence is for Uber or used to be.

SPEAKER_00Yeah, and I think those folks like this, there's a real need for something like this, and they were able to go and build, scale it even further beyond Uber. But I think that the pattern that I'm trying to say, there's a whole bunch of things that We had to like invent in those days for the scale that we had and over the years then we are like now as the industry also mature like now we have few few products that we build so somewhere we have we think there's some unique need we buy and if we build anything if we think that's interesting for the rest of the industry

SPEAKER_00then we try to open source wherever possible.

SPEAKER_01Yeah I think two more tools that I think is worth mentioning because I just remember them as one is Morpheus It's basically launched directly for Uber, but it was way more integrated into everywhere, right? Like you could go in there and run experiments, you could see the results immediately. It was almost like feature flagging and experimentation, but integrated into mobile feature flags. And it was like, I remember inside Uber, like all of these integrations, it was just integrated into a bunch of tools, all of them, right?

SPEAKER_00Yeah, and since we are also expanding to so many countries, you need a really good configuration platform, whether it's for configuring our backend services or also having feature flag system and also experimentation. Uber is a very data-driven company and everything goes through a feature flag, everything goes through an XP. And since we are a two-sided marketplace and three-sided marketplace, some of the experiments are also not

SPEAKER_00We also have switchbacks and we have to build a really deep experimentation that's baked into all of our tools and it can automatically generate analysis and what are the p-values, what metrics to use and that also led to working a lot on our metrics foundations, data foundations so that whatever analysis that we generate from this metrics are trustworthy and we can make business decisions based on that.

SPEAKER_01And I remember I was always wondering, why do we have our own feature flag system? But then I was thinking like, hmm, like show me a feature flagging solution for a vendor that supports cities. In fact, cities are first class citizens because inside Uber, like I think cities were so important and like a lot of our tools had cities as, for example, for cardinality, like every city had its own metrics. And then you start to realize like, oh, very few things treat, like Uber had a bunch of interesting constraints, right? Like that no one else had at this scale.

SPEAKER_00Yeah, absolutely. Yeah. Yeah. I think the city one is an interesting one. It's like so deeply baked into like our infrastructure. Like, for example, some of our like critical platform services, we also do sharding by city. Oh, sharding by city. Yeah. Because then we can also observe across different tiers of cities. Because we need to make sure, even now, across all of the regions where we serve our traffic, we shard a city to a specific region.

SPEAKER_00We can also at any point in time failover that city to across regions and even and we also have like a logical deployment groups where a group of cities are routed to a group of containers and this is for few services we don't do it for all the services we do it for some of the services that are where we need that level of that level of sharding and that level of reliability.

SPEAKER_01One other system I wanted to ask you about that I remember using and it was being developed in 2017, I believe, which is DevPod. And it felt really ahead of the industry at the time. How did that come along and what did it turn into?

SPEAKER_00Yeah. So I think the backstory for DevPod is because we already talked about the growth of microservices. Initially, we had micro reposts where every service was its own repository. And we didn't have a standard build system at that point. Each service had their own, like, and we had at least two, three build systems, two, three testing systems. And it was very hard for even for a central team when we wanted to do library upgrades and go across all the services and do upgrades. So around 2017-18, we then changed the strategy to getting to Monorepos,

SPEAKER_00even though they are microservices, but all of the services were deployed from the Monorepos. We had one Monorepo per language, so we had a Monorepo for Go, one for Java, one for mobile iOS, one for Android and so on. Once the scale of that repository was pretty big enough then like it was pretty hard to run the build and like getting all of those systems set up on your local laptop also then you have to like go through all kinds of setup issues. So we took on this thing of like DevPod which is like cloud developer boxes

SPEAKER_00and you can have any number of them. We had one devpod per monorepo. So if I'm working primarily in Java monorepo, I create the Java version of the flavor of the devpod and it has all the build system cached. I can have faster builds. It's working on cloud machines as a beefier than even the laptops. So then you could have like we could have really good SLAs on your build time, on your test time. And we don't have to worry about the setup issues because we can we can show the right tools in place.

SPEAKER_00And now the interesting part is as we got to the AI era, like even the agents need the similar workspaces and cloud boxes. And we are able to kind of latch on to that. And we also created like now a mega dev pod with all of the repositories. Repositories, because then I can have my agents create PRs across the repositories. It has all the tools it has to work in any repository, to look at context across any repository. This also helped us to create the AgentsMD, CloudMD at the top folder because we have standardized builds, standardized testing.

SPEAKER_00Once we set up the top level files, most teams will get like really good value out of the bat. They don't have to like really go into per module setup. And like on the DevPod, all of the AI tools are installed by default. So you can get a new DevPod. You don't have to go through even installing like Cloud Code, Codex, OpenGo. They're all there in the DevPod. So this made it really easy even for non-engineers in the company to create a DevPod and start playing with Cloud Code, start playing with Codex out of the bat.

SPEAKER_01Yeah. So I guess like having cloud development environments just like really early helped a lot.

SPEAKER_00Yeah, like I think all of those years of platformization bets also helped a lot as we expanded on our agentic journey because you could use all the things that we built making the life easier for engineers and it makes it easy for our agents as well.

SPEAKER_01I wanted to talk a little bit about before we got into what's on today, because I think a lot of things are connected, right? So like there was so many things being built for platforms, for tools, often custom for Uber. But having talked enough about that, what does it look like today? What does development look like today? What kind of tools do you use? And maybe you could... Give us a little bit of a primer of the foundations that you put in place specifically for AI and what you built and why.

SPEAKER_00Especially in the AI world, one thing we realized very early on was because the landscape is changing pretty quickly, harnesses are changing pretty quickly, models are changing pretty quickly. We wanted to invest in the right set of building blocks that are hardness and model agnostic so that no matter any of these things change, we can easily port across different models, different harnesses. So few things that we invested like, so we started working on a model gateway from The initial introduction of ChatGPT, like late 2013, early 2014.

SPEAKER_00So we had a model gateway back then, which would expose an OpenAI compatible API or Anthropic compatible API.

SPEAKER_01Sorry, so like this is 2023, 2024, right?

SPEAKER_00Yeah, so 2023, 2020, early 2024, when ChatGPT was first launched and GPT models were first launched. And we quickly noticed, you know, we wanted to make it easy for any teams in the company who want to like leverage and build agents with these models. And so we had a model gateway that would abstract all of the complexity behind where we had all of the security and policy guards in place. We had all of the visibility in place. So you could switch across different models or different vendors for these models. And you could just have the ID for the organization or the project and

SPEAKER_00then you can use any of the models exposed.

SPEAKER_01So this is what you just expose it into like a model selector and it piped into the model gateway and it routed to the right, you know, may that be ChatGPT at the time or Opus at the time, that kind of those models.

SPEAKER_00Yeah, and it was built on Michelangelo itself. So we create a Michelangelo Gen AI product.

SPEAKER_01Oh, Michelangelo. But that's an ML platform, right?

SPEAKER_00Yeah. And then now there's a Gen AI set of foundations on Michelangelo platform. So then you go to create a Gen AI project or a Gen DKI project on Michelangelo. Then you can set up the project. What is Model Gateway?

SPEAKER_00I can show like a quick primer into like you know what we had built and this was late 2023 when the first set of LLMs were launched and we quickly realized that we had a whole bunch of internal teams wanting to use it for different use cases and we wanted to make it really easy for them and this was built on top of the Michelangelo platform and one more component inside that they can set up

SPEAKER_01And Michelangelo, just to recap, that's the ML platform that was built from like 2015, 2016. So it was built a long time before LLMs.

SPEAKER_00Exactly. And now we have like a Gen AI or Agentic AI set of components also inside Michelangelo. And we're also building a lot more right now that those are not in the open source version yet. But we are actively building a lot of stuff for agentic AI world. But my model gateway was the first such one and the initial version we didn't have all of these middle version components and we built some of this over the last one, two years. But what the experience for any team in the company is, you know, they create a client,

SPEAKER_00they give their project ID, which they initialized in Michelangelo. And then the caller token, and that's all. And we exposed one OpenAI Compatible API or Anthropic Compatible API. And all of the models across all the vendors, open-weight or closed-weight, they're all behind this gateway. And the first thing we added there was what we call AI Guard.

SPEAKER_01What is AI Guard?

SPEAKER_00Essentially, we wanted to make sure we are not sending any sensitive content back to the user. So this was one entry point where we could add this middleware that we can enforce consistently across all the projects. And security team had ability to like, you know, then they could iterate on all of the AI guard specific models, whether it's on content, whether it's on safety policy, whether it's on like a whole bunch of comms policy, where like five different set of fine-tuned models that they built.

SPEAKER_00Which runs on every request and every response with strict latency guardrails. This helped a lot to make sure that all of our projects have same standard baselines. And then because every project is tracked here, We could have per user, per caller attribution. Then as we progressed later this year, we were able to add spent years for projects, spent years for users. And since all the requests go through one place, we could also add audit logging, session traces, and a whole bunch of infrastructure around this.

SPEAKER_00Every model request at Uber goes through this. So there's no model request that doesn't go through this. So that gives us one single entry point across all the models.

SPEAKER_01And I assume like you can just try billing as well, like costs and all of those things.

SPEAKER_00Exactly. Like you can track billing, cash sheet rate and all kinds of things. You know as we progress into like there's a lot of discussion of like you know model routing and all of that I think this could be like a good entry point that we can leverage because all of our models go through this I think on the same journey of like being able to make it easy for engineers to switch across models the next thing was how do we make it easy for engineers to switch across harnesses like if I'm using cloud code then if I switch to codex then I need to like carry forward all of my settings all

SPEAKER_00the skills all the tools all the plugins So we internally wrap all of the harnesses with a wrapper called AIFX, AI Foundations. And this is pre-installed on Devpods or you can install on your local machine. And we have a concept of personas. So then that personas comes with a whole bunch of default plugins and skills so that you don't even have to install. Like any engineer, they get a whole bunch of default skills that they can use out of the bat.

SPEAKER_00We wrap everything, all of the settings are managed. So even if you are using any of the hardnesses on your local or on DevPod or any of your DevPod, the settings are synced. So then centrally we can manage one set of settings that we can propagate across all the users and any user-specific settings, they are synced across all of their environments. And again, this makes it really easy because now we have one entry point for switching the harnesses, switching the tools. And then we also made it like there's a status line where we can give tips,

SPEAKER_00where we can have tracking of your spend across harnesses, within that harness, in that session. So all of those additional things that we could add without any explicit setup from the engineers.

SPEAKER_01Nice. And then speaking of tracking spend, how do you approach that right now? Like as an engineer, I do, or if I just join as a, you know, like a senior engineer somewhere, like, do I need to be aware of spend, of costs? What's the thinking there?

SPEAKER_00See, we want to make sure engineers have access to all, like any of the tools that are available. And we have this notion of spend tiers. So by default, you get certain, certain tier. And then, then We have many levels of tiers. Once you are close to that tier, then you can raise a request to go to the next tier and you just need a simple manager approval so then you can discuss with the manager.

SPEAKER_00The ROI for that project, like, how are you spending? And it is just a conversation around responsible use of this, right? But we don't want to restrict engineers, like, even if they're working on, like, if this is going to the right project use, They should be able to spend whatever they need because ultimately we want to maximize the gain from this. But you also want to make sure there's a responsible use. So we try to do the minimal amount of paperwork needed to cross the tiers

SPEAKER_00and they will see across any of our tools how much they're spending within that hardness and across all the hardnesses. And we also have different tiers for different managed regions. So like then for all of the interactive tiers, interactive harnesses, we have some spent here for some of our background coding agents. We have a separate independent tier. So we make sure they can easily manage across all of these.

SPEAKER_01Yeah, but I guess it just makes sense. Even if I do my side projects, I will be aware of my own spend on my whatever limit I have. And I guess, I guess, do I sense it correctly that the thinking is just don't use tokens on stupid stuff that is that makes no sense or accidentally, you know, you might be just burning and you don't know. So like, just get that feedback. Like these things are not free.

SPEAKER_00Exactly like I think just having that feedback and we also have like a plugin called cost check that will run across all of your sessions and give you personalized tips based on your usage to say hey like this is the kind of thing this much you're spending on this project These are the tips that you can use to further improve your cost profile. And I think all of this creates a good conversation between managers and engineers, but also engineers will also see good feedback

SPEAKER_00into how they are efficiently spending across. And that helped us a lot with all of the real-time coaching with context help, model sizing, So that engineers are also aware of working in this new age. We also don't want inefficient spend.

SPEAKER_01And what other foundations that you built are related to AI? Oh, MCP gave way.

SPEAKER_00Oh, yeah. I think the other backstory was around, I think, early 2025. We started as Cursor kind of picked up. We enabled Cursor and MCP was a Spec that was created around that time. And we quickly realized that we can really increase the power of AI by exposing all of our internal tools in an easy way across things. We could potentially leverage this beyond just coding.

SPEAKER_00We could leverage it for all kinds of software engineering tasks. So around April of 2025, we started working on MCP Gateway, similar to Model Gateway. This is one gateway for all of the tools. So what we did was because we already had good ideal registry across all of our services, whether they're using Thrift or Proto, since we have the ideal specs, we auto-generate the MCP specs for any of these services.

SPEAKER_01And just to add, I guess, folks who don't know, like Uber services either emit Thrift or Prada buffers as strong contracts, right?

SPEAKER_00Yeah. And like we initially we are just ripped and later we added support for protobuf as well. But all of the new services in the last few years, they're all protobuf. The ones that were created like a few years And as they go through rewrites, usually teams switch to proto. But whether they're in thrift or proto, since we have all of the ideals in one single registry, we could use that registry to generate A default mcp spec for them

SPEAKER_00so they could anyone in the company they can go to this tool mcp gateway they can see all of the mcp servers that we auto-generated and if you go to any of that you could see all of the Tools that were generated from that existing API. So these might be the APIs from those services that are projected as MCP tools. And we auto-generate the request pack, response pack, the description. Later, we also,

SPEAKER_00based on the API descriptions, you generate LLM optimized descriptions for the APIs and fields so that we can, like, we don't, like, there are, we saw some tools with, like, 3,000, 4,000 tokens, so then we can optimize it for modeling. We have a framework for creating a custom MCP and proxy their custom MCP also through MCP Gateway. The other interesting thing we did was

SPEAKER_00Even all of our vendor tools, whether it's Google, Slack, all of them are also exposed through MCP Gateway. This reduced a lot of burden on configuration management since we always spoke about AI effects, right? So I could install any of the tools in the same manner, whether it's internal MCP tools or external MCP tools. In the last few months, we also projected all of these MCPs as CLIs. So you could execute any MCP as a CLI.

SPEAKER_00So then if some agentic hardness wants to use it as CLI, they can use it as CLI. If they want to use traditional MCP, they can use MCP.

SPEAKER_01And do I understand that this all comes because the APIs to start with were already either generated or written in, like a lot of them were generated that you already had a platform for the APIs. So adding MCP was somewhat complicated. I mean, trivial, of course, it's work, but you know what I mean. And now generating CLIs is also somewhat trivial because you had a foundation before AI, right?

SPEAKER_00Yeah.

SPEAKER_01And so when you have it, that's what I'm starting to see with Uber. You built a bunch of stuff before 2022 and you could just kind of add AI infra pretty easily on it.

SPEAKER_00Yeah, and the same team that works on our API Gateway also worked on MCP Gateway. So they're already in the business of building API Gateway. So they know how to do any kind of protocol transformations, any kind of API Gateways. So it was like we were able to pretty quickly build this. So I think this was probably like, especially once we got our Google Workspace and all of our Slack tools also into this, it kind of like created all kinds of Like amazing agency workflows.

SPEAKER_00People could automate anything right at that point, like from Slack to Google, Jira, Linear, like any any tool, vendor tool or any internal service is at a CLI call away. And then the other set of foundations as we like started thinking about this space was See I think all of these go in this camp of like vendor-neutral hardness-neutral building blocks that we think like you know as like you would have to do this irrespective of what happens because you would need agents need some way to access your tools.

SPEAKER_00Then the other thing was around skills like, you know, we jumped onto this earlier this year. Since we already have this notion of monorepos, we created a skill monorepo in some sense that has all of the core skills that central platform teams build and all of the team specific skills individual teams can build.

SPEAKER_01And are these skills, these are just like the skills folders, like the plugins that you can use for cloud code, etc.

SPEAKER_00Exactly. So there's plugins and skills that you can use across all the harnesses. Then we had like one single quality gates that can run across all of them. And we made it portable across all of the harnesses. So you install with the same AIFX tooling that I mentioned. And like if you do plugin add that skill, it will update the configurations across all of the harnesses, whether you are at that point using that harness or not. So that even if you switch across harnesses, you have these plugins.

SPEAKER_00And since all of the skills are using the tools from the MCP Gateway, we also have one standard mechanism for all the skills to access any of the tools. Now we are working on some tools where engineers can go select their skill, It will tell them all kinds of tips to improve their skills. The next thing that in-flight that you're working is how do we then collect the traces from the harnesses and generate improvement patches for the skills so

SPEAKER_00that we can continue Because there's a lot of discussion around the skill quality, skill evals and how do we improve that continuously. Like when you first build a skill, they might not be perfect and you need some mechanism to continuously improve that. I think both the MCP and skills, this gave us like a lot of like benefits. And like if I look at like in the last few weeks, there's around probably 30,000 skill executions per day.

SPEAKER_01This is kind of interesting for me to see because if you told someone like, oh, we're building a skills repo, like what most people would think inside a startup, right? I'm not talking about Uber, but like, oh, you just do a get... Just a repo and you have a bunch of scales there, just a bunch of scale folders and that's kind of it. But it's pretty clear that it's more complex, right? Is it more complex because of how many people, engineers you need to support? Is it because you're also running evals because you keep cleansing these scales

SPEAKER_01or like tracking if they're used or not used, those kind of things?

SPEAKER_00Yeah, absolutely. I think it's like now with any of these latest models, it's very easy to create like a V0 version of the scale. It's very hard to make it really good across all kinds of use cases. And that's why you want to To really make sure we have one place, we can update all of these skills. In fact, when we exposed the CLI projection for all of our MCPs, just a couple of engineers using AI,

SPEAKER_00they updated all of the skills in this repository to use the CLI pattern because it's more token efficient. Because if every skill says I want MCP ABC, MCP DEF, Then if I install these two plugins, I'm going to have like six MCPs across these two always in my context and it is so much context load right and like then we replace all the skills with the CLI pattern so that that had a lot of impact on our like token efficiency as well and the other thing is because We auto install this skill marketplace

SPEAKER_00for everyone. It reduces one more step of configuration and so then like the discoverability is easy, the setup and configuration and management is easy and we could build whole bunch of like eval pipelines around that, discoverability pipelines around that. We still have teams that have like their own repositories but majority of the skills go through this. My favorite is like context graph.

SPEAKER_01What is contact draft?

SPEAKER_00The backstory is back in December, one of the engineers built an AI assistant that you can go to, we call it Cortana, you can go to web or you can go to Slack and ask any question from any code base. So it would search across all of our repositories and answer questions. And then we saw people asking all kinds of questions like it could be research it could be like what is how to run this query like what is the data source for this who owns this like can you do RCA for this whole bunch of software engineering tasks

SPEAKER_00and when we're looking at some of these traces like it was clearly obvious that you know we could optimize the number of turns it takes for the models to get grounding because we have so many systems Historically, all of these are siloed systems. Engineer has to go into the task management platform, engineer has to go to our on-call dashboard, has to go to our own dashboard that we talked about. Then we're like, okay, how do we plug all of that into work? We have 100 plus node types and 130 plus edge types and this is live

SPEAKER_00If you click on any person, you know what assets they manage, what design docs they wrote, what features they're working on, what on-call events they worked on, what are the configuration changes, what incidents they manage. And if you take any node, it has a relationship to all other nodes. And what this allows us is now the same assistant I talked about. It has the context graph as the default grounding. So if you ask any question, It will first use this to like navigate,

SPEAKER_00okay, like what is this person asking about? Once you get grounded into, oh, this is about this domain, this set of services, then it can go deep and read other documents, read other things. So it reduces your agent races and the amount of turns it needs to give you a good answer. And we already see like in a whole bunch of categories of questions, The quality goes up, the cost goes down, the latency goes down, and we are still early in harnessing the full potential of this,

SPEAKER_00and we are super excited about all of the things that we can build on top of this going forward.

SPEAKER_01And then the nodes, are they people? Are they tools?

SPEAKER_00This is the schema visualization. So every person in the company will have like, there's a node for me, a node for every employee. And then it will have relationships to all of the instances, like every PR that I generate, there will be a node for that. And there's a link saying, okay, Uday created this PR. Uday manages this asset. I work on this product feature. I contributed to this design doc.

SPEAKER_01So then the nodes are both people, but they're also assets.

SPEAKER_00Yeah, exactly. Like one simple example is, let's say if you ask any question like how many mobility trips around the world are with cash payment type. Like you first need to understand what tables exist, what columns exist. Are there other people who wrote these queries? Are they from science team? Are they from mobility science? Like now all of this data is interconnected. You can go and check are there other queries that were done in the last 30 days around this and what kind of tables exist.

SPEAKER_00Go and explore those columns because again all of this is interlinked. Then you can quickly get to Like the blessed queries that we have and answer these kind of even complex questions pretty quickly. Otherwise, you would have to go through many, many dogs, many, many like tool calls. This makes it really efficient. Like it's much harder for us to manually write all of these cipher queries, but agents are really good at navigating the graph, like how they're really good at navigating file system with bash.

SPEAKER_01It seems very futuristic to me, but it's pretty cool. And I assume it's a ton of data to store just because Uber has like 10,000 plus people working there easily. So...

SPEAKER_00Yeah, the good part is, you know, similar to how our API Edge Gateway team worked on MCP Gateway effort. Our graph platform team that manages graph or our production graph with billions of nodes, they also helped build this graph. So this is, in compared to our production graph, this is much smaller. So this is, so it's not like a scale challenge for them anymore. I already talked about the AI assistant Cortana which kind of plugs some of all of

SPEAKER_00the context craft, tools, skills and it's omnipresent in Slack, Web, CLI. So anyone in the company like even someone in Ops in some country they can ask any question about how is this built, where is this located. Who works on this? What was the history behind this? And like it is able to take all of this data, all of our Google documents and then and then give them like a really good answer. And it breaks down kind of the information silos across orgs, right? Like they're able to like self serve, ask anything, execute any skill anyone has built.

SPEAKER_01And I guess it must have access to the data sources, at least for the right people, so it can go into things like logs and monitoring, those kind of things.

SPEAKER_00Yeah, it has logs, monitoring, you can ask, okay, what is the health of this service? It can go look at all of its dependencies, what kind of storage systems it uses, run all the queries and give you a 360 degree health report for that asset. And we carry forward all of the existing user permissions, so then if I'm only able to access the stuff that I had access to. Otherwise, that data is not accessible.

SPEAKER_01Yeah, because inside Uber, I remember it used to be called break glass when you added a permission for a system you didn't have access to. You go through an approval flow, which goes through your manager or director or whatnot. But I know Uber was really big. At my time, they were really big on this already.

SPEAKER_00Would I just mention how Uber wrapped up AI foundations work but already more foundations work is to come.

SPEAKER_01Our presenting sponsor, TurboPuffer, knows this all too well. They are rebuilding their storage architecture from first principles. They are doing this to make more types of storage faster, cheaper, and more reliable at scale. The new system called TurboPuffer v3 is not yet in prod, but it already passes CI. Still, it's not nearly as performant as the current production version, v2, and so the TPuff team is now perf-maxing query plans until they are as fast or faster than v2. They're doing this in public, documenting how they increase performance all the way up to

SPEAKER_01the v3 production launch. For example, we can see how the team increased full-text search performance 9x by batching reads. A lot of this is basic foundational work right now, but they will be sharing some very cool systems work over the next few weeks. But hang on, why is the team doing a full rewrite? It's because search is changing, and so is TurboBuffer. T5 started as a Vector database, but it is no longer VectorDB. It has evolved into a high-performance database for search. The V3 discards the Vector Primary Index, and all indexes will share the same columnar storage system.

SPEAKER_01Every query will run on vectorized query engines, which is something you don't usually see in search databases, Which will result in both very fast search and just generally a very fast database. The TurboPuffer team is keeping a work log of their journey and you can follow along at turbopuffer.com slash v3. It's a blast to see how it's being built. I also want to mention our season sponsor Lanier, the place to coordinate work between humans as well as agents. One of the biggest changes for us software engineers is the sheer amount of code that we need to review. Uber has built custom tools to deal with this but so has Linear.

SPEAKER_01It's called Linear Diffs and it lets engineers review PRs right inside of Linear. One thing I really like about Linear Diffs is how a diff sits next to an issue inside a project and so it's super simple to see all of the context in one place and know why the change has happened without needing to jump through different tools. For large PRs, guided reviews walk you through the core change with an explanation of what has changed. Guided reviews also group the glue code and the supporting edits separately. These small but important changes make a large PR readable in one sitting.

SPEAKER_01And linear also supports stacked PRs. If you've used stacked PRs, you'll know that they're a great way to break up complex work. And if you've not yet used them, you'll probably love them when you try them out. How we do code reviews keeps changing as AI agents take on more of the work. These neat changes coming from Linear help us engineers understand code reviews more and expect Linear to evolve diffs to keep up with how we work more with AI agents. To learn more and to get six months off of Linear, head to linear.app.pragmatic. With this, let's get back to Uber and how building a feature looks like today in terms

SPEAKER_01of tools used by devs at the company. ERD and ERD, I guess for those who don't know, product requirements documents and ERD was engineering requirements documents, right? Like it's like the product spec and engineering plan for those not familiar inside Uber.

SPEAKER_00Yeah, so I think we went through some of the AI foundations and like now if we think about the lifecycle of a feature, then how can we Truly uplevel each of those life cycle with the right set of AI foundations and also rethinking those tools for the AI era. And some we are more farther ahead, some we are early in the discussion, like I think even the notion of a combined There's a lot of cool ideas, but we also have all kinds of agentic skills for working with any of your

SPEAKER_00Design documents. And as I was saying before, whether it's a product design doc, engineering design doc, launch reviews, XP reviews, production readiness reviews, all of those things are in one place. I'm filtering some really old projects so that we can look through them. But you will know exactly who is working on them, track the status of that. And all of this data also goes into the graph as well so that we know like full history of what are the source of truth design docs,

SPEAKER_00approval status, who works on them. And each of these also have like the Jira link or the linear link. So then we can trace from this project, how was the execution translated to. This gives like a source of truth system for both humans and agents to know the specs for every project.

SPEAKER_01Yeah, I guess planning is still really important, especially for like something that is not just a trivial, you know, like one prompt or one shot project.

SPEAKER_00Yeah, exactly. And because we need to like, there's so many with the kind of Product heterogeneity and country heterogeneity like we need to like really think about all kinds of corner cases more than like they know the the happy path to think about all kinds of corner cases and like how they should exist and since we also operate in physical world the number Yeah, absolutely. I think that's why it really becomes important to think about the right plan upfront. And this will also

SPEAKER_00Make it easy for both the engineers or agents to take that plan and decompose and execute on that. Once we have the plan, then the next thing is how do you convert that into, decompose that and execute. Now we're going to what we call Minion internally. So this is our background coding agent. So around 15% of all of the production PRs go through Minion right now. Essentially, you know,

SPEAKER_00this whole bunch of PRs that some PRs engineers want to do interactively, but we also want to make it really easy. Once you have for the task that I have really good spec or I want to do minimal steering, I can hand off that to a Minion. And we had again we had we did multiple iterations is the latest iteration we call it Mahini and v2 but like you can use it from web you can use it from slack you can do a plan mode and then let it execute behind the scenes or you give one you can give the prompt once and let it execute

SPEAKER_00behind the scenes if it's like a simple simple prompt and it has access to this is using the mega dev pod so then it has access to All of the repositories, all of the code base that we have, it can work across repositories and generate one PR or multiple PRs across repositories based on the plan and spec that you have. When we initially started, mostly maintenance kind of tasks used to be offloaded to Minion. As we progress on this journey, as we are adding more and more tools into the Minion environment

SPEAKER_00where Minion can Self-validate its work in web or mobile or backend APIs. Our goal is we can increase the kind of tasks people can do with Minion over time, increase the percentage of tasks over time. I think we can also see how it's

SPEAKER_01integrated in Slack. You can just mention, do you mention at Minion or what do you say?

SPEAKER_00You can do at minion and then it will show you the status. And you can also like you don't need to go to the web. You can steer it on Slack itself. Maybe you have like a Slack thread with the With the EM, with the PM, you can iterate on the design. If it's a simple thing, you are just iterating and get alignment there. And then say, hey, at Minion, follow this thread. And can you create a PR? And then it will take that Slack thread context and then create a PR for you.

SPEAKER_00So it's been really useful. Even in our support channels, I would see like someone reports a bug, the on-call ask Minyan, hey, can you see what is the issue? And then Minyan will come back with, oh, this is the root cause. Oh, this looks simple enough. Let's just fix it right there. And then we would like fix it. Like there's no then loop of like adding it to backlog, doing in the next. It's improved since hasn't it?

SPEAKER_00Yeah, so this is showing all of my devpods. Since I'm working on context graph, you see that. But yeah, if you want to create a new devpod, you'll see which flavor you want. For minion, there is a devpod flavor that has all of the repositories, and optimized for minion usage. But for interactive, it will create a devpod, and within a matter of seconds, You will have that running.

SPEAKER_01That's impressive. I'm sure you do some clever stuff like having things warm or pre-built so you can copy those kind of things.

SPEAKER_00Yeah, it's like, it's running on, all of, this is, because this is developer stuff, like, we have, like, a separate multiple clouds that we are using for this, and it's, like, Kubernetes cluster, and, like, everything is warmed up, so then when you ask it, like, we are able to just clone from the pre-built image, and it has build caches, everything, repositories, everything, already cloned, like, it's ready for you to build, so, we use Bazel across all of our repositories for build systems, so,

SPEAKER_00I was able to go into a repository like run Bazel for that and then just like we went we went through this in the order of this like you know seconds where any engineer like if they if I want to make a code change here like within order of seconds I create DevPod It's one single way to build any module and then anyone knows this command. And now agents also knows the same way, right? In the ClaudeMD, AgentsMD, they know how to build, test any module in any repository and they're able to use the

SPEAKER_01exact same commands.

SPEAKER_00Yeah, so I think that covers kind of like, you know, what the kind of stuff that you have to do in code generation. So you can do interactive hardness, you can do like through Minion. The next thing is, okay, like you have the PR. What do you do after that? Then, um, Maybe go through self-healing CI first. One thing that we realized is because our PR volume has gone up year over year. Oh yeah, it must have explored. Yeah, our P50 cycle time, P90 cycle time, both for inner loop where you have the PR and you want to get it to a green state

SPEAKER_00and create the remote PR. Or once you have your PR created, Getting the human review, getting all the CI things to pass. I think reviews become a big bottleneck, same as like any other company. And we see year over year the delta and we want to like really, we're like really focused on like breaking down every aspect of this. And like how can we go really deep and optimizing it for the AI era. So first thing that we did was we took a whole bunch of like simple CI jobs that used to run once you create the PR and we moved it to inner loop

SPEAKER_00and we kept all of this behind one single command like AI FX validate. So then when agent creates a PR, it can run the linter, the coverage rules, the build rules, the template rules, like all of these things so that the probability of this PR is in a good condition like goes up. Then we hooked on ability to do visual validation where I can spin up emulator and test it. I can spin up like a web and test it. I can spin up like a staging environment where I can deploy this change, run the APIs and test it.

SPEAKER_00And now Minion or even my interactive harnesses, these agents know how to have the skills for this. And they can go through this verification loop. And let's say you create the PR and still something fails. Then we have like a self-healing CI. You can apply to all the PRs or per PR. So this is like a uber specification. Oh, nice.

SPEAKER_00And once you do all of this, then we have this thing called Code Inbox, this is showing a mock version of that, where any engineer can go, they can see all the PRs that are waiting for them, all the PRs that they created, You can just quickly take a look. You have a video as well.

SPEAKER_00Yeah, I can see the bill checks and all of that. And we're also adding risk score, blast radius of this PR and so on. So all of this will give engineers really good signals to figure out, okay, is it good to land and which parts of the PR I should focus on. And, yeah, I think we are still, like, we are aggressively, like, working on this whole bunch of ideas that we have, but we want to make sure, like, you know, as we are ready for the influx of PRs, we, in fact, had to, like, add internal code read for our CI system because year over year,

SPEAKER_00the number of CI jobs also doubled or more than doubled. So we had to, like, also, like, beef up a whole bunch of foundations in this space so that we can be ready for the influx of new PRs.

SPEAKER_01Yeah, and I guess you're also seeing that there's no way engineers can review all PRs with the same rigor as we did in the past. So I guess you're building tools to signal which ones to pay more attention to, which ones might be low risk in so many ways, maybe in impact, maybe in blast radius. What is your take on what will happen with code review over time? Do you think there will always be human code review or for some things it might go away?

SPEAKER_00Yeah, absolutely. See I think one is for us like we want to err always on the safer side because the cost of an incident for uber is like really high so even with all of this tooling we like right now we still require an engineer to review Every PR that goes in, once the more confidence we get in our risk scoring, and we can also backtest it with years of data, right? We have all of this data,

SPEAKER_00and the more confidence we have in our AI code reviews, then maybe for some class of low risk PRs, we could enable, we call it green lane, like auto landing. But we want to be really, really sure about all of the dimensions that we want before we go there. But right now, the main priority is how do we make it really easy for engineers to get all the signals they need to review faster. And if we do a good job there, then I'm sure we can increase the percentage of Autoline PRs over time.

SPEAKER_00But that's kind of P1, P0 is first getting all the signals in the right place.

SPEAKER_01Nice. Yeah, but CodeInbox is nice. I like it.

SPEAKER_00This is a thing that we should touch upon is like maybe how I can do sanity testing.

SPEAKER_01And sanity testing, just for those who don't know the internal lingo, that's the manual testing, right?

SPEAKER_00Yeah, manual testing. Yeah, I want to do, let's say I'm working on some new feature. It touches two or three services. How do I know if this feature is working end-to-end? So then typical workflow for engineers is like pre-agent equalities. We call it Slate. Essentially, you can create your own instance that deploys the branch that you are working on and you can route your test account UID directly.

SPEAKER_00To the instance that you created.

SPEAKER_01So when you log in you are executing that branch.

SPEAKER_00Exactly, that branch and rest of the upstream and downstream goes to the other staging setups. So that way you can test end-to-end without worrying about other parts of the infrastructure and this will carry forward across the service graph. So you can say from your mobile app or this is the test account I want to test it will go through other services and when it touches the service under test then it will go to your slate instance if you're testing two or three then it will go to the service under test across all two or three

SPEAKER_00so you can do even multi-service tests with this now that we had this we we had to just teach agents how to deploy slate And we also give it all of the past APIs that you can use. Oh, nice.

SPEAKER_01Basically, this was a thing where I as an engineer wanted to sanity test, meaning I want to manual test up my changes. I had a way to do this in a pretty cool way. And now we can just give it to agents and they can actually test their own changes end to end in a safe way.

SPEAKER_00Yeah so here we are showing a web version of that and you can see in fact like the this is like the CLI pattern we went through it's using slate mcp create new slate environment and it's saying this is the environment name and then once it creates the environment it's saying deploy service to this environment and giving the service name this is a test service and now It has its own environment. This is like the Slate UI. And if you see here, you can pin test account UIDs to that particular instance.

SPEAKER_00And you will see all the services that are in that particular environment. You can see the Gitsha and... Now, the agent internally can use all of these skills and tools. So if you see what it did was, it did A-B testing of that web thing. It ran agent browser in that slate environment with what is on main and what is on my branch. And it created a report for me to review.

SPEAKER_00The full set of changes that happened and so then like after this we can go to the like a report that it generates like see generated a report that shows before and after and showing me like all that and there's no regressions and this is very easy for like you know engineers to validate and we can attach some of the screenshots and videos to to the PR and the same thing we have for iOS Android web And if you want to do back-end APIs,

SPEAKER_00then you can also test the back-end APIs.

SPEAKER_01I just wanted to add one of the things that, again, you might not appreciate if you're not outside of Uber, but Slate is so important because Uber literally today still has thousands of microservices, or maybe not micro, but to run a change on just one service when you want to do end-to-end, that's just difficult, right? This is why Slate took years to build to start with.

SPEAKER_00Yeah, and you also have to think about the... Routing of the pinned accounts.

SPEAKER_01This is the test account that I want you to route to my branch that is just deployed in a staging environment. That's not trivial at all.

SPEAKER_00In fact, how this works is one of the things that we We built back in maybe 2017-18. It's called Jaeger, which is a distributed tracing infrastructure. It's open source now. And all of our services have Jaeger spans. Essentially, wherever the call originates, there's a parent span. And at every hop, there's a child span. And there is a, you can add any baggage on that trace, which can be propagated across the call stack.

SPEAKER_00So when you pin from the slate web, we are essentially adding, saying that for these are the Services under test and this is the account that I want to route. So as it is going through our service mesh, service mesh knows this baggage and then it can say, oh, I want to route this request not to this environment, but to the slate environment. So because we had tracing built out and because it's like propagated across the stack, now you could actually test and we can reliably route these kind of test requests to the right environments so that it makes it really easy to test.

SPEAKER_01This thing took years to make sure that all existing services were retrofitted with this and all new services were built with this in mind. I'm starting to sense a trend where Uber built stuff over years, pre-AI, just infrastructure that is easy to test for humans or easy to automate. And now with AI, it seems like, oh, you just built this the last year, but actually you built on top a foundation that was built for years, right?

SPEAKER_00Yeah. I'm sure I think if we did not have system-like slate, it would have been really hard for agents to self-validate. Then you cannot automate this thing. And as you said, this kind of stuff really helps. And I'll also talk about one more pretty interesting thing, which is so uber-specific. What is Capri? Yeah. And this is like very, very hard to get right. And like, we had like even many versions of build like this in the past,

SPEAKER_00but I think this is probably our best version and we can like really scale it to all the teams. But essentially, the idea is like context, because we have, even if you look at our core Rider app, right? Like we have hundreds of screens.

SPEAKER_01And the Rider app is what most people think is Uber app, right? Yeah, Uber app, exactly.

SPEAKER_00Yeah, like we have different business, different product lines, different cities, different like whole bunch of dimensions. Like if I want to test any new thing, how do I know every combination of these has no regression? And if you were to like create all permutations across this, you would have to maintain thousands and thousands of tests across the stack. And that alone is like, again, really hard problem in itself. So for some of our most complex platforms like we call it Capture Replay like essentially you

SPEAKER_00can capture real production traffic with just one middleware because now we have standardized language frameworks across all the services is very easy to hook this into and It will capture not just the inbound and outbound at that service layer, but also any of the downstream APIs that service calls, those requests and those responses are also captured. So in some sense, you have like full request response and all of your outbound request responses captured. Now, with all of these captured requests,

SPEAKER_00they go through some pipeline and then you can build simple rules that says, oh, if you see this kind of field in the request, this is the value for that dimension. So then you can capture all kinds of dimensions that that platform team cares about. And once you have all the requests and all the dimensions, you can in some sense generate the full set of live dimensions and live combinations that that platform is currently supporting, which we call it MatchGuard that I can run pre-deploy in our CI CD that shows

SPEAKER_00for all of the PRs that are going in this deploy, let me validate against The full combinatorial set of like combinations this platform supports and show the data before and after. So then you get really high confidence if am I breaking maybe cash trips only in Brazil with this configuration, which would have been really hard for engineers to manually test or agents also to manually test.

SPEAKER_01So step one is you capture what you want to capture and then you have the system that will rerun it, check that there's regressions.

SPEAKER_00Yeah. So like then I'm going to one of the domains that has this enabled so you can see all the sessions that it captures. I can go to any of the sessions. Then you can see all of the metadata and tags. So essentially it's saying, oh, this request is tagged as marketplace type eats. It's tagged as line of business eats. It is in this kind of terminal state. It knows captured all the requests. You can also see all of the outbounds that it used. If we click on that, it will also show you The request and response for the outbounds.

SPEAKER_00And if you see at the top, there is slate workflow UID or search environment name. So then let's say if someone, if I'm working on a bug that's only affecting for some reason, San Francisco, Uber Eats trips, I fix that bug. Then I can create a slate environment for that bug for my code changes. I can go here and then say replay Just this request on my slate environment and show me the delta here.

SPEAKER_01Show me the difference and you're expecting a fix or some change.

SPEAKER_00Yeah, I'm expecting the fix because I know the before request, after request, what I'm expecting. And then I can do it for a specific dimension or I can use it. And then we also generate what we mentioned like match guard, which is the full set of dimensions possible for that. So then you can create like a When I was there, we didn't have this, but we have something called Deputy and Cerberus, but those were, I guess, very early versions.

SPEAKER_00Yeah, so in fact, what Deputy and Cerberus allowed was ability to route any request from production to your testing instance and Cerberus allowed from your local testing instance back to the staging servers. But now that we have Slate, we don't necessarily have to use Deputy and Cerberus because you can deploy a Slate environment and we don't need to use any of them.

SPEAKER_01I just want to stress because I think it's hard to see from the outside, but that's really hard to build this because every single, again, you have thousands of services, all of them have to support these things from routing, for example, to capturing production, to replaying, right? Like this was a lot of work, I assume.

SPEAKER_00Yeah, and now I think the same, like similar to how we exposed like slate to agents, we can expose all of the Capri functionality to agents so agents can check, oh, I'm trying to fix this. Let me get all of the sessions of this type and then see what request looks like. Can I compose a request of that type on my slate instance and see did it fix or did it not fix? Let me go in a loop until I fix this.

SPEAKER_01Yeah, yeah. And now the agent can, even though it cannot see, but it can see the delta and it can tell that nothing has changed, for example. So it can get feedback loops.

SPEAKER_00Exactly, exactly. I think that helps a lot. Now, the agent can also capture replay on Slate with any of these. We already talked about the Edge Gateway.

SPEAKER_01Edge Gateway, that's just kind of Uber's internal API, right? It used to be RT API, real-time API, but now it's called Edge. It's a Go version in Go, if I remember, right?

SPEAKER_00Yeah, it's in Go and now it supports a whole bunch of endpoint types. So there's a sectional API where there's gRPC, webhooks, GraphQL. So again, all APIs go through one single entry point where we can build a whole bunch of tooling around that on monitoring, on alerting, And like even there is like a like where there's all the reliability features. The Edge also has its own capture replay similar to the Capri. It's using Capri behind the scenes.

SPEAKER_00And like if you want to look at We also automatically tagged which service service service service service. Which is a required dependency, which is an optional dependency. So that metadata is also there. So as we are even looking at alerts and diagnosing alerts, all of this rich context really helps both engineers to debug faster and also agents

SPEAKER_00to debug faster. Yeah, we already talked about experiments. The last thing we will cover is like what we call Shepard and Fox. After we moved to Monorepo, like then we moved to Monorepo because upgrading libraries was very hard, right?

SPEAKER_01On all of the different repos, yeah.

SPEAKER_00Exactly. Then you move to monorepo, but then now someone has to go and upgrade this library usage across all of the modules in that repository. So how do we automate that? So we first took a stab at using open rewrite policies. If it's a simple signature change, you can write that in open rewrite policy. And then Shepard knows what are the unique modules owned by different teams. And then it automatically creates PR for each of those modules.

SPEAKER_00When AI came into picture, then we could add like a AI-based shepherd where, okay, if I want to do fixed context propagation across all the services so that tracing works better, I can create set of prompts and skills that then agent can run across every module and fix all the issues in every service faster. So this is kind of a large scale Changes whether you want deterministic changes or whether you want like agentic changes across the modules that gave us like we were able to do a lot of migrations over the years through this

SPEAKER_00and we call each one as a campaign and if you see J unit 4 to So JUnit 5, there will be a YAML file with the policy and you'll see a whole bunch of PRs created for each module and a campaign owner can track it, the status of that and go through that. So it makes it really easy to manage these migrations. A few months ago, then we took the same platform and added like one more feature. We also want to make it easy. We want to continuously improve the code base, not one-time migrations. Let's say I want to...

SPEAKER_01So like code maintenance?

SPEAKER_00Yeah, code maintenance. So we call it Fox. Essentially, we want to create self-improving code bases since we already have a whole bunch of agent skills. Essentially, any service at Uber, they can go here and enroll their service into different skills.

SPEAKER_01They can say,

SPEAKER_00I want dead cloud cleanup or performance analysis or noisy logs. There's a whole bunch of these skills that you can onboard to. Oh, smart. And what Fox does is every Monday, essentially every Sunday, when we don't have that much CI load, it will generate a new PR for this service using that code cleanup. Can I clean up any code? Can I improve the performance of this? Can I fix any noisy logs? So it's kind of automated then code base improvement.

SPEAKER_00I like

SPEAKER_01it because I heard people do this thing where they would set an AI loop, let's say in cloud code or codec saying, look for ways to improve my code base and they would execute it every night or once a week. But it sounds like this is just a lot more optimized. And I love that you can enroll your code base because a code base is owned by an engineer or team at Uber, right?

SPEAKER_00Yeah, exactly. And you can see, because you're rapidly scaling, we are still in the opt-in phase. Once we improve some of the skills, we'll move some of them to opt-out phase. But any time you obtain a skill for a service, then every week, in some sense, you are getting a PR created, like a managed agentic loop. So then you don't have to like, you are continuously improving this. In some sense, the more toil we reduce,

SPEAKER_00the more amount of time engineers can spend on the true feature work. So like constantly thinking of, okay, how do I eliminate more toil from engineers and automate all of these things. And so we constantly look at all kinds of PRs with maintenance category PRs. Do we need to create a new skill? Do we need to improve that skill so that That class of PRs can be onboarded to this automated manage loop.

SPEAKER_01This is nice. I like it. And then finally, I heard about wisdom. What is wisdom?

SPEAKER_00Oh, yeah. So we use wisdom because all of our employees dog fooder products. So if you're using a rider app or earner app, and if you see any bug, then you can, from the app, report that bug. We now have all kinds of AI stuff where like it will automatically route it to the right team based on the stuff and then when you report a bug because it's from the employee like beta apps we can capture whole bunch of logs and telemetry around

SPEAKER_00that that makes it really easy for the team owning that feature to fix it so In this case, it's like an internal feature. It's essentially like if you want to test any feature in a different country, you can mock the location on the device. So if someone reported that issue, the first thing Now how do you reproduce that before it is it used to be a huge problem to reproduce so now we know that the set of screens the user has gone through because we have that in the logs and then we can give the engineers like this is

SPEAKER_00the reproduction steps and this is the expected behavior and this is the observed behavior and so that we can boil down to the issue quickly And all of the telemetry that we collected from the app, you can see all the logs in one place, whether it's analytics, network requests, all kinds of feature flags. So that also gives a lot of context. And now that we have all of this context, we build a managed agent called DebugAssist that looks all of this, looks at the code base, and it will try to pinpoint what is the root cause and what is the impact,

SPEAKER_00evidence timeline around that. Now, once I think, okay, this is the right root cause, I can hand off this to Minion that we talked about. This root cause looks good. Like, why don't you create a PR for this? Minion will take all of that context that we have. It creates a PR. It will attach that PR to the Jira task. The Jira task also has the root cause analysis. So anyone who's working, they know The root cause analysis, if someone updated the root cause on the comments or like a new PR, again,

SPEAKER_00all of that is data that we can then figure out, OK, how do we improve the debug assist? How do we improve minion? We didn't get the right RCA or we didn't get the right PR from the RCA. Like both will be able to continuously improve.

SPEAKER_01And so do I understand correctly that you can capture and reproduce bugs so much easier, right?

SPEAKER_00Yeah, and we also added ability to, because for the employee reported bugs, we have all the logs, I can take my test device and I can ask it to simulate what that employee has experienced, so then Once I give the bug UID, we can essentially serve the same network responses that we served to the person who filed so that I can experience the same issue in my test device app too. So that way,

SPEAKER_00then once you fix it, then I can see this actually got fixed and then it makes it really easy to reproduce.

SPEAKER_01I love the recording of the thing because that's so important. It's hard enough to do even on the web. You need to build custom tools for yourself if you want to do a website. But it's awesome that you did it.

SPEAKER_00Yeah, I think this helps a lot like then in reproducibility and like getting to the right root cause.

SPEAKER_01So you built all of these systems and especially last year, it seems so many brand new systems that are just doing things like, you know, they're maintaining the cold quality agents are validating themselves. How do you think the job of software engineers inside of Uber is changing? And what are you learning from all of these new tools? It seems like you're learning yourself.

SPEAKER_00You know like in some sense like if you're the agentic AI world like you're constantly trying to find the next bottleneck and trying to like remove that and figuring out how you can like both using AI and both using like both systems engineering like reduce that bottleneck because ultimately our goal is to maximize the number of features that we can ship across all of our products and that will require us to look at the entire life cycle How do we rewire that for the new world and what kind of tools

SPEAKER_00that we need to build and I think this is going to be like an ongoing journey right and there's a lot of discussion of you know this concept called capability overhang where I feel like even right now like there's so many things that we could you leverage the latest models for but we don't have the Sufficient orchestrating capabilities or the harnesses or the tools around that to leverage those models in the right places in the right way. I feel like the more we work on this, the more we can unlock,

SPEAKER_00the more features we can ship, the more productivity that we can unlock. And we are trying to measure a whole bunch of this and constantly think about what is the next set of foundations that we need to build to make it easy for engineers to work and to make it easy for agents to work.

SPEAKER_01You mentioned productivity. How do you think about engineering productivity? Because when I was at Uber, this was again up to 2020, us managers did look at things. It wasn't the main thing, but we did look at things like diffs per engineer just to show if someone didn't contribute for like two weeks, it was like, what are you doing here? Or if someone did a lot of code reviews, that's actually what we wanted to notice. But it sounds like a lot of those things, the code output, the code artifacts are less relevant potentially. What is a productive engineer in your mind right now inside of Uber or inside of your team?

SPEAKER_00Yeah, so I think before the AI world, we were measuring the standard stuff like code throughput and stuff like that. Those are the easy ones. And then we also have every few months, we do the developer experience surveys. We look at NPS, sentiment across all kinds of dimensions. We look at our code quality dimensions, what's our incidence per def, what's our code coverage, and all of these metrics, right? I think now, once you go to AI, I feel like we can leverage the new set of capabilities to get more nuanced

SPEAKER_00into this discussion. Because now, we are internally talking about not necessarily just diff velocity, but feature velocity. Like, how much of all of this additional throughput is going towards unlocking new features? And how can we even change our organizational processes so that we can reduce the P50 feature time? Because Like, you can create 100 divs, but if they are not all feature divs, then like, maybe it's not necessarily increasing the value that we can provide to the users, right?

SPEAKER_00So, like one, for example, one of the things we did is tagging all of our PRs to different categories, like on complexity level, and is it a feature PR, bug fix, maintenance, testing, documentation on different categories, and we could see year over year, how is that mix changing? Luckily, like, at least like right now, I can see Our percentage of feature PRs is going up. Our percentage of maintenance PRs is going down. So that gives us good indication that we're in the right direction.

SPEAKER_00We see the code volume go up and the percentage of PRs for features going up. Then we are investing a lot. Because before, even though we had the U-Plan tool, sometimes you might not have the right tracing from this product design dog to this engine design dog to this particular task. So it becomes a bit harder to measure what is the cycle time end to end reliably across all the features. So now we're using AI to automatically cluster these things and tag any missing things so that we can reliably get what is

SPEAKER_00the P50 feature velocity for different complexity of features for every half over the last two years, three years. And then we can go into the bottleneck In the PRD phase, is the bottleneck in the launch phase, is the bottleneck in the rollout phase? What is going on there and what kind of things we need to invest to accelerate the number of features that we can ship per half?

SPEAKER_01And how is your mental model changing of software engineering compared to pre-AI and pre-Ds tools? Is it still the same job? Have things changed?

SPEAKER_00I mean, at least in my total tenure as in engineering, there's never been this much change to how I work. Like, for example, like, you know, we say, like, all the engineers at Uber are hands-on, like, even distinguished engineers at Uber, like, are hands-on, they, like, they create PRs. And I used to think I was hands-on engineer, but my hands-on definition year over year has completely changed. I'm like, even personally, I'm doing like way more PRs than what I was doing last year. Like it's more easy to do all of these things, all of these things, right?

SPEAKER_00And I'm able to like explore more number of ideas than what I could possibly do in the past. But the job of the engineering is, you know, in some sense, how do you build the right context for the agents to produce reliable output like in some sense like you need to get really good at context engineering before I was operating with some context to do some tasks so I'm always thinking about okay if I want to offload some of this work to an agent what kind of context it would need and how do I set up the environment with that right context so that it can produce that work then you have to become really good

SPEAKER_00at like you know how do you give the right specs right constraints for the agents apart from building the right context then multitasking has become really like a thing before if you're like it would take so much time to create a PR so like at least laser focused on that but now you're constantly switching between so many things like you so you need to become really good at multitasking across all the sessions figuring how do you scope the how do you scope the work how do you track the work How do you verify the work really quickly?

SPEAKER_00Because if you're not able to verify the work and pick any code smells from the agent, then that's hard. Then you're shipping some recreations into production, right? So the context engineering, ability to multitask, ability to review and verify the outputs, these things have become much more important now because you're able to parallelize a lot more than before.

SPEAKER_01One thing I'm picking up is maybe a new primitive is we used to build software thinking, how do I build this software and how do I verify it? Whereas now the question that you said is often, how do I set up an agent to be able to build this and to be able to verify it? So it's kind of like almost a meta, like there's more of that, right?

SPEAKER_00Yeah, I'll give an interesting example. I was working with some of the engineers that work on our customer support team. We are going through the journey of having AI agents handle some of our customer support policies. That process would go from, oh, you have some runbook that human agents would follow. So then from that runbook, you are going through a whole bunch of steps to create a policy that AI agent can follow. Then you have to think about, okay, there is some dynamic elements here, then which APIs I need to call to get that data,

SPEAKER_00which attributes I need to use, then I need to configure my system, change bunch of configurations, then I need to do manual evals before launching. Then I had to do continuous evals post-launch. And then in the last few months, we were able to take down all of the steps. How do I now take all of the context and all of the principles that I want to use for the policy conversion and teach agents to be able to do that so that I have a really good V0 policy. That an agent can create. Then you automate all of the configuration setup,

SPEAKER_00manual testing, finding the right APIs, finding the right properties so that we can hook the right dynamic elements into this. Again, like that used to take weeks, now it's like order of Then, instead of doing manual evals, then how can you use all of the past context to create synthetic evals on top of that so that you can accelerate, you can deploy this and then verify this quickly. Essentially, what we were discussing, which is Now, instead of doing that work,

SPEAKER_00how can you build the right tools and harnesses on top of this so that you can automate the workflow? Obviously, you need the right guardrails and human verification steps in between, but you're designing a system that can do these things instead of relying on so much manual work across all of these steps.

SPEAKER_01Yeah, it sounds like you're learning a lot because these are all new concepts which you're now using, you're learning about it, you're doing the limitations. I see what you mean by more learning, more PRs. And as closing, what is a tool that you really like using? This can be a digital tool, this can be a non-digital tool, something you find helpful.

SPEAKER_00Yeah, I know the last few days I've been hooked to like some of the personal agents like Instang, Muse, Grokbots of the world. I'm playing around with all of them. It's helping automate a whole bunch of manual work that I do in my personal life. And I can in some sense follow the same process, give it all of the context it needs to do that task for me, give me all of the inputs I need. Then I can have human in the loop verification step and do that task. But it's helping a lot to automate all of the manual personal work.

SPEAKER_01Awesome. Well, Uday, thank you so much for walking through all of the internals that are being used at Uber. I found it super interesting, especially that I haven't seen this for quite a while.

SPEAKER_00Thank you so much for giving us the opportunity. I feel like we are still pretty early in this. There's so much to work on. There's so much to build. And we are super excited to be on this journey, to build the right stuff and to see how the software engineering changes and how at the end of the day, unlock the value for the company, value for our business by working on the right side of things.

SPEAKER_01I hope you enjoyed this look into Uber's tooling as much as I did. It's now been six years since I left Uber, so a lot of the tooling that Ulay showed me was also new to me. Uber was known for having about 5,000 services with about 2,000 engineers 10 years ago. But since then, the company started to consolidate services into well-sized services and today it has fewer microservices than they did a decade ago. Today it's in the ballpark of 4,500 to 5,000 even though these services are not as micro and they're much more well-sized instead. One piece of software that I really like is Capri.

SPEAKER_01The event capture and replay software. Back in my time at Uber, between 2016 and 2020, we had two tools called Deputy and Cerebrus that could do some level of production data replay, but it wasn't all that easy to use and it was just really hard to capture a real base to start with. With Capri, it's far more trivial to capture sessions, then replay them, and this works really well to reproduce bugs and also to test that a fix worked for a certain bug fix. Building a system at Capri is especially complex when you have closer to 4,500 or 5,000 services running like Uber.

SPEAKER_01Another thing that was interesting is how we're seeing where CodeReview is headed. It's slowly dying everywhere. But looking at the tooling Uber built around CodeReview with CodeInbox, every PR still gets merged after a human engineer decides that it's ready to do so. But Uber is building a lot of automated checks. Many of them are with AI to give more signal about the riskiness of the change and the quality that AI agents make of it. And so in the end, it's a lot easier for engineers to decide which PRs they should spend time looking at it in depth Versus ones that are safe to approve with the symbol LGTM,

SPEAKER_01AKA you might decide you're not going to read the code. Finally, I had the sense that Uber investing 10 plus years in developer productivity is really, really paying off with their AI integrations. For example, they built cloud development environments called Devpods back in 2017. Uber got to reuse almost all of this infra in building out their cloud agents called Minions. The same is true for existing systems to validate software, to safely rule them out, and now agents can use these as tools or as feedback loops. So it feels like companies that are getting some of the most benefit from AI are

SPEAKER_01the ones that have had developer platform teams pre-AI as well, at least in the case of Uber. Do check out the show notes for more of The Pragmatic Engineer deep dives on Uber's engineering culture. Uber is probably one of the companies I have covered in the most detail so far. If you've enjoyed this podcast, please do give it a rating and tell your colleagues and friends about it. Thanks, and I'll see you in the next one.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Software factories and code reviews with Harjot Gill

    Nov 18, 20261 hr 27 minAudio

    Brought to You By: • turbopuffer - a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. • Entire - The Git forge for agent-human collaboration. Built for the scale and...

    Credit the people on this episode

  2. Software engineering after code becomes cheap with Satya Nadella

    Nov 16, 20261 hr 3 minAudio

    Brought to You By: • turbopuffer - a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. • Linear - the product development system for teams and agents. • WorkOS -...

    Credit the people on this episode

  3. Scaling Slack and GitHub with Maude Lemaire

    Nov 5, 20261 hr 19 minAudio

    Brought to You By: • Antithesis - verify your system’s correctness without human review or traditional integration tests - and avoid bugs or outages. • turbopuffer - a vector and full-text search engine built on object...

    Credit the people on this episode

  4. Software engineering at Google & CS education with Titus Winters

    Oct 29, 20261 hr 40 minAudio

    Brought to You By: • turbopuffer - a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable • Linear - the product development system for teams and agents • WorkOS -...

    Credit the people on this episode

  5. The revised rules of engineering management with Will Larson

    Oct 21, 20261 hr 56 minAudio

    Brought to You By: • turbopuffer - a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable. • Entire - Git hosting, rebuilt for the agentic era. Every agent session, prompt...

    Credit the people on this episode