Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow

The AI Native Dev - from Copilot today to AI Native Software Development tomorrow artwork

From Living Room Hack to 30 AI Agents at Cyera

Jul 21, 20261 hr 6 minAudio
ShareXLinkedInFacebookWhatsAppEmail

An engineer builds an AI agent to manage his own life, decides an unrestricted "does everything" agent is too dangerous to trust, and ends up creating the internal agent platform that now runs 30 agents across his entire company. Ori Shoshan, tech lead at Cyera, walks through the guardrails, citation system, and knowledge graph that turned "let the agent do anything" into "let the agent do exactly what it's supposed to, and nothing else." What we cover: - Why one engineer built his own AI agent in his living room, and how it grew into Cyera's internal agent platform - Whitelisting tools instead of blacklisting them, plus the "escape hatch" that keeps agents honest - Backing every claim with a citation and using a second model to catch hallucinations before they reach a human - Trading RAG for a knowledge graph the agent can walk like a wiki - Turning "use this platform" into "build your own agent" to drive adoption across an entire engineering org - Running agents on confidential data that can investigate everything but can only ever say what's been cleared Chapters: 00:00:00 - Introduction 00:02:04 - Meet Ori Shoshan and what Cyera does 00:06:50 - The living-room spark: why Ori built his own agent 00:11:09 - Whitelisting tools instead of blacklisting them 00:12:55 - Why every claim needs a citation 00:15:35 - Reducing hallucinations with clean-context verification 00:21:04 - Taking Borg to Slack: the first agents at work 00:25:33 - Why naming your agent drives adoption 00:39:53 - Knowledge graphs over RAG 00:52:00 - The AI Captains: scaling adoption with carrots, not sticks 🌐 Tessl: https://tessl.io 🔔 Subscribe for weekly episodes on AI-native development Have you built guardrails like this into your own agents? Let us know what's worked (or blown up) for you in the comments.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

SPEAKER_02OpenClock could really do everything. It felt very uncontrolled and just a disaster waiting to happen. LLMs in an agent harness. If they have an open check to do everything, then it's inevitable that they will eventually do something terrible. It's just like evolution. We came up with this idea of having an agent that's within the controlled environment and has access to the data. And you can have it investigate the issue and troubleshoot it and it has access to everything, right? But one big constraint, it cannot output except for structured validated output.

SPEAKER_02So it can only output for a fixed vocabulary, which you can verify doesn't take any data. So those two agents, they're called Mulder and Scully, because they worked on the X files.

SPEAKER_00The AI Native Dev is a podcast with developers and engineering leads at the cutting edge of AI and agentic coding. Join your host, Guy Fajani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today. This is the AI Native Dev. We just wrapped up two amazing days at AI DevCon in London.

SPEAKER_01But the great thing is that we get to do it all over again in New York City this November.

SPEAKER_00You're absolutely right. We're gonna be back in the city that never sleeps on November 3rd and 4th for more amazing sessions, really engaging, hands-on workshops, and much more.

SPEAKER_01Yep, all that great networking, partying, eating and drinking that you've come to expect from AIDEF CON. We think we have one of the best hallway tracks in the business, and it's the perfect compliment to our incredible speakers and presenters.

SPEAKER_00We'll both be in person and virtual with live-streamed access to all mainstage keynotes and talks.

SPEAKER_01Sign up right now for our super blind bird ticket for just$100, only available for a limited time.

SPEAKER_00We're really excited to be headed back to the big apple. We hope to see you all there.

SPEAKER_04This will be within Sayera, which is a data security company. We'll hear more about that in a sec. And to talk about that, we have Ori Shoshan, who has been one of the key leads uh building that out. Uh so welcome to the show, Ori. Thank you. Nice to be here. Um so we're we're gonna go a little bit in chronological order. So just to give people a little bit of a teaser around what it is that uh that we're gonna talk about. Uh I think you've built a really, really interesting and oftentimes quite unusual platform for how to kind of work with agents. That's a that's a compliment. It's like you're a little bit weird.

SPEAKER_04It's good. Who wants to be normal? Uh actually, like a lot of many uh smart, sort of small innovations and learnings. Uh I like you've built uh a different approach to harness engineering that uh starts maybe like whitelists tools versus blacklist them. Uh you have made some really smart infra investments in creating context graphs. Uh you It's my uh gentleman's side that has to have control over them. That's been great. Uh and then you know, a lot of things and a lot of uh I love the sort of small earnings around how do you permeate through the org, you know, things like uh having people take ownership of their agents or they care about it.

SPEAKER_04So and there's all these like great nuggets uh that we're gonna dig into. But instead of jumping into those, let's let's do this properly and kind of run through a little bit of intros and origin stories. So, first tell us a little bit about what is it that you do at Sayera and maybe a little bit about what Sayera is.

SPEAKER_02Uh I'm a tech lead at Sayera. Um, I work all across the org. A lot of my work is focused on uh the DSPM product, so that's the data security posture management, as well as leading AI adoption. Sara is an AI data security company. Um we combine AI security, well, everybody wants to do AI security nowadays. It's a boss. Um and before AI really boomed, um Sara did that data security, right? So we're a data security platform, meaning in a nutshell, we uh scan your data, we find whether sensitive data is, what type of data it is, and then we let you do the really important stuff with it, like um deciding uh what your posture should be, um seeing what's exposed and how it's exposed and fixing that.

SPEAKER_02And in AI security, also enabling you to um allow AI access to that data, uh, but in a controlled and secure way. Um which is it it's quite important to to know where your sensitive data is and what you do with it if you wanna build agents and and let agents on freely. Because otherwise your risk leaks, um, and if you're scared of data leaking, you're kind of paralyzed, right? So it's it's really all about control. Yeah, left on my German side.

SPEAKER_05Yeah.

SPEAKER_04I love the um uh the generally analogies in companies that have have kind of uh grown up uh out of the problem of the empowered workforce, which I think Sierra and sort of data security poster management or sort of DSBM, you know, fits that sort of mold where everybody gets to do a thing, like everybody gets to create data in the cloud, you create those, and then of course you have chaos because everybody's empowered to do that. Um and so here you know like income, a variety of solutions, many in the security space, specifically in DSPM, to like wrangle where where do I have data, what's sensitive and what's not.

SPEAKER_04And then as we get into agent land, you know, agents are uh even less trusted, you know, empowered uh entities that you're sort of running along. Uh and so they have to uh they have to be reined in even even more. And so I love these kind of realities in which uh kind of solutions built for the for the pre-AI era, but for the empowerment era actually very much sort of applied to this.

SPEAKER_02Yeah. Oh, and and about me. I I forgot to tell about myself. Um so I joined Sierra about a year ago uh for the Autorize acquisition. Um so Autorize uh was my company. I joined together with my team and my co-founder uh Thomas Greenwald. Um and together we lead AI adoption in Sierra R D. And that's sort of uh how it all connects.

SPEAKER_04And I guess just for context a little bit, that's how I got to know you. So I was a part of the uh Autorized journey. It was uh super interesting to talk about sort of uh uh intent-based access control and all that. I still believe in that. You know, okay it will meet. It'll still happen. Uh but uh you know, love sort of uh working now in the on the Sayra side, you know, collaborating there. Um so so so this is sort of the background, you're sort of in Sayra, clearly data security company, um I think in the order of magnitude of sort of hundreds of engineers on it. Um how how did you come? So you're doing AI enablement or you're building it, but I think your origin story was more about like scratching your own each, right?

SPEAKER_02Right. So it's 2026. Um and I'm I'm sitting in my living room, it's it's January. So it's now 2026.

SPEAKER_04We're still in 2026, like I don't know, timelines are a little bit murky in AI. But it's January, so we're like five, six months at a moment.

SPEAKER_02It's January 2026, and I'm I'm sitting in my living room and I'm I'm doing something on my phone, and I ask myself, why am I using my hands to do this? What why is it is it not an agent doing it for me? It's 2026, it's not 2025, as we all know, in late November, December. So yeah. 2025, um, frontier models really took a leap, at least from my perspective. Um I saw that I was able to build a primitive harness back then, basically, you know, like a coding agent with with some tests, and allow an agent to run much longer than it would have been able to before, with the just one generation of of the frontier models before.

SPEAKER_02And then so a few weeks after that, I'm in my living room living room thinking how how why why am I still doing things on my own? I need a personal assistant agent to do it for me. I'm building a house and all that, and I needed something to keep track of um of all of my commitments and uh the argument I currently have with the contract contractor for the house. Um and I really wanted an agent that would kind of be like OpenClaw.

SPEAKER_02Um, but I didn't like uh OpenClaw at the time. I think it was still called its its previous name, ClaudeBot or something. Uh Moldbot, ClaudeBot, one of those. Yeah, it went through phases, it's like a teenager growing up. Yeah. Um but I didn't like that OpenClaw could really do everything and anything. It's it felt very uncontrolled and just a disaster waiting to happen. Because I I kind of view AI as or rather, not not AI, but um LLMs in an agent harness. If they have an open check to do everything, then I feel like it's inevitable that they will eventually do something terrible.

SPEAKER_02It's just like evolution, right? They will explore that path if it's there. Uh so I felt like hooking up OpenClaw to all my stuff and having it manage my life, my life is kind of precarious. So I started building out my own personal assistant, um which uh where I took a bit of a different approach where instead of letting it do anything, I let it use a specific set of tools. And then um I got it on WhatsApp and I would text with it and uh it would be able to uh mention where uh uh where its data is from, and then I I figured actually I would like to know it's not making stuff up.

SPEAKER_02Where where is that sourced from? And I figured, well, I actually have that data, I have that link, so why don't I just have the harness tell me uh where where the what the source actually is, right? Um and then eventually I realized actually I want this at work too. I mean, why am I still reading Slack messages and and following on sales for most time-consuming tasks? Yeah, why don't I just know everything all the time, but only the most important stuff, and I want to have links to all of them.

SPEAKER_05Yeah.

SPEAKER_02I don't want to wonder, is this something the model made up or or is this backed by data?

SPEAKER_05Yeah.

SPEAKER_04So this is uh I want to sort of unravel a few things. I I I love the sort of the personal to company paths on it. I think in part because you're you're kind of allowed to play a little bit more at home, right? As much as you are cautious about your sort of uh your own kind of personal blast radius, you know, it's sort of a little bit different than playing with sort of company data. So so as you as you build those out, I think the first thing you talked about is is sort of this sort of containment of uh of only giving the agent some specific subset of tools on it. I guess how how crippling is that, right? Because like some like clearly kind of from an AI or sort of AGI maximalist path, it's a lot of like, you know, do not get in the way of intelligence, right?

SPEAKER_04Just sort of let it uh let it do uh uh the path. Uh I guess how much was that felt that you know when you if you contain it and you need to choose every time which tools are available, and maybe like a little bit of you know, there's probably still some leeway that you have to give it. So what did you find was effective in terms of of the lines?

SPEAKER_02So I I would give it quite a bit of tools and and it would be the the one tool I would not give it is bash, which is basically do whatever you want. Yeah. But it would have a bunch of other tools. Uh for example, to read all my WhatsApp messages and browse my email and uh but the how do you say um the attack surface uh for each of them would be well known. Um and I'd I'd always leave an escape hatch. So I could have uh so what what you don't want is for an LLM to try and use a set of tools and really be trying to do something else if it had bash l2 whatever, and then do something that doesn't make sense just because it doesn't have the option.

SPEAKER_02That's a bad failure mode because you may not notice unless you actually look at the thinking uh tokens for the for the model.

SPEAKER_05Yeah.

SPEAKER_02So I always leave a tool that's that's a sort of escape hatch, whether that's um doing something else what but with a human in the loop or recording feedback. So for the model to be able to tell you, I wanted to do this, but I have no no option to do that.

SPEAKER_03Yeah.

SPEAKER_02And then you go and add that. Um and you very quickly arrive at a 50 at a set of 50 tools um which the model has access to and it can do pretty much everything it wants. Uh but you know that it can only do those things and structurally, or I guess the the harness enforces that it can't do anything disastrous. Like it can't send all my personal notes to the contractor over email.

SPEAKER_04Yeah, yeah, that sounds like uh you know maybe definitely a negotiation sort of sides, or uh maybe even like just sort of a goodwill. So uh okay, let's sort of come back to that a little bit in the context of the company on it. The other piece you talked about is this notion of citations and sort of bring proof. I guess the sort of the association that comes to mind is what we see in Google, right? Oftentimes today, when you Google something, you get the preview, you've got the sort of the uh the equivalent of the blue links today, right, that uh talks about those references. Um what's um uh is it just about sort of sourcing the information? Is it similar, you know, what you built for your for the personal agent there, or is it something a bit more uh distinct from that when when you use it for your sort of uh uh maybe kind of more elaborate?

SPEAKER_04Because Google is very much about like lookup of information, so maybe a little bit more expect expected uh that the uh that there will be a link for it.

SPEAKER_02Yeah, so that the there is um I took inspiration from Chat GPT actually, where they at some point they started adding links next to where that the the claim is sourced. Um and but the the core need I had was to know if the model is lying to me easily because you get like a wall of text from an LLM and then you're thinking okay, well which of this is based on fact and which of this is reasoning that's correct, and which of this is reasoning that's incorrect, and and it's really cognitively exhausting to to have to constantly evaluate that.

SPEAKER_02And it's one thing when you're working with a coding agent and you're in an interactive session and you're in the zone, but for the personal assistant use case, you really don't want to be paying so much attention because that's kind of the use case.

SPEAKER_04Um so I wanted So you don't care about like your you're fine for it to take more turns uh for the exchange of getting the correct result at the end of it because you're not sitting there supervising, but if you get the wrong answer or you get something that then that that can be that can be um uh can cause substantial damage or at least waste a lot of time.

SPEAKER_02Yeah. Well I I I sort of assumed that uh LLM is gonna get the model is gonna get some stuff wrong. And then I wanted to make it more obvious to me when that's happening. Um and the way to do that is to have uh basically instead of having the model be able to just produce prose, uh it would be able to produce uh structured output, and that output is a uh a set of blocks where each block is uh a tuple of a citation and a source reference. Um sorry, uh a claim and a source reference.

SPEAKER_02The source reference is a citation. Um and then when the the output is rendered to me, I just get a list of well, each claim is just a paragraph, say, and then it has uh uh a source reference and the link is automatically rendered. And once I had that, I realized that I could actually use that to do uh some context engineering and do uh claim verification as well. Right, because um LLMs tend to hallucinate more or make the wrong conclusion when they have a ton of stuff in the context and they end up paying attention to the kind of in the dumb mode, you know.

SPEAKER_04They've sort of uh a lot of their sort of uh context window is taken up by historical actions.

SPEAKER_02They might end up you know uh answering in the wrong answer and and because they're kind of looking for a needle in the haystack, right? And they might look at the wrong needle. Attention is the scarce resource, right? Yeah, attention is all you need. Um so I realized okay, now I have the ability to take each claim and pair it with the raw data that's supposedly behind it, at least the LLM uh says so, and then I can take those two pieces of data and ask um another model with a clean context, which has the claim and the data, is this claim supported by this data?

SPEAKER_02And in that mode of operation, uh they tend to hallucinate a lot less uh because there's no noise. Uh so that reduces the amount of hallucinations overall. So each claim independently tends to be um more accurate, that there's far fewer hallucinations, and then if it is still a hallucination, I can just say that the source. And it's easy to tell if it made a citation that makes no sense. Right? It cited something completely irrelevant. Yeah. It does happen quite rarely.

SPEAKER_02Yeah. Because the LLMs have the models have gotten quite good, right?

SPEAKER_05Yeah.

SPEAKER_04I love the uh so I I like that, and I think uh I think in general, uh kind of I guess a principle of working with multiple agents is that any individual agent might hallucinate, but if you layer on even one more uh to sort of you know review the work, uh then uh then you reduce dramatically the sort of the chances of a hallucination. And I think you're you're you took it a step further because you're even requesting the original kind of claimer uh to uh to create something in a slightly structured format on it. So I uh I like that.

SPEAKER_02So that's sort of the escape hatch here was I let them actually uh go past, you know, go if they if they don't have uh a source reference, they can still uh produce a claim, but it has to be marked as opinion. And that's where all the reasoning goes. So I know to take those with a grain of salt, and the rest is what I use to evaluate the response.

SPEAKER_04I I I really like that. I think um uh oftentimes like the thing that is annoying when you're sort of interacting, and indeed, you know, the LLMs have gotten better uh in this, but you know, what used to be very annoying is it's not so much that the LLM makes a mistake, but that it says these mistakes with such confidence on it. So even just labeling it as like, hey, I have proof of this and I do not, this is my opinion, uh is probably useful.

SPEAKER_02I think it possibly even helps the LLM to have that in the context as the as the output. Right. So it knows how much to build on it. Yeah.

SPEAKER_04So let's sort of move this back into work. So you've you you've sort of learned, and I'm I imagine you sort of started bringing this into the company before all of these things were sort of resolved on it, but I don't know. But um as you as you brought it into the company, so how does this manifest uh in in bringing into the company? What did you have to set up and and what does it look like today when you run these agents uh inside uh the context of Sayara?

SPEAKER_02Running it at home. I ran it on a Nuke PC and I didn't even know it had a fan. But uh turns out it has a fan. Um and then bringing it into Sayra, at first I started running it locally, um, but I wanted it to run obviously 24-7. Um and I wanted it to have access to some things and be unable to access some things which maybe my laptop can access. Um again, my my German has to know like what I I have to know what's the possible failure mode and be okay with it.

SPEAKER_04Generally, like those are good tendencies when you work at a data security company. Right. So after you you set up the kind of the uh so you handle the sort of the ops part of it, you're gonna run this, now you have it in some sort of Sayera uh uh owned or sort of controlled um agendic environment. And then you mentioned that you have different agents that that can run on it. So that's already like a bit of a distinction. So before you kind of created this one agent to uh to be your personal assistant, you didn't need to probably like multiples of them. Um I guess what um I'm I'm curious a little bit, because I think a lot of companies talk about like the the the blank slate uh problem, right?

SPEAKER_04Or how do they get going? What are what what's a a type of agent that got created? We'll come back maybe a little bit to how you create agents on it. But maybe we'll just sort of talk a little bit about the use cases. So within the context of the company, how did you interact with uh with with yourself, I guess, and with sort of users? Because at the beginning it was like you know to help you read Slack messages. Um and what are what are the use cases that end up you know proving to be valuable?

SPEAKER_02Right. Um so the first one was a type of personal assistant or it's its work mode is chief of staff. Um and then I uh looked at making it more generally available, uh, which initially the obvious choice was to make it into a Slack bot that you can DM and tag in threads and uh an actual sort of a chat interface for it. Yeah, just you know, bring the bot to to the to where people walk already. Um and so the first use cases, the the easiest ones to build were an on-call triage agent where an alert pops up, there's a discussion about it, you can ask the agent, hey, can you cause this?

SPEAKER_02Um uh what's the impact, which tenants did the impact, stuff like that. Uh very operational RCA type of agent.

SPEAKER_04Um this is uh you you create the agent, or you know, someone creates an agent, but then multiple people use it. Like this is a deviation from what you had before. Before you were like there was a one-to-one, one user to this agent, uh now there are multiple uh users to this agent, but collaboratively or just like every every individual might sort of uh uh at the uh the uh the bot and and get a response?

SPEAKER_02So early on, really it was just every individual, but there were some uh emerging properties out of that. Well the agent can read the thread so it can tell when it can see the previous discussion, it can see people talking. about using the agent, uh people complaining about hallucinations. And then when you tag it, it sees the prompt, but it also sees the context. So you might respond to something else as well. Okay.

SPEAKER_04So it's Slack is like naturally a multi-person kind of a multiplayer environment on it. So it just sort of we got that for free.

SPEAKER_02So that was the interactive use case. And then another use case was something our CTO Tamar proposed. We have these channels on Slack where uh SEs, AEs and so on might ask questions of product and RD. And you can answering some of these questions can be done by looking at the knowledge base, the internal knowledge base, the public knowledge base, sometimes the code, sometimes historical questions.

SPEAKER_02And I think in many cases you just don't know exactly what you want to search for. Kind of like searching on Google. Yeah if you know the right keyword you're gonna find the answer but sometimes you first have to fight figure out what it is. So agents are quite good at that.

SPEAKER_04Yeah. But in this case like the the the way that people interact with it unlike the previous one is they don't add the bot they they just ask the question in the channel.

SPEAKER_02And then the bot would uh ask them using an ephemeral message which is the type of message that shows up on Slack when only you can see this. Like when you tag somebody in a channel and Slack bot says hey this person isn't in the channel should I add them? That's an ephemeral message. So the bot would ask you do you want me to try and answer this? And then if you click click, yes it's gonna answer publicly in the thread for everybody to see. And that's actually an adaptation because initially it would just answer and it would annoy some people.

SPEAKER_02So I had it just start asking I figure just treat it like a human right if you ask for permission and somebody wants you to, it's fine. They won't be upset.

SPEAKER_05Yeah yeah cool.

SPEAKER_04Okay so so uh we kind of scattered a little bit over here as I kind of asked different paths but you have the bot it's running in your Slack yeah uh uh you know it has one use case which is when you or one mode of operation I guess when you sort of interact with it uh and it just sort of answers questions and one that it's sort of a proactively I guess on a on a schedule or just monitoring a channel so it reviews every message?

SPEAKER_02So it's just it's just monitoring the channel. It's called ambient mode. And it has a classifier that uses a cheap model to decide is this a question, is this an announcement or something else? And only if it's a question does it actually offer to answer. And there are a bunch of other use cases that's where the multibot part comes into play. So initially there was just one Slack bot uh my own I I called it Borg uh for the Star Trek reference that's the you know the cyborg hive mind and the thing about the platform is all the would-be agents of the platform are gonna share the same knowledge and they're just all gonna be the Borg yeah yes they're gonna all be assimilated so the joke was too good plus I gave it uh I took Sayera's um if you go to website you'll see we have this flying uh space pod icon so I put a little cute face on it and that's Borg um so then uh the multi-bots right so uh I started out with just my own bot but the way that he would interact with his agents is by tagging them right so now um somebody else uh wants to create a bot the way for them to do that would be to create a new Slack app and give it a name and possibly a personality um and this was something I did for a functional reason I just wanted them to be able to customize the system prompt, the tooling the context um that the the platform is aimed at engineers, right?

SPEAKER_02So they would uh potentially rewrite the the code that sets up the agent when it gets a prompt and be able to do context engineering. But a really nice side effect of that was they ended up naming their agent and really feeling like it's their thing and it really helped adoption. Because it was no longer you know I'm adding a bit of code to Ori's agent. It's my agent. Which was really uh kind of my goal with this for this to be um a platform that everybody on Sierra can use to build agent and um they get the data infrastructure, the citations, um the knowledge graph which maybe we'll talk about in a in a sec.

SPEAKER_02But also the guardrails and to satisfy my my uh need for uh for those controls.

SPEAKER_04So so let's uh cool okay so at this point we've kind of advanced down the story of it now we have these bots running they're connected to Slack. They can do a variety of use cases which we you know we we touched a couple we probably should sort of cover some more but then uh they don't you don't just sort of lump on all the many use cases just to to Ori's uh Borg uh but rather now there might be you know there are multiple agents uh I love the I love that sort of uh human insight of it which is once you name it you care for it on its fun too like that they make stupid jokes while they're answering well and you can do because you can kind of add your personality but then if you if you care for it there's you know there's just like the the investment within that but it also lowers the friction because people allow themselves to modify because it's fine it's mine I'm not getting in anybody's way so on one hand you kind of get that foundation.

SPEAKER_04Maybe let's indeed go back a little bit and and understand um uh first like maybe let's talk a little bit more about some use cases and then we'll come back to understand how did that citation platform kind of manifest. We probably should talk about the knowledge graph there, but how did that sort of come back? But first let's come back a little bit to the to the use case in example. So like one I get it's internal knowledge based lookup I think everybody relates to that that's sort of a common use case. You start talking about some some ops issues like root cause analysis of it maybe say a few words about that and I know you have a few other ops use cases.

SPEAKER_02Right so Sarah is a complex system um and by by its very nature data security deals with a lot of data. Some of it takes quite a bit of time to scan um just because of the sheer scale um and so some operations can run for quite a while and that makes them hard to troubleshoot. I mean if something ends in 10 seconds you you look at the operation it's done. So these on-call or operational troubleshooting agents would have different triggers.

SPEAKER_02So some of them interactive um some of them are like the ambient answering questions agents but they would actually respond to alerts um or uh um tickets from customers being opened and do uh initial triage uh some of them would do triage in terms of uh assigning it to the right team based on what's in the in the ticket um and then there's an entirely different class of agents still all Slackbots um but we have say the um the costs agent which analyzes the the cloud costs for Sarah and then creates a report and root causes changes so it would see an increase in cost and try to attribute it to increase in workload or uh change in the code um and help you sort of uh keep it under control over time.

SPEAKER_02Kind of like spring cleaning type uh housekeeping uh agents yeah exactly um and then you also have agents which which help product managers uh for example by looking at um feature requests by customers or mentions on slack um for features by customers and sort of help the PM see that there's a trend forming uh way before it would have been detected otherwise because before you had to have um say an an SE or an AE say we need this feature and then it would need to people would need to realize it's actually the same thing.

SPEAKER_02LLMs are good at finding those similarities and and saying hey I think this this is actually similar pay attention here and here.

SPEAKER_05Yeah.

SPEAKER_04And this is like how many how many people have built agents? Because like these are uh uh this is sort of shared infrastructure but they're also very different agents that that people have built uh they share the same sort of uh I guess kind of patterns around interfaces and the slack integration on it um how like what is the result uh you mentioned developers are the ones that are building uh the agents but are there like five agents, 50 agents, 500 agents? Like what's the what's the rough order of magnitude?

SPEAKER_02It's around 30 ish agents um built by about the same number of developers. And the the number of users is I don't have an exact number but I imagine it's around several hundred because it's it's not just the developers in the company. Everybody used them some even indirectly.

SPEAKER_04Cool so I like I love the use cases and again they're sort of their they're uh they're very practical example of uh of of in general kind of you know building agents you know when people talk about sort of you know building kind of enterprise automation but I love that it's bringing bringing it into the heart into Slack. But all of this is substantially enabled by both the citation side and the tooling side uh maybe actually let's talk about the constraints first the sort of the tooling side uh piece so before we talked about how you kind of whitelist the tools versus blacklisting how does that work here like you know all of these things they need different tools do you share the tools between the agents do you define them like how does that work?

SPEAKER_02Alright so the the agents all have a shared baseline of tools and you can exclude some of them or you can add uh custom tools to you to your agent basically like skill like skills except they have a how do you say a schema that gets validated and you can run code and do deterministic uh checks. For example another example of an agent um is an agent that actually doesn't operate in Slack uh but it works on sentry so errors that show sentry is an error management platform and then um errors show up on sentry and we want to basically same thing as the Slack agents right we want to triage them figure out the root cause if it's a bug fix it if it's an environmental issue.

SPEAKER_04So these are just basically settings as a part of the configuration of the agent which I guess you can think of as a configuration of the harness you know because this is the sort of the environment that you're in the it is the harness yeah yeah it is the harness kind of with the tools. Okay so that's fairly straightforward and I guess how how often do people like before you talked about kind of your mechanisms for knowing that you need another tool because that's probably the the piece that is is uh you know like you the the dash dash dangerously accept uh sort of the dash dash yellow is uh is also convenient which is like I don't know what it's going to need you know can you just just let it do its thing and uh and trust it so I guess what's the what's the loop here you know what's the the interaction model and how often does it happen that people you know like it fails because it doesn't have a tool and they come back to it.

SPEAKER_04Does it does it typically sort of stabilize like they do that a few times and then generally it kind of works or you know what's the what's the experience there?

SPEAKER_02I think it's actually quite rare um already from the af after the first few agents were were spun up. Because we we kind of plateau out around uh 60 ish tools that most agents have and that basically um because the tools are shared between the different agents. Yeah so somebody adds a tool all the other agents get it get it automatically and then it just becomes possible so the way that we discover that a new tool is tool is needed so the obvious choice is somebody would tell me uh that's quite bad because in in many cases people wouldn't tell you or wouldn't even know to tell you um and then there are a few other mechanisms so uh the citations they show up in line in the text um whether the interface is on Slack or on other platform and then what also shows up is the feedback mechanism.

SPEAKER_02So you get to thumbs up, thumbs down and if you thumbs down you can write why um so that's the the obvious one it goes and and when you when you specify feedback that gets linked up against the original data that backed this claim and not the the specific answer. So already the agent is learning just based on if it queries the same data it's gonna see that in the context. So you have that and then if somebody is talking in the conversation after the the agent has responded and he's like man I just wish it could you know give me a an update every 15 minutes.

SPEAKER_02So that that gets recorded right it's just on Slack. And then one one other mechanism that the platform has is a schedule mechanism. So the agents can uh trigger have a trigger every schedule and the first one I added as an example really was the dream state which is to which is where the agent spins up every week and goes and reviews all of its interactions, feedback, um and also the the responses people wrote just around the answer and they might say something like that.

SPEAKER_02And then the dream state is gonna go and submit a pull request to improve the agent. I like to joke that this part of the agent is the part that replaces me as it sort of chooses to sort of evolve it.

SPEAKER_04So this is the loop like this is the part in which it it sort of optimizes it. And in those contexts the the agent might come back and say I need a new a new tool or of course it might sort of change its own more thing I forgot.

SPEAKER_02There's also a tool called submit feedback which is for the agent to submit feedback. So if it's trying to do something and it so it's looking at the tool list right and it it realts on it and I don't know where those exist on it I need I need help. And then that gets saved and when the Dream state runs it's gonna see that in the context of everything and possibly propose that change. Got it.

SPEAKER_04Okay. And this is uh it's interesting that the tools are shared between the agents I guess I I expect at some point in the future you might find yourself sort of isolating and kind of further constraining uh the agents but just hasn't come up because you're already sufficiently contained.

SPEAKER_02So there is there is a bit of um divergence between them but it mostly has to do with functionality because not all of the agents are are only Slack bots at this point. There are other interfaces like the sentry one I mentioned.

SPEAKER_03Yeah.

SPEAKER_02And so so sending a Slack message is a tool right and and going on about about control and and and being and the agents being constrained the agent isn't able to select where it's sending a message to but it gets triggered in a particular context and then it can call send message but it only sends it to the same thread that is unlocked for it. It doesn't it can't choose the destination so it can't make mistakes you know provided that the harness only allows it to be triggered in the right context.

SPEAKER_02But that's deterministic we know how to deal with that most of the time.

SPEAKER_04I think uh that very much is aligned with kind of my view right I think when you have the agentic stack today you have the models that are kind of our our operating systems we're building on and then on top of that you have tools and context and harness and the the context is kind of uh uh sandwiched you know between the sort of the tools and the uh and the harness with the harness creating constraints uh and so that drives some determinism and some sort of security but then also the tools drive another uh another set of uh constraints and determinism um so I like that you're sort of building that and I love also like and as we had conversations about this it also sounds like you didn't set out to do this but in practice this also saves you substantial cost because you're talking about like a fairly notable amount of sort of use here you know it's scanning a whole bunch of data it's doing all that work and I guess have have costs stayed contained like it hasn't really expanded?

SPEAKER_02Pretty much yeah the the actual usage is quite cheap relative to what I would expect. And I think it is indeed because it's it's very structured right the constraints keep it sort of on on the rails. Yeah um this setting up the the the knowledge graph uh which which we'll get into was quite expensive but it was a one time cost.

SPEAKER_04So let's so let's go there. We've sort of we've now sort of teased it up enough uh on it so okay and this also like relates I think to the citations infrastructure right so maybe talk a little bit about the knowledge graph and maybe we can kind of slip into citations. I know you have that sort of reviewer aspect of it so how does that work in sort of the setup?

SPEAKER_02Yeah. So the the knowledge graph is really so okay I didn't mention this but uh quite early on when I had the idea of uh maybe I want citations, maybe I want I want all my data to be backed by a source reference I went and read I am using you know quotes here yeah quotes because AI version of red I I did assisted reading using Chat GPT of a bunch of uh white papers and all of these concepts data provenance claim verification they've been explored uh quite thoroughly um over the years and especially in the in the last year um and it just made sense to put them together in in this way you know having all of your data backed by a source and then using that to do claim verification and hooking up feedback into that it all felt natural it didn't come to me initially it came gradually um so the knowledge graph is an extension of that um I was looking so the the obvious thing most coding agents or interactive agents you're gonna give them uh access to data maybe to an LLM wiki you know markdown uh files and they're gonna go and explore.

SPEAKER_02Now as a user I know that sometimes they will not explore all the things that they should be exploring and then you're kinda down to um will they find a hint or something that pushes them in the right direction. And Andrei Karpati wrote about this that having those links in the LLM wiki helps them uh find um find the the data they're looking for.

SPEAKER_03Right.

SPEAKER_02But it's really rag in in the sense that using using a rag to build context and just dumping that into the agent right off is no longer necessary because agents are better at exploring nowadays. Agentic search versus rag is a yeah so and and I I see that but again going back to my need for uh for control over agents I I f I didn't like that sometimes the results would be inconsistent. And so my way to solve that was okay let's take all the data put it into a knowledge graph a database really um with interconnections between the entities almost a rag but then don't actually use it like a rag let the agent walk the connections like it would an LLM wiki on disk just you can have many many more pieces of data because now you it's it's indexed and the search is fast and finding uh relations is fast.

SPEAKER_04This is just to sort of clarify a little bit about what type of data might be in there. Like you know this is like in the organizational context.

SPEAKER_02Right so it's anything from our code, um Slack messages, uh a particular exception popping up basically anything um but um setting this up has quite a bit of cost because extracting the entities and all that that goes through LLMs. And then probably the most important piece was working on data quality which is still very manual.

SPEAKER_02Because if I just took all of Sarah's data um the relevant data at least obviously not all the data and just stuck it in the knowledge graph I think there would be mostly noise I don't know maybe bots on Slack sending alerts repeatedly probably dwarfs the amount of messages that humans send which is the signal. So figuring out the different data sources that we want our agents to have access to how to model that in a graph. So an account, an exception, a service those are entities that that need to be in the graph.

SPEAKER_02And then as the data is ingested seeing where uh where the noise is and actually going and cleaning that up. So it took me well manually we installed yeah yeah but yeah it took me a few weeks to to finish ingesting because it was was really bottlenecked on my own ability to look at data. I would of course look at the the noisiest pieces first right you group by the source and the uh the origin and uh and then and then clean that up um but now we have about a million or so pieces of data in the graph as opposed to many many millions that would have been there otherwise and the agents find surprising things which is a good sign I think as to the the amount of signal in there.

SPEAKER_04So this is a uh so you you gather data from a bunch of places and I imagine like once that data is gathered and you know you kind of cleaned up the the sources, those are not repeatable, right? You're you're now kind of keeping it up to date with uh with new data as it comes in of that shape, I guess, you know, kind of uh from from the relevant Slack channels and the other you gather that up. And and then I guess what are the tools that the agent uses to consult that this is like a data lake, right? This is not kind of a context lake, a context graph you know you call it the knowledge graph. You can use whatever word And you say that the agent is able to kind of mine that data in more uh efficient ways doing it.

SPEAKER_04What what tools are make that possible?

SPEAKER_02So there are a few generic tools. Um an exact search. So if it's looking for a particular service, for example, uh it would use an exact search and find entities that have that um that value in them. So that maybe backing up uh a little bit. So the knowledge graph has episodes, which is things that happened and got ingested. Um and then it has entities and relations between those entities. Um so if it's looking, say, at a particular um exception or a slack message, um in that slack message and exception, we you have the entities that participate.

SPEAKER_02So in a system prompt, which is shared to all the agents. So there's a piece of the system prompt, a block of the system prompt that's shared to all the agents and agents and explains kind of how to walk the graph and and what a SIR agent should behave like.

SPEAKER_05Yeah.

SPEAKER_02And then you have another piece that's per agent. So that shared piece uh explains you should first use find exact if it makes sense. If not, use the semantic search. Once you find a relevant entity, go ahead and walk the graph using find related. So that just finds the it would it's kind of like listing the directory and looking at uh interconnected entities. So in in Carpaty's LLM wiki, those would be the dimensions within the file. Um and then once it does that, it has a bunch of specialized tools per data source, right?

SPEAKER_02So it might find an episode uh of a Slack message, and then it has a tool to read that Slack messages thread, um, or look up more inform more messages by that person, and and so on. Um so those are the bulk of the tools, right? There are just there's probably five or six tools per data source, and then there's the generic tools. Um and those are all read-only, and there are a few write tools that uh that involve with actually solving problems, like performing a code review, uh, posting a comment somewhere, um, submitting a pull request.

SPEAKER_02Cool. Good.

SPEAKER_04So the so the context graph is uh uh is a great tool to basically spare a bunch of this work during uh during the uh the agentic run. Uh and I love that again, you sort of you you were driven to create it because you wanted uh determinism or you know like consistent success when it should have the answer uh that that path on it. But it probably is a pretty big kind of time and money saver as well, because now it doesn't need to you know kind of blumber its way, you know, sort of through uh uh through the data uh and it doesn't read unnecessary data uh or it doesn't repeat that, which I guess kind of comes back to you could just ask for structured data or sort of uh cleaning it up.

SPEAKER_02Yeah, the goal was context engineering so that the setting it up so that the right data is is in the context. Um and I wanted to, for example, if if we go back to the very first use case, it was finding a piece of data and finding what's related to it. So if somebody is talking about an exception, I want to see the pull request that introduced it, I want to see the uh the exception, the logs, the the impact it has. And for the agent to reliably do that, in my experience, it has to have a way to find those relations.

SPEAKER_05Right.

SPEAKER_02Um and they're there. Like if I as a human can go and look at um at a log line and and something like that and and find them, then all I have to do is build a tool that sets it up. So when it calls find related, those things just pop up.

SPEAKER_04Yeah, I mean I think it's basically the conversation about structured versus unstructured data. And uh and I think what we're sort of seeing is agents are are so powerful that you can give them highly unstructured environments. You can give them bash and give them files, and they will get a lot done uh on it. But when you come to doing things at scale, reviewing large volumes of data uh and you know, performing actions repeatedly, and you want to improve reliability, I I I think there are there are multiple paths you can take.

SPEAKER_04You know, there's the path of um you know better instructing the agents to eventually figure it out uh in in sort of what uh I mean I I guess you you can make significant uh progress, but it is uh it's you're just playing with kind of make smarter individuals, right? You know, in the uh elements, but you're not actually sort of equivalent them. Uh there's you know another path you have around sort of giving them uh um making the data more structured, so uh making it easier for them to sort of look up the right information at the right place.

SPEAKER_04And so uh that is uh that is helpful, of course, you kind of adapt the context, and then you can take it further and you can say, okay, now that I have that structured data, I can actually offload some of the cognitive effort into tools uh and uh and run that, which makes a lot of sense because it's basically how we operate with with humans.

SPEAKER_02Um Yeah, I actually really think about LLMs as humans. I think that you're just it's not it's not it's really tempting to think of them as computers, but really the only other thing in the world which we know that you know if if they have the right training and they have the right information, they generally do a pretty good job are humans, but they're not infallible, and neither are LLMs, right? So you're you're really helping them. And it's so having these constraints and tools, that's kind of how I work with myself as an engineer, right?

SPEAKER_02I I try to set up my development environment so that the compiler yells at me when I do something wrong. I I try to set myself up so that my own mistakes are caught before they actually have an impact. So I do that for my friend Bog too. Yeah.

SPEAKER_04I think uh I think that that is a very effective way. I think about that as well uh fair bit. And there are somewhere in between humans and sort of systems, you know. Sometimes you use a system analogy and sometimes you use Eventually they're just they're gonna be systems. So so cool. So I just want to take stock a little bit, right? Like at this point, we talk about how you've gone from uh choosing the uh citations-based and sort of tools whitelisting approach to create the agent. You created uh this sort of shared repository with infrastructure for everybody to create an agent, uh, and then you started by hosting yours and giving Slack interactions uh to it with both the kind of explicit and ambient mode um of interaction, which is pretty cool.

SPEAKER_04Basically, you know, who's who's um who's in the driver's seat, right? Like who initiates the conversation uh changes over there. Um and then people started uh uh kind of evolving and uh and building uh their own agents. I really, really like that sort of hack of like name it and it's yours uh and uh build those out uh and uh and expand them. Uh and then we talked a little bit about the sort of uh efficiency uh component, you know, to improve both accuracy, enable citations, and uh um reduce costs in the process of that in the context graph.

SPEAKER_04Um so you know, a ton, a ton of evolution on it. Uh I think uh we're we're sort of running out of time here a little bit, but maybe just a a couple of uh a couple of things that you know when we were talking about this, I found interesting. One is um when you when you teed this up, like in an organization, you're you know, you're this you know, engineering leader who's you know building, being an IC here, building a bunch of things, promoting it out. But when you permeate it through the organizations, you worked with this AI captains uh group on it.

SPEAKER_04So because oftentimes again like affecting cultural change is uh is a recurring uh need of it. So tell us a little bit about that group and how does it work?

SPEAKER_02Alright, so to get started, um we we created uh Tomir and I created a small group called the AI Captains um and we basically uh picked a few people who we already knew were more uh were interested in in building agents and and stuff like that. And a little bit before before that point, we're talking like December or something like that, um I built my first agent because I tried to accomplish a task.

SPEAKER_02It was classifying uh the amount of time it takes us to fit to fix a bug from uh the the pull request that introduced it to the pull request that fixed it. Um and I tried doing it with a coding agent, but it had some success, but it was very inconsistent, and and at the scale that we are getting meaningful statistics out of something so inconsistent was really hard for me. So I figured okay, let's try and uh build an agent and and give it hints to keep it on the right path.

SPEAKER_02Um and sometimes you would see that you would see that the system hints leak into your context when you're working with a coding agent, you would have the agent suddenly react to a system hint. It would say something like, Got it, the command response is this way, so don't tell the user. And and it excellently tells you that. Um and I saw what a massive difference it made to actually constrain the agent and give it just the right tools it needs to do the job. And of course it takes a few iterations, right? It's not a one-hit wonder, but the consistency and the quality actually improved.

SPEAKER_02So I wanted um, I felt like the best way to get other people to build agents is to give them that sort of Eurica moment as well. So like they would be like, Holy shit, I need to be writing more agents. That's what I felt. Um and so we we formed that group, if it was just 10, 12 people initially. Um and then we did like an agent workshop. Um and it was all really simple, and uh, and then with the uh with this agents platform, uh they would each spin up their own agent.

SPEAKER_02And spinning up an agent to get a carbon copy of of Borg is really just it's copy-paste, right? It's a configuration file. Uh, but once you do that, you're really seeing kind of the the power of it, and it it lowers the barrier from there's this unknown thing that's building agents to oh, there's a Python class I can modify. Yeah, lower the agent.

SPEAKER_04Easy enough. So you share them, you want to you want them to sort of see the light, and then you kind of make it easy for them to get in the game, which I guess comes back to that fork the agent, name the agent, don't change or his agent.

SPEAKER_02So the blast rate is just lower the barrier of entry, um and make it easy to solve all the operational problems, which are really not that interesting, whether you run your agent, uh, what are the guardrails like, like having all that figured out, which are hard questions to answer, but not interesting from the application standpoint.

SPEAKER_04I thought that was like an effective, sort of simple, repeatable, but sort of effective way to uh to promote things.

SPEAKER_02I think it worked really well. And and itself, it it became self-sufficient too. Like once those first people um really adopted it, um some of them became massive champions of it within the organization. And I I reinforced that. So uh I would tell people you don't need my code review, you can ask those people. Um and there are the guardrails, they do require uh my code review, but I added some of them to uh be the code owners for that as well. The guardrails are really how we're keeping Claude or other coding agents from you know really going too crazy and removing uh removing some of the safeties.

SPEAKER_02Um so it would really reinforce that. And kind of my approach has been all carrots and and no stick, just make it really easy, um, make it safe. Really making it safe is making it making it really easy. It's another way of doing that. Yeah, avoid uh concerns about blast radius and sort of concerns. So you just didn't know. Exactly. Make it so people don't have to worry and reinforce it when somebody, you know, somebody is effective and it he takes the the reins, be like, good, yeah, keep going and point people at um yeah, sort of celebrate the successes.

SPEAKER_04Yeah. And I think the the the last use case I wanted to uh to just sort of like uh get from you as we as we uh as as we sort of maybe like wrap up a little bit after that is uh is is how you kind of dealt with secret data. So like you're in the world of data security, uh a lot of that data is quite sensitive. And so on one hand, like as a company, you need to deal with the fact that you know you have you need to secure and sometimes troubleshoot problems when things don't work, on uh data that is uh that you're not supposed to be able to see, or that maybe like there's uh you know specific individuals that have been under ND8 or whatever it is have the right kind of enough clearance to be able to sort of see that data.

SPEAKER_04So so I I I love you you've also used agent a little bit to like export your intelligence in sort of importing the data on it. So maybe tell us a little bit about that use case, which found like a a good uh novel use for uh for agents.

SPEAKER_02Yeah. That's one of the use cases I think that actually we weren't really able to think about before agents were were a thing that everybody in Sierra are thinking in terms of. Before you start building affinity to like what is this thing, right? Yeah, because okay, I'll I'll talk about it then it'll be obvious. In some cases, you could have, for example, the the on-called triage agents, they have access to to all the data, and you have access to the same data, and it's easy to see. Um, and it's easy for them to troubleshoot and it's easy for you to see that it's correct.

SPEAKER_02Um, but in in other cases, you may be working in in an environment where um you're not allowed to see the data or even access the data um for regulatory reasons or because of uh customer commitment. But obviously there are still bugs. So you do need uh some way to deal with that, and up until we had agents, really that the only way to deal with that is to get a particularly small group and pre-approved group of people who can see the data and they can then go and troubleshoot it.

SPEAKER_02But um as the company as company as as a company like Sierra grows larger and larger, I mean we we grew quite quickly. I think a year and a half ago we were massively smaller. Um you may need more people than that small group, right? And how do you do that then? So we came up with this idea uh of having an agent that's within the controlled environment and has access to the data, and you can have it investigate um the issue and troubleshoot it and and it has access to everything, right?

SPEAKER_02But one big constraint, it cannot output um except for structured validated output. So uh this troubleshooting agent has access to everything, but then it can only output for a fixed vocabulary, which you can verify doesn't leak any data.

SPEAKER_03Yeah.

SPEAKER_02Um and then so it does its investigation, outputs in the fixed vocabulary, and then that goes that can actually go out of that perimeter uh to other people who are not approved because now you know nothing sensitive was was sent out. And then we have another agent which reads that data um and has access to the code and all of the uh wider availability data and can put together an explainer for humans backed on that other agent's conclusions.

SPEAKER_02Um so those two agents, they're called Mulder and Scully, uh, because they're they they worked on the X files.

SPEAKER_00Hey everyone, hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development. Whether that's talking about the latest tools, the most efficient workflows, or defining best practices. But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button. It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.

SPEAKER_04Alright, back to the episode. This is basically a way of uh having kind of two agents talk uh uh across a wall, you know, when when you sort of can't see the data. And I guess the result of that is you can troubleshoot issues uh a lot more efficiently uh with because expertise varies like with the people around it.

SPEAKER_02You can have somebody on a on an RD team that doesn't have access to the data program the agent with all of the knowledge of that team and how to troubleshoot and examples and and then the agent goes and and does that in in a controlled environment and the feedback um is is fixed and safe. And then going back to that escape hatch uh thinking, uh the controlled agent can still output to the pre-authorized small list of people. If it does need to say something that it can't say using the vocabulary, it goes and outputs that free text uh to that group, and then we can go and expand the vocabulary after human review by the approved people.

SPEAKER_05So it all scales comes together.

SPEAKER_04Yeah, yeah, scale those people. So I thought that was very cool, and I uh I feel you know not just about confidential data, but also when you think about on-prem support, when you think about you know all these paths, I think that's very compelling.

SPEAKER_02Many use cases where you can't you just can't have the data go out, but you still need to operate.

SPEAKER_04So Ori, thanks for sharing all this. Like I love the journey on it. I mean, sort of we told it, I felt like the sort of the the destination and all those insights are super useful. But I think the way to achieve like the sort of the journey to sort of iterate and how how did you learn and what did you add uh later, uh, I think is is you know just as uh educational for you know a lot of organizations that are going through that process. So thanks for sharing. And I guess just before we wrap up, I'll I'll ask you what are you excited to do next uh for for this whole journey.

SPEAKER_02So I think right now that the most exciting thing is uh working on our uh code harness, coding harness that gives us all of that infrastructure. But honestly, I feel like if you ask me two weeks from now, it might be something else because that there are a lot of ideas that we couldn't have had before, like this this confidential information agent. I don't think I could have thought of that uh a few months ago before I had internalized thinking in terms of agents. Um so I'm really just excited about that.

SPEAKER_02It's sort of a meta excitement, it's not a particular thing I I want to build, but I'm I'm excited to to be able to execute on them so quickly, right? Because now building an agent once you have all this set up is is really easy. Getting it access to the right data, like this confidential agent sounds like possibly a heavy lift with lots of moving parts. Two days. Right. That's nuts. It's it's not just LLMs, but having all this set up and also honestly, that we're a data security company, um and like you know what data you have and and where it is, and you you can let the agents access it and know they're not gonna go wild.

SPEAKER_02Yeah, I think that's really really important. And if I can just give one tip to uh to the audience. Um since having AI adoption is really about adoption, it's about making people uh adopt the this new mode of thinking and building agents. Um so I would recommend trying to do it with a platform engineering approach. That's why I went all carrots. Um, because you're really trying to get people to adopt a new type of agency, huh?

SPEAKER_02Uh for themselves, and you can't tell them to do that, they have to internalize it. So you have to lower the bar and make it look enticing and make it theirs. Yeah. Um and I would love to say I thought about all that before, but it's kind of in hindsight.

SPEAKER_04I think it makes a lot of sense, and I think uh I think we've learned a lot from you know the DevOps transformation and the value of the paved roads, and you know, you you enable that. And I think the other advantage I'll sort of highlight is that it it it helps people along on the journey because you've paved the road and it's easy for them, but it also still allows them to sort of bushwhack their way kind of along the along the side and maybe find shortcuts. And so most people will just opt for the paved road, uh, and then the few that goes to the side, some will regret and come back, and then some will will find a new path, and so there's an opportunity for learning.

SPEAKER_04Yeah, because you can learn from that and actually. If you just sort of centralize a new constrain as opposed to like you just work with the sticks, then the result of that is is you're saying, well, it's only these sort of smart people in the platform team. They're the only ones who can innovate, uh, and the rest just sort of apply it. And so enabling a paved road but allowing someone to go to the side uh is uh is a powerful path.

SPEAKER_02I think it's it's even it's easier to have that mode of thinking as as the platform team for this because agents are new to everybody.

SPEAKER_03Yeah.

SPEAKER_02Right. So uh it didn't require any mental gymnastics on my part to assume that there are things I didn't think about.

SPEAKER_04Cool. Well, Ori, thanks a lot for uh coming on the show. Love what you've been building. Looking forward to uh explore the the future. Uh and thanks everybody for tuning in, and I hope you join us for the next one.

SPEAKER_00The AI Native Dev is brought to you by Tesla, the package manager for Skills and Content. Your hosts are Guy Pajani and me, Simon Maple. Our producer is Tom Dowler. The AI Native Dev is not just a podcast, it's a community. And we host monthly meetups at the Tesla offices in central London. Visit Tesla.io forward slash community to learn more. I hope to see you there.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Your Software Factory Needs a Bill of Materials | Daniel Meppiel

    Nov 19, 20261 hr 1 minAudio

    Developers are about to write skills instead of code, and Microsoft's Daniel Meppiel argues the software factory that runs them needs versioning, a bill of materials and signed receipts before enterprises will trust it....

    Credit the people on this episode

  2. Paul Stack: I'll Never Write Infrastructure as Code Again

    Nov 17, 20261 hr 2 minAudio

    Paul Stack helped build Terraform. Now he says he'll never write another line of infrastructure as code, because an agent harness runs his team's production infrastructure instead. Watch all the AI DevCon NYC 2026...

    Credit the people on this episode

  3. Edith Harbaugh: Code Is Cheap. The Last Mile Isn't.

    Nov 10, 202649 minAudio

    Automated software delivery is moving faster than the people who used to approve every release. LaunchDarkly co-founder and CEO Edith Harbaugh tells Guy Podjarny why rollback should be autonomous, why one-shot coding...

    Credit the people on this episode

  4. Maggie Appleton: Agentic Coding Has Outgrown the Text Box

    Nov 3, 202652 minAudio

    Agentic coding has outgrown the text box. Maggie Appleton, staff research engineer at GitHub Next, argues that walls of CLI output and AGENTS.md files serve coding agents far better than the humans steering them. She...

    Credit the people on this episode

  5. Caleb Sima: AI Attackers Move at Machine Speed

    Oct 27, 202655 minAudio

    AI attackers now move at machine speed, while most security teams still move at meeting speed. Former Robinhood CSO Caleb Sima has seen an AI attacker up close, and he argues the only way to keep up is to take humans...

    Credit the people on this episode