Episode 402 – Declarative Agents with Andrew Connell
34 minAudio
Welcome to Episode 402 of the Microsoft Cloud IT Pro Podcast. In today’s episode, we’re thrilled to have Andrew Connell (AC) joining us once again. If you’ve been with us for a while, you’ll recognize AC as a repeat guest and one of the early voices who encouraged us to launch this podcast. In this episode, we’re diving deep into the world of Microsoft Copilot agents from a bit of a developer’s perspective. We expand on insights from Episode 400 and helping us understand some additional intricacies of how Copilot works under the hood. We’ll break down the different billing models and some limitations in what they provide and why having at least one $30 license is key to unlocking the semantic index. We’ll also explore how Copilot uses semantic indexing to ground responses in your organization’s data, and how it respects user permissions through access control lists. But it’s not all smooth sailing - AC also walks us through some of the current limitations, like missing metadata and large file indexing. He also shares a potential workaround using Azure AI Search for those needing more comprehensive coverage. Finally, we this weeks Microsoft Build Conference and tease a potential future episode where ACw will return to talk about custom Copilot development. Andrew Connell LinkedIn YouTube BlueSky Newsletter: The Full Stack Dev’s Microsoft 365 Playbook Podcast: Code. Deploy. Go Live. - codedeploygo.live Code. Deploy. Go Live on YouTube Show Notes Episode 400 - Microsoft 365 Copilot Declarative Agents and Copilot Studio Agent Builder Create Copilot agents in Microsoft 365 Business Chat and Teams Grounding an agent to specific public websites Semantic indexing for Microsoft 365 Copilot Data, Privacy, and Security for Microsoft 365 Copilot About the sponsors Would you like to become the irreplaceable Microsoft 365 resource** for your organization? Let us know!
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Welcome to episode 402 of the Microsoft Cloud IT Pro podcast recorded live 05/16/2025. This is a show about Microsoft three sixty five and Azure from the perspective of IT pros and end users, where we discuss a topic or recent news and how it relates to you. This week, Ben and Scott have Andrew Connell or AC back on the show to loop back into some topics from episode 400 around custom agents, but from a bit of a developer perspective.
They discuss some differences besides price in the various billing models and limitations around creation of the semantic index for Copilot. They also discuss how Copilot uses the semantic index and access control lists for security when retrieving your content. Another topic of conversation is potential issues you may encounter around large files, ASPX pages, and missing metadata when it comes to Copilot responses, as well as potential workarounds by using Azure AI search.
Finally, they wrap up the show by talking about some topics you may want to pay attention to from this week's Microsoft Build Conference. Let's dive into the show. For all of you listening, it's not just Scott Knight today. There's a new I would say a new voice. It's kind of an old voice. I don't know how many times Wow. Just digging that lightning. Out swinging today. It's a Friday morning. Said, we have a popular repeat guest. Andrew Connell's back on the show. But then, no,
you just went to the end of it. Just said it all yeah. I shoulda had you kick it off then, Scott. Now I'm all we're we're all off today. This is what happens on a Friday morning when it's been a long week. But, yeah, we do have Andrew Connell back on the show to talk more about CoPilot agents. So I don't know how many times we've had you on now, Casey. It's probably like four or five. Has it been that many? I know it's, like, one or two. I didn't know I didn't know if it's been I don't know. But we're go hey. It's a long time listener. Let's just say second or third or fourth time caller. Yeah. Exactly. A popular that popular caller on our show
that just keeps coming back. But, no, we did, like, episode I think it was was it 3 no. It was 400 was the Copilot agent once. And you were listening to it, and you sent us a message and said, hey, Ben. Let's talk about some more stuff around Copilot agents that you've come to discover. And I would say even from a developer's perspective, like, Scott and I are very IT pro, which everybody knows, and you are very developer. But I think that also gives you a new perspective sometimes on how some of the stuff works under the covers. You uncover some stuff. You're like,
Ben and Scott, let's talk about some other stuff that relates to that. So we're gonna take a little bit of a journey adding some color, some additional information to what we discussed on episode 400. So if you haven't listened to that one, you may even wanna listen to that one first. And then even maybe dive into some of the development stuff with Copilot agents and how you can expand Copilot agents even more from if you have access to a developer, if you are a developer listening to the show. Yeah. That sounds great. But yeah. You guys, first of all, like, congratulations
on 400 episodes doing this for eight years. That's a huge accomplishment. It is I've done eight I've done a podcast for that long and that many episodes, and I will I know how big how much effort it is and how how much, like, staying power you've done with it. So huge kudos to you guys for doing this. I remember when you started this, and it's like it's really I I was listening to this on your episode 400 on the way to the airport, and I, like, did a note to myself. I'm like, oh my god. Gotta congratulate these guys. That's freaking awesome. I didn't realize they were at 400. And then I listened to the rest of the episode, and I was like, I'm gonna get to the airport, and I gotta send a loom
over to to to you guys and be like, I got some stuff I wanna talk about. Yeah. Well, thanks. And thanks for encouraging us to start it because I was gonna say you're one of the catalysts. So Yeah. There you go. You, like, propelled us into doing this. You're like, you really should do it. And then you got me to go chase down Scott and convince Scott to do it with me. So thank you. And CJ as well. You guys were both an inspiration to starting it episode one eight years ago. I missed doing it, and I'm looking forward to starting another one by the time this goes live. I think I might have it might
be live by then, so we'll see. Oh, alright. Keep our eyes out for us. Yeah. Put a link. We'll throw it in the show notes. Alright. Sounds good. Yeah. Where do you wanna start? Some of the stuff that we talked about. I don't know. There were there's kind of a couple things. I don't know if we wanna start with some of the stuff around deploying it, different models for pricing it, or if you wanna start with some of the other topics and what's going on in the background. Where if you were gonna explain this to somebody, where would you start that journey? Doctor. Let me build off what you guys said in episode 400 because there was a few things that I thought were not that you missed anything, but there
was a few extra things that I've seen that when I talk to a customer that it's a it's like a you should know this before you go into things. So one of them was you guys were talking a little bit about the billing, and there was, like, the difference between $30 per user per month, or do I spend $20.200 for 25,000 messages? Doctor. Yep. Doctor. And or do I go with the consumption model and pay a penny per message? Of course, all USD pricing. Sorry. All my numbers are USD pricing. I'm sure you could buy this outside of outside of The US. Doctor. Yeah. Doctor. There's one thing that I see when people do that that is a huge blind spot that a
lot of people don't realize. And if they go with the consumption model first, they will have this copilot sucks type attitude to it because it's like, I thought it was supposed to be grounded in all my data. So there's there's two aspects to it. So first of all, one thing to understand, and I think you guys talked a little bit about this, is that when Microsoft says that it is grounded in your data and they always point to Microsoft Graph, that is kind of true. I mean, it's not incorrect. But to be a little bit more technical about it is, like in SharePoint, you put content in SharePoint, and then for search to
work in SharePoint, there's an indexer that creates an index so so that when you do execute a keyword search, it doesn't look at the content in SharePoint. It looks at the index, finds the results, and it gives you gives you the results and it's links to the content in SharePoint. Yep. So that's how Copilot is grounding your information. So it's not it's doing a similar model. What it does is it gets data from Microsoft Graph about you and your organization. So that's data from SharePoint, data from chat messages, Teams transcripts, meeting transcripts, all that kind of stuff. And what it does is
it does another kind of indexing called a semantic index, which is not keyword index, but a semantic index is more like let's just say, AI based index that is a little bit more these words are closer to these words, and it's just better for, like, AI based queries. What Microsoft is doing is that when they look at that content in graph, when they do a semantic index of it, they are storing that in what's called the semantic index. So, that kind of sits on top of Graph so that when Microsoft when you run a prompt against Copilot, Copilot is looking up content from the semantic
index to get content that's relevant to you and your organization to ground the response in your data and not just like the foundational knowledge that the model has that like ChatGPT can leverage from, like, the the the training from the different models. Okay. So Office three sixty five search uses search index. Copilot uses semantic index. You've got these two indexes. That's correct. Yeah. And and and it's important to, like, call out, like, semantic index is technically generated with information from the graph. So you're, like, layers of layers in here. And
and I think it gets a little confusing to customers because we maybe throw out terms like AI training, AI fine tuning, AI inferencing. Like, the reality is most folks aren't out there doing, like, AI training typically in the terms you're hearing maybe, like, I don't know, me, like, Band Aid about. Like, there there's only so many open AIs and anthropics and and things out there that are training these foundational models. So more and more, I think what I see in in my day to day is kind of folks who are leveraging existing models, and they're effectively
into some type of augmentation, be it rag that's out there, be it leveraging systems like m three sixty five Copilot that then build on graph, build on semantic index, but then can also do rag and things like that for further grounding, like contextually in the moment versus, oh, hey. I was able to fine tune this model or push out this specific piece of organizational knowledge and and kind of do a little mini retraining run along the way. So it's all these layers upon layers in there. And it's funny to
me how it all comes back to, like, the old days of SharePoint and, like, information architecture. Like, how did you organize this stuff? How did you permission it? How how did that look? And then you and then we're back to the same thing of, like, now it's not just figuring out, like, hey. Why did that search result surface in my search index because the permissions were wrong on that SharePoint site or that list or whatever it was. But now it's, oh my gosh. Where else was that thing potentially off the rails? Oh, over here in Teams, because it was actually in a Teams message, but it was in a group Teams message that I had going with some other folks that were out
there. And because The Graph is able to surface that information, all of a sudden, you end up in this in this little bit of a a weird spot. And then you combine that with the fuzziness of I I don't know if you ever do this. I I go into teams all the time just or Copilot the app or Copilot in teams, and I say, hey. Search my messages for the last two weeks or the last six months and summarize blah blah blah that I discussed with, you you know, these people. Well, you can tell Copilot that it doesn't actually know how to execute a time based search based on
your messages. So, like, what it pulls back could be completely random, could be outside the boundaries of that. It was all just what was available to it in in its grounding knowledge. Doctor. Right. Right. And I like, the way I like to the way I explain this to customers is that, like, think about if you are doing your taxes for your business, right? If you sit down, you have a question for an accountant, that the accountant is somebody you just you just hired, they have all of the knowledge they've learned from being trained as an accountant when they were in college, right, and just factual stuff on how QuickBooks works. QuickBooks works. They're gonna give you an answer
to whatever your problem or whatever your question is. That's like using ChatGPT. What's using Microsoft three sixty five Copilot is he has that information or she has that information, but they also have all of your previous tax returns, and there are have access to your QuickBooks. So now they have context about you that they can add to the actual question. That's what the semantic index is. It's context about you and your organization. Technically, there's two semantic indexes. There's one for users and there's one for organizational data. So, but Copilot figures out which one it's gonna go through and grab. So, there's a few
things about that though that the semantic index has some interesting aspects to it that people need to understand I think that really helps in understanding why Copilot can or can't do something. One of those things is around the billing. So, I talked about a second ago the difference between you can start with $30 per user per month. Doctor. Yep. Doctor. It's like the all you can eat plan, then there's the 25,000 messages for $200 a month, or it's paid per the message down on the consumption or metered plan. If you're a brand new customer to Copilot and you decide, I'm going to start this a
little bit easy. I don't wanna do the $30 per user per month with a twelve month commitment, so one license is a $360 commitment for a year. If you say, I don't wanna do that. Instead, I wanna do the consumption model and kind of ease my way into it. Here's the thing that people don't realize, and Microsoft does not document very well, if at all. You don't get a semantic index until there is at least one active Microsoft three sixty five Copilot license, the $30 per user per month. So if you have 10 employees and you want them using Copilot,
if you just get the consumption model, all Copilot has is the all Copilot has access to is basically the same stuff that you get from ChattGPT, just the foundational knowledge from the training model. If you're a developer and you go through and extend it and do things like what you guys talked about, like adding actions to it, where you can implement the the RAG pattern, the retrieval of meta generation pattern where you where Copilot can look at your query or your prompt and say, oh, I need this extra data from this other endpoint over, like, in Dataverse. I can pull that stuff in, but it doesn't have
any it won't have any knowledge about your chats, your your transcripts, your calendar, your meetings, stuff like that. It won't have that context because the semantic index does not exist. It is not until you decide to spend, I will have one person get that $30 per user per month of my 10 employees, and then I'll do consumption, and it completely changes the experience for everybody that has that now has that consumption model in your organization. So now it's like a lot of people look at this and go, like, Copilot sucks. It's no better than ChatGPT. It doesn't have any it's not grounding
any of my any of its knowledge and stuff that's with my organization. It's because it doesn't have the semantic index, the trigger to create it, one act of life. So that's bizarre. So because I've never tried this with consumption models. So if I went in, bought one of the consumption model or the message pack, went into, like, Teams chat Copilot and said, go give me this information about my email or people I email or documents in SharePoint, it's not gonna return any of that. I can't speak for certain about the 25,000 the message packs. Okay. But I can say on the consumption model, yes, that is correct. The penny per message one. The the penny
per message one does not turn on a semantic index. The thing that turns it on is the the $30 per user per month license. A message pack might also do that. I think it does, but I can't be certain on that. Again, Microsoft doesn't do a good job of documenting this. You gotta find the right person and catch them at the right time and and get the answer. I have not that part I haven't I don't have a solid answer on. My belief is that a message pack does also do that because a message pack is also gonna have, like they also talk about if you have a message pack, it's gonna be, like, fifteen fifteen messages just to go through and ground
data because that's how much a query is gonna cost to the semantic index. Right. So I I think so, but I'm not certain. And it's I'm assuming maybe. Again, this is not anything that's documented having to do with that cost. If you're paying $200 a month, that probably helps to offset the cost of building the semantic index. $30 a month, same thing. If you're just paying a penny per message, you're like, how do we recover the cost of having the semantic index sitting out there because you're only paying a penny per message? I that would be my speculation as to maybe
the rationale, but I agree with you. Like, it should be right in that pricing table of, with this, you do not get a semantic index or grounded in your data with these two. Like, just give us a table and show us that. I completely agree with you on that point. That part to me is very frustrating. That was not it's not as clear. One of the other reasons why I say I think it's only limited to the consumption model is that this wasn't known until the consumption model was announced and when it wasn't even really known at that time, but that'severything kind of points back to that being like the big deal. So,
Ithat's why I think the message pack does have it, but again, I'm not a % certain. Got it. Do you feel overwhelmed by trying to manage your Office three sixty five environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligink is here to help, much like like you take your car to the mechanic that has specialized knowledge on how to best keep your car running, Intelligink helps you with your Microsoft cloud environment because that's their expertise. Intelligink keeps up with the latest updates in the Microsoft cloud to help keep your business
running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several thousand employees, they want to partner with you to implement and administer your Microsoft cloud technology. Visit them at inteligync.com/podcast. That's intelligink.com/podcast for more information or to schedule a thirty minute call to get started with them today. Remember, Intelligink focuses on the Microsoft cloud so
you can focus on your business. I'm curious. This is not a question that we necessarily talked about. But I know, like, with sensitivity labels, you're going in and starting to exclude SharePoint sites from Copilot, that type of stuff. It also affects SharePoint search, but, yeah, Copilot and SharePoint search are pulling from two different indexes. Is it because the semantic index is still using the search index to build it, or is it just because some of the functionality is the same? I think it's the functionality is the
same. I don't know if they're also using SharePoint as a way to populate the semantic index. They they may be, but I don't I don't think I got it. You don't know or looked into that why that's the case. Well, no, and I and I also don't think that they would be using the keyword indexing for that, for the search index that they've created for doing like SharePoint search or three sixty five search. Right. I think they would actually be going to the content, but that that actually brings up another interesting aspect too that that I that I thought was something that also catches people. And you so you guys, I think there's a little bit of a question around security and permissions. So like if like who
has access to the content? Does Copilot have access? Does Copilot and the user have access? Like how does that work? When it comes to the semantic index, it's the same as think of the same way as how SharePoint works. So when something goes into the semantic index, you have the ability to add an ACL to it, an access control list, the same way we do with, like, SharePoint search. So when something is indexed, it has an ACL attached to it or a collection of ACLs, and that collection says these people or these groups have access or do not have access to
this. So when Copilot does a query against the semantic index, Copilot's making the query, but it's making that query on behalf of the user that submitted the prompt. Okay. So it's getting so the data that Copilot gets back from semantic index is security trimmed for the user who issued the prompt. The same it Got it. Basically the same way how SharePoint search works. If I don't have access to a document and I do a search, I won't see it. You won't see it. It's just that the part that's me is now that is Copilot is doing it for me. So another question then,
how quick is the semantic index? Because like SharePoint search index, right, you can have the continuous index on so that when you change security, it's I mean, let's be honest. It's not always instantaneous because it still has to re index the content to re pull in ACLs when you change security to it. Yeah. Semantic index, any ideas? I think it's kind of the same delay as, like, how long it takes to share point to get in. And the other part is, like, how long is it how how fast is it actually querying that when you issue a prompt to Copilot and it's like dot dot dot Copilot's thinking. Yeah. That's how fast it is.
That's one of the reasons why it's like it's going to the semantic index. It's pulling some data back. It's going well, first, it goes to LLM, figuring out, like, what is the user's intent? And that tells it, Oh, I need to go over and find out find calendar message find calendar invites or transcripts or emails. It grabs that. It uses that as part of like it says that it's grounding that knowledge, but it it is. But that's actually pulling data to give it a like a a implementing the rag pattern on the fly. So, it's actually CopaD's going to the LLM multiple times, multiple round trips. So, that's one. So, another one though that's actually big is
so that's one about the the whole thing about the the semantic index getting created Doctor. Yep. Doctor. And then also the content that's going in. Here's another I I'm gonna call it a limitation, but make sure you listen to the whole part of this because there's like a big story here. Doctor. Okay. Doctor. So when let's say I put a 300 page PDF into a SharePoint document library, it's getting indexed into the semantic index, but only the first percentage of that file. And now, I'm going to use some numbers here, but these numbers are always getting bigger. And so, just take this
as an example, and this was true months ago, but I'm sure that when the time this episode comes out, when Build has happened, they would have made another announcement, so the numbers may be bigger. So, when if I have a 300 page PDF in there, and let's say deep in that PDF, there are lots of tables and good like documentation stuff in it and like real statistics, Only the first, let's say, 25 pages are getting indexed in the semantic index. Only the first four megs, essentially, of a piece of content is getting indexed. Now, granted, if I got 300
page PDF, usually only about 15% or 20% of that is actual content, and the rest of it is all like packaging and Adobe PDF crap and all that kind of stuff. Same thing with Word Docs and PowerPoints. But a lot of people look at this, they say, I know the answer is in that PDF. And let's just say it's on page two seventy four. But when I ask Copilot, it doesn't know the answer. It's because the semantic index doesn't get the entire file. It's limited. And it's just part because we're early with all this AI stuff, and Microsoft is still working to get the things big to make the the the capabilities of
what it can put in the index much bigger. So I I think it's big today. You you're already at 512 megabytes today. That four meg has been surpassed. So if that goes even higher, like, I don't know. I think about the majority of organizational content that you work with is not many of us are working with 500 meg word documents, maybe Excel sheets, thing things like that. But that that number always gets bigger and bigger, probably up to the limitation of just file size in SharePoint at some point. But context windows need to grow. I I
totally agree. I like context there. I actually have a one of my coaching clients is, in the, let's just say, the national security industry in The United States, and they're involved with the Department of Defense, and let's just say nuclear is involved in it as well. And they have some PDFs that are like three and four gigs of, like, nuclear safety stuff, and they're like, Why are we not seeing this stuff? So what we had to do instead to fix that was instead of just putting the stuff inside of a SharePoint document library, which they weren't doing anyway, but that's a whole head Snowden kind
of like be careful. Instead, what they had to do is they had to create a custom process that would take all content out of that PDF. They would then create the embeddings and then store it in a vector database, and then they could go through and find all the content based on the user's prompt. Best way to do that, use Azure AI search. It's another resource. And you can hook that up to a Copilot based agent, or you can write a REST API that a declarative agent that a developer could write to go get that data, and now you have full visibility
in the document. The big difference there, you as the developer, you have to pay for the Azure AI search resource and how much which is a a metered based thing. So how much you're putting in it and how much you're actually using it. Whereas, if you're doing a semantic index, that's part of the $30 per user per month all you can eat plan. Right? So Yeah. That's a there's a way around that. And, also, the Azure AI search doesn't have the security trimming because they don't have the capability to do ACLs on the stuff today. So that's a little Hey. It's it's a you can't have your cake. You need it too Right. Depending depending
on where you are. I I think there is friction there. Like, one of the exciting things about context windows growing and effectively this concept of, like, quote, unquote, AI memory, right, and and chat memory and all these things is you're not gonna have to do that extra hop of potentially multiple chunks. Like, I I do run into customers in this situation today to really massive PDFs, things like that, where they have to pre chunk, and then they have to go and vectorize that to generate they have to go generate embeddings, store all that stuff in a vector database, be it Pinecone, AI
search, whatever it happens to be, and manage it through. But if you've got some maturity and, like, you're there and you understand that stuff and you're kind of on the, quote, unquote, cutting edge, like, sure. It's right there. It it is available for you. I've always been surprised, like, even watching, like, PMs on our team be able to pick some of that stuff up and and just run with it, like, out of the box. Well, not out of the box. Like but the the number of us now that are sitting here writing Python scripts on the side to do weird machinations, like, it it it's ever growing
as as as we get up here and and we do these things. We we've been we've been prepping for our build session all week and thinking about, like, oh, which model do we show? How are we gonna demonstrate checkpointing here? What what are we gonna do in this scenario, in this thing to to really show it off where it shines? I'm not at the point of writing my own Python scripts yet. I am at the point of asking chat gbt telling it what I want, and it writes the Python script for me. But I'm not I'm not the point of writing my own. Yeah. Well, nobody should be writing Python from scratch anyway these days. Let let me yeah. I do it for you. Yeah. Those were the really big things that I
saw, and I know they're not exactly dev stuff, but I I don't know what else you guys wanna talk about in terms of, like, dev specific things. So I have another question that I think you said this in the video. I think we're okay to talk about this. Well, it's something that that This is the last podcast I'll have my my my MVP NDA. Well, that's why we can edit. Right? Todd's the only one listening. Todd, if you hear this is NDA NDA or anything about this is NDA, don't I know Todd. Todd will be good. Like, we talked about SharePoint index, surfacing content, size of files. One of the things you used to do or you
still do to help with SharePoint search and digging through libraries and lists and all that is metadata on these files to be able to do, like, filters on it and query on it. And then you get a query and you get your filter down the left side with your metadata and doing all of that. How does Copilot handle metadata and files? That's a great question. It's a great answer. It doesn't. So here's the thing. Copilot only okay. So it's not so much Copilot. Remember, all goes back to semantic Right. Semantic index. The semantic index only has the content from files inside
of SharePoint and OneDrive. It does not have content related to the metadata on those files in a document library. It does not have data from SharePoint lists. It does not have data from SharePoint pages. Microsoft knows all of this. They that is might even has changed by the time I'm saying this. It might even change by the time you're hearing this, but especially because build is next week because it's a huge ask for people. But I know, like, list data has been a huge blind spot when it comes to m three sixty five Copilot because it is that content is not being indexed
into the semantic index. Only the files inside of a document library in OneDrive are being indexed. Okay. And not even so not even ASPX pages because technically, I think of ASPX pages and, yeah, you look at companies' intranets or I know companies, good idea or not, have built, like, whole wikis and all their documentation in ASPX pages because they want it in that format. And then so they could also be very surprised when they go ask Copilot, and Copilot's, like, missing data. Maybe it's missing data from Power Apps that are writing to SharePoint lists
or Yeah. You're a developer or SPFX. I can do SPFX writing the SharePoint list. None of that's gonna come into Copilot. That's correct. But, again, I just wanna stress one thing, and that is that is the way it at least was recently. And I would whenever something like this comes up and cope with with, like, a limitation, it's always, I'm filing this back in my head. When I do a search, something's not happening. It's like, I remember that guy told me this thing at one point. Yeah. I wonder if that's going on. Let me go research or go to your research and be like, well, no. Andrew's wrong. I just saw
this build this week, and they said that now we're doing list content, like, going, yeah, well, that's probably that's a couple days after we recorded this. So this is changing. Microsoft is is like they're feverishly working Right. On everything with this to make it more powerful. So this stuff could is I would expect it to change and get better, but it's just one of the things you wanna be, like, let's double check and make sure this is the case. That's okay. Right. That's turned into my default answer. And I'm doing demos now or something, and I can't find something. I'm like, must have changed because it does. It feels like this stuff is changing, like, daily or hourly or
by the minute. I was teaching a class this week. I recorded a demo at 10:30 at night. I taught the class at nine the next morning, and it literally was different at nine the next morning in the exact same tenet. And so I was like, Yeah, stuff is flighting for build, so my bad. Welcome to Evergreen SaaS products. Exactly. Right. And this time of year is always fun too because the documentation often freezes before these events. So sometimes you start seeing new things come out before documentation's even there or or anything else along the way. So it's it's a little bit of a a challenge to to keep up with and
balance. But I think, like, we've been talking about this for years and years and years with just Office March, m three sixty five in general. Like, the only constant is change there anyway. So you gotta kinda be ready ready to pivot and and deal with those things as they do come up. And I think, like, back to earlier about those customers who are mature enough maybe to go ahead and chunk and and generate embeddings for for their own stuff, they also know that they're kind of on the bleeding edge and need to keep up
with some of this stuff. Yep. I I do worry sometimes about the not that's not like some, like, heavy hearted, like, the world's gonna end kind of thing, but just about the friction for customers there, just as the buttons are moving every day, the functionality is changing. Like, oh, I I I can do deep research today in in Copilot consumer, but I can't do it yet in m three sixty five Copilot. Like, those disparate experience experiences and things, they they do tend to add up. So you still gotta do the
TCO and figure out, like, where value is inflicted in my org. I completely I completely agree with that. I mean, it's like it's one of the things that I I find is like a service offer for my customers. I I I do, like, this coaching service, and it's like, hey. Look. You can't stay on top of this stuff. I'm the one that's my job. I'm tango staying on top of it. So it's pretty much we jump on a call. It's, like, going, what do you know that's different based on what we're doing? And I'm, like, going, okay, this is no longer the case, or this is the case, or be careful with this, or I know that this is gonna change, so I would not invest time here because stuff is gonna change in the next few months. So it's it's just the only thing that's different about what you said
to me, Scott, is that it's the change is just going so much faster. Like, we are on a much steeper downhill than we were on the mountain before. For sure. I I think pace to change is is very real there. So should you get into this world, all of a sudden, you end up in a place where you're kind of joining the space race of everybody else. And and you gotta be prepared for that. So the solution that you build today, should you take a dependency on any kind of SaaS surface or anything like that, it's or or a PaaS service even, you have to
know that it's gonna change. And it's probably gonna change more rapidly than you're used to in the past. That said, if you're still doing some of this stuff maybe in, like, an on prem context, like, it's one thing to say, hey. I'm using Azure AI studio or Azure AI search, and it's moving at this pace. Or I'm using AI Foundry, and it's going this fast versus I I took a dependency on langchain and Pinecone here, and and that was locally for me to do. So you you can have a little bit of that stability along the way if you need it. Yep. It's not gonna be super stable. Like
you said, the hill is still a hill. And and it's and it's a very it's it's a very steep one. Yeah. That we're all falling down or trying to climb up, I can't figure it out. It's I think it's a little bit of both. I can I can make a little bit of progress, but I fall four rings down? Yes. Yeah. %. So I know, Scott, you have meetings coming up. I have meetings coming up. AC has coaching and meetings coming up. But anything else, AC, any last thoughts, other things people should be aware of or even if people wanna learn more about Copilot
dev? I know we didn't get into all the developing your own agents. Maybe we'll have to have you on for the fifth or sixth time or whatever and talk more about the custom dev story, but, yeah, people wanna get in touch with you, any of that. Yeah. I I appreciate that. I guess it I would love to come back on happy to talk about the dev story. In fact, it'd probably be better if we did it after build too so that I can Alright. I know some stuff that's coming that we'll talk about that Microsoft will talk about next week. But, yeah, I have a company called Voitanos. I focus on making you the best Microsoft three sixty five developer in the Microsoft three sixty five ecosystem
and indispensable in your organization. So I do that through, like, SharePoint framework training, Teams app dev training, and Copilot training. I have a a six hour, two day virtual workshop that I do on building declarative agents for the developer audience, so not using Copilot Studio, but using Visual Studio Code. And I'm due to refresh that. I'm intentionally waiting until after build because I know things are gonna be changing. But I'm gonna I plan on I do that at conferences for a full day workshop. I did it at the Microsoft three sixty five community conference. I'm gonna do it again at the TechComm conference in Seattle in June. But I also
have that as a, as a live workshop that I'll offer on my site at voitonos.io. And if you, if you if you're interested too, I mean, I've got, like, the recordings from when I last delivered that in March are also available for, to enroll in on my course where you'll actually get access to the the updated ones once those are done. So, yeah, I mean, the best place to find me is just voitanos.io or search for I I'll put give you guys my link so you can put them in the show notes, but Perfect. LinkedIn, Blue Sky, YouTube, I'm easy to find. If you just search for Andrew Connell in SharePoint, it's I'm pretty
much the first one that's gonna come up. Perfect. Sounds good. Well, thanks, AC. Thanks for jumping on with us, filling in some of those additional details around CoPilot agents. And, appear I was gonna say the message takeaway from this is go pay attention at build, but this is gonna come out after build. So if you're listening to this Go back and watch build. Go back and watch build and look for maybe certain announcements around Copilot agents and some of the stuff you may have heard here. Watch for my recap. I will have a recap for eight when it comes to SharePoint, Teams app dev, and agents that
is, like, the build stuff, like, this is what you wanna focus on. I did it for Ignite. I did it for Build. It's a great way for me to get get up to speed on everything, and I plan on doing it again as Build goes on next week as well. Okay. Perfect. And if you have that before this show goes live, send it over, and we'll include that in the show notes as well for anybody listening. If you wanna get AC's recap, and if it's live or going to be live, we'll add that as well. Awesome. Sounds good, man. Awesome. Well, thanks, guys. Appreciate it. Yeah. Enjoy your weekend, and we'll talk to you guys later. Thanks. Alright. Thanks, Ben, Scott. And congrats again on episode
400. That's awesome. Alright. Thanks, AC. Appreciate it. Bye. If you enjoyed the podcast, go leave us a five star rating in iTunes. It helps to get the word out so more IT pros can learn about Office three sixty five and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening, and have a great day.
Welcome to Episode 441 of the Microsoft Cloud IT Pro Podcast. In this episode, Ben and Scott take a first pass at Microsoft Ignite 2026, starting with a question a lot of IT pros are asking: is the pace of change...
Welcome to Episode 440 of the Microsoft Cloud IT Pro Podcast. Recorded live at the North American Cloud and Collaboration Summit in Branson, Missouri, this MS Cloud on the Road conversation has Ben sitting down with Jay...
Welcome to Episode 439 of the Microsoft Cloud IT Pro Podcast. In this episode, Ben and Scott dig into two Azure agents with confusingly different names: the Azure Copilot Troubleshooting Agent and the Azure SRE Agent....
Welcome to Episode 438 of the Microsoft Cloud IT Pro Podcast. Back in Episode 429, Ben and Scott introduced LLM wikis. In this episode, they come back a few months later to compare what those wikis turned into after...
Welcome to Episode 437 of the Microsoft Cloud IT Pro Podcast. In this episode, Nick Burling, Chief Product Officer at Nasuni, joins Ben and Scott to talk about getting unstructured file data ready for AI. Nick digs into...