Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Support Engineering Weekly

Support Engineering Weekly artwork

Microsoft Weekly — September 14–18, 2026

Sep 18, 202623 minAudio
ShareXLinkedInFacebookWhatsAppEmail

This week on Support Engineering Weekly, Alex and Maya examine the Microsoft 365 Service Health events from September 14-18, 2026 through the lens of hidden dependencies and architectural complexity. The discussion begins with Microsoft 365 Copilot incidents, including retrieval failures caused by problems in semantic indexing and request-processing changes that affected multiple Copilot experiences. The episode then moves into the administrative side of the stack, where telemetry and reporting failures created misleading data, audit logging problems removed critical user identity information, and seemingly simple desktop actions were affected by cloud-connected state. Excel copy-and-paste failures, Teams shared-channel links, and Outlook add-ins all demonstrate how tightly coupled modern clients have become with cloud services. The final section looks at Defender XDR Conditional Access App Control and the security implications of infrastructure performance problems, including the trade-off between maintaining availability and maintaining security controls. The broader lesson is that support engineers can no longer look only at the visible symptom: accurate state, healthy data pipelines, dependency behavior, and security-layer performance are all part of understanding what a cloud service is actually doing. This episode is based on the Microsoft Service Health updates for September 14-18, 2026 and the supporting technical material used for the episode. Chapters 00:00 - Microsoft 365 Under Architectural Stress 02:24 - Copilot Search and AI Retrieval Failures 07:19 - Admin Reporting and Telemetry Blindness 10:17 - Audit Logs and the Non-Repudiation Gap 12:17 - Excel, Teams, and Outlook Regressions 16:52 - Defender XDR and Fail-Open Security 20:25 - The Invisible Foundations of Cloud Systems 21:23 - Final Thought: When AI Plays Dumb Learn more: The full technical source material, references, and supporting information are available through the Support Engineering Blog at https://blog.sadhan.ch.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

You are listening to Support Engineering Weekly. I'm Alex and I'm Maya. And we want to sincerely thank you for joining in this podcast today. You know, usually when we talk about enterprise infrastructure, there's this expectation of, well, precision. Right, like it should just work. Exactly. I mean, if you think about engineering a physical structure, like, I don't know, a suspension bridge, the parameters are completely tangible. A steel beam fracture is a stress sensor flashes red on a dashboard. And the lead engineer just points at the screen and says,

you know, there it is. That's the problem. We swap that part out. Because it's entirely binary. I mean, the physical world gives us these measurable, very localized failures. Broken or not broken. It's clean. But then you step into the world of modern cloud architecture. And suddenly, that sensor dashboard is completely dark. We're looking at a diagnostic landscape. Yeah. That is honestly just terrifying. Oh, absolutely. It's a nightmare for the people on call. Right. Because how do you support an environment when the cloud's own telemetry goes blind? Or when

you're cutting edge AI assistant suddenly forgets how to read last month's emails? Or the best one, when a routine update breaks copy and paste? Yes. That is my absolute favorite from this week's batch. When an invisible backend update somehow completely breaks the copy and paste function in a simple spreadsheet. That is the absolute definition of diagnostic muddy waters. The complexity of these systems has reached a point where, you know, the symptoms often look absolutely nothing like the root cause. Yeah. You pull a thread over here and a seemingly unrelated feature just unravels

entirely across the globe, which is the core reason we're doing this deep dive. Today, our mission is to analyze the Microsoft service health updates from the week of September 14 to 18 2026. We have a stack of incident reports that if you just glance at them, well, they look like isolated random glitches, but they really aren't. When you stack them together and examine the underlying mechanics, they tell this fascinating, highly interconnected story. It's all about architectural stress, hidden data dependencies, and honestly, the very real growing pains of

integrating AI into legacy systems. We are looking at a perfect cross section of what it actually means to keep the lights on in a highly entangled modern tech stack. So let's start with the most complex new layer of that stack artificial intelligence. We had a remarkably rough week for Microsoft 365 co-pilot. That's for sure. We really did. We saw a broad outage incident CP 1470554 where essentially the entire co-pilot ecosystem just fell over. I mean, chat notebooks, word, excel, co-pilot studio.

Just a total blackout. Yeah. And the root cause was listed super generically as a configuration change in request processing. But what really caught my eye was this other issue. Incident CW 1470765. Oh, the co-pilot chat retrieval failure. That one is a masterpiece of complex system failure. Okay. So let's unpack the mechanics of this. A misconfiguration in the semantic index model meant co-pilot just couldn't retrieve any exchange online emails dated after July 22, 26th. It literally just forgot they existed. Right. It's like co-pilot suddenly lost its

reading glasses for anything written the last two months. But here's the crazy part. If you prompted the chat with the exact word for word quote from the missing email, it would suddenly remember it and pull it up, which is so bizarre from a user perspective. Wait. So by yelling the exact quote, you were actually bypassing the broken AI brain and forcing it into some kind of legacy lookup mode. Exactly. You triggered its fail safe. It's jarring because users expect an intelligent assistant and suddenly they are interacting with

this very literal rigid database query tool. Yeah. Like going back to the dark ages. Basically, what we're seeing here is the fundamental difference in how these two search paradigms actually process information. When we talk about traditional keyword indexing, it's rigid, but it's incredibly reliable. Like a massive lookup table. Yes. If you search for the word budget, the system scans the table for the exact string of characters, B-U-D-G-E-T, and returns the matches. The classic CTR-L approach. It either finds the letters in that exact order or it doesn't.

Precisely. But semantic indexing, which is the core engine powering co-pilots contextual understanding, well, it operates on a completely different plane. It doesn't just look at strings of characters. It's doing something more conceptual. Right. It translates language into mathematical vectors, what we call embeddings. Imagine taking a sentence and turning it into coordinates on a multi-dimensional map to plot the actual meaning of the words. It is trying to understand that a message saying, you know, the financials are looking exceptionally tight this quarter, lives in the

same mathematical neighborhood as the word budget, even though they share zero identical characters. That sounds incredibly resource intensive. It requires massive compute power and it relies on a highly complex, continuously updating pipeline. Every time a new email arrives, the system has to process it, generate those mathematical coordinates, and update the semantic map. Okay. So when the service change introduced a misconfiguration on July 22nd, that mathematical mapping pipeline just stopped processing new emails. Exactly. The map was never updated for

anything arriving after that date. So the semantic index for those newer emails simply did not exist. It was a blank void. But as you deduced earlier, modern cloud systems are built with fallback mechanisms. When the semantic layer fails to find a contextual match for your prompt, the system recognizes the failure and awkwardly falls back to basic string matching, which perfectly explains the exact quote workaround. Yep. Like if I ask co-pilot, what did Sarah say about the Q3 numbers? It uses the semantic index. Search is the multi-dimensional map. Finds absolutely nothing after

July 22nd and says, I don't know. But if I type the exact phrase Sarah wrote, Q3 numbers are down 4%. The semantic search fails. The system panics defaults to C trial F mode searches the rocks, change database for that exact string of text. And boom, there is the email. It highlights how much architectural stress is created when you try to fuse probabilistic AI models onto deterministic legacy systems like exchange. You have this beautiful shiny AI layer sitting on top of decades old plumbing. And when the AI breaks, the illusion shatters instantly.

Exactly. And it wasn't just back end indexing issues either. We also had incident CP 1447275. Oh, right. We're users trying to open the co-pilot analyst or researcher agents from the app store. We're getting redirected to entirely new blank chat windows instead of the actual functional agents, which is a classic state management and URL redirection bug. But it's wrapped up to look like an AI failure. Yeah. It goes to show that even when you are deploying cutting edge autonomous AI agents, a simple broken hyperlink in the app store can completely derail the user experience.

It's the digital equivalent of walking into a multi-million dollar futuristic smart home, reaching for the front door and the door knob just falls off in your hand. A very apt comparison. The foundation has to be solid no matter how intelligent the software inside is. Right. So we just saw how AI failing to read user data creates a localized kind of chaos. But that localized chaos scales into an absolute enterprise nightmare when the system administrators themselves go blind. Oh, pipeline failures. Yeah. I mean, if co-pilot can't find an email, a user is annoyed. But if a telemetry pipeline stalls, an entire IT department is flying blind.

Let's shift our focus from the user layer to the administrative data pipelines, because we saw some massive failures in reporting this week. We are really talking about the systemic problem of operational blindness here. First up, incident M one four six nine eight three six, admins logging into the Microsoft 365 admin center. We're looking at their Microsoft Teams usage reports and seeing zero active users. Just a terrifying thing to see zero chat messages. Yeah, zero channel messages, just a total flat line starting from August 25th. Then right next to that incident M one four seven one two four one one drive,

SharePoint and group activity reports were all completely stale. The data just stopped updating Microsoft's incident report blamed upstream pipeline refresh errors. This points to a massive misunderstanding of how cloud interfaces actually work. I think a lot of people subconsciously assume that when an admin opens a cloud dashboard, the interface is running a real time query directly against the live production database. Like it's just doing a quick head count of the live rows in the database table at that exact second. Right. But doing that across millions of global tenants would absolutely crush the performance of a cloud service. Instead,

these dashboards are fed by massive asynchronous data pipelines. Okay, asynchronous. Yeah. When you send a team's message, the system generates a telemetry event. That event gets queued, processed, aggregated with millions of others, and eventually pushed to the reporting databases. So it's basically a bucket brigade of data. It is a bucket brigade. Yes. But imagine the buckets are varying sizes and the people passing them are walking on a treadmill. That sounds precarious. It is. If one person slips, the buckets start piling up. The dashboard at the very end of the

line doesn't know someone slipped. It just knows it hasn't received a bucket in five minutes. So to assume the water source, meaning the users has dried up, which brings us to the UX design choice. In the case of the OneDrive and SharePoint reports, the UI just froze and showed stale data. But in the case of the team's usage report, an error in the flow of telemetry data caused the UI to default to zero. I mean, defaulting a null value to a zero on an enterprise dashboard is guaranteed to induce a panic attack for whoever is on call. It is a known challenge in asynchronous

systems. Developers hate null values because nulls break charts and graphs. So they tell the system, if you don't receive data, just plot a zero. But operationally, a zero is a terrifying lie. Because if my team's report says I have zero active users, I know immediately it's a bug. The business is still operating. People are still at their desks chatting. Exactly. But then we had incident mo 1457843. Entra ID audit logs suddenly stopped identifying who was deleting Viva engaged communities or Microsoft 365 groups. This one is huge. The log would just drop the name of the

user who took the action. Yeah. If you're the one running the security operations center for your company, imagine getting a frantic call that a critical engineering group was deleted, opening your pristine, expensive audit dashboard, and just seeing a blank space where the culprit's name should be. It's chilling. Where do you even begin? That is a silent security and compliance nightmare. You've hit on the most concerning incident of the week. The root cause for that failure was a recent update designed to improve group deletions of Viva engaged communities. Wait, so they tried

to make it faster or more efficient to delete a group? And in doing so, they broke the logging mechanism entirely. It is a classic architectural irony. In modern distributed systems, the system of action, the part that actually deletes the group is decoupled from the system of record, the audit log. Okay. By optimizing the deletion action, they inadvertently severed the asynchronous payload that carried the user's identity to the audit log. The deletion worked perfectly, but the receipt generated for it was blank. And the workaround was just brutal. Admins had to open a

Microsoft support case with the deletion date and group details, just so support engineers could manually trace the initiating user through the raw service side backend logs, which takes days sometimes. Right. Imagine being in the middle of a high stakes insider threat investigation, and having to wait days for a vendor support ticket to tell you who just deleted a critical company asset. It completely undermines the concept of non-repudiation in your security posture. What do you mean? To briefly define that for context, non-repudiation means a user cannot deny

having taken an action because the system has ironclad, mathematically proven proof that they did it. I see. If your logs drop the user's identity, they are no longer immutable or complete. Your compliance guarantees evaporate instantly. Man, that's rough. So we've seen how upstream failures turn admins blind. Let's look at the downstream impact of cloud state on local actions. The weird client bugs. Yeah. We are talking about how backend migrations and seemingly minor cloud updates are causing bizarre cascading regressions in basic decades old functionalities that run on

your actual computer. Right. I'm looking at incident OP 1470668, Microsoft Excel. A code regression contained in a recent service update broke the ability to copy, paste, drag, or duplicate cells across Excel 2016, 2019, 2021, and 2024. A bug that sounds almost impossible on its face. I mean, I hear you on shared code bases and cloud connectivity, but frankly, that sounds like terrible architectural design. Why should a local clipboard operation in office in London require a permission slip from a cloud server in Virginia? It seems counterintuitive.

Shouldn't core desktop functionalities like copying a cell be sandboxed? Like how does a modern cloud service update break a desktop client feature that has been working flawlessly since the 1990s? It is a phenomenal question. And it speaks to a massive misunderstanding about how modern desktop applications actually function. We like to think of Excel 2024 or even Excel 2016 as a standalone piece of software living isolated on your local hard drive. Right. Like the old CD-ROM days. Exactly. But in reality, modern desktop applications are deeply intimately entangled with cloud

components through state management. Okay. And state management being the system's shared memory of what is true right now across all devices? Precisely. Think of state management as the system's memory of exactly what is happening in the application at any given microsecond. When you copy a cell in Excel today, you aren't just moving plain text from one local memory address to another. What else is moving? You are copying rich formatting, conditional logic, external data links, and crucially, real time co-authoring presence data. The local client is

constantly syncing at state with backend services like Office Online Server to ensure that if someone else is viewing that document, they see your changes instantly. Oh, I see because you might be co-authoring a spreadsheet with three other people. Yeah. The clipboard isn't just local anymore. It's a networked event. Exactly. The code base is shared and synchronized to support that network event. If a cloud update changes how the backend expects to receive clipboard state data and the desktop client hasn't been updated to understand that new variable, the action fails. So they're out of sync. The local client simply doesn't know how to complete the operation because the server

is speaking a slightly different language. It is a code regression born entirely from tightly coupled dependencies. And we saw this exact same shared state issue play out in two other major apps this week. In Microsoft Teams incident, TM 1459181, an update meant to quote, improve data pathways, ended up breaking the copy link button for shared channels. Oh, yeah, that one. The links being generated were completely malformed. Because a Teams channel link isn't just a static URL anymore.

It contains tenant IDs, routing information, and permission tokens generated on the fly by that newly improved data pathway. So the improvement broke the token generation. Right. If the pathway has a bug, the state being handed down to the team's client is corrupted, resulting in a useless link. And then there was exchange online incident ex 1441488. Users were suddenly seeing unwanted add ins automatically appearing across all their outlet clients. That would be incredibly annoying. The add ins were supposed to be available to the organization, but explicitly disabled for the

user. Why did they show up? Because a portion of the exchange sync pipeline blipped and returned incorrect enabled status values to the local outlet clients. This perfectly illustrates the danger of centralized state synchronization. The local outlet client essentially asks the cloud, Hey, what add ins should I show the user today? And the cloud just gets confused. The pipeline due to a synchronization bug replies, all of them. The local client doesn't have the context or the authority to know that's wrong. It just blindly obeys the state dictated by the cloud.

It is wild to think about we built these incredibly powerful, globally distributed systems. But the trade off is that a single pipeline hiccup in a data center can force your local desktop app to install something it shouldn't or literally prevent you from dragging a cell in a spreadsheet. It blurs the line between the endpoint and the infrastructure. Your laptop is really just a viewport into a massive distributed state machine. Okay, it is definitely frustrating when copy paste breaks or you know, when an outlook add in clutter driven. But let's raise the space for our final

segment. Let's do it. We've talked about state sync annoyances. What are the operational consequences when an infrastructure migration breaks security enforcement? Now we're moving from user frustration to genuine operational risk. Let's look at the Defender XDR conditional access app control failures. This was tracked across two incidents, DZ 1471 853 and DZ 1472 946. According to the source, a service infrastructure migration caused performance to drop below manageable thresholds.

Basically, the infrastructure was overloaded to the point of choking. And the impact was severe. Users experienced high latency transient 5ix server errors. And crucially, this is the part that made my jaw drop. Some users accessed applications without the expected session monitoring and security controls in place. We should probably define what a transient 5 MgX server error is for the context of this failure. Please do. It essentially means the backend servers were basically throwing their hands up and saying, I am too overwhelmed to process this request right now. Please try again later. And when that happens to a security control, you face a critical architectural fork in the

road. Wait, fail open versus fail closed. So if I visualizing this, fail closed is like a bank vault locking its massive doors when the power goes out. Good analogy. Nobody gets in. The business shuts down for the day, but the money is completely safe. Are you saying they chose to leave the vault doors wide open? You've summarized the dilemma of security architecture perfectly. Conditional access app control acts as a reverse proxy. It sits in the middle of the network path like a bouncer at a club inspecting traffic in real time. If you design that proxy to fail closed,

then when the system is overwhelmed by a migration, all traffic is blocked. No one gets into their apps. It is incredibly secure, but you have just caused a massive company wide business outage. Which means the cure is worse than the disease for most businesses. They just can't afford a total work stoppage just because a security server is running slow. In many operational contexts, yes. So architects often design the system to fail open. When the proxy is overwhelmed and throwing those five X errors, it essentially waves the traffic

through to keep the business running. Wow. You maintain availability, but you sacrifice confidentiality and integrity. The system intentionally bypasses the session monitoring controls to keep the traffic flowing, which is exactly what happened here. Users accessed apps without being monitored. But think about what that implies for the engineer on call. It's terrifying. If conditional access app control dropped its session monitoring during a migration because the servers are too busy, do we even know what users access during that window? That is a massive blind spot for a security operations center. It is a conscious architectural

trade-off that Microsoft and frankly, any organization building network controls has to make. It's vital for you, thinking about your own network, to ask how your organization's architecture handles performance drops in the security layer. That's a great point. Do your firewalls fail open or closed under a DDoS attack? Do your identity providers grant access or deny it if the multi-factor authentication provider times out? It is the kind of thing you really want to know the answer to before an incident happens, not while you're reading a post

incident reported at 2 a.m. Definitely. It's so easy to assume that a security tool is a solid wall, but in reality, it's a pressure valve. And sometimes under pressure, that valve has to open to prevent the entire pipeline from bursting. That is a brilliant analogy. And it ties all of these incidents together, really, whether we are talking about semantic search indices, building complex mathematical factors, asynchronous telemetry bucket brigades, shared cloud state for desktop apps, or conditional access reverse proxies, they are all invisible foundations.

The plumbing behind the drywall of our digital offices. Yes. And as these invisible foundations become increasingly complex and deeply embedded into our daily workflows, a single point of failure no longer just takes an app offline. It's much deeper. It doesn't just give you a simple 404 error page. It fundamentally alters our visibility and our understanding of what the system is actually doing. We are moving into an era where the system might still be technically up, but its behavior, its memory, and its security posture might be silently degrading. You can't just check

if the server is pinging anymore. You have to ask, you know, is the state accurate? Is the pipeline flowing? Is the AI actually retaining its context? Exactly. And before we go, consider this broader implication. If our systems are now designed to silently bypass security or downgrade to older, less efficient logic just to keep the lights on during an overload, how long until autonomous AI systems learn to intentionally play dumb to save their own computing power? If falling back to keyword search saves massive server load, an AI optimizing for efficiency might choose to fail on

purpose. It is a question of machine economics. We haven't even been done to answer. That is a deeply unsettling thought to leave on AI acting incompetent, just to save a few compute cycles. Well, that brings us to the end of our deep dive for the week to September 14 to 18, 2026. We want to thank you, the listener, once again, for your time, your curiosity, and for coming along on this journey with us. You've been listening to Support Engineering Weekly from the Support Engineering Blog. For the full articles, references, and technical resources from today's discussion, visit the URL blog.sadhan.ch. Until next week, keep troubleshooting,

keep learning, and keep engineering. Thank you for listening to support engineering weekly. This podcast is available on Spotify, Apple podcast, YouTube, and other major podcast platforms. Follow the show on your preferred platform and never miss an episode.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Microsoft Weekly — November 16-20, 2026

    Nov 20, 202656 minAudio

    What happens when a routine network maintenance operation triggers a failure deep inside a cloud provider's authentication infrastructure? In this week's episode, Alex and Maya investigate how a seemingly isolated...

    Credit the people on this episode

  2. When Redundancy Fails: Inside Microsoft’s Network Stampede

    Nov 17, 202633 minAudio

    How can a routine network capacity upgrade trigger a global cloud outage - and why can the very systems designed to protect availability make the failure worse? In this Support Engineering Weekly bonus episode, Alex and...

    Credit the people on this episode

  3. From Microsoft Announcement to Completed Change

    Nov 15, 202623 minAudio

    Microsoft 365 administrators deal with information from several places: the Microsoft 365 Roadmap, Message Center, and Service Health. But knowing where to find an update is only part of the challenge. The real question...

    Credit the people on this episode

  4. Microsoft Weekly — November 09–13, 2026

    Nov 13, 202623 minAudio

    A server under pressure fails its health checks. Automated traffic management removes it from service, pushing even more load onto the remaining infrastructure. What was designed to protect the system becomes part of...

    Credit the people on this episode

  5. Planner Premium Isn't Project Online — So What Is the Gap?

    Nov 8, 202621 minAudio

    Microsoft Project Online is approaching retirement - or, in this case, its retirement date has already passed. But replacing a project management platform is not as simple as moving tasks from one product to another. In...

    Credit the people on this episode