[English]Microsofts Cloud-Dienst Exchange Online ist vorgestern und gestern (28./29. März 2023) wohl gestört gewesen. Die Störung begann bereits am 28. März 2023 um ca. 14:00 Uhr, als Nutzer sich nicht mit anmelden konnten. Microsoft hat das bestätigt und schreibt, dass es wohl auch eine Dienstbeeinträchtigung gebe, so das E-Mails auf Exchange Online in Quarantäne geschoben werden. Blog-Oser Andreas P. hat mich per Mail informiert und die MS-Status-Informationen zukommen lassen (danke dafür). Hier ein schneller Überblick, was Sache ist.
Anzeige
Es ging bereits am 28.03.2023 um 14:51:55 Uhr los, als Microsoft ein Problem feststellte. Eingehende E-Mails von Benutzern wurden in Exchange Online gelegentlich und unerwartet unter Quarantäne gestellt. Microsoft vermutete abld, dass dieses Problem durch Restauswirkungen eines früheren Azure Active Directory (Azure AD)-Problems verursacht wird, bei dem Benutzer mit geolokalisierungsbasierten Richtlinien für den bedingten Zugriff betroffen waren. Ich hatte über dieses Problem im Blog-Beitrag Microsoft 365: Nutzer versehentlich per Geo-Location ausgesperrt (23. März 2023) berichtet. Diese Ursache hat Microsoft dann in weiteren Statusmeldungen bestätigt.
Wir haben eine Reihe von betroffenen IPs mit falschen Geolokalisierungs-Klassifizierern bestätigt und sind dabei, eine Regel anzuwenden, die sicherstellt, dass alle neuen Nachrichten über die erwartete Geolokalisierung geleitet und zugestellt werden. Sobald dieser Prozess abgeschlossen ist, werden wir uns darauf konzentrieren, die zuvor betroffenen Nachrichten erneut abzuspielen, um sie aus der Quarantäne zu befreien und das Problem vollständig zu lösen.
Ursache war wohl ein Fehler bei der Markierung der Geolokalisierungsdaten, der dazu führte, dass einer Reihe von betroffenen IPs falsche Geolokalisierungsklassifizierer zugewiesen wurden. Microsoft hat dann Backups zurückgerollt, um den Fehler zu beheben. Hier noch die betreffenden Status-Mitteilungen zum Vorfall.
Vorfall: EX534197, betroffener Dienst: Exchange Online, betroffenes Feature: signin, aktueller Status: Service degradation
Published Time: 29.03.2023 11:09:30
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: The deployment of the fix is taking longer than expected due to additional validation steps. We're continuing to monitor the progress of the fix as it deploys to all affected environments. We anticipate its completion time by our next scheduled update.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Start time: Tuesday, March 28, 2023, at 10:00 AM UTC
Root cause: A recent issue affecting our geolocation data marking resulted in a range of affected IPs being assigned incorrect geolocation classifiers, resulting in impact.
Next update by: Wednesday, March 29, 2023, at 10:00 PM UTC
Published Time: 29.03.2023 06:23:28
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We're continuing to monitor the progress of the fix and we anticipate its completion by the next scheduled update.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Start time: Tuesday, March 28, 2023, at 10:00 AM UTC
Root cause: A recent issue affecting our geolocation data marking resulted in a range of affected IPs being assigned incorrect geolocation classifiers, resulting in impact.
Next update by: Wednesday, March 29, 2023, at 3:00 PM UTC
Published Time: 28.03.2023 23:44:28
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We've successfully gathered the full list of affected IPs and initiated the process of replaying the impacted messages, which will release the incorrectly sorted email from quarantine. We're expecting this process to fully resolve the impact of this issue, and it may complete as early as the time of our next scheduled update.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Start time: Tuesday, March 28, 2023, at 10:00 AM UTC
Root cause: A recent issue affecting our geolocation data marking resulted in a range of affected IPs being assigned incorrect geolocation classifiers, resulting in impact.
Next update by: Wednesday, March 29, 2023, at 10:00 AM UTC
Published Time: 28.03.2023 21:19:34
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We've utilized a recent backup of our IP geolocation data to restore the affected IPs' incorrect geolocation classifiers, preventing further impact from the issue. Going forward, we're currently working to gather a full list of affected IPs so we can replay the impacted messages, releasing incorrectly sorted email from quarantine. We're aiming to begin the process by the time of our next scheduled update, at which point we should be able to provide a timeline for full remediation of the impact.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Root cause: A recent issue affecting our geolocation data marking resulted in a range of affected IPs being assigned incorrect geolocation classifiers, resulting in impact.
Next update by: Wednesday, March 29, 2023, at 4:00 AM UTC
Published Time: 28.03.2023 19:04:37
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We've confirmed a range of affected IPs with incorrect geolocation classifiers, and we're in the process of applying a rule to ensure any new messages are routed through the expected geolocation and delivered. Once this process is complete, we will focus on replaying the previously affected messages to release them from quarantine and fully resolve this issue. We anticipate having a better estimation for impact resolution by our next update.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Next update by: Wednesday, March 29, 2023, at 12:30 AM UTC
Published Time: 28.03.2023 17:18:21
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We suspect that this issue is caused by residual impact from a previous Azure Active Directory (Azure AD) issue in which users with geolocation-based conditional access policies were impacted. We're reviewing samples of affected users to investigate the affected API calls, confirm our hypothesis, and determine our next troubleshooting steps.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Next update by: Tuesday, March 28, 2023, at 10:30 PM UTC
Published Time: 28.03.2023 15:19:58
Title: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online
User impact: Users' incoming email is intermittently and unexpectedly quarantined in Exchange Online.
Current status: We're reviewing system logs and samples of affected messaged to isolate the origin of this issue.
Scope of impact: Your organization is affected by this event and any user may notice that some email that they view as legitimate are quarantined.
Next update by: Tuesday, March 28, 2023, at 8:30 PM UTC
Published Time: 28.03.2023 14:51:55
Title: We're looking into a potential problem
User impact: We're checking for potential impact to your users.
Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.
2015
