Am 10. August 2022 kam es bei den Diensten von Microsoft 365 zu einem Ausfall, der speziell Nordamerika aber auch den EMEA-Raum betraf. Nutzer hatten Probleme mit Office 365, Outlook und weiteren Diensten. Es lag wohl an einem Update für die Cisco Meraki-Firewall, die von Microsoft verwendet wird.
Anzeige
Auf Twitter hat Microsoft Probleme mit seinem Cloud-Diensten eingestanden, wie nachfolgende Tweets zeigen. Bereits in der initialen Nachricht wurde bestätigt, dass der Netzwerkverkehr über verschiedene Regionen blockiert war.
Schnell war klar, dass das Ganze mit von Microsoft verwendeten Firewall-Lösungen zusammen hängt. Blog-Leser Markus wies mich per Mail auf diesen Beitrag von Bleeping Computer hin (danke dafür). Der Ausfall wurde von einem Fehlalarm in der eingesetzten Cisco Meraki-Firewall ausgelöst, der verhinderte, dass sich Benutzer mit Exchange Online, Microsoft Teams, Outlook-Desktopclients und OneDrive for Business verbinden konnten.
Ein Mitarbeiter von Cisco hat das in diesem Forenbeitrag angesprochen. Eine von Microsoft gemeldete Sicherheitslücke CVE-2022-35748 löst die die SNORT-Regel 1-60381 aus, was zu Problemen mit der Kommunikation durch die Firewall führte. Inzwischen hat Microsoft das Problem aber wohl behoben (siehe auch die Hinweise bei Bleeping Computer). Blog-Leser Andreas P. hat mir noch nachfolgende Auszüge aus dem Statusbereich des Admin Centers zukommen lassen (danke dafür).
Published Time: 10.08.2022 19:56:28
The firewall partner is currently reviewing options to remediate impact.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 19:02:39
Title: Some users may be unable to connect to multiple Microsoft 365 services.
User Impact: Users may be unable to connect to multiple Microsoft 365 services.
More info: Impacted services include, but are not limited to:
– Outlook desktop client
– OneDrive for Business
– Microsoft Teams
Affected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact. Some firewall vendors have published guidance on disabling the impacting rules, and we recommend contacting your firewall vendor for further assistance.
Current status: We continue to work with the firewall partner to investigate a Snort rule which is contributing to impact. Our focus remains on mitigation and from user reports, disabling the specific firewall rule provides immediate relief. Additionally, we continue to investigate recent changes within the Microsoft-managed environment to rule out potential causes of impact.
Scope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.
Next update by: Wednesday, August 10, 2022, at 6:30 PM UTC
Published Time: 10.08.2022 18:41:46
We're continuing to work with the firewall partner to investigate the issue. Additionally, we monitoring feedback from impacted organizations that disabling a specific firewall rule, which blocks TLS 1.2, is mitigating impact.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 17:25:42
Title: Some users may be unable to connect to multiple Microsoft 365 services.
User Impact: Users may be unable to connect to multiple Microsoft 365 services.
More info: Impacted services include, but are not limited to:
– Outlook desktop client
– OneDrive for Business
– Microsoft Teams
Affected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact.
Current status: We've identified an increase in errors related to TLS 1.0 and 1.1 across Microsoft 365 services. We've confirmed that there have not been any recent changes to the service feature which is blocking the traffic. We're continuing to engage with the firewall partners to assist our investigation into the potential blocking of legitimate traffic. Additionally, we're working with impacted users to gather client logs.
Scope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.
Next update by: Wednesday, August 10, 2022, at 5:00 PM UTC
Published Time: 10.08.2022 17:01:14
We're directly working with some of the affected users to aid in our investigation while continuing to engage with our firewall partners. Analysis into Microsoft 365 client endpoints is ongoing.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 16:28:30
We're looking at recent changes made within the Microsoft-managed infrastructure and reviewing endpoints that are leveraging TLS 1.2. Additionally, we're contacting firewall partners to assist our investigation.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 15:53:29
Title: Some users may be unable to connect to multiple Microsoft 365 services.
User Impact: Users may be unable to connect to multiple Microsoft 365 services.
More info: Impacted services include, but are not limited to:
– Outlook desktop client
– OneDrive for Business
– Microsoft Teams
Current status: After analyzing system telemetry and Fiddler logs from impacted users, we suspect that third-party firewall devices are potentially blocking legitimate Microsoft traffic. Affected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact. We're continuing our investigation into the underlying cause.
Scope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.
Next update by: Wednesday, August 10, 2022, at 3:30 PM UTC
Published Time: 10.08.2022 15:25:09
Some customers are able to mitigate impact by disabling a firewall rule that is blocking TLS 1.2.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 14:57:17
We're reviewing Exchange trace logs (ETL) from users who are experiencing impact. We believe the issue may be related to Active Directory (AD) services and are investigating this further.
This quick update is designed to give the latest information on this issue.
Published Time: 10.08.2022 14:41:52
Title: Some users may be unable to connect to multiple Microsoft 365 services.
User Impact: Users may be unable to connect to multiple Microsoft 365 services.
More info: Impacted services include, but are not limited to:
– Outlook desktop client
– OneDrive for Business
– Microsoft Teams
Current status: We're reviewing system telemetry to isolate the source of the issue. Additionally, we're working with impacted users to gather network trace logs to assist our investigation.
Scope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.
Next update by: Wednesday, August 10, 2022, at 2:00 PM UTC
2015
Hier sind wahrscheinlich die Cisco Meraki Firewalls gemeint, unter dem Begriff Merics Firewall konnte ich nichts finden. Aber wie immer: vielen Dank für die ausführlichen Infos mit Hintergrund Informationen
Du hast Recht – Asche auf mein Haupt – war die Nacht irgendwie schon arg dunkel. Es ist korrigiert.
Meraki ist eine Firma, der ich alleins schonnwegen Ihrem merkwürdigen Lizenzmodell (Hardware fast verschenken, die ohne aktuelle Lizenz nutzlos ist) nicht über den Weg traue. Das erinnert mich zu sehr an die Tintendruckerhersteller.
Das hier
> https://www.reddit.com/r/sysadmin/comments/wkvbfr/comment/ijqdofi/?utm_source=reddit&utm_medium=web2x&context=3
liest sich für mich so, als würde ausgehend vom CVE dann jemand (hoffentlich nicht-automatisiert, aber weiß) Snort-Regeln erstellen, und die gehen dann halt in die Welt hinaus.
Ebenso liest es sich für mich so, dass das CVE quasi "zu schlecht" geschrieben und somit die Rule zu breit war. Womit MS ja dann doch wieder selbst Schuld wäre, weil das CVE von MS kam.
Kennt sich in dem Bereich einer besser aus und kann hier noch Input geben?
Ich sehe auf der CVE Seite nur Manager-Bla, viele Referenzen sind schon gelöscht und ich finde es gerade mega unnachvollziehbar -_-