https://doublepulsar.com/rackspace-cloud-office-suffers-security-breach-958e6c755d7f Open in app Home Notifications Lists Stories --------------------------------------------------------------------- Write DoublePulsar Published in DoublePulsar Kevin Beaumont Kevin Beaumont Follow Dec 3 * 4 min read * Member-only Save Rackspace Cloud Office suffers security breach Thousands of small to medium size businesses are suffering as Rackspace have suffered a security incident on their Hosted Exchange service. Yesterday, 2nd December 2022, Rackspace announced an outage to their Hosted Exchange Server: Updated followed through the day, but were a little vague: I got involved in the end, as I noticed something and documented it in this thread: Kevin Beaumont (@GossiTheDog@cyberplace.social) Attached: 3 images #Rackspace's managed Exchange documentation points to Exchange clusters which are all offline. When... cyberplace.social Chiefly at Rackspace's managed service uses the hostnames mex*.emailsrvr.com for Exchange and OWA: And then when looking at the most recent Shodan data, it was clear the Exchange cluster was showing Exchange long build numbers that were old: This Exchange build number is from August 2022, before the ProxyNotShell patches became available: Exchange long build numbers aren't always reliable... but it is a sign worth keeping in mind. I wrote about ProxyNotShell, a vulnerability I named -- sorry, here: ProxyNotShell -- the story of the claimed zero days in Microsoft Exchange | by Kevin Beaumont | DoublePulsar Early this morning, Rackspace clarified it is a security incident: Threat Context Now, it is possible the Rackspace breach happened due to other issues -- but as a general reminder I would suggest a few key points on the threat: * The Microsoft supplied mitigations for ProxyNotShell are bypassable. IIS Rewrite, which Microsoft used for mitigations, doesn't decode all URLs correct and as such can be bypassed for exploitation. If you relied on the PowerShell mitigation or EEMS application, your Exchange Server is still vulnerable -- Microsoft just haven't told you this clearly. The fix is to patch. * Although the vulnerability needs authentication, the exploits work without multi-factor authentication as Exchange Server doesn't yet support Modern Authentication at all, as Microsoft deprioritised the implementation work (covered in the other blog). * If you are an MSP running a shared cluster, such as Hosted Exchange, it means that one compromised account on one customer will compromise the entire hosted cluster. This is high risk. Threat response * It is very important that Exchange Server administrators get to both the latest Cumulative Update and Security Update for Exchange Server 2013/2016/2019. * In the case of Exchange Server 2013, this is CU23 Nov22SU aka build 10.0.1497.44. You can check your build numbers from Shodan, or Exchange administrators can use this Powershell below. Double check. $ExchangeServers = Get-ExchangeServer | Sort-Object Name ForEach ($Server in $ExchangeServers) { Invoke-Command -ComputerName $Server.Name -ScriptBlock { Get-Command Exsetup.exe | ForEach-Object { $_.FileversionInfo } } } * If you run Exchange in Hybrid mode or have on prem servers for management -- which is extremely common for almost every Exchange Online customer -- you still need to take action. The Microsoft blog says no action is required for Exchange Online. This is wordsmithing by Microsoft, and is wrong -- those on prem Exchange Servers you forgot about and rely on still need fixing. * If you use Microsoft Defender Vulnerability Management, check the build numbers in product rather than relying on the identified vulnerabilities (you'll notice why). * If you outsource Microsoft Exchange management and you have resources, you should request proof that each server is patched for every update. I realise most orgs who outsource won't have this resource. * You should sweep your Exchange Servers, if you had OWA presented to the internet in particular, for webshells (backdoors for later) and signs of post compromise activity. The easiest way to do this to run Microsoft Safety Scanner and Sophos Scan & Clean, which are one time scan tools, to automatically sweep and clean a system. If you are a Rackspace Cloud Office customer, they are saying recovery may take days. I have heard it may be a destructive attack, that could take an extended period for recovery. You should contact Rackspace support for assistance. Exchange Server patching is extremely convoluted, in some cases risky -- how many have us admins have had servers fail? -- and confusing. Microsoft need to modernise it. Also, Modern Authentication needs implementing in Exchange Server by Microsoft -- basic authentication is not okay in the space year 2022. I expect continued attacks on organisations via Microsoft Exchange through 2023. And for those who say 'use Exchange Online, fools!', there are some challenges around that still: The German federal data protection authority and 17 state regulators (DSK) published a report into Microsoft 365 after two years of work with Microsoft, and declare Microsoft 365 doesn't yet meet GDPR citing a wide range of issues. The result is MS customers and MS engineers are being left in this situation: Some rights reserved -- -- More from DoublePulsar Follow Cybersecurity from the trenches, written by Kevin Beaumont. Opinions are of the author alone, not their employer. Read more from DoublePulsar Recommended from Medium Skylar Skylar TryHackMe: Simple CTF Walkthrough [1] AMATERAS AMATERAS AMATERAS IDO Whitelist Announced [?] [1] Filip Olszak Filip Olszak Analysing maldocs with x64dbg and Process Hacker -- Part 2: Shellcode [1] Valera Fadil Valera Fadil {UPDATE} Peter & Wendy in Neverland Hack Free Resources Generator UltimateDAO UltimateDAO UltimateDAO: An Algorithmic Currency Protocol on Solana [1] Seif Hateb Seif Hateb Cybersecurity And Much More Newsletter -- Week 10 (2022) [1] Crocodile Jamie Crocodile Jamie And don't forget to insure your degree while you're doing the household appl Swarm Hive Swarm Hive in Ethereum Swarm We Are Millions -- Liberate Data Day -- Recap [0] AboutHelpTermsPrivacy --------------------------------------------------------------------- Get the Medium app A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store Get started [ ] Kevin Beaumont Kevin Beaumont 3.6K Followers Everything here is my personal work and opinions. Follow Help Status Writers Blog Careers Privacy Terms About Text to speech