https://arstechnica.com/information-technology/2021/08/worst-cloud-vulnerability-you-can-imagine-discovered-in-microsoft-azure/ Skip to main content * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Subscribe [ ] Close Navigate * Store * Subscribe * Videos * Features * Reviews * RSS Feeds * Mobile Site * About Ars * Staff Directory * Contact Us * Advertise with Ars * Reprints Filter by topic * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Settings Front page layout Grid List Site theme Black on white White on black Sign in Comment activity Sign up or login to join the discussions! [ ] [ ] [Submit] [ ] Stay logged in | Having trouble? Sign up to comment and more Sign up cloudy with a chance of compromise -- "Worst cloud vulnerability you can imagine" discovered in Microsoft Azure 30% of Cosmos DB customers were notified--more are likely impacted. Jim Salter - Aug 27, 2021 9:00 pm UTC Cosmos DB is a managed database service offering--including both relational and noSQL data structures--belonging to Microsoft's Azure cloud infrastructure. Enlarge / Cosmos DB is a managed database service offering--including both relational and noSQL data structures--belonging to Microsoft's Azure cloud infrastructure. Microsoft reader comments 150 with 81 posters participating, including story author Share this story * Share on Facebook * Share on Twitter * Share on Reddit Cloud security vendor Wiz announced yesterday that it found a vulnerability in Microsoft Azure's managed database service, Cosmos DB, that granted read/write access for every database on the service to any attacker who found and exploited the bug. Although Wiz only found the vulnerability--which it named "Chaos DB"--two weeks ago, the company says that the vulnerability has been lurking in the system for "at least several months, possibly years." A slingshot around Jupyter * Jupyter notebook functionality in CosmosDB enables many advanced data visualization techniques with relatively little coding experience or effort. Wiz * A privilege escalation vulnerability allowed anyone with a Cosmos DB account to filch the private key for any other Cosmos DB account, by way of the Jupyter notebook functionality. Wiz * Once an attacker has the victim's primary key, it's game over--full read/write/delete access is granted permanently, and cannot be revoked without replacing the affected keys. Wiz Further Reading Cloudy with a chance of neurons: The tools that make neural networks work In 2019, Microsoft added the open-source Jupyter Notebook functionality to Cosmos DB. Jupyter Notebooks are a particularly user-friendly way to implement machine learning algorithms; Microsoft promoted Notebooks specifically as a useful tool for advanced visualization of data stored in Cosmos DB. Jupyter Notebook functionality was enabled automatically for all Cosmos DB instances in February 2021, but Wiz believes the bug in question likely goes back further--possibly all the way back to Cosmos DB's first introduction of the feature in 2019. Wiz isn't giving away all the technical details yet, but the short version is that misconfiguration in the Jupyter feature opens up a privilege escalation exploit. That exploit could be abused to gain access to other Cosmos DB customers' primary keys--according to Wiz, any other Cosmos DB customer's primary key, along with other secrets. Access to a Cosmos DB instance's primary key is "game over." It allows full read, write, and delete permissions to the entire database belonging to that key. Wiz's Chief Technology Officer Ami Luttwak describes this as "the worst cloud vulnerability you can imagine," adding, "This is the central database of Azure, and we were able to get access to any customer database that we wanted." Advertisement Long-lived secrets Unlike ephemeral secrets and tokens, a Cosmos DB's primary key does not expire--if it has already been leaked and is not changed, an attacker could still use that key to exfiltrate, manipulate, or destroy the database years from now. According to Wiz, Microsoft only emailed 30 percent or so of its Cosmos DB customers about the vulnerability. The email warned those users to rotate their primary key manually, in order to make certain that any leaked keys are no longer useful to attackers. Those Cosmos DB customers are the ones which had Jupyter Notebook functionality enabled during the week or so in which Wiz explored the vulnerability. Since February 2021, when all new Cosmos DB instances were created with Jupyter Notebook functions enabled, the Cosmos DB service automatically disabled Notebook functionality if it wasn't used within the first three days. This is why the number of Cosmos DB customers notified was so low--the 70 percent or so of customers not notified by Microsoft had either manually disabled Jupyter or had it disabled automatically due to lack of use. Unfortunately, this doesn't really cover the full scope of the vulnerability. Because any Cosmos DB instance with Jupyter enabled was vulnerable, and because the primary key is not an ephemeral secret, it is impossible to know for certain who has the keys to which instances. An attacker with a specific target could have quietly harvested that target's primary key but not done anything obnoxious enough to be noticed (yet). We also can't rule out a broader impact scenario, with a hypothetical attacker who scraped the primary key from each new Cosmos DB instance during its initial three-day vulnerability window, then saved those keys for potential later use. We agree with Wiz here--if your Cosmos DB instance might ever have had Jupyter notebook functionality enabled, you should rotate its keys immediately to ensure security going forward. Microsoft's response Microsoft disabled the Chaos DB vulnerability two weeks ago--less than 48 hours after Wiz privately reported it. Unfortunately, Microsoft cannot change its customers' primary keys itself; the onus is on Cosmos DB customers to rotate their keys. According to Microsoft, there's no evidence that any malicious actors found and exploited Chaos DB prior to the Wiz discovery. An emailed statement from Microsoft to Bloomberg said, "We are not aware of any customer data being accessed because of this vulnerability." In addition to warning 3,000+ customers of the vulnerability and providing mitigation instructions, Microsoft paid Wiz a $40,000 bounty. Promoted Comments * ReaderBot Ars Praefectus jump to post SamStephens wrote: Sorry, but whilst lots of S3 is complex, including the billing model, making sure your buckets are not publicly accessible is incredibly easy. When you create a new bucket from the console, the bucket defaults to not being public, and completely blocking being public. You have to explicitly chose to expose a bucket publicly. As far as creating buckets programatically goes, securing them from public view is as simple as Code: ExampleBucket: Type: AWS::S3::Bucket Properties: PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true You missed Code: AutoEnrolledInExpermentalBugRiddenFeatures: default 3788 posts | registered 7/6/2007 reader comments 150 with 81 posters participating, including story author Share this story * Share on Facebook * Share on Twitter * Share on Reddit Jim Salter Jim is an author, podcaster, mercenary sysadmin, coder, and father of three--not necessarily in that order. Email jim.salter@arstechnica.com // Twitter @jrssnet Advertisement You must login or create an account to comment. Channel Ars Technica - Previous story Next story - Related Stories Today on Ars * Store * Subscribe * About Us * RSS Feeds * View Mobile Site * Contact Us * Staff * Advertise with us * Reprints Newsletter Signup Join the Ars Orbital Transmission mailing list to get weekly updates delivered to your inbox. Sign me up - CNMN Collection WIRED Media Group (c) 2021 Conde Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1 /20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy. Your California Privacy Rights | Do Not Sell My Personal Information The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices