https://www.theregister.com/2021/12/22/aws_outage/ [user] [user] Sign in The Register(r) -- Biting the hand that feeds IT [magn] [burg] [burg] Topics Security Off-Prem All Off-PremEdge + IoTChannelPaaS + IaaSSaaS (X) On-Prem All On-PremServersStorageNetworksHPCPersonal Tech (X) Software All SoftwareAI + MLApplicationsDatabasesDevOpsOSesVirtualization (X) Offbeat All OffbeatDebatesColumnistsScienceGeek's GuideBOFHLegalBootnotesSite NewsAbout Us (X) Vendor Voice All Vendor VoiceAdobeAmazon Web Services (AWS)Amazon Web Services (AWS) MigrationGoogle CloudGoogle Cloud's ApigeeGoogle Workspace LenovoNutanixRapid7Red hatSophosVeeamVirtru (X) Resources * Whitepapers * Webinars * Newsletters Situation Publishing * The Next Platform * Devclass * Blocks and Files Get our Weekly newsletter [paasiaas] PaaS + IaaS AWS power failure in US-EAST-1 region killed some hardware and instances Some servers and EBS volumes won't make it back Simon Sharwood, APAC Editor Wed 22 Dec 2021 // 23:58 UTC 35 comment bubble on white --------------------------------------------------------------------- 35 comment bubble on white # reddit Twitter Facebook linkedin WhatsApp email [https://www.theregis] Copy A small group of sysadmins have a disaster recovery job on their hands, on top of Log4J fun, thanks to a power outage at Amazon Web Services' USE1-AZ4 Availability Zone in the US-EAST-1 Region. The lack of fun kicked off at 04:35AM Pacific Time (PST - aka 12:35PM UTC) on December 22nd, when AWS noticed launch failures and networking issues for some instances in its Elastic Compute Cloud IaaS service. 26 minutes later the cloud colossus 'fessed up to a power outage and recommended moving workloads to other parts of its cloud that were still receiving electricity. [paasiaas] Power was restored at 05:39AM PST and AWS reported slow recovery of services, however a 6:51AM update admitted that ongoing networking issues were hampering efforts at full restoration. [paasiaas] [paasiaas] Services including Slack and Asana reported service difficulties as a result of AWS' mess. At the time of writing, AWS has still not fully restored networking. [paasiaas] And restoration may not be possible for some customers: at the time of writing, the most recent update on AWS' status page offers the following grim news: As is often the case with a loss of power, there may be some hardware that is not recoverable, which will prevent us from fully recovering the affected EC2 instances and EBS volumes. We are not quite at that point yet in terms of recovery, but it is unlikely that we will recover all of the small number of remaining EC2 instances and EBS volumes. That's the digital equivalent of waking up to a lump of coal on Christmas Day. The incident is AWS' second outage in a fortnight: on December 15th the operation's US-WEST-1 went missing for around 30 minutes. The US-EAST-1 region also browned out for eight hours in September 2021. AWS advises customers not to rely on a single Availability Zone (AZ). The outfit's architecture places two or more AZs within a single Region, and each Zone is physically distant from the others so that a single physical infrastructure incident can't take out the whole Region. Using multiple regions therefore improves resilience - and cost. * AWS flicks the switch on an Indonesian region * AWS postmortem: Internal ops teams' own monitoring tools went down, had to comb through logs * The big AWS event: 120 announcements but nothing has changed Not every user follows AWS' guidance about using multiple AZs, so when incidents like this strike their servers and data will become unavailable. US-EAST-1 is AWS' biggest and oldest region. Cloud economist Corey Quinn rates its importance as follows: A multi-day full outage of us-east-1 will have an observable effect on the world economy. That is not an exaggeration. -- Corey Quinn (@QuinnyPig) December 8, 2021 AWS offers a service level agreement of 99.95 per cent uptime for compute instances - or just under 22 minutes a month of downtime. If AWS misses that mark, it offers a ten per cent service credit, a sum that grows to thirty per cent if uptime drops below 99 per cent. If uptime falls below 95 per cent, customers are given 100 per cent of their fees as credits. AWS also automatically waives fees if EC2 Instance are unavailable for more than six minutes inside a single hour. Good luck if you're one of the AWS customers faced with the need for a sudden rebuild. (r) Get our Tech Resources # Share reddit Twitter Facebook linkedin WhatsApp email [https://www.theregis] Copy 35 Comments Similar topics * AWS * Cloud Computing * Outage Broader topics * Amazon * Jeff Bezos Narrower topics * Azure * Edge Computing * Google Cloud Platform * Hybrid Cloud * iCloud * Kubernetes * Paas * Private Cloud * Public Cloud * S3 * Serverless Corrections Send us news --------------------------------------------------------------------- [paasiaas] Other stories you might like * Tencent unexpectedly offloads stake in fellow Chinese giant JD.com Surprise dividend for shareholders may also yield approval from monopoly-wary Beijing Simon Sharwood, APAC Editor Thu 23 Dec 2021 // 15:55 UTC comment bubble on black While Chinese web giant Tencent has announced an unheralded special dividend: shares in giant Chinese e-tailer JD.com. Tencent acquired around 15 per cent of JD.com in 2014. Now it will divest around 86.4 per cent of that stake - or 457,326,671 shares. The company's Tencent's filing [PDF] announcing the deal spells out that 21 Tencent shares entitles investors to one JD share. The filing explains the timing of the dividend by explaining that Tencent likes to invest in companies until they are "consistently capable of self-financing their future initiatives." JD.com has reached that stage, the filing explains, so shareholders deserve to benefit from Tencent's investment. Continue reading * Db2, where are you? Big Blue is oddly reticent to discuss recent enhancements to its flagship database Cloud-native RDBMS is coming, on AWS and Azure. Yet IBM's only preaching to the choir Lindsay Clark Thu 23 Dec 2021 // 12:02 UTC 21 comment bubble on white It's the time of year when one might wonder what happened to that avuncular family figure whose existence was so reliably dull they passed into history almost forgotten - a little like Db2, IBM's flagship relational database that has faded from users' collective memory. Big Blue's system of choice for its mainframes and big Unix/Linux boxes is still very much alive and kicking and has even delivered a smattering of news in recent weeks, with a high profile attendee at the early December International Db2 Users Group European Conference telling us that select users were told of the new "Db2u", a set of containers for Db2 aimed at users exploring or working with the database in the cloud. Conference speaker, Db2 DBA, and commentator Ember Crooks had posted earlier this year that "db2u sounds amazing to me" and that "db2u, conceptually, is a nearly cloud-native way of doing a relational database management system." But she also pointed out that "the problem I run into is that db2u is only available on RedHat OpenShift." Continue reading * Wifinity hands customers bills for Wi-Fi services they didn't want but used by accident after software 'glitch' let 'fixed term' subs continue Firm admits problem and then tries to cash in from own screwup Gareth Corfield Thu 23 Dec 2021 // 09:56 UTC 14 comment bubble on white Members of the British Armed Forces Soldiers have reacted with anger after a British Wi-Fi provider failed to automatically end their time-limited contracts before charging them for consuming data "without subscription". Ofcom-regulated Wifinity prompted a wave of fury when it told former subscribers on Ministry of Defence bases that they would be receive bills running into the hundreds of pounds for data services that former customers didn't want and hadn't signed up for. Those bills were then "revised" without explanation down to sums of PS10-PS30, as seen by The Register, in what one ex-customer told us was a cynical attempt to trick recipients into paying up to avoid further hassles. Continue reading * Fisher Price's Bluetooth reboot of pre-school play phone has adult privacy flaw 'Chatter' can be bugged thanks to kindergarten-grade security Simon Sharwood, APAC Editor Thu 23 Dec 2021 // 08:02 UTC 39 comment bubble on white A Bluetooth phone designed to evoke the carefree days of early childhood has been found to instead threaten the very adult prospect of being surveilled in your home. The phone is the Fisher Price Chatter Special Edition, a device that adds Bluetooth and a speaker to the smiling, brightly coloured, wheeled, rotary dial phone on which it's previously been possible to make calls only by using one's imagination. The phone also bears the name "60G LTE" - which stands for "60 great years, Let's Talk Everywhere" and an infomercial for the handset opens with "The past has finally arrived" before lampooning mobile phone ads quite nicely. Continue reading * Alibaba Cloud slapped by Chinese ministry for mishandling Log4j Beijing's not saying what cloudy contender did wrong Laura Dobberstein Thu 23 Dec 2021 // 05:58 UTC 9 comment bubble on white China's Ministry of Industry and Information Technology has suspended Alibaba Cloud's membership of an influential security board to protest its handling of the Log4j flaw. The move appears odd as The Apache Software Foundation credited Alibaba Cloud's Chen Zhaojunfor identifying and reporting the Log4J flaw in the first place. You might think Alibaba Cloud deserves a parade for identifying a dangerous flaw, and showing that Chinese bug-hunters can match it with the world's best. But according to Chinese outlet The 21st Century Herald, Chinese authorities were displeased with the cloud giant's response. Continue reading * James Webb Telescope launch delayed again, this time by weather New launch date could be a marvelous Christmas gift to humanity Simon Sharwood, APAC Editor Thu 23 Dec 2021 // 03:58 UTC 20 comment bubble on white The James Webb Telescope has been cleared for launch, only for weather to delay its ascent for at least a day. Work on the 'scope commenced in 1996, ahead of a planned 2007 launch. The instrument's journey to the launchpad has been long and hard: in 2005 it was redesigned to control cost overruns, construction occupied another 11 years, then the craft was damaged during tests in 2018. Then came COVID, which in 2020 slowed the process of integrating the 'scope with the Ariane rocket that will (eventually) lift it into orbit. Further issues with that lifter, and the 'scope, saw October 2021 slated as the launch date. Continue reading * Would you like a side of data with your chips? Silicon-slingers start bundling info with their hardware Intel, Nvidia, and others have figured out you need help getting started with AI Agam Shah in New Jersey Thu 23 Dec 2021 // 01:59 UTC 2 comment bubble on white Some chip makers are starting to supply data as a value-added service. Silicon-slingers, especially those with products for AI, know that their products are worthless without information from which knowledge can be extracted; it's like selling an empty brain. As a result, they're building or acquiring data sets to make their products more attractive. Continue reading * Electric fastback fun: Now you can surf the web from the driving seat of your Polestar 2 With Vivaldi. While parked Richard Speed Wed 22 Dec 2021 // 12:01 UTC 26 comment bubble on white Chromium-based browser Vivaldi has ported its eponymous Android Tablet browser to Android Automotive, making it the first web browser on the operating system and available in the Polestar 2 electric car. Continue reading * Boffins' first take on asteroid dust from Japanese probe: Carbon rich, less lumpy than expected The five grams of Ryugu we got suggests it's made of the same stuff as the outer Sun Laura Dobberstein Wed 22 Dec 2021 // 10:03 UTC 69 comment bubble on white Researchers have published the first analyses of samples plucked from asteroid 162173 Ryugu by Japan's spacecraft Hayabusa2, revealing, for the first time, the physical properties and composition of a carbonaceous asteroid. The 5.4g of asteroid sample collected from two surface locations on asteroid Ryugu landed in the South Australian outback a year ago before being shipped to Japan for investigation. Some of the space pebbles went to NASA, but the bulk remained with Japan's Aerospace Exploration Agency JAXA and its scientists. Continue reading * A proposal to beat below-the-belt selfies: Crowdsourced machine learning using victims' image stashes Empty your inboxes, people, we can duck this problem together Mark Pesce Wed 22 Dec 2021 // 08:05 UTC 63 comment bubble on white Column It's often said that the second most important job in Australia - behind Prime Minister - is captain of the Australian men's cricket team. The last holder of that office recently resigned in disgrace after it emerged a flurry of texts he exchanged with a staffer crossed the line from racy to harassment as one message included an unsolicited dick pic. After his resignation, a friend who is both a cricket fan and an AI expert messaged me. Continue reading * Nvidia says its SmartNICs sizzled to world record storage schlepping status But did so operating as NICs - the real fun starts in mid-2022 when VMware's hypervisor for SmartNICs should land Simon Sharwood, APAC Editor Wed 22 Dec 2021 // 06:02 UTC 7 comment bubble on white Nvidia has claimed a world record for storage IOPS using its Bluefield data processing units (DPUs). The Bluefield kit is Nvidia's take on a "SmartNIC" - a network interface card bulked up with a decent CPU so it can run workloads such as firewalls or encryption engines, so that CPUs can be freed for more important tasks. Hyperscalers routinely use SmartNICs and the likes of Nvidia, Intel, and VMware are working to bring them into mainstream data center scenarios. Nvidia's benchmark saw a pair of HPE ProLiant DL380 Gen 10 Plus servers hooked up with a dual-100GB ethernet Bluefield aboard each, in a configuration that offered 400Gb/s wire bandwidth between initiator and target. The company used the Storage Performance Development Kit (SPDK) and FIO test, and also targeted the Linux kernel. Continue reading ABOUT US* * Who we are * Under the hood * Contact us * Advertise with us MORE CONTENT* * Latest News * Popular Stories * Forums * Whitepapers * Webinars SITUATION PUBLISHING* * The Next Platform * DevClass * Blocks and Files * Continuous Lifecycle London * M-cubed Situation Publishing The Register - Independent news and views for the tech community. Part of Situation Publishing SIGN UP TO OUR DAILY NEWSLETTER Subscribe Twitter Facebook LinkedIn feeds no-js Biting the hand that feeds IT (c) 1998-2021 Do not sell my personal information Cookies Privacy Ts&Cs