https://arstechnica.com/security/2024/05/dns-glitch-that-threatened-internet-stability-fixed-cause-remains-unclear/ Skip to main content * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Subscribe [ ] Close Navigate * Store * Subscribe * Videos * Features * Reviews * RSS Feeds * Mobile Site * About Ars * Staff Directory * Contact Us * Advertise with Ars * Reprints Filter by topic * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Settings Front page layout Grid List Site theme light dark Sign in DNS DISARRAY -- A root-server at the Internet's core lost touch with its peers. We still don't know why. For 4 days, the c-root server maintained by Cogent lost touch with its 12 peers. Dan Goodin - May 23, 2024 5:10 pm UTC A root-server at the Internet's core lost touch with its peers. We still don't know why. Enlarge reader comments 54 For more than four days, a server at the very core of the Internet's domain name system was out of sync with its 12 root server peers due to an unexplained glitch that could have caused stability and security problems worldwide. This server, maintained by Internet carrier Cogent Communications, is one of the 13 root servers that provision the Internet's root zone, which sits at the top of the hierarchical distributed database known as the domain name system, or DNS. Here's a simplified recap of the way the domain name system works and how root servers fit in: When someone enters wikipedia.org in their browser, the servers handling the request first must translate the human-friendly domain name into an IP address. This is where the domain name system comes in. The first step in the DNS process is the browser queries the local stub resolver in the local operating system. The stub resolver forwards the query to a recursive resolver, which may be provided by the user's ISP or a service such as 1.1.1.1 or 8.8.8.8 from Cloudflare and Google, respectively. If it needs to, the recursive resolver contacts the c-root server or one of its 12 peers to determine the authoritative name server for the .org top level domain. The .org name server then refers the request to the Wikipedia name server, which then returns the IP address. In the following diagram, the recursive server is labeled "iterator." [iterative_DNS_resolver_process] Enlarge Lion Kimbro Given the crucial role a root server provides in ensuring one device can find any other device on the Internet, there are 13 of them geographically dispersed all over the world. Each root sever is, in fact, a cluster of servers that are also geographically dispersed, providing even more redundancy. Normally, the 13 root servers--each operated by a different entity--march in lockstep. When a change is made to the contents they host, it generally occurs on all of them within a few seconds or minutes at most. Advertisement Strange events at the C-root name server This tight synchronization is crucial for ensuring stability. If one root server directs traffic lookups to one intermediate server and another root server sends lookups to a different intermediate server, important parts of the Internet as we know it could collapse. More important still, root servers store the cryptographic keys necessary to authenticate some of intermediate servers under a mechanism known as DNSSEC. If keys aren't identical across all 13 root servers, there's an increased risk of attacks such as DNS cache poisoning. For reasons that remain unclear outside of Cogent--which declined to comment for this post--all 12 instances of the c-root it's responsible for maintaining suddenly stopped updating on Saturday. Stephane Bortzmeyer, a French engineer who was among the first to flag the problem in a Tuesday post, noted then that the c-root was three days behind the rest of the root servers. A mismatch in what's known as the zone serials shows root-c is three days behind. Enlarge / A mismatch in what's known as the zone serials shows root-c is three days behind. The lag was further noted on Mastodon. Optional caption By mid-day Wednesday, the lag was shortened to about one day. [root-c-wednesday-640x577] Enlarge By late Wednesday, the c-root was finally up to date. Page: 1 2 Next - reader comments 54 Dan Goodin Dan Goodin is Senior Security Editor at Ars Technica, where he oversees coverage of malware, computer espionage, botnets, hardware hacking, encryption, and passwords. In his spare time, he enjoys gardening, cooking, and following the independent music scene. Advertisement Promoted Comments [238795] Architect_of_Insanity It boggles my mind that alarms didn't go off all over the internet when one of the roots drifted by more than a few minutes - days just means we're not paying attention at all. JFC, this is a big deal. May 23, 2024 at 5:31 pm [avatar] arc-tu-rus What are the reasons behind Cogent, a private for-profit organization, running a dns root server? Afaik, only two out of the 12 organizations managing root are servers are for profit organizations. How is transparency and independence guaranteed in cases like this? May 23, 2024 at 5:37 pm Channel Ars Technica - Previous story Next story - Related Stories Today on Ars * Store * Subscribe * About Us * RSS Feeds * View Mobile Site * Contact Us * Staff * Advertise with us * Reprints Newsletter Signup Join the Ars Orbital Transmission mailing list to get weekly updates delivered to your inbox. Sign me up - CNMN Collection WIRED Media Group (c) 2024 Conde Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1 /20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy. Your California Privacy Rights | [privacyopt] Do Not Sell My Personal Information The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices