[HN Gopher] DNS glitch that threatened Internet stability fixed;...
___________________________________________________________________
DNS glitch that threatened Internet stability fixed; cause remains
unclear
Author : dangle1
Score : 75 points
Date : 2024-05-23 17:16 UTC (5 hours ago)
(HTM) web link (arstechnica.com)
(TXT) w3m dump (arstechnica.com)
| bewaretheirs wrote:
| Cogent's statement on c.root-servers.org:
|
| "2024-05-23 - On May 21 at 15:30 UTC the c-root team at Cogent
| Communications was informed that the root zone as served by
| c-root had ceased to track changes from the root zone publication
| server after May 18. Analysis showed this to have been caused by
| an unrelated routing policy change whose side effect was to
| silence the relevant monitoring systems. No production DNS
| queries went unanswered by c-root as a result of this outage, and
| the only impact was on root zone freshness. Root zone freshness
| as served by c-root was fully restored on May 22 at 16:00 UTC."
|
| Edit to add:
|
| This was mentioned on Tuesday on the dns-operations list:
|
| https://lists.dns-oarc.net/pipermail/dns-operations/2024-May...
| bewaretheirs wrote:
| The problem was evident if you ran
|
| dig @${server} . soa
|
| c.root-servers.net showed an older serial number than all the
| others yesterday.
|
| This seems like a major hole in their monitoring systems.
| gorkish wrote:
| If it takes 3 days to be noticed by an outside party, I dare say
| that whatever caused the issue probably isn't the first problem
| to fix.
| overstay8930 wrote:
| Internet stability is not threatened by the loss of a single root
| server, it's not even a blip on the radar. In fact most people
| wouldn't even notice if they all went down for a few minutes, DNS
| operators do not actually use these servers directly. You can
| even run your own if you are crazy enough.
|
| Everyone likes to meme on DNS but it's literally designed to make
| these servers as unimportant as possible. It's a really well
| designed system that will probably be around as long as humans
| exist.
| denton-scratch wrote:
| > that will probably be around as long as humans exist.
|
| My goodness, that's a strong claim to make, about a glue layer
| in a computing technology that's been around for just 50 years.
| I mean, arguably the internet as we knew it is already dying.
| The original design of DNS was flawed (no security), and the
| patches to fix those flaws are far from perfect.
| bdamm wrote:
| Replacing Internet infrastructure is incredibly difficult.
| We'll all be switching over to IPv6... any decade now.
|
| DNS was flawed, but it was very good for the time, and it
| still is the keystone around which the entire Internet
| functions. That's pretty incredible for a system that has
| indeed received a lot of patches. And DNSSEC might turn out
| to be just as well designed as the original DNS. Besides,
| what are we going to replace it with, namechain? I'll put you
| on hold while all the ISPs switch over.
| jazzyjackson wrote:
| it's Good Enough and Not Broke, so there's little incentive
| to iterate
|
| I wouldn't mind some alien technology taking over for TCP/IP
| but that's squarely within the "unforseeable" IMO
| adamomada wrote:
| A bit pessimistic, but maybe the subtext is we won't be
| around for very much longer
| jedberg wrote:
| > DNS operators do not actually use these servers directly.
|
| No, but a lot of servers do. In 2020 the root servers saw 120
| billion queries a day, which was double what they saw in 2018:
|
| https://blog.apnic.net/2020/09/28/scaling-the-root-of-the-dn...
|
| It's probably closer to half a trillion queries a day now.
| sebzim4500 wrote:
| IIRC a large chunk of those is spam from chrome which most
| people wouldn't notice if they stopped working
| deathanatos wrote:
| ... under what circumstance is Chrome hitting the root DNS
| servers?
| ectospheno wrote:
| Old bug fixed in 2020. OP didn't know of the fix it
| seems. Was covered on Hacker News at the time. Still top
| google result.
| deathanatos wrote:
| Well that's horrifying.
|
| For those that are wondering like me:
| https://blog.apnic.net/2020/08/21/chromiums-impact-on-
| root-d...
|
| Essentially, it generated 3 random, _single-label_ (so,
| "TLD") queries, and ran them through DNS. Whatever
| recurser that lands at obviously wouldn't have random
| junk in its cache (unlike probably any other root record)
| and thus had to go to the root servers for it.
|
| They did this to try to not display an omnibox result for
| single words (the example in the article is "marketing")
| on what I'm going to just call "broken networks"; see TFA
| if you're curious.
| bewaretheirs wrote:
| Note that running your own mirror of the root on the
| recursive nameserver(s) on your network will keep the
| traffic generated from these sorts of queries off the
| rest of the internet.
|
| If you're already running your own recursive resolver on
| reasonably up-to-date software it can be surprisingly
| easy to set this up; see rfc8806 for some config
| examples.
| Repulsion9513 wrote:
| Honestly the issue here isn't that they let the zone get stale.
| The root zone doesn't change that much. So a couple of TLDs had
| to hold off on making changes, who cares.
|
| The issue here is that they didn't notice until someone with a
| direct contact at Cogent emailed them to tell them.
|
| A corollary to that issue is the fact that the last news item is
| the _same thing_ : they failed to get updates and their
| monitoring failed to tell them. Supposedly their monitoring was
| fixed then, in 2019. Why should we believe it's any more fixed
| now?
| egberts1 wrote:
| Probably accidentially firewalled the DNS-XFER/IXFR port for a
| few days.
|
| I know this similar scenario happened to our lab setup of root
| servers.
| sparrish wrote:
| Cogent sucks. Their routing issues are so common and often
| severe. I seriously don't know how that org still exists.
| ipython wrote:
| Not surprising behavior from Cogent, who to this day refuses to
| peer with Hurricane Electric's IPv6 network [see
| https://en.wikipedia.org/wiki/Hurricane_Electric and related
| footnotes].
___________________________________________________________________
(page generated 2024-05-23 23:01 UTC)