[HN Gopher] HE.net problem
       ___________________________________________________________________
        
       HE.net problem
        
       Author : greyface-
       Score  : 68 points
       Date   : 2024-07-04 19:52 UTC (3 hours ago)
        
 (HTM) web link (mailman.nanog.org)
 (TXT) w3m dump (mailman.nanog.org)
        
       | bediger4000 wrote:
       | Is there some context that would aid in understanding this?
       | Because it doesn't make sense.
        
         | trvrprkr wrote:
         | Hurricane Electric (HE) provides, among other things, DNS
         | services. When a domain is placed in `clientHold`, as has
         | happened to HE due to a spurious phishing report, it causes the
         | domain to no longer resolve. So the DNS records for HE and _all
         | of HE 's customers_ are gradually becoming unresolvable as
         | caches expire.
        
           | KennyBlanken wrote:
           | For anyone not well versed in DNS who needs an ELI5 level:
           | 
           | This impacts anyone using HE.net for their domain name
           | service because a domain name specifies its nameservers (in
           | an NS record) using fully qualified domain names. I believe
           | it's ns1.he.net and ns2.he.net.
           | 
           | User at WildISP tries to go to a site. Resolving nameserver
           | at WildISP.net fetches the NS records for
           | samplewhamplesite.com from a root server and gets pointed to
           | ns1.he.net...which no longer exists because the entire domain
           | disappeared from the root servers.
           | 
           | It looks like things are back up?
           | 
           | Edit: nope, still in clienthold.
        
           | dave78 wrote:
           | They offer a nice (free) secondary DNS service, which I use
           | for all my domains. From my limited interaction with them,
           | they seem like a great company. They also provide some nice
           | IPv6 tutorials as well, and offer free IPv6 tunnels for those
           | who can't get native IPv6. All in all they seem like a model
           | company that believes in giving back to the internet
           | community in meaningful ways.
           | 
           | Unfortunately, I use them to host secondary DNS for all my
           | domains, so this Network Solutions stupidity is hitting home
           | for me right now. Then again, I had enough issues with NS
           | back in the 1990s that as soon as that monopoly was broken I
           | switched and never looked back. TBH I'm a little surprised
           | that HE would use NS, but perhaps they did thinking that NS
           | would provide proper enterprise-level support for a major
           | backbone provider and not cut them off on a holiday with no
           | recourse - behavior you might expect or fear from a cut-rate
           | provider.
        
           | toast0 wrote:
           | Edit: since the domain is fixed, this isn't reproducable
           | anymore.
           | 
           | An interesting thing is that if you have a .net domain with
           | he.net nameservers, the .net authoritative servers will give
           | you full glue records with the A/AAAA for nsX.he.net. But if
           | you ask for he.net, you get back NXDOMAIN.
           | 
           | Example domain removed; no longer relevant.
           | 
           | NetworkSolutions is a truly terrible registrar, and anyone
           | who still has a domain there should use this occasion to
           | switch to somewhere better, which is almost anywhere. IMHO,
           | it may be worthwhile for a real business where their domain
           | is important to move to an expensive corporate registrar like
           | MarkMonitor or CSC, and looking into registry lock, etc.
        
         | LinuxBender wrote:
         | Looking to see what escalation numbers I may still have...
         | 
         | [Edit]: Sent them what I have but it isn't much. Most human
         | contacts have been replaced by menu's. I hope the last one I
         | sent helps.
         | 
         | It appears someone managed to get NetworkSolutions (now
         | web.com) to put the domain he.net on clientHold for one
         | Phishing complaint, unrelated to he.net meaning someone did a
         | lookup on bgp.he.net and reported he.net instead of the domain
         | they were looking up. If true it means web.com did not do any
         | due diligence, investigation, etc... and just put a very
         | popular domain with many shared services on hold. The domain
         | he.net is used by MANY services and those services are likely
         | going to start breaking. I have no idea what's going on over at
         | web.com.
         | 
         | As a side note, I recently transferred all my domains away from
         | NetSol/web.com as they lost the ability to define custom name
         | servers (glue in the root servers) which is bizarre to me,
         | given many of the meme-style resellers of resellers can still
         | do this. I had many other strange experiences with them,
         | including having to teach their escalation contacts how to use
         | dig outside of their own name servers.
        
           | fragmede wrote:
           | > As a side note, I recently transferred all my domains away
           | from NetSol/web.com as they lost the ability to define custom
           | name servers
           | 
           | FFS is that what happened? They recently bought the registrar
           | I use and a domain of mine with custom nameservers isn't
           | working and I haven't had a chance to troubleshoot.
        
             | LinuxBender wrote:
             | I doubt they have fixed it and I doubt they have plans to,
             | but in the off chance they have make sure it is prefixed
             | with dns(number) or ns(number) as they will not give you a
             | useful error message if you have not done so. _e.g.
             | ns0.yourdomain.tld_ but that probably won 't work either. I
             | used to use bare apex domains for a couple decades but they
             | and several other registrars stopped supporting that. They
             | do not give useful error messages to the clients or to
             | their support. AFAIK there is nobody left that knows how to
             | debug the back-end.
             | 
             | This is just a suggestion but I would consider moving the
             | domains somewhere else. It seems web.com is just hoovering
             | up registrars but not migrating everything to a
             | centralized, modern, maintained system. In my experience
             | and opinion they do not know how the back-end of the old
             | Network Solutions systems work.
        
         | KennyBlanken wrote:
         | https://www.icann.org/resources/pages/epp-status-codes-2014-...
         | 
         | > This status code tells your domain's registry to not activate
         | your domain in the DNS and as a consequence, it will not
         | resolve. It is an uncommon status that is usually enacted
         | during legal disputes, non-payment, or when your domain is
         | subject to deletion.
         | 
         | > Often, this status indicates an issue with your domain that
         | needs resolution. If so, you should contact your registrar to
         | resolve the issue. If your domain does not have any issues, but
         | you need it to resolve, you must first contact your registrar
         | and request that they remove this status code.
        
       | speckx wrote:
       | I can't believe that NetworkSolutions doesn't have a list of
       | significant ISPs by domain name or something. I would proactively
       | work with significant ISPs before suspending the entire domain
       | and interrupting their services.
        
         | KennyBlanken wrote:
         | There's a reason people were practically partying in the
         | streets when Network Solutions lost their monopoly.
         | 
         | The fact that their netops / support staff have no way to
         | escalate an issue like this is the bigger issue. IMHO the giant
         | ISP shouldn't get a better response time or procedure; everyone
         | should be able to get help in an emergency like this.
         | 
         | Really, NANOG have themselves to blame. Half the reason NANOG
         | was started was to help high level techies bypass low-level
         | operations/support staff.
         | 
         | Everyone should have worked together to make sure their own
         | orgs could appropriately handle calls/issues like this.
         | 
         | Instead, with the issue mostly fixed _for them_ , they stopped
         | caring, and all the peons and techies who don't work for
         | Fortune 500 or famous tech companies are still fucked...
        
           | bifrost wrote:
           | HE participates in NANOG, what are you even talking about.
        
         | bifrost wrote:
         | Its not really going to affect their services, maybe email
         | in/out of it but thats about it.
         | 
         | NetSol should've done better though, this makes them look like
         | clowns.
        
           | a-dub wrote:
           | > Its not really going to affect their services, maybe email
           | in/out of it but thats about it
           | 
           | idk. how do you know that their management capability can run
           | sans dns?
        
             | bifrost wrote:
             | Because they run their own DNS servers and inside of their
             | network they're going to be using their own infra...
        
               | a-dub wrote:
               | idk. aren't they tier 1? what if it breaks peering? is
               | all their management capability strictly "inside" their
               | network? what would happen to the rest of the internet if
               | they dropped off?
        
       | codetrotter wrote:
       | Status in whois response is "active" for me, and he.net resolves
       | from DNS for me, so I guess it's been fixed?
       | 
       | But why did they list a @he.net email address as one of the
       | emails to contact them on? Wasn't the problem that he.net would
       | not resolve because of that very client hold?
        
         | KennyBlanken wrote:
         | Confirmed at 3:30 CT
        
           | codetrotter wrote:
           | I read a bit more in the linked email thread. Some other
           | message said that caching would likely make it still resolve
           | in most places and that checking the root servers would show
           | the problem and that elsewhere it would show up around 48
           | hours after, when the caches expire.
           | 
           | Still not sure if the status it shows in Whois means that
           | it's resolved or not.
        
           | LinuxBender wrote:
           | Indeed
           | 
           |  _was_ Domain Status: clientHold
           | https://icann.org/epp#clientHold
           | 
           |  _now_ Domain Status: clientTransferProhibited
           | https://icann.org/epp#clientTransferProhibited
           | 
           | Verified he.net is correctly listed in the root servers,
           | though I did not check prior.                   dig +nocookie
           | @i.gtld-servers.net he.net              ;; AUTHORITY SECTION:
           | he.net.   172800 IN NS ns1.he.net.         he.net.   172800
           | IN NS ns2.he.net.         he.net.   172800 IN NS ns3.he.net.
           | he.net.   172800 IN NS ns4.he.net.         he.net.   172800
           | IN NS ns5.he.net.              ;; ADDITIONAL SECTION:
           | ns1.he.net.  172800 IN AAAA 2001:470:100::2
           | ns1.he.net.  172800 IN A 216.218.130.2         ns2.he.net.
           | 172800 IN AAAA 2001:470:200::2         ns2.he.net.  172800 IN
           | A 216.218.131.2         ns3.he.net.  172800 IN AAAA
           | 2001:470:300::2         ns3.he.net.  172800 IN A
           | 216.218.132.2         ns4.he.net.  172800 IN AAAA
           | 2001:470:400::2         ns4.he.net.  172800 IN A 216.66.1.2
           | ns5.he.net.  172800 IN AAAA 2001:470:500::2
           | ns5.he.net.  172800 IN A 216.66.80.18
           | 
           | which belong to Hurricane [1]
           | 
           | [1] - https://bgp.he.net/AS6939#_whois
        
             | toast0 wrote:
             | I don't have a paste, but a similar query gave me NXDOMAIN
             | during the incident. I did get the A/AAAA glue records for
             | nsX.he.net when asking gtld-servers[1] about my .net,
             | though.
             | 
             | [1] note that these are tld servers run by Verisign, not
             | root servers organized by ICANN or IANA and run by multiple
             | organizations. A few decades ago, I think root-servers.net
             | did do root and also the three letter tlds, but that was a
             | long time ago, before everything got split up into multiple
             | roles.
        
       | basilgohar wrote:
       | First off, Hurricane Electric has always been one of the "cool"
       | companies on the Internet since the early days (of public
       | Internet, at least). So, my goodwill with them tells me that
       | Network Solutions, a company that most certainly has NOT been one
       | of the cool companies, is at fault. I feel and know this to be
       | the case now, as well.
       | 
       | Having said that, I am also greatly pleased that the NANOG
       | mailing list is public and plebes like us can see these kinds of
       | discussions, which I feel are becoming harder to find as time
       | goes on as services get consolidated into larger, government-
       | friendly corporate private digital gardens.
        
       | byyll wrote:
       | Great news! It's a private company and they can do whatever they
       | want! Just like HE likes to censor.
        
       | omoikane wrote:
       | The hold appears to have been removed as of few minutes ago:
       | 
       | https://mailman.nanog.org/pipermail/nanog/2024-July/225903.h...
       | 
       | I didn't know HE.net was using Network Solutions, maybe they will
       | switch to someone else soon. My personal experience with Network
       | Solutions wasn't so great (lots of trouble and human intervention
       | needed to transfer a domain out of their service).
        
       ___________________________________________________________________
       (page generated 2024-07-04 23:02 UTC)