[HN Gopher] Async DNS
___________________________________________________________________
Async DNS
Author : todsacerdoti
Score : 89 points
Date : 2025-12-12 16:52 UTC (6 hours ago)
(HTM) web link (flak.tedunangst.com)
(TXT) w3m dump (flak.tedunangst.com)
| albertzeyer wrote:
| The first linked article was recently discussed here: RIP
| pthread_cancel (https://news.ycombinator.com/item?id=45233713)
|
| In that discussion, most of the same points as in this article
| were already discussed, specifically some async DNS alternatives.
|
| See also here the discussion: https://github.com/crystal-
| lang/crystal/issues/13619
| frumplestlatz wrote:
| I am always amused when folks rediscover the bad idea that is
| `pthread_cancel()` -- it's amazing that it was ever part of the
| standard.
|
| We knew it was a bad idea at the time it was standardized in
| the 1990s, but politics -- and the inevitable allure of a very
| convenient sounding (but very bad) idea -- meant that the bad
| idea won.
|
| Funny enough, while Java has deprecated their version of thread
| cancellation for the same reasons, Haskell still has theirs.
| When you're writing code in IO, you have to be prepared for
| async cancellation anywhere, at any time.
|
| This leads to common bugs in the standard library that you
| really wouldn't expect from a language like Haskell; e.g.
| https://github.com/haskell/process/issues/183
| (withCreateProcess async exception safety)
| AndyKelley wrote:
| What's crazy is that it's _almost_ good. All they had to do
| was make the next syscall return ECANCELED (already a defined
| error code!) rather than terminating the thread.
|
| Musl has an undocumented extension that does exactly this:
| PTHREAD_CANCEL_MASKED passed to pthread_setcancelstate.
|
| It's great and it should be standardized.
| frumplestlatz wrote:
| That would have been fantastic. My worry is if we
| standardized it now, a lot of library code would be
| unexpectedly dealing with ECANCELED from APIs that
| previously were guaranteed to never fail outside of
| programmer error, e.g. `pthread_mutex_lock()`.
|
| Looking at some of my shipping code, there's a fair bit
| that triggers a runtime `assert()` if
| `pthread_mutex_lock()` fails, as that should never occur
| outside of a locking bug of my own making.
| gpderetta wrote:
| You can sort of emulate that with pthread_kill and EINTR
| but you need to control all code that can call
| interruptable sys calls to correctly return without retry
| (or longjmp/throw from the signal handler, but then we are
| back in phtread_cancel territory)
| paulddraper wrote:
| IO can fail at any point though, so that's not particularly
| bad.
| kccqzy wrote:
| It's extremely easy to write _application_ code in Haskell
| that handles async cancellation correctly without even
| thinking about it. The async library provides high level
| abstractions. However your point is still valid as I do think
| if you write library code at a low level of abstraction (the
| standard library must) it is just as error prone as in Java
| or C.
| javantanna wrote:
| Just curious how you approached performance bottlenecks --
| anything surprising you discovered while testing?
| 01HNNWZ0MV43FF wrote:
| It's weird to me that event-based DNS using epoll or similar
| doesn't have a battle-tested implementation. I know it's harder
| to do in C than in Rust but I'm pretty sure that's what Hickory
| does internally.
| frumplestlatz wrote:
| it's a weird problem, in that (1) DNS is hard, and (2) you
| really need the upstream vendor to solve the problem, because
| correct applications want to use the system resolver.
|
| If you don't use the system resolver, you have to glue into the
| system's configuration mechanism for resolvers somehow ...
| which isn't simple -- for example, there's a lot of complex
| logic on macOS around handling which resolver to use based on
| what connections, VPNs, etc, are present.
|
| And the there's nsswitch and other plugin systems that are
| meant to allow globally configured hooks plug into the name
| resolution path.
| AndyKelley wrote:
| (1) DNS is hard
|
| It's really not.
|
| Just because some systems took something fundamentally simple
| and wrapped a bunch of unnecessary complexity around it does
| not make it hard.
|
| At its core, it's an elegant, minimal protocol.
| bwblabs wrote:
| It falls into the category that most people _think_ they
| understand DNS, the same as JavaScript, or e.g. elections,
| but the devil is in the detail. And I can tell you, at
| least for DNS (and Dutch Elections), it 's kind of tricky,
| see fun cases like https://github.com/internetstandards/Int
| ernet.nl/issues/1370 and I thought the same before I had my
| current job which involves quite some tricky DNS stuff (and
| regarding this we also sometimes encounter bugs in unbound
| https://github.com/internetstandards/Internet.nl/issues/180
| 3 )
|
| But maybe DNSSEC is the 'unnecessary complexity' for you (I
| think it's kind of fundamental to secure DNS). Also without
| DNSSEC they needed RFC's like
| https://datatracker.ietf.org/doc/html/rfc8020 to clarify
| fundamentals (same goes for
| https://datatracker.ietf.org/doc/html/rfc8482 to fix
| stuff).
| tptacek wrote:
| Lots of elegant, minimal things are hard to use
| effectively.
| benatkin wrote:
| Another related article:
| https://ziglang.org/devlog/2025/#2025-10-15
| brcmthrowaway wrote:
| Who can fix getaddrinfo?
| AndyKelley wrote:
| There are steps that three different parties can take, which do
| not depend on other parties to cooperate:
|
| POSIX can specify a new version of DNS resolution.
|
| libcs can add extensions, allowing applications to detect when
| they are targeting those systems and use them.
|
| Applications on Linux and Windows can bypass libc.
| brcmthrowaway wrote:
| What about macOS?
| AndyKelley wrote:
| they already have CFHostStartInfoResolution /
| CFHostCancelInfoResolution
| btown wrote:
| For those using it in Python, Gevent provides a pluggable set of
| DNS resolvers that monkey-patch the standard library's functions
| for async/cooperative use, including one built on c-ares:
| https://www.gevent.org/dns.html
| petcat wrote:
| gevent. Man that's a blast from the past
| btown wrote:
| Still alive and kicking in production for us! For situations
| where many requests are bound by external HTTP requests to
| third-party suppliers, it's an amazing way to allow for
| practically unlimited concurrency with limited cores.
| dweekly wrote:
| I was able in an afternoon to implement a pretty decent
| completely async Swift DNS resolver client for my app. DNS
| clients are simple enough to build that rolling your own async is
| not a big deal anymore.
|
| Yes, there is separate work to discern what DNS server the system
| is currently using: on macOS this requires a call to an
| undocumented function in libSystem - that both Chromium and
| Tailscale use!
| AaronFriel wrote:
| A lot of folks think this, but did you also implement EDNS0?
|
| The golang team also thought DNS clients were simple, and it
| led to almost ten years of difficult to debug panics in Docker,
| Mesos, Terraform, Mesos, Consul, Heroku, Weave and countless
| other services and CLI tools written in Go. (Search "cannot
| unmarshal DNS message" and marvel at the thousands of forum
| threads and GitHub issues that all bottom out at Go
| implementing the original DNS spec and not following later
| updates.)
| formerly_proven wrote:
| nsswitch _cough_
| frumplestlatz wrote:
| Even once you use the private `dns_config*()` APIs on macOS,
| you need to put in heavy lifting to correctly handle scoped,
| service-specific providers, supplemental matching rules, etc --
| none of which is documented, and can change in the future.
|
| Since you're not using the system resolver, you won't benefit
| from mDNSResponder's built-in DNS caching and mDNS
| resolution/caching/service registration, so you're going to
| need to reimplement all of of that, too. And don't forget about
| nsswitch on BSD/Linux/Solaris/etc -- there's no generic API
| that let's you plug into that cleanly, so for a complete
| implementation there, you need to:
|
| - Reimplement built-in modules like `hosts` (for `/etc/hosts`),
| `cache` (query a local `nscd` cache, etc), and more.
|
| - Parse the nsswitch.conf configuration file, including the
| rule syntax for defining whether to continue/return on
| different status codes.
|
| - Reimplement rule-based dispatch to both the built-in modules
| and custom, dynamically loaded modules (like `nss_mdns` for
| mDNS resolution).
|
| Each OS has its own set of built-ins, private, incompatible
| interfaces for interacting with things like the `nscd` cache
| daemon, and the nsswitch APIs and config files themselves
| differ across operating systems. And we haven't even discussed
| Windows yet.
|
| Re-implementing all of this correctly, thoroughly, _and_
| keeping it working across OS changes is extremely non-trivial.
|
| The simplest and most correct solution is to just:
|
| - Use OS-specific async APIs when available; e.g.
| `CFHostStartInfoResolution()` on macOS, `DnsQueryEx()` on
| Windows, `getaddrinfo_a()` on glibc (although that spawns a
| thread, too), etc.
|
| - If you have a special use-case where you need absolutely need
| better performance, and do not need to support all the system
| resolver functionality above (i.e. server-side, controlled
| deployment environment), use an event-based async resolver
| library.
|
| - Otherwise, issue a blocking call to `getaddrinfo()` on a new
| thread. If you're very worried about unbounded resource
| consumption, use a size-limited thread pool.
___________________________________________________________________
(page generated 2025-12-12 23:00 UTC)