https://cyounkins.medium.com/encrypted-dns-ntp-deadlock-9e378940b79f [ ] Craig Younkins Craig Younkins Follow Dec 29 * 2 min read Encrypted DNS + NTP = Deadlock Photo by Uzoma Ozurumba, CC BY-SA 4.0, no changes I'm generally a big fan of encrypted DNS for the security it provides with negligible performance impact. But I recently found out the hard way that DNS over TLS or DNS over HTTPS can deadlock with Network Time Protocol (NTP) on embedded devices without a battery-backed real-time clock. Many low-cost routers lack a battery for keeping time during power loss. After these devices lose mains power, they start with their clock initialized to some early time, usually the Unix epoch, January 1st, 1970. The device then uses NTP to retrieve the current time over the internet. The NTP source is specified as a set of domain names like pool.ntp.org, so the device begins DNS resolution of the chosen domain to begin NTP. As part of the encrypted DNS process, the TLS certificate of the upstream provider is validated. But TLS certificates are only valid within a certain time frame, which is specified by the 'Not Before' and 'Not After' conditions on the certificate. So validating them requires local time to be correct! $ openssl s_client -connect dns.google:853 | openssl x509 -text | grep -A2 'Validity' Validity Not Before: Nov 28 08:19:04 2022 GMT Not After : Feb 20 08:19:03 2023 GMT $ openssl s_client -connect 1dot1dot1dot1.cloudflare-dns.com:853 | openssl x509 -text | grep -A2 'Validity' Validity Not Before: Sep 13 00:00:00 2022 GMT Not After : Sep 13 23:59:59 2023 GMT And so a deadlock occurs -- we can't update local time because we can't resolve domain names because we don't have correct local time! To address this, I suggest using a few IP addresses for NTP servers so the device can bootstrap. Almost no one provides IP addresses for NTP servers, NIST is the only one I can find. It would be great to see Google or Cloudflare use their infrastructure to provide anycasted NTP IP addresses. I scanned the relevant RFCs and couldn't find any awareness of this circular dependency. If you use DNS over TLS or DNS over HTTPS on your router, add one of the NIST IP addresses to your NTP configuration before you lose power and it deadlocks on startup like mine did. Notes Thank you to David Orr for reviewing this post. Networking Openwrt DNS Embedded Systems Dns Over Https -- -- More from Craig Younkins Follow Hacker, entrepreneur, and quantified self nerd. cyounkins at gmail. Love podcasts or audiobooks? Learn on the go with our new app. Try Knowable AboutHelpTermsPrivacy --------------------------------------------------------------------- Get the Medium app A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store Get unlimited access Craig Younkins Craig Younkins 32 Followers Hacker, entrepreneur, and quantified self nerd. cyounkins at gmail. Follow Help Status Writers Blog Careers Privacy Terms About Text to speech