https://netmeister.org/blog/https-rrs.html
Signs of Triviality
Opinions, mostly my own, on the importance of being and other things.
---------------------------------------------------------------------
[homepage] [blog] [jschauma@netmeister.org] [@jschauma] [RSS]
---------------------------------------------------------------------
Use of HTTPS Resource Records
November 13th, 2023
Good news, everybody -- we have new DNS resource records! Well, not
new new, but, you know, newish. You've probably heard of them, or
even seen them actively in use, even though they moved from internet
draft to formal RFC9460 adoption literally while I was working on
this blog post during the last few weeks: the SVCB and HTTPS resource
records.
There are a few interesting things to note about these records: they
allow you to speed up your time-to-first-packet (by basically
stuffing the Alt-Svc HTTP header / ALPN TLS extension into the DNS);
let you do redirection on the zone apex without using CNAMEs; allow
for simple DNS load distribution and failover; obviate HSTS and the
cumbersome preloading process; and enable stronger privacy
protections via Encrypted Client Hello aka ECH (previously ESNI).
Pretty neat, all that.
The record's main drawback is basically the name: trying to search
the web for information about "https" is a lot like asking in your
local library whether they have any books with words.
You can find great explanations of these records elsewhere (for
example, here, here, or here), but let's give a quick example:
$ dig +short https https.test.netmeister.org
1 www.netmeister.org. alpn="h2,http/1.1" ipv4hint=166.84.7.99 ipv6hint=2001:470:30:84:e276:63ff:fe72:3900
$ host https.test.netmeister.org
$
With no A or AAAA records, but the above HTTPS record in place, you
should still be able to connect directly to https.test.netmeister.org
^1. On the wire, that looks like so (click the image to see the full
size):
Screenshot of Wireshark showing packet capture ofan HTTPS lookup
followed by immediate TCP handshake
You'll find our HTTPS record lookup in packet #624, followed by an A
record lookup in #625, and the HTTPS result in #626. Notice that we
then make a TCP connection immediately in packet #627 and begin our
TLS handshake in #630, without waiting for the (empty) A record
result, which finally arrives in packet #651, showing the use of the
ipv4hint from the HTTPS result.
HTTPS records in the wild
Despite just emerging from draft status, we're already seeing some
notable adoption across the industry: Firefox has been making HTTPS
lookups (albeit only over DoH) since May 2020, Apple's iOS and Safari
/ macOS since September 2020, Chrome has had partial support since
December 2020, and just recently enabled ECH by default. Various DNS
service providers also offer support for HTTPS and SVCB records
already.
So with all that, I was curious to see just what the actual adoption
of these records is in the wild. I ignored the more generic SVCB
record (since it requires knowledge of the scheme and possibly port)
and focused only on the HTTPS record, for which I then performed DNS
lookups for approximately 227 million second-level domain names
(e.g., example.com). I then repeated the lookups, prefixing each name
with www. Finally, I repeated the same exercise for the Tranco Top1M
domains^2.
I then analyzed the data collected for the different features the
record provides. Let's take a look at how these are used!
Presence of HTTPS records
Not surprisingly, overall adoption of these records is still low.
However, it is not negligible. As of October 2023, I found almost 10
million domains using an HTTPS record for their www service names
(i.e., ~4.4%), and around 9.1 million domains (~ 4.0%) using the
record on their bare second-level domain name; for the Top1M domains,
there were around 22.5K (25.5%) for the www service names, and almost
24K (25.6%) bare domains using HTTPS records:
Pie Chart showing presence of HTTPS Pie Chart showing presence of
records on www.domain HTTPS records on bare domains
Pie Chart showing presence of HTTPS Pie Chart showing presence of
records on Top1M www.domain HTTPS records on Top1M domains
SvcPriority
The HTTPS record has the following format:
SvcPriority TargetName SvcParams
The SvcPriority field indicates the mode of the HTTPS record: 0
indicates AliasMode (generally intended to allow aliasing at the zone
apex), any other value indicates ServiceMode.
As such, you might expect a SvcPriority of 0 to be more frequently
encountered on the bare domain names, with ServiceMode being
indicated for the www. subdomains. However, I found that virtually
all existing HTTPS records are in ServiceMode:
All bare domains All www. Top1M bare domains Top1M www.
---------------------------------------------------------------------
Priority Count Priority Count Priority Count Priority Count
-------------------- --------------------- ------------------ ------------------
1 9,902,756 1 10,069,638 1 255,940 1 254,605
0 4,905 0 5,173 0 45 2 68
5 78 2 3,285 10 10 0 13
2 76 10 132 9 7 9 4
28 560 18 1,010 11 68 8 others 28
others others others
---------------------------------------------------------------------
There are a handful of other priorities set, as well as a number of
records that have no priority set (i.e., erroneous records, likely
mistyped), but clearly ServiceMode is the primary use case right now.
I expect this to change as support for HTTPS records increases and
organizations begin to adopt them to resolve the "no CNAMEs at the
apex" problem. On the other hand, AliasMode currently does not permit
SvcParams to be set, so perhaps those will ultiately cause
ServiceMode to remain the dominant use.
TargetName
In ServiceMode, the TargetName and SvcParams within each RR
associate an alternative endpoint for the service with its
connection parameters.
RFC9460, Sec. 2.4.3
The TargetName field also shows that early adopters of these records
do not use them to redirect traffic: virtually all records have the
TargetName set to ., meaning the record owner's name is used:
All bare domains All www. subdomains
---------------------------------------------------------------------
TargetName Count TargetName Count
-------------------------------------------- --------------------------------------
. 9,899,763 . 9,142,658
star.fallback.c10r.facebook.com. 3,159 geo-routing.nexuspipe.com. 768
geo-routing.nexuspipe.com. 1,272 hmkb.mydefense.info. 128
5,681 others 6,197 4,845 others 7,504
Top1M bare domains Top1M www. subdomains
TargetName Count TargetName Count
------------------------------------------ ------------------------------------
. 254,600 . 255,926
star.fallback.c10r.facebook.com. 65 geo-routing.nexuspipe.com. 84
geo-routing.nexuspipe.com. 36 www 2
16 others 17 54 others 56
---------------------------------------------------------------------
The two other records besides . that stand out here are
star.fallback.c10r.facebook.com. and geo-routing.nexuspipe.com..
Facebook / Meta does not currently set HTTPS records on their primary
domain names but uses the star.fallback name as a CNAME redirect for
mistyped domains, which explains why this shows up a number of times
for all of their various typo-squatting and brand-protection names:
$ dig +nocomments +nostats https www.instagra.com
; <<>> DiG 9.18.19 <<>>+nocomments +nostats https www.instagra.com
;; global options: +cmd
;www.instagra.com. IN HTTPS
www.instagra.com. 7140 IN CNAME star.c10r.facebook.com.
star.c10r.facebook.com. 7140 IN HTTPS 1 . alpn="h2,h3"
star.c10r.facebook.com. 7140 IN HTTPS 2 star.fallback.c10r.facebook.com. alpn="h2,h3"
The other name (geo-routing.nexuspipe.com.) appears to be used by the
"NexusPipe" CyberSecurity company's DNS services to load-balance or
otherwise distribute traffic across different ports, one of the very
few uses of the HTTPS record using different priorities and ports for
this purpose:
dig +nostat +nocomment https www.fluxteam.net
; <<>> DiG 9.18.19 <<>> +nostat +nocomment https www.fluxteam.net
;; global options: +cmd
;www.fluxteam.net. IN HTTPS
www.fluxteam.net. 10 IN CNAME geo-routing.nexuspipe.com.
geo-routing.nexuspipe.com. 3472 IN HTTPS 10 geo-routing.nexuspipe.com. alpn="h2" port=8080
geo-routing.nexuspipe.com. 3472 IN HTTPS 6 geo-routing.nexuspipe.com. alpn="h2" port=2086
geo-routing.nexuspipe.com. 3472 IN HTTPS 2 geo-routing.nexuspipe.com. alpn="h2" port=2052
geo-routing.nexuspipe.com. 3472 IN HTTPS 1 geo-routing.nexuspipe.com. alpn="h2" port=443
geo-routing.nexuspipe.com. 3472 IN HTTPS 4 geo-routing.nexuspipe.com. alpn="h2" port=2082
geo-routing.nexuspipe.com. 3472 IN HTTPS 3 geo-routing.nexuspipe.com. alpn="h2" port=2053
geo-routing.nexuspipe.com. 3472 IN HTTPS 5 geo-routing.nexuspipe.com. alpn="h2" port=2083
geo-routing.nexuspipe.com. 3472 IN HTTPS 11 geo-routing.nexuspipe.com. alpn="h2" port=8880
geo-routing.nexuspipe.com. 3472 IN HTTPS 7 geo-routing.nexuspipe.com. alpn="h2" port=2087
geo-routing.nexuspipe.com. 3472 IN HTTPS 9 geo-routing.nexuspipe.com. alpn="h2" port=2098
geo-routing.nexuspipe.com. 3472 IN HTTPS 12 geo-routing.nexuspipe.com. alpn="h2" port=8443
geo-routing.nexuspipe.com. 3472 IN HTTPS 8 geo-routing.nexuspipe.com. alpn="h2" port=2095
SvcParams
RFC9460 defines the alpn, no-default-alpn, port, ipv4hint and
ipv6hint, and mandatory SvcParamKeys. In addition, this Internet
Draft defines the ech SvcParamKey for Encrypted Client Hello.
mandatory and no-default-alpn
These two SvcParamKeys are exceedingly rare. The only domains
observed using them are:
* lonios.com. (mandatory=ipv4hint,ipv6hint)
* www.0834-3658888.com. (no-default-alpn=)
* www.014.se. (no-default-alpn=)
* 014.se. (no-default-alpn=)
That's right: 4 out of ~10 million HTTPS records. That's it! Well, ok
then, let's look at the others.
alpn
The alpn SvcParamKey is widely used: 99.9% of all HTTPS records
observed do set this parameter key; only around 7.6K do not have it
set . The breakdown by frequency is:
All bare domains All www. Top1M bare Top1M www.
subdomains domains subdomains
---------------------------------------------------------------------
alpn Count alpn Count alpn Count alpn Count
------------------ ------------------ ---------------- ----------------
h3,h2 8,888,454 h3,h2 8,987,592 h3,h2 217,709 h3,h2 209,767
h2 252,445 h2 888,675 h2 37,585 h2 4,4066
h2,h3 685 h2,h3 7,669 h2,h3 145 h2,h3 157
h3 537 h3 1,689 h3 99 h3 98
15 65 12 60 2 2 5 9
others others others others
---------------------------------------------------------------------
This also speaks to the increasing adoption of HTTP/3.
ech
The ech SvcParamKey is virtually unused -- right now. When I first
ran my data collection, Cloudflare had just announced that they had
enabled ECH for all customers, and indeed millions of domains showed
ech parameters in their HTTPS records using around 207 unique ECH
values. However, soon after (and with decidedly less fanfare or any
specific reasons given), Cloudflare then disabled ECH again,
promising to re-enable it in "early 2024".
As such, as of late October 2023, only three of the Top1M and 16 of
all domains in total have ech parameters set:
* tls-ech.dev. * cloudflareresearch.com.
* 17-mai.com. * dramateket.com.
* cloudflare-ech.com. * encryptedsni.com
* cloudflare-esni.com. * epochbelt.com.
* cloudflare-http1.com. * join21.com.
* cloudflare-http2.com. * parachaexperiments.com.
* cloudflare-http3.com. * myechtest.site.
* cloudflare-quic.com. * protocols.team.
port
The port SvcParamKey is, not surprisingly, hardly used at all: for
HTTPS records, port 443 is the default.
All bare All www. Top1M bare Top1M www.
domains subdomains domains subdomains
---------------------------------------------------------------------
port Count port Count port Count port Count
--------------- ----------------- ---------------- ----------------
443 78 443 23 8880 7 8880 3
8443 63 8443 11 8443 7 8443 3
8880 62 8880 10 8080 7 8080 3
13 566 16 others 106 8 others 63 8 others 27
others
---------------------------------------------------------------------
ipv4hint and ipv6hint
IP hints are ubiquitous. Over 99.8% of HTTPS records have ipv4hints
set, over 92.5% have ipv6hints set. There are 12 domains that only
use ipv6hints and around 420K that only use ipv4hints.
bare domains www. subdomains Top1M bare Top1M www.
---------------------------------------------------------------------
# of
unique 107,131 131,271 91,446 41,623
IPv4
most
frequent 104.16.16.194 104.16.16.194 141.193.213.10 141.193.213.21
IPv4
# of
unique 102,582 105,297 79,842 83,093
IPv6
most
frequent 2606:4700:4400::ac4 2606:4700:4400::ac4 2606:4700:3037::681 2606:4700:3037::681
IPv6
---------------------------------------------------------------------
Now usually when I've done this sort of analysis, I've then also
reported on the distribution of IP addresses across Autonomous
Systems (AS), but this time there is hardly any use in doing so, as
almost all IPs map only into Cloudflare's networks:
AS Number Owner / Name Count
-------------------------------------------------
13335 CLOUDFLARENET, US 9,592,729
209242 CLOUDFLARESPECTRUM 228,058
273584 LINKED STORE BRASIL [...] 24,646
397273 RENDER, US 12,497
12996 DOMENESHOP Oslo, Norway, NO 11,209
16509 AMAZON-02, US 2,199
14061 DIGITALOCEAN-ASN, US 1,828
24940 HETZNER-AS, DE 1,262
1,805 other AS 20,319
-------------------------------------------------
This suggests that the adoption of the HTTPS record is -- at this
time, anyway -- effectively driven by Cloudflare setting the records
by default on all of their domains. Since that includes many small,
parked domains or domains with very little traffic, it's difficult to
judge how many organizations currently actually take advantage of the
record's capabilities. I'd guess that most aren't aware of them at
all, and active use is far, far less common.
Summary
As we have seen, despite being a just recently finalized RFC, the use
of HTTPS DNS records has already grown beyond just sporadic. If you
monitor your organization's DNS logs, you will find plenty of
lookups, as popular browsers have already started to at least
partially implement support for them.
On the domain side, however, it seems that very few organizations
explicitly set them. I'm curious to see how this adoption will
spread, and whether we will see regular CNAME records (with time) be
replaced by HTTPS records, or if we will primarily see that use at
the zone apex.
Generally speaking, I expect CDNs to lead the adoption efforts here,
as the benefits most obvious in their use cases, and as is evident
from the above findings as well. The adoption of ECH, effectively
tied to the HTTPS record, will hopefully also increase as we move
forward here. I know I'll be keeping an eye on that.
November 13th, 2023
---------------------------------------------------------------------
Footnotes:
[1] Safari gets this right. Firefox only looks up HTTPS records when
using DoH, but then also does the right thing. Chrome, as of October
2023 does not support other target names nor use the IP hints in the
record.
[2] Logically, the Top1M domains ought to all fall into the
comprehensive list of all second-level domain names, but
unfortunately not all second-level domains make available their
zones, meaning my list of second-level domains does not include
several of the names found in the Top1M list. (The missing names
generally are those found in country-code TLDs that don't publish
their zones for research.
---------------------------------------------------------------------
Links:
* This blog post as a Mastodon thread
* This blog post on the APNIC blog
* HackerNews Comments
* Lobsters Comments
* HTTP/SVCB or ECH specific:
+ A similar analysis
+ RFC9460 -- Service Binding and Parameter Specification via
the DNS (SVCB and HTTPS Resource Records)
+ Introduction to DNS SVCB/HTTPS records
+ ISC Webinar Slides
+ ECH test site
+ Another ECH test site
* Related research on internet centralization:
+ The gTLDs' New Clothes - A Look at Centralization in Naked
Domains
+ Whose Cert Is It Anyway?
+ Who reads your email?
+ Who controls the internet?
* Other DNS related content on this blog:
+ DNS Zone Stats
+ (All) DNS Resource Records
+ TLDs -- Putting the '.fun' in the top of the DNS
+ DNS Response Size
---------------------------------------------------------------------
Previous: [TLD Domain Count Stats]
---------------------------------------------------------------------
[homepage] [blog] [jschauma@netmeister.org] [@jschauma] [RSS]
---------------------------------------------------------------------