Post Abpxt45BLIMWRUB7BI by tykling@mastodon.social
(DIR) More posts by tykling@mastodon.social
(DIR) Post #Abpxt45BLIMWRUB7BI by tykling@mastodon.social
2023-11-15T17:03:45Z
0 likes, 0 repeats
I was investigating an MITM today where the attacker was using a real LetsEncrypt certificate. This was possible because an NS record domain expired and was re-registered by the attackerA few weeks ago there was also the jabber.ru MITM issue where a valid LE cert had also been issued.Both of these attacks could have been avoided by using CAA account pinning.You should add this on all domains today!For an example see the CAA record for bornhack.dk https://caatest.co.uk/bornhack.dkSpread the word!
(DIR) Post #Abpxt4x46xSp8bO9AW by feld@bikeshed.party
2023-11-15T17:11:25.625830Z
0 likes, 0 repeats
@tykling What's ridiculous about this is that Caddy, by default, will randomly get a certificate from LetsEncrypt or ZeroSSL and doesn't guarantee that the certificate will be renewed from the same issuerSo if you want CAA to not break your site in the future you need to know you must deploy extra configuration to pin to one of these two providers... *sigh*
(DIR) Post #Abpy5i3WY4sJtNQrRY by tykling@mastodon.social
2023-11-15T17:12:19Z
0 likes, 0 repeats
@feld you can pin both, no problem with that
(DIR) Post #Abpy5iuLNh7sXC92m0 by feld@bikeshed.party
2023-11-15T17:13:45.191547Z
0 likes, 0 repeats
@tykling it just means an attacker only has to look and see your CAA supports both but the current certificate is from LetsEncrypt, so they can repeat this attack by asking ZeroSSL to issue the certat least for your described scenario
(DIR) Post #AbpyHGT8wQnpoAvQmW by feld@bikeshed.party
2023-11-15T17:15:51.667984Z
0 likes, 0 repeats
@tykling actually it wouldn't really matter anyway. Both ZeroSSL and LetsEncrypt allow you to issue certs, throw away your account keys, and issue those same certs again under a new accountThe certificates are not in any way protected by your account keys
(DIR) Post #AbpyXaX27HE8fIWpLU by tykling@mastodon.social
2023-11-15T17:16:37Z
0 likes, 0 repeats
@feld that is what the account pinning prevents. The attacker doesn't control the private keys for the ACME account pinned in the CAA record. ZeroSSL would refuse to issue because the attacker is using a different account.
(DIR) Post #AbpyXbPGrcc1NVu8sy by feld@bikeshed.party
2023-11-15T17:18:44.590567Z
0 likes, 0 repeats
@tykling How? I don't see anywhere in the CAA specs that demonstrates this?What does as CAA record look like that pins it to your ACME private keys?https://datatracker.ietf.org/doc/html/rfc6844
(DIR) Post #AbpypErlPJqSgOdpXk by feld@bikeshed.party
2023-11-15T17:21:58.076013Z
0 likes, 0 repeats
@tykling ahh my eyes went right past it> A certificate issuer MAY specify additional parameters that allow customers to specify additional parameters governing certificate issuance. This might be the Certificate Policy under which the certificate is to be issued, the authentication process to be used might be specified, or an account number specified by the CA to enable these parameters to be retrieved.> $ORIGIN example.com CAA 0 issue "ca.example.net; account=230123"Do both LetsEncrypt and ZeroSSL honor this account parameter? I've never seen this "account" parameter used in the wild, but I don't look at CAs regularly
(DIR) Post #Abpz4P8FWNZ0rjaGkC by feld@bikeshed.party
2023-11-15T17:24:42.876370Z
0 likes, 0 repeats
@tykling Continuing this line of thought:your original scenario had the attacker control an NS. If they can control an NS they control whether or not it serves CAA records.Does LetsEncrypt and ZeroSSL ensure responses from *ALL* NSes are identical before proceeding?
(DIR) Post #AbpzlF1KzWBRxQmFiS by feld@bikeshed.party
2023-11-15T17:32:26.556100Z
0 likes, 0 repeats
@tykling The only way around this is to require the CAA record not be served from the NS but is a Glue record insteadI think this is a major security hole everyone is ignoring. It's much harder for an attacker to tamper wither your Glue records
(DIR) Post #Abq0EczCxz54gRnHM0 by tykling@mastodon.social
2023-11-15T17:33:19Z
0 likes, 0 repeats
@feld I believe LE currently does DNS checks from multiple AWS regions + from their own servers. So they likely would have gotten inconsistent answers and bailed out at that point. I don't know about ZeroSSL.The attackers likely had to try issuing multiple times to get lucky and have all the lookups hit the "bad" server.No guarantees here, but I would much, much rather have had CAA account pinning in place than not during this attack.
(DIR) Post #Abq0EdlQ4je55yLmV6 by feld@bikeshed.party
2023-11-15T17:37:43.639253Z
0 likes, 0 repeats
@tykling > The attackers likely had to try issuing multiple times to get lucky and have all the lookups hit the "bad" server.They can't be checking every NS then; they check a random one and I'm not sure why they'd need to do it from multiple regions (other than working around networking issues).So that's the whole attack scenario. Pwn a single NS and then try a couple times until it picks the NS you control.Very disappointing.
(DIR) Post #Abq0tUNDtQM3qRVixE by feld@bikeshed.party
2023-11-15T17:45:00.052376Z
0 likes, 0 repeats
@tykling every acme tool should strongly suggest you do DNS validation and then it can automatically add CAA records for you as it already has all the info required to do it
(DIR) Post #Abq5k4aMu03T1szMlk by tykling@mastodon.social
2023-11-15T17:55:29Z
0 likes, 0 repeats
@feld I agree that DNS-01 should be pushed, and pinned. ACME tools should check for CAA records and recommend they be added.But IMO you should never let any tool on an internet-facing server edit your zone directly, or an attacker compromising the server can also edit your zone.Instead you should make CNAMEs for the _acme-challenge records to a dedicated subzone which is used exclusively for ACME challenges. This has all the advantages of DNS-01, but doesn't hand over control of your zone :)
(DIR) Post #Abq5k5PlotAhbJ2PtA by feld@bikeshed.party
2023-11-15T18:39:26.633529Z
0 likes, 0 repeats
@tykling No, it's not a CNAME, you want a NS for situations like AWS/Route53new zone: _acme-challenge.foo.com.In the parent zone make an NS for:_acme-challenge.foo.com.This delegates requests to your new sub-zone. And give the server a key that can only modify records in that subzone.(if you are using BIND and maybe others you don't need to do this, IIRC you can be more granular without needing to make an entirely new zone)
(DIR) Post #Abq7FPAwV1FboqbH4i by feld@bikeshed.party
2023-11-15T18:56:18.083727Z
0 likes, 0 repeats
@tykling The main problem I have with DNS-01 though is that it's slow as fuck so if you have a distributed cluster of webservers and need to dynamically issue certificates you can just cheat by allowing any server automatically respond to any HTTP-01 challenge without them needing to be aware a challenge was issued. This setup assumes you have a method for servers to automatically find certificates on some kind of shared storage.Here's my Caddy solution for it:(acme_standalone) { # ACME http-01 challenge sends a request to /.well-known/acme-challenge/TOKEN # and expects the server to have created a file at that location with the contents # being a string of TOKEN.THUMBPRINT to validate the challenge. # # We are operating in a multi-node cluster and really want to do the http-01 style # ACME validation, so we can use this regex trick to forge a valid response to any possible # challenge from any node that receives the request. This is much faster than dns-01 challenges. # # The thumbprint is derived from the account key that the ACME client has generated at first run. # Our account name is our email defined at the top of this config. # With Caddy this key file would normally be located at: # $THE_CADDY_DATA_DIR/acme/acme-v02.api.letsencrypt.org-directory/users/USERNAME/USERNAME.key # # The same key is used for all providers, so this is compatible with LetsEncrypt, ZeroSSL, etc. # # We are using an extension for Caddy that stores the Caddy data in an S3 bucket, so the key # can be found there and you can use a script in the ops git repo to get the thumbprint. # # Due to the fact that we're storing the production Caddy data in S3 it is unlikely you will # need to generate a new thumbprint, but that is how you would retrace these steps. @achallenge { path_regexp ch ^/\.well-known/acme-challenge/([-_a-zA-Z0-9]+)$ } respond @achallenge "{re.ch.1}.REDACTED_THUMBPRINT_HERE"}
(DIR) Post #AbqFzFnoTnJJhf9fg8 by tykling@mastodon.social
2023-11-15T19:12:38Z
1 likes, 0 repeats
@feld maybe you want an NS record, I want a CNAME. Making a delegation and seperate zone for every challenge sounds very ineffective to me, but whatever works for you.A CNAME does exactly what is needed, and as a bonus it can carry the name being challenged in the CNAME target (below the challenge zone), so:_acme-challenge.www.example.com CNAME www.example.com.acme.example.orgAnyway, we clearly have different views on this, which is fine. I am not on bikeshed.party, so I will stop here :)