[HN Gopher] Email Authenticity 101: DKIM, Dmarc, and SPF
___________________________________________________________________
Email Authenticity 101: DKIM, Dmarc, and SPF
Author : alexblackie
Score : 435 points
Date : 2021-08-16 02:40 UTC (20 hours ago)
(HTM) web link (www.alexblackie.com)
(TXT) w3m dump (www.alexblackie.com)
| endymi0n wrote:
| > Email is at the centre of most of life and business, and so we
| must ensure it is trustworthy and authentic.
|
| While it's certainly true, to me it's missing the most important
| aspect for practical use here: Email must be _deliverable_.
|
| I only started down the rabbit hole of email authentication long
| after we discovered important client and outreach mail never
| reached its intended sender.
|
| Back then, most of this stuff was hidden in some layers of
| documentation in G Suite instead of in your face after setting up
| your custom domain (where it would belong imho).
|
| If you're setting up your freshly bought startup domain, please
| do yourself a favor and follow this or a similar guide from A-Z.
| estreeper wrote:
| To add to this, also remember to do this for anything else you
| might use to send email: servers/cloud services providers you
| use for transactional emails, any provider you might use for
| marketing emails, etc.
| Covzire wrote:
| Side question if anyone happens to know and wants to tell... is
| there anything going on with GMAIL's spam filtering in the last
| year? I've been getting 1 to 2 daily misses with brain=dead-
| obvious spam messages hitting my inbox, for about a year now.
| I've been using gmail since the invite-only-beta and it was
| never, ever this bad before. I just had 4 spam messages in a row
| like "C0NgRAtuLationS1! yoU Won!" and the like and marking them
| as spam hasn't done anything positive in the last year.
| vgb2k18 wrote:
| Not a full answer but a partial from the other side of the
| fence: setting up my website and domain-name to send emails
| fully self-hosted - I finished setting up DKIM, DMARC and SFP
| on my brand-new, not-publicly-blacklisted IP address, and
| finally... Time to send some test emails out. Results:
|
| [-] Outlook rejected my emails: vague reason given, but the
| fact is, Outlook rejects entire IP address allocations (for
| example, DigialOcean's entire VPS network), even individual
| DigitalOcean IPs which aren't publicly blacklisted. There is a
| proper process in-place for small upstarts like myself to get
| de-listed, but its not a given right that the process will
| favor the applicant.
|
| [-] Gmail. Email received. No problem, no spam label. And I'm
| quite thankful to Google/Gmail for accepting my mail based on
| metrics other than IP addresses alone.
|
| So while I can't answer why they're allowing obvious spam
| through their filter, I can say that at the present moment they
| are not as ruthless in blocking /16 networks (or whatever size)
| as some providers based on IP addresses alone (e.g., Outlook,
| E365).
| Doctor_Fegg wrote:
| Outlook is uniquely horrible. I'm this close to changing my
| signup form to proactively recommend third-party login
| whenever it detects someone entering an outlook/hotmail
| address.
| aj3 wrote:
| Subnet level blocking is certainly a thing, but I'm setting
| up new domains routinely and Outlook works fine. Even with
| cheap / free domains.
|
| Lack of verbose feedback / delivery errors makes debugging
| issues in your own really hard, so you might want to contact
| someone who knows a thing or two about emails for help.
| vgb2k18 wrote:
| > I'm setting up new domains routinely and Outlook works
| fine. Even with cheap / free domains.
|
| You say _domains_
|
| To ensure we're speaking to, not past, each other it will
| be essential to clarify one thing: what do you mean
| precisely by "setting up new domains", "cheap/ free
| domains"?
|
| _Domains_ , I'm not interested in chatting about those,
| but IP address space, then yes I'm all ears. If you can
| reliably setup new, fully self-hosted mail servers (i.e.,
| not using SendGrid, MailGun etc), with port 25 open for
| both send and receive, on cheap, accessible VPS servers...
|
| And...
|
| Send email messages from those new servers directly to your
| Outlook's inbox.
|
| Then you Good Sir, have the secret sauce.
|
| *Caveat: AWS does have a lot of un-blacklisted IP address
| space, however, port 25 remains blocked for new customers
| without a "valid use-case" and an unspecified amount of
| payment history. Similar for Azure. So if your methodology
| is built around AWS/Azure then kudos, but for new-comers,
| it would be unrealistic from my experience to suggest the
| path forward will be without major hurdles and/or delays.
|
| The lack of verbose feedback in the rejection reason from
| Outlook was, as I see it, a feature not a bug. A bit of
| friction to turn away the undetermined ones. I followed the
| trail of clues from the top search results of the error
| message and eventually got my new IP unblocked through the
| proper channel. So... I have made it from bottom tier
| (outright rejection of emails), to 2nd-bottom tier
| (directly to spam/junk folder). Next: build domain/ip
| reputation ever so slowly.
| aj3 wrote:
| Yeah, I didn't want to get stuck in details, but I'm
| setting up a couple of new mailservers (freshly
| registered domains, self-hosted VPS at cheap cloud
| providers (DO, Hetzner, OVH, AWS). And they do get
| delivered directly into Outlook, Google, Protonmail and
| other major providers after two days tops (some still
| employ greylisting for fresh IPs/domains).
|
| There's no secret sauce but there is a myriad of ways to
| shoot yourself in the foot when setting up a mail server
| from scratch. E.g. many standard setups may try
| delivering email over IPv6 if available which currently
| is a terrible idea but might be completely opaque to the
| user. Obviously, the results will also depend on what is
| it that you're sending as everyone uses point-based (or
| even ML) systems nowadays and headers + contents can earn
| you either negative or positive points.
| vgb2k18 wrote:
| Oh boy, thanks for the detail, love it. You've probably
| been more fortunate than you can appreciate for now. I
| mean, I've not yet had a DO IP which wasn't listed
| blocked by SNDS (Outlook private block-lists): https://se
| ndersupport.olc.protection.outlook.com/snds/ipStat...
|
| Would be interested to hear from others who've acquired
| new DO IPs that weren't blocked on that list though.
| Change my view of the whole scenario.
| scrollaway wrote:
| Yeah gmail's spam filter quality has gone way down. I'm not
| getting daily misses, but monthly definitely (on an email that
| doesn't receive much spam at all).
|
| What's more, though, is that there's actually a _lot_ of false-
| positives. Even emails with people I 've talked to in the past.
| Probably at least one false positive per week.
| Seirdy wrote:
| Be aware that poorly-configured mailing lists might not play well
| with DMARC.
| denton-scratch wrote:
| > DKIM is a much stronger signal than SPF
|
| I don't think so.
|
| SPF attests that the mail was sent by an IP address that is
| authorized to send for a certain domain.
|
| DKIM attests that various headers, usually including the From:
| address, are approved by the owner of the domain.
|
| SPF prevents random people on the internet sending mail that
| purports to come from your domain. DKIM prevents people inserting
| forged headers into emails. These goals are orthogonal; neither
| is a stronger signal than the other.
| LeonM wrote:
| Not exactly.
|
| SPF has many issues. One of which is that including external
| email providers will include large blocks of IP addresses to
| your SPF that you don't control. IP spoofing is also a thing.
| SPF also breaks horribly with forwarding email services (like
| catch-all inboxes). The SPF lookup limit of 10 additional
| lookups is way too low for most modern email setups.
|
| SPF is just not very robust, and badly understood by most
| people (leading to many configuration mistakes). Hence that
| receivers cannot get much trust from the SPF outcome alone.
|
| DKIM does send a "stronger signal" in the sense that it is more
| robust than SPF, and contains cryptographic proof. He who owns
| the private key is trusted by the domain (who published the
| public key) to send email on behalf of the domain. DKIM also
| does not suffer from the forwarding issue that SPF has.
|
| As a result, a valid DKIM signature often does result in a
| higher trust rating by the receiver, than just an SPF pass.
|
| DKIM + DMARC enforcement is the most reliable way to go.
| Obviously you still want to setup SPF as good as you can (until
| you start hitting size and lookup limits).
|
| Disclaimer: I'm the founder of Mailhardener, an e-mail
| hardening platform.
| mercora wrote:
| > IP spoofing is also a thing.
|
| IP spoofing is pretty hard to do with 2-way communication
| channels like TCP and a lot harder with TLS too. usually IP
| spoofing only works meaningfully with UDP and protocols that
| allow for transactions were you don't need replies from the
| target. to pull this of you would need to do BGP hijacking in
| order to receive return traffic which is a little above the
| efforts spammers will make.
| denton-scratch wrote:
| > Not exactly.
|
| Are you saying that my core point is wrong, that SPF and DKIM
| serve orthogonal purposes, and that it's incorrect that one
| is somehow "more robust" than the other?
|
| I can't see what I said that you disagree with, except that
| somehow "DKIM and DMARC enforcement is the most reliable way
| to go". You've just said that you disagree, without providing
| much evidence.
|
| You need DKIM if you want deliverability to (principally)
| GMail. You need SPF if you don't want your mailserver
| blacklisted. You don't need DMARC, really, unless you send so
| many emails that you need some kind of automated reporting
| infrastructure.
| nutsu wrote:
| > SPF attests that the mail was sent by an IP address that is
| authorized to send for a certain domain.
|
| Yes, but SPF ties the IP address only to the Sender address
| (MAIL FROM) - so utterly useless in stopping phishing emails
| for example, if attackers can use said domain in From address.
|
| > DKIM attests that various headers, usually including the
| From: address, are approved by the owner of the domain.
|
| As the From address (and implicitly domain) is the one people
| see (and trust) DKIM in a way should matter more.
|
| Also, misconfigured SPF records happen from time to time (on
| big domains actually), where bad actors can then take advantage
| of them and abuse delivery, much harder for a DKIM private key
| to end up in the wrong hands.
| mercora wrote:
| ive seen DKIM signed mails that did not have their key in DNS
| fail verification without being marked as spam by google. and i
| have (not) received mails because of failing spf validation
| made delivery fail silently so i think you are quite right in
| that regard at least for gmails interpretation of those
| signals...
| nutsu wrote:
| Gmail has a blackbox ML that takes SPF and DKIM into account,
| but with a certain scoring that might not matter that much
| compared to said IPs reputation in their system.
| lazyweb wrote:
| Somewhat relevant, my experience with running a small private
| mail server.
|
| https://jschumacher.info/2021/05/running-a-private-mail-serv...
| leephillips wrote:
| My experience is similar. I've been running my own for about 12
| years, for my own own personal email + a few other individuals
| and a couple of very small companies. I also use postfix +
| Dovecot (I'm impressed by the quality of those programs). I
| replaced Spamassassin a few years ago by a handful of procmail
| rules, which reduces CPU usage a bit. My setup handles a couple
| of hundred email per day, and I rarely have to pay any
| attention to it.
|
| That was a good article, and your site seems to have a lot of
| useful information. Does it have an RSS feed?
| cridenour wrote:
| Most Wordpress sites have a default one at /feed/ so in this
| case you can use: https://jschumacher.info/feed/
| leephillips wrote:
| Thanks, that works.
| throw0101a wrote:
| > _I replaced Spamassassin a few years ago by a handful of
| procmail rules, which reduces CPU usage a bit._
|
| rspamd is worth checking out as an alternative to SA if you
| need something more advanced than procmail.
| leephillips wrote:
| Thanks for the suggestion. A dozen or so spam emails still
| get through to my inbox every day. But that's still better
| than using something like Gmail and missing real emails
| because of false positives.
| eddieroger wrote:
| Thanks for sharing. I keep getting closer and closer to wanting
| to stand up my own mail server, but research always leads to
| "it's not worth it" posts, and it's nice to have a realistic
| point of view that is also pretty up to date. I also didn't
| know about mail-tester.com, which I'm adding to my toolbox.
| ozim wrote:
| It is not worth it if you need to send marketing campaigns in
| hundreds of mails per day or transactional email in hundreds
| that really needs to be delivered to corporate email boxes of
| "joe@megacorp.com".
|
| For my personal e-mail most of the time I am just on
| receiving end. For receiving e-mail I don't see much problem.
| For receiving email only you don't need to setup any of the
| DKIM/SPF/DMARC stuff.
|
| Occasionally I send some e-mail out but you are not going to
| be marked as spammer if you sent 1 e-mail a week from your IP
| address. Most likely it will be delivered and accepted by
| whatever provider receiving person is using, might be marked
| as spam - but well you probably can call up that person and
| ask them to check spam folder.
|
| For spam - don't put your email into every web page that asks
| you to do it and you should be good. My gmail account that is
| now ~20 years old started to receive loads of spam only last
| year when email lists from various leaks started to go around
| internet. My other email accounts that get loads of spam are
| on shitty providers that send their own spam.
| eddieroger wrote:
| The realization that my Gmail is nearing 20 years old was
| not a reminder I was prepared for today. If it's that old,
| how old I must be now. :-\
|
| I'm similar to your boat, so this is just one more reason
| I'm going to give it a go, I think. It'll take time to move
| the stuff I care about off Gmail anyway, so I can sort out
| the spam stuff as I go. And I'd really like the flexibility
| of controlling my mail in ways I can't really right now
| with a hosted provider anyway.
| croutonwagon wrote:
| I have been considering it over and over.
|
| For a time i DID host my own emails. Out of my house, with a
| business line and static IP.
|
| Google Apps was free at the time and I registered my domain
| and over time that has become the easiest (its still free for
| me because i grandfathered in back in like...2014.)
|
| With that said, more and more I want my own privacy and
| control. And VPS services are cheap. Im leaning toward moving
| to Mail in a Box or something similar to what OP talked about
| in his blog. Its really just me and my wife so not much
| volume. Could probably host it on Linode no problem.
| dane-pgp wrote:
| If we reach a point where the spaminess of an email can be
| correctly determined based on the reputation of the domain from
| which it was purported to be sent (after some cryptographic and
| policy checks), then is the problem of spam equivalent to the
| problem of bootstrapping trust for new domains?
|
| I suppose technically there is the problem that a domain with a
| good reputation can be hacked or bought, but the costs of that
| happening should be orders of magnitude higher than the cost of
| buying a new domain.
|
| Requiring new mail-serving domains to post a bond for good
| behaviour seems like it would similarly increase the costs of
| buying a good reputation, but the problem is coming up with a
| fair process to decide when a bond should be forfeited.
|
| I'd be reluctant to give the ITU any control over the global
| email network, so instead perhaps there could be some system for
| countries to delegate the bond-forfeiting power to companies of
| their choosing, with each country having a vote in proportion to
| the cube root of their population size. All existing domains
| would be grandfathered in to not need bonds, so the potential
| damage from abuse of this system should be minimal.
| anvandare wrote:
| Alternatively, just create electronic stamps* for email; cheap
| enough that it doesn't enter the minds of those that send a few
| emails per day but expensive enough to ruin the profit margins
| of spammers.
|
| * I'm thinking something similar to SSL: write email, create
| hash (one hash per recipient), submit signing request for hash,
| CA signs request and returns stamp (certificate), stamp is
| transmitted along with email. Additionally, allow people to
| give each other (or sites/newsletters they would like to
| receive) 'rubber stamps', allowing them to create (free) stamps
| for emails addressed to each other... And since no idea of mine
| is ever original I'm certain several smarter people have
| already written multiple RFC's on this and the idea never took
| off because of something I'm overlooking at the moment. Ah
| well.
| gary_0 wrote:
| https://craphound.com/spamsolutions.txt
| rlpb wrote:
| https://en.wikipedia.org/wiki/Hashcash
| dane-pgp wrote:
| > but expensive enough to ruin the profit margins of
| spammers.
|
| And mailing list operators, and charities, and free software
| projects, and...
|
| To be fair, your "rubber stamps" idea does mitigate this
| problem a lot, but I think the issue still remains that it's
| hard to get email providers to agree to drop emails from
| senders that haven't upgraded to this new system, as well as
| agreeing on a globally consistent set of CAs who are trusted
| to not act as a cartel.
|
| It's even more difficult to get all existing pieces of email
| sending infrastructure to simultaneously upgrade to support
| users entering their rubber stamps, not to mention the
| security concerns of dealing with the funds needed to buy the
| non-rubber stamps. Perhaps the big email providers could
| force everyone to implement this as of a certain flag day,
| but I imagine the response to this weird new tax would be
| less than enthusiastic.
| tonyedgecombe wrote:
| You could make the payment go to the recipient. Then for most
| people the cost should be neutral.
| tgsovlerkhgsel wrote:
| For most people, the cost should be negative, courtesy of a
| lot of companies that love to send newsletters.
| aj3 wrote:
| No, domain reputation does not solve the problem of spam (and
| especially [spear]phishing).
| megous wrote:
| Email authenticity and no mention of PGP or S/MIME, which would
| be the only authenticity solution that's actually end-to-end?
| cies wrote:
| Yups. PGP lost the non-tech email users. Too hard to set up for
| them (as they have to set it up).
|
| DKIM, Dmarc and SPF are not an end user burden.
| remram wrote:
| PGP is not meant to be run on the server, and therefore cannot
| make decisions about accepting or rejecting email.
|
| Additionally, PGP relies on a "web of trust" managed directly
| by the user. There is no way for a server to automatically
| discover the key from $YOURBANK to decide if the email is
| legit; you need the bank to give you their fingerprint through
| a side-channel. _pka DNS records help, but is that simpler than
| SPF?
| megous wrote:
| > PGP is not meant to be run on the server, and therefore
| cannot make decisions about accepting or rejecting email.
|
| Why not? To check a signature you're using a public key of
| the other party, so nothing prevents you from adding a milter
| that would check incoming mail against a list of public keys
| you trust, without compromising/sharing any secret key
| material with the server. (As long as you trust your server.)
|
| Anyway, point of PGP is that you can do the checks end to end
| between MUAs, not just between MTAs. Relying on DNS to
| distribute key material is not that great anyway, so PGP with
| keys shared via a better channel, is more secure. (Less
| convenient too, perhaps, depending on your MUA.)
| upofadown wrote:
| Yeah, DKIM, Dmarc, and SPF doesn't prove email authenticity. It
| only proves that an email came from a particular email server.
| omk wrote:
| The bonus in there is a very valuable piece of advice. Thought I
| have good understanding of email authenticity measures, it never
| occurred to me that my domains that don't send out any emails
| should advertise the block all policy.
| brightball wrote:
| This is one of the main points I make when giving talks on it.
| arximboldi wrote:
| I did run my own server with properly setup DKIM and SPF and all.
| Gmail and Hotmail would still mark my emails as spam whenever
| they felt like it. Sometimes, even for emails that were _a reply_
| to an email coming from their own servers. I almost lost a client
| because of that once. Maybe it was because of the IP range (I was
| in a cheap Scaleway server)?
|
| At some point I just accepted reality and paid for a third party
| email server (Runbox) but the situation with the monopoly of
| these giants marking as spam anything they don't manage made just
| really sad.
| xhkkffbf wrote:
| I have one domain that uses Google's G suite. But sometimes
| when I send mail from this domain to a gmail address it gets
| marked as spam. It doesn't leave Google's network or cloud of
| servers, but they flag it.
|
| It's not a very good system.
| calibas wrote:
| HN often talks about running your own email server like it's
| this wonderful example of a decentralized service that anybody
| can take part in. That's kind of true, but there's this whole
| other aspect that gets ignored.
|
| Email is not this wonderful, democratic system of equals. Email
| is more like a school playground where a handful of enormous
| bullies control everything. You either learn to work with them,
| cater to their demands, or you'll never be able to visit large
| sections of the playground.
|
| Gmail is relatively easy to appease, mostly just configuring
| your nameservers and email headers to Google's standards, then
| slowly building "reputation". Hotmail will block an entire
| domain (IP?) randomly every few years, then you fill out one of
| their infamous forms and pray they unblock you quickly. Yahoo
| (technically Verizon now) is even more difficult to work with,
| they block you out of nowhere and I've never figured out how to
| appeal. Seems like they do some kind of ratelimiting for
| smaller email servers, I don't know if it's some kind of
| punishment or just by default.
|
| Then there's the dozen other major email providers. Some easy
| to work with, others not so much.
| trulyme wrote:
| > Gmail is relatively easy to appease, mostly just
| configuring your nameservers and email headers to Google's
| standards,...
|
| Do you mean SPF, DKIM and DMARC or something else? I will
| check the records again, just in case, but afaik they are
| perfect and Gmail still often classifies my mail as spam. It
| is annoying as hell (and your comparison to playground
| bullies is very apt).
| calibas wrote:
| SPF, DKIM and DMARC, but there's other factors as well.
| Gmail will usually tell you exactly why something was
| flagged spam. If you go to "Show Original" SPF, DKIM and
| DMARC should all show "PASS" if everything is configured
| correctly.
| sarkron wrote:
| Hmm, not always. I've just setup my own mail server, I
| PASS SPF, DMARC and DKIM according to Gmail headers and
| yet my email is still classified as spam, as determined
| by "Gmail magic". I believe this is because the domain is
| relatively new and Gmail treats you as guilty until you
| prove your innocence (ie. Enough people mark my emails as
| not spam)
| trulyme wrote:
| Interesting, didn't know that as I don't use gmail
| myself. Thank you!
| traceroute66 wrote:
| > I did run my own server with properly setup DKIM and SPF and
| all
|
| Ah. But the million dollar question is _WHERE_ did you run that
| server?
|
| If you ran it off your home connection then there's your
| problem.
|
| Pretty much everyone on the receiving end (ISPs, corporates
| etc.) will either block or assign a low reputation to home ISP
| IP ranges.
|
| So basically you can do all the DKIM and SPF you like on your
| home MX server, but you're going to have the high hurdle of
| your untrustworthy IP range to fix first !
|
| EDIT: Just saw you said "cheap Scaleway" ... well that's a
| similar sort of problem. Perceived "spammer friendly" hosts
| tend to be assigned a low rep in spam scanning. You won't be
| auto-blocked like residential IPs are, just a low rep set. (for
| any lawyers reading, I am NOT making any comment on Scaleway
| here, I am speaking in general terms !)
| rsync wrote:
| "Gmail and Hotmail would still mark my emails as spam whenever
| they felt like it. Sometimes, even for emails that were a reply
| to an email coming from their own servers. I almost lost a
| client because of that once. Maybe it was because of the IP
| range (I was in a cheap Scaleway server)?"
|
| Like you, I have properly set up DKIM and SPF and blah blah.
|
| I also have a 12 year "clean" IP from he.net.
|
| Sometimes gmail will mark something I send as spam that is,
| like yours, a reply to their user _and is a recipient that I
| have a years long history of corresponding with_.
|
| That's frustrating but what really makes me angry is that there
| is no indication given that this occurs. Why can't google send
| a proper bounce or warning message ... or even an error code
| that I could at least see in my logs ?
|
| In the arms race between spammers and (providers) are error
| indications exploited in some way ?
|
| Or is google just being lazy ?
| arximboldi wrote:
| That's so frustrating. The biggest AI company in the world
| and their anti-spam system is so stupid? Really lowers your
| expectations on state of the art tech.
| gambler wrote:
| They're not stupid. It's part of a sophisticated long-term
| strategy. They deliberately make things complicated to
| maintain so that you give up and use their services. This
| strategy is working. Many small and mid-sized businesses
| today gave up on email maintenance altogether and
| outsourced it to either Google or Microsoft. Many
| individuals end up using Gmail, because it seems more
| "reliable" now that so many other people are on Gmail.
| rsync wrote:
| From my cold, dead hands ...
| lytefm wrote:
| I have a similar impression from dealing with
| deliverability issues.
|
| Relevant providers usually just require a few days of
| sending legit mail and occasionally asking to be
| unblocked, but there seems to be no chance getting into
| the inbox on GMail and Outlook with a "too low" sending
| volume.
|
| If it wasn't for regulations, I would have already given
| up and switched to GMail.
| discreditable wrote:
| Another tip for domains that don't send email: set a null mx
| record. This explicitly tells sending servers that your domain
| does not receive mail, and isn't just broken. Create an mx record
| priority 0, with a dot (.) for the value.
|
| https://tools.ietf.org/id/draft-delany-nullmx-02.html
| LeonM wrote:
| Yes, this is a good tip.
|
| I'd like to add that this is because with no MX record the MTA
| may still attempt to deliver email to the IP from the domain
| A/AAAA record.
|
| This is a lesser known feature of SMTP, it is specified in
| rfc5321 section 5: If an empty list of MXs is
| returned, the address is treated as if it was associated
| with an implicit MX RR, with a preference of 0, pointing
| to that host.
|
| https://datatracker.ietf.org/doc/html/rfc5321#section-5
| scrollaway wrote:
| Oh wow, TIL. Does that work? It says the draft expired in 2014.
|
| "[T]he SMTP client first looks up a DNS MX RR and if that is
| not found it falls back to looking up a DNS A or AAAA RR.
|
| Many domains do not accept email, but do have A or AAAA
| records. If they have no MX records, senders will attempt to
| deliver mail to those A or AAAA records. "
|
| Nothing like a good opt-out to make you feel like you're
| signing up to a Do Not Call list.
|
| I guess we can't blame email too much, it's an incredibly old
| protocol. But at some point, some quirks have got to be worth
| rethinking, right?
|
| Maybe we can deprecate the fallback from MX to A/AAAA for the
| next, oh I don't know, two hundred years before we can move on.
| throwaway210504 wrote:
| The document linked to in the GP eventually became RFC7505
| (https://datatracker.ietf.org/doc/html/rfc7505).
| scrollaway wrote:
| Thanks!
| discreditable wrote:
| Send a test to test@yaboo.com and see. I originally noticed
| it because of Yahoo doing it on misspellings of their domain.
| vgb2k18 wrote:
| >Another tip for domains that don't send email: set a null mx
| record. This explicitly tells sending servers that your domain
| does not receive mail...
|
| Most of my domains don't send mail, however I do prefer to
| receive mail to the standard aliases (webmaster@, abuse@, etc)
| and often a catchall. Even for domains I've got parked and
| awaiting deployment.
|
| Am I thinking too much, not enough, or is the opening phrase of
| the advice meant to be "Another tip for domains that don't
| intend to _receive_ email "?
| justusthane wrote:
| If you're receiving mail you need valid MX records, so you
| can't set a null record.
| ceejayoz wrote:
| You don't need a valid MX. It'll try the A record absent a
| MX.
| justusthane wrote:
| Yes, but if you set a null MX record than it wouldn't,
| right?
| snowwolf wrote:
| My recent experience of trying to get DMARC set up.
|
| 1. Don't use strict (adkim=s; aspf=s) if you expect your emails
| to be forwarded (https://www.dmarcanalyzer.com/forwarding-within-
| dmarc/).
|
| 2. Microsoft (Outlook/Office365) ignores DMARC p=reject by
| default (https://docs.microsoft.com/en-
| us/microsoft-365/security/offi...)
| ehPReth wrote:
| Hopefully ARC solves this:
|
| https://en.wikipedia.org/wiki/Authenticated_Received_Chain
| LeonM wrote:
| > 1. Don't use strict (adkim=s; aspf=s) if you expect your
| emails to be forwarded
| (https://www.dmarcanalyzer.com/forwarding-within-dmarc/).
|
| Neither of this settings have anything to do with forwarding.
|
| aspf and adkim are for SPF/DKIM _alignment_ , where 'r'
| (relaxed) allows subdomains, and 's' (strict) does not.
|
| aspf=s requires RFC5321.MailFrom and RFC5322.From to match
| exactly (no subdomain difference) for the SPF to be qualified
| as aligned.
|
| adkim=s requires the DKIM key to be published under the same
| subdomain as found in the RFC5322.From address for the DKIM to
| be qualified as aligned.
|
| Forwarding will break SPF in most cases anyway, this is
| unrelated to your aspf setting (unless you are forwarding
| between your own subdomains).
|
| DKIM should not break by forwarding, regardless of your adkim
| setting, because forwarding does (should) not rewrite the
| RFC5322.From address.
|
| For more explanation, see here:
| https://www.mailhardener.com/kb/dmarc#relaxed-vs-strict-mode
|
| > 2. Microsoft (Outlook/Office365) ignores DMARC p=reject by
| default (https://docs.microsoft.com/en-
| us/microsoft-365/security/offi...)
|
| They don't ignore it, they treat it similar to p=quarantine. So
| there still is a difference between p=none (or no DMARC at all)
| and p=reject.
| jpalomaki wrote:
| I would recommended to take these steps right in the beginning,
| when you setup email for a domain (or even when just registering
| a domain with no intent to use it for email).
|
| It's much less stressing to work with these when you don't yet
| have anything important going on.
| YPPH wrote:
| I never considered blocking email on domains I don't email from!
| I will implement that.
| somishere wrote:
| Same here. Haven't seen this mentioned before, seems like great
| advice
| brightball wrote:
| Yea, it's huge. Abuse is constant if you don't.
| hardwaresofton wrote:
| Shameless plug -- for people who want to automate this process, a
| guide that does this with Pulumi and SES in particular (while
| leaving space for an cloud-external self-hosted SMTP server):
|
| https://vadosware.io/post/setting-up-ses-with-pulumi
|
| I'm a big fan of Pulumi and have used this setup for 2 projects
| so far (it's part of a bigger framework of repeatable infra for
| SaaS projects I've been hacking on), and I can say it definitely
| delivers (pun intended).
|
| Code in the blog post might be a little stale -- I've just put an
| update in there to note that SES recently changed to using DKIM
| records primarily for domain verification, so the
| aws.ses.DomainIdentityVerification (Pulumi object) is no longer
| needed. One of the nice things about using Pulumi to do this is
| that since AWS allows you to use pre-made DKIM keys now, I read
| one right off disk (look into how to make a dkim key[0], it's
| interesting), and use it for both my personal email (I run
| maddy[1]) tied to the domain _and_ SES.
|
| [0]: https://lxadm.com/Generating_DKIM_key_with_openssl
|
| [1]: https://maddy.email
| alberth wrote:
| Slight off topic: how do sites know an email address from one of
| those "temporary email service providers" is a fake/short-term
| email address?
|
| There's a number of things I like to kick the tires on without
| giving my actual email. So I'll use a temp email and more times
| than not it's detected to be as such. How is that known?
| emersion wrote:
| Don't use SPF. It breaks mailing lists. DKIM does not, assuming a
| correctly configured mailing list.
| throw0101a wrote:
| If you run a mailing list, you probably need to look into ARC:
|
| > _Authenticated Received Chain (ARC) is an email
| authentication system designed to allow an intermediate mail
| server like a mailing list or forwarding service to sign an
| email 's original authentication results. This allows a
| receiving service to validate an email when the email's SPF and
| DKIM records are rendered invalid by an intermediate server's
| processing.[1]_
|
| * https://en.wikipedia.org/wiki/Authenticated_Received_Chain
| emersion wrote:
| ARC requires the recipient to trust the mailing list. Better
| to just allow recipients to check the original DKIM signature
| instead.
| exikyut wrote:
| Reading the sibling comments, what would your recommendation(s)
| be in terms of the mailing-list-friendly version of the posted
| article?
| fogihujy wrote:
| Recent versions of mailman is perfectly capable of handling
| both SPF and DKIM and thus bypassing problems with
| authentication.
| sshine wrote:
| I wanted to ask, how does it do that?
|
| But then I googled around:
|
| https://serverfault.com/questions/611575/will-mailing-lists-...
|
| It seems that SPF, DKIM and DMARC can all cause failure wrt.
| mailing lists depending on how messages are rewritten.
| emersion wrote:
| SPF will always fail. OTOH, when the mailing list doesn't do
| dumb stuff like mutating the subject or the body, DKIM will
| succeed.
| remram wrote:
| SPF really shouldn't fail. SPF checks use the envelope
| address, not the From header. See also
| https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme
| jeltz wrote:
| A properly configured mailing list should work just fine with
| both SPF and DKIM. Especially SPF should not be an issue at
| all.
| emersion wrote:
| A mailing list will _always_ break SPF. When forwarding
| messages to mailing list subscribers, the mailing list server
| will not match the host in the "From" header field.
| kiallmacinnes wrote:
| One of SPF's shortcomings is that is doesn't act upon the
| `From` header at all, which is the name/address the
| recipient most often is shown. Instead, it acts upon the
| `Return-Path` header, which a well configured mailing list
| will set to a domain it controls.
| emersion wrote:
| Alright, I took a shortcut: SPF will verify, but for the
| wrong domain. It'll break when used together with DMARC's
| alignment checks.
| jeltz wrote:
| Yes, it is DMARC which can break mailing lists. Neither
| SPF nor DKIM breaks them assuming the mailing list does
| not modify the mail.
| kiallmacinnes wrote:
| > SPF will verify, but for the wrong domain
|
| I can see that argument, but - it's kinda a philosophical
| question about "who the sender is". Is the the person who
| typed out the text? or is it the server which transcribed
| that text into N new emails?
|
| The ML server will verify the original authors SPF. The N
| recipients will verify the ML servers SPF - the chain
| (which matches the series of MTA's involved) is still
| verified end to end.
|
| > It'll break when used together with DMARC's alignment
| checks.
|
| Yea, DMARC is a much bigger issue for mailing lists, but
| that's no reason to say "A mailing list will _always_
| break SPF" - a well configured* ML has no issues with SPF
| at all.
|
| * And, yes - the definition of "well configured" had to
| change when SPF was introduced, that's of course
| annoying, but there has been many many years for ML
| operators to make these changes.
| emersion wrote:
| > The ML server will verify the original authors SPF. The
| N recipients will verify the ML servers SPF - the chain
| (which matches the series of MTA's involved) is still
| verified end to end.
|
| The recipients have no way to check that the mailing list
| server has checked SPF/DKIM/DMARC. Mailing lists very
| rarely drop messages because of a failing SPF/DKIM/DMARC
| check.
|
| ARC tries to fix this, but requires recipients to trust
| the mailing list server. Just using plain DKIM is much
| better, recipients can just treat ML-forwarded emails
| just like direct emails.
| brightball wrote:
| Failing SPF doesn't matter if a DMARC policy is in place.
| You just need to pass SPF or DKIM at that point. It's why
| it's important to setup both.
|
| Both fill in gaps in the use case of the other.
| emersion wrote:
| It depends. I've seen some servers with a DMARC policy
| which fails when SPF fails.
|
| DKIM completely supersedes SPF, hence my recommendation
| to just skip SPF.
| brightball wrote:
| It does, but it's more involved to setup so depending on
| how many different sources your domain is using you may
| not be able to use DKIM everywhere.
|
| Additionally, DKIM keys need to be rotated periodically
| just like SSL. Many services like Sendgrid or ProtonMail
| will handle this for you now by setting up multiple CNAME
| records so they can rotate the keys for you, but it only
| works with that sender.
|
| SPF helps to address the gaps. I totally agree that
| strict DMARC policy plus DKIM should be enough though.
| remram wrote:
| You're far from your original claim that "SPF always
| breaks mailing-list". You are now at "there is a way to
| configure DMARC that breaks mailing-lists".
| hbcondo714 wrote:
| FWIW, there's also BIMI[1], an industry standard that attempts to
| increase the wide adoption of email authentication with brand-
| specific indicators[2] next to properly authenticated messages &
| requires senders to publish these SPF, DKIM, and DMARC records.
|
| [1] https://tools.ietf.org/id/draft-blank-ietf-bimi-00.html
|
| [2] https://news.ycombinator.com/item?id=28160019
| LeonM wrote:
| At Mailhardener we built a BIMI validator tool [0], we also
| work with the AuthIndicators Working Group (the authors of the
| BIMI standard) to refine the standard.
|
| Please be aware that the BIMI standard is far from finished.
| There are some serious documentation ambiguities that pose a
| risk to how well BIMI might function, and how email providers
| will adopt the technology. Obviously, the CAs are now pushing
| hard for BIMI, since VMCs are a new source of recurring
| business for them.
|
| In my personal opinion I think that BIMI is pushed to early,
| which could leave brands attempting to adopt BIMI disappointed.
| Obtaining a VMC is quite involved (and expensive) for small
| businesses, and the potential benefits are (currently) quite
| small, as not many email service providers support BIMI.
|
| Don't get me wrong though, I think BIMI has a great goal of
| pushing the DMARC adoption rate (remember, you _must_ implement
| strict DMARC for BIMI to work). I just don 't see BIMI offering
| any reliable brand exposure for the coming years. Only time
| will tell I guess.
|
| [0] https://www.mailhardener.com/tools/bimi-validator
| aj3 wrote:
| That's a very controversial standard as it can be abused for
| tracking and does not really protect end users from spoofing.
| teddyh wrote:
| Controversial _and_ technically dubious. The whole premise of
| the BIMI standard hinges on the adoption of X.509 certificate
| authorities which will issue certificates (called "VMC") for
| a _graphical logotype_. However, the laws regarding
| trademarks are restricted in many scopes; legal juristictions
| and business area (trademark "class"), to name a couple. No
| VMC issuer could really do anything sensible with similar
| trademarks which are registered by multiple entities in
| different juristictions and /or different business areas. The
| _best-case_ scenario is that huge players like FAANG, CNN,
| etc. get VMC certificates and the rest of us get nothing.
| Which is, I guess, why the big players are toying with the
| idea and are, slowly, tentatively, pushing for it.
| Thoughtful wrote:
| and financially scandalous
| jiggawatts wrote:
| Thousands and _thousands_ of dollars for a BIMI
| certificate is just ludicrous.
|
| Certificates are certificates are certificates.
|
| They're 1KB files full of random numbers. What you use
| them for shouldn't alter their price... which ought to be
| zero.
|
| The facts that a handful of near-monopolistic CAs have
| managed to control protocols, set standards, and
| generally funnel money into their coffers that _need not
| have been spent at all_ is one of the worst examples of
| naked capitalistic greed I have ever seen.
|
| Rent seeking, pure and simple.
| LeonM wrote:
| Your view is very, very naive.
|
| > Thousands and thousands of dollars for a BIMI
| certificate is just ludicrous.
|
| VMCs are between 800 and 1500 USD
|
| > Certificates are certificates are certificates.
|
| The x509 certificate is, but the process to verify your
| ownership of the supplied 'mark' (your logo) is not.
|
| The VMC issuing procedure is very involved, it requires
| the CA to confirm with your local trademark office that
| your company owns the trademark. For VMC the CA is also
| required to verify (by phone) that the person who
| requested the VMC does indeed work for the company, they
| are also required to verify that the person who requested
| to certificate is allowed by the organization to do so.
|
| The CAs are actually doing work here, there are real
| costs.
|
| FWIW: Digicert currently sells VMC for a discounted price
| of 800USD, which they claim is not even economically
| viable. Without the discount expect VMCs to be ~1500USD.
| zyx321 wrote:
| Verifying that an SVG image is 100%identical (with zero
| room for interpretation) to a picture in a public
| database (available online free of charge)?
|
| And then clicking 'renew' once per year?
|
| How could you possibly charge less than a thousand
| dollars for that...
| LeonM wrote:
| No public database with all trademarks exists. And even
| if it does: who is going to maintain that public
| database? And how would you be able to trust said
| database? How will you proof that you really are the
| legal owner of the logo in that database?
|
| Again, the 'renew' click is not why this is expensive.
| It's just a cryptographic function that signs a bunch of
| data, Lets Encrypt has long proven that certificates can
| be created free-of-charge. However, having a human
| verifying that everything checks out is the expensive
| part. Having that human work in an environment, following
| procedures that passes public audits is expensive. None
| of the trademark offices is going to do your trademark
| validations for free. No-one is going to staff a call-
| centre for free.
|
| You are right about the technical part of creating a
| certificate being trivially easy, but I believe you truly
| underestimate the costs of running a CA that is capable
| of delivering VMCs.
|
| Maybe that some company can do it for less than the
| current prices, I don't know. Competition will show that
| eventually. But if you truly think that you can do it for
| less than the current CAs, then start a CA yourself and
| start competing.
| teddyh wrote:
| People have already paid for an actual trademark. Why
| should they pay _again_ , to a private company?
|
| Your argument is essentially _identical_ to the arguments
| usually made to explain why EV certificates are
| expensive, and nobody is buying those.
| teddyh wrote:
| From what I can see, it is the _very same CAs_ who used
| to peddle normal certificates (and are currently mostly
| failing to sell EV certificates) who are now desperately
| trying to save their business by pivoting to convince
| businesses that VMC certificates are the new big thing.
| wila wrote:
| BIMI is really cool. I played with it, setup the appropriate
| DNS records, created an SVG logo and ... well that's about
| where it stopped as registering a trademark is a bit of a step
| too far for getting all of this to work.
|
| It does pass the checks though [1], but does not really work
| because of the missing certificate which you can for example
| get at DigiCert [2] once your trademark has been approved..
|
| The domains I configured it for are: antwise.com and
| vimalin.com so you can see at [1] how it looks like when it is
| supposed to work.
|
| [1] https://bimigroup.org/bimi-generator/
|
| [2] https://www.digicert.com/tls-ssl/verified-mark-certificates
| raxxorrax wrote:
| The IETF contains some really questionable propositions lately,
| not sure what to make of this, but it doesn't seem to solve a
| problem.
|
| What does this provide in contrast to DKIM? Perhaps there are
| logistical problems with DKIM, but I wouldn't know. I wonder
| why it isn't more widely spread in the first place. The
| receiver can determine the requirement, which is the best
| solution imho.
|
| For Google it seems to be more of a marketing issue than
| security. Furthermore provider could track domain requests.
| That isn't just suspicious, this is almost malicious. There is
| a reason mail clients don't display pictures outright.
|
| I hope my mail client will never support this.
| politician wrote:
| Pay-each-recipient is a great use for a cryptocurrency. The
| incentives align senders to receivers without requiring all of
| our correspondence to be read by intermediate machines outside of
| our control to block nuisance email.
|
| Cue that email solutions checklist meme.
| a-dub wrote:
| that was, i believe, the point of decentralized cryptocurrency
| before it became decentralized cryptocurrency:
| https://en.wikipedia.org/wiki/Hashcash
|
| but then it makes email so environmentally unfriendly, may as
| well just send dead trees.
|
| edit: interesting, didn't know the idea of arbitrarily
| expensive computations as adding costs to combat abuse was
| first presented in the early 90s:
| https://en.wikipedia.org/wiki/Cynthia_Dwork
| LeonM wrote:
| I remember reading somewhere in the 90's that Bill Gates
| proposed this as a potential solution for spam.
|
| If I remember correctly his proposal was to have the sender
| solve a cryptographic puzzle, which takes a bit of CPU time.
| For normal every day use you wouldn't notice, but this subtle
| costs and throughput limit would make spamming no longer
| economically viable.
| Tepix wrote:
| If you're looking for an Ansible playbook to setup a mail server
| with DKIM, DMARC and SPF check out Sovereign on github.
| https://github.com/sovereign/sovereign The project could use some
| help to get up to speed with Debian 11 "bullseye".
___________________________________________________________________
(page generated 2021-08-16 23:02 UTC)