[HN Gopher] Don't send to Gmail over IPv6
___________________________________________________________________
Don't send to Gmail over IPv6
Author : croes
Score : 144 points
Date : 2022-06-03 14:22 UTC (8 hours ago)
(HTM) web link (www.spamresource.com)
(TXT) w3m dump (www.spamresource.com)
| pcdoodle wrote:
| Gmail is not Email
|
| We were lucky enough to have a product idea that sold out
| immediately after some coverage on a few tech sites (A tiny
| hardware product for nerds).
|
| Trying to update our customers, we sent out an email to about 500
| people via Shopify, the spot the little guys use to do eCommerce.
| No links in the email at all, just a text based update that we're
| here and working hard to keep up with demand.
|
| Ever since then, every email we send to a gmail user (including
| friends and family) get's bounced. It's not even in their spam
| box! We have since added google dns txt records and via something
| they call postmaster tools and we still can't get emails out.
|
| Just text, no links, no phone number, just a product update from
| a small company.
|
| Gmail is not Email anymore.
| jeffbee wrote:
| Gmail does not ever "bounce" mail for spam reasons. It either
| temp-fails the SMTP conversation with a 4xx error code, or it
| delivers it, to the Inbox or to Spam as indicated.
|
| If Shopify is transforming a temp-fail into a non-delivery
| receipt, that's their problem, and yours, but nothing to do
| with Google.
| Avamander wrote:
| There is not a single provider out there that won't punish you
| for sending bulk email out of the blue. It's unfortunate you
| found out like that, but you should try a new domain and let
| the old one cool down.
| CodeWriter23 wrote:
| 1-5 mandatory for standing up a mail server on IPv4 ijs
| ale42 wrote:
| Have a small mail server, pretty low-volume (max bursts are 40-50
| messages to Gmail, global volume to it is around 100 messages per
| week), delivering to gmail over IPv6. (Note: DMARC, DomainKeys
| and SPF all implemented) Mails are delivered reliably and have
| zero problems with it. Had man more issues with Yahoo, Hotmail
| and Apple, all of which only accept mail over IPv4.
| alaricus wrote:
| Don't use Gmail. Here are some better alternatives:
|
| https://proton.me/
|
| https://tutanota.com/
| robjan wrote:
| I get deliverability issues when sending from Protonmail
| rflrob wrote:
| It's one thing to advocate not using Gmail for your own mail,
| and another entirely to advocate not sending any mail to gmail
| accounts from your own mail server, which is what TFA is really
| covering.
| Vladimof wrote:
| Can you search your mail on tutanota yet? Also I wish they
| would have chose a different name...
| andix wrote:
| It's totally understandable why a lot of mail providers require
| correct DNS, rDNS, SPF and DKIM. This is standard in 2022. And
| DMARC is also not hard to implement, it's just a DNS record.
| zippergz wrote:
| DNS is not the hard part of DMARC. It's identifying ALL of the
| legitimate sources of mail from your domain, and keeping up as
| they change. Easy enough for a hobbyist, but often challenging
| for even small business.
| andix wrote:
| Just relay all the mails via your main mailserver and you're
| done.
| djbusby wrote:
| I've got two. One primary, corp email system (out/in) and
| then a second forward-hub for systems to send mails
| through.
|
| This way one doesn't accidentally bang up the other
| istillwritecode wrote:
| outlook.com is even worse. If you bring up a mail server, you
| have to apply to them for permission to send them email. Doesn't
| matter that you have DKIM, SPF, DMARC, or that the domain is 25
| years old. They don't care - you must apply through a process
| that ironically sends email that isn't DKIM certified in their
| response.
| buro9 wrote:
| This isn't specific to Google, but IP reputation was used heavily
| for IPv4... but there either isn't enough reputation for IPv6 or
| the systems haven't yet scaled to make sense of it (what works
| for the IPv4 address space doesn't necessarily work for the IPv6
| address space, i.e. storing and looking up reputation and the
| slow changes in IPv4 ownership and the rapid burn-through of IPv6
| ownership).
|
| What people on IPv6 are getting is one of two things: 1) Harsher
| defaults or 2) No reputation system applied.
|
| In the case of Google for email, it appears that it's harsher
| defaults.
| onphonenow wrote:
| My guess is that the defaults are designed to block a similar
| amount of spam if possible?
|
| Most major mail providers have access to one or more ipv4
| addresses. This allows for reputation + it's a bit more
| wasteful to burn through ipv4 blocks on spam campaigns.
|
| ipv6 address are plentiful, so you can burn through them on a
| spam campaign.
| nybble41 wrote:
| IPv6 addresses are more plentiful. However, you don't assign
| reputation to an individual IPv6 address but rather to an
| IPv6 _network prefix_ (a /64, or even /60). That gives
| roughly the same granularity as a single public IPv4 address:
| one ISP subscriber.
| onphonenow wrote:
| False, the cost of a prefix is MUCH MUCH lower if you are
| looking to spam via ipv6.
|
| Total prefixes are 18,446,744,073,709,551,615
| nybble41 wrote:
| Naturally there are far more IPv6 /60s than IPv4 /32s. As
| a spammer, though, you don't get to pick just any /60
| prefix you want--it has to be allocated to you.
| [deleted]
| rascul wrote:
| Are there any VPS providers who try and help with mail delivery?
| Perhaps by actively watching for and shutting down spammers, or
| investigating and taking action when one of their IP addresses
| show up on a blocklist? Or something else?
| nimbius wrote:
| SPF dkim and dmarc aren't exclusive to ipv6. I think what the
| author is bemoaning is just how exhaustive the requirements are
| to send to one of googles cloistered mailservers in the first
| place.
|
| check senderbase and the rbls, you might be there too if you're
| on a less gilded cloud provider like oracle or ibm. in general
| new mailservers take some TLC to repair the IP space you're given
| before providers will trust it.
| upofadown wrote:
| I don't know why everyone lets Gmail off the hook like this.
| Let's face it, Gmail is broken and has been for some time. It is
| time to move on.
| Avamander wrote:
| People have said that for a decade now. Any specific reasons
| why you're saying this?
| willis936 wrote:
| (2020)
|
| There is a comment saying it's still true in March 2022.
| Uptrenda wrote:
| There should be an amendment to the email protocol that routes
| emails to inboxes if they were paid for by some user-configurable
| amount. Cryptocurrency would be ideal for this unironically
| because its basically an open protocol (yeah I know the hacker
| news crowd hates cryptocurrency but whatever.)
| gorgoiler wrote:
| One in N of their IPv6 relays is also way more fussy than the
| others. What's even better for causing frustrating debugging
| sessions than nebulous spam fighting rules? Intermittently
| implemented rules.
|
| Google's borglike embrace of email is a sad thing for The
| Internet that slipped quietly by before it was too late.
| timcavel wrote:
| tenebrisalietum wrote:
| So what if we just change email to work like this: Mail server
| will not let incoming messages through unless you've first A)
| sent a message to that domain AND B) received a reply that
| doesn't look like an error or bounce.
|
| Yes, you'll have to send an email somewhere with little to no
| content to initiate reception of things like invoices, password
| resets, and monitoring emails. Is that such a bad thing?
|
| Yes, spammers will desperately try to get you to send an email,
| any email, to their domain, or cybersquat on common typos. These
| problems might have easier and better solutions than letting tech
| giants takeover a formerly open Internet protocol that anyone
| should have the ability to use.
| robjan wrote:
| That's sort of how Gmail does work in my experience. Once there
| is evidence of an account replying to email from a "less
| reputable" domain / IP, they tend to let all follow-up
| correspondence through.
|
| The problem is that people managing their own server wouldn't
| be able to initiate conversations with people who give them
| business cards, for example.
| W0lf wrote:
| That sounds a lot like the idea from Peter Huth's Mail-Terror-
| Blocker PRO, a theater drama studied guy that happened to
| become an _internet expert_ by accident here in Germany back in
| the 2000s. Unfortunately, his program failed in the most
| hilarious ways and quickly became a joke in the usenet groups
| back in the day.
| konschubert wrote:
| I don't know the guy, but what's the problem with somebody
| being theater drama studied?
|
| Like somebody can't be an expert at something without having
| gone through formal education for it?
| W0lf wrote:
| Nice framing. However, I was rather thinking about the
| _drama_ he created back then with his software due to a
| logical error. To me it had some resemblance of a nice play
| at the time.
| mgoetzke wrote:
| How would the first mail come through ?
| zerocrates wrote:
| They're saying you, the would-be recipient of later messages,
| would have to initiate by sending a message.
| Closi wrote:
| Yeah, but then the other persons mailbox will reject you
| because they haven't messaged you yet.
| Closi wrote:
| Well I guess you could base it on some sort of reputation
| system...
| bombcar wrote:
| Yeah, if _every_ single mail server worked that way, you
| couldn 't ever get off the ground.
| ldiracdelta wrote:
| Phew!! Glad he isn't lying with that title.
| bhauer wrote:
| The annoying thing about the major email services (Gmail,
| Outlook, et al) using reputation as a key input in their spam
| prevention algorithms is that they don't handle reputation in a
| way sympathetic to small and hobbyist mail server operators at
| all.
|
| Google and Microsoft have the computation and data storage
| resources to forever record how much spam they've received from
| the mail servers operated by any given small company or hobbyist.
| If a domain's authoritative mail servers have a history of never
| sending spam, when that company or hobbyist (for any reason) has
| to switch hosting providers and gets a new IP address,
| Google/Microsoft should recognize that the domain, and its
| referenced mail servers, has never been malicious.
|
| But instead, the majors treat a new IP address for a long-
| established domain as entirely new to the internet and assign it
| zero, or even negative, reputation. And they don't care to
| address this because, well, why care about hobbyists and the rare
| small company insolent enough to try to self-host email?
| w-j-w wrote:
| YetAnotherNick wrote:
| I could bet that most of mails from new IPs are spam because
| there are much fewer people willing to set up a mailserver than
| there are spammer operated mailservers. It is cheap to buy a
| new IP(comes free with many $2 hosting plans). The problem of
| identifying if a new IP is used for spam is not easy to solve.
| megous wrote:
| > The problem of identifying if a new IP is used for spam is
| not easy to solve.
|
| Huh? With dataset of ~100k classified hams/spams I get like
| >99% precisison in identifying spam/ham with just bogofilter
| and 0 heuristics whatsoever (my mail server accepts
| everything, and I just use bogofilter client side).
|
| I guratanee you MS has a dataset with > 100k emails, lol.
|
| It's not like this is unsolved problem.
| SoftTalker wrote:
| Right. While at the same time as they clumsily block any
| IPs with unknown reputation, they seemingly cannot reliably
| flag a poorly formatted, obvious and well-known fake
| invoice for "Norton 360 renewal" as spam (and really, it's
| worse than spam, it's a scam).
| Avamander wrote:
| > It's not like this is unsolved problem.
|
| My god, are you joking or what? Spam, a solved problem? Far
| from it.
| SoftTalker wrote:
| Why can I identify spam at a glance, with greater than
| 90% accuracy, from the subject line alone, and to Google
| it's an unsolved problem?
|
| Google reads and indexes every email that they deliver to
| a gmail inbox. They probably have the worlds largest and
| highest paid staff of ML experts, and they resort to
| auto-flagging based on IP address?
|
| Fix it.
| Avamander wrote:
| > to Google it's an unsolved problem?
|
| It's an unsolved problem for anyone.
|
| > Why can I identify spam at a glance, with greater than
| 90% accuracy
|
| I don't think you understand how terrible even 99.998%
| accuracy is at their scale.
|
| > They probably have the worlds largest and highest paid
| staff of ML experts, and they resort to auto-flagging
| based on IP address?
|
| You think they aren't using them for this? IP addresses
| very likely go into their model as well, when determining
| something is spam.
|
| > Fix it.
|
| They are and adversaries are breaking them again.
| kingjojo23 wrote:
| > I don't think you understand how terrible even 99.998%
| accuracy is at their scale.
|
| What does scale have to do with it, am I missing
| something? Wouldn't a spam detection rate of 99.999% mean
| that only every 100000th spam mail would get through on
| an account basis, how's that terrible?
| Avamander wrote:
| > What does scale have to do with it, am I missing
| something?
|
| The person I replied to said they can identify spam with
| 90% accuracy with a glance, my point was that it's
| abysmal as accuracy. Even orders of magnitude better it'd
| still be bad.
|
| > Wouldn't a spam detection rate of 99.999% mean that
| only every 100000th spam mail would get through on an
| account basis, how's that terrible?
|
| I didn't specify if it's per-account or a false positive
| rate, it's also actually not relevant to the point I'm
| trying to make.
|
| When you receive billions of letters a month small errors
| start to matter and they're doing a better job than the
| person I replied to thinks they do.
| bhauer wrote:
| > _The problem of identifying if a new IP is used for spam is
| not easy to solve._
|
| How is it not easy to check a reputation database for
| _domains_ when evaluating a mail server?
|
| Using DKIM records, Google could cross-reference the
| reputation for the sending domain and, where applicable,
| recognize that the domain in question has never been
| malicious or negligent. And in that case, extend a
| probationary reputation sufficient to allow the IP to
| establish its own new reputation.
| ipaddr wrote:
| The really annoying thing these days is google blocking zip
| files as spam requiring you to use google drive.
| rstupek wrote:
| Zip files are being used to infect computers with malware
| which would explain why they're blocking them
| DarylZero wrote:
| I guess they're not allowed on Google Drive either.
| walrus01 wrote:
| they are basing their filters on the reputation of the netblock
| that your individual ipv4 /32 (or ipv6 equivalent) is contained
| in. And very often when you are hosting with a low cost VM or
| VPS provider the historical record of _other_ IPs operated by
| the same hosting company in the same /24 or something is very
| poor.
|
| while I concur with all of the points you make, there is a
| logical and statistically accurate reason for some of these
| spam filters.
|
| even if your no-open-relay, rdns, spf, dkim, dmarc, SSL/TLS and
| other configuration is absolutely impeccable on your smtpd,
| your only recourse in a situation like this is to change to a
| new ISP that does not have a poor IP space reputation in all
| the adjacent IPs.
| Neil44 wrote:
| The netblock thing has happened to me. My Linode that I've
| had sending mail for 6 years gets blacklisted now purely for
| being on the netblock that it's on.
| apocalyptic0n3 wrote:
| Yep. I've encountered the exact same issues on multiple
| Linode servers in multiple data centers. I've also
| experienced it with an AWS server. It's often almost
| impossible to get it unblocked because the support teams at
| the email providers (if you can even contact them) don't
| realize that's what's happening. I didn't understand it
| myself until I got a call with an engineer at Microsoft who
| realized it in the middle of our call. He unlisted us on
| the call and we were fine for a while, but were eventually
| blocked again and haven't been able to get unblocked. It's
| really frustrating and I'm slowly ending all of my email
| services (both professional and hobby) as a result.
| hamburglar wrote:
| Indeed, although I think there is a little more rationale than
| simply not caring. The hobbyist/small mail server is
| historically more likely to go unpatched and pwnage is more
| likely to go unnoticed, so it's unfortunately a good defensive
| move to penalize them reputationally by default.
|
| PS yahoo is the worst for assuming poor reputation from a
| sender. In my experience they just introduce massive delivery
| delays at the drop of a hat even when the sender has a stellar
| reputation.
| alxlaz wrote:
| > I think there is a little more rationale than simply not
| caring.
|
| You're right in one more way: both Google and Microsoft offer
| email hosting solutions for small businesses. Their main
| (and, if you've got more than 8-10 accounts or so, the only)
| selling point is that it makes managing email hassle-free.
| Making it as painful as possible for small businesses to host
| their email server helps these services tremendously. If they
| had real interoperability (either of their own accord or
| because it were forced upon them through regulatory
| measures), the biggest cash cows of these services --
| companies that are well into "enough accounts that your own
| server would be much cheaper" territory but not large enough
| to afford or risk large infrastructure changes -- would
| evaporate pretty quickly.
| bhauer wrote:
| > _The hobbyist /small mail server is historically more
| likely to go unpatched and pwnage is more likely to go
| unnoticed, so it's unfortunately a good defensive move to
| penalize them reputationally by default._
|
| But to reiterate my earlier point: Google and Microsoft have
| the computational and storage capacity to have a detailed
| history of all domains' authoritative mail servers behavior.
| They will know whether a domain has a history of patch
| negligence.
|
| No, I think the far simpler explanation is that they just
| don't bother tracking reputation by domain. Or if they do,
| it's largely overshadowed by the weight given to IP-based
| reputation.
| hamburglar wrote:
| Oh, I'm not saying you're wrong. They absolutely could put
| the effort in to solve this problem.
| gnarbarian wrote:
| This seems like a major legal liability for Google. It could be
| shown that Google and other major email hosting providers act
| like a cartel by unfairly discriminating against companies who
| don't use a major email provider.
| nonameiguess wrote:
| They hurt more than competitors. If anyone from gmail is
| reading and gives a crap about the collateral damage from
| policies like these, they have set me back at least a few
| months in my attempts to adopt a child. The agency my wife
| and I had been getting licensed through dropped us, citing
| lack of responsiveness in completing paperwork. But I've been
| e-mailing them for months with questions clarifying how to
| fill out unclear forms and they've been ignoring me. I
| presented them with a history of all the sent e-mails they
| never answered, and even gave my e-mail address in person
| last time I was there, asking them to add me to an allow-
| list, but it didn't make any difference.
|
| It's particularly disappointing because even just getting the
| process started was delayed when no one answered our initial
| inquiries of interest for nearly a month. That was when I
| first figured out it was because my e-mails were getting
| routed to the agencies' spam folders. At this point, I'm
| probably just going to have to give in and get a gmail
| address for the sole purpose of completing an adoption, and
| then stop using it and go back to my real address.
|
| Who'd have thought this would be the thing preventing me from
| completely de-googling?
| superasn wrote:
| My loss isn't as big as this but I've gotten some Paypal
| disputes because those users didn't get any of the welcome
| emails (for a rather costly purchase). My issue has been
| more with Apple's mail for some reason though.
|
| Thankfully all my customers have been super understanding
| and were kind to take the disputes back once they were able
| to locate the emails in the Spam folder. Also I finally
| decided to just use a "reputed" smtp server to fix my
| problems (feels like extortion but what can you do).
|
| I've written this before, but it's so sad that something so
| important and vital like email is so broken and we are the
| mercy of corps like Google/Apple/Microsoft and everyone has
| their own secret policy.
| dorfsmay wrote:
| Hiding messages in a spam folder is essentially ghosting.
| Email servers should reject messages that they consider
| spam.
|
| I've run into countless problems because of my messages
| being marked as spam. The worse is that now it as become
| socially acceptable to ghost people, I never know who
| ghosted me, the mail server or the person! I've run into "I
| ignore your email, get the hint" a few times when thinking
| I was marked as spam and tried to contact people through
| other means.
| rascul wrote:
| Too many false positives to reject everything they
| consider spam.
| jawr wrote:
| I think everyone always looks at it from the perspective of
| the hobby mailer, when instead gmail looks at it from the
| recipients point of view.
|
| It's really really trivial to automate buying a domain, get
| some IPs and setup DKIM, SPF and DMARC. How can a provider
| determine intent?
|
| Gmails approach is to mostly use user signals, which in my
| opinion is the best way.
|
| At a company I've worked for, they sent some mail using
| sendgrid to a poorly made mailing list and we're in gmail
| spam for ages.
|
| My personal ran email mostly stays in inbox.
| alar44 wrote:
| Sure, then they relax things, spam explodes, and people are
| going to complain that they're not doing enough to fight spam
| or actively enabling it.
| throw10920 wrote:
| Google has _more_ than adequate capabilities to handle
| spam, as several comments point out above. Also,
| Hashcash[1], which is a virtual silver bullet for email
| spam, _especially_ when combined with existing trust
| mechanisms (i.e. the less reputation you have, the harder
| the Hashcash challenge).
|
| The problem is _not_ that the level of strictness that
| Google is applying is necessary to reduce spam; the problem
| is that Google doesn 't care and is in fact incentivized to
| _not_ accept email from non-Google domains.
|
| [1] https://en.wikipedia.org/wiki/Hashcash
| Avamander wrote:
| > Google has more than adequate capabilities to handle
| spam, as several comments point out above.
|
| Yes, and sender reputation is part of it :D that's why
| they can handle it.
|
| > Also, Hashcash[1], which is a virtual silver bullet for
| email spam, especially when combined with existing trust
| mechanisms
|
| Who do you think has to and can spend more compute or
| money, well-trusted Google or a small player? Far from a
| silver bullet.
|
| > i.e. the less reputation you have, the harder the
| Hashcash challenge
|
| Who do you think will have better reputation? I won't
| even start at how different people define spam. Though, I
| can promise you that on average SpamChimp would get to
| send a lot of spam for much less "HashCash" than a small
| player.
|
| > is in fact incentivized to not accept email from non-
| Google domains.
|
| Everyone is.
| sigstoat wrote:
| > Who do you think has to and can spend more compute or
| money, well-trusted Google or a small player? Far from a
| silver bullet.
|
| i'm not even a big fan of hashcash, but i don't think you
| understand it, or how it could fit in as a component of a
| broader spam mitigation system. (for better or worse)
|
| google wouldn't need to compute hashcash for their
| outgoing emails, because they have DKIM and a solid
| reputation. nor is its computational power somehow at
| odds with that of smaller players.
| Avamander wrote:
| > i don't think you understand it, or how it could fit in
| as a component of a broader spam mitigation system.
|
| It's not that I don't understand, I know how it won't.
|
| > google wouldn't need to compute hashcash for their
| outgoing emails, because they have DKIM and a solid
| reputation.
|
| Great, so what's the point? Small players would have to
| pay Google to deliver mail :D
|
| > computational power somehow at odds with that of
| smaller players.
|
| As I said in my other comment, either botnet devices can
| send tens if not hundreds of spam letters, or it's going
| to be slow and/or expensive for all the legitimate small
| senders. You really can't have both.
| BenjiWiebe wrote:
| The thing with hashcash is that a small player _doesn 't
| need_ much compute power, cause they aren't sending a
| million emails a minute.
| Uptrenda wrote:
| Hashcash is an interesting idea and very creative. But I
| do think you're right. It may impose reasonable limits on
| a single computer. But if an attacker has a botnet they
| can still send out a crap load. Botnets and spam
| practically go hand-in-hand as-is, so I don't see how
| hashcash helps there.
|
| Still... it is a creative idea.
| Avamander wrote:
| Oh, so it's ineffectual against spam, got it.
|
| Either botnet devices can send tens if not hundreds of
| letters, or it's going to be slow and/or expensive for
| all the legitimate small senders. You really can't have
| both.
| tzs wrote:
| > Yes, and sender reputation is part of it :D that's why
| they can handle it.
|
| They aren't doing a very good job of using sender
| reputation. If you have a domain with a several year
| record of no spamming whatsoever, and with every outgoing
| message using DKIM, they will still start blocking you if
| _other_ _senders_ who just happen to have an IP address
| close to yours start spamming.
|
| IP block reputation should only be used to set the
| default when dealing with new senders. If they have seen
| enough DKIM signed messages from a sender to know that
| the sender has a good reputation, IP block reputation
| should have weight 0 when receiving mail from that
| sender.
|
| Google could do this without any increase whatsoever in
| the amount of spam that shows up in their customers'
| inboxes. All that would change is that their false
| positive rate would go down.
| Avamander wrote:
| > They aren't doing a very good job of using sender
| reputation.
|
| That's your limited perspecive, sorry.
|
| > they will still start blocking you if other senders who
| just happen to have an IP address close to yours start
| spamming.
|
| Absolutely, how would they know how that provider assigns
| those IP's? A lot of spammers use entire /24's.
|
| > If they have seen enough DKIM signed messages from a
| sender to know that the sender has a good reputation
|
| A lot of spammers have DKIM, it's not a good reason to
| allow mail from a suspicious subnet.
|
| Pick a provider that deals with their spam complaints.
| That's the harsh truth.
| amenod wrote:
| I hope so! I am tired of explaining people that it's actually
| Google's fault they didn't receive my mail. I'll be happy
| when they pay dearly for the disservice they do to e-mail.
| Avamander wrote:
| It's not really Google's fault to be honest. The need to
| warm up new IP's has existed for a while and a lot of
| providers do it. Any postmaster with experience knows how
| and why it's done.
| black_puppydog wrote:
| The root comment literally explains that that's BS.
|
| Every post master knows _that_ it is done. But it doesn
| 't have to be that way, although it certainly can feel
| like it when a company like google decides to not budge
| on the matter.
| Avamander wrote:
| > But it doesn't have to be that way
|
| Sure, if you want spam. Don't like it, get people to
| deploy DKIM, then the domains will be used for reputation
| purposes.
|
| What the root comment says is BS, the industry uses these
| methods for a very real and practical reasons.
| tomatocracy wrote:
| Even if you have DKIM, SPF and DMARC all set up, at least
| Microsoft still seems to give a decent weight to IP
| reputation and assign a negative reputation to
| unknown/low use IPs.
| Avamander wrote:
| Absolutely, my second sentence says how that could
| change.
|
| I would have thought that it's fairly self-explanatory
| that anti-spam measures utilize the strongest signals. If
| sender domain becomes that, it will get more weight.
|
| So if in the future email providers could reject both
| SPF-less domains and DKIM-unsigned letters, IP's would
| definitely become less relevant. So, get people to deploy
| those things.
| BenjiWiebe wrote:
| No, it can be a different way and still not have spam, by
| trusting/tracking domains instead of IP addresses.
| Avamander wrote:
| I did write how using domains for reputation purposes
| instead could happen. The second sentence.
| cmroanirgo wrote:
| DKIM in no way helps to get past the cartels'
| "reputation" filters. I send maybe one email every few
| months to microsoft accounts & it's always received as
| spam. My server setup & ip have been solid for a decade.
| It's only ever the globalist providers that block me.
| Google is 50/50 I get through. Everyone else (eg
| Protonmail) is no problem.
| Avamander wrote:
| It absolutely does. Also that was an "if, then
| potentially" sentence about reputation tracking in the
| future.
|
| In your case, it's likely that your volume and sending
| patterns aren't consistent and trustworthy enough to keep
| track of your domain and IP reputation.
|
| You have to understand that they get millions of letters
| from new domains each day, sent from compromised
| Wordpress blogs and the alike. If you want to be
| deliverable, you have to be consistent and not
| suspicious.
|
| Or, more likely, there's some other mistake in your
| configuration somewhere.
| ghshephard wrote:
| You also wrote: The need to warm up new
| IP's has existed for a while and a lot of providers do
| it. Any postmaster with experience knows how and why it's
| done.
|
| I think the point people are trying to make, and I'm
| sympathetic to, is that if an ultra-low volume email
| poster, with a full-set of SPF DKIM and DMARC credentials
| configured _and_ zero history of sending spam - that the
| majors (Yahoo /Google/Microsoft) could start off by not
| sending email from that domain immediately to spam, just
| because it isn't a well established and trusted IP
| address.
|
| Alternatively - come up with something akin to D&B
| registration system so people can attest that they won't
| engage in spammy behavior.
| Avamander wrote:
| > I think the point people are trying to make [...] not
| sending email from that domain immediately to spam, just
| because it isn't a well established and trusted IP
| address.
|
| Yes, and I'm saying what's the prequisite for that to
| happen. As long as it's okay (which it currently is) to
| send unsigned mail, IP addresses have larger weight. DKIM
| needs more deployment for that to change.
|
| There's absolutely no way that IP-based reputation
| schemes will be deprecated before alternatives are
| viable. Sure it would be nice for a few people here, but
| no, won't happen before the ecosystem improves.
|
| > Alternatively - come up with something akin to D&B
| registration system so people can attest that they won't
| engage in spammy behavior.
|
| Already exists. That too gets abused.
| denton-scratch wrote:
| It's cheaper and easier to munch through lots of
| throwaway domains than to keep moving IP neighbourhoods,
| isn't it? I don't know - is free domain tasting still a
| thing?
|
| If you filter by IP block (or address!), it might be a
| block that has changed hands and is no longer spammy. Or
| it might be a block from the Zen Policy Blocklist, which
| blocks ranges that the responsible ISP has submitted as
| domestic or retail blocks that are supposed to send
| outbound mail through the provider's smarthost.
|
| If you filter by domain, that could be the envelope
| sender, the From:, the Reply-to:, or the domain of the
| SMTP client. Only the last is reliable; and you also have
| the IP address for the client. In my experience, the IP
| address is more useful, for longer, than the domain name.
| But any good blocklist should age quickly (i.e. old stuff
| should drop off the list).
| Avamander wrote:
| > It's cheaper and easier to munch through lots of
| throwaway domains than to keep moving IP neighbourhoods,
| isn't it?
|
| Depends on your approach. If you hack IoT devices then
| you have a lot of IP's. If you hack Joomla sites, you
| have a bunch of domains.
|
| > I don't know - is free domain tasting still a thing?
|
| Yes. There are also discounts and stuff like that.
| gnarbarian wrote:
| Are you using S/MIME certificates?
| smartbit wrote:
| Why are you asking? Is there a relation between spam
| handling and usage of S/MIME?
| Kadin wrote:
| Last time I looked into it (I run a mailserver and
| mailman list for one of my hobby groups), S/MIME wouldn't
| change your "spamminess" reputation score.
|
| DKIM, DMARC, and SPF do, though, and basically are table
| stakes if you want your mail (especially mailinglist
| messages) to go through to people at major providers.
| ajross wrote:
| > If a domain's authoritative mail servers have a history of
| never sending spam, when that company or hobbyist (for any
| reason) has to switch hosting providers and gets a new IP
| address, Google/Microsoft should recognize that the domain, and
| its referenced mail servers, has never been malicious.
|
| That algorithm doesn't work. The internet is filled with parked
| domains that have "never been malicious". This just creates a
| new market for clean domains that you can use to evade
| protections. It makes spam a little more expensive, but it
| still gets through.
|
| None of these tricks work. There are no tricks. All rules can
| be gamed. The only thing that can't be easily faked is reality:
| if Microsoft knows you're a big org with a well-managed IT
| group running your output email setup, then they know they can
| (probably) trust you not to spam their customers. If you are
| too small to prove that to MS in a scalable way, no amount of
| heuristic trickery is going to help you.
| bhauer wrote:
| I'm sorry, but I don't buy that argument.
|
| Google--a technology giant with algorithms and heuristics
| running most operations--is not up to the task of improving
| algorithms for email server reputation? No, they are
| definitely capable of improvement. It's simply not of
| interest to them.
|
| I can hardly blame them because the cost-benefit analysis
| clearly says, "why bother?"
|
| Of course anything can be gamed, but magnitude matters. I've
| had the same domain for 24 years, with many hundreds or
| thousands of email conversations between users of my server
| and those of the major email players over the years. I had to
| switch my mail server's IP address two years ago. Immediate
| zero reputation from many big players.
| Avamander wrote:
| > is not up to the task of improving algorithms for email
| server reputation? No, they are definitely capable of
| improvement. It's simply not of interest to them.
|
| Incorrect, if they stopped spending all that money and
| effort to keep up, their users would get flooded.
|
| > Of course anything can be gamed
|
| Sending spam/marketing e-mails is a multi-million industry.
| Both on illegal and legal markets. It's a constant race.
| BenjiWiebe wrote:
| Parked domains also don't have a record of sending good
| emails either.
| ajross wrote:
| Tough love: neither do our vanity domains (and yes, I have
| one too, and feel the same pain trying to get gmail to take
| mail). No one is going to pay someone to dig through our
| decades of mailing list activity trying to figure out
| whether to let a half dozen replies through.
|
| This isn't the 1990's anymore, recipient domains aren't
| just homes to a few hundred undergraduates or a dozen
| programmers. Email hosts manage communication for hundreds
| of millions of customers.
| kevincox wrote:
| FWIW I run a service that sends a decent amount of email to
| GMail users. While it was a bit slow to get started (messages
| being marked as spam) once I had sent for a month or two and
| had a few users that marked the messages as not-spam I don't
| appear to have any problems. I say this sending from a Digital
| Ocean IP address that occasionally changes. It appears that
| Google highly values domain reputation and that once I have got
| onto their good list I am doing OK.
|
| Disclaimers: I make sure to do everything else right. I have
| SPF and DKIM and my DMARC policy is to reject 100%. I also
| don't use IPv6 (DO Kube doesn't really support IPv6 well).
|
| I have found other major providers to be much worse. Microsoft
| seems to rely almost entirely on IP reputation and marks
| everything as spam, even accounts that have marked messages as
| "not spam" many, many times. Apple outright blocks the IP
| range.
| AshamedCaptain wrote:
| My experience is that all of SPF, DKIM and DMARC are almost
| completely ignored by Gmail -- one day I simply stopped DKIM
| and DMARC and Google keep happily accepting emails (and to
| this day I still send emails without). In fact they will
| happily accept emails even when the SPF check didn't pass and
| the policy clearly says strict reject aka -all .
|
| While on the other hand I fully agree with TFA: I have
| _never_ been able to send an email to Gmail from a IPv6
| address and have it not end up as spam, not even to accounts
| where I already whitelisted previous attempts.
|
| I don't think it's a reputation issue, since my IPv4
| addresses likely have much worse reputation. It's as if they
| just handicap all IPv6 addresses.
| kevincox wrote:
| GMail definitely respects my DMARC reject policy. But IDK
| how missing one of SPF or DKIM affect its spam decision.
| But with DMARC reject and both missing or invalid it will
| bounce the message every time I have seen.
| tomatocracy wrote:
| Automatic mail forwarding without altering the From
| address will cause DMARC alignment for SPF to break. This
| is a common enough legitimate setup that most providers
| seem to effectively downgrade the DMARC policy applied
| when they see this (usually reject becomes quarantine,
| quarantine becomes ignore).
| marcosdumay wrote:
| > sends a decent amount of email to GMail users
|
| That's why your experience does not apply to the GP.
| andix wrote:
| My experience with digital ocean was, that their IP subnets
| have a horrible reputation for email. Google is easy, but try
| to get unblocked from Microsoft (office 365 or hotmail). In
| the end I switched to another provider.
| tzs wrote:
| The only thing I found that seemed to help with Microsoft
| is for the recipient to find the mail in the spam box and
| reply to it.
| bombcar wrote:
| Back when domains were easy to spoof (I could setup a server
| and send mail as ycombinator.com easily enough) it made sense
| to track the IPs, but now that you have DKIM and SPF links to
| cross-check, you _should_ be able to use the domain reliability
| as a strong indicator. Sure you would have to catch people
| buying a "good" domain that expired, but that shouldn't be an
| insane hurdle.
|
| The real story is nobody cares.
| bhauer wrote:
| > _The real story is nobody cares._
|
| Exactly. Why should the major players care about those too
| small to matter?
|
| > _Sure you would have to catch people buying a "good" domain
| that expired_
|
| Adding to my earlier point, Google and Microsoft are well-
| enough connected to the domain registrars to know when that
| scenario has happened as well. If they put any effort into
| it, they could reliably determine whether a new IP address
| for an established domain is legitimate or a fraud. But as
| we've said, why put any effort into it when the only people
| complaining are not important?
| bombcar wrote:
| And it's even worse - if we theorize a email competitor
| appearing out of nowhere to rival Gmail, and people
| complaining, all that would happen is Google and Microsoft
| would special-case that provider, and the underlying issue
| wouldn't be solved.
| gnarbarian wrote:
| a class action lawsuit could make them care.
|
| as far as how to solve this problem technically, I think
| a reputation system based not on domains or ips but on
| email certificates is the real answer here.
| toast0 wrote:
| DKIM is certificates, so I'm guessing you're talking
| about sender certificates?
|
| How would that help? Spammers can get certificates too.
| Maybe it cuts down on some of the misconfigured http
| email senders, maybe, but not enough to matter. Scam
| sites run https these days.
|
| You can't use like age of activity of the cert to help
| because a) things get compromised, b) you need to rotate
| your certs frequently anyway.
| u801e wrote:
| Shouldn't the authority (or key) used to sign those certs
| be long lived? The certificates themselves should be
| rotated frequently, bit not the key used to sign them.
| Avamander wrote:
| There's only S/MIME, TLS and BIMI+VMC that use actual
| certificates.
|
| DKIM does not and DKIM keys should be rotated once in a
| while, but few do.
| Avamander wrote:
| Being able to tie together letters and senders, knowing
| who sent what, would help.
|
| It wouldn't help to fully trust, nothing would, it's a
| human problem, it would help to trust more.
| toast0 wrote:
| If the email is DKIM signed, it's expected that the
| sender was authorized to send the message.
|
| Any wide spread certificate program will just have the
| email address as the identity, and it will be authorized
| by establishing control of the email (just like the
| majority of certificates used for https are domain
| control only, no organizational verification, not that
| organizational verification means much anyway). Anyway,
| identity is hard; there are many people with my name,
| including a Pulitzer winning author.
| tshaddox wrote:
| Isn't the problem more about how to treat brand new domains
| the first time you encounter them? In order to be friendly to
| small/new email servers, you would presumably need to
| initially grant new domains a sufficient reputation for them
| to send mail reliably. But since domains are essentially
| unlimited, a bad actor can trivially circumvent your
| reputation system by spinning up endless domains. This seems
| like a fairly textbook example of a Sybil attack.
| marcosdumay wrote:
| Spinning up endless domains is something that can be
| detected perfectly well. Very few entities can do it in a
| way that interferes with other people.
| DarylZero wrote:
| Domains aren't free, they're limited.
| Avamander wrote:
| Keep in mind that there are a lot of domains out there
| without SPF records, there's really no lack of domains to
| abuse.
|
| Not to mention all the websites that get hacked or the
| uber-cheap registrars.
| toxik wrote:
| Using DKIm you don't actually have the problem that people
| can buy so-called good domains, because the system works with
| private and public keys, so not only do you need a good
| domain, you also need the private key for that specific DKIM
| in signature
| bombcar wrote:
| I wonder how many systems actually track DKIM signatures
| over time, beyond just checking at the moment of email
| receipt.
| Avamander wrote:
| Most, check results are usually kept in the final stored
| letter.
| BenjiWiebe wrote:
| You wouldn't need to.
|
| At the point of receipt, when verifying via DKIM that
| foobar.com did indeed send this email, then update your
| spam statistics for foobar.com and you're good.
| telmich wrote:
| The article has nothing to do with IPv6.
| bri3d wrote:
| The whole point of the article is that Google's sender identity
| scoring system is more strict when the sending IP is an IPv6
| one. That's a pretty clear cut link to IPv6, no?
| telmich wrote:
| Let me rephrase: the whole article is equally valid for IPv4.
| Being more or less strict is a claim the article makes
| without proof. And as far as my experience goes, there is no
| difference.
|
| So standing by it: the article has nothing to do with IPv6
| per se.
| bragr wrote:
| Google is really strict in general about these kinds of
| things. I had to go a few rounds with my VPSes before emails
| from them would consistently not end up in spam, but looking
| at the headers I'm mostly using IPv6 so I don't draw the
| conclusion "don't use IPv6" just "if you have IPv6, which is
| more likely than not now, be careful and read the docs"
| m348e912 wrote:
| Huh? I'm not even sure how you came to that conclusion.
|
| Gmail is probably tougher on mail servers using IP6 addresses
| because they're plentiful and I suspect spammers were having a
| field day setting up temporary mail relays forcing google to
| play whack-a-mole.
|
| I used to run my own email server years ago but spam and spam
| protection measures have made it time consuming and annoying.
| I'll leave it to the professionals.
| kazinator wrote:
| Don't send to Gmail over IPv6.
|
| .... Unless you're a kooky spammer re-sending a captured YouTube
| terms-of-service-change e-mail to the inbox of some Gmail user
| who bears no relation to the original recipient of that e-mail.
|
| Then, hey, you have no issues with delivery.
|
| https://news.ycombinator.com/item?id=31577087
| jeffbee wrote:
| Gmail has stricter requirements for v6 because they assume if you
| are using v6 then they can leave aside all the bozotic baggage
| they allow for legacy v4. In particular, your sending IP must
| have a PTR record and the name in that PTR record must have a
| AAAA record containing that address.
|
| This is mentioned in the 550 message, but I guess people don't
| read the logs.
| Filligree wrote:
| > Gmail has stricter requirements for v6 because they assume if
| you are using v6 then they can leave aside all the bozotic
| baggage they allow for legacy v4. In particular, your sending
| IP must have a PTR record and the name in that PTR record must
| have a AAAA record containing that address.
|
| This is, in fact, quite difficult to convince your ISP they
| should do.
| mschuster91 wrote:
| > This is, in fact, quite difficult to convince your ISP they
| should do.
|
| For server hosting, I have never seen a provider that doesn't
| allow me to either set the PTR record or at the very least
| keep it set to something that resolves back to the IP address
| in question.
|
| For _residential_ ISPs however the story is different - but
| who would want to send emails to googlemail from a
| residential IPv6 address without authenticating themselves?
| Only spammers would.
| plainolrandy wrote:
| "Authenticating themselves" to who? This, to me, is just
| another way of saying "residential users must use an
| approved sender and can't send mail themselves" since
| residential users will statistically never be able to get
| PTRs changed to their domain and instead will have to deal
| with them pointing to the ISP.
|
| It's another way to force people to use the handful of
| approved providers to send mail and it's really shitty.
| zinekeller wrote:
| I'm sure you'll disagree with me but if you're running a
| computer with an ephemeral address it's very _very_
| likely that you 're not intending to send mail but
| instead it's a malicious program unknowingly installed
| flogging mail, and if you're ISP doesn't provide a static
| address with PTR chances are that they're also negligent
| to the point that they're blocked by even the smaller
| providers simply due to too much spam.
| Avamander wrote:
| > It's another way to force people to use the handful of
| approved providers to send mail and it's really shitty.
|
| There are so many and cheap ways to have a matching PTR,
| so really not really.
|
| The majority of mail from residential ranges is spam and
| has been for a long time. It's unlikely to change at this
| point.
| mschuster91 wrote:
| > It's another way to force people to use the handful of
| approved providers to send mail and it's really shitty.
|
| You can always go and rent a server somewhere in a random
| datacenter, the lowest of the low VPS providers are at
| ~5EUR a month, and send and receive mail from there.
| Hardly a "handful of providers".
|
| There simply is no alternative to banning sending mail
| from residential IPs.
| jeffbee wrote:
| Round-trip DNS consistency is pretty basic and my ISP
| provides it by default. Note that I didn't say your ISP has
| to delegate to you, or that your PTR record needs to match
| your EHLO or anything of that nature.
| vajrabum wrote:
| Are IPv6 numbers not available from your local registry at
| low or no cost? Or in these late days must you rent them? Or
| is this just a problem of people not wanting or knowing how
| to run BGP so they can control their own fate?
| navaati wrote:
| Run BGP... with whom ? Let's say I have my own AS and a
| prefix given to me, which is indeed quite open and not too
| costly. If my ISP coming to my facility won't allow me to
| set a PTR record, they for sure won't allow me to run BGP
| session with them !
| icedchai wrote:
| There are VPS providers out there (like Vultr) that offer
| BGP. You can either use the IPs on their cloud instances,
| or tunnel IPs back to your home network, etc.
| gunapologist99 wrote:
| Vultr is actually really awesome. (And highly unusual in
| their BGP support!)
| nybble41 wrote:
| You're supposed to get your IPv6 prefix from your ISP's
| allocation, not directly from the regional registry.
| Simplifying the routing tables with hierarchical address
| assignment was one of the major selling points for the
| larger address size in IPv6. If everyone gets their own
| prefix independent from their ISP then the core Internet
| routing tables will continue to grow ever more complicated.
| jeroenhd wrote:
| The 550 messages are quite clear, but Gmail has the nasty
| tendency to accept email and then flag it as spam, stuffing it
| away in the spam folder to be automatically deleted in 30 days.
|
| Requiring SPF/DMARC/DKIM/PTR shouldn't really be a problem, but
| there are extra layers of spam filtering on top of the problems
| Gmail will give you feedback about.
| Avamander wrote:
| > Requiring SPF/DMARC/DKIM/PTR shouldn't really be a problem
|
| Oh I wish that were the case. One recent "lovely" example I
| stubled upon is Deutche Telekom (t-online.de) not willing to
| use SPF because it's not perfect enough for them.
|
| It's only the tip of the iceberg unfortunately.
| hdjjhhvvhga wrote:
| I remember that was the first thing I used to do on my mail
| servers, ever before DKIM etc. Why? I have no idea and I will
| never know. Is sending email via IPv6 important to me? No. Is
| mail delivery important to me? Yes. Why is Google such a bad
| player in this field? Because in spite of having the biggest
| email network they don't want to allocate relevant resources to
| handle spammers properly. In other words, they are OK with false
| negatives, that is us. For this reason, I'm fine with this ugly
| workaround.
___________________________________________________________________
(page generated 2022-06-03 23:01 UTC)