[HN Gopher] Gmail is a huge source of spam
___________________________________________________________________
Gmail is a huge source of spam
Author : caution
Score : 76 points
Date : 2021-02-25 17:24 UTC (5 hours ago)
(HTM) web link (drewdevault.com)
(TXT) w3m dump (drewdevault.com)
| skunkworker wrote:
| Recently in the last 6 months Gmail has gotten significantly
| worse in filtering out spam, I'll get really strange messages
| showing up in my inbox, where the red ssl ({{domain}} did not
| encrypt this message) shows up. It's quite frustrating
| jsnell wrote:
| I'm confused. What, exactly, is the author looking for Google to
| do about these "spam registrations"?
|
| The abuse problem that an identity provider can solve is bulk
| account creation, but a) they are already doing a better job of
| it than any other free identity provider, b) given how low the
| numbers in the post are, this is clearly not any kind of bulk
| account issue.
|
| (Also, not stating either the total number of suspensions or the
| proportion of suspensions that are for gmail seems like a
| conspicuous omission. The author states that 41% of all
| registrations are from gmail, and the absolute highest proportion
| of spammy registrations is 63%.)
| ddevault wrote:
| It is an issue of bulk registration. The reason why the numbers
| are low is because we detect it fairly quickly and teach their
| shape to our abuse prevention tools.
| jsnell wrote:
| I can only go by the numbers you actually posted. If you have
| a different data set that actually supports your thesis, why
| not post those numbers?
| ddevault wrote:
| We don't keep stats on discarded registrations, and usually
| the spammer gives up or changes tactics when they realize
| what's going on. How exactly does this invalidate my
| thesis?
| bo1024 wrote:
| This seems to be going by the "from" header in an email. Do you
| know these emails were actually sent from gmail addresses?
| ddevault wrote:
| All of these cases are after the user has verified their email
| address by acknowledging an email sent to their inbox, so we
| can be certain that these are actually from genuine gmail.com
| addresses.
| bo1024 wrote:
| Interesting, thanks.
| google234123 wrote:
| So, gmail makes up 41% of registrations and also a large
| percentage of suspended accounts? This doesn't seem very
| surprising to me. Also, for me, just because an account was
| "suspended" doesn't mean it was spam.
| ddevault wrote:
| A user has been suspended for any reason other than spam
| exactly once in the entire history of sr.ht.
| yjftsjthsd-h wrote:
| The guy who stored his photo library in hg?
| ddevault wrote:
| No, that person was just asked to stop doing it and then
| chose to leave the service entirely as a consequence. They
| were never actually banned.
|
| The person who was banned was a troll, being rude to other
| users and posting nonsense to various mailing lists and bug
| trackers.
| CameronNemo wrote:
| Anecdotally, my inbox is filled with spam from google forms.
| Somehow scammers are convincing google forms to send me an email
| with their message. I imagine it is difficult for my provider to
| distinguish spam google forms emails from non-spam.
| EvanAnderson wrote:
| I guess I'm just lucky. My current personal spam corpus goes back
| to 2019-05-30. Of the 39454 messages only 80 of them have a
| ".google.com" server as the immediately preceding "Received:"
| hop.
| alex_studer wrote:
| Maybe slightly unrelated to the main topic, but:
|
| > If you're worried about mail deliverability issues, don't be --
| it's more or less a myth in $CURRENTYEAR. If you set up DKIM
| properly and unlist your IP address from the DNSBLs (a simple
| process), then your mails will get through.
|
| Does this match most people's experience? Or is he just talking
| about the deliverability of smaller providers like Migadu? I run
| my own mail server and can never send emails to people on
| Gmail/Microsoft 365/etc unless it's in reply to an email that I
| received.
|
| I've checked DKIM and the DNSBLs, and everything seems
| correct...is it just a matter of not having enough volume for
| email providers to trust me?
| nr2x wrote:
| yeah, that's a 100% bullshit claim.
| throwanem wrote:
| Deliverability issues for personal servers are emphatically not
| a myth. One reason I moved off self-hosting mail a couple of
| months ago, after 17 uninterrupted years of doing so, was
| because of perennial problems with deliverability - and Gmail,
| while bad, wasn't even the worst; some large providers (glaring
| daggers at you, Microsoft) make it effectively impossible even
| to understand why mail is getting blackholed, much less do
| anything about it.
|
| (Fastmail, in case anyone is wondering. $50 a year, and there's
| nothing I want that they don't do.)
| bluecalm wrote:
| Not at all my experience. I was sending paying customers emails
| with their license key and a link to our starting page for
| years and it was fine. Recently our emails randomly land in
| spam v even though:
|
| -we have never sent one unsolicited email
|
| -100% of those emails are opened and read (I know because
| customers are activating their licenses). It's hard to imagine
| anyone mark the email with license they paid several hundred
| dollars for as spam
|
| -we get tons of engagement, we answer around 30-40 emails a day
|
| Something is terribly wrong with Gmail spam filter recently.
| Some of the emails from my customers land in my spam folder as
| well. They more often than not don't contain any links or
| anything out of the ordinary. Just a simple question or two.
|
| It started happening a few months ago. It's a huge pain. All
| Google support tells us is that their magical box they can't
| disclose thinks our emails are spam even though there is
| nothing suspicious about either our website, the link or the
| content of the email. They can't tell us what triggers the
| filter cause it will make the job of the spammers easier.
|
| I am now left to play the game against spammers: they try to
| imitate legit emails, if they get close enough then my emails
| are similar to spam, I have to modify them in random way, then
| spammers catch up etc.
|
| Yay for machine learning I guess. Doesn't work so well even you
| have an adversary trying to fool you.
|
| Btw, we use Gmail and it's Gmail spam filter which is causing
| problems. Google analytics is able to guess our monthly revenue
| and number of licenses pretty well. We send exactly as many
| emails. We are paying customer of Google Suite. It's still not
| their problem that their magic box puts our emails in spam but
| it's all on us.
| sildur wrote:
| It does match my experience. I have a personal mail server with
| every bell and whistle working and my emails arrive to gmail
| addresses. My IP address was used by a spammer in the past, and
| took me some time cleaning it up in a bunch of dnsbl lists. But
| it reliably works now.
| syntheticnature wrote:
| As someone who runs a discussion e-list server for a small
| volunteer org, Gmail has not been an issue with a good config,
| generally. The other big providers are annoying, to varying
| degrees, often silently dropping email. ISP email tends to be a
| pain in the neck, putting one on a blacklist if you exchange
| email with two different addresses they provide service to
| within one hour, though at least they tend to be noisy about
| it.
| ddevault wrote:
| Send me an email: sir@cmpwn.com
|
| I'll see if I notice any configuration issues and report back
| to this thread.
| alex_studer wrote:
| Done, thanks!
| ddevault wrote:
| Copying my reply to HN:
|
| Some feedback: you should add a PTR record (reverse DNS)
| for studer.dev and mail.studer.dev.
|
| I also noted that your DMARC record has p=quarantine, which
| depending on the details of your outgoing setup, may cause
| your emails to be placed in the spam folder if DMARC fails
| for any reason. I generally advise against using DMARC with
| any policy other than `v=DMARC1; p=none` or else you're
| liable to run into issues, it's not a very good standard.
|
| Otherwise, your configuration looks good to me. Also try
| https://www.mail-tester.com to see what they think.
| alex_studer wrote:
| In case anyone is curious: I checked and the PTR record
| for my mail server (168.62.161.121 -> mail.studer.dev) is
| valid. mail-tester.com also gives me a 9.9/10, but I just
| tried sending an email to a Gmail account and it went
| straight to spam :(
|
| I suspect it's probably a volume thing? As in I'm not
| sending enough email for Gmail to make any judgements
| about my mail server, since I'm only an individual. Of
| course there's not really a great way to resolve that...
| Nux wrote:
| There's a site for that: https://www.mail-tester.com/
| looperhacks wrote:
| Weird, for my server spam assassin tells me my dkim
| signature is invalid but the dkim check in the next
| category says that it's valid.
| alex_studer wrote:
| Same here!
| jacquesm wrote:
| No, it doesn't match my experience at all. Microsoft, Apple and
| Google all tend to randomly drop messages from addresses that
| were perfectly good the day before without any change in the
| sending mail servers configuration. It is super annoying
| because it actually forces me to use one of those to send email
| and that actually rewards those companies for creating this
| problem in the first place. I'd rather deal with spam than
| random delivery issues without any knowledge about it.
|
| Oh, and I get more and more false positives in gmails spam
| folder as well, and since they're deleted after 30 days I now
| have one more 'inbox' to monitor.
| loloquwowndueo wrote:
| +1 these days I need to wrangle two inboxes. Thanks google.
| Cu3PO42 wrote:
| You could pay for hosting from a smaller provider on your own
| domain. Sure it costs money, but you don't have to support
| those giants and you get to keep your own address.
| joshuaissac wrote:
| It seems they already do that, which is why their e-mails
| are getting dropped when sent to a recipient hosted by the
| giants.
| Cu3PO42 wrote:
| I was reading it as they are self-hosting and are
| therefore having their emails dropped. While I have had
| similarly terrible experiences with self-hosted mail, I
| have found even smaller dedicated providers have
| absolutely no issues with getting mail delivered.
| dunham wrote:
| Apple will silently drop an email (no bounce message) based
| on text content of attached pdf files. This happened at my
| day job, we found the page of the pdf that triggered it via
| divide and conquer.
| upofadown wrote:
| >Microsoft, Apple and Google
|
| Well what do you want for nothing? Randomly losing email is
| what the free email services do. That is hardly the fault of
| everyone else...
| elric wrote:
| Yeah that's my experience as well. And it's impossible to
| find anyone to talk to at Google/MS/Apple about an email
| issue -- unless you send really large volumes, then suddenly
| all kinds of support and tools become available. IMO it's a
| deliberate attempt to get more people to switch to gmail. But
| maybe I'm a cynic.
| noja wrote:
| No it doesn't. My mail server is massively rate limited even a
| year later.
| jnwatson wrote:
| Yeah, that's not been our experience, and this is just for
| password reset/logins. You pretty much have to use a dedicated
| provider for that.
| Cu3PO42 wrote:
| It absolutely does not. I tried setting up mail on my own
| server and by all accounts I did everything right. Mail-Tester
| and similar services found no issues, I had all the optional
| records including DKIM, SPF, etc. and yet I couldn't rely on my
| mail going through.
|
| Microsoft (Outlook.com) was particularly bad: mails were
| silently dropped without notice to me or the recipient even
| when I was replying to a mail they sent. I tried adding my
| e-mail address on my own servers to the trusted senders list on
| a MS account of my own and even then not all mail would go
| through.
|
| Gmail ocasionally put me in spam but at least always delivered
| my messages.
|
| At that point I gave up since I consider email to be too
| essential to be flaky and paid for Migadu. I have been very
| happy with their service and appreciate their pricing strategy
| (charging for number of mails sent, not the number of domains
| or mailboxes added). That said, I still dislike paying for
| hosting that I had sufficient capacity for on my own servers.
| joana035 wrote:
| Yes, it works for me.
|
| SPF, DKIM and proper DNS records and that is it.
| anonydsfsfs wrote:
| > Does this match most people's experience?
|
| It does not. I used to work for a non-profit that ran our own
| SMTP server, and we had issues with our subscriber e-mails
| getting marked as spam, mainly for Office 365 users. We set up
| DKIM, DMARC, and SPF, but it still happened. We ended up having
| to move to using Google's SMTP servers just so our e-mails
| would go through.
| kureikain wrote:
| It does but only for IPv4. If you have those:
|
| - reverse ip. origin ip -> dns -> resolve to ip
|
| - SPF
|
| - DKIM
|
| - DMARC
|
| - Make sure ip are not on DNSBL
|
| - Message ID
|
| - Try to connect to gmail mx server using TLS
|
| Then you have very low chance to get in spam unless you are
| sending spam and get blacklist. Time to time an gmail mx server
| will reject you, rate limiting, just try again with another and
| you will be ok.
|
| Source: I run https://hanami.run and have to deal with this a
| lot due to nature of an email forwarding service that people
| usually use as one-off email (anything@domain.com for their
| one-off service) so it attract a lot of spam which I have to
| filter and make sure it won't go to gmail/microsft
|
| I would say gmail is the best among big provide:
| Microsoft/Apple(iclouds) are garbage mail server where they
| just blindly trust DNSBL and block you no matter what. Gmail
| was way better.
|
| However, if you use IPv6, it's a bit harder to avoid being flag
| as spam. It's really random and I don't know why yet and I
| contacted them and now "Waiting for 2 weeks to get a response"
| alex_studer wrote:
| Out of curiosity, where do you send the emails from? (Are the
| IP addresses from AWS/some other large provider? Or something
| else?) I've heard that the IP address block has something to
| do with getting marked as spam.
|
| I've checked everything on that list and it's all good, and
| I'm sending from IPv4. The really frustrating thing is that
| Google's own Postmaster Tools [0] doesn't want to tell me
| anything without "a sizable daily volume of email traffic (up
| to the order of hundreds)".
|
| If it doesn't have to do with the origin IP's neighborhood,
| my only other guess is that I don't have enough email volume,
| which seems like a catch-22...
|
| [0] https://gmail.com/postmaster/
| kureikain wrote:
| I used Hetzner.
|
| At the time I allocate an IP address, it's was listed on
| DNSBL. I tried like 30times to get a good IP and gave up. I
| instead just allocate a new IP, wait a week. Started to
| send email to myself every 15minutes to a gmail address.
| Anytime it isn't showed up in Important, I go in to move it
| to important. If it showed up as spam, I marked it as not
| spam.
|
| After 2-3 weeks doing that, The IP is no longer listed on
| any blacklist and I started to roll them out.
|
| Don't worry too much about IP blacklist. They aren't super
| important to gmail. If an IP stopped sending spam, they
| will be removed from IP blacklist eventually. Give it a
| week.
|
| > "a sizable daily volume of email traffic (up to the order
| of hundreds)".
|
| I send about 8,000 emails per day and somehow that
| Postmaster Tools not reports anything at all. I would say
|
| If you send me an email vinh@hanami.run I can tell you what
| you can do to avoid being flagged at spam. Happy to jump in
| a 1-1 chat https://calendly.com/vinh-hanami/15min to help
| you on that.
| jacquesm wrote:
| Even replying to gmail messages or forwarding an older
| address to gmail will randomly fail. I've had some pretty
| important messages (for instance: invoices) randomly
| blackholed while all other mail arrived fine. It is super
| frustrating to answer someone's request for an invoice and to
| see that invoice then disappear without any notice that it
| was spam trapped.
| kureikain wrote:
| Yes, I heard you.
|
| To gmail point of view, they don't care about forwarding or
| replying to an email someone send you at all. All they care
| seems is about the message follow standard best
| practice(SPF/DKIM/DMARC/PTR) and send from a good IP.
| Sometime your IP can be flagged as spam wrongly on DNSBL.
| But it's very easy to get remove from DNSBL. Just submit a
| removal request, explain your situation and it's usually
| remove within a few days.
|
| Also, the reason while your important emails are marked as
| spam is depended on gmail mx server you connected to. I
| don't know the detail but I have seen gmail mx server
| rejected/spammed my email, but on subsequent try if I
| connect to different one(by resolving other MX record), the
| email went through just fine.
|
| If you send me an email vinh@hanami.run I can try to spot
| if anything may randomly flagged your email.
|
| Also, did you have multiple SMTP servers?
| [deleted]
| laurent123456 wrote:
| Only slightly related but it seems Google has also become very
| bad at filtering spam too? I keep getting agency spam from Indian
| companies, all with very similar characteristics (HTML emails for
| IT jobs in the US, where I don't live, from Indian companies). I
| marked hundreds of them as spam, and even a basic bayesian filter
| would be able to easily filter this but somehow gmail let it all
| go through.
| bluecalm wrote:
| Yep, it started becoming terrible a few months ago imo.
| boring_twenties wrote:
| I don't get any spam in my inbox, but I do get non-spam
| messages marked as spam all the time, to the point where I have
| to check my spam folder once or twice a day.
| x86_64Ubuntu wrote:
| I used to never get spam in my GMail. Now, every once in a
| while, I get spam in the form of Google Drive spam (pharmacy
| stuff) and then I get "You've won Amazon Rewards!!" type of
| email. It's very strange.
| kureikain wrote:
| It's very similar to my experience.
|
| 50% of spam come from a valid gmail account, as in, using gmail
| smtp server, come from gmail origin(.google.com reverse dns) have
| valid DKIM. The only way to filter out spam is look at the
| content and run some simple ML on it.
|
| 50% of spam come from non .com TLDs such as .cam .work from
| random servers around the world: China, Bulgari, Hungari, France,
| Germany...
|
| But it's very easy to block other TLDs because their IP are on
| DNSBL already. But what can I do with gmail? Gmail IP get on
| DNSBL all the time.
|
| Example: 209.85.222.196 is listed right now on
| spam.dnsbl.sorbs.net and ips.backscatterer.org
|
| But at the same time, this is just the nature of an email
| service. If we have a large user base then no matter what we do,
| a small percentage of users are abuser which are successfully to
| send out spam using our services before we can detect and ban
| them.
|
| On AWS SES, they requires user to keep a reputation of less than
| 0.1% emails flagged as spam. Go over than that and you will be
| put on a review queue. Reach 5% and get banned permanently.
|
| But before that occurs, spam are send out already.
| zokier wrote:
| It is unclear if Gmail is proportionally any worse than the
| others. Is the 119 suspended accounts more or less than 41% of
| the total number of suspended accounts?
| samblr wrote:
| >> 5X as many spam registrations on sourcehut are from gmail than
| from the second-largest offender.
|
| Wow, I'm looking at Google in new way now.
| throwaway3699 wrote:
| I get what you feel, but on the flip side, if Google was even
| more aggressive at policing accounts and even hit 1-2 false
| positives, only for a scathing post to follow on a popular
| orange news aggregator, how is that a better situation?
| sanxiyn wrote:
| False positives are okay. Google gets scathing posts because
| they suck at customer service. Hire people to do customer
| service like everyone else does already!
| throwaway3699 wrote:
| How does customer service detect a false positive as such
| any more than a machine can, without 1) enabling social
| engineering 2) damaging the privacy of unrelated users by
| snooping in their account?
| scotuswroteus wrote:
| search for: "unsubcribe" click "delete"
|
| done
___________________________________________________________________
(page generated 2021-02-25 23:03 UTC)