[HN Gopher] Email obfuscation: What still works in 2023?
___________________________________________________________________
Email obfuscation: What still works in 2023?
Author : surprisetalk
Score : 133 points
Date : 2023-11-22 14:12 UTC (8 hours ago)
(HTM) web link (spencermortensen.com)
(TXT) w3m dump (spencermortensen.com)
| sdflhasjd wrote:
| Given that the percentages of blocked spam are just 17, 50, 67 &
| 83, it appears that the sample was likely just 6 spam emails
| which isn't a lot. Also doesn't really explain the methodology. I
| assume it's against a control of no obfuscation which was the 6
| emails.
| mbauman wrote:
| So is the real story here that putting your email on the open
| web simply isn't a significant "vulnerability" anymore?
|
| Honestly, why would you spider and scrape sites when you can
| just grab a leaked dataset?
|
| It would be slightly more interesting to put the canaries in a
| more commonly scanned location, like a plain-text bio field on
| a popular site (like this one... or FB... or LinkedIn...).
| NeoTar wrote:
| Spidering and scraping is 'more' legal?
| marginalia_nu wrote:
| Scraping is pretty dangerous if you're running a spam
| business, since there are a lot of honeypots out there, and
| you'll pretty rapidly find yourself blacklisted. Email
| spammers generally want verified email addresses.
|
| Since it's far safer to buy email lists from some broker,
| you tend to get a lot more spam for signing up for things
| than posting your email publicly.
| aendruk wrote:
| To offer one corroborating point, my email address is posted
| in cleartext on the front page of my website and it receives
| spam at a rate of less than one message per day.
| capableweb wrote:
| Another: Used the same email for ~2 decades, never hidden
| it, giving it out freely and available on lots of public
| websites including my own. Get around ~30 spam emails per
| day, all automatically marked as spam. Around 2-3 gets
| through my filter each month.
| listenallyall wrote:
| Difficult to extract any useful info from this post
| without specifying what or where your spam filter is.
| Does it run locally? On your own mail server? An ISP's
| default spam filter? Gmail? CPanel? Have you tweaked any
| settings?
| aendruk wrote:
| We've been discussing observed rates of _attempted_ spam,
| not the effectiveness of downstream filters.
| listenallyall wrote:
| > not the effectiveness of downstream filters.
|
| But he chose to mention that all spam were marked as such
| and that only 1 or 2 get through. Readers will naturally
| be curious what methodology and tools are in use.
| aendruk wrote:
| "Difficult to extract any useful info from this post"
| reads as undue criticism, as if preemptive satisfaction
| of your curiosity about the tangent was compulsory.
| listenallyall wrote:
| I disagree that the criticism is "undue"... a post that
| waves off spam as if it's a non-issue ("2-3 each month"),
| when in fact many people do struggle with it, helps
| nobody. A post actually explaining how you built or
| configured an effective spam filter, however, benefits
| the HN community.
| pixl97 wrote:
| Is that pre or post IP filter attempted spam?
|
| Or what I'm saying is, if the SMTP server blocks by IP
| first before determining what mail is being delivered
| then the actual rate of potential spam to any particular
| email address is not being discovered.
| aendruk wrote:
| For my case above we have a decent explanation of the
| "SMTP stage" policy [1] but no visibility into its actual
| metrics.
|
| [1]: https://www.fastmail.help/hc/en-
| us/articles/360060591393
| capableweb wrote:
| The conversation was about how many spam messages we
| received, not how many were caught or how. But since
| you're curious: sendmail + SpamAssassin, running on my
| own server.
| listenallyall wrote:
| But you specifically mentioned the percentage of messages
| which are marked as spam, and how many get through your
| filter. If you choose to include that info, people are
| going to have questions.
| Xamayon wrote:
| My experience is similar; I have my personal email, which
| has been in use since the early 00s, in almost plain text
| (@ replaced with <at>) on the about page of a fairly
| popular site which gets several million visits a month. I
| only get 5-10 spam messages/day, most of which are filtered
| without issue. I do get a decent amount of email, but not
| true 'spam'. It's mostly just crap I've signed up for over
| the years and can't be bothered to get rid of.
|
| I honestly get more at my work email, which has never been
| posted anywhere... I wonder if spammers have started to
| assume the easy to get email addresses are suspicious or
| not valuable for various reasons.
| jjav wrote:
| > So is the real story here that putting your email on the
| open web simply isn't a significant "vulnerability" anymore?
|
| Yes. The whole obfuscate my email thing is silly. I have the
| same email since the mid 90s, I post it without any care to
| wherever.
| Ringz wrote:
| You should have posted it right here to prove your point.
|
| /s
| nvy wrote:
| Spam isn't a problem for you, I take it? What are you using
| to filter spam out?
| NeoTar wrote:
| > Given that the percentages of blocked spam are just 17, 50,
| 67 & 83, it appears that the sample was likely just 6 spam
| emails which isn't a lot.
|
| This is mentioned, but a little hidden (in the description for
| URL encoding):
|
| > This is based on a small sample size: just six bots that were
| observed over a one-year period.
|
| I think the number of spam emails would be a bad measure, since
| a single scan could result in many hundreds of spam emails.
| PinguTS wrote:
| What about accessibility for people with visual impairments?
|
| The author has warnings for the last three version, because of
| usability. I see lots of red flags also for the other versions in
| terms of accessibility.
| RugnirViking wrote:
| ultimately I'm not sure that is a tractible problem; any method
| a screen reader would need to use to decode to speech would
| require getting the address in clear text via programatic
| means, which is something that one specifically needs to avoid
| if we want to stop bots scraping addresses
| iudqnolq wrote:
| You could have a button that triggers a captcha that then
| authenticated a request for the plaintext email.
|
| Or you could just use a spam filter.
| RugnirViking wrote:
| you know, you're right, an optional audio capcha alongside
| these could work quite well. I stand corrected !
| runnerup wrote:
| Not really a block. I've worked on CAPTCHA-solving bots.
| But it does increase the cost. I'd probably not scrape
| for emails anyways though, using leaked datasets is
| likely far more economical.
| stereo wrote:
| What I'm missing from the article is an explanation of what the
| control is. The email with no protection, which you'd expect to
| be the control, blocked 17% of spam - 17% of what amount of spam?
| How does the author quantify the spam that's out there?
| RugnirViking wrote:
| it says in the page that its because all the mails were on the
| same page (and presumably then scaped by the same bots), and
| yet a couple mails were recieved by other addresses and not by
| the unprotected one. So it would seem that the control is the
| set of unique mails recieved by all addresses.
|
| "Surprisingly, the unprotected email address appears to have
| blocked a spam email. Either that message wasn't received, or
| an extra message was sent to one of the protected email
| addresses."
| progval wrote:
| <a href="mailto:email@example.com">email</a> blocked 17% but
| email@example.com blocked 0%
| OJFord wrote:
| Really? Seems like you'd have to actively try to be blocked
| by that, i.e. to extract 'email' but not the mailto. I
| suppose if you didn't actually scan the HTML, but instead all
| the contained text.. maybe it just tests an implementation
| detail basically.
| apitman wrote:
| The problem with email is that it fills so many roles. It's both
| the only universal chat program between trusted entities and the
| only universal way to allow people to cold call you.
|
| If you're really worried about spam, I recommend just keeping a
| different email address for each separate purpose.
| thdc wrote:
| Here's an older submission about using ChatGPT to de-obfuscate
| the more basic methods:
|
| https://news.ycombinator.com/item?id=38150096
|
| Some comments claim that it can break some of the more complex
| techniques presented in the article. I've tried it a few times
| myself with varying results that tend towards not working.
| throw555chip wrote:
| Feels like ChatGPT has to show up in every Hacker News thread
| now.
| boznz wrote:
| Seriously if you are still running email with HTML and scripting
| enabled you need to turn it off. Plain text is fine for
| everything use a secure messaging program like signal for
| anything else.
| earthboundkid wrote:
| I'm surprised URL encoding works so well. I've been base64
| encoding and using JS to decode, but if URL encoding is that
| reliably, it's probably not worth it.
| notatoad wrote:
| i'd be curious to know, of the spam emails received, what
| percentage were blocked by basic spam filtering?
|
| in 2023 email spam seems like mostly a solved problem - i very
| rarely get any that actually makes it through to my inbox. trying
| to solve the spam problem by protecting your email address from
| becoming public might have seemed like a valid strategy 20 years
| ago, but we have better tools now.
| gspencley wrote:
| The limitation of spam filters is false positives. For most
| people it's probably not a big deal to have one or two messages
| land in the spam filter .. then someone follows up and says
| "Hey did you get that legit email I tried to send you? I
| haven't heard back." But for certain business accounts, the
| amount of spam + false positives can get to unmanageable
| levels, where important emails are flagged as spam and left
| undiscovered because sifting through the spam folder regularly
| is as time consuming and annoying as if the spam just went
| straight to inbox.
| pizzafeelsright wrote:
| What service?
|
| I have gmail and O365 and some other web hosting provider. My
| work email is unknown, probably filtered 5x, plus on O365 so
| limited to known contacts.
| ape4 wrote:
| I wonder how much mail the example.com domain gets!
| theblazehen wrote:
| Given the MX record resolves to `.` I suspect not any
| (successfully)
| rpigab wrote:
| None, I always use those in tests, because they're owned by the
| IANA, so unless random domains that have no mx records now, but
| might in the future, the example org, net and com are safe.
|
| I do this because sometimes in companies, people will put DB
| dumps in the wrong environment that has an actual SMTP going to
| WAN, then shit happens. I also make sure the environments have
| a dummy smtp or mailcatcher, but it's better to be safe than
| sorry.
|
| See RFC2606: https://www.rfc-editor.org/rfc/rfc2606.html
| tyingq wrote:
| > 1.3 No protection Blocked 17% of spam
|
| I read the explanation, but it just sounds like something is a
| bit off in the methodology and metrics. Or at least in my
| understanding of them :)
| kmeisthax wrote:
| What they mean is that at least one bot doesn't read mailto:
| links and only cares about plaintext e-mails.
| tyingq wrote:
| Ah, thank you for that. I feel a bit dumb now, but that would
| have been a good inline explanation.
| gwbas1c wrote:
| Might want to add an intro paragraph. I initially thought that
| "Email Obfuscation" meant something different, like trying to
| trick a mail client.
| prajaybasu wrote:
| Tip: Almost none of the scrapers and bots run WebAssembly :)
| rekabis wrote:
| Intriguing, but has this been tested in the wild alongside a
| control eMail address in plaintext?
| joeig wrote:
| This has worked incredibly well for about two years now.
| davidy123 wrote:
| I have used a keyed email address scheme since the mid 1990s
| (me_context@domain.org). 99% of the email I get now is sent to
| the same 5 addresses, in particular me_boingboing, me_linkedin.
| Those addresses were never on a web page. The keyed addresses I
| used for my company occasionally get spam, but it's generally on
| topic (offering relevant services, though still imo spam). I also
| have a gmail address that is a common first/last name but I don't
| send from that address, I get a lot of presumably guessed address
| spam there. This suggests quite a bit of organization in how spam
| works these days.
| hunter2_ wrote:
| It's an interesting claim that defeating xor requires a JS
| interpreter but defeating the others (e.g., concatenation, rot18,
| etc.) does not. I mean, if a bot author wanted to scrape a site
| or sites where all obfuscation used the same xor routine, the
| author (not the bot!) could just read (not execute) the JS and
| customize the bot accordingly. Granted it restricts things to
| more targeted attacks rather than a bot that just crawls
| arbitrarily across the web, which is noteworthy.
| switch007 wrote:
| On the topic of spam..."Spam" to me these days includes email
| from companies who abuse the soft opt-in, service emails and
| "existing commercial relationship" rules.
|
| Cart abandonment, tips on how to stay safe online, requests to go
| paperless, newsletters, claim your free subscription that came
| with your purchase, thank you for your purchase (separate to the
| order confirmation), requests to leave a review, continue your
| application, offering support, "you haven't logged in for a
| while", new login from $device, thank you for completing step X
| ... it's endless.
|
| An endless torrent of excuses to get the their company name in
| front of your eyeballs.
| brightball wrote:
| "new login from $device"
|
| That's a security precaution, not spam.
| encom wrote:
| Except all my logins are from "a new device" because I use
| Cookie AutoDelete, there's no way to opt out of this spam,
| and I don't give a shit if anyone DID actually hack my Google
| account because it's only a way to keep track of my Youtube
| watch history and subs.
|
| I filter these as spam.
| brightball wrote:
| As long as you know the reason. For everybody else, it's
| not spam.
| switch007 wrote:
| It's not a precaution. A security precaution would be
| blocking the login because of unusual activity
|
| I get so many, I tune out. I bet I'm not alone
| brightball wrote:
| That's ideal, but it's a lot harder and involves more
| invasive tracking to determine normal activity.
|
| You really have 2 options from a UX standpoint. Either
| allow the login and notify you, which gives you less
| friction in your experience with the application if it
| really is you.
|
| Or they can stop you right on the login screen and send you
| an email with a code or a link to click before you go
| further. It's more secure, but it adds friction on the more
| common case that it really is you.
| calvinmorrison wrote:
| The big spam to me at work is now people using linkedin and
| other tools to automatically find people's emails, and send
| them targted automated email that are, plaintext old outreach
| emails. for some reason this is just not getting caught in my
| spam filter.
|
| Frustrating! I never opted in!
| throw555chip wrote:
| I deleted my LinkedIn account when it started becoming more
| like Facebook (nothing against FB, Wide likes it) but FB
| isn't for me.
|
| Also, I removed my public open source projects from GitHub
| after Microsoft bought them then removed them from GitLab too
| when the AI warnings about stealing copyrighted code started
| to sound more real.
| calvinmorrison wrote:
| I removed my github when they took my code and printed and
| put it in some archive. All rights reserved?
| Sephr wrote:
| I've discovered that the most effective trick is to use an email
| address that is considered invalid by most email scrapers but
| valid by most mail servers.
|
| I have an unobfuscated mailto: link on my blog and I receive
| barely any spam to my public email address, ~@eligrey.com
| brightball wrote:
| Same. I took over a programming conference earlier this year
| and it uses a .codes domain.
| jonchurch_ wrote:
| OP was saying that their email is literally ~@eligrey.com
|
| He's saying this works bc scrapers might not match a left
| hand side of an @ that contains no alphanumerics
| brightball wrote:
| I get that. I think the less common domain extensions are
| having a similar effect right now though.
| IshKebab wrote:
| I guess you have a different email address that you use for
| website signups, shopping etc?
| justinhj wrote:
| ~@eligrey.com implies it's not you, I would add an alias for
| ~~@eligrey.com for the comfort of any programmer friends.
| forgotpwd16 wrote:
| But ~user is user's home for Unix friends.
| dom96 wrote:
| I'm using CSS-based obfuscation on my website[1] and it works
| quite well.
|
| 1 - https://picheta.me
| pizzafeelsright wrote:
| I have successfully abandoned email.
|
| Sites that require email for logins get my common email that I do
| not check unless required to reset a password or links for
| authentication.
|
| Long form communication is done via texting or shared chat.
| mock-possum wrote:
| How do you review old communication from 8 years ago? Or find
| texts with particular criteria - "from x, subject like y,
| received within a week of date z, has attachment"
| Ringz wrote:
| He gave up on that too and leaves it to his parents.
| BeetleB wrote:
| > Long form communication is done via texting or shared chat.
|
| That's a nightmare.
|
| In reality, long form just doesn't exist in chat/text. For me,
| long form is at least 500 words. Years go by before I get "long
| form" text/chat of that order.
| hcurtiss wrote:
| Total aside, but I recently purchased a .us domain name for the
| first time. I was on my phone and signing up with the Namecheap
| app. I thought it odd that it didn't ask for privacy protection,
| but I figured I'd quickly turn it on after the fact and didn't
| get around to it until later that night. I used our home phone as
| the phone number for the site. The next day we got no fewer than
| fifty spam phone calls, and the email address I offered was
| quickly inundated with over a hundred spam messages. It was
| absolutely nuts. Fortunately, I used an email alias and just
| turned it off. Likewise, I have a Callcentric VOIP number for the
| home, so I was able to add a "press six to continue" prompt, but
| I can't change that number without a lot of pain and I now no
| longer get the automated phone calls from the school (we still
| get them through the app). Since then, the call volume has
| dropped off, but I can see in the logs that we still get several
| per day.
|
| After all of that I learned that privacy protection is not
| available for .us domains. No wonder they're so cheap.
|
| A long way of saying, if you sign up for a .us domain, definitely
| do not give them your real information.
| ulrischa wrote:
| Most scrapers search for anchor tags. So a span with the aria
| role=link could work too. This will also keep accessibility.
| Because I'm not shure what a screenreader will read for the
| encoded E-Mail. https://developer.mozilla.org/en-
| US/docs/Web/Accessibility/A...
| acylia wrote:
| I use JavaScript obfuscation[1] on my website and get very low
| amount of spam. However I am confident that some modern scrapers
| are using headless browsers and wait for the DOM to be fully
| loaded before extracting the email from the text data.
|
| [1]: https://ianisbernard.com/contact/
| hurtuvac78 wrote:
| It may not be the clerverness of the obfuscation that beats the
| spammer's bot. If you go to such lengths to obfuscate your email,
| then you may simply not be a bad target for my bot: response
| rates tiny. No good ROI for the spammer.
|
| Does not matter, it works then!! But not because of technical
| cleverness.
|
| No idea if what I am saying is true.
| djha-skin wrote:
| Turn your email into a PNG. It's absolutely easy to read, and My
| brother who worked for a scraping company said they haven't
| cracked that yet (as of 5 years ago). Even today, you would need
| a neural net to do it which makes it more expensive. They would
| also need another neural net to figure out whether or not it was
| an email address! It's just an image after all.
| latchkey wrote:
| I recently created a new company domain with email on it and I'm
| amazed at how quickly I'm getting spam on it.
|
| To be fair, most of it is directly from LinkedIn, but still...
| OmarAssadi wrote:
| While it doesn't stop spam, I have been using a catch-all email
| system for a while now.
|
| The benefit is that I know where someone got my email from, and I
| can then try to figure out whether the place has been
| compromised, or whether they're selling my email, etc. And I can
| just blacklist that particular address forever as well.
|
| Previously, I just did whatever@mydomain.tld, but I've switched
| to something similar to blame.email [1].
|
| This makes my emails look a little weirder, but it has stopped
| the weird looks I'd get when walking into a physical place, like
| my doctor, and telling them "Yeah, email me at <doctor's
| name>@<first><last>.com".
|
| It also makes it less obvious that its effectively a throwaway
| email, particularly combined with my domain; it looks fitting.
| And since each address is salted and hashed, it pretty much
| eliminates the risk of someone successfullying trying to phish me
| by sending me an email to something like
| `paypal@<first><last>.com`.
|
| Lastly, on my HN profile and elsewhere, I've got my "email", but
| despite them being unique, I still don't want to have to rotate
| it if it gets picked up by a spambot, so I've tried to do some
| plaintext simple "obfuscation" like in the article.
|
| I went for <address> ~AT~ <domain>.<tld> -- with the "AT" being
| Cyrillic rather than Latin - I figure at least some will get
| tripped up by not being able to use purely English regex.
|
| So far, I have yet to receive any spam with that strategy. Maybe
| I'm lucky or just not getting indexed, or maybe it's working a
| little.
|
| Still torn about how to handle Git or copyright/license headers,
| though; those addresses need to last a long time, in case anyone
| needs to reach out and ask for re-licensing/etc, and I figure
| it'd be annoying doing different emails for each repo.
|
| [1]: https://news.ycombinator.com/item?id=31820502 /
| https://blame.email/
| teravolts400 wrote:
| What worked really well for me: storing email in a base64
| encoding and then decoding it whenever user hovered over the link
| for my email.
___________________________________________________________________
(page generated 2023-11-22 23:01 UTC)