[HN Gopher] Information about a recent security incident at Mail...
___________________________________________________________________
Information about a recent security incident at Mailchimp
Author : guiambros
Score : 95 points
Date : 2023-01-21 05:07 UTC (17 hours ago)
(HTM) web link (mailchimp.com)
(TXT) w3m dump (mailchimp.com)
| davewritescode wrote:
| I just don't understand how this keeps happening?
| mrmincent wrote:
| Funnily enough I'm studying a cyber security course at the moment
| and had to find examples of social engineering for an activity,
| and came across this yesterday. I shared it with my classmates
| and they just couldn't believe that MailChimp been hit 3 times in
| a year with what sounds like the same social engineering attack.
|
| It does make you wonder when there's going to be a 4th, I
| certainly don't think I'd want my customers contact details in
| their system at the moment.
| NelsonMinar wrote:
| This problem is almost certainly Mailchimp-specific. The talent
| was decimated after the owners sold it to Intuit.
| kneebonian wrote:
| That's the thing you can't outsource IT security because in
| order to be secure you need to have institutional
| understanding.
| AtlasBarfed wrote:
| But if your devs are outsourced...?
| iamacyborg wrote:
| > This problem is almost certainly Mailchimp-specific.
|
| It's not. Multiple marketing automation/email service
| provider platforms were affected by similar hacks last year.
| NelsonMinar wrote:
| Once is a hack. Three times in a year is a systemic
| business problem.
| echelon wrote:
| Mailchimp was _gutted_.
|
| You have to understand the sort of mythical place the
| company had in the nascent Atlanta tech scene. It was one
| of the first Atlanta tech unicorns and was growing and
| hiring at a rapid clip. The founders promised never to sell
| and that they would give annual bonuses in exchange for not
| offering employee equity. The office was brimming with art,
| music, and perks that were unusual for Atlanta at the time.
| They said they were going to build a generational Atlanta
| tech company.
|
| I desperately tried to recruit my friends that worked at
| Mailchimp away, because I saw the writing on the wall. I
| had joined a promising startup offering significant equity,
| and more importantly, building real value in the world. To
| my disappointment, none of my Mailchimp friends joined me.
| They liked the culture too much, felt comfortable, and
| continued to tout the bonuses they were receiving despite
| Mailchimp offering lower total comp. I never stopped trying
| to pull them away, but I didn't succeed. I received counter
| offers and heard the Kool Aid stories. "The founders are so
| cool and so generous."
|
| The job itself isn't that interesting, either. Marketing
| and email advertising isn't intellectually attractive. To
| top it off, Mailchimp has a disgusting PHP monolith that
| I've heard is like pulling teeth to work with.
|
| I was right in the end. I walked away with a huge eight
| figure exit. Mailchimp employees got pay cuts, benefit
| cuts, and saw their culture ravaged. Intuit took what
| little niceties they had and threw it all to the wind.
| Macy's and Home Depot have better software jobs.
|
| I really don't like Mailchimp. The company is rotten.
| axpy906 wrote:
| You worked at Autotrader?
| AviationAtom wrote:
| The sad part is it's one of the hardest components to solve.
| High quality training can instill a healthy dose of paranoia,
| that is necessary, though.
| Enginerrrd wrote:
| I'm convinced there's no way to solve this except for regular
| "red shirt drills" linked to your job performance. (This is a
| term I borrowed from life-guarding where someone randomly
| pretends to drown to see if/how fast the guard will catch it
| and assess the skills of their response.)
| AviationAtom wrote:
| Good detection and response is a must, but creating a
| security culture helps greatly. Training, test campaigns,
| and further training for those that fail, does help.
| Nothing is a silver bullet, that is why we emphasize
| "Defense in Depth."
| codetrotter wrote:
| > a term I borrowed from life-guarding where someone
| randomly pretends to drown to see if/how fast the guard
| will catch it and assess the skills of their response
|
| They have a similar concept in cybersecurity.
|
| The Red Team
|
| > a group that plays the role of an enemy or competitor to
| provide security feedback from that perspective. Red teams
| are used in many fields, especially in cybersecurity,
| airport security, law enforcement, the military and
| intelligence agencies.
|
| https://en.wikipedia.org/wiki/Red_team
| nibbleshifter wrote:
| Even that doesn't work - some companies do exactly that
| with regular phishing exercises etc.
|
| Employees get really good at spotting the tests.
| AviationAtom wrote:
| The technical ones may, but they're also less likely to
| fall victim to the phishing schemes.
|
| It's those non-technical folks in admin roles that you
| tend to have to worry about most. Those folks aren't so
| good at picking out the tests.
|
| I actually found KnowBe4 to work fairly well on a large
| scale.
| nibbleshifter wrote:
| The non technical staff seem to learn to spot the knowbe4
| shit real quick lol.
| jabroni_salad wrote:
| Yep. All of KB4s stuff looks like it came from a single
| designer. You need to go through your spam filter and
| make custom templates from whatever you are receiving,
| and also make lookalikes from real business messages.
| Bankers doing wire transfers to wrong accounts because of
| a forged document has been a serious problem in my sphere
| lately.
|
| It used to be that if a mailbox got compromised they
| would just send spam about lottery wins and boner pills.
| Now they watch your messages and reply to a real request
| with a good looking response. A correct expected reply in
| a chain from an authentic account, just some numbers have
| changed. Then they will steal your contacts and register
| a similar domain and try to impersonate you.
| AviationAtom wrote:
| We had almost this exact scenario (look alike domain)
| play out with a customer. Their accounts payable
| department almost paid out half a million to a scammer.
| Fortunately the employee at the customer accidentally
| replied to our actual email address and our folks knew
| better and picked up the phone. The customer insisted up
| and down that their email system had not been
| compromised. It took telling their IT folks what and
| where to look before they finally realized they were
| compromised. Good IT/security teams make all the
| difference.
| AviationAtom wrote:
| It's all in how much effort you put into it. It's quite
| versatile and their customer success team is phenomenal.
| We had multiple pretty smart employees accidentally fall
| for it the first few campaigns and immediately reach out
| when they realized what they did.
| yjftsjthsd-h wrote:
| > Employees get really good at spotting the tests.
|
| Then surely either the test isn't good enough, or the
| problem is solved?
| syrrim wrote:
| If being able to spot the tests doesn't confer ability to
| spot real phishers, then the tests aren't very good.
| jacquesm wrote:
| It is. And then you receive mails from say Google or some
| other large company that look _exactly_ as though they are
| very bad forgeries of what you 'd expect. 'Action required',
| some weird vaguely related domain name, a bunch of links to
| click that definitely have nothing to do with the matter at
| hand.
|
| With crap like that it isn't all that strange that some of
| these attempts get through and on the social engineering side
| it can't be all that much better. After all we're
| continuously conditioned to trust that which we really
| shouldn't be trusting. Right up to the point where the party
| on the other side is abusing that trust.
|
| One example: my _real_ bank calling me and then asking me to
| verify that it is really me by asking me all kinds of
| information about my account... And from a number that isn 't
| associated with them. And yet, it was very real and when I
| refused to answer their questions they took it pretty badly
| as though _I_ was a scammer...
| Eleison23 wrote:
| [dead]
| guiambros wrote:
| I find it hard to believe it was just 133 accounts. I just got an
| email saying mine was one of the accounts compromised, even
| though I haven't used Mailchimp in over 12 years, and don't
| remember having ever used them for anything material.
|
| From what I remember either I deleted the account, or they closed
| for inactivity, so it's weird they still had my data.
| mrslave wrote:
| Unless they expressly commit to deleting it, or you were paying
| per GB for storage, consider it saved forever.
| password4321 wrote:
| I failed to convince them to undelete my account (the
| deadline is currently 2 years), but there is an HN support
| incident:
| https://news.ycombinator.com/item?id=18715866#18717279
|
| Back in the day I thought "who better to retain a mailing
| list" but turns out I'd have been best off with a Google
| Sheet. Now I have to scrounge around in backups to see if I
| downloaded it anywhere... maybe I should start a support
| ticket here of my own!
| davewritescode wrote:
| > I failed to convince them to undelete my account (the
| deadline is currently 2 years), but there is an HN support
| incident:
| https://news.ycombinator.com/item?id=18715866#18717279.
|
| A lot of companies have a habit of keeping data but not
| migrating dead customers to newer schemas and systems.
| blowski wrote:
| True, although I think the OP was highlighting the oddness of
| such a minor account being one of those compromised.
| ThePowerOfFuet wrote:
| [flagged]
| Pako wrote:
| GDPR to the rescue! /s
| daneel_w wrote:
| Does the scanning software in the update have any opinions about
| installations of LibreOffice? Asking because Microsoft really has
| a strong opinion about _not_ using Microsoft Edge, whose shortcut
| icon they keep resurrecting from the dead to haunt my desktop.
| Twice just the past 10 days.
| tybit wrote:
| While they don't specify it sounds like they don't even require
| 2FA to access their systems?
| Eleison23 wrote:
| [dead]
| steeleyespan wrote:
| I don't like Mailchimp, they won't allow any emails that even
| seem to mention anything crypto-related. Super annoying.
| tiffanyh wrote:
| Off topic: what do people think of MailChimps quirky web design.
|
| Like it or not?
| DavidIsATool wrote:
| Jarring, overwhelming, everything is too big, hard to process
| quickly. Not a fan.
| fukawi2 wrote:
| Again? This is the third incident in ~12 months IIRC.
|
| Is _every_ other company hiding their incidents, or are MC
| developing a habit of actually not "taking the security of our
| users' data seriously"?
| teknopaul wrote:
| Might it be that Mailchimp has Internet facing customer service
| facilities?
|
| Our lot doesn't, if you got and admin user and password that,
| in itself, doesn't get you very far.
| iinnPP wrote:
| The problem that people are overlooking is the profit motives
| of the people capable of penetrating systems. These motives are
| dictated by how companies treat security. By:
|
| - Feet dragging on free work or extremely low paid work - Not
| hiring any competent person or not paying enough to attract
| that person. - Refusing to fix internal problems that create
| the issues.
|
| The reality is, there are so many more people willing to use
| technology to scam you than there are people employed to stop
| those people. Probably because these currently bad people are,
| by design, hard to discover and thus go unaccounted for
| entirely.
|
| Nobody pays for the skillset and what people hire for clearly
| doesn't work. Until companies see consequences, nothing will
| change. So expect this not to change.
|
| Why on earth are you giving these companies your money and data
| anyway? It's truly not difficult to avoid breaches. Do a tiny
| bit of digging into the security model before you unload your
| list of customers into their servers. Take some bloody
| responsibility yourselves too.
| beardedwizard wrote:
| What exactly does Mailchimp publish about their security
| model that would allow anyone to predict this breach?
| iinnPP wrote:
| You should never trust the marketing department to
| accurately convey any security model and should assume
| anything published regarding security has been approved by
| marketing.
|
| Avoid companies that require data which isn't required.
| Overlook companies with previous vulnerability disclosures
| that leave you facepalming. Ignore companies that pre check
| marketing anything. Blacklist the companies with GDPR
| ignorant cookie "consent." Black hole companies that kill
| the planet to bring you advertising while telling you
| you're killing the planet.
|
| I would make a joke about what remains. It wouldn't be
| funny. There are still lots of companies left however and
| those are the ones that gave at least a signal about caring
| about you.
|
| And for the love of all things. If your company is forcing
| employees to sign up for garbage data farm software, SAY
| SOMETHING. Your data is important.
| InCityDreams wrote:
| >Blacklist the companies with GDPR ignorant cookie
| "consent."
|
| Could you explain further?
| dividuum wrote:
| As purely an outsider and not OP: it seems their
| contractors and support people have direct access to
| critical customer data simply by typing password and/or
| totp into a form. Seems it's not always the correct one.
|
| * Limiting access (maybe to email subject instead of all
| content?) might prevent some fallout as it might be more
| difficult to extract password reset emails sent by
| customers.
|
| * Limiting access from certain IP sources might make it
| more difficult to use captured login credentials.
|
| * Hardware key based authentication might prevent the type
| of phishing that seems to have happened here.
| iamacyborg wrote:
| * Allowing customer to only allow MC staff access for
| fixed intervals when support tickets are raised.
| BrandoElFollito wrote:
| The third case is very common.
|
| I would not even use the word "refuse" Sometimes the legacy
| system is so legacy that fixing it is a many years project.
| Sometimes you have such a messy environment that rotating
| credentials means a general crash. Sometimes you have legacy
| software that must stay legacy and you cannot patch + that
| software is so ingrained into your system that you cannot
| isolate it. Sometimes ...
|
| This is all bad design and bad architecture from scratch. Or
| "good architecture 25 years ago".
| kneebonian wrote:
| What I've found is it isn't companies won't pay for good
| people, they very much will, as a sec engineer you can make
| 200k pretty easily with just a few years of experience in a
| lot of places.
|
| The problem is once these people are hired they come in and
| aren't given power to do anything, or the "drive for
| security" isn't actually present in the organization once
| people realize it might actually force people to focus on
| things other than pushing features as fast as you can.
|
| This issue is further exacerbated by the fact that there is
| not an insignificant portion of info sec guys that believe
| they have to come in and save the organization from
| themselves and that they are the heroic white knight
| valiantly protecting the company from the unwashed masses of
| wild wild west cowboy developers and incompetent sys admins.
| rprospero wrote:
| I once worked with one of those heroic security guys
| before. He setup the firewall so that the public website
| was only accessible from a white list of known up
| addresses. New users would need to submit their IP address
| to him in person before they'd be allowed to browse the
| site.
|
| He insisted that this was industry best practice and it
| took two weeks before the site was online again.
| diarrhea wrote:
| At that point just take it offline entirely. For security
| of course.
| LewisVerstappen wrote:
| Well, considering how absurdly expensive they are compared to
| other ESPs, the money has to be going somewhere?
| iamacyborg wrote:
| They're expensive for their feature set but cheap compared to
| most good ESP's.
| mpclark wrote:
| What are savvy folks using instead of Mailchimp nowadays? I run a
| B2B news site and use Mailchimp for its daily and weekly
| newsletters. I like that MC handles the subscriber management and
| deliverability side of things, but they seem to be moving away
| from doing email well and towards random value added small biz
| marketing stuff. And my monthly bill has almost doubled since the
| Intuit acquisition.
| thereal_ba wrote:
| Aside from the above, Audienceful is a new company in the
| emailing space as well: https://www.audienceful.com
| password4321 wrote:
| 2022 Ask HN: What service are you using to send marketing
| emails? https://news.ycombinator.com/item?id=32535397
|
| https://hn.algolia.com/?query=mailchimp%20comments%3E10
|
| 2021 emailoctopus postmark mailersend|lite sendgrid campaign-
| monitor constant-contact
| https://news.ycombinator.com/item?id=28516219&p=2#28516522
|
| 2016 (self-hosted): mailtrain dada-mail-project+ses mailwizz!
| maybe:sendy | mailgun sparkpost
| https://news.ycombinator.com/item?id=11424189
|
| 2017 ses sendy mailblast mailgun postmark mailtrain mail-for-
| good kevy mailget https://news.ycombinator.com/item?id=15493127
| (+per-recipient customization tools)
| mabbo wrote:
| Do MailChimp employees not need a Yubikey or some other secure
| OTP generator to access customer data?
|
| Or does the attack involve the user sending their OTP as well?
|
| Either this is a pretty heavy handed attack, or MailChimp aren't
| following best practices in terms of security.
| vlovich123 wrote:
| For a corporate setting where you can replace keys easily, I
| would choose Fido keys and SSO protecting all access into the
| network instead of OTP/VPN as that is generally immune to
| social engineering attacks / leaked passwords. Additionally,
| it's trivial to manage those keys in a corporate setting
| centrally.
|
| For personal use, I still prefer OTP because I'm not fully
| comfortable that I won't lose some key and forget to revoke it
| everywhere and/or lose permanent access. Phones are getting
| very good at being that security key with backup but I'm a
| little hesitant to do that because I'm not sure of the interop
| story.
| rmatic wrote:
| while most implementations don't require 2 security keys,
| most services highly suggest that you enroll 2 keys, and that
| you keep one in a safe place for backup.
|
| e.g. Apple's new icloud Security Keys support requires 2 keys
| to enroll
| lfodofod wrote:
| You just ask the target to install teamviewer or anydesk...
|
| These are probably support staffers working from personal
| computers at home using some sort of remote desktop software,
| hard to enforce any security policies.
| mabbo wrote:
| This is one of the places where ChromeOS is an interesting
| option. If the employee literally can't install software,
| it's very hard to get owned by malware.
| iamacyborg wrote:
| > we take the security of our users' data seriously
|
| Clearly they don't if staff are able to just access customer
| accounts.
|
| SaaS companies _really_ need to stop giving themselves access to
| customer accounts.
| InCityDreams wrote:
| >> we take the security of our users' data seriously
|
| I take the war in Ukraine very seriously, and a few other things,
| but I'm still going on the piss tonight.
___________________________________________________________________
(page generated 2023-01-21 23:01 UTC)