[HN Gopher] Zulip Cloud security vulnerability with reusable inv...
___________________________________________________________________
Zulip Cloud security vulnerability with reusable invitation links
Author : Brajeshwar
Score : 76 points
Date : 2022-02-26 17:07 UTC (5 hours ago)
(HTM) web link (blog.zulip.com)
(TXT) w3m dump (blog.zulip.com)
| KennyBlanken wrote:
| https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-2170...
|
| > Zulip Server version 2.0.0 and above are vulnerable to
| insufficient access control with multi-use invitations. A Zulip
| Server deployment which hosts multiple organizations is
| vulnerable to an attack where an invitation created in one
| organization (potentially as a role with elevated permissions)
| can be used to join any other organization. This bypasses any
| restrictions on required domains on users' email addresses, may
| be used to gain access to organizations which are only accessible
| by invitation, and may be used to gain access with elevated
| privileges. This issue has been patched in release 4.10.
|
| Why does it feel like the completely misleading title was
| intentional to try and drive traffic to / SEO this crappy copycat
| CVE site?
| mosselman wrote:
| Is anyone using Zulip and has it made a difference to Slack for
| example?
| aidenn0 wrote:
| I liked it a lot, but I'm not an Apple user, and apparently the
| iOS app was (is?) not great. For me, it's like a vastly better
| version of NNTP, while slack is more of a slightly improved
| IRC.
| temp wrote:
| We use it at work. We first switched over to Mattermost, then
| to Zulip. Mattermost seemed like a clone of Slack and we were
| initially "satisfied", but didn't see it get improved that much
| nor noticed any attempts of it trying to tackle other problems.
| Then they raised their prices to even higher than Slack's,
| which we took as a bit of dick move seeing as we were self-
| hosting it + it has nowhere near the experience of Slack. Push
| notifications in general didn't seem to be that good.
|
| Zulip has a nice spin on the way discussions work with their
| concept of "topics". It lends itself well to how we
| conceptualize the discussions across our development teams,
| without unnecessarily creating many ad-hoc temp channels
| (something we'd have done on Slack). We're liking it thus far.
| That being said I do need to check why it's using about 35 gigs
| of RAM on our server.
| DoItToMe81 wrote:
| 35 gigabytes?! How many people are using that thing?
| temp wrote:
| Only about ~40 users, so it's definitely an issue. Haven't
| investigated yet but this reminded me that we should.
|
| (edit: actually it's now down to 17 gigs)
| tabbott wrote:
| Please report this in https://zulip.com/development-
| community/; this certainly sounds like a memory leak. We
| aren't aware of any other reports of memory leaks in
| Zulip in the last couple years, so we'd be very happy to
| help track down what's happening on your server.
|
| https://zulip.readthedocs.io/en/latest/production/require
| men... is a useful reference for folks interested in
| memory requirements for a self-hosted Zulip server.
| DoItToMe81 wrote:
| I briefly used Zulip as part of a friend's community, rather
| than work. My experience with it was that it's great at first,
| but when you start to get masses of messages and users, the
| client becomes laggy and unresponsive. Scrolling through
| messages particularly sucked because of this.
|
| I'm not sure if it was just our setup, because when I checked
| the official Zulip web chat, I had the same problem.
| luxurytent wrote:
| We use it at our small (15 person) startup. I like that topics
| are light weight, like git branches, but despite that we
| struggle to use them effectively (generally dumping thoughts in
| a select few topics)
|
| One major quirk of the interface is that can be difficult to,
| at a glance, identify what topic/dm you are actively on, which
| has lead to a handful of "Oops, what was supposed to be a DM"
| message deletes.
| Xylakant wrote:
| We've struggled with that in the past as well, but it got
| better over time. One thing that helps is to aggressively
| start moving messages to new topics once things go beyond a
| few in a general topic.
| sonofhans wrote:
| We used Zulip at $job[-1] and it was awfully painful. I've used
| nearly every chat client invented since the BBS days, and Zulip
| was one of my least favorite. We're using Slack now at $job and
| I vastly prefer it (even though I still call it "IRC").
|
| The biggest selling points of Zulip, from the POV of the people
| who put it in place and truly loved it, were that it's free/OS
| software, it has explicit threads as a first class citizen, and
| that it's very configurable. All those things are true.
|
| The day-to-day use was painful, though. It was riddled with UX
| problems, e.g., forgetting state between restarts, inconsistent
| hotkey support, multi-action use paths for common cases,
| frequent full reloads of its web view. I'd much rather use
| vanilla IRC than Zulip.
|
| Slack is just enough of a shim layer on IRC, and with good
| usability. It has threads, and in use it's not a problem that
| you can't name them like Zulip. The integrations are nice to
| have (e.g., type `/zoom` to start a Zoom meeting) but not a big
| deal. The UX is clear and predictable and (mostly) stays out of
| my way.
| tabbott wrote:
| Hi! I lead the Zulip project.
|
| I'm sorry to hear you had a bad experience with Zulip. If you
| can share details on the UX problems you had, perhaps in
| https://zulip.com/development-community/ where we can have an
| extended conversation, I'd be very appreciative. I personally
| review every issue reported in zulip, and some of the
| specific problems you cite are not unfamiliar and surprising
| to me. Specifically:
|
| * I'm not aware of a plausible explanation for Zulip
| frequently doing full reloads of the web view. Zulip's core
| design is to live-update everything [1] and there's only a
| handful of code paths that can trigger a full reload (the
| most common being when the server version was updated, and in
| that code path, the client has an algorithm where it waits up
| to 30 minutes for a moment when your window is idle and not
| focused to reload to get the updated version, to minimize the
| chance of disruption). * I'm also not sure what what state
| might be forgotten between web client restarts; almost all of
| our client UI updates happen via the client asking the server
| to change something, and the client updating via the same
| server-client push mechanism that updates other clients (the
| main exception to this is sending messages, where local echo
| is important for UX reasons). And at least the server-
| initiated reloads are designed to preserve your precise
| scroll position, compose box state, etc., and that's been
| true since ~2013. (Though we did fix a couple bugs where
| compose box state was not properly preserved in the last
| year).
|
| I know you're not using Zulip currently, so your memory may
| be imprecise, but any additional detail that could help us
| reproduce what you experienced would be awesome!
|
| [1] https://zulip.readthedocs.io/en/latest/subsystems/events-
| sys...
| dsr_ wrote:
| We use a self-hosted Zulip. It's the first chat system to make
| everyone reasonably happy -- devs, IT, customer service, and
| most of the execs.
| asah wrote:
| Love Zulip. Very hackable. Terrific core team who really care.
| jamesvl wrote:
| Personally, I like Zulip way better than Slack because of how
| it does message threads: _channels_ that have conversations in
| _topics_ "makes sense" to me.
|
| Pros: Server maintenance is blissfully minimal. Message
| threading that actually works. No server issues or downtime in
| the past 4 years. (Every time I see a HN "Slack is down"
| posting, I smile to myself and continue my work day...) It runs
| great for ~50 employees on an AWS t3a.large instance. Importing
| all the company's previous messages from Slack worked
| perfectly.
|
| Cons: If you have people used to Slack, there will be
| resistance / a learning curve. Knowing, "when do I create a
| topic?" takes practice. The mobile and desktop apps are not as
| polished as Slack's. Not as many integrations as Slack. You
| have to know enough to manage and secure a server yourself.
|
| The Zulip devs accept Github donations, btw, if anyone cares to
| support their work. (We do; no affiliation other than we're
| happy to have a private, self-hosted alternative.)
|
| edits: punctuation and adding that instance was a ".large"
| kibwen wrote:
| Sort of a weird context to ask for Zulip recommendations, but
| sure. I find Slack to be a miserably unproductive experience
| that makes it impossible to keep up with a team (or teams) of
| any significant size. Slack is good for recreation, not
| productivity. If the primary purpose of your organization is to
| be a social group, then by all means use Slack. Meanwhile,
| Zulip does what I need and does it well. It scales excellently
| to large organizations discussing broad arrays of topics. It
| gets out of my way and I don't find myself thinking about it.
| In that way it's the perfect tool, because it lets me spend
| energy focusing on engaging with the conversations rather than
| wrestling with the interface. I guess I'd describe it as
| natural and unobtrusive, the sort of thing that does its job so
| well that you only notice that it's there when something goes
| wrong and you realize what you've been taking for granted.
|
| Note that my experience is almost exclusively with the web
| client. My small amount of experience with the mobile app is
| that it's much less polished.
| mavelikara wrote:
| We use Zulip at work. I have used Slack in a previous job.
| Zulip's interface looks a little clunkier than Slack, but I
| like Zulip.
| capableweb wrote:
| Clojurians-Zulip is much better interface to catch up and read
| through existing information than Clojurians-Slack ever was
| (Clojurians is basically bunch of Clojure(Script) people
| helping/getting help from each other), for whats it worth. Best
| would be if they both could be fully public, but for now the
| archives seems to do the job well at least.
| paulhart wrote:
| Much easier to read: https://blog.zulip.com/2022/02/25/zulip-
| cloud-invitation-vul...
| dang wrote:
| Changed to that from
| https://cvepremium.circl.lu/cve/CVE-2022-21706. Thanks!
| [deleted]
| tabbott wrote:
| No team is capable of writing complex software without eventually
| introducing security bugs, and we (Zulip) are no exception. I'm
| disappointed that we introduced this bug, and would like to
| apologize to our users.
|
| As a bit of commentary, my experience is that most SaaS vendors
| fix security bugs quietly. Others will do an brief internal
| investigation and only consider doing a public disclosure if they
| believe data privacy regulations require them to do so, because
| they discovered that the vulnerability had been in fact
| exploited. Usually public disclosure decisions are made by
| business executives with PR considerations in mind, so that
| shouldn't be surprising. See, for example,
| https://www.zdnet.com/article/slack-resets-passwords-for-1-o....
|
| Zulip's core values include transparency, and our policy is to
| publicly disclose all security bugs that we discover with a
| formal CVE number, and this blog post is part of that practice.
| As noted in the blog post, we did an extensive audit and emailed
| every customer who our audit logs could not prove was not
| affected with enough detail to let them audit the situation. We
| did that despite our audit finding no evidence that this
| vulnerability was ever intentionally exploited, because we think
| it's the right thing to do.
|
| Some technical detail I can add for those interested is that our
| postmortem plans following this incident include further
| investment in the internal audit log system used for the
| investigation (`RealmAuditLog`, if any readers feel like grepping
| our codebase). Improvements we made in mid-2020 were very helpful
| in investigating whether this bug had in fact been exploited, and
| we take this as a sign that we should increase investment in that
| system.
| tabbott wrote:
| As a side note, I'm surprised that this specific security
| disclosure ended up on the Hacker News homepage, while our
| security release about an RCE issue affecting self-hosted Zulip
| servers last month did not
| (https://blog.zulip.com/2022/01/25/zulip-
| server-4-9-security-...).
|
| The core bug there an insecure secret generation algorithm in
| upstream Erlang/RabbitMQ, which seems more broadly notable to
| me!
|
| Despite our spending many hours composing detailed reports and
| lobbying security teams to get this problem fixed for everyone,
| not just Zulip, `apt install rabbitmq-server` continues to make
| a Debian/Ubuntu system vulnerable to RCE, and this fact has
| apparently been public for years.
|
| We did manage to get `rabbitmq-server` fixed in Debian testing,
| and it is scheduled to be included in the next Debian stable
| release, I would really have thought an RCE issue like this
| would result in an immediate security advisory.
|
| As a final remark, if you self-host Zulip, please subscribe to
| our announcements mailing list
| (https://groups.google.com/g/zulip-announce) and make sure your
| installation is up to date, so that you benefit from the hard
| work our security team does to keep Zulip users safe.
| Ndymium wrote:
| The blog post tries to paint the RabbitMQ cookie generation
| as a security issue, but the Erlang distribution cookie is
| NOT a security feature. Indeed, the Erlang documentation
| states:
|
| > "Security" here does not mean cryptographically secure, but
| rather security against accidental misuse, such as preventing
| a node from connecting to a cluster with which it is not
| intended to communicate.
|
| The purpose of the cookie is to guard against _accidental
| misuse_, such as a user connecting to the wrong node (prod vs
| testing?). The cookie is not meant to secure a system against
| unauthorised use. Any system relying on the cookie for
| security should already be considered broken. Thus I can see
| why there has not been such hurry to "fix" the issue.
| tommoor wrote:
| For anyone interested in the bug and fix:
|
| https://github.com/zulip/zulip/commit/7e991c8c7e59291296fecd...
| ssivark wrote:
| Is this bug another manifestation of the ubiquitous "confused
| deputy" problem that results from conflating authorization and
| authentication? (Trying to relate to some recent stuff I've read
| about the importance of "object capabilities")
| smartbit wrote:
| _Zulip in DockerP
|
| Zulip has an officially supported, experimental docker image.
| Please note that Zulip's normal installer has been extremely
| reliable for years, whereas the Docker image is new and has rough
| edges, so we recommend the normal installer unless you have a
| specific reason to prefer Docker._ [0]
|
| A server application without Helm charts? Stateful apps without
| Kubernetes Operator? For someone living with k8s all day, it
| feels so strange to need to make a VM, update it, then follow the
| installation steps and then for years coming having to keep them
| updated, migrate to new hosts, etc, etc.
|
| [0]
| https://zulip.readthedocs.io/en/stable/production/deployment...
| northrup wrote:
| I couldn't agree more. I'm unsure why it appears that docker is
| a such an afterthought.
| [deleted]
___________________________________________________________________
(page generated 2022-02-26 23:00 UTC)