[HN Gopher] AuthZ: Scalable permissions system
___________________________________________________________________
AuthZ: Scalable permissions system
Author : jbredeche
Score : 86 points
Date : 2021-06-21 15:32 UTC (3 days ago)
(HTM) web link (medium.com)
(TXT) w3m dump (medium.com)
| lmeyerov wrote:
| Love seeing the visuals here :) We work on scaling graph UIs, and
| I've been noticing the auth use case more and more!
|
| The discussion on Flattening is super important, though for most
| teams, I'd also add this dances around the costs of an extra
| piece of intertwined DB infra.
|
| For our internals, we ended up having to reject OPA and friends,
| including what Zanzibar does, because we decided our customers
| wouldn't want us to spend our infra innovation tokens on an extra
| DB and figuring out how it should intertwine on the hotpath for
| queries to the main DBs. For bigger/heavier sites like Carta
| (with especially complex rules) and especially high-traffic ones
| like Google, as hinted by the percentile charts, I can see their
| decision making way more sense. For most consumer & b2b sites
| though, I'm not as convinced, and especially long-term.
|
| We ended up going with Casbin as it can do ARBAC policies +
| reuses our SQL DB. So it's more like adding a lib than infra,
| even though it does come with policy features like an analysis-
| friendly DSL. Likewise, that means we get a few ways to tune bulk
| queries: most are automatically fast by smart default roles &
| logic, some via special-cased app-level opts (so out-of-DB), and
| if we ever need to, we can always do optimized in-DB queries.
|
| The options for policy engines for b2b apps is surprisingly slim.
| The whole experience has made me think that a SQL DB with a
| strong multitenant+arbac RLS story for Django/Rails/etc. users
| and a cloud autoscale-to-zero capability will make a killing for
| b2b SaaS's. Amazon Aurora and friends are close, but feels like
| it's still another couple years in practice.
| [deleted]
| spankalee wrote:
| It's wild that an equity management startup has to do this level
| of engineering. Goes to show how much more there is to build out
| in cloud and OSS platform offerings.
| SahAssar wrote:
| - A: But how do you solve authz? - B: authz - A:
| Yeah, how do you do it? What system are you using? - B:
| authz - A: ... Why are you not answering me?
| bingo_cannon wrote:
| How does this compare to Apache Shiro?
| gneray wrote:
| I'm not an expert on Shiro, but there are a number of high
| level differences, e.g.,
|
| - Shiro covers authorization, plus authentication, session mgt,
| etc. The Carta project is authorization only
|
| - Shiro relies on external data to make authorization
| decisions. The Carta project stores that data internally
|
| - Shiro is run as part of your app. The Carta project is run as
| a separate service - Related: Shiro is
| specific to Java.
| supermatt wrote:
| Shiro is a framework. This is a service that checks (static)
| permissions.
| IndignantNerd wrote:
| When a startup I work for was borderline manipulated into
| onboarding onto the Carta platform [0], I did some brief security
| review.
|
| I noticed they're loading third party JS with at least one
| postMessage handler with a wildcard origin. They were also
| sending the current page title (which can include things like
| company or investor name) to this provider. It appeared they were
| also using a version of Knockout.js that is 2+ years old. [1]
|
| This did not inspire a lot of confidence. The linked blog post at
| least takes security seriously, but it's clear they're patching
| over a lot of legacy cruft.
|
| There's a hell of a lot of sensitive data on Carta, which I'm
| sure they're already selling to their "partners" anyway, but IMO
| it's only a matter of time before they (or one of their
| "partners") are hacked and a whole bunch of companies suffer from
| information breach.
|
| IMO the main reason they haven't been hacked yet is simply
| because you need to pay $2k just to get access to their
| questionably secure app.
|
| [0] Your lawyer tells you that you need a 409a and Carta is the
| best. But if you only want the 409A, and don't want to get growth
| hacked onto Carta's platform, you're out of luck. They won't do
| the 409A for you unless you complete the full onboarding process,
| which includes appointing Carta as your company's transfer agent,
| and uploading the email addresses of your investors and allowing
| Carta to spam them with tasks like re-signing legal documents
| that you issued off-platform.
|
| [1] This is not a technical analysis and I'm just going by what I
| remember from watching my local chrome dev tools. Don't @ me.
| steevenwee wrote:
| I sense a lot of bitterness from this post. I don't think you
| have a full context on history of 409a valuations industry.
|
| Considering the fact that 409a valuations pre carta would cost
| $10k, $2k doesnt sound like a such an unfair deal. You could
| always go the legacy lawyer companies route and pay way more
| for a single 409a valuation.
|
| As for onboarding process - you do need a hell of a lot
| information to conclude said valuation.
| [deleted]
| tristanz wrote:
| Does anybody have any paid or open source solutions that they
| actually like? All the open sources solutions I've tried don't
| actually work for real world use cases with highly dynamic
| policies, users, groups, and objects (e.g. Google Docs). Often
| the issue is the inability to easily push down filters into the
| database or search index.
|
| Aserto looks very promising (https://www.aserto.com/), but it
| still in private beta.
|
| It's crazy this still is part of the stack where there are no
| great solutions.
| jbotz wrote:
| There is Ory Keto (https://www.ory.sh/keto).
| parkerhiggins wrote:
| I was surprised not to see a comparable evaluation to Ory
| Keto at all in this article. Ory Keto continued to evolve out
| of the shortcomings of OPA
| https://github.com/ory/keto/issues/47 .
|
| It's also based on Zanzibar, and it's used in production
| applications today. Cloud version coming soon (using
| cockroachdb) or use the open source version (postgres).
| denysvitali wrote:
| Ory Keto
| lmeyerov wrote:
| casbin, see above. less third party dependency risk heartaches
| and infrastructure maintenance barnacles.
|
| agreed that it's wild that this isn't solved for b2b apps in
| 2021. the 80's were when most fundamental innovation happened
| here, then fundamentally more flexibility in the xacml days
| (~arbac), yet here we are...
| samjs wrote:
| > It's crazy this still is part of the stack where there are no
| great solutions.
|
| Seems like a few others have come to the same conclusion :)
|
| We're working on this at Oso (https://osohq.com) - I'm the CTO.
| Oso is an open source framework for authorization. Policies can
| reference application directly, so any authorization decisions
| can be made dynamically based on the data. And your data
| doesn't need to leave your application.
|
| > Often the issue is the inability to easily push down filters
| into the database or search index.
|
| We actually support pushing down filters into the database
| directly through a feature we call... data filtering. We
| support Django and SQLAlchemy today
| (https://docs.osohq.com/python/guides/data_access.html) but
| will be making this available for non-ORM/non-Python users in
| the next few months.
| Bombthecat wrote:
| Doesn't opa do something like this?
| adamfeldman wrote:
| The article discusses Concord, a visualization tool they built.
|
| From the screenshot [1], it looks like it's hosted within some
| larger dashboard system - does anyone know if that's OSS or
| something custom?
|
| [1]: https://miro.medium.com/max/700/0*JATQRwLPT_MgrMHT
|
| EDIT: Ah, looks like Backstage - https://backstage.io/demos
| [deleted]
| buro9 wrote:
| > But we had a problem: There weren't any open source
| implementations available. We decided to build our own system,
| adding several of our own modifications to make it an even better
| fit for Carta
|
| Was this problem solved? Is AuthZ open source? (I couldn't see a
| link to a repo anywhere - but am reading on mobile and maybe I
| missed it)
| netvarun wrote:
| There is also Keto - an open source project based on the Zanzibar
| model [https://github.com/ory/keto]
| denysvitali wrote:
| This!
|
| Also, the full Ory suite is very well developed and worth
| looking into. Fully open source!
| twiddling wrote:
| Having XACML flashbacks
| supermatt wrote:
| What aspect of this makes it "highly scalable"?
|
| It seems that the scalability limitation is the same as that of
| every graph - the efficiency of scaling a graph beyond a single
| node.
|
| Not bashing what you are saying, just want to understand what
| makes it "highly scalable", as it isn't clear from the article.
|
| Also, I was wondering: Are ALL permissions stored as an
| Object::relation pair? i.e. do you need to register permissions
| for all new entities relationships, or do you have some way of
| storing more dynamic permissions?
| jzelinskie wrote:
| Full disclosure: I'm a founder of Authzed (YC W21)[0], which is
| a productized Zanzibar system.
|
| > just want to understand what makes it "highly scalable", as
| it isn't clear from the article.
|
| While this post might not cover in depth why their system is
| scalable, the Zanzibar paper[1] and this post[2] about Zanzibar
| covers why similar systems are scalable. A oversimplification
| would be to say that finding a path between two nodes in a
| graph can be recursively broken down and performed in parallel
| -- the Authzed implementation looks roughly similar to a map
| reduce.
|
| >Are ALL permissions stored as an Object::relation pair? i.e.
| do you need to register permissions for all new entities
| relationships, or do you have some way of storing more dynamic
| permissions?
|
| Zanzibar implementations do have graph schemas (called
| Namespace Configs in the paper) which offer set operations for
| computing dynamic relationships. When you create a new user or
| object in your app, you typically also write a few
| relationships into Zanzibar and the rest of their permissions
| are implicitly granted through the computed definitions in your
| schema.
|
| If you're interested in learning more about Zanzibar, we're
| giving a PapersWeLove talk that will be streamed on Twitch next
| week: https://www.meetup.com/papers-we-love/events/278148236/
|
| [0]: https://authzed.com
|
| [1]: https://research.google/pubs/pub48190/
|
| [2]: https://authzed.com/blog/what-is-zanzibar/
| AceJohnny2 wrote:
| Thanks for excellent explanation! I'm disappointed your work
| on Authzed hasn't hit HN front page before, not for lack of
| trying on your part!
| jzelinskie wrote:
| We haven't done an official "Launch HN" yet. For now, we're
| heads-down with our early adopters fleshing out developer
| UX and a friendlier API. We started by implementing
| Zanzibar as faithfully as possible, but that doesn't make
| this technology widely available to developers of all walks
| of life.
| AceJohnny2 wrote:
| Nevertheless, your separate posting [1] of your "What is
| Zanzibar" article deserved more attention, and IMHO is
| useful to drum up attention to your problem/solution
| space.
|
| [1] https://news.ycombinator.com/item?id=27143342
| victor106 wrote:
| authzed looks cool. Any plans to release a Java library?
| jzelinskie wrote:
| Our API is gRPC, so we can easily generate some Java
| bindings. For a more ergonomic client, we'd have to partner
| to flesh something out. Feel free to put some time on our
| calendar, everyone on the call will be engineer that's deep
| in the weeds: https://calendly.com/d/mzs5-cxvx/authzed-
| introduction
| samjs wrote:
| > What aspect of this makes it "highly scalable"?
|
| The idea is to put all data in one place, and then aggressively
| optimise for answering queries about that data. For
| authorization you're pretty much always asking specific
| reachability questions. That's where the design of the index
| comes into it.
|
| > Also, I was wondering: Are ALL permissions stored as an
| Object::relation pair? i.e. do you need to register permissions
| for all new entities relationships, or do you have some way of
| storing more dynamic permissions?
|
| In Zanzibar, there is a configuration format for computing
| relationships dynamically. E.g. "anyone who is an editor of a
| document is a viewer of a document". I'm not sure if they
| implemented something like that.
|
| If you're interested, I wrote a guide on relationship-based
| access control that includes a section on how Zanzibar(-like)
| systems fit in: https://www.osohq.com/academy/authorization-
| academy-chapter-...
| philsnow wrote:
| I hope "AuthZ" is just a temporary codename, that will be changed
| if/when they open source their work. authz/authn are very generic
| terms that already mean something and it's distasteful when
| companies squat on such terms. Imagine if a company came around
| and created a networking-related project and called it "TcP".
|
| > During our discovery process, we evaluated both open source and
| vendor products.
|
| > But we had a problem: There weren't any open source
| implementations available. We decided to build our own system,
| adding several of our own modifications to make it an even better
| fit for Carta.
|
| There's no mention or implication that they'll release it as open
| source, this seems to be an engineering blog post (for hiring
| lead gen or building the engineering brand of the company or
| whatever).
| simonw wrote:
| I was pleased to see a section in here on what they call
| "flattening" - the problem of "efficiently provide a list of
| resources my user has access to".
|
| I've made this mistake more than once when designing an
| authorization system. Sure, you can build a fast mechanism for
| answering the question "does actor A have access to object B" -
| but can you efficiently answer that for 10,000 objects at once?
|
| Say for example you're building a search engine, and you want to
| search across all public data but ALSO all of the data your user
| has access to that's private. You probably can't run the search,
| get back 10,000 results, then apply your "can X access Y" logic
| to all 10,000 of them.
|
| (I'd love to know how GitHub code search implements this - it
| shows me results from my private repositories mixed in with
| results from their public index).
| Diggsey wrote:
| The permissions system I designed at work solves the search
| problem, although it is not as powerful as what's described in
| the article.
|
| We have three main concepts in the sytem:
|
| - Capabilities (eg. "read profile")
|
| - Selectors (eg. "profiles belonging to customer X)
|
| - Groups (eg. "profile")
|
| All queries are of the form "Do I have capability X on resource
| Y" (where resource Y belongs to a particular group). A
| "permission set" is a mapping from a capability to a set of
| selectors.
|
| Selectors can be composed (eg. you could OR two selectors
| together) but are predefined. Since they are predefined, our
| search implementation can understand a subset of the selectors,
| and translate them into an ElasticSearch-compatible query.
|
| We then filter the results by evaluating the permissions
| normally: this catches any cases where there was no way to
| construct a query, or if the constructed query could not be
| completely precise. If this occurs, then we show a message on
| the page like "some profiles were hidden because you do not
| have permission to view them" and then we might show 19
| profiles on a page instead of the full 20, so that our
| pagination can still work efficiently.
| jzelinskie wrote:
| "Flattening", the equivalent of denormalizing a graph, is one
| way to implement performant ACL Filtering[0]. This is the same
| strategy used in the Zanzibar paper to gain the desired
| performance out of their Leopard index system. I tend to
| consider this largely a performance optimization strategy and
| not a necessarily the "solution" to this problem. We actually
| just this week wrote about how Authzed[1] actually reverse
| walks the graph in order to implement this API.
|
| Full disclosure: I'm a founder of Authzed (YC W21), which is a
| productized Zanzibar system.
|
| [0]: https://authzed.com/blog/six-month-profile-page/
|
| [1]: https://authzed.com/blog/acl-filtering-in-authzed/
| debarshri wrote:
| Maybe this is a naive question, at what point does an
| organisation decide to build a framework or a product internally
| than buying? Sure there was no opensource solution available but
| why not buy a product from other security companies. Just
| wondering.
| gneray wrote:
| It's a fair question, but in fairness to Carta, until recently,
| there have been ~zero commercial or open source solutions to
| building application authorization (beyond limited helper
| libraries or things like Active Directory, which solve only a
| portion of the problem). Developers have typically figured out
| the models on their own and implemented them in application
| code with IF statements, etc. and/or by building separate
| authorization services. Other publicly documented examples
| include teams at Slack (https://slack.engineering/role-
| management-at-slack/) and Gusto
| (https://engineering.gusto.com/layering-authorization-
| into-a-...)
|
| OPA is one solution that's emerged in recent years. The author
| describes his team's point of view after evaluating it for
| their purposes.
|
| Oso (https://www.osohq.com/) is a framework for application
| authorization. Disclosure: I'm a cofounder. Based on the post,
| the Carta team started their work in 2019, but we only open
| sourced Oso mid-2020, so they wouldn't have known about it.
|
| And in the last few months, a few other companies have
| announced offerings to address this problem too.
|
| So it's understandable why Carta chose to build. Now/soon other
| teams like them should have more options to choose from when
| they encounter similar challenges too.
| spockz wrote:
| Integration into your own systems for example. Or specific
| requirements for which it is not clear they can be met by the
| product. Or unawareness. Or a combination of the above.
| simonw wrote:
| Build v.s. buy is genuinely one of the harder problems in
| software engineering strategy.
|
| You'll often hear people say "you should never, ever build
| something if you can buy a solution instead" (where buy also
| includes using an open source package) - but there are actually
| a bunch of other things to consider:
|
| - Is the solution you're considering well maintained and likely
| to keep working? Open source projects sometimes stop being
| updated. Vendors go out of business.
|
| - What's your escape strategy if the vendor DOES go out of
| business, or massively increases their pricing?
|
| - Integrating with a vendor / package is often a non-trivial
| amount of development work. Have you considered this relative
| to building your own, custom solution?
|
| - Do you have specific requirements that go beyond what's
| available off-the-shelf?
|
| It's absolutely true that your engineering efforts should be
| spent on the things that make your organization unique: time
| spent on your customer's problems is more valuable than time
| spent reinventing a wheel.
|
| But sometimes, the available wheels really aren't fit for your
| purposes.
|
| I've built when I should have bought, and I've bought when I
| should have built. Getting it right is hard! Often it doesn't
| become clear if you made the right decision until several years
| after you've made it.
| ex_amazon_sde wrote:
| Ex-Amazon here. One thing than Amazon really understands is
| that if you build you also create the opportunity to develop
| skill and knowledge.
|
| Also you avoid vendor lock-in. Even Open source projects
| cause lock-in especially if they are overcomplex and fast
| moving.
___________________________________________________________________
(page generated 2021-06-24 23:01 UTC)