[HN Gopher] I was wrong - CRDTs are the future (2020)
___________________________________________________________________
I was wrong - CRDTs are the future (2020)
Author : pcr910303
Score : 131 points
Date : 2022-04-16 05:52 UTC (17 hours ago)
(HTM) web link (josephg.com)
(TXT) w3m dump (josephg.com)
| wngr wrote:
| (2020)
| dang wrote:
| Added. Thanks!
| sedatk wrote:
| This is a false dichotomy. OP can be wrong with their new
| prediction too.
| JadeNB wrote:
| > This is a false dichotomy. OP can be wrong with their new
| prediction too.
|
| It's not "I was wrong, therefore CRDTs are the future" but
| "CRDTs are the future, therefore I was wrong." As you say, this
| new prediction, too, can be wrong, but I don't see where any
| sort of false dichotomy comes into it.
| sedatk wrote:
| You're right that OP doesn't directly make that correlation,
| but I deduced it from his insistence on that one thing can
| either be "the future" or not. That kind of thinking usually
| stems from a dualistic perspective. I wanted to emphasize
| that. I find such extreme stances generally unhelpful and
| likely to turn out wrong again.
| mlyle wrote:
| We really have three OK-ish options for rich collaboration:
|
| * CRDT's
|
| * OT
|
| * _Maybe_ differential synchronization
|
| If a key OT partisan now backs CRDT, that's a significant
| development. It means CRDT's arguments have become
| compelling enough to convince people on the other side.
|
| Now, maybe something new and wonderful will pop up and
| supplant all of the above...
| williamstein wrote:
| In CoCalc, I use a fourth approach that I think isn't in
| your list.
| https://blog.cocalc.com/2018/10/11/collaborative-
| editing.htm... I don't claim it is better than any other
| approach, except it is easy to understand. Also I'm not
| sure that it isn't technically an extreme special case of
| OT.
| mlyle wrote:
| > Also I'm not sure that it isn't technically an extreme
| special case of OT.
|
| It sure looks like the trivial case of OT to me.
| smasher164 wrote:
| I've been delving into CRDTs lately, and one thing I've been
| wondering is if there's an implementation that is purely SQL-
| based. All operations and merges would happen through database
| queries. This may not be suitable for real-time use, but I've
| noticed that most CRDT implementations expect you to have the
| entire data structure in memory, forcing you to be clever with
| the way you model your data and save to disk.
| staticassertion wrote:
| I don't really understand why every single article on CRDTs is
| about collaborative text editing. Is no one else bothering to
| build systems with CRDTs outside of this space?
|
| Not to detract from the article, which I found interesting for a
| number of reasons - both in terms of the technology and future of
| that technology as well as the personal realization of "the thing
| I'm building won't be necessary one day".
| kareemsabri wrote:
| Figma uses them too.
|
| https://www.figma.com/blog/how-figmas-multiplayer-technology...
| ItsMonkk wrote:
| I'm extremely excited about CRDT Matrix[0], especially in the
| context of decentralized chains[1]. I think if we combine this
| message stream concept, with the entirely separate concept of
| torrent databases[2], you get the ability to get fully
| decentralized, self-hosted, web applications - all with no
| concept of some crypto-token, as it's all incredibly efficient.
|
| It's going to take some time for this to all come together, but
| it's exciting.
|
| [0]: https://news.ycombinator.com/item?id=29978659
|
| [1]: https://news.ycombinator.com/item?id=30560573
|
| [2]: https://news.ycombinator.com/item?id=30605252
| taeric wrote:
| Agreed. It doesn't help that the idea of concurrent editing of
| a single document is.... Not really that useful. Makes fun
| performances/demos. But that isn't really where the friction is
| in how folks collaborate.
| jitl wrote:
| Concurrent editing of documents is a $100B+ market
| taeric wrote:
| So is videoconferencing. And somehow, all offerings are
| some version of garbage. :(
| dustingetz wrote:
| CRDTs match multiplayer collaboration to edit a target free
| data structure, but they don't match multiplayer competition to
| put a database through a series of transaction states. Note the
| word "transaction" is almost the opposite of "conflict-free" in
| that the whole point of a transaction is to reject certain
| actions as resulting in an invalid state.
|
| Most of the applications I worked on in the last decade (web
| saas crud stuff) are dominantly transactional with maybe a thin
| slice of collaborative functionality. I suppose the next decade
| could look different but I don't see why that would be the
| case.
| xtracto wrote:
| I have a distributed/multiplayer domino and a scrabble (tm)
| clone game in javascript/redis that I'll change to CRDT. At the
| end of the day, the concept of updating a board with some tiles
| lends itself pretty well for CRDTs imho.
| samwillis wrote:
| There are people looking at how to use them outside of rich
| text editing, it's just that the technology was
| developed/initially applied to text editing.
|
| I'm more excited about the use of CRDTs outside of text
| editing, there are so many use cases. However there are still
| some problems to solve with the existing general purpose
| toolkits such as Yjs and Automerge. Both give you simple data
| structures and some more complex structures specifically for
| rich text. For a true conflict free system you need the CRDTs
| to match your data model exactly so you can never have a
| invalid state after a merge. This is almost never the case with
| existing toolkits.
|
| Take Yjs and it's ProseMirror bindings, Yjs can represent any
| internal state of a ProseMirror document and merge them.
| However it does not have an understanding of the specific
| ProseMirror schema being used, this could limit the valid
| states possible (a max length on a heading for example). When
| Yjs merges multiple edits you could end up with an invalid
| document for your schema, Yjs loads this into ProseMirror which
| then throws away the invalid data (with the heading example it
| just deletes the excess text, it may be better to dump it into
| the next line but should be configurable).
|
| These sorts of problems are "ok" in a text editor but would be
| much harder to cope with in a database or a 3d cad tool.
|
| Ultimately the general purpose CRDT toolkits need to have more
| internal types with a wider ability to capture intent. I also
| think they need to have the ability to understand the schema of
| the data structure they are tracking and how to handle more
| complex merges.
|
| The alternative to doing that with a general purpose toolkit is
| to build your own CRDTs from scratch for your data model. I
| don't think many people would go that route though.
|
| Once those sorts of problems have been solved I think we will
| see their use explode into new areas.
|
| It's still early days with them but exciting time ahead.
| lijogdfljk wrote:
| I am. But i know very little about CRDTs lol, so we'll see how
| that goes. I'm interested in converting some immutable, local-
| first data warehouse tooling i enjoy to a CRDT version. Prior
| it was more.. Git-like. Basically just Git with data structures
| inspired-massively from Noms[1].
|
| The thing i've found most interesting is it appears[2] that
| CRDT backends need to expose CRDT flavored types to users.
| Which is to say how i'm writing this combines the notion of a
| type, say `[i32]` with how you want the merges to work. CRDT
| works great but based on my amateur-hour researching on the
| subject i don't feel you can write a single CRDT merge strategy
| for a single data type ala `[i32]` and have it be always
| correct. Applications need to indicate enough context on what
| makes sense for a given data type.
|
| So yea, i agree with you. I'm interested in making a database-
| like thing, backed by CRDTs, but i also have seen very few
| general purpose implementations with CRDTs. It feels like i'm
| breaking "new ground", while having no idea what i'm doing and
| having no intention of being an actual researcher here. I'm
| just making apps i enjoy heh.
|
| [1]: https://github.com/attic-labs/noms [2]: appears, because i
| could easily be wrong
| derefr wrote:
| Collaborative text editing is an easy-to-understand example of
| a Hard Problem that can't be solved with any simpler
| abstraction. You can try to throw an MVCC database at it, a BFT
| consensus algorithm at it, whatever, but without having those
| substrates implement a CRDT (or its cousin, Operational
| Transformations), you won't get the results you want.
|
| More generally, CRDTs are useful when syncing any kind of
| change-events between peers (with no central ability to pre-
| linearize those events), where the ordering of change-events
| doesn't matter -- i.e. where the operations described by the
| events are "commutative", such that you'll get the same results
| out if you apply events {1,2,3} or {3,2,1} or any other order.
| As long as all peers see all events, all peers arrive at the
| same eventually-consistent position in the "lattice" of
| possible states. In other words, CRDTs + gossip protocols = the
| ability for peers to just process events as they hear about
| them, without a worry about receiving events from peers "out of
| order"; rather than needing to wait around for a some elected
| peer to forcibly linearize the events (i.e. without the need to
| introduce a serial write bottleneck.)
|
| A good example of the distributed-systems-backend kind of use
| of CRDTs, is in the Elixir web framework Phoenix, which does a
| sort of mesh delivery of web-socket messages between a cluster
| of backends. CRDTs are used to sync presence information (=
| which clients are connected to which backend) between the
| backends.
| staticassertion wrote:
| It just seems like an extremely niche, difficult problem,
| whereas CRDTs are a very general tool.
|
| edit: Oh you edited your post there's way more now, but I
| can't read it all now x_x but fwiw when I said "anyone else"
| I meant that I use crdts.
| kabes wrote:
| On the other hand: A lot of practical applications are fine
| with last write
| jbotz wrote:
| Previous discussion:
| https://news.ycombinator.com/item?id=24617542
___________________________________________________________________
(page generated 2022-04-16 23:01 UTC)