[HN Gopher] Lies I was told about collaborative editing, Part 2:...
___________________________________________________________________
Lies I was told about collaborative editing, Part 2: Why we don't
use Yjs
Author : antics
Score : 179 points
Date : 2026-03-13 01:38 UTC (3 days ago)
(HTM) web link (www.moment.dev)
(TXT) w3m dump (www.moment.dev)
| presspot wrote:
| Replacing CRDT with 40 lines of code. Amazing.
| GermanJablo wrote:
| I remember reading Part 1 back in the day, and this is also an
| excellent article.
|
| I've spent 3+ years fighting the same problems while building
| DocNode and DocSync, two libraries that do exactly what you
| describe.
|
| DocSync is a client-server library that synchronizes documents of
| any type (Yjs, Loro, Automerge, DocNode) while guaranteeing that
| all clients apply operations in the same order. It's a lot more
| than 40 lines because it handles many things beyond what's
| described here. For example:
|
| It's local-first, which means you have to handle race conditions.
|
| Multi-tab synchronization works via BroadcastChannel even
| offline, which is another source of race conditions that needs to
| be controlled.
|
| DocNode is an alternative to Yjs, but with all the simplicity
| that comes from assuming a central server. No tombstones, no
| metadata, no vector clock diffing, supports move operations, etc.
|
| I think you might find them interesting. Take a look at
| https://docukit.dev and let me know what you think.
| freekh wrote:
| Cool! We also build client-server sync for our local-first CMS:
| https://github.com/valbuild/val Just as your docsync, it has to
| both guarantee order and sync to multiple types of servers
| (your own computer for local dev, cloud service in prod). Base
| format is rfc 6902 json patches. Read the spec sheet and it is
| very similar :)
| huksley wrote:
| Looks really cool, I would love to use it in my DollarDeploy
| project. Documentation could be a bit better still, it is not
| clear, are content is pure markdown or it is typescript
| files? Which GitHub repo it synchronizes to? I prefer
| monorepo approach.
| freekh wrote:
| Awesome feedback! Will update the docs! The content is TS
| files. You can chose which repo GitHub you want to
| synchronize to - monorepo also works!
| freekh wrote:
| Should add: you can read more docs here:
| https://val.build/docs/create
| antics wrote:
| Hello again German! Since the product we make is, basically, a
| local-first markdown file editor, I would humbly suggest that
| the less-well-known algorithm we recommend is thus also local-
| first. But, I fully believe that you do a ton of stuff that we
| don't, and if we had known about it at the time, we very
| definitely would have taken a close look! We did not set out to
| do this ourselves, it just kind of ended up that way.
| eviks wrote:
| Tiny fail at undo: insert 1 before E, Ctlr+Z, move left/right:
| left editor moves around E, right editor moves around the
| nonexistent 1
|
| And for real "action" there should be a delay/pause button to
| simulate conflicts like the ones described in the blog
| GermanJablo wrote:
| Yes, the undo issue is a known bug in the website demo
| because it's messing with Lexical's undo functionality. It's
| not actually a DocNode bug. I'll fix it soon.
|
| The feedback about the delay/pause button is also good,
| thanks!
| samlinnfer wrote:
| Just use OT like normal people, it's been proven to work. No
| tombstones, no infinite storage requirements or forced
| "compaction", fairly easy to debug, algorithm is moderate to
| complex but there are reference open source implementations to
| cross check against. You need a server for OT but you're always
| going to have a server anyway, one extra websocket won't hurt
| you. We regularly have 30-50k websockets connected at a time.
| CRDTs are a meme and are not for serious applications.
| chrisweekly wrote:
| "CRDTs are a meme and are not for serious applications."
|
| That is one hot take!
| argee wrote:
| Let's balance the discussion a bit.
|
| https://josephg.com/blog/crdts-are-the-future/
| antics wrote:
| Author here, I did not specifically mention OT in the article,
| since our main focus was to help people understand the
| downsides of the currently-most-popular system, which is built
| on CRDTs.
|
| BUT, since you mention it, I'll say a bit here. It sounds like
| you have your own experience, and we'd love to hear about that.
| But _OUR_ experience was: (1) we found (contrary to popular
| belief) that OT actually does _not_ require a centralized
| server, (2) we found it to be _harder_ to implement OT exactly
| right vs CRDTs, and (3) we found many (though not all) of the
| problems that CRDTs have, are also problems in practice for OT
| --although in fairness to OT, we think the problems CRDTs have
| in general are vastly worse to the end-user experience.
|
| If there's interest I'm happy to write a similar article
| entirely dedicated to OT. But, for (3), as intuition, we found
| a lot of the problems that both CRDTs and OT have seem to arise
| from a fundamental impedance mismatch between the in-memory
| representation of the state of a modern editor, and the
| representation that is actually synchronized. That is, when you
| apply an op (CRDT) or a transform (OT), you have to transform
| the change into a (to use ProseMirror as an example) valid
| `Transaction` on an `EditorState`. This is not always easy in
| either case, and to do it right you might have to think very
| hard about things like "how to preserve position mappings," and
| other parts of editor state that are crucial to (say) plugins
| that manage locations of comment marks or presence cursors.
|
| With all of that said, OT is definitely much closer to what
| modern editors need, in my opinion at least. The less-well-
| known algorithm we ended up recommending here (which I will
| call "Marjin Collab", after its author) is essentially a very
| lightweight OT, without the "transformation" step.
| samlinnfer wrote:
| Having a central server is not necessary, but we have one
| anyway and we use it, especially if you have a permissions
| system. It lets us use the "Google wave" algorithm which
| vastly simplifies things.
|
| https://svn.apache.org/repos/asf/incubator/wave/whitepapers/.
| ..
|
| > This is not always easy in either case, and to do it right
| you might have to think very hard about things like "how to
| preserve position mappings," and other parts of editor state
| that are crucial to (say) plugins that manage locations of
| comment marks or presence cursors.
|
| Maintaining text editor state is normal. Yes you do need to
| convert the OT messages into whatever diff format your editor
| requires (and back), but that's the standard glue code.
|
| The nice thing about OT is that you can just feed the
| positions of marks into the OT algorithm to get the new
| positional value. Worst case, you just have the server send
| the server side position when sending the OT event and the
| client just displays the server side position.
| DonHopkins wrote:
| Josh eloquently explains how Google Wave's DACP
| (Distributed Application Canceling Protocol) works:
|
| https://www.youtube.com/watch?v=4Z4RKRLaSug
| ianhorn wrote:
| I always mentally slotted prosemirror-collab/your recommended
| solution in the OT category. What's the difference between
| the "rebase" step and the "transformation" step you're saying
| it doesn't need?
| antics wrote:
| Great question. Matt has a comment about this here, and he
| has an actual PhD on the subject! So rather than doing a
| worse job explaining I will leave it to him to explain:
| https://news.ycombinator.com/user?id=mweidner
| gritzko wrote:
| One way to minimize impedance mismatch is to work with DOM-
| like or JSON-like structures mostly immune to transient bugs,
| which I am doing currently in the librdx project. It has
| full-CRDT RDX format[1] and essentially-JSON BASON[2] format.
| It does not solve all the problems, more like the set of
| problems is different. On the good side, it is really
| difficult to break. On the bad side, it lacks some of the
| rigor (esp BASON) that mature CRDT models have. But, those
| models are way more complex and, most likely, will have
| mismatching bugs in different implementations. No free lunch.
|
| [1]: https://github.com/gritzko/librdx/tree/master/rdx [2]:
| https://github.com/gritzko/librdx/tree/master/json
| MrJohz wrote:
| Are there any major libraries for OT? I've been looking into
| this recently for a project at work, and OT would be completely
| sufficient for our use case, and does look simpler overall, but
| from what I could tell, we'd need to write a lot of stuff
| ourselves. The only vaguely active-looking project in JS at
| least seems to be DocNode (https://www.docukit.dev/docnode),
| and that looks very cool but also very early days.
| samlinnfer wrote:
| https://github.com/josephg/sharejs
|
| https://github.com/FirebaseExtended/firepad
|
| https://github.com/Operational-Transformation/ot.js
|
| https://github.com/ottypes/docs
|
| Really nice demo: https://operational-
| transformation.github.io
| antics wrote:
| Author here. I think it depends what you're doing! OT is a
| true distributed systems algorithm and to my knowledge there
| are no projects that implement true, distributed OT with
| strong support for modern rich text editor SDKs like
| ProseMirror. ShareJS, for example, is abandoned, and predates
| most modern editors.
|
| If you are using a centralized server and ProseMirror, there
| are several OT and pseudo-OT implementations. Most popularly,
| there is prosemirror-collab[4], which is basically "OT
| without the stuff you don't need with an authoritative source
| for documents." Practically speaking that means "OT without
| T", but because it does not transform the ops to be order-
| independent, it has an extra step on conflict where the user
| has to rebase changes and re-submit. This is can cause minor
| edit starvation of less-connected clients. prosemirror-
| collab-commit[5] fixes this by performing the rebasing on the
| server... so it's still "OT without the T", but also with an
| authoritative conflict resolution pseudo-T at the end. I
| personally recommend prosemirror-collab-commit, it's what we
| use, and it's extremely fast and predictable.
|
| If you just want something pedogocically helpful, the blessed
| upstream collaborative editing solution for CodeMirror is OT.
| See author's blog post[1], the @codemirror/collab package[2],
| and the live demo[3]. In general this implementation is quite
| good and worth reading if you are interested in this kind of
| thing. ShareJS and OTTypes are both very readable and very
| good, although we found them very challenging to adopt in a
| real-world ProseMirror-based editor.
|
| [1]: https://marijnhaverbeke.nl/blog/collaborative-editing-
| cm.htm...
|
| [2]: https://codemirror.net/docs/ref/#collab
|
| [3]: https://codemirror.net/examples/collab/
|
| [4]: https://github.com/ProseMirror/prosemirror-collab
|
| [5]: https://github.com/stepwisehq/prosemirror-collab-commit
| MrJohz wrote:
| In our case, we're not using a text editor, but instead
| building a spreadsheet, so a lot of these collab-built-
| into-an-editor are, like you say, pedagogically useful but
| less helpful as direct building blocks that we can just
| pull in and use. But the advice is very useful, thank you!
| nhatcher wrote:
| Interesting! I am building a spreadsheet and the next few
| months will be building the collaborative side of it. I
| think many of the things that work for text don't
| necessarily translate for spreadsheets.
| ghProTip wrote:
| Cheers for plugging prosemirror-collab-commit! Nice to see
| it's getting used more.
| gritzko wrote:
| When I was starting my research into collaborative editing
| as a PhD student 20+ years ago, rebase-and-resubmit was
| well known. It was used in one Microsoft team collab
| product (I forgot the name). It is 100% legit algo except
| intermittently-connected clients may face challenges (screw
| them then).
|
| Unless you have to support some complicated scenarios, it
| will work. I believe Google Docs initially used something
| of the sort (diff-match-patch based). It annoyed users with
| alerts "lets rebase your changes", esp on bad WiFi. So they
| borrowed proper OT from Google Wave and lived happily since
| (not really).
|
| One way to think about it: how many users will your product
| have and how strange your data races / corner cases can
| get. At Google's scale, 0.1% users complaining is a huge
| shit storm. For others, that is one crazy guy in the
| channel, no biggie. It all depends.
|
| TLDR: people invented OT/CRDT for a reason.
| antics wrote:
| First of all, thanks for chiming in! I wish someone would
| collect stuff like this and write it down in some sort of
| "oral history of collab editing."
|
| Second of all, I actually think we're more aligned than
| it seems here. What we're really advocating for is being
| super clear about what your end-user goals are, and
| deriving technology decisions from them, instead of the
| reverse. Our goals for this technology are (1) users
| should be able to predict what happens to their data, (2)
| the editor always run at 60fps, and (3) we are reasonably
| tolerant of transient periods of disconnection (up to,
| say, 30s-1m).
|
| Because of (1) in particular, a lot of our evaluation was
| focused on understanding which situations users would be
| unable to predict what was going to happen to their data.
| This is only our own experience, but what we found (and
| the impetus for part 1 of this series) is that almost
| 100% of the time, when there is a direct editing
| conflict, users interpret the results of the dominant
| CRDT and OT implementations as silently corrupting their
| data. So, the name of the game is to decrease the
| likelihood of direct editing conflicts, _e.g._ presence
| carets in the live-collab case. In particular, we did not
| notice a meaningful difference between how users view
| reconciliations of OT and CRDT implementations.
|
| Since our users could not tell the difference, and in
| fact viewed _all_ options as equally bad ( "disastrous"
| as one user said), this freed us up to consider a much
| broader class of algorithms, including prosemirror-collab
| and prosemirror-collab-commit.
|
| I know there is a rich history of why OT is OT, but our
| final determination was made pretty simple by the fact
| that the source of the majority of race conditions in our
| view come from the difficulty of integrating CRDTs and OT
| directly into _modern_ editing stacks, like ProseMirror.
| As far as I am aware, prosemirror-collab-commit behaves
| as good or better on every dimension than, say, an
| OTTypes implementation would... and mostly that is
| because it is native to the expressive `Transaction`
| model of the modern editor. If we had to do interop I
| think we would have shipped something noticably worse,
| and much slower.
|
| If you have a different experience I would love to hear
| about it, as we are perennially in the market for new
| ideas here.
| GermanJablo wrote:
| Author of DocNode here. Yes, it's still early days. But it's
| a very robust library that I don't expect will go through
| many breaking changes. It has been developed privately for
| over 2 years and has 100% test coverage. Additionally, each
| test uses a wrapper to validate things like operation
| reversibility, consistency across different replicas, etc.
|
| DocSync, which is the sync engine mainly designed with
| DocNode in mind, I would say is a bit less mature.
|
| I'd love it if you could take a look and see if there's
| anything that doesn't convince you. I'll be happy to answer
| any questions.
| MrJohz wrote:
| I've looked through the site, and right now it's probably
| the thing I'd try out first, but my main concerns are the
| missing documentation, particular the more cookbook-y kinds
| of documentation -- how you might achieve such-and-such
| effect, etc. For example, the sync example is very terse,
| although I can understand why you'd like to encourage
| people to use the more robust, paid-for solution! Also just
| general advice on how to use DocNode effectively from your
| experience would be useful, things like schema design or
| notes about how each operation works and when to prefer one
| kind of operation or structure over another.
|
| All that said, I feel like the documentation has improved
| since the last time I looked, and I suspect a lot of the
| finer details come with community and experience.
| GermanJablo wrote:
| Thanks! I've recently made some improvements to the
| documentation. I agree the synchronization section could
| be improved more. I'll keep your feedback in mind. If
| you'd like to try the library, feel free to ask me
| anything on Discord and I'll help you.
| r0b05 wrote:
| What is OT?
| hsx wrote:
| Operational Transformation:
| https://en.wikipedia.org/wiki/Operational_transformation
| fritzo wrote:
| Agreed. In my limited experience, conflict resolution rules are
| very domain specific, whereas CTDTs encourage a lazy attitude
| that "if it's associative and commutative it must be correct".
| leksak wrote:
| What does OT stand for so I can learn more?
| cantaloupe wrote:
| Operational Transformation
| bawolff wrote:
| Reminds me a bit of google-mobwrite. I wonder why that fell out
| of favour.
| truetraveller wrote:
| Very likely AI slop, very hard to read. Too many indications. HN
| should have another rule: explicitly mention if article was
| written (primarily) by AI.
| utopiah wrote:
| Funnily enough I had 2 HN tabs open, this one and
| https://news.ycombinator.com/item?id=47394004
| antics wrote:
| I'm the author. Literally 0% of this was written with AI. Not
| an outline, not the arguments, not a single word in any
| paragraph. We agonized over every aspect of this article: the
| wording, the structure, and in particular, about whether we
| were being fair to Yjs. We moved the second and third section
| around constantly. About a dozen people reviewed it and gave
| feedback.
|
| EDIT: I will say I'm not against AI writing tools or anything
| like that. But, for better or worse, that's just not what
| happened here.
| truetraveller wrote:
| Apologies. Was it at all edited by an AI?
| comex wrote:
| It doesn't strike me as AI. The writing is reasonably
| information-dense and specific, logically coherent, a bit
| emotional. Rarely overconfident or vague. If it is AI then
| there was a lot more human effort put into refining it than
| most AI writing I've read.
| kaiwenwang wrote:
| It appears Moment is producing "high-performance, collaborative,
| truly-offline-capable, fully-programmable document editor" -
| https://www.moment.dev/blog
|
| There seems to be a conflict of interest with describing Yjs's
| performance, which basically does the same thing along with
| Automerge.
| antics wrote:
| Author here. To be clear, we do not in ANY WAY compete with
| Yjs! We are a potential _customer_ of Yjs. This article
| explains why we _chose not to be_ a customer of Yjs, and why we
| don 't think most people building real-time collaborative text
| editors should be, either.
| crashabr wrote:
| You have an amazing tagline. This is the first time I read a
| tagline and thought: this is exactly what I was looking for.
|
| But the product seems much more narrow than an actual tool
| run the whole business in markdown. I was hoping to see
| Logseq on steroids, and it feels like a tool builder
| primarily. I love the tool building aspect, but the
| fundamentals of simply organizing docs (docs, presentations,
| assets etc, the basics of a business) are either not part of
| the core offering or not presented well at all.
|
| I love the idea of building custom tools on top of MD and
| it's part of my wishlist, but I feel little deceived by your
| tagline so I wanted to share that :)
| antics wrote:
| This is great feedback, thank you. I will say that IS our
| goal... but we only really launched last week and are still
| figuring out what resonates with people and what they
| really want! It sounds like you're saying that the
| organization aspects are not there, which is very helpful
| to know... I am not quite sure I understand if you _also_
| think the toolbuilding is lacking?
|
| If you are open to it, I'd love the opportunity to hear
| more. Here or email (alex@moment.dev) or our Discord
| (bottom right of our website) or Twitter/X... or whatever
| you prefer.
| kaiwenwang wrote:
| That doesn't make sense. If you are a customer that implies
| you pay for it, so people can be users of Yjs which is free
| and open-source, but not customers.
|
| The logic that makes sense is you are using your own framing
| (Moment.dev will later be paid and people will be customers)
| to interpret Yjs.
|
| Moreover, the 'social proof' posted by the following later on
| by 'auggierose' and 'skeptrune': -
| https://news.ycombinator.com/item?id=47396154 -
| https://news.ycombinator.com/item?id=47396139
|
| Appears, to me, to be manufactured. The degree of
| consolidation in this 'SF/Bay Area tech cult' which I've
| noticed, although I am unsure if others are aware, that tries
| to help other members at the expense of quality, growing
| network wealth through favoritism rather than adherence to
| quality, is counterpoint to users whose interest is high
| quality software without capture.
|
| While you may not like me describing this, it is not in your
| own interest to do this because it catabolizes the base layer
| that would sustain you. Social media catabolizes actual
| social networks, as AI catabolizes those who write
| information online. Behavior like this ruins the public
| commons over time.
| antics wrote:
| I'm not sure I fully understand, but to be clear, we
| actually _do_ voluntarily pay for the Free and OSS software
| we use. For example, we support `react-prosemirror`
| directly with monetary compensation. And if we used Yjs, we
| would have paid for that too. So in that sense, I do think
| of us as customers!
|
| It's hard to tell, but I think you also might be saying
| that criticizing the FOSS foundations of our product
| actually hurts the ecosystem. I actually am very open to
| that, and it's why we took so much time writing it since
| part 1 came out. But the Yjs-alternative technology we use
| is all also F/OSS, and we also do directly support it, with
| actual money from our actual bank account. All I'm
| recommending here is that others do the same. Sorry if that
| was not clear.
|
| The rest of your reply, I'm not sure I grok. I think you
| might be suggesting that we are sock-puppeting `auggierose`
| or `skeptrune`, and that we are part of some (as you put
| it) "cult" of the Bay area! Let me be clear that neither of
| these things true. I don't know anyone at Mintlify
| personally, and in any event we are from Seattle not the
| Bay!
| kaiwenwang wrote:
| No, you're not sock-puppeting it yourself. But you all
| are probably friends and cross-promoting. It's a common
| business strategy these days, but to some underhanded
| seeming compared to straightforward ways.
|
| Anyhow, we just have different norms of being. I still
| stand by my above statements and observations, which you
| reject but has plausible deniability, so we'll just leave
| it as is.
| antics wrote:
| Hi folks, author here. I thought this was dead! I'm here to
| answer questions if you have them.
|
| EDIT: I live in Seattle and it is 12:34, so I must go to bed
| soon. But I will wake up and respond to comments first thing in
| the morning!
| drpotato wrote:
| Just wanted to say thanks! This is a great write up and
| resonates with issues I encountered when trying to
| productionise a yjs backed feature.
| samwillis wrote:
| It's disingenuous to suggest that "Yjs will completely destroy
| and re-create the entire document on every single keystroke" and
| that this is "by design" of Yjs. This is a design limitation of
| the official y-Prosemirror bindings that are integrating two
| distinct (and complex) projects. The post is implying that this
| is a flaw in the core Yjs library and an issue with CRDTs as a
| whole. This is not the case.
|
| It is very true that there are nuances you have to deal with when
| using CRDT toolkits like Yjs and Automerge - the merged state is
| "correct" as a structure, but may not match your scheme. You have
| to deal with that into your application (Prosemirror does this
| for you, if you want it, and can live with the invalid nodes
| being removed)
|
| You can't have your cake and eat it with CRDTs, just as you can't
| with OT. Both come with compromises and complexities. Your job as
| a developer is to weigh them for the use case you are designing
| for.
|
| One area in particular that I feel CRDTs may really shine is in
| agentic systems. The ability to fork+merge at will is incredibly
| important for async long running tasks. You can validate the
| state after an agent has worked, and then decide to merge to main
| or not. Long running forks are more complex to achieve with OT.
|
| There is some good content in this post, but it's leaning a
| little too far towards drama creation for my tast.
| antics wrote:
| Author here, sorry if this was not clear: that specific point
| was not supposed to be an indictment of all CRDTs, it was
| supposed to be much more narrow. Specifically, the Yjs authors
| clearly state that they purposefully designed its interface to
| ProseMirror to delete and recreate the entire document on every
| collab keystroke, and the fact that it stayed open for 6
| _YEARS_ before they started to try to fix it, does in my
| opinion indicate a fundamental misunderstanding of what modern
| text editors need to behave well in _any_ situation. Not even a
| collaborative one. Just any situation at all.
|
| I think it's defensible to say that this point in particular is
| not indicting CRDTs in general because I do say the authors are
| trying to fix it, and then I link to the (unpublicized) first
| PR in that chain of work (which very few people know about!),
| and I specifically spend a whole paragraph saying I _hope_ that
| I a forced to write an article in a year about how they figured
| it all out! If I was trying to be disingenuous, why do any of
| that?
| cowboy_henk wrote:
| > sorry if this was not clear
|
| It's easy to make that mistake reading your post because of
| sentences like
|
| > I want to convince you that all of these things (except
| true master-less p2p architecture) are easily doable without
| CRDTs
|
| > But what if you're using CRDTs? Well, all these problems
| are 100x harder, and none of these mitigations are available
| to you.
|
| It sure sounds a lot like you're calling CRDTs in general
| needlessly complex, not just the yjs-prosemirror integration.
| antics wrote:
| To be clear, we _ARE_ arguing CRDTs needlessly complex for
| the centralized server use case. What I am describing in
| the "delete and replace all on every keystroke" problem is
| the point at which it became clear to me that the project
| did not understand what modern text editors need to perform
| well in _any_ circumstance, let alone a collab one.
|
| I think this is still reasonable to say because the final
| paragraph in that section is 100% about how they might fix
| the delete-all problem, and I hope they do, so that I can
| write about that, too. But also, that the rest of the
| article is going to be about how you have to swim upstream
| against their architecture to accomplish things that are
| either table stakes or trivial in other solutions.
| hrmtst93837 wrote:
| You can split CRDT libs and compose them however you want, but
| most teams never get past the blessed bindings, because
| stitching two moving targets together by hand is miserable even
| if you know both codebases. Then you're chasing a perf cliff
| and weird state glitches every time one side revs.
|
| In theory you can write better bindings yourself. In practice,
| if the official path falls over under normal editing, telling
| people to just do more integration work sounds a lot like
| moving the goalposts.
| skeptrune wrote:
| we're about to implement collaborative editing at Mintlify and
| were considering yjs so this couldn't have come at a better time
| antics wrote:
| Author here, my personal mission is for people implementing
| this to have clear, actionable advice. Which is something we
| did not when we started. If you want to chat about it I'm happy
| to help, just email me: clemmer.alexander@gmail.com
| auggierose wrote:
| And let's not forget that the official paper on Yjs is just plain
| wrong, the "proofs" it contains are circular. They look nice, but
| they are wrong.
| breakingcups wrote:
| Could you elaborate on that or share a source? It sounds like
| it'd be not just interesting but important to learn.
| auggierose wrote:
| https://dl.acm.org/doi/epdf/10.1145/2957276.2957310
|
| Try to understand 3.1-3.4 in this paper, and you'll find that
| the correctness proof doesn't prove anything.
|
| In particular, when they define <_c, they do this in terms of
| rule1, rule2, and rule3, but these are defined in terms of
| <_c, so this is just a circular definition, and therefore
| actually not a definition at all, but just wishful thinking.
| They then prove that <_c is a total order, but that proof
| doesn't matter, because <_c does not exist with the given
| properties in the first place.
| mweidner wrote:
| This was my impression as well. If you ignore the paper and
| just look at the source code - and carefully study Seph
| Gentle's Yjs-like RGA implementation [1] - I believe you find
| that it is equivalent to an RGA-style tree, but with a
| different rule for sorting insertions having the same left
| origin. That rule is hard to describe but can eventually be
| proved commutative; I'm hoping to include this in a paper
| someday.
|
| [1] https://josephg.com/blog/crdts-are-the-future/
| lostmsu wrote:
| From the "40 line CRDT replacement": const
| result = step.apply(this.doc); if (result.failed) return
| false;
|
| I suspect this doesn't work.
| antics wrote:
| Author here. I'll actually defend this. Most of the subtlety of
| this part is actually in document schema version mismatches,
| and you'd handle that at client connect, generally, since we
| want the server to dictate the schema version you're using.
|
| In general, the client implementation of collab is pretty
| simple. Nearly all of the subtlety lies in the server. But it,
| too, is generally not a lot of code, see for example the
| author's implementation:
| https://github.com/ProseMirror/website/tree/master/src/colla...
| minikomi wrote:
| Component library page in the docs gives 404
| anentropic wrote:
| I'm curious how these approaches compare with MRDTs implemented
| in Irmin
|
| https://gowthamk.github.io/docs/mrdt.pdf
| dsnr wrote:
| It should be noted that this is about text editing specifically,
| and for other use-cases YJS is using other code
| pathways/algorithms, but you have to be careful how you design
| your data structure for atomic updates.
| ralferoo wrote:
| I just read part 1 as well as part 2, for me it raises an
| interesting question that wasn't addressed. I correctly guessed
| the question posed about the result of the conflict, and while
| it's true that's not the end result I'd probably want, it's also
| important because it gives me visibility of the other user's
| change. Both users know exactly what the other did - one deleted
| everything, the other added a u. If you end up with an empty
| document, the deleting user doesn't know about the spelling
| correction that may need to be re-applied elsewhere. Perhaps they
| just cut and pasted that section elsewhere in the document.
|
| But there's another issue that the author hasn't even considered,
| and possibly it's the root cause why the prosemirrror (which I'd
| never heard of before btw) does the thing the author thinks is
| broken... Say you have a document like "Qing Lai means 'please
| go'" and independently both the Chinese and English collaborators
| look at that and realise it's wrong. One changes it to "Qing Zou
| means 'please go'" and the other changes it to "Qing Lai means
| 'please come'". Those changes are in different spans, and so a
| merge would blindly accept both resulting in "Qing Zou means
| 'please come'" which is entirely different from the original, but
| just as incorrect. Depending on how much other interaction the
| authors have, this could end up in a back and forth of both
| repeatedly changing it so the merged document always ended up
| incorrect, even though individually both authors had made valid
| corrections.
|
| That example seems a bit hypothetical, but I've experienced the
| same thing in software development where two BAs had created
| slightly incompatible documents stating how some functionality
| should work. One QA guy kept raising bugs saying "the spec says
| it should do X", the dev would check the cited spec and change
| the code to match the spec. Weeks later, a different QA guy with
| a different spec would raise a bug saying "why is this doing X?
| The spec says it should do Y", a different dev read the cited
| spec, and changed the code. In this case, the functionality flip-
| flopped about 10 times over the course of a year and it was only
| a random conversation one day where one of them complained about
| a bug they'd fixed many times and the other guy said "hey, that
| bug sounds familiar" and they realised they were the two who'd
| been changing the code back and forth.
|
| This whole topic is interesting to me, because I'm essentially
| solving the same problem in a different context. I've used CRDT
| so far, but only for somewhat limited state where conflicts can
| be resolved. I'm now moving to a note-editing section of the app,
| and while there is only one primary author, their state might be
| on multiple devices and because offline is important to me, they
| might not always be in sync. I think I'm probably going to end up
| highlighting conflicts, I'm not sure. I might end up just re-
| implementing something akin to Quill's system of inserts /
| deletes.
| ralferoo wrote:
| I see someone has downvoted my actually relevant post. Not sure
| why, but anyway.
|
| I also tried out the behaviour of their example. Slowing the
| sync time down to 3 seconds, and then typing "Why not" and then
| waiting for it to sync before adding " do this?" on client A
| and " joke?" on client B. The result was "Why not do this?
| joke?" when I'd have hoped that this would have been flagged as
| a conflict. Similarly, starting with "Why not?" and adding both
| " do this" and " joke" in the different clients produced "Why
| not do this joke?" even though to me, that should have been a
| conflict - both were inserting different content between "t"
| and "?".
|
| Finally, changing "do" to "say" in client A and THEN changing
| "do" to "read" in client B before it updated, actually resulted
| in a conflict in the log window and the resultant merge was
| "Why not rayead this joke?" Clearly this merge strategy isn't
| that great here, as it doesn't seem to be renumbering the
| version numbers based on the losing side (or I've misunderstood
| what they're actually doing).
| gritzko wrote:
| The actual point of the post: Y.js is slow and buggy.
| ghProTip wrote:
| Couldn't agree more with the gist of the argument, especially in
| the context of ProseMirror.
|
| That's why I created prosemirror-collab-commit.
| nchmy wrote:
| Fantastic article. I was particularly interested because
| WordPress has been working to add collaborative editing and the
| implementation is based on yjs. I hope that won't end up being an
| issue...
|
| It would have been nice if the article compared yjs with
| automerge and others. Jsonjoy, in particular, appears very
| impressive. https://jsonjoy.com/
| chuckadams wrote:
| The transport for collaborative editing in Wordpress 7.0 is
| HTTP polling. Once per second, even if no one else is editing.
| It jumps to 4 requests/sec if just two people are editing. And
| it's enabled by default on all sites, though that might not be
| the case when it leaves beta.
| nchmy wrote:
| The transport is a completely different concern... (though
| there's also a websocket implementation).
|
| They use Yjs:
| https://make.wordpress.org/core/2026/03/10/real-time-
| collabo...
| miloignis wrote:
| (Xpost from my lobsters comment since the Author's active over
| here):
|
| I really disagree with this article - despite protestation, I
| feel like their issue is with Yjs, not CRDTs in general.
|
| Namely, their proposed solution: 1. For each
| document, there is a single authority that holds the source of
| truth: the document, applied steps, and the current version.
| 2. A client submits some transactional steps and the
| lastSeenVersion. 3. If the lastSeenVersion does not match
| the server's version, the client must fetch recent
| changes(lastSeenVersion), rebase its own changes on top, and re-
| submit. (3a) If the extra round-trip for rebasing changes
| is not good enough for you, prosemirror-collab-commit does pretty
| much the same thing, but it rebases the changes on the authority
| itself.
|
| This is 80% to a CRDT all by itself! Step 3 there, "rebase its
| own changes on top" is doing a lot of work and is essentially the
| core merge function of a CRDT. Also, the steps needed to get the
| rest of the way to a full CRDT is the solution to their logging
| woes: tracking every change and its causal history, which is
| exactly what is needed to exactly re-run any failing trace and
| debug it.
|
| Here's a modified version of the steps of their proposed
| solution: 1. For each document, every
| participating member holds the document, applied steps, and the
| current version. 2. A client submits (to the "server" or
| p2p) some transactional steps and the lastSeenVersion. 3.
| If the lastSeenVersion does not match the "server"/peer's
| version, the client must fetch recent changes(lastSeenVersion).
| The server still accepts the changes. Both the client and the
| "server" rebase the changes of one on top of the other. Which one
| gets rebased on top of the other can be determined by change
| depth, author id, real-world timestamp, "server" timestamp,
| whatever. If it's by server timestamp, you get the exact behavior
| from the article's solution.
|
| If you store the casual history of each change, you can also
| replay the history of the document and how every client sees the
| document change, exactly as it happened. This is the perfect
| debugging tool!
|
| CRDTs can store this casual history very efficiently using run-
| length encoding: diamond-types has done really good work here,
| with an explanation of their internals here:
| https://github.com/josephg/diamond-types/blob/master/INTERNA...
|
| In conclusion, the article seems to be really down on CRDTs in
| general, whereas I would argue that they're really down on Yjs
| and have written 80+% of a CRDT without meaning to, and would be
| happier if they finished to 100%. You can still have the exact
| behavior they have now by using server timestamps when available
| and falling back to local timestamps that always sort after
| server timestamps when offline. A 100% casual-history CRDT would
| also give them much better debugging, since they could replay
| whatever view of history they want over and over. The only
| downside is extra storage, which I think diamond-types has shown
| can be very reasonable.
| antics wrote:
| I know it seems that way, but it's actually not 80% of the way
| to a CRDT because rich text CRDTs are an open research problem.
| Yjs instead models the document as an XML tree and then
| attempts to recreate the underlying rich text transaction. This
| is much, much harder than it looks, and it's inherently lossy,
| and this fundamental impedance mismatch is one of the core
| complaints of this article. Some progress is being made on rich
| text CRDTS, _e.g._ , Peritext[1]. But that only happened a few
| years ago.
|
| Another important thing is that CRDTs by themselves _cannot_
| give you a causal ordering (by which I mean this[2]), because
| definitionally causal ordering requires a central authority.
| Funnily enough, the `prosemirror-collab` and `prosemirror-
| collab-commit` _do_ give you this, because they depend on an
| authority with a monotonically increasing clock. They also also
| are _MUCH_ better at representing the user intent, because they
| express the entirety of the rich text transaction model. This
| is very emphatically _NOT_ the case with CRDTs, which have to
| pipe your transaction model through something vastly weaker and
| less expressive (XML transforms), and force you to completely
| reconstruct the `Transaction` from scratch.
|
| Lastly for the algorithm you propose... that is, sort of what
| `prosemirror-collab-commit` is doing.
|
| [1]: https://www.inkandswitch.com/peritext/
|
| [2]: https://www.scattered-thoughts.net/writing/causal-
| ordering/
| voctor wrote:
| I think Y.js 14 and the new y-prosemirror binding fix a lot of
| the encountered issues
| antics wrote:
| It _might_ fix the replace-everything bug. It definitely does
| not fix any of the other issues I mentioned. Even just taking
| the permissions problem: Yjs is built for a truly p2p topology
| you as a baseline will have a very hard time establishign which
| peers are and aren 't allowed to make which edits. You can
| adopt a central server, but then the machinery that makes Yjs
| amenable to p2p is uselessly complicated. And if you cross that
| bridge, you'll still have to figure out how to let some clients
| do mark-only edits to the document for things like comments,
| while other can edit the whole text. That _can_ be done but it
| 's not at all straightforward, because position mapping is very
| complicated in the Yjs world.
| mweidner wrote:
| The PowerSync folks and I worked on a different approach to
| ProseMirror collaboration here:
| https://www.powersync.com/blog/collaborative-text-editing-ov...
| It is neither CRDT nor OT, but does use per-character IDs (like
| CRDTs) and an authoritative server order of changes (like OT).
|
| The current implementation does suffer from the same issue noted
| for the Yjs-ProseMirror binding: collaborative changes cause the
| entire document to be replaced, which messes with some
| ProseMirror plugins. Specifically, when the client receives a
| remote change, it rolls back to the previous server state
| (without any pending local updates), applies the incoming change,
| and then re-applies its pending local updates; instead of sending
| a minimal representation of this overall change to ProseMirror,
| we merely calculate the final state and replace with that.
|
| This is not an inherent limitation of the collaboration
| algorithm, just an implementation shortcut (as with the Yjs
| binding). It could be solved by diffing ProseMirror states to
| find the minimal representation of the overall change, or perhaps
| by using ProseMirror's built-in undo/redo features to "map" the
| remote change through the rollback & re-apply steps.
| antics wrote:
| Hi Matt! Good to see you here. For those who don't know, Matt
| also wrote a blog about how to do ProseMirror sync without
| CRDTs or OT here: https://mattweidner.com/2025/05/21/text-
| without-crdts.html and I will say I mostly cosign everything
| here. Our solution is not 100% overlap with theirs, but if it
| had existed when we started we might not have gone down this
| road at all.
| mweidner wrote:
| Your part 1 post was one of the inspirations for that :)
|
| Specifically, it inspired the question: how can one let
| programmers customize the way edits are processed, to avoid
| e.g. the "colour" -> "u" anomaly*, without violating
| CRDT/OTs' strict algebraic requirements? To which the answer
| is: find a way to get rid of those requirements.
|
| *This is not just common behavior, but also features in a
| formal specification [1] of how collaborative text-editing
| algorithms should behave! "[The current text] contains
| exactly the [characters] that have been inserted, but not
| deleted."
|
| [1] http://www.cs.ox.ac.uk/people/hongseok.yang/paper/podc16-
| ful...
| Kjue wrote:
| Am I correctly understanding that you (Moment) have chosen to use
| Prosemirror and that with that using Yjs was the hard part? Or
| did you mean to say in the article that you used Yjs directly? It
| would be less prone to misunderstanding if it read "why we don't
| use y-prosemirror" and you would lose a lot of potential audience
| for the post.
|
| I tried to understand what was wrong in Yjs, as I'm using it
| myself, but your point is not really with Yjs it seems but on how
| the interaction is with Prosemirror in your use case. I can see
| why you're bringing up your points against Yjs and I'm having a
| hard time understanding why you don't consider alternatives to
| Prosemirror directly. Put another way, "because this integration
| was bad the source system must also be bad". I do not condone
| this part of your article. Seems like a sunken cost fallacy to me
| and reasoning about it at anothers expense, but perhaps not.
| Hoping to hear back from you.
| antics wrote:
| So, we are basically making two points.
|
| First, the fact that Yjs bindings, by design (for ~6 years),
| replace the entire document, does in my opinion, indicate a
| fundamental misunderstand what rich text editors need to
| perform well in _any_ circumstance, not just collaborative
| ones. As I say in the article... I hope to be able to write
| another article that this has changed and they now "get" it,
| but for now I do not think it is appropriate to trust Yjs with
| this task for production-grade editors. I'm sorry to write
| this, but I do think it's true! I'm not trying to bag on
| anyone!
|
| Second, and more material: to deploy Yjs to production in the
| centralized case, I think you are very much swimming against
| the current of its architecture. Just one example is
| permissions. There is no established way to determine which
| peers in a truly-p2p architecture have permissions to add
| comments vs edit, so you will end up using a centralized server
| for that. But that's not free, CRDTs are mechanically much more
| complicated! For example, you have to figure out how to
| disallow a user to make mark-only edits if they have
| "commenter" access, but allow editing the whole doc for
| "editor" access. This is trivial in `prosemirror-collab` (say)
| but it's very hard in Yjs because you have to map it "through"
| their XML transformations model.
|
| I'm happy to talk more about this if it's helpful. But yes, we
| are trying to say some stuff about Yjs specifically, and some
| stuff about CRDTs generally.
| karel-3d wrote:
| Sorry that I am too stupid to understand what Moment is.
|
| It is a collaborative markdown file that also renders very fast.
| So far so good.
|
| And then... it somehow adds Javascript? And React? And somehow AI
| is involved? I truly don't understand what it is, and I am (I
| think) the end customer...
|
| edit: I tried it and I just get "Loading..." forever. So, anyway,
| next time.
| twhitson wrote:
| Hey karel-3d, I'm one of the engineers working on Moment and
| would love to help figure out the issue you're running into.
| Would you mind reaching out via our Discord or email
| (trey@moment.dev)?
| karel-3d wrote:
| OK I will be happy to help. I didn't mean to be dismissive!
| Will ping you tomorrow
| torginus wrote:
| The biggest evidence against collaborative editing working and
| being useful is that programmers don't use it. We go through the
| pain of having git branches and manual merges.
|
| We're the nerdiest bunch in the world, absolutely willing to
| learn and adapt the most arcane stuff if it gives us a real of
| percieved advantage, yet the fact that Google Docs style CRDTs
| have completely elided the profession speaks volumes about their
| actual usefulness.
| ansc wrote:
| >Google Docs style CRDTs
|
| Google Docs is OT though.
| zamfi wrote:
| > The biggest evidence against collaborative editing working
| and being useful is that programmers don't use it. We go
| through the pain of having git branches and manual merges.
|
| Hmm -- this seems a bit apples and oranges to me: collaborative
| editing is sync; git branches, PRs, etc. are all async. This is
| by design! You want someone's eyes on a merge, that's the whole
| rationale behind PRs. Collab editing tries to make merges
| invisible.
|
| Totally different use case, no?
| rishabhjajoriya wrote:
| Collaborative editing looks deceptively simple until you deal
| with real-world concurrency and network issues. Operational
| transforms and CRDTs both introduce their own tradeoffs.
| mlysk wrote:
| Great article - you mentioned "two most popular families of
| collab editing [...] OT and CRDT". One thing you should look into
| is the work of https://braid.org - Its combining crdt with ot.
| Work that is inspired by that build the basis of loro which
| allows to prune history (helping with the tombstone issue you
| mentioned)
___________________________________________________________________
(page generated 2026-03-16 23:00 UTC)