[HN Gopher] Learn Yjs Interactively
___________________________________________________________________
Learn Yjs Interactively
Author : paulgb
Score : 289 points
Date : 2025-01-16 22:13 UTC (1 days ago)
(HTM) web link (learn.yjs.dev)
(TXT) w3m dump (learn.yjs.dev)
| jakelazaroff wrote:
| Hey HN! I'm the developer at Jamsocket who made this. In case
| you're not familiar with Yjs, it's a CRDT library for building
| collaborative and local-first apps.
|
| The thing is, if you're not used to working with distributed
| state there's definitely a learning curve. Even simple things
| like incrementing a counter -- the "hello world" of JavaScript
| framework demos -- get tricky when dealing with multiple clients.
| Worse, a lot of tutorials are just like "install this library and
| text editor integration and boom you have an app", which doesn't
| give you a good mental model for what's actually happening.
|
| So we made Learn Yjs! It's an interactive Yjs tutorial. I wanted
| it to be really intuitive for people just getting their feet wet
| with local-first development, so there are lots of explorable
| demos and coding exercises. The idea is to use interactive
| examples to build an understanding from the ground up.
|
| Hope you like it :)
| nhatcher wrote:
| Woah! Beautifully done! I wonder if I will finally understand
| CRDT with this. Will go through it over the weekend.
|
| Congratulations!
| santa_boy wrote:
| I am just looking at this but quick question. Is there an
| example for building a local-first app? Also, I working with
| plain vanilla HTML & JS and Alpine. Can this be used in my
| apps?
| paulgb wrote:
| Y-sweet (which this is built with) allows you to make a Yjs
| app local-first by passing "offlineSupport" to the provider
| when constructing.
|
| There's a demo of this here: https://demos.y-sweet.dev/color-
| grid
|
| The source for that demo is here: https://github.com/jamsocke
| t/y-sweet/blob/5fa6941cf5568f4a3f...
|
| That demo uses react, but there is a vanilla js version of
| the same demo here: https://github.com/jamsocket/y-sweet/tree
| /main/examples/vani...
|
| (The vanilla js version is not local first, but it would just
| be a matter of passing the same offlineSupport flag to make
| it so.)
| santa_boy wrote:
| Thank you. This is the direction I needed. Will try out
| steve_adams_86 wrote:
| I just noticed that if you install the example service
| after setting up an account, it installs a version of
| @y-sweet/react which doesn't offer the offlineSupport prop.
|
| The version specified is ^0.5.1. After switching to 0.7.1
| as the repo you linked uses, all is well. Probably not a
| big deal for most newcomers, but it might be worth updating
| dependencies in the example service.
| paulgb wrote:
| Thanks Steve, good catch!
| adeptima wrote:
| Thank you. Keep going. Very good topic to expand
| KaiMagnus wrote:
| > "install this library and text editor integration and boom
| you have an app", which doesn't give you a good mental model
| for what's actually happening.
|
| I was interested in working with CRDTs and collaboration in
| general for a while now and this has been the biggest issue
| whenever I tried to get into it. Websockets also seem a bit
| harder to find (free) hosting for.
|
| Btw, your other tutorials on CRDTs were also a great help.
| Imustaskforhelp wrote:
| Cloudflare workers can host soketi which is websocket.
| jakelazaroff wrote:
| Thank you!
|
| As far as websocket hosting, there are a few services out
| there. We offer a pretty generous free tier for Y-Sweet [1]
| with 10GB of storage. There's also PartyKit and Liveblocks,
| though I'm not sure what their pricing is like.
|
| [1] https://jamsocket.com/y-sweet#y-sweet-pricing
| stopachka wrote:
| The interactive demos are beautiful! Is there a library you used
| to build it?
| jakelazaroff wrote:
| Thank you! The site as a whole is built with Astro and the
| demos are "islands of interactivity" built with React.
|
| The demos are really running Yjs under the hood -- each
| "client" has its own document, and when the user clicks I set a
| timeout to simulate the latency and then manually merge the
| documents into each other. (There's a third document too for
| the timeline in the center.)
|
| Other than that, I'm using dnd kit [1] for the drag and drop
| functionality in the todos demo, Motion [2] for the animations
| and CodeMirror [3] for the text editor.
|
| [1] https://dndkit.com
|
| [2] https://motion.dev
|
| [3] https://codemirror.net
| stopachka wrote:
| Thank you Jake! I just played with the motion website and
| loved it. I am going to try to use it sometime
| techan_liquid wrote:
| Great work Jake! I love the Astro and the "islands of
| interactivity". Is it possible to see the repo, if it's
| public? I couldn't find it anywhere.
| downrightmike wrote:
| The banner image game is silly but fun
| zipy124 wrote:
| Reminds me of the fun reddit used to have with places and such:
|
| Team purple ftw: https://imgur.com/a/Oxc6jUA
| sergiotapia wrote:
| is Yjs a platform like Convex? are they competing? would love a
| comparison page potentially! :pray:
| paulgb wrote:
| You can think of Yjs as a protocol for data synchronization. It
| gives you a way to describe a JSON-like data structure (i.e.
| nested lists and maps), and keep them in sync across multiple
| devices.
|
| Yjs itself doesn't provide a platform, but it's an open
| protocol so there there are service providers (like
| ourselves[1]) that offer Yjs backends as a service (other
| notable providers are TipTap/Hocusocus and Liveblocks).
|
| [1] https://jamsocket.com/y-sweet
| showdown wrote:
| what's the main differences in your opinion between y-sweet
| and Hocuspocus?
| paulgb wrote:
| The biggest difference is that Y-Sweet is built around
| object storage (i.e. S3 and friends) rather than a
| database.
|
| This allows writes to scale horizontally without a central
| database being the bottleneck. It's also a much cheaper way
| to store lots of documents accumulating over time on S3
| than in a database. This is because compute scales with how
| much data you _actually access_ , not how much data you
| store.
|
| I wrote about why object storage is a good fit for this
| here: https://digest.browsertech.com/archive/browsertech-
| digest-fi....
| oefrha wrote:
| I used Yjs in a small side project a while ago. The client side
| was fairly easy to learn and use. On the server side though, I
| wanted to deploy a single binary, but examples in languages other
| than Node (e.g. for the Rust port Yrs) seemed close to
| nonexistent, so I ended up slightly adapting
| https://github.com/yjs/y-websocket (Node) with persistence based
| on LevelDB. That's suboptimal for me. I wonder if there are good
| resources for implementing the server-side with persistence in,
| say, Rust.
|
| Btw, IIRC existing resources for the client side are heavily
| focused on text editor integrations, so this is really helpful. I
| had to fumble a bit myself because I wasn't using a text editor
| integration.
| paulgb wrote:
| You might like Y-Sweet[1], an open-source Rust server that
| powers the banner on Learn Yjs. It uses S3 or the local
| filesystem for storage, and it compiles down to a musl-linked
| binary that weighs a few megabytes.
|
| https://github.com/jamsocket/y-sweet
| oefrha wrote:
| Thanks, I'll look into it. It's pretty low on the list of
| search results for Yjs server, I somehow missed it the last
| time.
| dangoodmanUT wrote:
| another banger from the jamsocket team
| __MatrixMan__ wrote:
| I finally got a four leaf clover and somebody ruined it after
| like 2m.
|
| Edit: Wait, where did everybody go? I think we broke it, or maybe
| I got banned. Kudos to the creator just the same. Neat stuff.
| jakelazaroff wrote:
| You definitely aren't banned! Maybe people just tapered off for
| a bit -- I see a bunch of cursors now.
|
| There are two more secret emojis, by the way :)
| adeptima wrote:
| Yjs is nice. Easy to get p2p results.
|
| Yjs backend, its persistence, eventual conflict resolution (one
| of editors stayed too long offline), history rewind seems like a
| tough engineering challenge to crack.
|
| Always wanted to have a nice experience with block editor like
| Platejs https://platejs.org/
|
| There are nice attempts simplify dev experience with solutions
| like Liveblocks https://liveblocks.io/text-editor However most
| options come at the expense of controlling your data.
|
| CRDT keyword search on HN bring consistently good results and
| interest in the topic, but no good options on open source backend
| side. Checked it many times in the past. Please correct me if I
| missed something.
|
| https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
|
| Most people expectations it should work at least as in Google
| Docs or Notion, and on the dev side it should be store privately
| in Postgres JSONB like format without going deep into details.
| blixt wrote:
| I agree. In theory you need to just distribute serialized
| patches but in a real world backend scenario you may need to
| integrate with knowledge of current document state, user
| identity, and possibly even some level of access control.
|
| I've wanted to use Y.js with a Go backend multiple times but
| gave up each time due to time constraints as it's hard to find
| simple reference implementations.
|
| I've been checking back over the years but it still seems hard
| to do this outside of Node.js.
| adeptima wrote:
| Same experience. Go is my default choice for backend too
|
| It can easily be a full time job
|
| You need to mirror all logic and encoding/decoding part
|
| https://github.com/yjs/yjs/blob/main/src/utils/encoding.js
|
| https://github.com/yjs/yjs/commits/main/src/utils/encoding.j.
| ..
|
| I was thinking about spining off nodejs instance just for
| persistence and syncing with postgres
| jakelazaroff wrote:
| Author here -- I think our own almost server is almost
| exactly what you're looking for? Y-Sweet [1] is open
| source, written in Rust and persists documents to S3. We
| offer a hosted version if you want to quickly check it out,
| which you can use with your own S3 bucket if you want to
| self host eventually.
|
| This is the first I've heard of Platejs, but we do have a
| tutorial on integrating a block editor using BlockNote [2]
|
| [1] https://github.com/jamsocket/y-sweet
|
| [2] https://docs.jamsocket.com/y-sweet/tutorials/blocknote
| 5Qn8mNbc2FNCiVV wrote:
| Fyi, building a collaborative editor right now and it's still
| hard to do outside of Node
| adeptima wrote:
| Still better time spend than doing leetcode.
|
| It's my go-to argument against leetcode style interviews.
|
| I would rather ask a candidate to spend 30 mins and do
| research together on collaborative editing, or visualizing
| distances used in pgvector or similar vector database.
|
| Imagine how far the whole colabarative editing space moved
| forward if 1% of leetcode grinding were rerouted
| danielEM wrote:
| > The trick is a technique called fractional indexing, in which
| indices are fractions rather than integers.
|
| How many times you can use fractional indexing before it stops
| working?
| danielEM wrote:
| Sorry, just did that test - it is 1075, not massive, I would
| risk saying that may be major issue of this lib.
|
| Here is the code:
|
| let left = 0, right = 1
|
| let rightPrevious = 2
|
| let counter = 0;
|
| while (right > left && right < rightPrevious) {
| rightPrevious = right; right /= 2;
| counter++
|
| }
|
| console.log(counter)
|
| Did look very briefly into repo, but didn't find a place where
| it is handled and don't have time to dig it, but if that issue
| is true I may help with resolving it. LMK
| paulgb wrote:
| Good point. Usually fractional indexing is done with
| arbitrary-precision numbers to avoid exactly this issue. We
| landed on using JS floats to teach the technique because it
| makes it more intuitive to understand what's going on, but
| you make a good point and I'll make sure we add that caveat
| and link out to an arbitrary-precision fractional index
| library.
| essentia0 wrote:
| So perplexing - in the interactive demos, the latency slider acts
| not as a network latency but more a debounce buffer? Why? Why? I
| just don't get it
| paulgb wrote:
| We wanted to go with the simplest model of latency that would
| suffice to demonstrate the problems that come up when dealing
| with state mutated in multiple locations. The advantage of this
| approach is that we can do it without hiding any state from the
| reader -- the pie chart exposes the only internal state that
| exists. It's not meant to be an accurate network simulation,
| just to build intuition.
| patwolf wrote:
| I'm currently using Yjs to build an app that needs to work
| offline. It's not actually a real-time or collaborative app.
| However, if you think of the server as being one collaborator and
| the app being another, then it's easy to imagine use cases for
| it.
| 6gvONxR4sf7o wrote:
| The thing I never got past with yjs was that the generalization
| of the todo app item-moving thing in this tutorial gets messy so
| quickly. If you have multiple objects that relate to each other,
| then you want them in the same Document, and keeping the
| relationships in sync has lots of gotchas (like needing the
| fractional indexing in the tutorial).
|
| But then you accumulate a database worth of items and the simple
| toy model of putting everything into one document amounts to
| having to ship the whole DB (and it's history) to anything that
| wants to read any items. So you need to start breaking the items
| into separate objects and persisting them yourself, which has no
| documentation except for this tutorial [0] (spoiler alert: it's
| just an empty "Work in Progress - come back later" and has said
| that for years and years and years).
|
| So you try to build a Provider to manage things, and the
| documentation story is the same [1].
|
| Then you find out about Subdocuments and get happy that there's a
| solution to this common problem [2], but then learn that some
| Providers can't handle any relationships between subdocuments,
| and some Providers don't even support subdocuments, which goes
| back to issue of DIY providers not being documented.
|
| So then you try to read some code to learn about it (after all,
| it's open source, we can't expect the dev to do all the work for
| us for free; let's do some work too!) and realize that what yjs
| really is at its core is a protocol. And that protocol -- the
| very essence of yjs -- is also totally undocumented and spread
| across a bunch of relatively idiosyncratic code. Unless you count
| a paper that documents how a yjs precursor worked a decade ago.
|
| I didn't intend this to turn into a rant, but I've spent so much
| time trying to get it working over the years, and the gap between
| the toy 'Look it's magic!' demos and anything real is just so
| wide and the documentation for that part has been empty except
| "Work in Progress" for a decade.
|
| I'm hopeful that some of the companies trying to monetize yjs,
| like the author of this post, are willing to step into that gap
| (which is who I'm writing this for, btw).
|
| [0] https://docs.yjs.dev/tutorials/persisting-the-document-
| to-a-...
|
| [1] https://docs.yjs.dev/tutorials/creating-a-custom-provider
|
| [2] https://docs.yjs.dev/api/subdocuments
|
| [3] I was really hoping that as I wrote this, the situation has
| changed, but it looks like all the documentation is just as WIP
| as five years ago and same for subdocument support.
___________________________________________________________________
(page generated 2025-01-17 23:01 UTC)