[HN Gopher] Loro: Reimagine state management with CRDTs
___________________________________________________________________
Loro: Reimagine state management with CRDTs
Author : czx111331
Score : 190 points
Date : 2023-11-13 10:51 UTC (12 hours ago)
(HTM) web link (www.loro.dev)
(TXT) w3m dump (www.loro.dev)
| rudasn wrote:
| The performance of this looks really interesting, looking at the
| demo gif they have on the page.
|
| I wonder if this is something that can be used for versioning
| database columns / fields.
| meiraleal wrote:
| great post explaining CRDT and the tool.
| hugodutka wrote:
| We've been using https://github.com/electric-sql/electric for
| real-time sync for the past month or so and it's been great.
| Rather than make you think about CRDTs explicitly, Electric syncs
| an in-browser sqlite db (WASM powered) with a central postgres
| instance. As a developer, you get local-first performance and
| real-time sync between users. And it's actually faster to ship an
| application without writing any APIs and just using the database
| directly. Only downside is Electric is immature and we often run
| into bugs, but as a startup we're willing to deal with it in
| exchange for shipping faster.
| erikaww wrote:
| How do you handle migrations?
| jitl wrote:
| The docs say Electric propagates migrations (DDL) on Postgres
| synced tables to their "satellite" clients
| hugodutka wrote:
| Every postgres migration is done through an Electric proxy
| and it converts it into a corresponding sqlite migration that
| it can apply later on the client. In case of a migration that
| would be somehow breaking you can also drop the client-side
| sqlite database and resync state from postgres.
| btown wrote:
| What kinds of bugs have you run into? Any large-scale
| corruption of data at rest?
| hugodutka wrote:
| We have run into queries that corrupted the database client-
| side, but fortunately that doesn't propagate into postgres
| itself. In that case we had to drop the client-side db and
| resync from a clean state.
|
| The corruption was also caught by sqlite itself - it threw a
| "malformed disk image" error and stopped responding to any
| further queries.
|
| Also bugs around syncing some kinds of data - one bug that's
| already been fixed was that floats without decimal points
| would not get synced. https://github.com/electric-
| sql/electric/issues/506
|
| In general electric's team is very responsive and fixes bugs
| when we bring them up.
| randyl wrote:
| Any idea on what the root cause of the sqlite corruption
| was? There's some discussion on the SQLite forums about
| corruption with wasm (I've encountered it myself on a
| personal project), but from what I understand no one has
| identified a cause yet.
| matharmin wrote:
| How do you deal with shapes and permissions not being available
| yet?
| hugodutka wrote:
| There's a workaround - if a table has an "electric_user_id"
| column then a user with that id (based on their JWT) can only
| read rows which have the same id. It's basic but it works for
| us. https://electric-sql.com/docs/reference/roadmap#shapes
| lgessler wrote:
| How are conflicts resolved?
| hugodutka wrote:
| With something called Rich-CRDTs - they were invented by
| Electric's CTO. They have a section in the docs and some blog
| posts dedicated to it: https://electric-
| sql.com/docs/reference/consistency#rich-crd...
| DylanSp wrote:
| I've been wondering how well Electric's been working for people
| ever since I heard about it; good to hear that it's been useful
| for you.
|
| Couple of questions:
|
| - How big is the WASM blob that you need to ship for in-browser
| SQLite? Have you had any noticable issues from shipping a large
| payload to the browser?
|
| - What are you using to persist the SQLite database on clients?
| Have you been using the Origin Private File System?
| hugodutka wrote:
| This is the WASM blob and it's 1.1 MB uncompressed.
| https://github.com/rhashimoto/wa-
| sqlite/blob/master/dist/wa-.... No issues - it's cached by
| cloudflare.
|
| We're using IndexedDB. Here's a writeup on alternatives
| https://github.com/rhashimoto/wa-sqlite/issues/85 and a
| benchmark https://rhashimoto.github.io/wa-
| sqlite/demo/benchmarks.html
| DylanSp wrote:
| Gotcha, interesting. 1.1 MB isn't _too_ bad, especially
| with Cloudflare providing a local PoP. And if this is for
| Hocus, I 'm guessing your frontend isn't used much on
| mobile devices with iffy connections.
|
| That writeup on different SQLite VFS's for in-browser use
| is helpful, thanks for linking that.
| CMCDragonkai wrote:
| I see that the libraries are written in Rust, would this work in
| a nodejs app as a wasm or as native plugin?
| anentropic wrote:
| the docs show installing and using a wasm version from JS:
| https://www.loro.dev/docs/tutorial/get_started
| jitl wrote:
| The demo code for Loro looks very easy to use, I love how they
| infer a CRDT from an example plain JS object. I've played with a
| Zod schema <-> Yjs CRDT translator and found it kinda annoying to
| maintain. However this looks so easy I worry about apps building
| with too little thought about long term data modeling. Migrations
| on CRDTs are challenging, so it's important to "get it right" at
| the beginning. I'm curious how this design works with longer
| term, more complex CRDT apps.
| chris_st wrote:
| Curious how this compares with Automerge/Automerge-repo [0].
| Looks like Automerge is at 2.0.
|
| 0: https://automerge.org/blog/2023/11/06/automerge-repo/
| Inviz wrote:
| Well it's honestly about time. I've tried to build something like
| this personally with OTs, but it can be pretty brutal with all
| the fuzzying and N-way merges. I even chose one of rich editors
| just because it supports OT (then i learned it's only in
| commercial version not even available for small-timers).
|
| I like the completeness of the Loro solution: the state, the rich
| text, the tree. Local-first database approach sounds like a great
| idea. Wondering how large is the code size overhead for using
| this though.
| matharmin wrote:
| How does this compare to Yjs/y-crdt?
| singhrac wrote:
| This looks really neat. I appreciate that you reference the
| previous work in this area (by josephg, Ink & Switch, Fugue
| etc.).
|
| I think the roadmap says that WASM is next as a target, and that
| makes sense for prioritization. Would you also consider
| Flutter/Dart as a target, even if at the level of "we checked
| once that flutter_rust_bridge can call Loro"?
| bxff wrote:
| Congratulations on the launch! Cannot wait to see Loro in action.
| moklick wrote:
| Looks great! We will check it out! And nice to see that you are
| using React Flow in your example
| bzmrgonz wrote:
| This is amazing. Please share with the big projects which need it
| the most.. collabora and libreoffice. Also, a product which the
| world needs badly.. would be a software which would abstract git
| and present text to lawyers as regular word processor, but in the
| backend it's git for the win.
| rapnie wrote:
| > Also, a product which the world needs badly.. would be a
| software which would abstract git and present text to lawyers
| as regular word processor, but in the backend it's git for the
| win.
|
| Ink & Switch Upwelling [0] goes into that direction. A must-
| watch is the StrangeLoop 2023 talk by Martin Kleppmann "New
| Algorithms for Collaborative Editing" [1] that excellently
| explains things.
|
| [0] https://www.inkandswitch.com/upwelling/
|
| [1] https://yewtu.be/watch?v=Mr0a5KyD6BU
| aatd86 wrote:
| Can someone explain to me what happens when there is a
| destructive update on one side while the other side is still
| relying on some old version?
|
| Can this even be reconciliated?
|
| Or is it append only?i.e. No delete operation.
|
| UIs have delete operations in general.
___________________________________________________________________
(page generated 2023-11-13 23:01 UTC)