[HN Gopher] A Local-First Case Study
___________________________________________________________________
A Local-First Case Study
Author : paulgb
Score : 82 points
Date : 2024-10-01 18:51 UTC (4 hours ago)
(HTM) web link (jakelazaroff.com)
(TXT) w3m dump (jakelazaroff.com)
| com wrote:
| This is great! I was quite excited to see Ink & Switch's Embark
| and now this...
|
| Jake makes creating a local-first multiplayer app seem so simple.
| xrd wrote:
| Ink & Switch is like the Medici family of the Internet era.
| Medici's funded the piano, and I&S is funding local-first. I love
| what they do.
| er4hn wrote:
| > Architecturally, Y-Sweet acts as a bus: clients connect to the
| Y-Sweet server rather than directly to each other. Whenever a
| client connects or makes changes, it syncs its local document
| with the Y-Sweet server. Y-Sweet merges the client's document
| into its own copy, saves it to S3 and broadcasts updates to other
| clients. Since CRDTs are guaranteed to eventually converge on the
| same state, at the end of this process all clients have the same
| document.
|
| I had thought that the advantage of CRDTs was you do not need a
| centralized server and that if you do have a central server
| Operational Transforms are easier. Am I missing why CRDTs are
| used here?
| paulgb wrote:
| (One of the authors of Y-Sweet)
|
| You're right, that is one of the advantages of CRDTs, but it
| turns out to be hard to realize on the web -- aside from RTC
| (which has its own dragons), you still need a server in the
| mix.
|
| The other thing an authoritative server solves is persisting
| the data. Because one server is the authority for a document at
| a time, you can use S3 or R2 for persistence without worrying
| about different servers with different versions of the document
| colliding and erasing each other's changes.
| er4hn wrote:
| Oh, this is interesting. Can you elaborate (or link to some
| post) about what sort of issues you run into? I find the
| concept of CRDTs to be very interesting, but if you still
| need a centralized server I question the value of them over
| OTs. I'd love to understand more about if this is a
| connectivity issue, a CRDT issue, or what.
| paulgb wrote:
| The main issue you run into is that the web is designed
| around client-server communication; you can do peer-to-peer
| communication in theory but because of NAT and firewalls,
| many of those end up not being peer to peer in the end.
|
| Plus, if you want the data to persist so that two people
| can collaborate even if they are never online at the same
| time, you need a server anyway.
|
| CRDTs as data structures support peer-to-peer, it's just
| that in many use cases that aspect of CRDTs is not needed.
| er4hn wrote:
| Okay that all makes a lot of sense thank you.
| com wrote:
| I think that using a bus decomplicates the connectivity story -
| having a "cloud peer" removes quite a bit of coding and testing
| from implementing true peer to peer discovery and
| communications functionality.
|
| Bonus points: you could potentially rip out the bus and replace
| it with something that involves peer to peer connectivity
| without changing client data structures.
| jakelazaroff wrote:
| Author here! A few thoughts on this:
|
| - First and (maybe most importantly), WebRTC in browsers
| requires a central server for signaling. So unless web browsers
| loosen that constraint, a "true" P2P web app without a central
| server is unfortunately infeasible.
|
| - My understanding is that with Operational Transforms, the
| server is "special" -- it's responsible for re-ordering the
| clients' operations to prevent conflicts. I mention a little
| later in the article that Y-Sweet is just running plain Yjs
| under the hood. So it is a central server, but it's easily
| replaceable with any other instance of Y-Sweet; you could fork
| the code and run your own and it would work just as well.
|
| - Peers will only sync their changes if they're online at the
| same time. That means that the longer peers go without being
| online simultaneously, the more their local documents will
| diverge. From a user experience point of view, that means
| people will tend to do a lot of work in a silo and then receive
| a big batch of others' changes all at once -- not ideal! Having
| a "cloud peer" that's always online mitigates that (this is
| true for any algorithm).
| saurik wrote:
| The ability to run your own server--or even not requiring a
| bespoke set of operations--isn't a special property of a
| CRDT; the same thing should be doable with something like
| ShareJS and it's generic tree/JSON structures.
|
| FWIW, though, the author of ShareJS had said some pretty
| strong things pro-CRDT in the past and even kind of lamenting
| his work on OT, so...
|
| https://news.ycombinator.com/item?id=24194091
| josephg wrote:
| Hi, that's me!
|
| OT would work fine to make this collaboratively editable.
| It's just not local first. (If that matters to you.)
|
| With an OT based system like sharejs or google docs, the
| server is the hub, and clients are spokes connecting to
| that hub. Or to put it another way, the server acts as the
| central source of truth. If the server goes down, you've
| not only lost the ability to collaboratively edit. You've
| also usually lost your data too. (You can store a local
| copy, but sharejs not designed to be able to restore the
| server's data from whatever is cached on the clients).
|
| With Yjs (and similar libraries), the entire data set is
| usually stored on each peer the server is just one node you
| happen to connect to & use to relay messages. Because
| they're using Yjs, the author of this travel app could
| easily set up a parallel webrtc channel with his wife's
| computer (in the same house). Any edits made would be
| broadcast through all available pipes. Then even when
| they're on the road and the internet goes down, their
| devices could still stay in sync. And if the server was
| somehow wiped, you could spin up another one. The first
| client that connects would automatically populate the
| server with all of the data.
|
| But whether these local first characteristics matter to you
| is another question. They might be a hindrance - for
| commercial data, centralisation is often desirable. I can
| think of plenty of use cases where replicating your entire
| database to your customers' computers (and replicating any
| changes they make back!) would be a disaster. It depends on
| your use case.
| er4hn wrote:
| On an unrelated note, I'm getting local-first case studies
| everytime I leave a wifi space since my cell phone is still
| suffering from the Verizon outage (Los Angeles area). My
| conclusion has been I can read stuff like calendar invites
| (Google), I can save notes on stuff to do in Trello, but I cannot
| queue up IMs to send in GHC.
| NeutralForest wrote:
| I've been hanging local-first circles but haven't made to switch
| to write anything with it yet. Is there a typical stack people
| recommend to get started? I'm not really sure where to start,
| especially in terms of backend.
| paulgb wrote:
| I'm biased as one of the authors of Y-Sweet, but I think Jake's
| choice of Yjs + Y-Sweet is as good as any to start with. We've
| worked a lot at making the localhost environment easy to get
| started with, and Yjs is the de facto CRDT with the most
| written about it.
|
| https://github.com/jamsocket/y-sweet
| NeutralForest wrote:
| I'd like to start with something that's more DB oriented like
| SQLite both locally and in the browser for example but I
| understand a documented-oriented approach might make more
| sense.
| paulgb wrote:
| In that case, there are a number of recent options.
| https://www.instantdb.com/ and https://electric-sql.com/
| come to mind. I wrote about a number of them here:
| https://digest.browsertech.com/archive/browsertech-digest-
| tr...
| NeutralForest wrote:
| Thank you!
| ochiba wrote:
| There is a fairly comprehensive list of frameworks/tools
| here: https://localfirstweb.dev/
|
| I work on PowerSync and we did a Show HN last year:
| https://news.ycombinator.com/item?id=38473743
|
| Also see InstantDB Show HN:
| https://news.ycombinator.com/item?id=41322281
| matlin wrote:
| You'd probably enjoy Triplit then--especially if you're
| using Typescript.
|
| https://triplit.dev
| xnx wrote:
| Is there some version of local-first that doesn't require a
| webserver, but does seamlessly sync state to a consumer cloud
| service like Google Drive? I'd love to write apps that have all
| the speed and portability of local apps, but the data isn't tied
| to a specific device. It seems like it would be feasible to have
| a large JSON blog background synced to a cloud file service after
| some threshold of accumulated change or time.
| michaelmure wrote:
| Not a file storage but https://github.com/git-bug/git-bug push
| and sync with any git remote. There is a generic data structure
| you can use to build your conflict-free type.
| ochiba wrote:
| This made the rounds on HN recently:
| https://tonsky.me/blog/crdt-filesync/
|
| A PoC of using Dropbox for a local-first app
| chris_pie wrote:
| https://remotestorage.io while it's a specific protocol for
| storing user data on a compatible server, their library also
| provides Google Drive integration
| wonger_ wrote:
| I love how the map automatically updates based on the places
| typed in the editor. A great visual aid to a text-based workflow.
|
| I got confused by this comment though: > To
| determine when to re-render, "reactive" frameworks like Svelte
| and Solid track property access using Proxies, whereas
| "immutable" frameworks like React rely on object identity.
|
| I thought React was just as reactive as all the other JS
| frameworks, and that the state/setState code would look similar.
| jakelazaroff wrote:
| Author here! Maybe I could have worded that better --
| basically, when you call setState in React, it compares the
| identities of the new state and old state to determine whether
| to re-render. Svelte and Solid use "signals" to automatically
| determine when to re-render, with the drawback that you're no
| longer interacting with the "raw" value but a Proxy. But
| neither way would be able to detect Yjs mutating its internal
| state; you're correct that even in React you would need some
| sort of wrapper code "tricking" it into re-rendering when
| appropriate.
| IggleSniggle wrote:
| I haven't worked frontend in a long time but just from reading
| that snippet I would assume React does a `obj===obj` (identity)
| comparison for state update under the hood, while Svelte/Solid
| are doing a Proxy trap on the accessors like `parent.obj =`
| would result in an intercept fn being called when you hit that
| 'obj' access. Proxy being a little more complicated to reason
| about and setup in totality but a lot more powerful and
| flexible.
| jazzypants wrote:
| React is not "reactive" according to the classic definition,
| but even the creator of Solid.js thinks that doesn't really
| matter.
|
| https://dev.to/this-is-learning/how-react-isn-t-reactive-and...
| mentalgear wrote:
| The future is local-first. IF you haven't yet learned about it
| and how to break-out of the expensive cloud/serverless cage, this
| is a good start: https://localfirstweb.dev/
| k__ wrote:
| Awesome stuff. Reminds me of the offline-first movement from ~10
| years ago.
|
| I'm currently looking into TinyBase to make working with high
| latency decentralised services more bearable.
|
| Would be cool if there were a better comparison of the different
| solutions for storage and sync, with features, good use-cases,
| etc.
| tobyhinloopen wrote:
| That's a nice font
| RobCodeSlayer wrote:
| This is awesome! Since you're using CRDTs, do you have any plans
| to make it collaborative? I would find it useful to build an
| itinerary with multiple people
| jakelazaroff wrote:
| It is collaborative! Start a document and send someone else the
| link :)
| catchmeifyoucan wrote:
| Great write-up, I've been looking for a solution like this for
| adding syncing to my local-app!
|
| I love that it's document stored in S3, and it's probably going
| to be way cheaper than if hosted elsewhere in a database. Can't
| wait to try it out soon
| tkiolp4 wrote:
| Would be perfect is somehow it could work without S3. Would be
| awesome if the internet could just work in p2p mode out of the
| box, just some JS and HTML and you have 2 computers talking to
| each other collaborating on a doc without the need of a server
| (or S3)
| josephg wrote:
| You often still want a server somewhere to relay messages.
| Imagine I type something in my computer at home and turn the
| device off before leaving the house. Then on the road I pull up
| the document on my phone. I should see the latest version of
| the document on my phone. However, there were no moments where
| both devices were online at the same time.
|
| For this to work, my home computer needs to upload the changes
| somewhere my phone can access them. For example, a home server
| or a dumb box in the cloud.
|
| It's very difficult to make this work without a server kicking
| around somewhere. So long as the server is fungible (it can
| easily be replaced for basically any other server), I don't
| really see the problem with keeping a server around to relay
| messages.
| braden-lk wrote:
| We built LegendKeeper using Yjs! (It's a mapping app as well, but
| fantasy). Ended up rolling our own sync server to handle large
| scale multiplexing, as we have D&D game-masters with 25,000+
| documents to manage. (I don't know how they do it, tbh!)
|
| We opt for the central server as a super-peer and use the Yjs
| differential update system to avoid loading docs in memory for
| too long. While there are many things about local-first that are
| a huge pain in the ass, the UX benefits are pretty huge. The DX
| can be nice too! Getting to focus on product and not on data
| transit (once you've got a robust sync system) is pretty sweet.
| The first 4 weeks of launching our Yjs-based system was rough
| though; lots of bugs that virally replicated between peers. It
| requires a really paranoid eye for defensive coding; after
| several years, we have multiple layers of self-healing and
| validation on the client.
| jakelazaroff wrote:
| I think a lot of people -- including myself -- would be very
| interested in a longer write-up of that system if you're ever
| interested in sharing more :)
___________________________________________________________________
(page generated 2024-10-01 23:00 UTC)