[HN Gopher] Reverse engineering of Linear's sync engine
___________________________________________________________________
Reverse engineering of Linear's sync engine
Author : flashblaze
Score : 151 points
Date : 2025-05-29 04:29 UTC (2 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| tonetheman wrote:
| Whatever a linear sync engine actually is... sigh. Needs more
| information.
| ralfhn wrote:
| Linear.app is a product management tool like Jira. The article
| describes how they sync their data to their backend
| nologic98 wrote:
| Is this applicable for a consumer mobile app to use for a local-
| first architecture (either conceptually or literally)?
| isaachinman wrote:
| You could achieve something almost identical with Replicache +
| (Mobx or Orama). Only mentioning Mobx because it's what Linear
| uses. That level of the implementation is interchangeable.
| artman wrote:
| Most certainly, if the data that the mobile app consumes is
| bounded and the same data is accessed frequently. Uber for
| example could have benefited from a sync architecture immensely
| (I tried to implement one back in the day, but was too late to
| the party as hypergrowth blocked any attempts at switching
| architectures). Sync architectures are not only great from a
| user experience point of view, but also for developer
| productivity and velocity. Sync takes care of a slew of
| problems that makes feature development slow. I gave a talk on
| this at last year's Local First conf
| https://www.youtube.com/watch?v=VLgmjzERT08&t=4s.
| bhl wrote:
| Ecosystem for local-first and mobile is pretty immature, at
| least for Swift.
|
| In comparison to the web where there's so many libraries e.g.
| Zero, LiveStore, LiveBlocks, I've yet to find a good GRDB
| (sqlite abstraction) integration / client.
|
| Offline-first is definitely very strong, but now how do I get
| data into a remote database with conflict resolution support?
| satvikpendem wrote:
| I simply eschewed a relational database and instead used a
| CRDT like Yrs, Loro, Automerge, etc as my main source of
| truth. The benefit is that they work well on mobile as well
| as every other platform, given they're all written in Rust.
| mappu wrote:
| If you're using AI to write all those em-dashes, please add a
| disclaimer.
|
| For humans i would say a shorter summary is Linear.app syncs a
| client IndexedDB with the server using naive last-write-wins, no
| conflict detection, no OT, no CRDT. There's a global sync ID that
| the server is in control of. Most of the article describes
| minutae of the json schema.
| pottertheotter wrote:
| Never thought someone would be anti em-dashes.
| dheatov wrote:
| I am anti-reading content generated by probabilistic model of
| human language, especially if published without much editing.
| Em-dash is a strong indicator of such.
| jurip wrote:
| It used to be it was easy to tell apart Mac and Windows
| users by em dash usage. Now apparently Mac users are
| considered LLMs.
| ljm wrote:
| Do people have to lower their literacy to the level of a 6
| year old and write like complete dumbasses in order to
| convince you that something isn't AI generated?
|
| I'm sure that pointing out the word 'delve' or the use of
| em-dash says more about the literacy of the reader than it
| does about the humanity of whoever wrote it.
| chrismorgan wrote:
| I can't comment about other venues, but on Hacker News it's
| not at all. The type of people to assiduously use
| appropriate dashes, quotation marks, _& c._ have always
| been heavily represented here.
| MangoToupe wrote:
| > Em-dash is a strong indicator of such.
|
| I see the blind superstition stage of AI has set in
| evaneykelen wrote:
| On macOS, typing two consecutive hyphens automatically gets
| converted to an em-dash in many applications: no AI involved
| necessarily.
| notpushkin wrote:
| I've built a custom layout for that (and a bunch of other
| symbols I frequently use). [?] hyphen for en-dash, [?] |
| hyphen for em-dash (and [?] M is for minus):
| https://typo.ale.sh/
|
| (The idea isn't new, of course: the default macOS layout's
| 3rd layer is absolutely bonkers. I think Ilya Birman was the
| first: https://ilyabirman.net/typography-layout/)
| jdxcode wrote:
| those are the default macos keybindings for en-dash and em-
| dash characters
| bitpush wrote:
| What's the closest opensource library that implements this sync
| (or similar) scheme? ElectricSQL? ZeroSync? Firebase? Something
| else?
| jaccola wrote:
| Firebase (Firestore is the DB) is the best I've ever used but
| not open source. MongoDB has Realm which achieves similar, is
| open source and is OK.
| ochiba wrote:
| Realm's sync functionality (Atlas Device Sync) has been
| deprecated by MongoDB:
| https://www.mongodb.com/docs/atlas/app-
| services/sync/device-...
| rapnie wrote:
| There is a proliferation of sync tools and little
| standardization. Here some in local-first space [0]. Martin
| Kleppmann in his talk last year spoke [1] the need for a
| generic sync protocol, which was very interesting.
|
| [0] https://localfirstweb.dev/
|
| [1] https://www.youtube.com/watch?v=NMq0vncHJvU&t=1016s
| drunkan wrote:
| Someone maintains a list here
|
| https://www.localfirst.fm/landscape
|
| LiveStore shows recreating linear as one of their examples
| though I haven't tried it. It was on the front page recently
| https://news.ycombinator.com/item?id=44105412
| devmakasana wrote:
| Linear's sync engine maintains a local, in-memory object graph
| (backed by MobX) and persists all changes to IndexedDB, allowing
| immediate, offline-first updates.
|
| We build same experience at www.teamcamp.app
| artman wrote:
| I think the impressive part here isn't Linear's sync engine, but
| the fact that Evan Hu went through painstakingly reverse-engineer
| the engine by inspecting traffic and obfuscated code and was able
| to write documentation that is correct and more complete than
| what Linear publishes internally.
| jtwaleson wrote:
| I have a first attempt at a sync engine for my app, but it's very
| primitive. Just a websocket that sends updates based on database
| triggers. If you miss one, you have to do a full reload. I know
| I'll need something better in a year or so.
|
| Any advice on what route to take with creating a sync engine for
| a product like mine? Self-hosted, single binary web app (Rust) +
| Postgres db. Frontend is based on VueJS. I've looked at the
| readme of Yjs and was considering that. I'm a solo dev for now.
|
| I'm tempted to feed Cursor this description of the reverse
| engineered solution of Linear, but I doubt it'll be successful.
| jgeurts wrote:
| Take a look at Electric SQL: https://electric-sql.com/
| jasonjmcghee wrote:
| Yjs isn't a sync engine, it's a data structure for managing
| distributed concurrent updates and ensuring they are conflict
| free.
|
| Whether you use it feels orthogonal to the problem you're
| describing.
|
| ---
|
| For a minimal scope solution, have you considered making a
| table in your database where you log each update? Then you can
| keep an id of your most recent update locally and on websocket
| reconnection ask for the updates after your current change.
|
| Similar to how in-app notifications work.
|
| ---
|
| For local-first, you can use things like:
|
| https://tinybase.org/ https://electric-sql.com/
| https://livestore.dev/
|
| But they are pretty foundational. You use them as your storage
| layer in the front end. So worth considering the scope of the
| change.
| jtwaleson wrote:
| Thanks, that helps! Like I said I had only very briefly
| looked at Yjs.
|
| The thought of an "updates" table has crossed my mind yes,
| but after some time you want a "materialized view" instead of
| replaying the history from the start, and that's where it
| gets complicated ;)
|
| I'll take a look at those alternatives. I'd rather have
| something stable than having to re-invent the wheel. Thanks
| again!
| satvikpendem wrote:
| I'm using Loro as the CRDT as well as Iroh for byte transfer,
| works well. You can look at ElectricSQL as a Postgres sync
| engine but it won't do conflict resolution for you and it's
| hard doing CRDT operations on relational databases on general.
|
| Look into these as well:
|
| https://www.typeonce.dev/article/how-to-implement-a-sync-eng...
|
| https://www.sandromaglione.com/newsletter/lessons-from-imple...
|
| Same author, not necessarily sure why it's on two different
| domains with different content but they open sourced their sync
| engine. If you're interested in this topic, I'd follow. Their
| newsletter as they have great stuff.
| ochiba wrote:
| You can look at PowerSync: https://www.powersync.com/
___________________________________________________________________
(page generated 2025-05-31 23:01 UTC)