[HN Gopher] LiveStore: State management based on reactive SQLite...
___________________________________________________________________
LiveStore: State management based on reactive SQLite and built-in
sync engine
Author : akoenig
Score : 125 points
Date : 2025-05-27 09:50 UTC (13 hours ago)
(HTM) web link (livestore.dev)
(TXT) w3m dump (livestore.dev)
| jessmartin wrote:
| Just saw the talk on it at Local-first Conf. So many snyc engines
| these days. LiveStore is exploring an interesting space in the
| idea maze: marrying event sourcing with sync engines.
|
| I couldn't believe how robust LiveStore is already. I've been
| trying it out for a new project over the last few weeks and it's
| so smooth.
| schickling wrote:
| Thank you! So glad you like it!
| apitman wrote:
| How is the conf going? Would have loved to attend but couldn't
| quite swing it. Can't wait for the videos to drop.
| schickling wrote:
| Hi folks, creator of LiveStore here (prev. founder Prisma).
|
| Very excited to launch LiveStore in beta today after having
| worked on it over the past 4 years. I've built it for myself
| working on Overtone, an ambitious music client aiming for a
| native-grade high-performance app feel.
|
| LiveStore embraces SQLite by adding a signals-based reactivity
| layer and combines it with event-sourced based syncing (similar
| to Git).
|
| Happy to answer any question! Looking forward to thoughts and
| feedback!
| shortcord wrote:
| I'm so glad to see this.
|
| I was just looking into LiveStore this past month for a hobby
| project, but it was behind a beta preview, so I hope to be able
| to dig in soon to see if it can be useful to me.
|
| I love how you're pushing forward the localfirst conversation.
|
| If you've spent any time building offline capable web apps with
| some sort of syncing mechanism, you begin to very quickly see
| the usefulness of a sync engine.
| wg0 wrote:
| I have evaluated the local first landscape and few solutions
| are as neat as this one! The only other (more mature seems to
| be) is tinyBase but that's based on a different model (CRDTs vs
| Event sourcing)
|
| Few questions:
|
| Would it not be possible to store and rely on larger amount of
| data on SQLite and keeping it on disk as an option? I see that
| there's a limitation of 1 GB. So maybe just a config flip to
| change the mode of persistence?
|
| Also - multi-tenancy could be an interesting scenario. Think
| JIRA but each organisation needs their own namespace and each
| user/client wouldn't need all the tickets from all the
| organisation but only of their own (or even more scoped to
| department and team) therefore the local database would alway
| be a subset of the whole universe.
|
| An out of the box sync server that could be run directly on
| Bun/Node (without Cloudflare) could be a nice addition.
|
| But overall, really elegant solution. I like it, might give it
| a try in an idea I am evaluating various tools for.
| Unfortunately, the multi-tenancy story is critical for that.
| jessmartin wrote:
| curious: did you literally use the Local-first Landscape[1]
| to do the evaluation? (full disclosure: co-author of
| landscape)
|
| [1] https://localfirst.fm/landscape
| wg0 wrote:
| Yes already did through most of them. Either we have
| libraries around CRDTs or we have full BaaS (think Parse if
| anyone remembers or Firebase) soft of things that possibly
| can be self deployed but have a very heavy and iron clad
| curtain on the underlying storage medium (Postgres/SQLite)
| etc exposing a world view of their own.
|
| tinyBase seems to be the only one that stands out in that
| regard or this LiveStore now.
|
| PS: Thank you for putting this landscape. That really has
| enabled me or anyone else to look at what options are
| there. No amount of googling would do that kind of
| compilation. I find it very useful.
| rramon wrote:
| Do you think it (in combination with React, WASM) could replace
| something like the Juce framework that most music apps use? I'm
| a beatmaker myself and would like to get into music apps but
| found the Juce stuff and C++ always scary.
| apitman wrote:
| Juce is more common for music creation apps right? I believe
| Overtone is more of a Spotify alternative.
| rapnie wrote:
| > LiveStore extends its local event-sourcing model globally by
| synchronizing events across all clients, typically through a
| central sync backend.
|
| Is that a requirement or could there also be federated nodes,
| or a full p2p modus operandus? Think for use in decentralized
| social networking.
| ochiba wrote:
| I really enjoyed the talk at Local-First Conf today -- well
| done. I thought it was very well explained and made compelling
| arguments for the event-sourcing materialized into SQLite
| architecture.
|
| Thank you for championing SQLite and especially OPFS Wasm
| SQLite on the web -- we (PowerSync) are clearly also big
| proponents of it, so love to see other projects having success
| with it too.
| growthwtf wrote:
| Sounds very cool. Not impressed with the cross platform claim
| when the first thing I see is that android web is not supported,
| though.
| schickling wrote:
| That's a good point. I'm in touch with the Android/Chrome team
| about the underlying issue.
|
| I was hoping the underlying Android web issue would have been
| fixed by now (as I first noticed it ~3 years ago with some
| indication for progress), but looks like LiveStore needs a
| custom workaround for it. You can track the progress here:
| https://github.com/livestorejs/livestore/issues/321
|
| I hope you understand that bridging the gaps between various
| levels of supported web APIs take a lot of effort and is non-
| trivial when building ambitious systems like LiveStore.
| wlib wrote:
| I've been working on a very similar sync problem and hit this
| too. I think the way forward is to use a broadcast channel to
| elect an individual worker that communicates with all other
| contexts using the broadcast channel.
| dugmartin wrote:
| fyi, I've done something similar and the cleanest method
| for leader election is to use the web locks api:
|
| https://developer.mozilla.org/en-
| US/docs/Web/API/Web_Locks_A...
|
| It is supported by all browsers. You just have all the tabs
| try to lock the same resource and have them return a
| promise in the lock callback. The first one wins and when
| that tab closes the next one in line gets automatically
| elected.
|
| The leader can then use a broadcast channel and act as a
| server for all the other tabs to serialize access to any
| shared resources.
| wlib wrote:
| That's very elegant, thanks!
| growthwtf wrote:
| Very interesting context. Thank you for sharing.
| brianzelip wrote:
| A recent episode of the excellent LocalFirst.fm podcast (hosted
| by the author) explores LiveStore a bit,
| https://www.localfirst.fm/24
| schickling wrote:
| Thanks for sharing this episode. Planning to do a dedicated
| episode about LiveStore some time soon. :)
| akshayKMR wrote:
| Neat to ship dev-tools alongside, looks like you've been dog-
| fooding this library for quite some time.
|
| What are your thoughts on:
|
| - Handling compaction for long lived apps/pages?
|
| - IMO events are nice, but also require discipline and good
| design w.r.t code (Thinking of how application layer evolves over
| time, possibility of maintaining code for older client versions
| and/or schema migrations.
|
| - Overtone looks sick. I've been using Spotify less and less
| because of janky UI and constant UI changes, would love a
| replacement. I see that it supports multiple sources, will it
| offer offline playback?
| chrisweekly wrote:
| "sick" in the slang sense, ie "amazing / awesome"
|
| (clarifying for non-native English readers)
| schickling wrote:
| Great questions!
|
| > Handling compaction for long lived apps/pages?
|
| That's a very common question and something I'm planning to
| ship a solution for soon. The basic idea is to give each event
| some more semantic "meaning" by annotating the event definition
| which allows you to express which events "semantically
| overlap". For example in a todo app you could express that the
| "todoCompleted" event for a given task id can compact other
| "todoCompleted" / "todoUncompleted" events for the same task
| id.
|
| You can track the progress of this topic here:
| https://github.com/livestorejs/livestore/issues/254
|
| > IMO events are nice, but also require discipline and good
| design w.r.t code
|
| Yes, I agree with that. When it comes to data there is "no free
| lunch" - it's all about tradeoffs. I prefer the tradeoffs of
| event sourcing though for my own use cases such as Overtone.
|
| That being said for many situations (like older client versions
| etc) there are pretty straightforward ways to address those
| concerns. Always depends on your application use case and
| possible tradeoffs though.
|
| > Overtone looks sick. I've been using Spotify less and less
| because of janky UI and constant UI changes, would love a
| replacement. I see that it supports multiple sources, will it
| offer offline playback?
|
| Very excited to hear! I'm sharing your frustrations which is
| why I'm building Overtone (next to many other reasons). Re
| "offline playback": That will depend on where your music is
| coming from. e.g. for your own music collection in Dropbox (or
| similar) it will be supported. For music streaming services
| like Spotify it will depend on their terms.
|
| Hope that all makes sense?
| gbalduzzi wrote:
| This looks very promising but I'm a bit cautious not to fall in
| an hype trap.
|
| I was experimenting with something similar for a custom local-
| first app with multi device support.
|
| Would it be possible to add optional E2E encryption to this?
| While reading the docs, by adding encryption to the event payload
| you would be almost there already, even though it would prevent
| log compaction on the server
| schickling wrote:
| Totally agree re "hype trap". I'm building LiveStore for myself
| while working on Overtone which mostly informed the design
| decisions. I'm building LiveStore/Overtone to last!
|
| > Would it be possible to add optional E2E encryption to this?
|
| Yes, that's something that should already be possible, though I
| haven't done this myself yet. Happy to help if you're running
| into any issues.
|
| Will definitely keep this use case in mind while working on
| compaction. One solution could be that only clients could do
| the compaction.
| eiiot wrote:
| LiveStore is the only sync engine I've used for Expo / RN that
| actually just works out of the box. In a space that is seemingly
| mature it's nice to see something polished and I'm excited to see
| how it develops over the next year!
| parsadotsh wrote:
| Congrats on the launch!
|
| Would this fit into the "1. Serialization" strategy described
| here? https://mattweidner.com/2024/06/04/server-
| architectures.html...
|
| If so, would it run into the same issue ProseMirror-collab did as
| mentioned in the article? As in: A frequently updating low-
| latency client can lock out another high-latency client's
| updates.
| oliwary wrote:
| Interesting - a lot of problems I face seem to fit into this
| general framework, and I feel like I keep reinventing the wheel.
| How is conlict resolution handled?
| rudi-c wrote:
| This looks like interesting technology! Congrats on the launch,
| it's great to have people exploring the space of realtime data
| storage & sync. Some thoughts, that I hope you can find
| constructive.
|
| The landing page draws comparison to Figma, Linear and Notion.
| But they are vastly different use cases.
|
| Figma is document-centric, which means that:
|
| - All data is tied to a single document, limiting their size.
|
| - Requires that single backend session holds a document in-memory
| and deals with split-brain issues.
|
| - Operational complexity arises around deployment.
|
| - User interactions are often continuous (one event / frame),
| imposing tighter latency requirements.
|
| - Generally requires that the document stays loaded in a memory
| in a stateful backend session.
|
| - Technologies like Jamsocket are targeting such use cases.
|
| Notion & Linear, on the other hand, have collaborative editing
| but not to the same degree of realtimeness. - Data is not tied to
| a single document.
|
| - Data can grow unbounded.
|
| - Data is typically more relational.
|
| Presenting both these uses cases adds confusion. By using SQLite
| (emphasis: database and "SQL"), it makes the technology appear at
| first glance more suited for the Linear/Notion use case. However,
| the opposite appears to be true after reading
| https://docs.livestore.dev/evaluation/when-livestore/.
|
| - "All the client app data should fit into a in-memory SQLite
| database".
|
| - "Reasons when not to use LiveStore" -> "Your app data is highly
| connected across users".
|
| The Figma-like document use case does seem like something that
| LiveStore could support. When it comes to designing a data model
| for collaborative apps (CRDT or CRDT-like), the most simple and
| flexible solution is to store every object as a map of [Object
| ID] -> [Property Name] -> [Property Value]. Assuming that the
| properties names & values come from a fixed set and are typed
| (this will generally be true unless you allow for arbitrary user-
| defined fields), that looks like a database row! So why not just
| store it in a database, indeed.
|
| However, among databases, there are object stores and relational
| databases. The latter is useful, as applications often want to
| represent relationships between different objects. But if we
| support multiplayer-like use cases which implies the absence of
| server-side transactions, how should conflicting events involving
| the creation/deletion/relationships between objects be handled?
| Is it entirely on the application to think about it? This is an
| interesting topic that I believe LiveStore seems well-positioned
| to innovate in.
|
| I also find the "local-first" emphasis to be at odds with all of
| Figma/Notion/Linear. Local-first software tends to have a limited
| number of concurrent users accessing the data in-mind. SaaS
| applications that are truly targeted at collaboration tend to
| have a lot of different requirements. On the other hand, those
| applications do tend to have relatively limited offline support,
| which a local-first application tends to emphasize more.
|
| The Figma-like use cases also has additional requirements. But I
| understand this is beta and look forward to seeing further
| development!
| imslavko wrote:
| I worked with Rudi at Figma and of course support his comment -
| Figma seems to be mentioned for marketing, not for the actual
| technical comparison.
|
| For others looking for more details on how Figma's sync engines
| differ and why 2 sync engines emerged, I had a long thread
| about it here:
|
| https://x.com/imslavko/status/1890482196697186309
| IdPreferNotTo wrote:
| ...incurabel...next!
| pdyc wrote:
| seems you are using wa-sqlite. Can you elaborate on your strategy
| for persisting data offline? are you using opfs(which variant
| AccessHandlePoolVFS?) or indexddb? Also how are you mitigating
| against unstable implementation of opfs in different browsers and
| 7 day retention policy of safari for indexeddb?
|
| since sqlite now also provides official wasm builds why did you
| decided to go with wa-sqlite?
| bilater wrote:
| Landing page is awesome! Love the way you demo the software. This
| is a crowded space with Zero/Convex/Tinybase etc and others so
| this is a great way to stand out.
| maxjustus wrote:
| This looks amazing! One small thing I noticed: I was watching the
| demo video on the landing page and at 1:07 your voice
| inexplicably pans almost completely to the left speaker - not a
| big deal but figured I'd apprise you of it.
___________________________________________________________________
(page generated 2025-05-27 23:01 UTC)