[HN Gopher] Show HN: Triplit - Open-source syncing database that...
___________________________________________________________________
Show HN: Triplit - Open-source syncing database that runs on server
and client
Hey HN, we're Matt and Will, the co-founders of Triplit
(https://www.triplit.dev). Triplit is an open-source database
(https://github.com/aspen-cloud/triplit) that combines a server-
side database, client-side cache, and a sync engine into one
cohesive product. You can try it out with a new project by running:
(npm|bun|yarn) create triplit-app As a team, we've worked on
several projects that aspired to the user experience of Linear or
Superhuman, where every interaction feels instant like a native app
while still having the collaborative and syncing features we expect
from the web. Delivering this level of UX was incredibly
challenging. In each app we built, we had to implement a local
caching strategy, keep the cache up to date with optimistic writes,
individually handle retries and rollbacks from failures, and do a
lot of codegen to get Typescript to work properly. This was spread
across multiple libraries and infrastructure providers and required
constant maintenance. We finally decided to build the system we
always wanted. Triplit enables your app to work offline and sync in
real-time over websockets with an enjoyable developer experience.
Triplit lets you (1) define your schema in Typescript and simply
push to the server without writing migration files; (2) write
queries that automatically update in real-time to both remote
changes from the server and optimistic local mutations on the
client--with complete Typescript types; (3) run the whole stack
locally without having to run a bunch of Docker containers. One
interesting challenge of building a system like this is enabling
partial replication and incremental query evaluation. In order to
make loading times as fast as possible, Triplit will only fetch the
minimal required data from the server to fulfill a specific query
and then send granular updates to that client. This differs from
other systems which either sync all of a user's data (too slow for
web apps) or repeatedly fetch the query to simulate a subscription
(which bogs down your database and network bandwidth). If you're
familiar with the complexity of cache-invalidation and syncing,
you'll know that Triplit is operating firmly in the distributed
systems space. We did a lot of research and settled on a local
first approach that uses a fairly simple CRDT (conflict-free
replicated data type) that allows each client to work offline and
guarantees that they will converge to a consistent state when
syncing. It works by treating each attribute of an entity as a last
writer wins register. Compared to more complex strategies, this
approach ends up being faster and doesn't require additional logic
to handle conflicting edits between concurrent writers. It's
similar to the strategy Figma uses for their collaborative editor.
You can add Triplit to an existing project by installing the client
NPM package. You may self-host the Triplit Server or pay us to
manage an instance for you. One cool part is that whether you
choose to self-host or deploy on Triplit Cloud, you can still use
our Dashboard to configure your database or interactively manage
your data in the Triplit Console, a spreadsheet-like GUI. In the
future, we plan to add APIs for authentication, file uploads, and
presence to create a Supabase/Firebase-like experience. You can
get started by going to https://triplit.dev or find us on Github
https://github.com/aspen-cloud/triplit. Thanks for checking us out
and we are looking forward to your feedback in the comments!
Author : matlin
Score : 138 points
Date : 2024-06-25 13:53 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Jayakumark wrote:
| Is there a pre-existing demo page.
| matlin wrote:
| We've setup a demo environment to get a feel for it here:
| https://demo.triplit.dev/
|
| It's actually running two clients and the server inside the
| browser window so even though it's simulating real-world, it's
| the actual code that powers Triplit.
| Jayakumark wrote:
| thanks, but if i add a new todo in sync mode left say "uni" ,
| go offline and change one side to "universe" and other to
| "unicorn", one overwrites another its random on who wins.
| Shouldn't it maintain both ?
| matlin wrote:
| Technically speaking both values are preserved in the
| history but only one "wins" because each attribute is a
| last-writer-wins register. This works for most use cases
| e.g. you wouldn't want two concurrent edits to checkbox to
| leave it simultaneous true AND false state.
|
| However, when it comes to collaborative text you're
| absolutely right and support is on our roadmap. A
| researcher in this space has already implemented his
| collaborative sequence library on top of Triplit with the
| Quill text editor and you can check it out here:
| https://github.com/mweidner037/list-positions-
| demos/tree/mas...
| arcticfox wrote:
| This is really cool. Feels like the future of app development.
|
| But also I'm getting old, and I had the same feeling when
| RethinkDB came out. Do you guys have any thoughts on how your
| system compares to what they were doing and what eventually
| happened to them?
| matlin wrote:
| I can't speak to RethinkDB's history but we're really focused
| on a specific use case which I think helps. Specifically,
| Triplit is made for web developers so we have great TS support,
| React and Svelte bindings, etc which makes adoption and
| marketing much easier.
| c01nd01r wrote:
| Any plans to add Vue.js support?
| matlin wrote:
| Yep! We've just prioritized based on what's been requested
| in our Discord and surprisingly Svelte has been requested a
| lot more than Vue so we did that first but Vue is on the
| way.
| wernst wrote:
| We definitely think its the future! I've always found this
| article captures the ethos well: https://tonsky.me/blog/the-
| web-after-tomorrow/
| terpimost wrote:
| Pretty cool guys. It would be nice to know the differences to
| other local first like solutions out there.
| matlin wrote:
| There are a lot of new local-first tools these days but broadly
| speaking Triplit lends itself to more typical client-server
| applications with a central, authoritative server where other
| tools in this space are oriented around decentralization and
| server-agnostic systems.
| curtisblaine wrote:
| Since this is LWW, does it mean the amount of information on the
| client scale linearly with the number of operations? (meaning:
| the more users modify a database, the more the operations log
| grows)? Or do you do checkpoints? How does it scale space-wise
| when the user does millions of ops per day?
| matlin wrote:
| Triplit does persist the history of edits to a given attribute
| but indexes on the latest values makes querying stay fast.
| However, LWW Registers by nature doesn't actually require
| storing the history that's just our current implementation that
| allows clients to sync efficiently even after being offline for
| a while.
|
| In terms of scaling to a million-ops per day we are probably
| not quite there yet but one advantage to having the server be
| the authority is that, in the future, the Triplit server could
| track the timestamps that each client last synced at and
| progressively prune the history similar to how Postgres handles
| VACUUM'ing dead tuples.
| troyjfarrell wrote:
| Looks like it could be a more batteries-included/opinionated
| alternative to RxDB (https://rxdb.info). The relational queries
| might help some people who tend to think in SQL as opposed to
| documents (as in CouchDB or MongoDB) and the WebSockets for
| synchronization will help people get started more quickly. (RxDB
| provides interfaces for those who want to implement their own
| storage engine and/or synchronization backend.)
| candiddevmike wrote:
| Why did you choose to license this under the AGPL?
| victor9000 wrote:
| So you have to AGPL your product because of the database that
| it uses lol, hard pass.
| matlin wrote:
| I'm not a legal expert but my understanding is that your
| perspective is a common misconception.
|
| I'd recommend reading this:
| https://drewdevault.com/2020/07/27/Anti-AGPL-propaganda.html
| andrewmutz wrote:
| What types of usage do you think the AGPL would prevent
| that the GPL license would have allowed? What situations
| were you trying to prevent?
| matlin wrote:
| From my understanding, if someone were to create a
| private fork of Triplit Server add some modifications and
| then launch a hosted service (i.e. serving the modified
| Triplit Server over the internet) they wouldn't be
| compelled to open-source their modifications under GPL
| but would under AGPL.
| andyp-kw wrote:
| Why would your whole product need to be AGPL just because the
| database is ?
|
| It's like saying that Oracle owns my product because I use
| MySQL.
| scosman wrote:
| MySQL is GPL not AGPL. Very different.
| matlin wrote:
| Our hope with the AGPL license is that it will make Triplit
| easily self-hostable while ensuring that anyone who makes
| modifications contributes those back to the community.
| candiddevmike wrote:
| > while ensuring that anyone who makes modifications
| contributes those back to the community
|
| The AGPL doesn't really require this though. It requires the
| source code to be distributed to users, it does not require
| anyone to contribute those modifications back.
| matlin wrote:
| Right, that's a good clarification.
| pcthrowaway wrote:
| Based on you making your frontend libraries AGPL, I believe
| any frontend using them would have to be generally compatible
| with the GPL family (so would also have to be *GPL licensed)
|
| Is my understanding incorrect?
| resoluteteeth wrote:
| Yeah this seems like a major issue. I think the AGPL is
| strictly stronger than the GPL so if it was possible to use
| a GPL or AGPL library from non-GPL code then surely there
| would be no reason for the LGPL license to exist?
| pcthrowaway wrote:
| Yeah, unfortunately I think it's debatable whether LGPL
| is even OK for a closed-source project to use, since IIRC
| LGPL specifically talks about linking compiled binaries.
|
| I guess there's the argument that transpiled, minified
| javascript program is a compiled binary? I'd love to know
| whether these things have been tested in various courts.
| imkyle wrote:
| This is just using sqlite under the hood and none of this will
| scale. Why build your own query builder as well when you could
| just hook in one of the many ORMs in the NPM ecosystem and add
| support for 50+ other databases. This also isn't really something
| you could drop into an existing project very easily. Overall, I'm
| not impressed.
|
| Here are some other options I prefer:
|
| https://feathersjs.com/ https://rethinkdb.com/
| jstummbillig wrote:
| Something something Dropbox
| AlexErrant wrote:
| What does "scale" here mean? Sqlite can go quite far
| https://use.expensify.com/blog/scaling-sqlite-to-4m-qps-on-a...
|
| I agree with you w/r/t "Why build your own query builder"
| though since (AFAIK) you can't touch the underlying SQL. I'm
| making heavy usage of FTS5 in my app, and it doesn't look like
| they support that.
|
| And, to be fair, feathers/rethink also have the weaknesses you
| list:
|
| - Why build your own query builder
|
| - This also isn't really something you could drop into an
| existing project very easily
|
| Personally, I've been pretty happy with
| https://github.com/vlcn-io/cr-sqlite/ which is just a sqlite
| extension that can run in the browser.
| matlin wrote:
| Can you point me to what gave you this impression? We implement
| our own query engine for Triplit and use a fork of tuple-
| database[1] for storage which is low level database that _can_
| store to SQLite (we do on the server) but in the browser it
| uses either an in-memory btree or IndexedDB.
|
| Regarding compatibility with our databases, we do have some
| internal experiments of running a Triplit Server in front of
| existing databases as almost like a syncing cache. Still very
| experimental but maybe you'd be more into that use case.
|
| 1. https://github.com/aspen-cloud/tuple-database
| crabmusket wrote:
| Very cool to see a use of tuple-database in the wild! I've
| known about that project for a while and thought it looked
| really cool, but I don't have a use for it.
|
| What's the reason for the fork?
| matlin wrote:
| No real reason other than we wanted to customize it and
| move quickly. I'm sure Chet, the maintainer, wouldn't
| oppose any of the changes we've made.
| salviati wrote:
| Do the databases you propone work offline too?
|
| If they don't they're not a good match at all.
| imkyle wrote:
| I use React Query[0] for frontend data management regardless
| and my apps work great as PWAs.
|
| 0. https://tanstack.com/query/latest/docs/framework/react/qui
| ck...
| matlin wrote:
| React Query (and other query caching libraries) are great
| until you have overlapping data between your queries. E.g.
| you edit a message in one place but other queries that show
| that same message (like a preview in your chat list) don't
| update.
|
| Having a complete query engine on the client makes this a
| lot easier to deal with. It's why most popular messaging
| apps will use SQLite on the client to render the UI and
| build their own sync engine to keep it up to date with the
| backend.
| satvikpendem wrote:
| That is not a CRDT level solution though, it doesn't work
| with multiple clients with disparate conflicting data.
| mrtesthah wrote:
| What is Triplet's syncing latency?
| matlin wrote:
| I don't have official measurements but it's fast enough for
| most things short of realtime cursors or anything updating on
| an sub-second interval--we'll be adding a presence API for
| things like that. It'll also depend on the complexity of the
| query.
|
| Do you have a specific use case in mind?
| mrtesthah wrote:
| That is my use-case: syncing things like cursor positions,
| scroll offsets, and other realtime metadata. I'll await your
| presence API.
| armincerf wrote:
| We've been using triplit to store user settings that can be
| managed by admins, it's important users always feel like the app
| runs locally and they often don't have good internet but we still
| needed sync as the users often switch devices and admins need to
| see and manage other users settings.
|
| Overall triplit has been really great, both as a frontend dx and
| also their support - whenever we find an issue or have a feature
| it gets handled very quickly by the team which is awesome!
|
| As soon as they have an answer for HA deployments we will be
| moving more critical data there instead of Postgres
| matlin wrote:
| It's been great working with you to get things right. Your
| feedback and bug reports have helped immensely to shape
| Triplit!
| satvikpendem wrote:
| I believe I saw your video presentations on YouTube [0] from the
| Local First Discord server [1], so nice to see your Show HN here.
| I am not using TypeScript so I might not be your target audience,
| I'm using local-first especially for mobile apps where
| connections can be spotty unlike the web which by definition (at
| least on first load) is always connected to the internet, and I'm
| using Flutter for this use case, as well as a backend in Rust.
| Other local first solutions are generally agnostic to the client
| and server because they work directly on the database layer, ie
| ElectricSQL and PowerSync will sync the client and server
| database. Other solutions with CRDTs can also work on the client
| and server with some FFI, ie automerge is in Rust so in my case
| it'll work via FFI on the Flutter side via flutter_rust_bridge
| (or WASM on the web) and of course on my Rust backend server, but
| it looks like you are looking to offer a more classic client-
| server syncing solution instead needing conflict free resolutions
| on disparate clients, ie the server is the source of truth.
|
| So with all that, my question is, is there a reason you guys went
| with more of a full language level solution rather than being
| more agnostic to the client and server? It seems harder to
| support other languages and frameworks other than JS based ones
| in the future, but I suppose the market for that is already big
| enough. ElectricSQL etc also have SDKs for TypeScript as well as
| Flutter and others so they are similar to your solution but it
| seems like they can support more clients and servers in the
| future just by building SDKs for them.
|
| Another question, looks like you eventually want to compete with
| Supabase but they are already experimenting with database level
| syncing and CRDTs in Postgres [2] and might catch up with your
| solution, any thoughts on that?
|
| [0] https://www.youtube.com/playlist?list=PLTbD2QA-
| VMnXFsLbuPGz1...
|
| [1] https://localfirstweb.dev/
|
| [2] https://news.ycombinator.com/item?id=33931971
| matlin wrote:
| That's cool that you saw my presentation!
|
| Re: Flutter (and other native support) it's something we've
| thought a lot about (Flutter specifically comes up often) but
| we decided to focus on pure-Typescript to start because it's a
| big enough market (I'm personally a big believer in the future
| of PWA's) and we know we can make the best experience by
| focusing on it. I do think that we'll eventually make something
| more platform agnostic but not sure when that will happen.
|
| Re: ElectricSQL & Supabase: both teams are obviously very
| talented and thoughtful and, I believe, will continue growing
| in the SQL space which is the most fundamental difference
| between our approaches. I think Triplit can make the best
| experience for developers by avoiding SQL and think there's
| undoubtedly room for both philosophies.
| sagarjs wrote:
| So it's not possible to use this with an existing postgresql
| database?
| matlin wrote:
| Not currently but we have an internal tool that does bi-
| directional syncing using Postgres's replication protocol and
| WAL2JSON. It's not quite ready for prime time but we're hoping
| to get it into people's hands soon.
| satvikpendem wrote:
| Look into ElectricSQL which works with existing Postgres, it is
| what I'm leaning towards using too.
| alex_lav wrote:
| This looks great! I would like to use this tool.
|
| I'm looking through the docs for more info on this line:
|
| > The server supports different storage adapters, such as SQLite
|
| What are the other storage adapters? I may just be blind, so if
| so please forgive me!
| matlin wrote:
| There are adapters for LevelDB, LMDB, and File storage. We just
| use SQLite out of the box because it's fast and reliable. We'll
| make those other adapters more accessible in the future.
| tanishqkanc wrote:
| I've been using Triplit on my React native app for a while and it
| works great. Highly recommend. It's the only local-first db that
| hits these points for me:
|
| - Good sane query language (not SQL) - Great typescript support -
| Offline support - React native support
|
| The cherries on top is that it's open source and self-hostable.
| matlin wrote:
| Thanks for the kind words and help identifying issues in
| Triplit as you've built your app!
| syedmsawaid wrote:
| Would this be pluggable to any backend? Like say, Ruby on Rails
| or ASP.NET?
| matlin wrote:
| Triplit uses its own standalone server (similar to other
| database servers) which you can talk to via HTTP
| https://www.triplit.dev/docs/http-api
| kvonhorn wrote:
| I used to work for a startup called TripIt 15 years ago. Took a
| double take when I saw your name.
| munzman wrote:
| would be great to have Rust bindings so it could work with Tauri.
| With tauri growth and the upcoming mobile devices support coupled
| with SQLite recent hype, this could bridge the gap and solve many
| problems for offline-first apps, thus become the go to solutions
| for many dev shops.
| matlin wrote:
| I thought Tauri was just using native web renderers? Seemingly
| Triplit should work out of the box.
___________________________________________________________________
(page generated 2024-06-25 23:00 UTC)