https://github.com/YousefED/Matrix-CRDT Skip to content Sign up * Why GitHub? + Features + Mobile + Actions + Codespaces + Packages + Security + Code review + Issues + Integrations + GitHub Sponsors + Customer stories * Team * Enterprise * Explore + Explore GitHub + Learn and contribute + Topics + Collections + Trending + Learning Lab + Open source guides + Connect with others + The ReadME Project + Events + Community forum + GitHub Education + GitHub Stars program * Marketplace * Pricing + Plans + Compare plans + Contact Sales + Education [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} YousefED / Matrix-CRDT Public * Notifications * Fork 5 * Star 258 * Use Matrix as a backend for local-first applications with the Matrix-CRDT Yjs provider. MPL-2.0 License 258 stars 5 forks Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Wiki * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Wiki * Security * Insights main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags 3 branches 1 tag Code Latest commit @YousefED YousefED update readme ... 8476b3f Jan 18, 2022 update readme 8476b3f Git stats * 22 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Node 16 (#2) Jan 4, 2022 .vscode hello world (#1) Jan 4, 2022 examples update readme Jan 18, 2022 packages/matrix-crdt add comments Jan 5, 2022 test-server hello world (#1) Jan 4, 2022 .gitignore hello world (#1) Jan 4, 2022 .npmrc hello world (#1) Jan 4, 2022 .nvmrc Node 16 (#2) Jan 4, 2022 CONTRIBUTING.md update readme Jan 10, 2022 LICENSE Initial commit Jan 2, 2022 README.md update readme Jan 18, 2022 jest.config.js hello world (#1) Jan 4, 2022 lerna.json Node 16 (#2) Jan 4, 2022 package-lock.json rich text sample Jan 10, 2022 package.json hello world (#1) Jan 4, 2022 prettier.config.js hello world (#1) Jan 4, 2022 tsconfig.build.json hello world (#1) Jan 4, 2022 tsconfig.json hello world (#1) Jan 4, 2022 View code [ ] Matrix CRDT TL;DR Live demo Motivation Usage Usage with Yjs SyncedStore API Parameters Architecture Snapshots WebRTC (experimental) Development Credits README.md Matrix CRDT Discord Matrix npm version Coverage Status Matrix-CRDT enables you to use Matrix as a backend for distributed, real-time collaborative web applications that sync automatically. The MatrixProvider is a sync provider for Yjs, a proven, high performance CRDT implementation. TL;DR Create apps like this: screencapture And connect Matrix as transport + backend storage. Instead of chat messages (primary use-case of Matrix), we send an event stream of data model updates (for the rich-text demo, these are document edits) to Matrix. Live demo In the examples directory, you'll find some live examples: * Collaborative Todo list * Collaborative rich text editing Motivation CRDTs (Conflict-free Replicated Data Types) make it easy to build decentralized, fast, collaborative local-first applications. Read more about the benefits of Local-first software in this essay When building local-first software on top of CRDTs, you probably still need a backend so users can access their data across devices and collaborate with each other. While Matrix is primarily designed for messaging (chat), it's versatile enough to use as a backend for collaborative applications (see Architecture). The idea is that by building on top of Matrix, developers can focus on building clients and get the following benefits from the Matrix ecosystem out-of-the-box: * An open standard and active community * Multiple server implementations (including hosted servers) * Authentication (including support for SSO and 3rd party providers) * Access control via Rooms and Spaces * E2EE * A decentralized architecture with support for federation Usage Matrix-CRDT currently works with Yjs or SyncedStore. Usage with Yjs To setup Matrix-CRDT, 3 steps are needed: * Create a Yjs Y.Doc * Create and authenticate a client from matrix-js-sdk * Create and initialize your Matrix-CRDT MatrixProvider import { MatrixProvider } from "matrix-crdt"; import * as Y from "yjs"; import sdk from "matrix-js-sdk"; // See https://matrix.org/docs/guides/usage-of-the-matrix-js-sdk // for login methods const matrixClient = sdk.createClient({ baseUrl: "https://matrix.org", accessToken: "....MDAxM2lkZW50aWZpZXIga2V5CjAwMTBjaWQgZ2Vu....", userId: "@USERID:matrix.org", }); // Create a new Y.Doc and connect the MatrixProvider const ydoc = new Y.Doc(); const provider = new MatrixProvider(ydoc, matrixClient, { type: "alias", alias: "matrix-room-alias", }); provider.initialize(); // array of numbers which produce a sum const yarray = ydoc.getArray("count"); // observe changes of the sum yarray.observe((event) => { // print updates when the data changes console.log("new sum: " + yarray.toArray().reduce((a, b) => a + b)); }); // add 1 to the sum yarray.push([1]); // => "new sum: 1" SyncedStore You can also use SyncedStore and use Matrix-CRDT as SyncProvider. API new MatrixProvider (doc, matrixClient, room, awareness?, opts?): MatrixProvider The MatrixProvider syncs a Matrix room with a Yjs document. Parameters Name Type Description doc Y.Doc The Y.Doc to sync over the Matrix room. A matrix-js-sdk client with matrixClient MatrixClient permissions to read (and/or write) from the room. { type: "id"; id: string; } The room ID or Alias to sync room or { type: "alias"; alias: with. string } A y-protocols Awareness awareness instance that can be used to (optional) awarenessProtocol.Awareness sync "awareness" data over the experimental webrtc bridge. opts MatrixProviderOptions (see Configure advanced (optional) below) properties, see below. MatrixProviderOptions Additional configuration options that can be passed to the MatrixProvider constructor. Defaults to: { // Options for `ThrottledMatrixWriter` writer: { // throttle flushing write events to matrix by 500ms flushInterval: number = 500, // if writing to the room fails, wait 30 seconds before retrying retryIfForbiddenInterval: number = 30000 }, // Options for `MatrixCRDTEventTranslator` translator: { // set to true to send everything encapsulated in a m.room.message, // so you can view and debug messages easily in element or other matrix clients updatesAsRegularMessages: false, // The event type to use for updates updateEventType: "matrix-crdt.doc_update", // The event type to use for snapshots snapshotEventType: "matrix-crdt.doc_snapshot", } // Experimental; we can use WebRTC to sync updates instantly over WebRTC. // See SignedWebrtcProvider.ts for more details + motivation enableExperimentalWebrtcSync: boolean = false // Options for MatrixReader reader: { // How often to send a summary snapshot (defaults to once every 30 events) snapshotInterval: number = 30, }, } Architecture CRDT updates (in our case, Yjs document updates) are very similar to (chat) Messages, that Matrix has been optimized for. Matrix-CRDT bridges Yjs documents to Matrix Rooms. and Yjs updates to Matrix events (regular chat messages are also a specific event type in Matrix). Yjs document updates are sent as base64-encoded events to the Matrix room. When registering a MatrixProvider, we: * Listen to new matrix-crdt.doc_update events in the Matrix Room, and apply updates to the Yjs document. * Listen to Yjs document updates and send these to the Matrix room as matrix-crdt.doc_update events. CRDTs are specifically designed to be eventually consistent. This means that the state of your data is eventually reconciled, regardless of the order of update events that reach each client or server (as long as you eventually get all updates). This makes it possible to work offline, or for servers / clients to be out of sync for a while. Snapshots To reconstruct your application state (that is, the Yjs document), we eventually need to access all previous events. When there have been a lot of updates, it would be inefficient to read the entire document / room history from Matrix. Matrix-CRDT sends periodic snapshots that contain a summary of all previous events. When retrieving a snapshot (stored as a Matrix event with type matrix-crdt.doc_snapshot), clients can reconstruct application state from that snapshot and don't need to fetch events occuring before that snapshots last_event_id (stored on the event). WebRTC (experimental) Matrix-CRDT by default throttles sent events every 500ms (for example, to prevent sending an event every keystroke when building a rich text editor). It also does not support Yjs Awareness updates (for presence information, etc) over Matrix. You can use the (experimental) WebRTC provider to connect to peers over WebRTC and send updates (regular and Awareness updates) instantly. Ideally, we'd replace this with Matrix Custom Ephemeral events when that Spec has landed. Development See CONTRIBUTING.md for instructions how to work with this repo (e.g.: installing and building using lerna). Credits Matrix-CRDT is built as part of TypeCell. TypeCell is proudly sponsored by the renowned NLNet foundation who are on a mission to support an open internet, and protect the privacy and security of internet users. Check them out! NLNet About Use Matrix as a backend for local-first applications with the Matrix-CRDT Yjs provider. Topics sync real-time messaging matrix crdt yjs collaborative-editing local-first Resources Readme License MPL-2.0 License Stars 258 stars Watchers 5 watching Forks 5 forks Releases 1 tags Packages 0 No packages published Used by 1 * @YousefED @YousefED / Matrix-CRDT Languages * TypeScript 99.2% * JavaScript 0.8% * (c) 2022 GitHub, Inc. * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.