[HN Gopher] SurrealDB - Document-graph database, for the realtim...
___________________________________________________________________
SurrealDB - Document-graph database, for the realtime web
Author : nerdyadventurer
Score : 168 points
Date : 2022-08-22 12:44 UTC (10 hours ago)
(HTM) web link (surrealdb.com)
(TXT) w3m dump (surrealdb.com)
| typingmonkey wrote:
| I am developing a web/JavaScript database for several years now
| [1]. What I do not understand is, how graph based databases,
| (like surrealDB or gundb) plan to have great performance when
| they run in the browser. The thing is that you are always limited
| to store your data inside of the browsers IndexedDB. So as soon
| as you do not work with the same document-based model similar to
| the IndexedDB API, you will have big troubles when it comes to
| performance. For example how can you store and query graph based
| data in IndexedDB without heavily overfetching either on read or
| on writes? IndexedDB is already way too slow.
|
| [1] https://github.com/pubkey/rxdb
| edgyquant wrote:
| Forgive my ignorance but What about the file system store? Is
| that the same thing?
| typingmonkey wrote:
| There are 2 things called "file system api" in the browser.
| One is the "File System Access API" to store files on a users
| OS. This cannot be used to store data of a database.[1] The
| other one is the "real" filesytem api, which is not supported
| by most browsers. [2]
|
| [1] https://developer.mozilla.org/en-
| US/docs/Web/API/File_System...
|
| [2] https://caniuse.com/filesystem
| tobiemh wrote:
| Hi typingmonkey. So SurrealDB abstracts away the document
| aspect of IndexedDB so it can be used as a key-value store. We
| then use that key-value store (in a similar way to RocksDB) to
| store the data in SurrealDB. You can see our abstraction layer
| here: https://github.com/surrealdb/indxdb . It's actually
| pretty performant in our tests so far, and we'll be releasing
| this soon as a WebAssembly library, enabling SurrealDB to run
| in the browser locally.
|
| I know rxdb (am a stargazer) so I'm really interested to hear
| suggestions or thoughts around this area!
| typingmonkey wrote:
| Thank you for these links, I will inspect that.
|
| As far as I know, accessing IndexedDB from WebAssembly only
| works by passing the data through the JavaScript layer
| (correct me if I am wrong). So from how I think WebAssembly
| works, it is likely not any faster then directly doing that
| in JavaScript. I mean, the storage layer does not have to do
| any heavy computations, so tunneling the data would be more
| expensive then what you get from plain js.
|
| When you store the data into an IndexedDB based key-value
| store, how does the querying work? Do you still use
| IndexedDBs secondary indexes when a query only matches a
| range of documents, or does the K-V store has any method to
| support querying a subset of the data without fetching the
| whole database state?
| tobiemh wrote:
| Yes you're right. Accessing IndexedDB from WebAssembly
| needs to go through JavaScript. The only benefit from our
| perspective is that building it in Rust enables us to use
| the same logic and code that powers our entire database
| platform. From a performance aspect, you're right, it still
| needs to go through JavaScript.
|
| So we don't use any features from IndexedDB at all really.
| We literally treat IndexedDB as a key-value store interface
| like RocksDB or something like that. So all of the indexing
| is done within SurrealDB, and is stored in IndexedDB as a
| set of binary keys->values (Uint8Array if I remember
| correctly).
|
| When we want to query a whole table, or a subset of a
| table, SurrealDB determines what individual keys to load or
| what key ranges need to be loaded to perform the query.
|
| Would love to chat further about this - it's a really
| interesting area of discussion for me!
| typingmonkey wrote:
| Ok now I understand.
|
| You can come to the RxDB discord for a chat if you want.
| I am really interessted in using WebAssembly for browser
| side storage.
|
| I look forward to the day where someone reimplements
| IndexedDB via Webassembly, with the exact same API, so
| that we have a faster IndexedDB which uses the old one as
| storage layer :)
| revskill wrote:
| Does it support http webhook on table events ?
| tobiemh wrote:
| Hi revskill, absolutely you could do something like this:
|
| DEFINE EVENT changed ON person WHEN $before.email !=
| $after.email THEN http::post('https://my-webhook.com', $after);
|
| It's very flexible! We're also releasing support for the
| fetch() function within the JavaScript functions soon, which
| will enable you to do the same thing from within the embedded
| JavaScript functions!
| tobiemh wrote:
| Thanks for posting this nerdyadventurer! I'm Tobie founder of
| SurrealDB. We've got some really big things planned for the
| database. Any feedback is really welcome!
| mijoharas wrote:
| Hey, random tangential question: Why do you have `user-select:
| none` on the page? is there a reason that you don't want people
| to be able to copy/paste any of the paragraphs? Or is there
| some other reason
|
| (I noticed as I found it jarring since I'll often select text
| while I'm reading it.)
|
| EDIT/NOTE: Also, I imagine it might somewhat hurt things
| getting spread by word of mouth. If I wanted to share this with
| someone I'd naturally copy some sections, paste them into a
| message to someone that might be interested along with a link
| to the site. This isn't possibly with your landing page.
| tobiemh wrote:
| That's a really good note/point/addition mijoharas. We'll get
| it changed ASAP!
| mijoharas wrote:
| No worries, and no rush or anything. Was really just
| curious more than anything!
| tobiemh wrote:
| Hi mijoharas, no reason to be honest! It's just what we've
| done for other sites we've built in the past - a design
| reason more than anything else!
|
| You're not the first one to make this point, so we'll get
| this removed from our site!
| packetlost wrote:
| What's the underlying storage engine? I've been doing some
| design work on a new hybrid LSM/Extendible Hashing system in
| Rust and am curious what this is using underneath.
| tobiemh wrote:
| Hi packetlost. Currently the underlying storage engines are
| memory, and distributed TiKV. For the next beta release
| (coming this week) we also have a local RocksDB
| implementation and a distributed FoundationDB storage
| implementation.
|
| In the long run, we want to implement our own key-value store
| so that we can offer some of the temporal/versioning aspects
| (which would allow you to view your data in the graph as it
| appeared at a particular point in time).
|
| For the browser, SurrealDB sits on top of a key-value store
| which is powered by IndexedDB, so that SurrealDB can run in
| the browser, using WebAssembly.
| packetlost wrote:
| My hybrid LSM/Ext. Hash system may be of interest actually.
| It's purpose built for tracking historical data similar to
| Datomic's data model, but emphasizing constant-time read
| operations for current data while keeping the read speeds
| for historical data reasonable. In short: 2 level LSM tree
| (log file, immutable page files) with a extendible hashing
| to index into the page files. The identity of each record
| is the key + tx instant (time of COMMIT, or some other
| monotonically increasing number that is unique to a
| transaction set). Optionally you can index over historical
| data by having a record per-key that just appends a new tx-
| instant + offset for better longitudinal query performance.
| the_duke wrote:
| Is your work public? I'm also working on a (loosely)
| datomic inspired database with pluggable backends, so
| this might be very interesting for me.
| tobiemh wrote:
| Hi the_duke. The base implementation of that key-value
| store (written in Golang) can be found here:
| https://github.com/surrealdb/rixxdb
|
| I don't think the implementation of the ideas in the
| thesis were added to the public project in the end.
|
| We will be working on this in due course in Rust,
| however, and the implementation will be completely open
| source!
| tobiemh wrote:
| Hi packetlost this is a really interesting topic to me. I
| wrote my MSc thesis on this exact topic -
| https://surrealdb.com/static/whitepaper.pdf ! I've learnt
| a lot since writing this thesis, and would have many
| different opinions if I wrote it today, but very
| interested in this area of research!
| packetlost wrote:
| Same! I'll read through that paper when I have time. What
| are your thoughts on how you would implement something
| like that today?
| tobiemh wrote:
| At the time I implemented the versioning using separate
| trees on top of the Radix tree. Today I would augment the
| tree with t he versioned nodes, and would use a
| Concurrent Adaptive Radix Tree for the base tree. This
| would allow high concurrency. Then at the base of the
| tree I would store a temporal link to each tree
| modification, so that you could access the changes in
| time order!
|
| Very, very interested in chatting further about this!
| tobiemh wrote:
| I would be really interested to see / find out more about
| your new hybrid LSM/Extendible Hashing system!
| danielvaughn wrote:
| Very cool, love the idea of JS/WASM extensions. In the past
| couple of years I've seen other new entries in this space -
| CockroachDB and Planetscale come to mind. What would you say
| makes SurrealDB unique, if you don't mind me asking?
| tobiemh wrote:
| Hi danielvaughn thanks so much for the comment! We love
| CockroachDB and Planetscale! With SurrealDB we intended to
| build a database where you can use the best of the relational
| database world, the document database world, and the graph
| database world. You don't have to choose up front how you
| store your data, but you can use many different concepts to
| model and analyse your data once stored.
|
| Then with the ability to connect directly form the client
| application or from web browsers, you don't have to build any
| API layer or logic layer in front of the database. Your data
| access permissions and authentication sit right inside the
| database (only if you want of course), enabling you to build
| applications quickly, directly against the database.
| [deleted]
| [deleted]
| lovasoa wrote:
| The database's feature set is very interesting !
|
| One thing to note is the license, which is incompatible with OSI
| open-source licenses :
| https://github.com/surrealdb/surrealdb/blob/main/LICENSE
| tobiemh wrote:
| Hi lovasoa thanks for your comment! We did try to stay as open
| as possible, but unfortunately the license (as you noted) is
| not an actual OSI approved license. We've got a whole page
| about why we chose this license, what the limitations are, and
| how we are contributing to open source
| (https://surrealdb.com/license).
|
| Effectively though we didn't want to limit the database usage
| in anyway except for preventing people from offering a paid-for
| hosted SurrealDB-as-a-Service. Some database companies offer a
| 'core' open source database, and then put their enterprise
| features behind an enterprise license (or close source them
| completely). We wanted to build in the open, but at the same
| time it is our intention to offer a cloud database version so
| that developers can get going really easily in due course. This
| doesn't mean that developers have to use it though :) !
|
| Let me know if you have any other questions or points!
| lovasoa wrote:
| Why wouldn't putting only some enterprise features under your
| "business" license work for you ?
|
| I don't think anyone will want to make a business out of the
| database you wrote until it gets massively popular. And it
| won't get massively popular if it's under a proprietary
| license.
| tobiemh wrote:
| Hi lovasoa, this is definitely an option, and we're open to
| suggestions in this area!
| lovasoa wrote:
| Great ! I would love to be able to use a version of
| SurrealDB, even with a more limited feature set, in my
| open-source projects.
| tobiemh wrote:
| Ok lovasoa, we will definitely give this a thought!!!
| Just out of interest (and so I can understand your
| position / usecase better), how would the current license
| prevent you using it in any open-source projects at the
| moment?
| jpambrun wrote:
| I appreciate that and fully understand, but, in many places,
| having a non standard licence pretty much guaranties it can't
| be used. I can't imagine volunteering myself to engage with
| legal to get this approved.. I don't need this is my life..
| tobiemh wrote:
| Hi jpambrun we're definitely open to comments and
| suggestions around this!
| jpambrun wrote:
| I imagine it was rhetorical, but as you probably imagined
| I have no suggestions. I understand your needs and
| imperatives, but they are unfortunately incompatible with
| mine. Getting a green light on using custom licence is
| near impossible. Procurement and contracting for database
| as service is also a nightmare.
|
| I think mongo and elastic could get away with it because
| they already had captive users. For illustration, I
| didn't really look at your offering, I went to the
| licencing section and just summarily disqualified. I am
| also unlikely to use it personal pet projects, because
| the learnings cannot be leverage in my paying job.
| tobiemh wrote:
| No not rhetorical - was genuinely interested in any
| suggestions. Out of interest, if the product was split
| (like other database products) into a 'core' open-source
| version, and an 'enterprise' limited license version,
| would this make any difference?
| jpambrun wrote:
| Depends what is behind the custom license and how well
| it's communicated. If it's only the db as service
| management magic I wouldn't mind. Anything more and I
| would worry I would miss out on upcoming features few
| years down the line.
| satvikchoudhary wrote:
| How does it compare to ArangoDB which is also a popular open
| source document-graph database? What features does it add/remove?
| tobiemh wrote:
| Hi satvikchoudhary! It's a good question. Obviously I know of
| ArangoDB, but I don't know it well enough to comment on the
| main differences, so I'll focus on what SurrealDB is aiming to
| achieve instead.
|
| SurrealDB is aiming to be at the intersection of relational,
| document, and graph databases, whilst still remaining simple to
| use with an SQL-like query language, for developers coming from
| the relational database side. We are only at the beginning of
| the journey, but SurrealDB is designed to be run embedded, or
| in the cloud, with the ability to query it directly from a
| client application or from a web browser (and only access the
| data that you're allowed to see).
|
| With our native client libraries (coming soon), SurrealDB will
| be able to be embedded within Node.js, WebAssembly, Python, C,
| and PHP applications, in addition to running as a server.
|
| We wanted to create a database that people didn't have to
| manage, so they could focus on building applications, not the
| infrastructure. We wanted users to be able to use schema-less
| and schema-full data patterns effortlessly, a database to
| operate like a relational database (without the JOINs), but
| with the same functionality as the best document and graph
| databases. And with security and access permissions to be
| handled right within the database itself. We wanted users to be
| able to build modern real-time applications effortlessly -
| right from Chrome, Edge, or Safari. No more complicated
| backends.
|
| I'm not sure how all of this compares to ArangoDB, but happy to
| learn!
| smarx007 wrote:
| What kind of interface and format is used to work with graphs?
| Many graph queries (your query language looks inspired by Cypher
| to me) require returning subgraphs, which need to be represented
| as a graph and not a tree (JSON, XML, etc.). Do you support
| TinkerPop-like APIs or RDF formats? Or do you use any
| alternatives (JSON is not such an alternative, of course)?
| geenat wrote:
| Cool. Any insight to how Surreal DB compares to Cockroach DB?
| tobiemh wrote:
| Hi geenat, CockroachDB is an awesome database! SurrealDB and
| CockroachDB are quite different, in that CockroachDB is
| effectively a highly-available distributed PostgreSQL database
| (aiming at that similar functionality), while SurrealDB is
| aiming to merge the relational / nosql / document / and graph
| databases together. Hence the functionality is quite different.
| To make SurrealDB distributed we have built it on top of TiKV
| (on a side note TiDB which is built by the creators of TiKV is
| more of a competitor to CockroachDB than SurrealDB is).
|
| We will definitely in due course provide some documentation and
| comparisons with many of the other databases out there, and
| also we intend to do benchmarking with the other databases as
| soon as we have implemented a few of the performance issues
| that we know about
| (https://github.com/surrealdb/surrealdb/labels/performance)!
| hiyer wrote:
| This is really cool. So do you have your own storage engine for
| the single node case, and you're using TiKV for the distributed
| one? Or is the local storage case also built on an existing
| engine like RocksDB?
| tobiemh wrote:
| Hi hiyer! We've got a single-node storage implementation using
| RocksDB coming this week. With regards to running in
| distributed mode, TiKV (and soon FoundationDB) will be able to
| be used for storage.
|
| In the long run we would like to build our own single-node key-
| value store (based on a Concurrent Adaptive Radix tree), but
| this is a little way off still!
| mackeyja92 wrote:
| Looks very interesting. Kind of reminds me of RethinkDB with its
| real-time features. Was any inspiration from them?
| tobiemh wrote:
| Absolutely mackeyja92. We were users of RethinkDB back in the
| day. We wanted to bring the best features/functionality of
| NoSQL databases (MongoDB/RethinkDB/CouchDB/...), graph
| databases (Neo4j/OrientDB/...), and relational databases
| (MySQL/PostgreSQL/...) together into one platform, and then add
| some awesome functionality on top of it!
|
| In addition we really wanted the realtime functionality to be
| available when connecting directly from the browser (like
| Firebase), and that's something that RethinkDB didn't offer.
| purplerabbit wrote:
| Realtime is undoubtedly the dream... What restrictions or
| limitations are present for live queries?
| Diggsey wrote:
| The features are very impressive, but why should I trust
| SurrealDB with my data over say, PostgreSQL?
|
| Some things I'd like to know more about:
|
| - Testing strategy - Point-in-time restore? -
| Consistency/isolation model - Performance data - Migrations
| petepete wrote:
| For a start, it's blazingly fast!
| phailhaus wrote:
| So is dumping to /dev/null. A trustworthy database needs a
| lot more than being fast, unfortunately.
| petepete wrote:
| Don't make me start using sarcasm tags. Sigh.
| edgyquant wrote:
| Yep especially considering the motto is always databases
| are slow. Of course they shouldn't be unusable but the
| database isn't meant to be your clients data model
| tobiemh wrote:
| Hi phailhaus, good point. We don't want to go for speed
| (for instance by writing to /dev/null) at any cost, but
| want SurrealDB to be a reliable and performant backend for
| any application in the long run. Obviously in our opinion,
| writing to /dev/null wouldn't make SurrealDB a proper
| database :) . We have a way to go to catch up with
| databases like PostgreSQL and other databases in terms of
| stability and performance, but we will strive to get there!
| Zealotux wrote:
| But is it web scale?
| tobiemh wrote:
| Lol. Unfortunately only /dev/null is truly web scale :) !
| tobiemh wrote:
| Hi Diggsey, great question. We are currently focussed on
| functionality and stability, and then will draw our attention
| to performance. Coming this week we have a RocksDB storage
| implementation. We've only just launched our initial beta
| version, and we know there is a lot of improvement and work to
| be done (some of these performance issues we know about already
| and are on our Github issues list).
|
| With regards to the consistency/isolation model, SurrealDB sits
| on top of a number of key-value stores. By using the
| distributed highly-available TiKV storage backend,
| https://tikv.org, (and we have a FoundationDB integration in
| the works), the database is designed to be highly-scalable and
| highly-available. The same guarantees (albeit just single-node,
| so no high-availability or scalability) will be available with
| the RocksDB implementation coming this week. By sitting on top
| of these key-value stores, SurrealDB ensures that all
| transactions are ACID compliant. We don't want to go for speed
| (for instance by writing to /dev/null) over anything, but want
| SurrealDB to be a reliable and performant backend for any
| application. Obviously we have a way to go to catch up with
| PostgreSQL (launched in 1996), but we will strive to get there!
| PeterZaitsev wrote:
| Publish Code on GitHub proclaim your love to Open Source... but
| actually use Proprietary License...
| tobiemh wrote:
| Hi PeterZaitsev, many of our core components that are used
| within the database are open source under the MIT / Apache 2.0
| license. With regards to the core database code, it is free to
| use with no limits on number of nodes, data size, number of
| connections, number of connected users, features, or
| functionality. The only limit is for providing a paid-for
| hosted database-as-a-service. In addition, after 4 years, our
| complete source code for that version release becomes Apache
| 2.0 licensed. We've tried to keep this as open as possible for
| developers / businesses / enterprises to use, whilst at the
| same time protecting ourselves from some of the events that
| have happened in the database world over the last few years.
| It's definitely not our intention to just publish our code on
| Github just to pretend to be open source!
| lovasoa wrote:
| On the other hand, you are able to use all of the libraries
| that you use only because they didn't make the same choice as
| you, and chose truly open-source licenses...
|
| In particular, the scaling abilities of your product seem to
| be based on the capabilities of TiKV, which itself is based
| on RocksDB. These are both databases in and of themselves,
| and if any of these two had adopted the same license as you,
| then you wouldn't have been able to release the product you
| are releasing...
| AtNightWeCode wrote:
| Many warning signs with this. From ridiculous claims to license.
| I'm out.
| tobiemh wrote:
| Hi AtNightWeCode, we're just a team of 2 developers at the
| moment, and completely bootstrapped. This is only our initial
| release, and we have a long journey ahead of us. We realise
| there is lots of work to do. It would be great to get an
| understanding of what you mean in more depth :) ! With regards
| to scalability and high-availability, in distributed mode you
| run SurrealDB on top of TiKV (and soon FoundationDB), so we
| don't claim to provide this functionality ourselves (yet).
|
| With regards to the license we wanted SurrealDB to basically be
| open source, but with the only limitation of not being able to
| provide a Database as a Service platform. So in a business or
| enterprise use, there is no limit at all. You can run SurrealDB
| with as many nodes as you want, and as many users as you want;
| you can provide a hosted database internally, or to employees,
| contractors, or subsidiary companies. The only limitation is
| providing a paid-for, hosted, SurrealDB database platform.
|
| After 4 years, all of our code becomes licensed with Apache 2.0
| license.
|
| In addition, all of our libraries, client SDKs, and many of the
| core components used in the database are completely Apache 2.0
| or MIT licensed (https://surrealdb.com/opensource).
| AtNightWeCode wrote:
| The license is not really the big issue.
| yewenjie wrote:
| Somehow I don't see people getting as excited with new
| programming language releases compared to database releases.
| Maybe that is because databases require less effort to integrate
| in your stack?
| felipelalli wrote:
| It's because there are a lot of good languages but no one
| single great "surreal" database. (I'm not saying the SurrralDB
| is "the one").
| yewenjie wrote:
| As you said, there are a lot of good languages, but not a
| single great one. Similarly there are multiple good
| databases, but not a single great one.
| endisneigh wrote:
| Benchmarks and comparisons to other popular databases? Landing
| says no complicated new languages but introduces SurrealQL?
|
| Would be nice to see a Jepsen test
| tobiemh wrote:
| Hi endisneigh. We are currently focussed on functionality and
| stability. There are a number of changes to our code that we
| know about
| (https://github.com/surrealdb/surrealdb/labels/performance),
| that we want to make really soon, and then as soon as those are
| complete, we'll run some benchmarks against other databases,
| and detail some comparisons.
|
| It is however hard to perform these direct comparisons, as the
| functionality is pretty different to a lot of the database out
| there, but nevertheless it definitely is something we want to
| get to in due course!
| hajhatten wrote:
| weddpros wrote:
| I'm glad to see a team with a new-ish approach in the DB world...
| I like the object-like querying! and the JS embedding.
|
| It really looks to me like SurrealDB could become big in the
| future, ambitious feature set.
|
| DBs take a lot of time to mature, so I want to congratulate you
| and wish you the best
| jaimemh wrote:
| Thank you very much weddpros! My brother and I have been
| building it and building apps on top of it for 7 years now. We
| launched SurrealDB in public open beta 4 weeks ago. Just the
| two of us at the moment! We have some really big things planned
| for SurrealDB.
| tobiemh wrote:
| Thanks so much weddpros! We definitely have some ambitious
| plans for SurrealDB, and we have lots of exciting stuff to
| come!
| CSDude wrote:
| Product looks great. I totally feel the need for new open-source-
| but-not-for-AWS-to-steal-my-product licenses; however I'm not
| sure about that vague license for database would work for you.
| A "Database Service" is a commercial offering that
| allows third parties (other than your employees and
| contractors) to access the functionality of the
| Licensed Work by creating tables whose schemas are
| controlled by such third parties.
|
| So, if as a service provider, I control the schemas can the
| customers query the data using your SQL? Can I just collect data,
| put them to database and proxy their SQL to DB? i.e I'm the co-
| founder of Resmo (https://www.resmo.com) where we let customers
| run SQL on their cloud/saas assets configuration, can I use
| yours? The current wording seems to let me because we don't let
| any customer to define their schema, resources schemas are
| defined by us, for now.
| gpm wrote:
| It's not obvious to me that they're trying to stop you. This
| reads as "AWS/GCP/Azure don't steal our work" while otherwise
| trying to be as wide a license grant as possible.
|
| I'm sure they'd love to sell you support services, but many of
| these database companies do that by convincing you there is
| additional value, instead of trying to use licenses. I think
| because licenses create a scary risk that the db company might
| increase prices or even stop offering them altogether and you
| have no real recourse.
| jaimemh wrote:
| Thank you CSDude. We wanted SurrealDB to basically be open
| source, but with the only limitation of not being able to
| provide a Database as a Service platform. So in a business or
| enterprise use, there is no limit at all. You can run SurrealDB
| with as many nodes as you want, and as many users as you want;
| you can provide a hosted database internally, or to employees,
| contractors, or subsidiary companies. The only limitation is
| providing a paid-for, hosted, SurrealDB database platform.
|
| After 4 years, all of our code becomes licensed with Apache 2.0
| license.
|
| In addition, all of our libraries, client SDKs, and many of our
| core components are completely Apache 2.0 or MIT licensed
| (https://surrealdb.com/opensource).
|
| We have a whole page describing the license here:
| https://surrealdb.com/license!
|
| We would definitely like to chat if you are interested in using
| SurrealDB!
| nerdyadventurer wrote:
| Source : https://github.com/surrealdb/surrealdb
|
| Features : https://surrealdb.com/features
| BankHottas wrote:
| I really like that the features page is so transparent about
| what's completed/in progress/planned
| jaimemh wrote:
| Thank you very much BankHottas! My brother and I are building
| it in the open and want to be as transparent as possible. We
| have some really big things planned for SurrealDB that aren't
| on our features page, but which we will be adding very soon!
| mackeyja92 wrote:
| I wonder how this would pair with Elixir & LiveView. My initial
| thought is that it would work together quite nicely.
| tobiemh wrote:
| Hi mackeyja92 great suggestion! We're not big Erlang/Elixir
| users, but we definitely want to integrate with Elixir and
| LiveView in due course! We actually received a suggestion to do
| this just yesterday!
|
| We would love to hear any suggestions or ideas in this area.
| Let us know in whatever way you want
| (https://surrealdb.com/community)!
| erikpukinskis wrote:
| Very cool... live queries are a game changer!
|
| What's the testing story? Do I need to connect to your servers in
| order to do e2e testing? Any plans to offer a mock server or
| anything like that?
| tobiemh wrote:
| hi erikpukinskis, obviously SurrealDB can be run on your local
| development machine, or in your own cloud, so you would never
| have to connect to our servers for testing. In addition, we are
| releasing a number of client libraries which (in addition to
| connecting to a local or remote database server), will also
| allow you to run SurrealDB embedded i the language of your
| choice.
|
| Coming soon are Nodejs native module (with embedded database
| functionality), Python (with embedded database functionality),
| WebAssembly (so you can run SurrealDB completely in your web
| browser), and C (with embedded database functionality).
|
| In due course, we want to have native integrations for PHP,
| Java, and Swift too (and then more to come down the line).
|
| This will mean that you could run tests without having to
| connect to any external service/platform/server!
|
| Let me know if you have any other questions!
| yawnxyz wrote:
| Hi tobiemh, will there be any plans to add native support for
| Svelte / Sveltekit?
| tobiemh wrote:
| Hi yawnxyz - absolutely we intend to have support for Svelte in
| due course! We aren't Svelte users ourselves, so any
| suggestions or ideas are very welcome! Do let us know in the
| community - https://surrealdb.com/community
| [deleted]
| catchmeifyoucan wrote:
| This looks pretty awesome! My biggest question is support for
| mobile. I wonder if there's any planned integration with React
| Native?
| jaimemh wrote:
| Hi catchmeifyoucan. We definitely plan to support React Native
| in due course. We are also happy to take any
| suggestions/contributions/ideas in this area to make it a
| really great integration.
| jxi wrote:
| RayVR wrote:
| perhaps I'm daft, but why should I care that you've written it in
| Rust? The number of times Rust is mentioned on the landing page
| feels...odd.
|
| I love shiny new things. However, without some sense of how this
| compares to alternatives, it's difficult to see why I should dive
| down the surrealdb rabbit hole. Feature lists are ok, but what I
| really want is context. Why should I care about what surrealdb
| can do?
| bluejekyll wrote:
| Did the link change? I just read through that landing page and
| didn't really notice Rust mentioned. It's mention a bit on the
| GitHub readme, but that would be expected as getting started
| with the code?
|
| Also, people should care about the language something is
| written in, especially in OSS, as that implies a number of
| things. Specifically, what the ecosystem of that language is
| like in terms of maturity, how easy it is for you as a
| developer to get started with the project, what the build tool
| chain is and will work in your environment.
|
| I know I disqualify projects that I will look at purely based
| on the language and toolchain it uses since I know it will be
| harder to integrate into my companies workflows.
| tobiemh wrote:
| Hi RayVR, being built in Rust definitely isn't a main selling
| point, but it has made building SurrealDB easier. We initially
| built our prototype in Golang, but decided to re-write it in
| Rust so that we could improve on certain aspects that were hard
| to do with Golang.
|
| That aside, we wanted to create a database that people didn't
| have to manage, so they could focus on building applications,
| not the infrastructure. We wanted users to be able to use
| schema-less and schema-full data patterns effortlessly, a
| database to operate like a relational database (without the
| JOINs), but with the same functionality as the best document
| and graph databases. And with security and access permissions
| to be handled right within the database itself. We wanted users
| to be able to build modern real-time applications effortlessly
| - right from Chrome, Edge, or Safari. No more complicated
| backends.
|
| It's only the beginning of our journey (we're just a
| bootstrapped team of 2 at the moment), and we welcome all
| feedback!
| arunc wrote:
| > certain aspects that were hard to do with Golang.
|
| Do you mind sharing those aspects?
| tobiemh wrote:
| Absolutely arunc!
|
| You can actually see the Golang version - it's on a
| separate branch in the main repository.
|
| We had written our query parser from scratch (which would
| read character by character). This worked well, up to a
| point, but was hard to extend. Building in Rust we've used
| the nom crate for parsing, and it has enabled us to do some
| pretty clever and advanced stuff with the SurrealQL query
| language (futures, embedded JavaScript functions, while at
| the same time remaining performant.
|
| Other things we wrote from scratch (at the time), were our
| own binary serialisation framework (similar to MsgPack, but
| with slight differences and performance improvements). We
| then had to write our own serialisation logic on top of
| this, so that each data type would serialise and
| deserialise efficiently. With Rust, using the Serde crate,
| this is done completely differently, and allows us to have
| the same performance (or better), but with no custom
| writing of data tags!
|
| In addition, many other improvements, especially around
| generics mainly. It made building SurrealDB much quicker,
| but it also means that all data passed around throughout
| the system is able to be reasoned about easier. Instead of
| brute-forcing with the race checker, you really have to
| have a concrete idea of how and where your data will be
| used. This starts out harder (fighting against the borrow
| checker), but in the long run it makes writing safe and
| performant software so much more pleasant.
|
| I'll try and go into this in detail in one of our future
| blog posts!
| arunc wrote:
| Fantastic, thank you!!
| debuggerpk wrote:
| Would love to hear about the pain points you had with go.
| tobiemh wrote:
| Absolutely debuggerpk. My answer to arunc above goes into
| it in slight detail! We will definitely plan to do an in-
| depth look at the decision to transfer from Golang to Rust
| in a future blog post!
| zinxq wrote:
| In the context of "History doesn't repeat, but it rhymes", this
| is just what happened when Java appeared. Everyone noting it
| was written in Java (as, believe it or not, it was the cool new
| language).
|
| It might have actually been worse, because there was a strong
| penchant for naming products starting with a "J" to indicate
| the fact (i.e. JNotepad, JDatabase, etc).
|
| It might actually be a good marketing technique to get other
| Rust aficionados to try your product. But otherwise, there
| isn't any real value now that "write once, run anywhere"
| doesn't just belong to Java anymore.
| arunc wrote:
| Even Javascript rode on the Java hype/marketing albeit having
| no correlation with it.
| gpm wrote:
| No real connection, but amusingly I think javascript ended
| up delivering on java's "run anywhere" pitch at least as
| well as java did in the end.
|
| I can run javascript in a query to this DB apparently for
| instance, I can't do that with java.
| dmytrish wrote:
| From the operations point of view: Rust code is usually fast,
| safe, free from data races, easy on RAM, possible to optimize
| very well, compiles to native binaries (easy to deploy). Unlike
| C, it's easy to test and provides good abstractions, so the
| code generally will be less buggy.
|
| From the point of view of a developer: when it comes to
| extending the database, writing an extension in Rust might be a
| pleasant and welcome experience. Rust interfaces with
| C/Python/Javascript easily and compiles to WASM, so the code
| written in it can be easily reused on the front end.
|
| What's not to like?
| tobiemh wrote:
| Hi dmytrish that's exactly it! We are really close to
| releasing our WebAssembly driver (built in Rust), a native
| Python driver which can connect to and embed SurrealDB (built
| in Rust), and a native Node.js driver which has the same
| functionality as the Python driver (built in Rust). The plan
| is to do the same for the C driver in due course too.
|
| The simplicity of what you described means that we can
| release a common set of libraries with similar functionality,
| but using a single common library.
| gpm wrote:
| It mentions rust 3 times on a very very long landing page, and
| once is in a list of languages it has bindings for. That's
| really not "the number of times" your comment implies.
|
| It is worth mentioning in my opinion mostly because that you
| should care that it isn't written in
|
| - C/C++, because of security concerns
|
| - Java/Scala/<JVM family> because embedding these languages
| (and this is available as an embedded database) is a pain, also
| because of common (though not inevitable) memory bloat.
|
| - Python/Ruby/..., because of common performance issues (DBs
| being performance sensitive software)
|
| Obviously there is perfectly good software written in all those
| languages, and terrible software written in rust. The the odds
| of it being good however are substantially increased when you
| find out it's written in rust because for this domain, out of
| the common programming languages, rust really does have the
| best set of tradeoffs. A landing pages goal is to give you as
| many signals that "this software might be worthwhile" as
| possible, and being written in Rust (or really not being
| written in pretty much every other common programming language)
| signals that right now.
| dataflow wrote:
| > It mentions rust 3 times on a very very long landing page,
| and once is in a list of languages it has bindings for.
| That's really not "the number of times" your comment implies.
|
| I count 7 times. Once with the cargo install, once "built
| entirely in Rust", once "built with Rust", three times quoted
| under "Loved by Developers", and once under "Database
| Libraries and Clients".
| dalegs wrote:
| jxi wrote:
| Will you add support for connecting from IntelliJ/DataGrip?
| tobiemh wrote:
| Hi jxi thanks for the question! Our plan is to make SurreealDB
| accessible and usable from as many
| codebases/platforms/architectures as possible.
|
| We've got plans for integration into all of the big IDEs down
| the line, so we absolutely will add IntelliJ and DataGrip to
| the list.
|
| We'll definitely want to get as much feedback in this area when
| we get to it, so do join the community
| (https://surrealdb.com/community) as we love all ideas and
| suggestions!
| gardenfelder wrote:
| Very interesting license structure:
| https://github.com/surrealdb/surrealdb/blob/main/LICENSE
| tobiemh wrote:
| Hi gardenfelder, we've got a whole page on our site explaining
| the thought process and our intentions around the license -
| https://surrealdb.com/license !
| carvking wrote:
| It would be helpful if people would include [WAITLIST] in the
| title.
|
| Too many waitlist things going on - I can understand the need for
| it, but I'm more inclined to think "Scarcity marketing" in most
| cases.
|
| Of course if some new products couldn't deal with an influx of
| new users in a timely fashion, better to have a waitlist.
|
| But for the first N people it should be auto-accept.
| tobiemh wrote:
| Hi carvking! The database can be used completely without our
| own cloud hosted product (for which there is a waitlist on the
| site). We're still a little bit away from offering our cloud
| hosted product, but the product itself can be downloaded,
| tried, hosted, and hacked on without waiting for our hosted
| cloud version.
|
| We're definitely not into the 'signup to our waitlist' product,
| just for marketing purposes. It's only there as some users
| expressed an interest in registering for it, as they didn't
| want to run SurrealDB themselves. Just to reiterate, all of our
| hosted cloud functionality is and will be available in the
| product which anyone can download and use for free :) !
| carvking wrote:
| Ah - fair enough, my built in autocomplete needs some fine
| tuning.
|
| I immediately saw "join waitlist" and came to complain.
|
| My bad - be well :)
| tobiemh wrote:
| Haha no worries carvking! Thank you!
| techn00 wrote:
| I've been seeing a lot of projects lately that use Rust for
| database-related work. How can someone get started writing a
| database these days?
| apavlo wrote:
| > How can someone get started writing a database these days?
|
| Take my class. It starts next week:
| https://15445.courses.cs.cmu.edu/fall2022/
| iamlucaswolf wrote:
| 15-445/15-721 got me a job. Thank you so much for making them
| publicly available.
| tobiemh wrote:
| Hi techn00! It's definitely taken a while for us to get to this
| point. We initially starting planning to build SurrealDB back
| in 2016, and started building on top of our initial database
| prototype (built in Golang) back in 2018. Over that time we've
| made many changes, both to the system design, and to the code,
| and architecture. it's just the beginning of our journey, and
| we've got some really stuff planned.
___________________________________________________________________
(page generated 2022-08-22 23:02 UTC)