[HN Gopher] How rqlite is tested
___________________________________________________________________
How rqlite is tested
Author : otoolep
Score : 239 points
Date : 2025-01-14 20:21 UTC (1 days ago)
(HTM) web link (philipotoole.com)
(TXT) w3m dump (philipotoole.com)
| Ericson2314 wrote:
| Now that there is testing like
| https://turso.tech/blog/introducing-limbo-a-complete-rewrite...
| going on, tbh the testing described in the linked piece doesn't
| seem so good.
| dangoodmanUT wrote:
| Probably a lot better than 99.999% of software? This is made by
| one person, with code that cannot be deterministically tested
| (external SQLite) But you seem to be cargo culting, as this
| testing has existed in DBs for nearly a decade
| otoolep wrote:
| rqlite has been in development for about a decade too!
|
| https://github.com/rqlite/rqlite/blob/master/CHANGELOG.md#10.
| ..
| dangoodmanUT wrote:
| kudos do you even just for sticking with it for that long,
| but I know it's well loved
| IshKebab wrote:
| It's not cargo culting. Deterministic Simulation Testing has
| been the standard for silicon verification practically
| forever (though it isn't called that because nobody does
| anything else).
|
| Applying it to software is a logical step if you want similar
| levels of quality assurance.
|
| Do you _need_ that level of assurance for a database?
| Probably not in most cases. But I think it 's a reasonable
| thing to mention on a post that is touting database
| reliability, especially because there are real databases that
| do use it.
| ncruces wrote:
| Then maybe point of one of those databases:
| https://foundationdb.org/ https://tigerbeetle.com/
|
| Or if you really wanna keep the SQLite theme:
| https://github.com/losfair/mvsqlite
|
| Why promote the vaporware project when comparing with
| something that's been in development for 10 years?
| reitzensteinm wrote:
| The parent comment was not written by the same author.
| ncruces wrote:
| Politely asking about DST like
| https://news.ycombinator.com/item?id=42707760 is
| perfectly reasonable.
|
| The cargo culting bit is using vaporware that was just in
| the news cycle to shout down good work that has been done
| for decades.
| IshKebab wrote:
| > Why promote the vaporware project
|
| Probably because it's been in the news very recently and
| most people learned of DST in software from it.
| abrookewood wrote:
| FYI this is a reference to Deterministic Simulation Testing
| (DST), which certainly sounds interesting:
| https://apple.github.io/foundationdb/testing.html
| someguy101010 wrote:
| I also enjoyed this in video format
| https://youtu.be/JLlIAWjvHxM?feature=shared&t=2049
|
| Always have been envious of that performance testing setup that
| is shown here
| fegu wrote:
| There seems to be some copy pasta in the FAQ on if any node can
| be contacted for writes or reads, the paragraph on reads mentions
| writes.
| otoolep wrote:
| Thanks for flagging -- fixed.
| atoav wrote:
| My feedback regarding the presentation is that I think there
| should be slightly more focus on _why_ one would choose
| rqlite over say sqlite. That probably means more info on the
| distributed part.
|
| I happen to know raft and the kind of problem it solves, but
| the average reader might not. A practical demonstration of
| the problems it solves might be in order.
|
| So that also means describing for whicj applications rqlite
| is more wellsuited (and for which it might be worse) than
| other databases.
| otoolep wrote:
| I actually touched on this topic somewhat in a recent talk
| at GopherCon: https://youtu.be/8XbxQ1Epi5w?t=2223
|
| The section of the talk at the timestamp above is titled
| "Should I use Raft"?
| didip wrote:
| I love how dedicated you are with this project, Philip. Been
| watching it for many many years.
| denysvitali wrote:
| In case you missed it, OP is also the author of rqlite
|
| (I for one didn't notice at first)
| otoolep wrote:
| Yes, that is right. So here's my disclaimer: I'm the author of
| rqlite.
|
| I really admire the testing that the SQLite team does[1], and
| the way it allows them to stand behind the statements they make
| about quality. It's inspiring.
|
| IMO there have only been two really big improvements to
| software development relative to when I started programming
| professionally 25 years ago: 1) code reviews becoming
| mainstream, and 2) unit testing. (Perhaps Gen AI will be the
| third). I believe extensive testing is the only reason that
| rqlite continues to be developed to this day. It's not just
| that it helps keep the quality high, it's a key design guide.
| If a new module cannot be unit tested during development, in a
| straightforward manner, it's a strong sign one's decomposition
| of the problem is wrong.
|
| [1] https://www.sqlite.org/testing.html
| cynicalsecurity wrote:
| 1. Is this a one man project? Why? What if the author dies?
|
| 2. Why Go? Go is garbage collected, how is this even a good idea
| for a database engine in the first place?
| ChocolateGod wrote:
| > Why Go? Go is garbage collected, how is this even a good idea
| for a database engine in the first place?
|
| First, the database itself is Sqlite, which is C and arguably
| the most widely deployed database engine in the galaxy.
|
| Secondly, Go is popular for network services as it's easy to
| write (relative to languages like C++, Rust etc), is memory-
| safe and is "fast enough" for the network/disk to be your
| bottleneck and its concurrency features make it easy to use
| hardware efficiently without having to worry about scheduling
| in your application, among other reasons.
| anilgulecha wrote:
| > the most widely deployed database engine in the galaxy.
|
| The planet perhaps. We don't really know at the galaxy level.
| At best we're atleast 200k years away from knowing for sure
| :)
|
| /nitpick
| shakna wrote:
| NASA has used sqlite on a number of systems, so the reach
| is certainly beyond just the planet.
| ChocolateGod wrote:
| known galaxy then!
| aabhay wrote:
| Making good on the username, for sure!
| dangoodmanUT wrote:
| This is HN outrage bait if I've ever seen it
| the-alchemist wrote:
| can't wait for the https://jepsen.io/ report!
| otoolep wrote:
| It's not done by Aphyr himself, but someone ran Jepsen-like
| testing a couple of years back.
|
| https://github.com/wildarch/jepsen.rqlite/blob/main/doc/blog...
| qaq wrote:
| Anyone using rqlite in prod ?
| otoolep wrote:
| Yes, Replicated for example:
| https://www.replicated.com/blog/app-manager-with-rqlite
|
| It's still in active use by them. https://www.textgroove.com/
| is another company that uses it, though I know less about their
| use case.
| qaq wrote:
| Cool, rqlite looks like a really nice option for c2 storage
| layer
| joostdecock wrote:
| Yes, it's our backing database for Morio [1], an open source
| observability plumbing tool.
|
| We're still building it but we're dogfooding it and run it in
| production ourselves.
|
| Can't recommend it enough
|
| [1] https://morio.it/
| computerfan494 wrote:
| I have been using it in production for more than a year. There
| were some early hiccups associated with my use-case that were
| resolved on upgrade and with a bit of tuning. Otherwise it has
| been very stable. I'll say that I wish the official language
| bindings were tested more like rqlited itself. They've been
| more problematic than the database.
| qaq wrote:
| Thank you gonna give a spin as c2 storage layer
| kopirgan wrote:
| Thanks for the post...did not know about this.
|
| Just went through the website and read through some documents. It
| is quite easy to read and understand.
|
| One part I couldn't follow was security - other than items listed
| (file system, HTTPS, IP based filter), is it correct to say that
| if you know & have access to the URL API endpoints, any query can
| be run directly off the db with curl or such tools? How is this
| aspect managed in production? Sorry if this question is
| inappropriate or too dumb.
| otoolep wrote:
| rqlite supports a few levels security, which you can use
| separately or together.
|
| You can enable BasicAuth on the HTTP endpoints. You can also
| require that any client presenting BasicAuth credentials has
| the right permission for the operation the client is trying to
| perform. Finally you can also enable Mutual TLS, requiring that
| any client that connects must first present a cert that is
| signed by an acceptable CA.
|
| For full details on security check out
| https://rqlite.io/docs/guides/security/
| Hixon10 wrote:
| What do you think about deterministic simulation testing, which
| is currently trending?
| aguynamedben wrote:
| rqlite is my favorite "out there" project that is actually
| freaking genius!
| scottyeager wrote:
| Have been following rqlite for a while and just started using it
| in one project. So far it delivers on the promise of simplicity.
| Thanks!
| codethief wrote:
| Could you elaborate on what kind of project you're using rqlite
| for?
| t43562 wrote:
| The testing Pyramid makes sense. The problem for (perhaps) a lot
| of us will be that we're working on things where some or even all
| of the levels are missing and we have to try to bring them to a
| sensible state as fast as possible....but we have limited ability
| to do so. It's managing imperfection.
|
| We're also possibly working with multiple teams on products that
| interact and it ends up being "nobody's job" to fill in the e2e
| layer for example.
|
| Then when someone bites the bullet to get on with it....the whole
| thing isn't designed to be tested. e.g. how does anyone do
| testing with Auth0 as their auth mechanism? How do you even get a
| token to run an E2E type test? I have to screen scrape it which
| is awful.
|
| Without those E2E tests - even just the test that you can login -
| the system can break and even when it's a test environment that
| makes the environment useless for other developers and gets in
| everyone's way. It becomes the victim's job to debug what change
| broke login and push the perpetrator to fix it. With automated
| e2e tests the deployment that broke something is easy to see and
| rollback before it does any damage.
|
| I suppose I'm challenging the focus in a sense - I care about e2e
| more because some of those issues block teams from working. If
| you can't work because of some stupid e2e failure, you can't get
| fixes out for issues you found in the unit/integration tests.
| hitchstory wrote:
| The pyramid makes sense for certain types of application - very
| logic heavy with light integration touch points with everything
| cleanly dependency injected. rqlite fits this pattern.
|
| If you're building a logic _lite_ application which has a _lot_
| of integration touch points (I find most commercial code
| actually fits this pattern) then it makes zero sense - an
| integration test heavy test suite is what you want - an upside
| down pyramid if you will.
|
| If you have a ball of mud on your hands then it doesnt matter
| what kind of app it is, E2E tests are the _only_ thing that
| makes sense and you need to build _very_ sophisticated fakes
| (requiring a lot of engineering skill) to avoid things like
| flakiness bugs and the token scraping problem you referred to.
|
| If you're writing a parser, you probably want 100% unit tests
| and property tests for all those millions of parser edge cases.
| No pyramid of any shape is required, just a slab.
|
| The testing pyramid reminds me of microservices: the people who
| came up with Their Grand Idea had no clue what it was about
| their _specific_ circumstances that made their approach work
| for them but still managed to market it as The Way.
|
| Realistically, the ultimate shape of your test suite should be
| an emergent property based upon a series of smart, local
| decisions - ideally TDD'ed tests which match the kind of code
| you are writing right now. Making a particular shape a goal is
| asinine.
| dpc_01234 wrote:
| It's very common mistake in SWE to generalize and extrapolate
| from own situation.
|
| The way things to do things is very dependent on the project.
| They way one maintain, debug, test e.g. a math library vs
| embedded sw vs a video game vs a database vs a enterprise app
| vs a service in a distributed system are all just different.
|
| Almost all advice should be qualified by "in this domain, one
| this type of project ...".
| andrewaylett wrote:
| A couple of extra points I've found useful:
|
| The first test of a class of tests is the hardest, but it's
| almost always worth adding. Second and subsequent tests are much
| easier, especially when using:
|
| Parametrised tests let you test more things without copying
| boilerplate, but don't throw more variants _just_ to get the
| count up. Having said that:
|
| Exhaustive validation of constraints, when it's plausible. We
| have ~100k tests of our translations for one project, validating
| that every string/locale pair can be resolved. Three lines of
| code, two seconds of wall-clock time, and we know that everything
| works. If there are too many variants to run them all, then:
|
| Prop tests, if you can get into them. Again, validate consistency
| and invariants.
|
| And make sure that you're actually testing what you think you're
| testing by using mutation testing. It's great for having some
| confidence that tests actually catch failures.
| otoolep wrote:
| >The first test of a class of tests is the hardest, but it's
| almost always worth adding.
|
| Agreed, this has been my experience too.
| oweiler wrote:
| That's why I prefer test frameworks that make parametrised
| tests so easy that they become the default.
|
| Kotest and Spock are such frameworks.
___________________________________________________________________
(page generated 2025-01-15 23:01 UTC)