[HN Gopher] Hold Off on Litestream 0.5.0
___________________________________________________________________
Hold Off on Litestream 0.5.0
Author : mtlynch
Score : 84 points
Date : 2025-10-14 16:10 UTC (6 hours ago)
(HTM) web link (mtlynch.io)
(TXT) w3m dump (mtlynch.io)
| emschwartz wrote:
| Extremely helpful. I've been eagerly awaiting v0.5 but have been
| holding off on deploying it until I had more confidence that it
| would work and be stable. Reading this, I'm definitely glad that
| I waited.
| ComputerGuru wrote:
| No need for all the "I love Litestream" disclaimers, these are
| serious issues that speak for themselves.
| jasonthorsness wrote:
| The change log is massive, so this is not unexpected to have a
| few issues. Maybe the two-year backlog of features and updates
| should have been batched into a few releases instead of this one
| big one? But there was a beta, and this is ripping the bandaid.
|
| https://github.com/benbjohnson/litestream/releases/tag/v0.5....
| mtlynch wrote:
| I think the most disruptive part is the migration to the new
| LTX format,[0] which I'd imagine is hard to do incrementally.
|
| [0] https://fly.io/blog/litestream-v050-is-here/#the-ltx-file-
| fo...
| benbjohnson wrote:
| Litestream author here. Yes, that's correct. LTX was the
| biggest hurdle to get over and was impossible to switch over
| incrementally. The storage layer change brings a lot of
| benefits and enables a lot of future work that we're really
| excited about.
|
| Thanks to mtlynch and everyone else who has submitted bug
| reports. We're squashing issues and working to get everything
| stable as quickly as possible.
| liuliu wrote:
| Feels like a side-effect of forever 0.x version symptom (I am
| guilty of as well). Even though semi-ver says 0.x can do
| whatever, people don't associate enough disruptive changes to
| it, whereas 0.4.x if it is 1.x, then it is much clearer this is
| a 2.x release.
|
| All things considered, this is probably just a tiny footnote in
| this software's life.
| ncruces wrote:
| They jumped from v0.3.x to v0.5.0 after a couple of years of
| v0.3.x (with an unreleased v0.4.x in between).
|
| That alone should hint everyone it was a big leap.
| ncruces wrote:
| I've been trying to get a headstart on the lightweight read
| replicas for my Go SQLite driver and, yes, there are issues.
|
| https://fly.io/blog/litestream-revamped/#lightweight-read-re...
|
| But the concept is really cool, and it's workable. I already have
| a version that mostly works.
|
| https://github.com/ncruces/go-sqlite3/tree/litestream/litest...
|
| With Litestream v0.5.0 doing the replication, you'll get some
| transient read failures that really shouldn't happen.
|
| If you build Litestream from head (or wait for v0.5.1), that'll
| be fixed. You'll still have similar issues around the time of a
| snapshot, if you keep only one snapshot.
|
| Other than that, I'm pretty confident it just works. But the
| performance probably sucks. I need to add a layer of caching. But
| that's it.
| ncruces wrote:
| And, meanwhile, v0.5.1 has been released. :)
| jadbox wrote:
| "But the performance probably sucks" The performance of what
| sucks? Replication? Why would you need to cache that?
| ncruces wrote:
| The idea behind Litestream is that you have a "primary" which
| has the SQLite database on a local disk, and you use
| Litestream to replicate it asynchronously to (e.g.) object
| storage.
|
| What's in object storage is not an SQLite file, but enough
| information to restore your SQLite file at various points in
| time. The way it's stored is optimized for frequent
| incremental updates and less frequent point restores.
|
| The lightweight read replicas VFS allows you open an SQLite
| database read-only directly from the data in object storage,
| _without_ having to first do a point restore, downloading all
| the data, and creating a local copy of the database.
|
| You "stream" the data directly from object storage as needed,
| get a consistent view of it, and can query it while the
| "primary" is concurrently updating it.
|
| It works, but SQLite kinda expects data to be local and low
| latency, which this might not be.
|
| SQLite has a page cache, which you can use, but everytime
| there is a write, the entire cache is dropped.
|
| Which means if you poll for updates regularly (which for
| various reasons you have to) you'll get frequent latency
| spikes.
|
| One way to improve this is to add a page cache that has an
| understanding of which pages have changed and which have
| stayed the same. Prefetching would help to, but SQLite is
| really unhelpful there.
|
| This is the piece I'm still missing. The rest mostly works
| already with Litestream v0.5.1.
| jadbox wrote:
| That's a rabbit hole. Thank you for sharing so much
| context. I see clearly that SQLite needs a page-updated-
| aware page cache, even if it was optional.
| andersmurphy wrote:
| Thanks for the heads up. I encountered similar issues so staying
| on 0.3.13 for now. But excited to give 0.5.x a go once the dust
| settles.
|
| Livestream, sqlite and caddy are incredibly at making single
| box/vps ops a breeze.
| placardloop wrote:
| > One of the benefits of Litestream 0.5.0 is that there's now an
| official litestream Docker image. All of my previous Docker
| containers required a lot of boilerplate to download the correct
| version of Litestream and make it available in my container, but
| now it reduces to a single Dockerfile line
|
| There's been an official Litestream container image for over 3
| years at this point (since version 0.3.4, it's at the same Docker
| Hub as 0.5.0).
| mtlynch wrote:
| Oh, thanks for the correction! I can't believe I never noticed
| that. I've updated the post.
|
| After your comment, I thought, "Oh, I should contribute a PR to
| the repo to add the Docker badge so the Docker image is obvious
| to everyone," but it turns out the badge has been right there
| for four years.[0]
|
| What I suspect happened is that I tried to use the Litestream
| Docker image once, discovered that image was amd64-only (until
| 0.3.9), so I didn't use it because I needed ARM, and then I
| just kept copy/pasting my workaround from project to project.
|
| [0]
| https://github.com/benbjohnson/litestream/commit/6acfbcbc64d...
| acrispino wrote:
| mtlynch, since you're a litestream+backblaze user, did you
| encounter this with 0.5.0?
| https://github.com/benbjohnson/litestream/issues/747
| mtlynch wrote:
| No, I saw that bug in the issue log, but I never hit that
| personally.
|
| If you're running into it, I'd test again with the latest
| release (0.5.1) and see if it's still present. If so, I suspect
| it would get more traction from the dev team if the report was
| complete, as it's currently missing repro steps and the
| litestream.yml is not compatible with 0.5.0 (it still has
| "replicas" plural).
___________________________________________________________________
(page generated 2025-10-14 23:01 UTC)