[HN Gopher] Kopia: Fast and secure open-source backup software
___________________________________________________________________
Kopia: Fast and secure open-source backup software
Author : thunderbong
Score : 159 points
Date : 2023-09-15 17:32 UTC (5 hours ago)
(HTM) web link (kopia.io)
(TXT) w3m dump (kopia.io)
| monlockandkey wrote:
| Can someone please help decide what is the "best" backup
| software?
|
| - Restic (https://restic.net/)
|
| - Borg backup (https://www.borgbackup.org/)
|
| - Duplicati (https://www.duplicati.com/)
|
| - Kopia (https://kopia.io/)
|
| - Duplicay (https://duplicacy.com/)
|
| - Duplicity (https://duplicity.us/)
| keep_reading wrote:
| Bacula
|
| I've tried all of these, and none are as reliable or powerful
| as Bacula.
|
| It's way more complex at first, but you will have peace of
| mind. And backup/restore speed is way faster.
|
| You can even easily setup automatic restore jobs to prove your
| backups work!
| MenhirMike wrote:
| Do you know how Bacula compares to Bareos? Bacula is on my
| to-do list to look at (also because I need tape backups), but
| the Bareos fork seems to have a more modern interface - but
| I've not stress tested either solution. The fact that Bacula
| has a Debian package and Bareos does not pretty much settles
| it already, but just curious if someone has actually tried
| both.
| bhaney wrote:
| I can't give you a meaningful comparison between all of those,
| because I haven't used all of them, but I can say that I've
| been pretty happy with Restic in the time I've been using it.
|
| Do you have any odd requirements that one might serve better
| than the rest? If you just want bog-standard backups, any of
| them will probably do.
| Linux-Fan wrote:
| Bupstash (https://bupstash.io/) beats Borg and Kopia in my
| tests (see https://masysma.net/37/backup_tests_borg_bupstash_ko
| pia.xhtm...). It is a modern take very close to what Borg
| offers regarding the feature set but has a significantly better
| performance (in terms of resource use for running tasks, the
| backups were slightly larger than Borg's in my tests).
| ntolia wrote:
| I've been using Kopia for my personal use and for products I have
| helped build at a couple of enterprise backup companies! It's
| also used by other open-source backup projects that focus on
| specific ecosystems (Velero and Kanister for Kubernetes, Corso
| for Microsoft 365 backup).
|
| I am obviously biased but it's pretty amazing. AMA.
| x0x0 wrote:
| Can you compare vs eg Duplicati?
|
| Do you (sorry, but just checking) repeatedly test backups? Eg
| pull monthly and bit verify that they're correct? Are you aware
| of anyone testing in this way?
|
| Thanks so much!
| riedel wrote:
| I can only compare it from a user experience point of view. I
| tried duplicati for my windows laptop and was never quite
| happy. Kopia just worked from day one. The front-end still
| has a few bugs here and there particularly if you on windows
| electron eating sockets, WebDAV mounts not always working),
| however the backend seems very reliable (only did one full
| restore, but I also did not note any reports).
|
| It still has a lot of potential, IMHO. You e.g. find some
| hints how to use it with AWS storage tiering in the docs.
|
| I am just a very happy user!
| ntolia wrote:
| I haven't looked at duplicati in a while and, it has evolved.
| While Duplicati's feature set looks similar now, I would need
| to benchmark it both for efficiency and final backup sizes.
|
| And, while not directly, I know a number of companies,
| including mine, do test restores all the time.
| SillyUsername wrote:
| No mention on the website, is it possible to schedule backups?
| vladgur wrote:
| Im curious if its able to use NAS Storage or a combination of
| offline/online storage
|
| I got a Synology in my house that could be utilized
| viciousvoxel wrote:
| Yes, it can. If your Synology model can run docker, running a
| minio instance is a good option.
|
| https://kopia.io/docs/repositories/
| proxyon wrote:
| Where does one even acquire a VPS that makes this worth it? Most
| VPS pricing I've looked at is significantly more expensive than
| something like BackBlaze or IDrive. So what even is the point of
| rolling your own backups if you can't get cheap terrabytes in the
| cloud? And no I'm not going to consider something like S3 because
| Amazon's pricing is obnoxious and confusing. Edit: $70 / month
| for 3TB of S3. Significantly more expensive than all of the
| managed SaaS backup providers.
| cycomanic wrote:
| AFAIK kopia has a S3 backend so can backup to idrive E2. That
| said I have a vps from greencloud with 2 TB (and 4 cores) for
| $80 a year which is very price competitive. There are actually
| lots of smaller vps providers that offer cheap storage vps.
| Lowendtalk.com is a good place to find out about offers in
| particular around black Friday.
| wmf wrote:
| S3 is the most expensive object storage; there are plenty of
| cheaper options like B2, Wasabi, or Coldline.
| danielhep wrote:
| I haven't tried it but it should work with Hetzner Storage
| Boxes over SFTP. Extremely price competitive.
| baal80spam wrote:
| Why not just use rclone?
| cpach wrote:
| AFAIK rclone is a sync tool. A backup tool is not the same as a
| sync tool.
| freedomben wrote:
| rclone is way underrated and should be used by a lot more
| people. But the various UIs are still a bit technical. Though
| if I were writing backup software nowadays, I'd probably write
| a skin over rclone.
| wmf wrote:
| Compression is still experimental. Can rclone pack small files
| to reduce API calls and storage space?
| momirlan wrote:
| +1 happy customer, for my laptop documents
| xgbi wrote:
| We are trying to use it for large backups of a production item,
| and it has not been a complete smooth ride all along.
|
| We have many files (millions) and lots of churn over ~80Tb total.
|
| Kopia has exhibited some issues:
|
| - takes about 120GB (!) of ram to perform regular maintenance &
| takes about 5hrs to do so. There are ideas floating around to
| cherry pick the large inefficiencies in the GC code but it's yet
| to be worked on. I'll try to have a internship accepted to work
| on this in my company.
|
| - there's a good activity on the repository but the releases are
| not quick to come and the PRs are not very fast to be examined
|
| - the local cache gets enormous and if we try to saddle it, we
| have huge download spikes (>10% of repo size) during maintenance.
| Same as above: pb is acknowledged but yet to be solved
|
| - the documentation is very S3 centric, and we discovered too
| late that the tiered backup (long term files go into cold storage
| on s3) is only supported on S3, while we use azure. We
| contributed a PR to implement it in June, yet to be merged (see
| point 2)
|
| So, not too bad, especially for a small-ish project maintained by
| mainly one person (from the looks of my interactions on slack and
| seeing the commit log). The maintainer is easy to reach and will
| answer, but external prs are slow. If I could use zfs cheaply on
| azure via s3, I'd use it over kopia, but as of now, it works.
| jiggawatts wrote:
| "ZFS on Azure via S3" is missing just an Apple technology to
| win the "mixing vendors randomly" bingo.
| haunter wrote:
| Love the asciinema demo. More projects should use it. Sometimes I
| have the urge to make a pull request on Github projects without
| any screenshot or video.
| candiddevmike wrote:
| I personally don't see the point of CLI videos like asciinema,
| just let me scroll through the damn CLI history.
| amarshall wrote:
| > Kopia does not currently support cloud storage that provides
| delayed access to your files - namely, archive storage such as
| Amazon Glacier Deep Archive. Do not try it; things will break.
|
| Sigh... and unfortunately all too common for there to be no cold
| storage support.
| ntolia wrote:
| However, you should be able to use Glacier Instant Access. It
| will get you a bunch of the way there.
| justin_oaks wrote:
| I use Restic for personal backups and at work, but I thought I'd
| try Kopia to see if it'd be a good fit for my less techie
| relatives. This was about a year ago.
|
| The UI didn't seem like a good fit for those who are less
| technical. I don't remember the specifics.
|
| Does anyone have recommendations for backup services for the
| average user?
| Unfrozen0688 wrote:
| Compared to Veeam how is it?
| ntolia wrote:
| Very different products and can't compare them. Veeam is
| enterprise-grade and used for a larger variety of mission-
| critical workloads. Kopia is meant for end-user backups (though
| folks use it for a bunch of other things too).
|
| Note that Veeam contributes to Kopia -
| https://www.veeam.com/sys451
| rsync wrote:
| Kopia runs over plain old sftp and can be pointed at any old sftp
| endpoint:
|
| https://twitter.com/rsyncnet/status/1643361320534953984
| guerby wrote:
| Replaced borg by kopia a while ago, I have "kopia snapshot create
| /" on most of the machines I manage, just works.
| izoow wrote:
| Why did you switch?
| anderspitman wrote:
| The state of backup tech is surprisingly bad, and runs OS deep.
| Even with modern solutions like restic, you have no guarantee of
| a consistent backup[0] unless you pair it with a filesystem that
| supports snapshots like btrfs, ZFS, etc, which basically no one
| other than power users are even aware of. Interestingly, Windows
| ships snapshot support by default[1], but you need admin
| privileges to use it.
|
| Also, it's unclear to me what happens if you attempt a snapshot
| in the middle of something like a database transaction or even a
| basic file write. Seems likely that the snapshot would still be
| corrupted. So for databases you're stuck using db-specific
| methods like pg_dump.
|
| All this complexity makes it very difficult to make self-hosting
| realistic and safe by default for non-experts, which is the
| problem I'm having.
|
| [0]: https://forum.restic.net/t/what-happens-if-file-changes-
| duri...
|
| [1]: https://learn.microsoft.com/en-us/windows-
| server/storage/fil...
| [deleted]
| asmor wrote:
| The advantage and disadvantage of Linux simultaneously is that
| you need to pick such a filesystem or work around the
| limitations, but it's your choice. The OS underneath really
| should not be responsible. Apple solves it with APFS snapshots
| and Microsoft has Volume Shadow Copy (which requires NTFS or
| ReFS).
|
| I personally use compose for all my services now and back up my
| compose.yaml by stopping the entire stack and running a restic
| container that mounts all volumes in the compose.yaml.[1] It's
| not zero downtime, but it's good enough, and it's extremely
| portable since it can restore itself.
|
| [1]:
| https://gist.github.com/acuteaura/61f221ada67f49193bc1f93955...
| adobrawy wrote:
| Filesystem snapshot for database is not different than database
| crash. Databases are designed to handle crashes well (WAL
| etc.).
| ntolia wrote:
| Unfortunately, this is not true. You need to grab all the DB
| files (WAL, etc.) in a consistent manner. You can't grab them
| while writes are in progress. There are ways though. Look at
| what Kanister does with its recipes to get consistent DB
| snapshots to get a sense of the complexities need to do it
| "right."
| thesh4d0w wrote:
| > Also, it's unclear to me what happens if you attempt a
| snapshot in the middle of something like a database transaction
| or even a basic file write. Seems likely that the snapshot
| would still be corrupted.
|
| You just quiesce the database first. Any decent backup engine
| has support to talk to a DB and pause / flush everything.
| cientifico wrote:
| For database, in my experience, is better to dump the whole
| database and backup the dump for the reasons you explained
| viraptor wrote:
| Depends on your database size, type, change rate, etc.
| Dumping the database to a file is fine for toy and small
| cases, but not for a 1+TB store that's under heavy writes.
| throwing_away wrote:
| I noticed this project while comparing restic/borg and am
| thinking about trying it.
|
| Initially I thought this was a corporate project and was looking
| for the monetization model, but then I found
| https://github.com/kopia/kopia/blob/master/GOVERNANCE.md
|
| I feel like the project might benefit from making their
| governance model more prominent on the website.
| IshKebab wrote:
| How does this compare to Rustic?
| SomeoneOnTheWeb wrote:
| I've been using Kopia for about a year to backup sensitive data
| multiple times a day into an off-site encrypted storage and it's
| worked like a charm so far.
|
| The presence of a WebUI is so nice compared to CLI-only tools.
| hsshah wrote:
| How does it compare with Arq Backup (for Mac)? It's a bummer that
| Arq has still not added support for Azure.
| unwind wrote:
| Hm interesting name, I spent a few seconds on the site to figure
| out origin but no luck.
|
| "Kopia" means "copy" in Swedish and probably more Nordic
| languages, too. Very hard to pronounce in English so it would be
| interesting to hear it said.
| j1elo wrote:
| My first impression was that it could have also been spanish
| "Copia" (copy), but built for KDE (thus following the naming
| trend of those apps)
| fishnchips wrote:
| Also in Polish and other Slavic languages probably, too.
| fishnchips wrote:
| Ah, just noticed. The founder is Polish. That explains the
| name.
|
| As a Pole, I actually greatly appreciate these Slavic names
| in tech.
| trwired wrote:
| Kopia also means a copy in Polish and the author is Polish. The
| first paragraph in the software's Github page also confirms the
| Polish origin of the name: https://github.com/kopia/kopia/
|
| Tangentially, as far as OSS names of Polish go, kopia is pretty
| tame. A popular deduplicating app is named czkawka (hiccup).
| Now that choice is just mean towards non-Polish speakers. :)
| traceroute66 wrote:
| > as far as OSS names of Polish go, kopia is pretty tame
|
| Well indeed.
|
| There's a project on GitHub with 1.7k stars called
| GitKurwa[1].
|
| Now that's proper untame Polish. ;-)
|
| [1] https://github.com/jakubnabrdalik/gitkurwa
| semi-extrinsic wrote:
| Oh my, this is a fantastic view of life - from the czkawka
| github page: Czkawka is a Polish word which
| means hiccup. I chose this name because I wanted
| to hear people speaking other languages pronounce it, so feel
| free to spell it the way you want. This name is
| not as bad as it seems, because I was also thinking about
| using words like zolc, gzegzolka or zoladz
| vladgur wrote:
| I had to google pronounciation of czkawka and to me it
| sounds exactly how i would pronounce it - chkavka or
| chkavka in Russian
|
| https://en.wiktionary.org/wiki/czkawka
| turtles3 wrote:
| This sounds very much in the spirit of Grzegorz
| Brzeczyszczykiewicz[0]
|
| [0] https://m.youtube.com/watch?v=AfKZclMWS1U
| havaloc wrote:
| I was using Vorta/Borgbase until I discovered that Vorta had an
| issue with restores. Kopia is pretty neat, but make sure you test
| as always.
| trailbits wrote:
| What was the problem with restores?
| jopsen wrote:
| I've been happy with kopia for a while now...
|
| But I probably ought to double check that restoring works :)
| johnchristopher wrote:
| > I was using Vorta/Borgbase until I discovered that Vorta had
| an issue with restores.
|
| Such as ?
| MenhirMike wrote:
| CTRL+F "tape" "lto" - 0 results.
|
| I know that Tape Backups are not hip and sexy, but CloudNordic
| showed us just last month why they still matter even in 2023 and
| beyond, so you'd definitely want to look at an additional
| solution for your large servers, with a proper rotation/retention
| strategy (e.g., GFS). You _need_ offline backups, if you think
| you don't, you just got lucky for now - or have data that can be
| recreated from other sources.
|
| For an online hot/warm solution, I'd use sending ZFS Snapshots
| into a backup server to then compress and encrypt them there,
| though keep in mind that for running systems, it may still not be
| enough (e.g., backing up a running Postgresql server through a
| file system snapshot may not be enough - there's an entire
| section in the documentation about backup options).
|
| That said, it's good to have more options, and you really want to
| use something for your personal stuff as well, so the more
| options there are, and the more user-friendly/turnkey they are,
| the better!
|
| Just be aware that backup solutions in a corporate/network
| environment are more complicated than just copying some files
| across. And also remember: Good companies test their backups -
| but great companies test their restores.
| [deleted]
| remisharrock wrote:
| I have also used it for a year or so and restored some files
| individually without any problem. I use a remote ssh storage
| through tailscale, just very stable interestingly! Only small
| problem was that my server broke during an uptade because I
| created a systemd service to start it and the parameters changes
| to start it. Apart from that, very stable for now.
| bhaney wrote:
| Any comparisons to Restic? Looks like basically the same thing
| but with a GUI available.
|
| Edit: Found this very ad-hoc "benchmark" from over a year ago
| claiming that Kopia managed significantly better deduplication
| than Restic after several backups (what took Restic 2.2GB, Kopia
| did in <700MB). No idea if the advantage falls off outside of
| this particular benchmark, but if it doesn't then that's a pretty
| big improvement.
| https://github.com/kopia/kopia/issues/1809#issuecomment-1067...
|
| Edit Edit: Never mind, this benchmark was from before Restic
| supported compression, which is the why its size is so much
| larger. Feels like that should have been mentioned.
| ntolia wrote:
| One thing I will mention is that other backup projects have
| switched from Restic to Kopia. Velero from VMware comes to
| mind.
| [deleted]
| alyandon wrote:
| Restic only (relatively) recently implemented compression so
| I'd run my own tests before making a decision. I'm currently
| thinking about migrating off borg to restic and disk space
| usage is very similar based on my testing.
| [deleted]
| pizza234 wrote:
| UX is very important for backups (considering that users can be
| very lazy, so the least friction, the better), so a GUI is an
| important component; those who don't care about UX surely have
| infinite ways to perform their backups.
|
| My dealbreaker with Restic was near-realtime backup - the
| discussion has been open for 5 years now:
| https://forum.restic.net/t/continuous-backup/593); this is also
| a UX problem. I haven't checked if Kopia supports it (or has
| better support, anyway), though.
| anderspitman wrote:
| That link is corrupted. HN thinks the ");" is part of it:
|
| https://forum.restic.net/t/continuous-backup/593
___________________________________________________________________
(page generated 2023-09-15 23:00 UTC)