[HN Gopher] Restic 0.13.0
___________________________________________________________________
Restic 0.13.0
Author : soheilpro
Score : 96 points
Date : 2022-03-27 18:22 UTC (4 hours ago)
(HTM) web link (restic.net)
(TXT) w3m dump (restic.net)
| tekronis wrote:
| Besides Homebrew, restic is also available via MacPorts:
| https://ports.macports.org/port/restic/
| mekster wrote:
| Unless it's for experimenting, I've stopped caring for backup
| solutions other than borg and zfs as the only way to prove their
| stability is to have them exist for a while without big
| complaints and new ones all seem to have complaints.
|
| Just having no data loss isn't enough which is the absolute base
| point but huge memory consumption and other operational issues
| are also showstoppers.
| cma wrote:
| Borg being single-threaded is painful in the era of consumer 12
| and 16 core CPUs, and even prosumer 64-core.
| sigio wrote:
| Don't notice this much.... disk is usually the bottleneck,
| and otherwise it will be the network to the remote backup
| location. Still backups complete in seconds:
|
| Repository: ssh://backup/./backups/mungedhostname.borg
| Archive name: 20220327-2201 Archive fingerprint: 8b710144579c
| 8d531e7c4a0192304323081b14a71445557608d859494bbe84b6 Time
| (start): Sun, 2022-03-27 22:01:35 Time (end): Sun, 2022-03-27
| 22:01:52 Duration: 17.28 seconds Number of files: 45678
|
| Original size Compressed size Deduplicated size This archive:
| 24.22 GB 9.35 GB 47.78 MB All archives: 363.60 GB 140.91 GB
| 7.81 GB
|
| Unique chunks Total chunks Chunk index: 59857 852257
| silon42 wrote:
| I like that it runs in the background without heating my
| laptop much.
| lucgommans wrote:
| This point hides a lot of goodness in something that I didn't
| even understand on the first read:
|
| > - We have added checksums for various backends so data uploaded
| to a backend can be checked there.
|
| All data is already stored in files with as filename the
| sha256sum of the contents, so clearly it's all already
| checksummed and can be verified right?
|
| Looking into the changelog entry[1], this is about verifying the
| integrity upon uploading:
|
| > The verification works by informing the backend about the
| expected hash of the uploaded file. The backend then verifies the
| upload and thereby rules out any data corruption during upload.
| \n\n [...] besides integrity checking for uploads [this] also
| means that restic can now be used to store backups in S3 buckets
| which have Object Lock enabled.
|
| Object lock is mentioned in passing somewhere down the changelog,
| but it's a big feature. S3 docs:
|
| > Object Lock can help prevent objects from being deleted or
| overwritten for a fixed amount of time or indefinitely.
|
| i.e. ransomware protection. Good luck wiping backups if your file
| hoster refuses to overwrite or delete the files. And you know
| Amazon didn't mess with the files because they're authenticated.
|
| Extortion is still a thing, but if people would use this, it
| more-or-less wipes out the attack vector of ransomware. The only
| risk is if the attacker is in your systems long enough to outlast
| your retention period and creates useless backups in the meantime
| so you're not tipped off. Did anyone say "test your backups"?
|
| For self-hosting, restic has a custom back-end called rest-
| server[2] which supports a so-called "append-only mode" (no
| overwriting or deleting). I worked on the docs for this[3]
| together with rawtaz and MichaelEischer to make this more secure,
| because eventually, of course, your disks are full or you want to
| stop paying for outdated snapshots on S3, and an attacker could
| have added dummy backups to fool your automatic removal script
| into thinking it needs to leave only the dummy backups. Using the
| right retention options, this attack cannot happen.
|
| Others are doing some pretty cool stuff in the backup sphere as
| well, e.g. bupstash[4] has public key encryption so you don't
| need to have the decryption keys as a backup client.
|
| [1] https://github.com/restic/restic/releases/v0.13.0
|
| [2] https://github.com/restic/rest-server/
|
| [3]
| https://restic.readthedocs.io/en/latest/060_forget.html#secu...
|
| [4] https://github.com/andrewchambers/bupstash/
| Mo3 wrote:
| I have to say, this is an excellent solution and I am seriously
| contemplating on deploying it for all of our servers.
|
| I'm a little unclear on one thing, are alternative S3 providers
| supported?
| bccdee wrote:
| Yes! I back up directly into Backblaze B2 using restic.
| Mo3 wrote:
| Excellent. One more question. Assuming someone gains access
| to a node - will they be able to access the backups or screw
| with them in any way?
| lucgommans wrote:
| Depends which back-end you use and which configuration.
| (Assuming "node" here means a backup client and not a
| backup-hosting server.)
|
| If you just put stuff on some standard storage (FTP, B2,
| etc.) without any permissions set up, then no backup client
| could stop the attacker from using the credentials you've
| deployed to the backup client to login to your storage and
| delete your backup files. This is not really specific to
| restic.
|
| Accessing old files from a backup client is a weakness.
| Though, personally, I don't store things on a system that
| this system is not supposed to know, so if someone
| compromises this system and they see data from the past
| year... that's nearly the same as just seeing the data that
| is on this system today. Nevertheless, this could be solved
| by using public keys, so the backup clients has an
| encryption but no decryption key (of course it's not as
| trivial as this sounds, e.g. how could it still do
| deduplication etc.), but restic does not do this.
| Mo3 wrote:
| Got it. So I guess the best solution to go forward here
| would be to deploy Minio and then backup the backup
| directory itself in regular intervals onto different
| storage not accessible from anywhere.
| tome wrote:
| These seem to be the supported backends Local
| directory sftp server (via SSH) HTTP REST
| server (protocol, rest-server) Amazon S3 (either from
| Amazon or using the Minio server) OpenStack Swift
| BackBlaze B2 Microsoft Azure Blob Storage
| Google Cloud Storage And many other services via the
| rclone Backend
|
| https://github.com/restic/restic#backends=
| Mo3 wrote:
| I saw that, I was specifically wondering about different
| regular S3 implementations like Linode object storage,
| Digitalocean spaces, Wasabi, etc
| wedowhatwedo wrote:
| Did you try your back end to see if it works? It's not that
| hard to download the executable and try it.
| Mo3 wrote:
| Yes, just now. Here's the piece of information I was
| looking for - rclone supports S3
| soheilpro wrote:
| I'm so glad to finally see the --dry-run option in Restic.
| wmf wrote:
| Still no compression?
| spasche wrote:
| I'm afraid not: https://github.com/restic/restic/issues/21
| timmg wrote:
| I have a hand-coded backup system for my photo library that
| writes to S3. It runs every night at 2AM.
|
| The one feature I have that's important to me is this: it will
| figure out what files need to be uploaded and then upload as many
| as possible _for an hour_ then stop.
|
| That means that it runs for at-most an hour a night.
|
| The reason I need/wanted this feature is that I might come home
| from a trip with (eg) 30G worth of photos. My (cable) internet
| will upload at around 1G an hour. I don't want this thing to
| saturate my internet for 30 hours straight. Instead, it backs up
| a small amount every night for 30 days.
|
| Am I the only one that wants a feature like this? I've never seen
| it in any other backup system. (At alternative might be to have
| configurable bandwidth for uploads.)
| [deleted]
| seized wrote:
| Rclone will do exactly what you want, upload to S3 and the
| --max-duration will stop new transfer from starting after a
| given duration.
|
| There are also throttle options for bandwidth. I use that
| combined with Node-Red and a smart plug on my monitors, if
| monitor power draw exceeds a threshold then the upload throttle
| is changed via the rclone API.
| timmg wrote:
| Ok, that looks like a nice tool. Thanks!
| crecker wrote:
| I wonder if you can obtain that behavior with a bash script,
| but it's boring to write scripts and I do not know if SIGTERM
| can exit Restic gracely.
| lucgommans wrote:
| If you've ever ctrl+c'd restic, you'll know the message
|
| > signal interrupt received, cleaning up
| crecker wrote:
| Thanks! Will try tonight to play with Restic
| lucgommans wrote:
| Makes perfect sense. Restic kind-of supports this because you
| can just kill the client after an hour and, tomorrow, it'll see
| which objects are there already.
|
| I'm not deep enough into the project to know whether this is
| like an officially supported use-case, but restic was of course
| made with the idea that interruptions can happen (your computer
| can crash) and should be handled safely, and for the
| deduplication it'll cut files up in a deterministic way and
| thus (as I understand it) store those chunks in a deterministic
| place.
| lykron wrote:
| You could do this with the `timeout` command.
|
| `timeout -s SIGINT 1h restic...`
|
| That would let restic run for one hour, and then once the hour
| elapses send a SIGINT which will stop the process (see https://
| github.com/restic/restic/blob/a29777f46794ea4e35548f...)
| sydney6 wrote:
| And restic would then resume the backup next time it runs.
|
| [1] https://restic.readthedocs.io/en/latest/faq.html#will-
| restic...
| sydney6 wrote:
| Restic has support for rate limiting built-in.
| teaudel wrote:
| My internet upload speed is bad so I do want something like
| that.
|
| I would also like to be able to "stage" a backup: figure out
| what needs to be transmitted and then create the data files
| that need to be transmitted without actually immediately
| transmitting it.
|
| That would allow me to do things like backup my laptop to
| another computer in my house that can upload the files over my
| slow connection overnight when my laptop isn't on; and to let
| me bring the backup files to a place (work/university/library)
| with a fast connection so large backups don't take days or
| weeks (especially initial backup).
| pdimitar wrote:
| Take a look at NNCP[0]. Haven't tried it yet but read several
| articles about it and I am looking forward to finding time to
| use it seriously.
|
| [0] https://www.complete.org/nncp/
| funcDropShadow wrote:
| I am using restic to backup my laptop and workstation to my
| nas. At night rclone syncs the restic repositories to S3. I
| can restore both from my nas as well as from s3.
| rvieira wrote:
| For someone using Kopia, is there any major advantage in
| switching to restic?
| paraph1n wrote:
| Any thoughts on how this compares to duplicacy?
| Tijdreiziger wrote:
| ...or Duplicity?
| sigio wrote:
| Both restic and borg are lightyears ahead of duplicity....
| backups are soo much faster (and smaller), restores are much
| easier.
| seized wrote:
| Duplicacy is much faster than Restic for backups and. That
| was was the overwhelming consensus when I was researching
| backup software.
|
| I use Duplicacy and Rclone, each for different types of data.
| crecker wrote:
| Where can I read about the best solutions for backups? For nerdy
| people
| rsync wrote:
| 'borg'[1] has, in recent years, become the de facto standard
| for secure, encrypted, you-control-the-keys backups. It has
| been referred to as "the holy grail of backups"[2].
|
| Two of the better howtos that we have seen for borg are [3][4].
| [4] is geared toward OpenBSD users.
|
| [1] https://borgbackup.readthedocs.io/en/stable/
|
| [2] https://www.stavros.io/posts/holy-grail-backups/
|
| [3] https://jstaf.github.io/2018/03/12/backups-with-borg-
| rsync.h...
|
| [4] https://rgz.ee/borg.html
| CameronNemo wrote:
| Restic and BorgBackup really seem to be the favored solutions
| out there. Restic for encryption, Borg for deduplication and
| compression. Or maybe bacula if you want pull based backups
| instead of push based.
|
| https://restic.readthedocs.io/en/stable/
|
| https://www.borgbackup.org/
|
| https://www.bacula.org/documentation/documentation/
| photon-torpedo wrote:
| BorgBackup can be made to operate in pull mode too, see: http
| s://borgbackup.readthedocs.io/en/stable/deployment/pull-...
|
| At work I've implemented a variant of the ssh-agent method
| described there. Admittedly, it requires some scripting :)
| lucgommans wrote:
| There is also https://github.com/restic/others which has some
| keywords (e.g. is it encrypted, does it do compression) for
| most FOSS backup solutions. It can be outdated or incomplete
| for some entries, though.
| O_H_E wrote:
| When talking about backups for nerds, https://www.rsync.net/
| always deserves a mention.
| funcDropShadow wrote:
| ... as well as the http://taobackup.com/ .
___________________________________________________________________
(page generated 2022-03-27 23:02 UTC)