Have you restored a Docker volume from a backup? If so did it work out? reddit / selfhosted ---------------------------------------------------------------------- Significant-Neat7754 · 2023-12-01 19:44 The backup solution could be Duplicati, Restic or Borg. My question is specifically regarding permissions. If you have restored a Docker volume/database from a backup, did it restore the permissions correctly? If so, were you able to get a container running from that backup smoothly without having to tinker with permissions again? Thank you for answering! ---------------------------------------------------------------------- Vyerni11 · 2023-12-01 20:02 Yes. All my docker volumes, are bind mounts. I ensure whenever I add a new container, any persistent data is written as my user (1000). Duplicati backs up, and restores this perfectly. I've done multiple actual restorals, and every 6 months I run a full DR test from scratch, to verify my backups, and procedures all work correctly. ---------------------------------------------------------------------- Simplixt · 2023-12-01 20:04 Yes, borg preserve permissions. And btw: Why ask on reddit, just test it by yourself! A backup where you just trusting people in the internet that it will work is useless. Just try it yourself and get confident :) ---------------------------------------------------------------------- thekrautboy · 2023-12-01 20:08 Yes. ---------------------------------------------------------------------- BrodyBuster · 2023-12-01 20:09 Same. Works flawlessly. I do have some containers that run as root, but so does my rsync backup, so no issues there. ---------------------------------------------------------------------- Significant-Neat7754 · 2023-12-01 20:34 You're right. But I have got so much help from the /r/selfhosted community over the years, I trust the most voted/best rated answers 😊 ---------------------------------------------------------------------- Astorek86 · 2023-12-01 21:33 I'm just using `tar`. As long as you're run it as "root", it preserves Permissions and Ownerships of the files. You must be "root" for both backup and restore. ---------------------------------------------------------------------- haaiiychii · 2023-12-01 21:41 I just rsync my docker volumes, but when I moved my server all I did was rsync my /docker directory (which contains everything I have for docker) to the new server, ran compose up -d and worked as if I never changed machine. Did the same for a friend who moved server too. I've also had to stop a container, delete the directory of that volume and copy the backup volume and start that. Always worked and never had an issue. Permissions or otherwise. ---------------------------------------------------------------------- Ursa_Solaris · 2023-12-01 22:49 I "tested" (screwed something up, but luckily made a manual copy because I knew I might) a manual backup recently. I had to manually correct some permissions because it was a manual and hasty process. Otherwise, it all worked fine. I need to test a proper restore from Borg when I have time. ---------------------------------------------------------------------- Impressive-Cap1140 · 2023-12-01 23:03 Same with Duplicati? ---------------------------------------------------------------------- BrodyBuster · 2023-12-01 23:31 Sorry … not sure what you’re asking. I use rsync, but all my docker containers also use bind mounts, which get backed up nightly. ---------------------------------------------------------------------- Big-Finding2976 · 2023-12-01 23:34 Do you do anything before doing the full DR test to protect yourself in case it doesn't work correctly and you end up with a load of corrupted files? ---------------------------------------------------------------------- www_creedthoughts · 2023-12-01 23:53 Do you stop all of your containers before doing the backup? ---------------------------------------------------------------------- ExperimentalGoat · 2023-12-01 23:55 What's the practical difference between the bind mount and volume mount for your setup? ---------------------------------------------------------------------- Vyerni11 · 2023-12-01 23:58 I have the luxury of running restore on a spare machine ---------------------------------------------------------------------- Vyerni11 · 2023-12-01 23:58 Yes. Via a script ---------------------------------------------------------------------- www_creedthoughts · 2023-12-02 00:00 I do the same. I thought you were offering a way to do without shutting down the containers, which would be cool. ---------------------------------------------------------------------- Vyerni11 · 2023-12-02 00:13 Nah. I'd prefer to shut down at night anyway to run the backup to ensure data integrity. Containers are only down for 15 minutes or so at 1am. ---------------------------------------------------------------------- Vyerni11 · 2023-12-02 00:22 Mounts in my user directory, as opposed to docker controlled locations somewhere in /var ---------------------------------------------------------------------- z3roTO60 · 2023-12-02 00:28 I have this strange issue with the container for Node-RED. All of my docker containers on my different hosts are bind mounted volumes. All except for Node-RED. That is the one container that, for who knows what reason, will not fire up properly or maintain persistent data with a normal bind mount. Works perfectly for a docker volume that I created outside of the `docker-compose.yaml` file though haha ---------------------------------------------------------------------- Big-Finding2976 · 2023-12-02 00:56 That's a very good option to avoid messing things up when testing. ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 02:00 Yeah, Duplicati can run as root ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 02:05 I don't and haven't had a problem, but I wouldn't recommend it. YMMV and do so at your own discretion. ---------------------------------------------------------------------- adamshand · 2023-12-02 04:25 You may already know this, but ... The only thing you absolutely must shutdown (or quiesce) before backing up is databases. For everything else, it's a calculated risk. The backup might skip a locked file or corrupt a file being actively written as it's backed up ... but the chances of it happening to the same file over multiple backups is typically very small. ---------------------------------------------------------------------- mrmclabber · 2023-12-02 04:57 Yes, I just image my entire docker VM lol. ---------------------------------------------------------------------- Vyerni11 · 2023-12-02 06:55 >ou stop all of your containers before doing the backup? I could, but given it all runs at 1 (Or 3, can't remember) in the morning, I'm asleep. And given it's all automated an only runs for 20-30 minutes each night, it doesn't bother me to let it shut it all down. It at least provides me more reassurance that all the files will be good and completely as they were. If they were mission critical, at that time, I'd alter it to not stop certain services ---------------------------------------------------------------------- ExtensionCricket6501 · 2023-12-02 06:56 I setup my stuff before thinking of this and luckily I was mindful enough to run \`getfacl\` to backup permissions, but most of my stuff turned out to not have too much problems with permissions, so I never knew if the permission restore was necessary or not. I don't think this is exactly optimal so I've been trying to do better in a newer setup. ---------------------------------------------------------------------- love_tinker · 2023-12-02 07:06 I got trouble with postgres, after an electric lost, i can't access my table. Due to low priority, i dispose it and restore from backup files. ---------------------------------------------------------------------- shaunjanssens · 2023-12-02 10:14 I just started self hosting so I never had to restore a backup for real but tested it multiple times to be sure my setup works. I’m also planning to test it every 3 months. ---------------------------------------------------------------------- lilolalu · 2023-12-02 10:24 https://docs.docker.com/storage/volumes/ ---------------------------------------------------------------------- CheebaSweets · 2023-12-02 12:14 Glad to hear Duplicati can restore this perfectly. I've been backing up for some years now using it but haven't had to restore as of yet. ---------------------------------------------------------------------- Charming-Molasses-22 · 2023-12-02 13:31 Has always worked out for me so far. Only program I had an issue with was nginx-proxy-manager. Some files were owned by root so the backup program couldn't back up those specific files. And you've just reminded me that I need to look into caddy as a replacement for NPM. ---------------------------------------------------------------------- agent_kater · 2023-12-02 13:59 If you mount a host directory into the container (the best solution IMO) it's no longer a "docker volume", is it? ---------------------------------------------------------------------- lbgdn · 2023-12-02 14:19 According to their [docs](https://nodered.org/docs/getting- started/docker#using-a-host-directory-for-persistence-bind-mount) bind mounts should work just fine. ---------------------------------------------------------------------- lbgdn · 2023-12-02 14:25 A backup you haven't tested restoring is not really a backup. Corollary: an important part of a backup, which is often forgotten, is testing the restore. ---------------------------------------------------------------------- scionae · 2023-12-02 15:12 I've used Duplicati but after migrating my project from one pc to another and trying to restore one of the backups, nothing worked so I ditched it. I'm now using docker-volume-backup and it's very bery powerful. No CLI, works with labels, but it has everything I need and then some. ---------------------------------------------------------------------- CheebaSweets · 2023-12-02 17:40 It was definitely ignored and not even considered by me. Thank you for your insight and pointing out the significance. It seems so obvious to do so now haha