Are backups this simple? reddit / selfhosted ---------------------------------------------------------------------- tribal2 · 2023-12-30 21:49 I'm fairly new to self-hosting and trying to get my head around backing my server up to a mounted NAS. I'm running Ubuntu Server on a mini PC, with docker running around 15 containers. I have all of my docker compose files, config data and bind mounts stored in \~/docker like this: \~/docker /container1 docker-compose.yml /data /container2 docker-compose.yml /data etc Most of my containers use bind mounts in the above data folders, but I also have a handful of volumes kept in the default docker volumes location of /var/lib/docker/volumes. Is it as simple as just running a nightly cron job to back up both the '\~/docker' and '/var/lib/docker/volumes' directories to my mounted NAS directory? (or another offsite/cloud backup location of my choice) Or is there a better way? There are heaps of backup posts/solutions out there, but this struck me as being the simplest approach... ---------------------------------------------------------------------- Writinguaway · 2023-12-30 22:09 Really depends what you are running. Say a Wordpress site with some config files? Yeah your strategy is going to work. Say a MySQL database with constant write changes? No, you’re going to want to couple your strategy with a mysqldump. Your strategy will work for most docker containers with simple config files though. ---------------------------------------------------------------------- Eirikr70 · 2023-12-30 22:10 Hello u/tribal, in theory, you don't even have to back up the `/var/lib/...` volumes since they are not meant to be persistent (their data will be rebuild when you remove and restart the containers. If all of your data reside in the \~/docker directory, then it is all you have to back up. You might also choose to back up your system with such a tool as [Timeshift](https://github.com/linuxmint/timeshift), in case your system has problems and you want to come back to a previous version. I also recommend that you read about [Duplicati](https://github.com/linuxserver/docker-duplicati). It allows you to keep several successive backups in case your data gets corrupt one day and you are willing to come back to a previous state. Last but not least, you should read about the backup strategies of your apps. Some apps have dedicated strategies for backing up their databases (for instance two popular apps as Vaultwarden and Immich). Then you should set up their own dedicated backups. Have fun ! ---------------------------------------------------------------------- tribal2 · 2023-12-30 22:44 Thanks for this - I'm using Ubuntu Server so I'm not sure Timeshift is an option, but I will look more into Duplicati. I've been a little put off by a few posts where people have had issues/corrupt files. I'm fairly new to docker, so definitely still getting up to speed with everything - but I thought volumes were often persistent? Generally I've been using bind mounts more, largely because they were a little simpler to wrap my head around when getting up to speed with docker, and it didn't seem to make much difference when researching the various differences - I especially liked the idea of having my data/files in a single neat file structure too. I currently have x3 volumes which were all created by Immich - these are 'immich\_model-cache', 'immich\_pgdata' and one for immich redis. Do I really not need to back these up? ---------------------------------------------------------------------- Eirikr70 · 2023-12-30 23:04 `immich_pgdata` and `immich_model-cache` are non-persistent volumes, otherwise they would have been mounted on dedicated directoeirs. I don't see any volume dedicated to `redis` in the [docker-compose sample](https://github.com/immich-app/immich/blob/main/docker/docker- compose.yml) of Immich, so I wouldn't speak for it. As for `timeshift`, I use it on Debian, so I suppose you can use it on Ubuntu : [https://www.linuxcapable.com/how-to-install-timeshift-on- ubuntu-linux/](https://www.linuxcapable.com/how-to-install-timeshift- on-ubuntu-linux/) Since we talk backup strategy, make sure you have a proper security strategy ! ---------------------------------------------------------------------- tribal2 · 2023-12-30 23:09 Thanks - though I'm still slightly confused by immich\_pgdata not being necessary to backup. Isn't this the immich database which is full of my data for my library? I've found other [comments](https://www.reddit.com/r/selfhosted/commen ts/18fzvhx/comment/kd08m2s/?utm_source=share&utm_medium=web2x&context= 3) from people indicating they do back this up. Are they wrong? ---------------------------------------------------------------------- Eirikr70 · 2023-12-30 23:14 The backup of the database is described [there](https://immich.app/docs/administration/backup-and-restore). Unfortunately, backuping the database the way you plan to do it is useless : you won't be able to restore it ! This is a beginner's error. We have all been through it. ---------------------------------------------------------------------- tribal2 · 2023-12-30 23:41 Thanks I'll take a look into this ---------------------------------------------------------------------- Nils-22 · 2023-12-31 00:39 Just want to jump in here to clear some things up! You definitely have to backup your volumes that are in “/var/lib/docker/volumes“! They store persistent data and need to be backed up the same as your bind mounts. „immich_model-cache“ is an exception to this the data stored in this does not need to be backed up. Additionally I would recommend stopping the containers before you do the backup so that you ensure that no files are opened by another application. This ensured consistent backups and no corrupted backups when you later try to restore them. This also makes database backups possible and a viable option although the native backups are better. ---------------------------------------------------------------------- tribal2 · 2023-12-31 00:48 Thanks - I had assumed so generally speaking. I think I’m going to take two approaches - a basic nightly copy/paste to a remote NAS as per the original post of my docker data + volumes, and also a more bespoke approach to individual containers where necessary if the data is particularly important (e.g. Immich per the other comments about ensuring proper db backup). Hopefully the basic approach will cover 99% of scenarios, with some additional bespoke safeguards in place for services I particularly care about. ---------------------------------------------------------------------- Eirikr70 · 2023-12-31 01:57 Sorry, u/Nils-22 is right. ---------------------------------------------------------------------- tenekev · 2023-12-31 05:54 Check out https://github.com/Minituff/nautical-backup. It does automatically what others described. It stops the containers to prevent corruption during backup. Creates the backup and restarts the container. Then pushes the backup where you need them. All on a schedule. It works with bind mounts. There was a similar image doing pretty much the same thing with volumes but I can't recall the name. ---------------------------------------------------------------------- Spuxilet · 2023-12-31 07:27 Why you have 2 places for docker? Docker already has it's own root directory and it manages it by itself. Only root can modify/delete those files. For docker management i use portainer. It's a great too. I suggest use docker volumes, as bind mounts have limited functonality compared to docker volumes [https://docs.docker.com/storage/bind- mounts/](https://docs.docker.com/storage/bind-mounts/) Then backup will be very easy. Portainer has API. I have written a small script that stops all stacks gracefully, stops portainer itself and docker service. Then rsyncs docker root dir. When rsync finishes it starts all stacks again. Everything that is not important to me like movies, tv shows or something else is in bind mounts, which i do not backup. ---------------------------------------------------------------------- lmm7425 · 2023-12-31 16:15 Are you running any databases? They can be backed up by backing up the file structure, but they really should be done using native tools like mysqldump or pgdump.