How do you backup your docker containers when so much is owned by root? reddit / selfhosted ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 06:44 I am trying to setup a restic job to backup my docker stacks, and with half of everything owned by root it becomes problematic. I've been wanting to look at podman so everything isn't owned by root, but for now I want to backup my work I built. I've spent the last few days moving all my docker containers to a dedicated machine. I was using a mix of NFS and local storage before, but now I am doing everything on local NVME. My original plan was having everything on NFS so I would worry about backups there, and I might go back to that. ---------------------------------------------------------------------- Croatwink · 2023-11-01 07:04 Good question and I can't wait to see what people say. But let me answer your podman questions because I've been using rootless podman for years. I have the exact same problem, because you run podman rootless as a user account on the host, but inside the container it still runs with whatever UID the upstream project setup to use. You can of course change this, and maybe I should, but the end result so far is that even files in my podman volumes are owned by another user. Don't ask me how podman manages to do this rootless, but it does somehow. So I've been using podman volume export/import to backup my podman volumes, and it works very well. Even if the container's files are owned by a different UID. The problem only comes up when my backup script is trying to see if there are any files in the podman volume, then of course it can't read the volume contents because all of it is owned by another UID. It can still import, and export the contents as a volume tarball, it just can't list it. ---------------------------------------------------------------------- jippen · 2023-11-01 07:04 Any state worth saving in a docker container should be in a volume. Depending on environment, I usually either nfs mount those volumes, or use whatever tools the hosting provider has for me. In the homelab, docker containers run on a proxmox machine. I also have a truenas server running a proxmox Backup vm. It filters it's backups into zfs where the other normal nas backup setup takes over. ---------------------------------------------------------------------- VirtualDenzel · 2023-11-01 07:12 I build my own docker images. All my images are build to run as set id / guid when specified in ansible. This way only my servicedaemon can do stuff. Also makes sure i never have issues with borgbackup etc. ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 07:27 I have some with setid/guid, but not consistently. ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 07:28 I do have volumes, all stored with the docker-compose.yml file in \~/stacks/ ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 07:30 If you use PUID/PGID with docker, is there still an advantage with podman? ---------------------------------------------------------------------- linxbro5000 · 2023-11-01 07:36 I have a backup-script (running as root) that * stops all containers * runs the restic-backup * starts all containers ---------------------------------------------------------------------- sk1nT7 · 2023-11-01 07:48 I run duplicati docket container as UID=0 and GID=. Then bind mount the NFS share into the container where all docker bind volumes are stored. Due to the correct UID/GID mapping, the duplicati container has full access to all volumes. No permission errors anymore. ---------------------------------------------------------------------- ElevenNotes · 2023-11-01 07:53 Yes, because the containerd does not run as root, so no priviledge escalation possible. ---------------------------------------------------------------------- ElevenNotes · 2023-11-01 07:54 Don't run docker as root, don't run containers as root, pretty simple to be honest. ---------------------------------------------------------------------- rrrmmmrrrmmm · 2023-11-01 07:54 Can't you run a restic container where you mount everything? If the restic container is insecure, everything is of course. But yes, I also migrated to rootless Podman for this reason and a bunch of others. ---------------------------------------------------------------------- esturniolo · 2023-11-01 08:07 KISS method: Script that copy the data to the /tmp dir, compress it, encrypt it and move it to destination using rclone. Running every hour, 4 hours or 24 hours, depending the container. Never fails. The backups nor the restore. ---------------------------------------------------------------------- root-node · 2023-11-01 08:56 For backups I use [Nautical Backup](https://github.com/minituff/nautical-backup). For the "owned by root" problem, I ensure all my docker compose files have [P]UID and [P]GID set to 1000 (the user my docker runs under). All my 20 containers have no issue running like this. How are you launching your containers? Docker compose is the way, I have set the following in all mine: environment: - PUID=1000 - PGID=1000 user: 1000:1000 ---------------------------------------------------------------------- RydRychards · 2023-11-01 09:07 I used to let root start the backup job ---------------------------------------------------------------------- Croatwink · 2023-11-01 09:07 Sure there is, you still get separation from the host OS. One of the many goals of containers is to contain a process inside a type of jail. So the container can run with any UID it wants in its own namespace, but outside it does not have access to other containers, or other parts of the host OS. So if one service is breached it can't take others with it as easy. ---------------------------------------------------------------------- fuuman1 · 2023-11-01 09:38 I asked myself the same question a while ago. I run a lot of services and to do a custom backup for every single one of them is crazy. There are config files, sqlite, MySQL and PostgreSQL databases. I don't like that the docker ecosystem is not giving me the ability to backup volumes so in the end I went with the same way which ist already mentioned: a script stopping all containers to avoid corrupt data, creating the borg backup on my NAS and starting everything again. Executing as root because of the permission issue you mentioned. I don't like that solution but at least it's a backup and it's easy to maintain. ---------------------------------------------------------------------- habskilla · 2023-11-01 09:42 Run duplicati as a service on the host, thus runs as root. You now have full access to all docker volumes on the host. ---------------------------------------------------------------------- PaulEngineer-89 · 2023-11-01 09:52 Don’t backup the container!! Map volumes with your data to physical storage and then simply backup those folders with the rest of your data. Docker containers are already either backed up in your development directory (if you wrote them) or GitHub so like the operating system itself, no need tk backup anything. The whole idea of Docker is the containers are ephemeral. They are reset at every reboot. ---------------------------------------------------------------------- Mean_Einstein · 2023-11-01 10:34 There is a Linux capability to give a binary read rights to all files, regardless of file permissions. So the backup can be started as an unprivileged User, with rights to read everything, without interfering with the actual file permissions. It's called DAC Override or something ---------------------------------------------------------------------- mirisbowring · 2023-11-01 10:37 I had this before but this created struggles with some containers since they do start specific checks and scans during startup which resulted in high cpu and disk load. Since unraid supports zfs, i am using this for the docker stuff and do snapshots to external disk as backup no need to stop containers anymorw ---------------------------------------------------------------------- Zeal514 · 2023-11-01 10:47 First, I try not to have it owned by root. But some containers have special privileges that need to be followed. So rsync -O will copy the directory retaining permissions and ownership of all files. ---------------------------------------------------------------------- thisiszeev · 2023-11-01 10:56 I have a script for automated dumps of MySQL or MariaDB databases. Then you can use something like rsync to put it where you want it. https://git.zaks.web.za/thisiszeev/linux-server-tools Slowly I am adding more and more useful scripts to the above repo. ---------------------------------------------------------------------- Do_TheEvolution · 2023-11-01 11:00 From my basic selfhosted experience... I litearlly do nothing just run kopia/borg to backup my docker shit that uses bind mounts. And it really works to just copy it back and start container. But yeah if there is something I care about I schedule database dumps like here in [bookstack](https://github.com/DoTheEvo/selfhosted-apps- docker/tree/master/bookstack#backup-of-just-user-data) or [vaultwarden](https://github.com/DoTheEvo/selfhosted-apps- docker/tree/master/vaultwarden#backup-of-just-user-data) ---------------------------------------------------------------------- nobackup42 · 2023-11-01 11:02 I converted they to use Podman. ---------------------------------------------------------------------- atheken · 2023-11-01 11:18 I use restic for my laptop, but if you can run a file system that supports snapshotting natively, it's a much more efficient and broadly supported backup mechanism. I have my state stored on a ZFS volume that I snapshot and pipe to B2 (via an rclone mount). I do a weekly "full" snapshot, and then an hourly(?) incremental snapshot. The docker containers themselves should never really need to be backed up. If the containers follow common patterns and you store all of your data in volumes, you should be able to destroy the containers, pull them from a registry and have things start back up without an issue. ---------------------------------------------------------------------- atheken · 2023-11-01 11:23 I mean, snapshotting and piping it to an rclone mount is arguably simpler than trying to do your own ad hoc file syncronization, also does not require 2x the storage space. ---------------------------------------------------------------------- SamSausages · 2023-11-01 11:25 I do this at the file system level, not the file level, using zfs. Unless the container has a database, I use zfs snapshots exclusively. If it has a database, my script dumps the database first and then does a ZFS snapshot. Then that snapshot is sent via sanoid to a zfs disk that is in a different backup pool. This is a block level backup, so it only backs up the actual data blocks that changed. ---------------------------------------------------------------------- MoneyVirus · 2023-11-01 11:43 only backup of the data i need to backup (mapped volumes). Restore: create fresh container and map volumes again. ---------------------------------------------------------------------- SamSausages · 2023-11-01 11:45 https://www.reddit.com/r/selfhosted/comments/17l65qg/comment/k7cnm3g/ ---------------------------------------------------------------------- trisanachandler · 2023-11-01 12:28 Try to not run containers as root? ---------------------------------------------------------------------- DSPGerm · 2023-11-01 12:33 Have you ever restored the containers from duplicati backups? I've heard a common issue with duplicati is that their backups always seem to break. ---------------------------------------------------------------------- nyrosis · 2023-11-01 12:38 ZFS snapshots combined with replication to another box. That and a cronjob on packaging up my compose/config files. ---------------------------------------------------------------------- trexxeon · 2023-11-01 12:50 I get the gist of using podman because of the escalation thing.. But how often have anyone had a security issue with docker? I’ve never even heard of someone being rooted because of this ---------------------------------------------------------------------- cbunn81 · 2023-11-01 12:51 This is one reason I prefer FreeBSD jails. They are each in a separate ZFS filesystem, with a separate filesystem for configuration files. So all I have to do is regular snapshots and send those to a backup pool. ---------------------------------------------------------------------- SnakeBDD · 2023-11-01 12:52 All Docker containers have their persistent data in a BTRFS mount. A cronjob takes a snapshot of the volume, then calls `btrfs send`, pipes that through tar and then directly to AWS. ---------------------------------------------------------------------- sk1nT7 · 2023-11-01 12:59 Yeah that seems to be a common phrase you hear about duplicati. However, I have never had any problems and could always recover all files if needed. Even complete database backups of postgresql/mariadb etc. No issues at all. ---------------------------------------------------------------------- nerdyviking88 · 2023-11-01 13:01 I don't run containers as root. ---------------------------------------------------------------------- ButCaptainThatsMYRum · 2023-11-01 13:21 The VMs are backed up nightly. ---------------------------------------------------------------------- TheFluffiestRedditor · 2023-11-01 13:22 It's moments like this that I wish ceph had.... - I should check that - has snapshots. Shit. I need to take advantage of these. ZFS and its integrated volume management plus snapshots was one of the major reasons I loved Solaris 10. I'm so glad it's migrated to Linux. Snapshots are love. ---------------------------------------------------------------------- froli · 2023-11-01 13:23 Great idea. I already do something similar(minus the btrfs part) for Vaultwarden. Mind sharing the script/commands? I setup my host with btrfs but I have 0 knowledge of it so I didn't take advantage of it until now. I already have my docker volumes mapped to /docker/*stack* so I'm gonna create a sub-volume and move that there. I'm mostly interest in your btrfs snapshot and send commands but if you don't mind sharing the whole thing that would be great. ---------------------------------------------------------------------- DSPGerm · 2023-11-01 13:25 That's good. I used it for a short time for desktop backups and never had an issue when I tested backups but I never used it in docker itself and never used it to backup any containers. ---------------------------------------------------------------------- Bonsailinse · 2023-11-01 14:02 If you work with databases it’s still safer to stop incoming data for the time of the backup. I don’t know why a higher CPU load would be a problem, those checks don’t run long or do so much your system would be under much stress. Do your backups at 3am if you still think the minute of highe load would cause any problems. ---------------------------------------------------------------------- human_with_humanity · 2023-11-01 14:30 Do u add bothe the user and env variables or just one? ---------------------------------------------------------------------- human_with_humanity · 2023-11-01 14:30 Do u add bothe the user and env variables or just one? ---------------------------------------------------------------------- root-node · 2023-11-01 14:45 I add both because why not. It doesn't hurt. ---------------------------------------------------------------------- WiIzaaa · 2023-11-01 14:47 This. Containers are not meant to store data ! ---------------------------------------------------------------------- doodeoo · 2023-11-01 15:04 This is the only correct answer. Containers are ephemeral and stateless. If you're not mounting a volume, think of what happens if the container does during your process for exporting the data. A failure mode not possible if you mount. ---------------------------------------------------------------------- katbyte · 2023-11-01 15:29 i don't, i created a docker VM (and a couple others) and then i backup the VMs (proxmox + PBS make this very easy) with all their data in /home/docker/config/\* i used to have them run off networked storage but i found it to be to slow/have other issues ---------------------------------------------------------------------- jerwong · 2023-11-01 15:32 I use restic and ran into the same problem with root ownership. The simple answer is that you run restic as root. I have the script called by root's crontab so it will have all the correct permissions to access everything. ---------------------------------------------------------------------- epsiblivion · 2023-11-01 15:33 the documentation has instructions how to run restic without root and still backup files that are usually requiring root privilege to read https://restic.readthedocs.io/en/stable/080_examples.html#backing-up- your-system-without-running-restic-as-root ---------------------------------------------------------------------- akmzero · 2023-11-01 15:45 I mount all my containers to datasets, backup your compose files and the storage and boom, all done. Can't comment on restic, but there's no need to backup the containers or host (vm or lxc however you run it). I haven't set this up yet, but if you use ansible you can just redeploy that and put your feet up. ---------------------------------------------------------------------- sowhatidoit · 2023-11-01 15:47 Not OP, but thanks for explaining this. I've been tinkering with Docker Containers for a couple of years now and this is the simplest explanation I've come across in regard to volumes. Can the volume be just a folder in my user's home directory? ---------------------------------------------------------------------- SitDownBeHumbleBish · 2023-11-01 15:59 Yes by having something like this in your docker compose file: ./config:/config https://docs.docker.com/storage/volumes/ ---------------------------------------------------------------------- Croatwink · 2023-11-01 16:05 Just a tip, but you should try using streaming replication instead of sql dumps. That means you can finish the backup job much faster because it only streams the changes made since the last backup. So you can backup more often, and you can also make incremental backups. For postgres I use pgbackrest, and for Maria I use mariabackup, mysql has an equivalent to that. ---------------------------------------------------------------------- rocket1420 · 2023-11-01 16:19 I understand how to do what the first guy said. How do you "snapshot" and how is your solution much different? ---------------------------------------------------------------------- akmzero · 2023-11-01 16:41 I mount all my containers to datasets, backup your compose files and the storage and boom, all done. Can't comment on restic, but there's no need to backup the containers or host (vm or lxc however you run it). I haven't set this up yet, but if you use ansible you can just redeploy that and put your feet up. ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 16:49 I am only trying to backup the compose files and volumes. ---------------------------------------------------------------------- ikus060 · 2023-11-01 16:55 Adding my two cents here. Your Docker container should not contain any data worth backing up. A best practice is to mount the data into the container. For example, if you have a PostgreSQL database running in Docker, the container should mount /var/lib/postgresql/data, which contains the database data. As a best practice, I usually bind mount from /srv//. So, everything under /srv/ must be backed up by whatever tool you want to use (rclone, rsync, rdiff-backup, etc.). However, you might still want to keep track of which Docker image was used or the specific configuration of each Docker container. In case you need to recover from data loss, the configuration of your server will be valuable to recreate the same environment. Depending on your needs, you may consider managing all your server configurations using Ansible or a similar project. Alternatively, you may want to back up the Docker configuration itself, but I don't know a tool doing that. ---------------------------------------------------------------------- SillyLilBear · 2023-11-01 17:03 I am only concerned with the docker compose and volumes, I am not looking to backup the images. ---------------------------------------------------------------------- Minituff · 2023-11-01 17:06 I built [Nautical Backup](https://github.com/minituff/nautical-backup) to solve this exact problem. Let me know if you need any help setting it up. ---------------------------------------------------------------------- Minituff · 2023-11-01 17:12 Hey there, I just wanted to say thank you for using Nautical. If you get any weird behavior or want any additional features feel free to let me know :) ---------------------------------------------------------------------- root-node · 2023-11-01 17:15 Thanks, it's working so far :) ---------------------------------------------------------------------- SamSausages · 2023-11-01 17:25 Thanks for the tip, I'll have to check that out. I pretty much just did the dump because it was quick and easy to add to my script and I already knew how to do it. ---------------------------------------------------------------------- TheSmashy · 2023-11-01 17:29 sudo crontab -e ---------------------------------------------------------------------- techbandits · 2023-11-01 17:31 I use a Synology Nas with Active backup for business and back up the VMs that host the containers. ---------------------------------------------------------------------- atheken · 2023-11-01 17:34 Doing a script to "make a backup" leaks. There's dozens of reasons it could produce an incomplete/incorrect backup from a set of files. It's also pretty inefficient. You have to basically make a duplicate copy (which takes I/O), read it and compress it, which takes more I/O and disk space, and _then_ you can actually ship it off the system. If you are running a file system that supports snapshotting (ZFS, BTRFS), you can take **very cheap** point-in-time snapshots that are consistent to the moment you took them and are close to non-blocking, and they are very easy to move off the system using simple unix pipes and common file system operations. So if I mount a ZFS pool (`$POOLNAME`) at at `/data` and an `rclone mount` at `/backups`, I can run a command like this: ``` ## take a point in time snapshot of the pool (an O(1) operation) zfs snapshot -r $POOLNAME@full ## pipe that snapshot through a gzip/encryption operation and zfs send $POOLNAME@full | gzip/encrypt > /backups/$POOLNAME_full.snap ``` ZFS is copy-on-write, so the checkpoint you create with the snapshot command happens _instantaneously_, and then any changes to the files are diffs based on that snapshot. I use rclone to mount the backup destination on disk, but you can really pipe the zfs send to any command, like `s3cmd put` if you want. You can also take snapshots that are just the diff between two, or against the full pool, and you can mount snapshots under other directories, like `/2023-11-01-data/`, but it won't take "double" the disk storage. You can also do some cool stuff with software RAID using ZFS, but that's a whole other thing. ZFS itself supports some types of encryption and compression, but those are more advanced features. It's well worth your time to learn how ZFS and BTRFS can save you some admin and give you some data integrity. ---------------------------------------------------------------------- atheken · 2023-11-01 17:45 Yeah, I remember when it was experimental on _OS X_, and it's too bad it was encumbered by patents, I vastly prefer it to the AppleFS thing they made. ---------------------------------------------------------------------- rocket1420 · 2023-11-01 18:09 Right, most people aren't using ZFS or btrfs, so snapshots aren't a thing. ---------------------------------------------------------------------- gnu_man_chu · 2023-11-01 18:22 For those using ZFS this is great information to have. Thanks for sharing this level of detail u/atheken! I would also agree with u/rocket1420 as well that perhaps this wouldn't be a proper solution for database backups. Great for data though! ---------------------------------------------------------------------- icebear80 · 2023-11-01 18:22 Store your compose files in GitHub and use e.g. Portainer to deploy them as stacks. Store the bound confit and data volumes in one common place and backup from there. That’s all that’s needed normally. ---------------------------------------------------------------------- kukubaorch · 2023-11-01 18:25 Compose file is only the scheme of creating a stack of images. Images themselves are stored on docker/GitHub servers in most cases as already build or as dockerfile (template for building). If you want to "back up" the container, you can map folders to each container, so for example you can map your ~/Media on your host to /data on your container. This will be "shared" folder, so everything in mapped folders is available to both container and host at the same time. This folder is not destroyed on container rebuild as it is saved on a host device and not volatile container volume ---------------------------------------------------------------------- adamshand · 2023-11-01 18:47 Hey cool, Nautical looks great. Thanks! ---------------------------------------------------------------------- atheken · 2023-11-01 18:49 They might be different use cases. On the other hand, for SQLite, this is perfectly acceptable, and probably would be fine for any database that has a WAL. ---------------------------------------------------------------------- adamshand · 2023-11-01 18:51 Not running services as root is good practice, and "defence in depth" is always good ... but I agree with you. Especially in a homelab situation, it's mostly a theoretical concern. ---------------------------------------------------------------------- atheken · 2023-11-01 18:52 I’d have to go on a deep dive, but if the DB has a WAL, you can snapshot the volume and it should give you a pretty decent recovery point. You can probably even make the db flush writes before taking a snapshot to further improve the fidelity of the snapshot. I’m not sure “most people” are or are not using those file systems, but you can spend an hour learning how to get a ZFS pool configured or an hour writing a backup script that “leaks.” ---------------------------------------------------------------------- newbutler · 2023-11-01 18:53 this only works if the image supports one of the two methods. using the 2nd option can even break the container. ---------------------------------------------------------------------- adamshand · 2023-11-01 18:56 Didn't know about that, thanks!! ---------------------------------------------------------------------- root-node · 2023-11-01 18:56 Really? do you have an example, as mine are running without issue. The only one I know specifically that does not work like this is the [Unifi](https://github.com/jacobalberty/unifi-docker) one as it's hard-coded to user 911. ---------------------------------------------------------------------- NekoiNemo · 2023-11-01 19:00 > I don't like that the docker ecosystem is not giving me the ability to backup volumes If you use btrfs like a modern person - Docker creates a new btrfs subvolume for each volume (and image) so you can just snapshot those, without needing to stop anything. That's also probably the reason Docker doesn't have their own backup solution - why duplicate something the system itself already capable of? ---------------------------------------------------------------------- wreinoriginal · 2023-11-01 19:22 >the documentation has instructions how to run restic without root and still backup files that are usually requiring root privilege to read > >https://restic.readthedocs.io/en/stable/080\_examples.html#backing- up-your-system-without-running-restic-as-root Can I ask you why did you choose rsync over rclone? ---------------------------------------------------------------------- Minituff · 2023-11-01 19:28 I used Rsync in my original backup scripts that I adapted it Nautical. I could pretty easily add Rclone as an option until Nautical. What features would you be looking for? ---------------------------------------------------------------------- rokejulianlockhart · 2023-11-01 19:44 I use PID -1 ---------------------------------------------------------------------- throwaway53377329 · 2023-11-01 20:58 I am Root! ---------------------------------------------------------------------- Evelen1 · 2023-11-01 21:22 I do it like this https://flemmingss.com/a-guide-to-backup-with-the- appdata-backup-plugin-for-unraid/ ---------------------------------------------------------------------- tech2but1 · 2023-11-01 22:19 One of the problems with this is everyone is releasing apps on docker for ordinary people to use but they don't put any built in backup methods in or provide any simple scripts. It leads people to think backing up the container is the way to do it. ---------------------------------------------------------------------- ithilelda · 2023-11-02 00:42 if you use compose, then the docker-compose.yml file IS all you need to run a container. the only other thing is runtime data in volumes, and they live in the volumes directory under your docker root. backing up ia as simple as sudo tar cf. ---------------------------------------------------------------------- jeremy_fritzen · 2023-11-02 01:06 That's what I wanted to do but then, do I need to stop the containers before doing backing up volumes mapped to them? I suppose yes. So it means that the backup system/program needs to be aware of the running containers in order to make sure it won't backup corrupted data (in case of concurrent writing operations). Does it mean that we have to make a specific file to manage docker containers before and after performing the actual data backup? Is there a better way to take care of it? ---------------------------------------------------------------------- Vlinux · 2023-11-02 01:52 Depends on the type of service running in the container and using the volume. For databases, you can use the database backup/export features to create a backup without shutting it down. For most things, I don't bother to shut down any containers before backing them up though. Most things don't seem to be that sensitive to being backed up live. Plus, it's mostly just me using my systems. ---------------------------------------------------------------------- Vlinux · 2023-11-02 02:04 I have compose files for all my containers, and all containers have their data mapped to folders in a specific path on the filesystem. My backup process runs as root (maybe not 100% best practice, but I'm confident in my rsync script setup), so no issues with data permissions. It backs up the compose files and all container data folders. I've actually had to restore a Raspberry Pi (after the SD card died) that was running some containers and was backed up with this process, and after installing a new SD card, a quick ansible playbook run to install and setup docker, copying the compose file and data back, and starting it up, I had it all back up and running no problem. For the few databases I have, like MariaDB (for NextCloud), I'm currently just copying a backup of their mapped data folder like all the other containers. Not sure if that might cause a corrupted backup or not, but for the things I have databases for, the data would be fairly easy to reconstruct (nothing critical), so I'm not too worried about it for my use cases. Future things I'm looking into though include using some kind of distributed filesystem or docker volume plugins for remote/replicated/distributed storage as an "automatic" backup of my container data. ---------------------------------------------------------------------- DapperProspectus · 2023-11-02 02:05 Podman is the way to go. ---------------------------------------------------------------------- karitchi · 2023-11-02 07:54 A simpler method would be to stop/start the Docker daemon instead of containers, it works smoothly. ---------------------------------------------------------------------- Not_your_guy_buddy42 · 2023-11-02 08:11 I've been on a selfhosting binge the past week and trying to work my way towards best practice. At least for the container databases I've been starting to use [tiredofit/docker-db- backup](https://github.com/tiredofit/docker-db-backup) (does database dumps) but also discovered this [jdfranel docker backup](https://gitlab.com/jdfranel-docker/db-backup) as well which looks great as well. I save the dumps on a volume mounted from NAS. btrfs and there is a folder replication (snapshots) tool. So far, so good. Where I am now stuck: I want to keep the docker volumes, as bind mounts, also on my NAS share. If the containers run as a separate non root user (say 1001) then I can mount that share as 1001... sounds good right? But somebody suggested running each container from their own user. But then I would need lots of differently owned directories. I wonder if I could keep mounting subdirs of the same NAS share, as different users, so each of them can have their own file access? Perhaps that is overkill. ---------------------------------------------------------------------- boomertsfx · 2023-11-02 08:44 I wish docker used zfs datasets for volumes, but it doesn't :( ---------------------------------------------------------------------- root-node · 2023-11-02 09:04 > ...running each container from their own user... Ideally this is the perfect option from a security standpoint, this as well as each container having it's own network too. In a homelab it's not really required unless you are exposing your network to the internet or are better at creating/managing containers. If you are just starting out, just keep everything simple. ---------------------------------------------------------------------- webbkorey · 2023-11-02 09:55 My docker configs and stuff are all /home/user/containername I then backup my whole user folder. Some of my apps also have scheduled backups, and those go to another folder on my Nas. ---------------------------------------------------------------------- Not_your_guy_buddy42 · 2023-11-02 10:34 Thanks for your reply and for giving me some perspective! I at least tested it anyway and happy to learn I can mount a bunch of NAS subdirs of the same share as different users, with dir\_mode=700. Rolling out the stacks with ansible so perhaps I do have a chance of automating the user-per-stack thing. ---------------------------------------------------------------------- root-node · 2023-11-02 11:12 Have a look at https://github.com/minituff/nautical-backup, it does a similar thing ---------------------------------------------------------------------- Not_your_guy_buddy42 · 2023-11-02 11:24 PS i figured out how to do it in ansible but you are still right lol, better keep it simple first and figure out rootless docker but with one user ---------------------------------------------------------------------- atheken · 2023-11-02 12:20 What would that do for you rather than pointing a volume at a local directory that is in a ZFS mount? FWIW: Docker _can_ use ZFS for image storage, so if you have tons of images that use the same base image, you can save some space there, though I don't know that it's worth it for most situations. ---------------------------------------------------------------------- boomertsfx · 2023-11-02 12:21 Yeah it's really bad for image storage. For volumes it could create a new dataset and perhaps do snapshots when you upgrade images, etc ---------------------------------------------------------------------- McGregorMX · 2023-11-02 13:07 I have my config and data volumes mounted to a share on truenas, that share replicates its snapshots to another truenas server. This is likely not ideal for everyone, but it works for me. My friend that also uses docker has it backed up with duplicati. ---------------------------------------------------------------------- wreinoriginal · 2023-11-08 17:48 Sorry for the late reply, I missed this thread. Thanks for the reply anyway! I was considering the best solution between rsync and rclone, I've seen that the latter is often mentioned in backup workflows, but I don't have a clear idea yet so I don't have any specific requests. Then I found your solution very interesting and, thinking of adopting it, I was wondering what was the reason for choosing rsync (which I have used often, while rclone not yet). I think it would be useful to be able to choose whether to use one or the other depending on the type of remote server to be used.