Root Filesystem Backup Software reddit / selfhosted ---------------------------------------------------------------------- sssRealm · 2023-12-05 05:38 I see lots of recommendations for backup software on here, but do any of them support backing up a root partition without taking it offline? I have a command line Debian server want to do regular backups of the root drive to a dedicated SMB NAS on the same network. The server has Apache running on it, so I don't want anything to conflict with that. I've spent days with rsync with many flags, mount flags, attributes, but I never can get it to replicate all permissions correctly. I just want to backup root, I don't care about saving the boot sector, etc. I know how to make it bootable if I had to do disaster recovery. What do you recommend? ---------------------------------------------------------------------- ttkciar · 2023-12-05 05:42 rsync ---------------------------------------------------------------------- sssRealm · 2023-12-05 05:49 Have you used rsync to restore a root partition from a SMB share and have it work correctly? ---------------------------------------------------------------------- discoshanktank · 2023-12-05 05:49 Proxmox backup server has an agent you can push to bare metal servers too ---------------------------------------------------------------------- sssRealm · 2023-12-05 05:59 Does it require a Proxmox server? ---------------------------------------------------------------------- lilolalu · 2023-12-05 06:17 I think the common strategy in Linux backups is not to make a "root" backup or disk image, but backup all the files you need to recreate that system but from a fresh install. - /etc for the settings - a list of all installed packages https://askubuntu.com/questions/9135/how-to-backup-settings-and-list- of-installed-packages The OS but also other services have a lot of open files that are accessed every single moment for write operations, think about it, how should a backup tool make a working copy of a file that's currently being written to? ---------------------------------------------------------------------- sssRealm · 2023-12-05 06:25 Good point. I could probably be good with /etc and /var. Do you have a recommendation for backup software? ---------------------------------------------------------------------- lilolalu · 2023-12-05 06:27 Very easy and effective, just relies on rsync: rsnapshot More features (encryption, deduplication): borg I personally like the companion "Borgmatic" to use with Borg, it simplifies the setup process a lot ---------------------------------------------------------------------- gryd3 · 2023-12-05 06:28 Yes. Also used it to migrate a server from one box to another. ---------------------------------------------------------------------- sssRealm · 2023-12-05 06:31 Really? Well, I can't even get it to duplicate the 700 permissions of /root folder ---------------------------------------------------------------------- BeYeCursed100Fold · 2023-12-05 08:08 proxmox backup server (pbs) does not require proxmox virtual environment (pve). Side note: PBS does not have a windows client currently. ---------------------------------------------------------------------- vogelke · 2023-12-05 08:56 This assumes utilities that handle null-terminated filenames, like GNU find and tar. root# cd / root# find . -xdev -print0 | sort -z > /var/tmp/root.toc root# tar --no-recursion -b 2560 --null -T /var/tmp/root.toc \ -cf /var/tmp/root.tar The sort's not needed, but if I ever have to look at the list of files, sorting makes it more convenient. root# tar tf /var/tmp/root.tar | head -5 ./ ./.rnd ./COPYRIGHT ./bin/ ./bin/[ root# tar tf /var/tmp/root.tar | tail -5 ./var/unbound/ ./var/yp/ ./var/yp/Makefile ./var/yp/Makefile.dist ./zroot/ If you have lots of files changing on the fly and you're using ZFS (or something that can do snapshots), take a snapshot first and run this from the snapshot directory instead of /. ---------------------------------------------------------------------- lynx769 · 2023-12-05 11:01 I use restic for everything. You can put the repo darn near anywhere you want (NAS, AWS, Backblaze, etc) either natively or using rclone. [https://restic.net/](https://restic.net/) ---------------------------------------------------------------------- Unix_42 · 2023-12-05 11:51 I use dump/ restore on \*BSD systems. Locally and over ssh to a backup host. I use tar where dump is not available. Example: `$ ssh user@server "cd / && doas tar -cpvz -f - /" > backup.tar.gz` \-c Create new archive, or overwrite an existing archive, adding the specified files to it. \-p Preserve user and group ID as well as file mode regardless of the current umask. \-v Verbose operation mode. \-z Compress archive using gzip. \-f - Read from or write to archive. A hyphen (-) can be used to represent standard input or output. The command above executes tar on the server. The "-f -" option tells tar to send the output to the standard output instead of a file. In this case the standard output is being sent through the SSH session and then piped to dd which writes the data to the local disk. TLDR: Make full backup from remote server onto local system. ---------------------------------------------------------------------- daYMAN007 · 2023-12-05 14:32 timeshift with a mounted remoute harddrive. It uses rsync under the hood but should be pretty easy to setup. ---------------------------------------------------------------------- sssRealm · 2023-12-05 15:07 >times Does Timeshift work in CLI? ---------------------------------------------------------------------- Cylian91460 · 2023-12-05 15:08 You can use dd. WARNING TEST IN A VM BEFORE YOU CAN DELETE YOUR ENTIRE DISK. You can use dd if=/dev/rootdrive of=where/to/backup ---------------------------------------------------------------------- daYMAN007 · 2023-12-05 15:09 Yes it has a cli tool, although it will probably pool all the x dependencies while installing. ---------------------------------------------------------------------- sssRealm · 2023-12-05 16:19 ooooh, I'm going to try out Timeshift! It looks like it supports BTRFS snapshots. My /var is BTRFS, I have 3 GB on root and 8 TB on /var ---------------------------------------------------------------------- sssRealm · 2023-12-05 16:49 Timeshift is not a backup program. It's like Windows System restore. ---------------------------------------------------------------------- CGA1 · 2023-12-05 17:24 Also, Vorta is a very nice GUI for Borg. ---------------------------------------------------------------------- daYMAN007 · 2023-12-05 18:29 But it creates a complete backup of your harddrive... Which makes it a backup program? ---------------------------------------------------------------------- sssRealm · 2023-12-05 18:30 "User Data is Excluded by Default Timeshift is designed to protect system files and settings. It is NOT a backup tool and is not meant to protect user data. Entire contents of users’ home directories are excluded by default. " ---------------------------------------------------------------------- sssRealm · 2023-12-05 23:18 >borg Does borgmatic work with version 1 or 2? ---------------------------------------------------------------------- lilolalu · 2023-12-05 23:20 Docs say "First, install Borg, at least version 1.1." ---------------------------------------------------------------------- gryd3 · 2023-12-06 05:00 I tend to use rsync with -avxHAX \-a includes a lot of thing, but not ACLs (A) or xattrs (X) . \-v to get a some feedback \-x for 'SingleFileSystem' to stop it from traversing other mounted drives, tempfs, etc.. rather than building an extensive 'exclude' list. \-H for hardlinks. That said... 'how' /root is created does matter. rsync /root/\* /newroot / newroot will be created with default permissions, and contents of root will be copied to it. rsync /root/ /newroot/ newroot will be the copy of root (including permissions) rsync /root /newroot/ newroot will be created with default persmissions, and 'root' will be copied into it. /newroot/root \*While newroot has default permissions, the root subdirectory created inside will copy the permissions. ---------------------------------------------------------------------- ghoarder · 2023-12-06 10:54 The way I do it (which by no means is the best way) ​ * All apps are a Docker Container * All docker containers are installed with docker compose * All docker volumes are mapped to two places, /opt for config mounts and /mnt for large data mounts. * Main data volume is BTRFS raid 1 * Backup data volume is BTRFS raid 0 * Daily Snapshots are taken using BTRBK * Snapshots are streamed from the main volume to the backup volume * 7 daily Snapshots are stored on main volume * 14 daily snapshots, 10 weekly and as many monthly Snapshots as storage permits are stored on the Backup volume * A monthly backup is sent to Gsuite with rclone * URBackup (as a container) is setup to backup some other SBC Arm devices ​ To me this works well because recovery is a piece of cake, just need to rebuild a server, install docker and run my compose files. Snapshots are fast being instant on the main data volume and only the delta COW being copied over to the backup volume. I can easily reclaim resources if I'm not using some software for a bit by downing the container, e.g. OnlyOffice uses 20% of a CPU core constantly even when idle. ​ Why backup as raid 0 and main as raid 1? Partly I want the main volume to be highly available and if one drive dies the volume should carry on while rebuilt, plus cost constraints. I started with the main array as the only one but wanted a true backup not just raid 1, but couldn't afford to buy enough storage to run raid 1 with space for snapshots so I went with raid 0, this allowed me to get a backup volume that is twice the size of the main one, if I had the budget it would be raid 1 or more like raid1c3. ​ Issues and downsides, it's easy to shoot yourself in the foot with BTRFS if you don't know what you are doing, and sometimes even if you do know what you are doing. Not all apps are dockerised, I have built my own Dockerfile for some apps that aren't but that's added hassle (to be fair I've tried to make them highly efficient, I could probably just start with an Ubuntu base and apt install the apps not worrying about image size.) ​ The future? An offsite box I can steam BTRFS snapshots too as my rclone setup is slow, more storage and raid1 for the backup volume. When I win the lottery. ---------------------------------------------------------------------- sssRealm · 2023-12-06 18:20 Thanks for the input everyone! For me there was no shortcut, I spent many hours grinding through different software. Looks like I'm settling on a combination of Borg, Rsync and a script to backup a MariaDB database. Basically it's Borg for system/config files and Rsync for large files. It doesn't fit my stated goal of complete root backup, but I feel prepared if SHTF