Idea: backup BSD and other *nix clients not supported directly by Veeam using Bacula as an intermediate reddit / BSD ---------------------------------------------------------------------- icantthinkofone · 2019-08-19 10:07 I've been using FreeBSD for 15 years on a daily basis for both servers and workstations. This is the first time I've ever heard of Veeam. I still do not care to learn anything about it. ---------------------------------------------------------------------- jdrch · 2019-08-19 12:43 Help me come up with a device-level, incremental, automatically consolidated bare metal restoration backup solution for FreeBSD, then? ---------------------------------------------------------------------- qci · 2019-08-19 15:20 In most cases you don't want to have any automatisms to restore an entire installation. This allows you to have the server up and running before everything is restored and also restore on disks that have been structured differently. Every BSD uses dump utility for backups and restore for restoring. This is one of the best backup solutions you'll see. Exception is ZFS that unfortunately does not have dump, but uses zfs send / recv with snapshots. You have everything. Just use it. ---------------------------------------------------------------------- dlangille · 2019-08-19 17:51 I don't do bare metal restores. I backup the the data, not the OS. Dead box? Reinstall the OS, install the packages from the repo, restore the config files, restore the data, done. I see no value in backing up stuff which is easily obtained: OS & packages. ---------------------------------------------------------------------- jdrch · 2019-08-19 18:30 > you don't want to have any automatisms to restore an entire installation I beg to differ. My systems are set up so that they're amenable to that. > you to have the server up and running before everything is restored Modern software technology allows you to do all at once. There's no technical need to restore OS, user data, etc. separately if you're restoring to the same hardware config the device backup was made from. > restore on disks that have been structured differently. I consider that a different use case entirely, which is why I do have the filesystem backed up. Implementation details [here](https://www.re ddit.com/r/DataHoarder/comments/cq171c/blueprint_and_progress_status_f or_mixed/). > dump IIRC `dump` doesn't support consolidation, which forces the OS to make full dumps every once in a while. Modern backup software does automatic consolidation that creates full backups from existing incremental backups, thus cutting down on source block device I/O. > This is one of the best backup solutions you'll see Yikes. As an active user of all 3 major OS paradigms and having researched and evaluated various solutions on each in my own mixed environment I'd have to strongly disagree on that. Compared to most enterprise backup/restore/recover solutions `dump` is incredibly lacking. The difference between `dump` and them, though, is that `dump`'s userbase seems complacent with its features while the userbases of other solutions realize more is and should be possible and demand it. `zfsnap` is great, though, especially combined with the ability to browse said snapshots from within Insight on Lumina DE. No other DE has that natively, AFAIK. I actually [wrote Project Trident's documentation](https://github.com/project-trident/trident- docs/blob/master/zfssnap.md) on how to get started with it. > zfs send / recv with snapshots `zfs send` is not a bare metal recovery solution. Or, at least, it's not described as such in its documentation. Could you kindly point to a tutorial or guide showing me how to restore an entire system - OS, user data, and packages - to bare metal from a recovery or boot environment using it? FWIW [Bareos seems to be the ticket](https://www.reddit.com/r/freebsd/ comments/cscgi2/idea_backup_bsd_and_other_nix_clients_not/exem6pa/). ---------------------------------------------------------------------- jdrch · 2019-08-19 18:49 > I don't do bare metal restores. Suit yourself, my good man. > easily obtained: OS & packages So in other words recovery is impossible without an internet connection? 😕 ---------------------------------------------------------------------- qci · 2019-08-19 20:40 Please read the man page of dump. It supports incremental backups and restore does not care about the target file system. zfs send also supports incremental backups. I am able to recover bare metal system from dump and zfs send. I have done it many times and never worried about any single aspect of the recovery I have been performing (see below what I did recently). And the most important thing for me is that I don't need to worry about my restore procedure. I have almost never wanted recovery on same disk configuration, but mostly on larger/better drives. It doesn't make sense to buy old drives again. You basically always want to improve something. Can you recommend me a tool that backups a geli-encrypted pool using ssh, compressing then encrypting the backup images safely with AES. And then restoring the datasets on 2 drives with 2 pools while the first one is holding ZIL and L2ARC for the second one, using ssh, decompression, configuring data for 2 copies and transparent compression, creating a swap and beadm-compatible layout and encryping the home partition with pefs and modifying PAM to integrate it? Thanks ;) ---------------------------------------------------------------------- jdrch · 2019-08-19 20:48 > incremental backups You know, every time I discuss `dump` with someone from the BSD community I always mention I'm looking for backup consolidation and then they *completely* ignore that want in their reply. `dump` does not consolidate backups, which is problematic for reasons that ... I'd hope I don't have to explain to a technically proficient audience. > does not care about the target file system. Same for zfs send. Where did I say I need my backup solution to not care about the target? If I didn't say that, then why are you recommending a feature I didn't ask for ... > I am able to recover bare metal system from dump and zfs send. I have done it many times and never worried about any single aspect of the recovery I have been performing (see below what I did recently). And the most important thing for me is that I don't need to worry about my restore procedure. This is why I asked for a tutorial/guide. Do you have one? > never wanted recovery on same disk configuration, but mostly on larger/better drives. It doesn't make sense to buy old drives again. You basically always want to improve something. You realize that modern backup solutions allow you to extend the volume the restored filesystem is on to fill the entirety of a new, larger drive (or even shrink it if necessary) so that isn't a concern, right? Different drive size hasn't been a problem for modern filesystem restoration for a *very long time* now. > recommend me a tool that backups a geli-encrypted pool using ssh, compressing then encrypting the backup images safely with AES. And then restoring the datasets on 2 drives with 2 pools while the first one is holding ZIL and L2ARC for the second one, using ssh, decompression, configuring data for 2 copies and transparent compression, creating a swap and beadm-compatible layout and encryping the home partition with pefs and modifying PAM to integrate it? What point are you trying to make? My needs aren't the same as yours, and so no, I can't recommend one to you because I don't know of one. *But at the same time* I'm not gonna try and sell you on a solution that doesn't meet what you asked for either, because I have enough respect for you as an operator to think you've carefully considered your use case and know what's necessary for it. I asked for help with my chosen solution, but **note that I did not try to evangelize that solution to you**. I just asked for help with it. Ignoring what someone asked for and pushing your ideal that doesn't fit their needs is unhelpful and arrogant. If you think what I'm asking for is impossible, then say so. But don't waste users' time by telling them about solutions that do ABC when they specifically asked for XYZ. ---------------------------------------------------------------------- dlangille · 2019-08-19 21:15 > So in other words recovery is impossible without an internet connection? No, impossible without a network connection. The packages and OS are available locally. ---------------------------------------------------------------------- dlangille · 2019-08-19 21:15 > No, impossible without a network connection. And even then, you could do all that from a thumbdrive. ---------------------------------------------------------------------- jdrch · 2019-08-19 21:16 > The packages and OS are available locally Oh. Well that's different. Do you have a script that mirrors the repositories every so often, or what? ---------------------------------------------------------------------- dlangille · 2019-08-19 21:17 > Oh. Well that's different. Do you have a script that mirrors the repositories every so often, or what? Or what. poudriere builds all packages locally. ---------------------------------------------------------------------- jdrch · 2019-08-19 21:18 > poudriere builds all packages locally Aaaack. That's incredibly involved. Good that it works for you, though. ---------------------------------------------------------------------- dlangille · 2019-08-19 21:20 If you ever want a package which is not compiled with the default configuration, poudriere is your best bet. All the /var/db/ports/... options I think it is, have the default configuration settings you want. It really is nice to have the patches compiled locally overnight without waiting for the build servers to catch up. It also provides considerable independence, both from a build and security point of view. ---------------------------------------------------------------------- qci · 2019-08-19 21:25 I just wanted to give you a hint that one day after making a backup with a tool that might be broken or not available, you might have a lot of headaches to restore. Backups need to be simple because the restore procedure might be needed in case of emergency where panic and stress might be a problem. I decided against Bacula and earlier Amanda, because I wanted to be able to know exactly how the backup images are constructed. I didn't want to show off my skills. I wanted to show you that it's important to navigate out of every situation while restoring from a backup collection that is carefully prepared to solve most things that might happen. One time I had my server flooded by rain and could not get the RAID controller anymore. I had to restore the company server within half an hour on entirely new hardware. It proves to be a solid concept again and again. I don't really know what backup consolidation is. I just use a naming schema like: hostname-filesystem.date.level.deltalevel.gz.aes. I can tell from the name what's inside and what the steps are to recover the filesystem. ---------------------------------------------------------------------- jdrch · 2019-08-19 21:30 > which is not compiled with the default configuration Gotcha, makes sense. That's not my use case, but I can definitely see where you're coming from. To me, that's all the more reason to do device-level backups (to avoid package rebuilds if you don't have to do them) but do you! ---------------------------------------------------------------------- dlangille · 2019-08-19 21:32 In my case, the packages are always available. No rebuild necessary. poudriere does an 'svn up' every night and rebuilds any of the packages on my list. ---------------------------------------------------------------------- jdrch · 2019-08-19 22:46 > poudriere does an 'svn up' every night and rebuilds any of the packages on my list. That's pretty cool. Definitely a workflow I appreciate.