Suggestions for a robust backup system reddit / selfhosted ---------------------------------------------------------------------- _Burgr · 2023-11-03 05:27 Hello, I'm looking for some tips on where to start for building my own server. I currently have an adequate machine (right now running Linux Mint but that can change, as there's not anything on it at the moment) that I plan on using and I primarily want it to be a backup system. Right now as it stands, more of my data is either saved on my primary desktop, in OneDrive, or in Google Drive / Google Photos. What I'm looking for is to configure server to do scheduled jobs to copy data from OneDrive and Google Drive / Google Photos and save it. Not sure how to handle the OneDrive side of it, but a Google Takeout for my Drive and Photos data is something I'm looking for I'm planning on uploading most of my things to OneDrive and having my own server take care of automatic backups (yes I'm thinking of having my files saved in OneDrive for convenience, I just want a local backup system) I have no prior experience with configuring a server for this kind of thing, but I'm relatively tech savvy and know my way around computers so I'm just looking for some suggestions / ideas / best practices here. ​ I plan on setting up my machine in some sort of RAID configuration (I'm thinking of RAID 5 or 6?) However, in practice I've never actually set up a machine using RAID before, is it a hardware specific thing? (Which I'm willing to go do) or is it something that can be done in software? (Is there perhaps a preferred / simple Linux distro I can use?) For reference, the specs of machine I plan on using as a server are: * CPU: Intel i3-10100F * Motherboard: ASRock H510M-HVS Micro ATX LGA1200 * RAM: Kingston HyperX Fury 16 GB (2 x 8 GB) DDR4-3200 CL16 * SSD: Kingston 240GB A400 (I got this for cheap I'll probably replace it) * And I plan to buy some extra hard drives for, y'know the actual storage part Any feedback is appreciated ---------------------------------------------------------------------- troywilson111 · 2023-11-03 05:50 Great project! Home lab/server is an awesome tool to have running at your disposal. Linux mint is more for a desktop than a server. Pick any enterprise class Linux like Ubuntu is an easy start. As for a software RAID, ZFS is super simple and is easy to install from apt-get install and super easy to setup your drives with. Keep your backup simple if you are going to use multiple cloud services for backups. Take a look at rclone.org. Great backup/sync software with active development for years. Rclone is a beast and scalable beyond what you will ever use. You can write up a simple bash script with a few rclone one liners in it and you can have a fully encrypted and automated backup in no time. This will get you a nice start. Then you can setup a reverse proxy and run any service you want from home depending if you have an internet cap. DM me if you would like. ---------------------------------------------------------------------- j0hnp0s · 2023-11-03 09:53 As troy has said, you can use rclone to sync files to cloud services. Or you can use syncthing to sync between machines This should protect you from hardware failure. Most likely, you do not really need raid in a home setup. It's an extra point of failure and it only offers uptime. It makes sense for commercial services that do not afford even brief downtime windows. But at home it does not really matter most of the times. You only need working backups. What you have not planned for, is protection from mistakes. Meaning, what happens if you delete something accidentally and do not notice it until a month later? What you need to plan for, is some archiving mechanism to keep a few weekly or monthly copies to protect you from yourself ---------------------------------------------------------------------- GolemancerVekk · 2023-11-03 10:30 Hardware raid means having a dedicated PCI card that does the RAID. Pro: very good performance, and it has a built-in battery that makes sure operations always finish properly in case of power outage or machine crash. Con: the RAID configuration is proprietary and if your card ever breaks you most likely need to replace it with the same card or a card from the same vendor to access your disks. You manage hardware RAID from a special BIOS that you enter at boot (not the regular BIOS for your motherboard). Software raid is done exclusively in the operating system. For Linux this means a driver such as MD or ZFS. Pro: software raid is super flexible and complex and can do tons of things that hardware raid can't, it's also simple to access your disks even if you change Linux distribution completely because the driver is the same. Con: potentially subject to corruption in cases such as OS crash or power outage (if you don't use an UPS). ZFS attempts to address this problem as well with its RAIDZ models but it's more complex to use than MD. You manage software RAID from Linux using command-line tools. There's also a thing called "fake RAID" or "hybrid" that's actually software but also proprietary, and usually comes embedded in the motherboard chipset. You manage fake RAID from the regular motherboard BIOS. I don't recommend this because it has all the worst from both worlds: it's proprietary but very simplistic. Obligatory warning about RAID: it's about availability, not backup. Think of it like something that keeps your machine in operation continuously. If a disk crashes you can continue to work until you replace it. But it won't help you if you accidentally modify or delete files. If your main goal is backup I would question the need to build a machine for it. You can simply connect backup disks to whatever your current workstation is, copy files to them, then store them in a drawer. You don't need a server running 24/7 for that. There are backup tools that are very easy to use and also very efficient. Personally I use Pika Backup, which is a very friendly UI built on top of Borg Backup. I connect a backup HDD over USB, fire up Pika, select the correct backup matching the HDD I plugged in, and tell it to do another backup. Thanks to Borg it will only actually copy the files that are new or changed, so the files that are unchanged won't take space twice. This is called deduplication. It can also do compression (to reduce space taken) and optionally encryption (if you want to protect the backups with a password – but be careful with this if you're gonna want to access the backups 10 years from now). Please don't use SSD for long term backup, SSD can lose data if left unpowered for long periods of time. HDD's are cheaper per MB anyway. You don't have to buy an "external drive" for backups, they are more expensive and often not worth it. You're paying mostly for the brand and for a fancy enclosure (the box). You can buy an USB-to-SATA adapter (check out Axagon for example) and plastic waterproof boxes for the HDDs (IcyBox and LogiLink make some nice ones but there's no shortage of them). Don't trust a single HDD for all your backups, use at least two in parallel (with the same exact backups), and buy different brands or different models (to avoid the case they're both from the same batch and have the same defect). You can buy pretty much anything from Seagate, Toshiba or WD, just get whatever is cheapest and the right size you want. Most of the price differentiation for HDDs comes from their ability to run 24/7, how hot they get, how much noise they make and other stuff that only matters at runtime and you won't care about if you only do a backup and then put them back in the drawer. What I do is run backups like I described above with 2 HDDs in parallel, then when they fill up I get another 2. If you think you really need a server and RAID I can assist with that too.