Backup question reddit / selfhosted ---------------------------------------------------------------------- InvaderToast348 · 2023-12-09 23:57 I am re-evaluating my backup solution, and was wondering if you guys have any thoughts? I have a macbook and a desktop PC running ubuntu desktop. Inside the desktop, I have a big hdd where all backups go, and then that is copied to an external usb hard drive. My current solution is to share a folder on the desktop pc using SMB, then connect to it on the macbook and use freefilesync to backup. This system works ok, but is painfully slow; it can take many hours to copy 2-10 gigabytes (of modified & new data) - mostly because it is a lot of small files (a few kilobytes each, 10k+ files) as my laptop is primarily used for software and web development. I tried switching to NFS, but it had strange issues where it wouldn't detect all files so the backups would be incomplete. I am also considering getting a usb-c to ethernet adapter which would hopefully dramatically increase the speed if directly connected to the PC via ethernet. I have so far looked at SMB, NFS and FTP / SFTP. I have also looked at running these services in docker so that I can access them remotely using my current VPN and RP setup; also, all of the benefits of containerisation - except, I don't know if it would have a significant impact on the transfer speed. ​ So, the question: What method / protocol /... do you use to backup device(s) to a central server / PC over the network? ​ I would really like to stick with freefilesync as the backup software if possible; however, if another software has support for a faster protocol I would definitely consider switching. I would prefer open-source if possible, but I will research all suggestions. Any contributions are welcome, I just need a little push in the right direction. This community has always been really great, I very much appreciate all your help. ​ BTW, sorry if this is the wrong sub - please kindly let me know. I assume it fits here because I am selfhosting everything, but if there is a dedicated or better-fitting sub it would be great to know. ---------------------------------------------------------------------- R3AP3R519 · 2023-12-10 03:56 If you put the macbook in target disk mode you can use thunderbolt to plug into the desktop and mount the macbooks disk directly as a thunderbolt ssd. It's very fast. Then you can use rsync or probably freefilesync. [https://wiki.archlinux.org/title/rsync](https://wiki.archlinux.org/ti tle/rsync), You can also use rsync over SSH if needed. ---------------------------------------------------------------------- NinjaFragrant7710 · 2023-12-10 09:01 You could try Veeam Backup Agent for windows, linux and mac. It is free, and as a dedicated backup software it has a lot of advantages. For example it backs up only what has changed since last backup, encryption, compression etc. It can backup locally to an smb share and i believe it has cloud options as well. For the best performance make sure you are using a wired connection (at least 1G) and that your target storage is fast enough to write the data. Also use off hours to backup. ---------------------------------------------------------------------- InvaderToast348 · 2023-12-10 10:27 Thank you. I did come across target disk mode, but unfortunately my motherboard doesn't have the pins for a usb c connector. I'll do some research on btrfs and rsync over ssh, thank you for the suggestions. ---------------------------------------------------------------------- InvaderToast348 · 2023-12-10 10:31 Thank you, all good points. I'll have a look for a usb c ethernet dongle I mentioned, maybe that's just the bottleneck. If I remember correctly, I think I have tried (or at least researched) Veeam, but I'll have another look. ---------------------------------------------------------------------- martin4233 · 2023-12-10 10:49 Take a look at restic. It supports backing up to filesystem and supports SFTP and is quite fast. ---------------------------------------------------------------------- InvaderToast348 · 2023-12-10 10:50 Thank you, I'll add it to the list :)