How do you store important global data for self hosted projects/containers? reddit / selfhosted ---------------------------------------------------------------------- anunknownmind · 2023-09-30 07:50 When I started working with self hosted projects I saved important commands/paths/login data in a single plain text file on a NAS. The NAS and the server for my self hosted projects are different devices. So in case my server breaks I still have all information required to rebuild it at a different place. I now have so many docker containers that the text file got pretty big which makes it really hard to find the right information when you need it. How would one store and style this in a more structured way? ---------------------------------------------------------------------- j0hnp0s · 2023-09-30 07:58 I have a git repository for each project And I have my credentials on a password manager ---------------------------------------------------------------------- nik_h_75 · 2023-09-30 07:59 Logins in a plain text file? I recommend vaultwarden as a password manager. As for commands as such - I just keep stuff in a text file (but it's not that complicates/long) :) ---------------------------------------------------------------------- anunknownmind · 2023-09-30 08:01 Since I already host GitLab I was thinking about this as well but what happens when my server (where GitLab is hosted on) goes down? ---------------------------------------------------------------------- VTi-R · 2023-09-30 08:11 Clone the repo somewhere else. Either offsite to a mate's place, or github (just don't store private keys in it) etc. I think Azure DevOps is private and free for up to 5 users or something, if Github private isn't free? ---------------------------------------------------------------------- anunknownmind · 2023-09-30 08:12 I know saving logins in a plain text file is dangerous but since the server is not exposed to the internet and a password manager would add another layer to maintenance this is a rather minor issue in my situation. The text file however usually consists of these entries **per project**: * Login Data & Tokens for Web Interfaces * Database-Logins * Important internal file paths (config locations) * Backup file paths * Important commands for individual services within the containers * IP-Addresses and Ports With about 20 Projects in total this has gotten a litte bit out of hand :/ ---------------------------------------------------------------------- redoubledit · 2023-09-30 08:14 I find the password manager to be the single most used service for me. I am hosting Vaultwarden for a backup but am paying for 1password. So no added maintenance per se. ---------------------------------------------------------------------- nik_h_75 · 2023-09-30 08:16 Aren't you using docker compose files? By backing up your compose files (and persisted volume data) you can always restore somewhere else. ---------------------------------------------------------------------- Stetsed · 2023-09-30 08:21 So I use docker-compose.yml files for all projects which are documented inside of my bookstack wiki for all apps. Currently any non-large datasource will be stored inside of the VM and anything that is larger is done via NFS(Linux ISO collection). However soon I will setup GarageHQ as an S3 compatible backup solution to actually start having backups replicated across my servers and one on a VPS. ---------------------------------------------------------------------- anunknownmind · 2023-09-30 08:21 I already considered cloning to a private GitHub account but feared the possibility of a human error (aka some mistake I could make when committing) which would lead to devastating consequences. Is it easy to set a private GitHub repo to public by accident? ---------------------------------------------------------------------- anunknownmind · 2023-09-30 08:29 Yes, docker compose files get backed up, but not the whole persisted volume data. For backing up data inside containers I use the native commands of the individual services and clone their output to the NAS. However, not all projects are hosted within docker containers since I developed a bunch of projects by myself. ---------------------------------------------------------------------- Murky-Sector · 2023-09-30 08:47 >Is it easy to set a private GitHub repo to public by accident? Not at all. You set the account at private and never have to worry about it. Its also very secure. They actually scan for things like keys and send you an alert to remind you to fix it. ---------------------------------------------------------------------- usmanatron · 2023-09-30 09:52 I think it's worth asking yourself if it's worth self hosting something like this. Password management is the one thing I don't self host because I think the fallout should it break is too great. So I've paid for a bitwarden account and I use that for all my credentials. On the other hand, If you use something like vaultwarden and just ensure you backup regularly to somewhere very safe, then that should be enough to recover from a disaster. My advice though is to be pragmatic and consider using other services where it makes sense to do so. For example, my restic backups are stored (encrypted!) in Azure Storage. If my whole thing falls over then at least I'm covered and I consider that an acceptable risk. (Sorry for the tangent!) ---------------------------------------------------------------------- [deleted] · 2023-09-30 10:39 [deleted] ---------------------------------------------------------------------- rocketmonkeys · 2023-09-30 13:41 I use a dedicated note taking app for this type of stuff, in my case Joplin. And while the Joplin server is self-hosted as well, all information is available on any of the client devices, PC, phone, even if the server is down ---------------------------------------------------------------------- maximus459 · 2023-09-30 13:59 3-2-1 backups.. (where possible) Have 3 copies of your data.. - On-site, online: The data your love services are using - on-site, offline: some backup in the same site in a different media that can be accessed of something happened to the main services. This can be an external hard disk, NAS, etc.. - off-site, offline: this should be in a different location than your main servers. Maybe git, or Google drive, S3 etc.. ---------------------------------------------------------------------- pigers1986 · 2023-09-30 14:56 all compose files + password vaults are stored in two places , vps and dropbox ---------------------------------------------------------------------- ericesev · 2023-09-30 15:27 Depends on the data: For domains, host names, IP addresses, config paths, etc. I put them in ansible-vault encrypted files within git. These are then substituted into the template config files when ansible runs. For encryption keys or passwords, those are pgp encrypted. Each host has its own key. The keys/passwords are encrypted using both the host's key and my key (on a Yubikey). I have these files listed in .gitignore, so they're not saved to git. For Docker containers I pin them to a specific version. When I upgrade the version I'll snapshot the ZFS data volume with the old version number. That way I can always restore the data along with the version of the software used to create the data. I also store a copy of each Docker image in a private repo. My docker-compose files reference the private repo rather than the origin so it can always be retrieved from there. For backups, I use rclone with encryption to store data on a cloud provider. This saves an encrypted copy of my git repo (including any uncommitted files) as well as saving the current data volume for each docker container. The encryption key, and config, for rclone is encrypted with my PGP key and stored in cloud storage. I do the same for TOTP seeds and recovery codes. I do the same for backups of passwords in my password manager. And the same for the master password for my password manager (it's too long to remember). I have three Yubikey devices with the same PGP encryption key (siging keys are different). That avoids losing access to the data if a Yubikey breaks. The PGP master key is stored offline. The Yubikey requires a password and a touch before allowing use of the key. ---------------------------------------------------------------------- AmIBeingObtuse- · 2023-09-30 18:52 I use Memos to store all my compose/stacks and then either important information It's kind of like Twitter but for notes not for social posts. https://github.com/usememos/memos Highly recommend ---------------------------------------------------------------------- ZaxLofful · 2023-09-30 20:08 I use Gitea and NetBox SoT, then have them backed up to Azure geo- replica. You could also use a VPS or offline physical copy for restore in a disaster scenario. ---------------------------------------------------------------------- YYCwhatyoudidthere · 2023-09-30 20:45 The old advice is the best advice. ---------------------------------------------------------------------- CodenameJackal · 2023-09-30 23:03 This might be blasphemous and on this subreddit but I’m too deep in OneNote to switch to anything else. I have a Bitlocker for passwords and OneNote for all documentation. ---------------------------------------------------------------------- JuryDesperate2699 · 2023-10-15 13:31 Aren't you hosting you self made projects in containers? ---------------------------------------------------------------------- anunknownmind · 2023-10-16 05:53 No, because I simply don‘t know how to turn them into containers ---------------------------------------------------------------------- JuryDesperate2699 · 2023-10-20 19:34 I would suggest you to invest an hour in it. You will save days with it.