Fire. Everyone is safe, it was just the garage...but I've got water damage..... reddit / selfhosted ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 01:25 Detached garage burnt to a crisp last Wednesday, everyone made it out with no harm, but the fire fighters blew out my windows on the first floor and also flooded the basement. HP Proliant DL360 took an indirect hit, stakes the front of the server. Raid 1 configuration. Three of seven drives went out. Looks like the boards got water damaged and corroded. One of them actually rusted pretty badly. The server itself never shut down or power faulted and i didnt notice anything was amiss until the next afternoon. I tried to clean up the boards on the hard drives and was able to get 2/3 to display green lights, but the server will still not boot fully. Error PXE-E61..media fault/bad drives. What's worse is that it looks like there is a bit of classic green corrosion on the hdd bay contacts inside the server itself. I will attempt to pull the whole thing apart and clean up those mounting points with a light toothbrush, isopropyl, and some Q tips. I am prepared to source new boards for the drives (the disks do not appear to be scratched or locked up) Can anyone suggest some helpful techniques that might bring my baby back to life? Am I SOL until insurance comes through with a new box? ---------------------------------------------------------------------- thekrautboy · 2023-12-02 01:33 Sorry for your bad luck... but this honestly has nothing to do with selfhosting software. Try subs like /r/Homelab /r/HomeServer etc instead. ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 01:39 Good places to crosspost mate. Good idea. As for it pertaining to self hosting though...I mean... I assume most people don't run containers on their refrigerators? ---------------------------------------------------------------------- Psychological_Try559 · 2023-12-02 02:31 Give it time, but maybe it'll be a toaster? ---------------------------------------------------------------------- Psychological_Try559 · 2023-12-02 02:33 Sorry to hear about the fire, but glad everyone is ok. Looking forward to hearing about what works and what doesn't when it comes to repairs of the drivers. ---------------------------------------------------------------------- Abe677 · 2023-12-02 03:24 Good reminder about the hazards of self -hosting. ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 03:47 Onsite backups for most of the sensitive things, but the minecraft servers and other home networking things are borked unless the drives can be recovered. But yes- good reminder because those backups would be worthless if the entire house was now charcoal. ---------------------------------------------------------------------- [deleted] · 2023-12-02 03:47 [removed] ---------------------------------------------------------------------- ephemeraltrident · 2023-12-02 04:03 As far as homelabs go… I have an AP and a switch on my fridge ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 04:28 i mean, okay i guess you could literally run a container On TOP of your fridge X-D ---------------------------------------------------------------------- Blaze9 · 2023-12-02 04:49 I don't think you can ever be truly self-hosted unless you own property in different physical locations and can sustain/maintain two physical servers for backup. I have a main server that has a ton of data on it, most of it I can obtain again. But for the personal data like photos, databases, container configs, appdata... No other choice than to backup to a 3rd party solution. I chose backblaze B2. Fully automated via duplicity. One copy goes to B2, another copy goes to a local server in a different room. My main apps (Vaultwarden, PhotoPrisim, Jira, etc) are backed up every 5 hours. All other data is backed up nightly. ---------------------------------------------------------------------- thekrautboy · 2023-12-02 05:18 > As for it pertaining to self hosting though...I mean... I assume most people don't run containers on their refrigerators? Thats not the point. This sub is explicitly about **selfhosting software services**, not about any hardware at all. Check the sidebar and the wiki. By your logic every sub about software would automatically be also about hardware, and vice versa too. But that simply doesnt work. Of course some subs are about both. ---------------------------------------------------------------------- burps_up_chicken · 2023-12-02 05:47 The folks over in /r/DataHoarder know quite a bit about storage and recovery as well. ---------------------------------------------------------------------- Lukebekz · 2023-12-02 06:07 > I assume most people don't run containers on their refrigerators? I hope they now do ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 06:54 Sidebar? Wiki? What are you on about? ---------------------------------------------------------------------- thekrautboy · 2023-12-02 06:59 Subreddits have sidebars, the menu you can see on the right side. They contain infos about that sub like what its about, rules and important links etc. Here it also has a wiki. ---------------------------------------------------------------------- omgpop · 2023-12-02 07:08 10 year old account w/ 40k karma who doesn’t know how subreddits work? ---------------------------------------------------------------------- VexingRaven · 2023-12-02 07:35 Better reminder of the hazards of not following 3-2-1 for backups. ---------------------------------------------------------------------- VexingRaven · 2023-12-02 07:38 There's absolutely nothing counter to the spirit of self-hosting to use a trusted provider where needed. You still control your data. You can encrypt it if you want. Self-hosted isn't about who owns the physical hardware. It's about having control over your own data and the applications you use and not being at the whim of some corporate CEO. Even if backblaze closes up shop tomorrow, you can easily just start uploading your backups somewhere else. Heck even if you are hosting your servers themselves somewhere, as long as you control the data you can easily just go somewhere. You still have control. ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 08:10 > I don't think you can ever be truly self-hosted unless you own property in different physical locations What ---------------------------------------------------------------------- dunkelziffer42 · 2023-12-02 08:48 With an E2E encrypted backup mechanism it might actually be possible with a single location. But you at least need physical control over all places with running servers that touch unencrypted data. ---------------------------------------------------------------------- Bean86 · 2023-12-02 08:49 That's what family and old hardware is for - at least in my case. I moved my 10 year old Synology (5bay) to my mother once I replaced it with my own NAS at home. I already had an old RPi3 running there. I only used it for the very important stuff so it's sitting idle most of the time and barely using any energy. ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 08:56 Oh, in regards to backups? I thought you meant you needed two physical locations for self hosting in general ---------------------------------------------------------------------- dunkelziffer42 · 2023-12-02 08:58 Aren’t backups a part of self-hosting „in general“? ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 09:05 Sure, but I'm wondering why they said, "I don't think you can ever be truly self-hosted unless you own property in different physical locations". Self hosting has no such requirement ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 09:11 I honestly don't care, but they're referring to this in the subreddit's description > This subreddit focuses on the software that is considered to be self-hosted, regardless of where it is being hosted. ---------------------------------------------------------------------- uberduck · 2023-12-02 09:14 You can always add more redundancies. The benefits of self hosting. /s ---------------------------------------------------------------------- RobinBeismann · 2023-12-02 09:22 Or have friends with similiar hobbies. I'm backing up my VMs to a friend via proxmox backup server, obviously encrypted and he is backing up his VMs to me. Fileshare data goes directly (encrypted) onto Hetzner Storagebox via sftp using the inbuilt TrueNAS cloud sync. ---------------------------------------------------------------------- sysop073 · 2023-12-02 09:37 The rest of the sentence that you keep cutting off was: > and can sustain/maintain two physical servers for backup. ---------------------------------------------------------------------- alex2003super · 2023-12-02 09:42 I mean, pretty much all smart refrigerators run some flavor of Linux. The Samsung Family Hub uses a customized version of Tizen OS (based on GNU/Linux), which I guess could very trivially run some OCI containers if somehow you managed to gain root access to it. It would be quite funny to try. ---------------------------------------------------------------------- Daniel15 · 2023-12-02 09:52 Any good backup system is E2E encrypted. I use Borgbackup and their threat model is based on having a trusted client with an untrusted server. It can use both a keyfile and a password for encryption. ---------------------------------------------------------------------- FuriousRageSE · 2023-12-02 10:42 >One copy goes to B2, another copy goes to a local server in a different room. Hope the "different" room is accompanied by ".. atleast 8000km away" or something. ---------------------------------------------------------------------- DyceFreak · 2023-12-02 11:50 Correct me if I'm wrong, but I don't think **OP's selfhosted software** is working and maybe he wants to talk about it here? >regardless of where it is being hosted. His software doesn't work. After some troubleshooting, it was determined to be a hardware issue. If you want to argue technicalities, it's still decently relevant to sub you gatekeeping chode. ---------------------------------------------------------------------- Spielopoly · 2023-12-02 12:40 There is no sidebar in the app. (Technically there is but it’s kind of hidden and you have to click on a few specific links to access it which most people don’t) ---------------------------------------------------------------------- schmerold · 2023-12-02 12:42 I have been wanting to monitor my smoke detectors for some time now, but was too cheap to give Ring $200 per year. I bought a X-Sense system from Amazon, each module is priced similar to standalone modules, however the monitoring is $50 per year, my insurance is covering $44 of the cost. Worth checking out! ---------------------------------------------------------------------- OppositeFisherman89 · 2023-12-02 13:13 Sure, but I'm wondering why they said, "I don't think you can ever be truly self-hosted unless you own property in different physical locations and can sustain/maintain two physical servers for backup". Self hosting has no such requirement ---------------------------------------------------------------------- thekrautboy · 2023-12-02 13:46 > There is no sidebar in the app. Thats no excuse tho. ---------------------------------------------------------------------- amunak · 2023-12-02 13:46 It's not a hazard of self-hosting, it's a hazard of having any data, period. Even huge "cloud" providers sometimes have blunders that lead to data loss (or at least severe outages) for their customers. Nobody is immune to that, and that's why literally everyone needs a solid backup strategy if they care about their data at all. ---------------------------------------------------------------------- persiusone · 2023-12-02 13:57 Physically separated backups are important. If you don't own multiple properties, consider a family member or good friend to let you drop a backup server at their location. Fire can suck, but the water hazard can be the more likely threat, especially if you have a suppression system or similar. Make sure you have a kill switch for your UPS systems and power or some way to ensure power is totally cut off. ---------------------------------------------------------------------- SamSausages · 2023-12-02 14:04 And why the the 3 (offsite copy) in 3-2-1 backups is so important! ---------------------------------------------------------------------- vnangia · 2023-12-02 14:24 Yeah, I was thinking exactly that. I've got all of the irreplaceable stuff backed up to a remote Synology at a friend's place in the same urban area as me, but only a small subset of the irreplaceable stuff backed up to my mom's place an ocean away because of space considerations. Need to think about it some more... ---------------------------------------------------------------------- microlate · 2023-12-02 14:26 If you have insurance then write up your servers as well get new ones ---------------------------------------------------------------------- Neldonado · 2023-12-02 15:00 I’m using duplicity for local backup but want to dig into b2 as well. Do you create new backups or can you do one backup to two locations? ---------------------------------------------------------------------- sysadmagician · 2023-12-02 16:09 You can't compete with OVH by copying their infra design :) Glad you are all safe! ---------------------------------------------------------------------- VexingRaven · 2023-12-02 16:44 Pretty sure it's the 1 that means offsite. 3 copies, 2 different types of media, 1 offsite. ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 16:45 Yes, but the data is more important and not something insurance can write me a cheque for ---------------------------------------------------------------------- SamSausages · 2023-12-02 16:49 Whatever way it is, it’s 1 of the 3 copies 😂 ---------------------------------------------------------------------- Necronotic · 2023-12-02 17:26 What causes the fire was it lab relates or not? Sorry if you've mentioned this already, but I didn't see it. Also, I hope you can recover most, if not all, of your data, and I'm very glad to hear no one was injured. ---------------------------------------------------------------------- h0w13 · 2023-12-02 17:27 Exactly, there are plenty of simple solutions. This shouldn't be a deterrent for anyone looking to self host. Backblaze is another great option. My parents are unknowingly hosting my off-site backups on a 2-bay NAS I plopped in their house years ago 😀 ---------------------------------------------------------------------- campr23 · 2023-12-02 17:36 I use the cloud for backup. Backblaze R2. Use the AWS S3 synchronisation tool. First 10Gbytes free. ---------------------------------------------------------------------- Fluffer_Wuffer · 2023-12-02 17:47 Ditto, I brought a load of Tuya / Smartlife detectors a few days ago, and seperate CO and gas detectors etc.. I feel a bit guilty from the reviews are hit and miss, but the exact models I got are CE marked, so we'll see when they arrive. ---------------------------------------------------------------------- Saint_Ferret · 2023-12-02 18:46 Fire was not lab related in any manner, in fact I am quite unlucky that the server was damaged at all; the water from the fire hoses busted out a window on the house and soaked the first floor allowing water to pour down on the area where the lab is set up in the basement. Also had the ONT shorted out, but the rest of the network (switches, routers, and AP) all unscathed. Server may be repairable to some degree...that will require break down and corrosion remediation if possible. ---------------------------------------------------------------------- TheBoatyMcBoatFace · 2023-12-02 21:19 Go buy 91% rubbing alcohol and go to town. Pour that shit over everything and brush with a tiny brush. Literally pour it on the circuit boards. I had a server go swimming and was able to get it running great. That Dell r710 is still running 6 years after the swimming incident. ---------------------------------------------------------------------- Saint_Ferret · 2023-12-03 00:59 Was able to get 2/3 drives to show green lights instead of no lights, and the third showing an amber fault. But I've got to tear in to the server and clean the ports on that side as well. I'm mostly concerned about any corrosion that might be under the jumper where I cannot scrub. Short of desoldering those joints? Any suggestions? Hopes and prayers? ---------------------------------------------------------------------- Necronotic · 2023-12-03 13:25 Wow, that is rather unlucky. Hope you can recover and repair the stuff damaged, and nothing like this happens again! ---------------------------------------------------------------------- kmisterk · 2023-12-05 17:10 ## Post Removed Hey, Saint_Ferret! Thanks for your submission on selfhosted. Your post has been removed due to a violation of the purpose of the subreddit: **Self Hosting** ## Not Self-Hosted When it comes to posts regarding applications in this subreddit, they must feature a self-hosted tool, or a tool that *can be* self-hosted, or some kind of related information, help request, or otherwise related to a tool that is something that one can self-host. If you believe this post *is in fact* relevant to /r/selfhosted, feel free to [message the mods](https://www.reddit.com/message/compose/?to= /r/selfhosted&subject=Removed For Not Being Self- Hosted&message=[Removed Post](https://www.reddit.com/r/selfhosted/comments/188s3yd/-/\)) with your concerns.