Why is starting with Self-hosting so daunting? reddit / selfhosted ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:02 I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree. I’ve developed applications and hosted them through docker on Google Cloud for school projects. I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot) Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting. Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up. Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running! ---------------------------------------------------------------------- Salty-Masterpiece-31 · 2023-10-26 11:11 Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs. ---------------------------------------------------------------------- Perpetual_Nuisance · 2023-10-26 11:12 It isn't..? ---------------------------------------------------------------------- -quakeguy- · 2023-10-26 11:13 Hosting different things, let alone hosting them that results in a somewhat secure and user friendly endpoint access can indeed get complicated pretty fast. Which is why that is also an actual career path where people get paid good money to implement it. ---------------------------------------------------------------------- Svidthekid · 2023-10-26 11:19 Check YouTube. I haven’t found a lot of written out guides but someone has made a video on it, I bet. Christian Lempa, Techno Tim, Db Tech, NetworkChuck come to mind for self hosting tutorials on containers and new applications. I’m sure there’s a lot more as well. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:19 I actually hate GPT, dislike it’s answers and find myself knowing better than it most times. I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple. I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now… ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:20 Well, clearly I’ve been having problems with it, and disregarding that is just plain rude. Maybe you can offer why you think it isn’t or just not say anything at all? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:22 Thank god someone gets it. I guess I do have the bad habit to do everything from scratch, fully understand what’s going on, and since I’m using my own home network for this, I’m quite concerned with doing everything securely haha. So, from what I gather in your comment, I should just focus on broader knowledge and hope for the best? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:23 Thanks for the suggestions! The algorithms keep feeding me people who just explain what stuff is and it drives me nuts. You wouldn’t believe the amount of videos, articles and blogs I’ve seen on setting up a DNS server just for it to be about either a cache or an explanation of how it works. I’ll look into these later! ---------------------------------------------------------------------- Salty-Masterpiece-31 · 2023-10-26 11:28 "Selfhost authorative dns" returns this for example https://wiki.selfhosted.show/DNS/ And we have two guides for the most common dns implementantations. Depending on the vpn software you can push the dns ip as part of the configuration or by using DHCP. I also found the pihole docs for unbound rather helpful https://docs.pi-hole.net/guides/dns/unbound/. ---------------------------------------------------------------------- lupercal93 · 2023-10-26 11:37 Jeff Geelong is another good resource in youtube, especially if you ever want to get into infrastructure as a career. Github is also an amazing resource and RTFM. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:37 For me, I have that as the 4th result, after some Reddit and IBM which probably would’ve discouraged me from continuing my search. I’d have to read on it. Also, TIL PiHole doesn’t necessarily need to run on a Raspberry Pi. I guess assumptions really do come back to bite me in the ass haha ---------------------------------------------------------------------- deeohohdeeohoh · 2023-10-26 11:37 I use Pihole for this in my home network. It has the capability to configure local domains for your internal network resolution as well as create whitelist/blacklist and use as a DHCP server alternative to your home router -- you just need to configure your router to point to the Pihole instance for DNS on your LAN network configuration to get started with the DNS piece ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:39 Reading the docs is my way usually to learn anything, only resorting to other sources when the docs aren’t clear or I don’t know which docs I should be reading haha ---------------------------------------------------------------------- FuriousRageSE · 2023-10-26 11:42 >Most blogs and articles I find are about what they are which I already know. Exactly.. I neither want "what z is" nor all the history behind z, the Z inventors life, his grand parents lives etc. I just want a solution to the problem im having.. ---------------------------------------------------------------------- issa62 · 2023-10-26 11:45 Use unraid ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:45 Honestly, I get they’re trying to be educational for beginners way more clueless than me. But after two years of an IT degree I know some stuff, and the sheer amount of internet text I’ve read just to find absolutely nothing new and no solution even though the title is exactly my problem is unreal ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:46 What’s that even? And how does that solve anything? ---------------------------------------------------------------------- NeverMindToday · 2023-10-26 11:46 >I guess I do have the bad habit to do everything from scratch, fully understand what’s going on, and since I’m using my own home network for this, I’m quite concerned with doing everything securely haha. Seeing that you're a software engineering student, that ~~bad~~ habit will make you a better engineer eventually. I reckon you have extra incentive to stick with it and push through the pain barrier vs someone just trying to save a few bucks and take some shortcuts. ---------------------------------------------------------------------- -quakeguy- · 2023-10-26 11:47 I'm very similar. I never feel I can truly understand how something works unless I've spent time building the same thing on my own, entirely from scratch and having fallen into every single bottomless pit on offer during the process. First few years it can be absolutely brutal and very mentally draining, but I can say with certainty that it absolutely does works and simply gets easier over longer periods of time. ---------------------------------------------------------------------- revereddesecration · 2023-10-26 11:50 Pi runs Raspbian which is just Debian with customisation applied. So of course it can run elsewhere. You don’t know as much as you think you do perhaps 😉 ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:52 Hahahahahaha, don’t know if it’ll make me a better engineer haha. I despise web development because the sheer amount of necessary shortcuts and abstracted complexity in the modern landscape make me deeply uncomfortable! ---------------------------------------------------------------------- froid_san · 2023-10-26 11:56 I've studied automotive and never really graduate, but I love reading manuals since I was a kid and always believe that even I don't know what am I doing as long I can read It's manual, guide, documentation and read support I could get a grasp of it even if it takes more time. A part that made it easier for me to understand self-hosting is the raspberry pi. It let me thinker self-hosting without worrying about power bill, how can I easily reinstall its OS even if I make bunch of mistake, make different copies of my setup on different sd card without worries of hdd or ssd. Then from that just upgraded stuff from what I've learned. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 11:57 I seriously thought it was a product, rather than software tbf. The name always sounded so “corporate” I never considered it. I definitely know more about the theory than the practice. I’m clueless as to what my options even are so I can’t argue with that. But I did know about the Linux “inheritance” of distros if you wanna call it that, and I’m fully aware of what that entails. Just honestly didn’t look at it twice cause I thought “there must be an FOSS option” without realizing what PiHole really is. Just a case of prejudice biting me in the ass I guess. ---------------------------------------------------------------------- Krieg · 2023-10-26 12:01 Invest some time in understanding Docker containers and then install something like Portainer (for example on top of Proxmox + VM, or ProxMOX + LXC, or some Linux or bare metal) and then installing new apps will become very trivial once you understand how to "create" the containers and how to map the volumes and ports. ---------------------------------------------------------------------- revereddesecration · 2023-10-26 12:02 Sounds like the next step in your journey is combing through this list and seeing what’s out there: https://github.com/awesome- selfhosted/awesome-selfhosted So much great stuff! But most of it has drawbacks, like missing features or less attractive UI. But it’s free and open source so we love it all the same. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:03 I wish I could afford a Pi. Would be so cool. Unfortunately I’m stuck with my gaming PC from 6 years ago that I recently updated from. Much more powerful hardware but I can’t just swap out the drive or not worry about power usage sadly haha. Still, I’ve had to reinstall the OS about 8 times last year alone haha, but we’re still learning (most because I forgot the password tbh haha ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:09 I’m sadly familiar with Docker, and documentation around it sucks. Half of it is presented as magic, and I need some technical info as to how it works to fully grasp it. Took me months to figure out what layers are, how to create a container, and pass it environmental variables. Volumes still escape me technically even though I know how to make one. Proxmox is just a hypervisor right? Never really looked into much of how that stuff works, but this is exactly what bothers me so much about Selfhosting. People online present things as solutions to problems I didn’t even have to begin with. I know you also offered the possibility of running on bare Linux, but VMs solve issues I haven’t even come across, and trying to set one up is yet another rabbit hole to go down, which is precisely why I’m so frustrated with Selfhosting. Nothing against you personally, I know you’re trying to help haha, but I felt like this precisely exemplifies what I’m complaining about haha. I just can’t leave things be, I need to fully dive into and understand whatever I set up so I guess it’s my bad habit hehe ---------------------------------------------------------------------- Jealy · 2023-10-26 12:10 It's somewhat bittersweet to be in a position further along than yourself, don't take it for granted. I sometimes wish I was still in the infancy of my journey... discovering, learning, implementing new things. Don't get me wrong, you're always discovering and learning, but some of the base technologies are fire & forget (except, don't forget!). Once you get into the swing of things and everything starts to come together, both in your lab and your mind, it'll feel great! ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:11 I’ve read that repo a million times! My self-hosting needs are more esoteric and I mostly play around with it. I’ve no need for media services or 90% of what that repo offers yet! I mostly want to end up self-hosting my own apps, but I need some foundational knowledge ---------------------------------------------------------------------- AllisonIsReal · 2023-10-26 12:13 There's a lesson in here somewhere about patience. Get good at skimming. I was looking for how to do something I was unfamiliar with the other day and I had to sift through 15 results across four different search strings before I found the solution that was going to work for me. But because I'm good at skimming it only took me 1/2 hour to discover and implement. Google isn't magic and this is why someone else recommended Chat GPT to help with some of the sifting, especially early on. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:14 Hahahahahaha, I guess I just gotta push through it then! ---------------------------------------------------------------------- Rorixrebel · 2023-10-26 12:18 As one of my favorite anime guy says. Learning and finding new stuff is exhilarating. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:19 I usually go through way more than 15 results, but in vastly higher amounts of search strings. Usually if Google doesn’t turn up what I’m after in 1-2 results, I’ll try something else or go to Reddit, Youtube or StackOverflow to search instead, since I find more often than not 3rd result onwards things become decreasingly useful. I obviously ignore the “Sponsored” results as well, and generally that works out quite well for my other purposes. But Selfhosting is kinda the exception here. I guess I just gotta get better at skimming. But I’m banned for some mysterious reason from ChatGPT sooo ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:19 It’s also overwhelming and exhausting ---------------------------------------------------------------------- InfamousAgency6784 · 2023-10-26 12:19 > at a theoretical level Theory + experience => skill I say this because I am not sure how much of what you find "rough" comes from you not being familiar with it and how much comes from real hurdle (not that getting familiar with a subject is not a hurdle in itself). --- > Most blogs and articles I find... Those are nice to get an idea but come on, read the doc. Do spend that time. Unless the service was extremely well designed and small in scope, getting a serviceable mental map of how it works can take days (or weeks if you really have no clue). There is no real shortcut to this. If the doc is not good enough, either walk away or engage with the dev/community to get things right. It might be the first time in your life where you are confronted to having to get things perfectly right for a service to work at all (I say this neutrally, without meaning to offend) but this is how it is: this is what "work" means. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:24 When I say at a theoretical level I mean I’m familiar with it from University lectures and reading about what it is, but it is true I’ve never actually tweaked my networking in a practical sense enough to be familiar with it, which is exactly why I want to get into self- hosting. As for the docs, I read them, I truly do. But docs are not where you find how to do something, is where you find how to implement it. By this I mean, if I wanna setup an authoritative DNS server, I need to find how I set one up. Once I know what software I need to use, I can read the docs to figure out how to wield said software. Just stuck on the step before being able to dive into the docs (or stuck on having too many docs to read, no middle ground) ---------------------------------------------------------------------- Krieg · 2023-10-26 12:24 Have you been running and configuring the containers per hand? To me what makes everything easy is some orchestation, I use Portainer. When I want to install a new app I just go to their Github and they will have an example of their Docker Compose file, I copy paste it and put it in Portainer as a "stack", set the volume how I want it and that is, I launch the container. If you already spent the time understanding the basics then Portainer should be relative easy to grasp. ​ P.S., volumes is just a mapping from a directory in your host machine to a directory in your container. So that your data does not die when your container stops. ---------------------------------------------------------------------- Jealy · 2023-10-26 12:26 Defo, I remember originally getting my head around what containers where and being confused... and I've worked in IT for over a decade (albeit in a Windows environment). ---------------------------------------------------------------------- Woke_killa · 2023-10-26 12:32 I implement everything with the help of chatgpt without problems. Of course, as the service on github has a lot of issues I skip it. ---------------------------------------------------------------------- InfamousAgency6784 · 2023-10-26 12:35 > As for the docs... No, they are. All the other resources are shortcuts to get a proof-of- concept out there more quickly. Actually the best-written software bring you to PoC stage from the get-go. E.g. for OpenSSH, just install the package and start the server. Now it works and you can use the rest of the doc to do what you want. --- > if I wanna setup an authoritative DNS server, I need to find how I set one up Yes, so you read the doc. That's a pretty unfortunate choice for an example because [bind](https://bind9.readthedocs.io/en/v9.18.19/) has excellent doc, walking you through an [introduction to DNS and Bind itself](https://bind9.readthedocs.io/en/v9.18.19/chapter1.html), then [what machine you need to run it](https://bind9.readthedocs.io/en/v9.18.19/chapter2.html) and then [how to get your PoC](https://bind9.readthedocs.io/en/v9.18.19/chapter3.html). The rest of the doc is all about how to wield it correctly, as you said. I am aware that some software does not document nearly all of that but the vast majority brings you to PoC state without trouble before you can tune things. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:49 Well, you’ve proven my point. In order to know how to setup an authoritative DNS server I need to read the docs for bind. But in order to know bind is the answer to my problem I need to read articles and blogs. There is no way to go from Authoritative DNS server to bind without reading some more on the internet in blogs and whatnot. Once I know about bind, I can read it’s docs to set it up or to figure out if it’s the right thing for me, but I need to know about it first. I only ever use something other than the docs when I’m either looking for something more specific than the docs, the docs suck or I can’t find it in the docs. Really not against reading through them at all. But with a lot of programs that’s also an issue cause a lot of docs just expect you to be familiar with that area of knowledge (at least with some libraries I work with such as Spring in Java, which assumes constantly you know about HTTP and APIs when explaining how to set an HTTP API with Spring. Not saying it’s bad, you probably need that background knowledge anyway, and the doc writers cannot be bothered to bake it into the docs, but it gives people who are completely clueless like me more and more homework in a snowball that becomes quickly unmanageable) ---------------------------------------------------------------------- rayjaymor85 · 2023-10-26 12:50 You're kidding? I'm an avid home-labber trying to get into software engineering and I'm finding the software dev work to be super daunting lol ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:53 I guess I could try looking into Portainer, but I’ve been managing my limited Docker experience manually. Usually building the images myself too many of them for my own apps or apps that don’t support Docker (been trying to dockerize myself a Minecraft Server for a while). ---------------------------------------------------------------------- Ieris19 · 2023-10-26 12:56 It comes with experience I guess, I’ve got a bad habit of researching to the core and many times have a hard time grasping things like containers without understanding how it’s setup technically. Sometimes I find a decent explanation, but specially for libraries that do “magic” I gotta go diving into the source to understand what’s going on, else I have trouble understanding what I am doing and what I should be doing. Which makes it so hard because networking is very low level and I’m very unfamiliar with this environment ---------------------------------------------------------------------- CactusBoyScout · 2023-10-26 13:03 Yeah I’m trying to learn Authetik SSO for my self hosted apps right now. It’s by far the most confusing aspect of this hobby that I’ve attempted so far. But it’s fun learning. ---------------------------------------------------------------------- Deses · 2023-10-26 13:14 Skill issue. It was pretty easy for me. ---------------------------------------------------------------------- RobertBobert06 · 2023-10-26 13:17 Because that means nothing? "Why are computers hard" "Why is car hard" Maybe figure out what you're trying to do? "I can't figure out VPNs" is a pretty weird starting point considering you can just google VPN and click literally anything ---------------------------------------------------------------------- InfamousAgency6784 · 2023-10-26 13:20 > But in order to know bind is the answer to my problem I need to read articles and blogs. Yes and no. Either you have the experience/knowledge to know what you need (e.g. "I need an authoritative DNS server to solve problem X"), in which case it all comes down to "what are my options?", which any search engine will gladly answer (and the doc will detail without ambiguity whether or not it's a good fit). Or you don't, in which case you either need to build that knowledge or you walk away. Blogs and articles get stale very very quickly and very often, they are not written by competent people. In the grand days of "host your own mail server", this very thing has led to so many open relays that ISPs and server providers started to block SMTP by default, with convoluted steps to unlock it. See last paragraph. --- > a lot of docs just expect you to be familiar with that area of knowledge Yes. It's just like a mechanics does not like to read literature explaining for a 1000th time how an engine works or a surgeon who loathes having to go through an anatomy lesson every time they try to read content. If you don't know what you are doing, learn about it first. Often, a wikipedia article and a couple more random reads go a long way towards understanding these kinds of things. Also with time, your area of soft expertise expends. I have never had to configure bind as an authoritative DNS, yet I know what DNS is, roughly how it works and how to navigate to the right places to get the specific info I need quickly. That's what experience brings. --- > but it gives people who are completely clueless like me more and more homework in a snowball that becomes quickly unmanageable Why do you think people have do different trades and learn about these kind of things at Uni level? That's the point of any degree or education system actually: you build an understanding, lesson after lesson, year after year, just to be able to understand/manage what you are going to be taught next. If you/I need to perform heart surgery tomorrow, your/my patient will undoubtedly die. Or phrased differently, it gives people who are completely clueless at heart surgery, like me, more and more homework in a snowball that becomes quickly unmanageable. That's how knowledge works. And as I've been repeating (again, without offense), you either learn about it, step by step, or you do something else. These things are not easy and getting competent/skilled at them requires work. --- I might be very wrong about what I am going to say, and again, this is not a slight, I don't mean to offend anyone, but it looks like, so far, you were able to find articles and blog posts about most of the stuff you wanted to achieve. So you've been mostly following tutorials. It's rewarding: you get things working. But the problem with said resources (besides going stale quickly or not being competently done) is that they are made, originally (before it's copied 100s of times by copycats), by people who took the time understanding how things work and do the hard work for you. Now that you want more bespoke or niche things, you seem baffled that no one had written a convenient tutorial for you, even to get yourself started. But the "tutorial world" is an illusion. The best way of doing self-hosting with minimal amount of frustration is by getting up to speed with all that background knowledge first or as you go, this is how you go beyond "tutorials". I am aware this is not how you phrased it but tutorials are the only way "clueless" people get something working. Most things out there are not written for clueless: they assume a minimum amount of knowledge. Finally I want to reiterate that I am not judging you. I stand by what I said: it is hard and not everyone want to spend time getting experts at those things. It's fine. Conversely, if you really want to and stick with it, I am certain you can achieve that level of expertise and, in a couple years, maybe, look at that post again and think "I really got upset over little there but it's true it looked insurmountable back then, good that I stuck with it". :) ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:22 Want a shiny gold star for that? ---------------------------------------------------------------------- Perpetual_Nuisance · 2023-10-26 13:23 Don't you think you should give at least \*some\* relevant information? With your knowledge and experience, I wonder not alone what could be daunting about self-hosting for you, but also why you still haven't learned to ask [smart questions](http://www.catb.org/~esr/faqs/smart-questions.html)? "*I find self-hosting a Plex server and suite of \*arrs daunting*" is VERY different from "*I find it daunting to securely make self-hosted apps public-facing using self signed certs.*" One problem is not identical to the other just because they both occur on a computer. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:27 While this post is a bit of a generic rant, I do know what I’m trying to do, and the issue is that as soon as you go into anything slightly more complex than setting up a VPN, you’ll be bombarded with a thousand words that barely mean anything, everyone and their mother has a different opinion on what’s optimal, minimal and desired and to top all of that, most resources out there focus on making you understand what things are rather than how to set it up. My issue and what I was ranting is why is most shit on the internet so unhelpful, hoping to find someone who’s had a similar struggle and learn how to get better. And I’ve succeeded, many people have given me useful advice. I never said anything remotely as vague as “computers are hard” I think my post clearly states my issue is with resources being unhelpful for complete beginners ---------------------------------------------------------------------- indykoning · 2023-10-26 13:27 The responses you get here are from people that have already perfected their setup so take them with a grain of salt. I mean I have my setup running with Traefik, Crowdsec, Authentik and that's before the request even reaches the application. 2 months ago that was only Traefik. A year before that I was using Nginx Proxy Manager instead of Traefik because it was easier to manage and understand. Half the fun is evolving your homelab. Trying to start out with the full stack of things someone suggests is daunting and nearly impossible. Take things one step at a time. And honestly if you don't understand what the documentation is talking about, YouTube videos are great. I've had to use it lots to understand how Authentik works but now I understand the docs ---------------------------------------------------------------------- Rorixrebel · 2023-10-26 13:31 This is the right attitude to have towards self hosting ---------------------------------------------------------------------- Qxt78 · 2023-10-26 13:35 As with everything in life you need to crawl before you run. And as someone mentioned before skills issue. You can assume most people who self host are Linux engineers / Devops engineers. So they do these kind of setups daily. This is why these channels exist. So we can assist or guide. So you can learn as well. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:39 I wanna preface this making something clear, I have probably never managed to follow a tutorial. I usually have an idea and try to run it. What I’m getting stuck at is precisely at the “what are my options” I did say that I understand why docs expect you to be familiar with knowledge, I’m even complaining that I don’t need to read what something is a million times and sometimes I just need to get a solution to my problem. My biggest issue here is lack of experience, but after two years in an IT education, I know some basics. I am familiar with countless topics and have a general idea of how things work in theory, but getting these ideas into a setup is what’s hard. Uni doesn’t help at all with this. I think my problem is quite honestly the opposite of what you present, I need to know deeper knowledge than what’s enough and have trouble actually conceptualizing things that are presented as magic. For example, Docker presents itself as magic in most of the docs. Volumes, layers, and so much more are explained with how to make one and what to use them for rather than what they are. It might just be the way I memorize things is weird, or that I’m stuck with too little knowledge and way too deep. I do mostly agree with your comment though, I just feel like I’m shit at explaining myself cause I’m clearly not getting my point across ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:45 What I find daunting is how unhelpful most of what I read is, perhaps I didn’t make that clear. It’s the whole thing, it’s about my inability to get started because the documentation is so inaccessible. I’ve studied in uni for two years and can barely understand some shit I read, because for every layer I peel off and understand, 6 more are revealed under. Virtualization lead to containers and VMs, which leads to Hypervisors and Orchestration, which lead down a path exponentially growing. But I can’t just setup a container and call it a day, because I’m using my own home network and people really make a point about how dangerous public facing services are, so that sends me down the rabbit hole of how to do it securely. This is mostly a rant about how overwhelming and inaccessible self- hosting’s been for me, and looking for advice to overcome that, more than a problem in need of fixing with my specific setup. Because I’m stuck for the same reason in ALL my projects, not just one of them ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:47 Thank god someone gets it! Biggest problem I have is how to even get started. All the advice I read is about things I do not care about, or things way more complex than they should be for me as a beginner. For example, someone else in this thread talking about learning Docker better suggested I run Promox and a VM with Portainer to use containers which I have a vague notion of what that means and is completely overkill for me. Yet it’s so hard to find that middle ground between knowing what the fuck I’m onto, but not having a perfected setup ---------------------------------------------------------------------- Ieris19 · 2023-10-26 13:50 I’m a Software Engineer in the making too, but it’s like no middleground exists. When I try to learn either stuff is way more complex than I’m ready to handle or way too fucking simple. It’s like I know how to crawl, but I can’t walk, but all the stuff I find teaches me how to crawl or run with no in between! I COULD learn to walk by attempting to run repeatedly, but that’s just so daunting… ---------------------------------------------------------------------- sarinkhan · 2023-10-26 13:58 Hello! I don't think it needs to be a big deal. It is complicated if you take the whole chain in one go. If you want to make it simpler, start by self hosting a stuff on your network, just for local use. Figure out docker and docker compose. Be happy with it. Then if you need outside access, try twingate. It is super easy, and you get access to your stuff in a secured way. Then you add more services, and you start monitoring your network and machines. At some point you begin to rely on the thing you deployed, and you think about how to back up your stuff. So you deploy a Nas. You make backups and stuff. Then at some point you realize your router does not suffice for you and you deploy a pfsense box of a opnsense box. At this point you may want to open services to other people, so you begin to harden your infrastructure, set vlans and stuff, you consider your reverse proxy, firewall with inspection of packets, etc... My point is you do things bits by bits, starting locally and figuring out stuff when you need them. When I teach code, that's the same thing: students that never coded Fing things daunting. But we divide the project in little tasks, iterate on one task until we validate it, then use what we did for the next task. ---------------------------------------------------------------------- Qxt78 · 2023-10-26 13:59 Understandable. Some devs make a awesome product really difficult to install and others make it really simple. Then running these apps in a home / self hosted environment becomes problematic and have their own challenges. Like the networking part. Most software engineers don't ever deal with that. So they won't know what to do themselves. But that is why we are here (In this channel) to advise and assist where we can. ---------------------------------------------------------------------- PaulEngineer-89 · 2023-10-26 14:00 Do it in stages. Don’t try to do everything at once. Start with Docker and leave the firewall alone. It’s there for your protection. At this point you can get a Docker running like you are used to, maybe tied to the host bridge only, so local access only. Moving up to the next level if you open up the server firewall only and join the Docker to the bridge network instead of host, it is now on your LAN but you are typing xxx:yyy. If you have a VPN such as Tailscale you can access it. At this point a password at a minimum is a good idea. If you don’t need public access stop here. Specifically run Tailscale. Seriously. Cloudflare is nice but for down and dirty Tailscale is easy. Set up a free account. From the command line serve your application. Type Tailscale serve help and just follow the instructions to launch your Docker as an application. It will be something like Tailscale serve https / http://127.0.0.1:1234. That’s it! Note that Tailscale does the SSL for you if you don’t mind letting it like abi d where I let it translate https to http. You can set up ssl direction but you have to load the key into your Docker container and you wanted simple. For public access realistically SSL is pretty much a requirement. Again Tailscale helps. First get on the web site and set up a “vanity name”. You get to choose basically 2 random words like “server.leopard-space.ts.net”. Also rename your server now if needed. Go back to the web site. Go to the hist and try to turn funneling on. It will lead you to a menu to set it up. Tailscale has to open the name server on their end and SSL is pretty much required so need to go through the process. Once set up external users can connect through Tailscale’s public access point. The ---------------------------------------------------------------------- Dairalir · 2023-10-26 14:04 As a software engineer of 17 years, you’re only 2 years in, you ain’t hot shit yet and have lots to learn. It can be difficult. Also, you’re probably just not great at searching/finding answers yet. Sometimes you need to look further down the Google results, or comb through closed issues/PRs in GitHub, or read a weird issue of StackOverflow that has a comment that links to another issue with the exact problem. Researching / finding answers / problem solving / learning new things will be the #1 skill that will get you somewhere. ---------------------------------------------------------------------- Acktung · 2023-10-26 14:06 Self-hosting is like any other topic in computing. You have to read, read and read. If you want to self-host a reverse proxy, go to the official documentation of the reverse proxy. What you are doing is searching for magic recipes in blogs and YouTube, that is why you feel it so "daunting". ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:06 This sounds very sensible, but “host bridge”? A password with a Tailscale VPN (how would you even implement that)? Why tailscale serve and not just port forwarding? Why add someone else’s server in front of my own? What’s funneling? The post is really helpful don’t get me wrong, but this is my issue with self-hosting. SO MANY NAMES ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:09 I never claimed to be hot shit, but I wouldn’t be as annoyed if I didn’t come across so much shit that I fully know already. I am stuck in tutorial hell without even looking for tutorials if that makes sense. The docs are either to simple or too hard with no in between. I feel like I’m no longer a baby learning how to crawl, but I’m also not yet learning to run. But it seems like the only way I can learn how to walk is by repeatedly failing at running until I get the hang of it ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:12 Quite the contrary, I’m stuck at finding a reverse proxy in the first place. If I didn’t know nginx had a reverse proxy, which is the only one I know about, where would I even start finding the docs? I can’t repeat this enough, but I rarely ever do tutorials, I find them basic and lackluster I don’t think I need specifically a reverse proxy rn so I don’t really have a clue about that kind of service specifically, but even finding WHAT to use to do an authoritative DNS was a challenge in its own right that I only solved somewhere else in this thread. ---------------------------------------------------------------------- StarSyth · 2023-10-26 14:13 Its only daunting if you want to do everything from scratch. Depending on your needs you could easily build off an existing platform for home labs and servers. Its more of a tinker / experiment / use existing, then reverse engineer the parts you need to. A good article on Home Server OS is here: [https://www.smarthomebeginner.com/best-home-server- os-2023/](https://www.smarthomebeginner.com/best-home-server-os-2023/) You can Find some good Self-Hosting projects / apps here: [https://selfhosted.libhunt.com/](https://selfhosted.libhunt.com/) And finally, here are two examples of decent, simple to install all- in-one solutions, Yunohost is my go to for friends who want to start self-hosting as it sets up everything including SSO, DNS (via Adguard Home) and Certs via Lets Encrypt all in a nice web gui and admin panel. Yunohost: [https://yunohost.org/#/](https://yunohost.org/#/) HomelabOS: [https://homelabos.com/](https://homelabos.com/) ---------------------------------------------------------------------- d-cent · 2023-10-26 14:13 I am right there with you. I am a Mechanical Engineer and if we tried putting out a product with the limited documentation that these come out with, we would be taken off the project. ME projects have documentation meant to guide a 10 year old through the process and details. The documentation would fill a 3" 3ring binder. Yet these network products are released with 1 page of upper level description of the product. Yeah I already know what the product already does, that's why I got it, how about the actual documentation? ---------------------------------------------------------------------- NSMike · 2023-10-26 14:14 Because most people who develop these things are, frankly, terrible at good documentation, or understanding the end-user perspective. There's also a downward spiral effect when you start getting into these things, because lots of them require dependencies, or ask you to do things but don't explain why, and you're just left wondering why you added that line to a config file somewhere, but if you don't put it there, nothing works. A vertical slice of the amount of knowledge you need passes through so many different disciplines, operating systems, GUIs, and programming languages that it would look like a Milhojas cake. I've been a technical writer in the software industry for 17 years. The number one challenge in my work is extracting all of the information I need to write good documentation from the experts elsewhere in my company. ---------------------------------------------------------------------- Ashragnorok · 2023-10-26 14:14 This is a swiss army knife of hobbies. I would say that yes it can be hard to get into if you try to do everything at once. I would pick one to three goals you want to accomplish and do those items. Once you have those documented and implemented, then pick up another task. ---------------------------------------------------------------------- beje_ro · 2023-10-26 14:14 I would say the opposite: self hosting nowadays is very easy. Is the multitude of options and configuration possibilities that is daunting. We want too much and this increases complexity. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:16 Honestly, my proficiency in languages and my frustrations with docs are making technical writing really appealing to me. Any advice on that career path? I’ve always heard it exists but never actually heard anyone using it as their job title ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:17 That makes sense, it frustrates me to no end I gotta learn so much shit already ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:19 Perhaps, wouldn’t know what to compare it to. I think I don’t want that complexity myself, but everyone else seems to want it, which adds useless layers I need to learn about only to realize I don’t need them yet, but can’t find a way to run them without the added complexity (looking at the apps that are only documented on how to run in Docker and no option to run on the OS bare) ---------------------------------------------------------------------- DeadOfKnight · 2023-10-26 14:21 If you want security and ease of use for remote access, just use Tailscale. Twingate if you share access with others. ---------------------------------------------------------------------- Wixely · 2023-10-26 14:22 The amount of times I've completely rebuilt my home setups is... many. But to get where I am now, I would never have been able to plan it properly without the learning experience of "failing" so many times. And there will always be a better way to do it as things need upgrading so in a way it is perpetual. Start with the projects you want first. Get them working and make sure you can export the data properly before you begin to rely on them. Then once you have small bits done, add extras, like cloudflare, vpn, portainer, proxmox. There will be a lot of formatting and reinstalling and going back to the drawing board, just accept it as a learning experience but take each piece once by one. If there is a specifc >yet I can barely find info on HOW to set up this things. There are plenty of guides out there that makes assumptions that you should already know things, which can be frustrating, I found this especially true with docker related documentation. If you find yourself on a github page, check the Releases tab. If it's still confusing there, check for a docker folder in the github and search "docker ". Start with popular more documented projects that will usually include beginner instructions that less popular software will omit. Even then if you keep looking you can usually fine one good doc that clears things up. Also, in this realm, chat GTP excels at obscure configurations so it's worth giving it a shot when stuck. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:25 That doesn’t even scratch the surface with the issues I face… ---------------------------------------------------------------------- Wixely · 2023-10-26 14:26 > If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now… Windows has a hosts file too. C:\Windows\System32\drivers\etc\hosts ---------------------------------------------------------------------- Perpetual_Nuisance · 2023-10-26 14:26 Step by step. Start with something that you're interested in, do that and learn more about it. Then move to the next thing. Yes, it can be a lot. Yes, it can be challenging. Yes, rabbit holes are all around. That's just how it is. There are setup scripts and such, but it doesn't seem you're looking for that. ---------------------------------------------------------------------- utopiah · 2023-10-26 14:26 Probably because you trust us, random strangers with entirely different needs and experience. I recommend to do what YOU need the way YOU want. Yes, it will not be "optimal" nor as "secure" as it "should" but by doing it, step by step, you will learn enough to gradually make better decision. Don't do everything at once the "right" way, just take a first step. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:27 Welp, maybe I’m wrong but when I looked into it Windows did some more stuff with those? Or maybe I’m wrong? Idk, maybe what I read just didn’t mention that ---------------------------------------------------------------------- samjongenelen · 2023-10-26 14:27 It sounds you want to skip steps and get fast results a lot of the time. While this will somewhat work, it sounds like you are not really suited for a career in engineering. Most of it is reading docs and gathering information - making sure you are not wrong. If you want fast results and WYSIWYG, currently frontend work is most suited. Of course I could be wrong, but I think I also read you are still in school. This would give you some time to think if you really want to work in this field. Best of luck :) ---------------------------------------------------------------------- Wixely · 2023-10-26 14:27 > Pi runs Raspbian which is just Debian with customisation applied. I think it's more that it's an ARM architecture that catches people out. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:30 I am not really looking for fast results, but it is discouraging that every layer I discover and understand leads to 12 different areas of knowledge. I guess I’m frustrated with how much of a, like someone else put it in this thread, a “swiss army knife” kind of hobby this is. Also, docs explain the basics or the advanced stuff with no middle ground in my experience ---------------------------------------------------------------------- CrunchCrisps · 2023-10-26 14:36 Maybe you should read the documentation for the programs you try to install ---------------------------------------------------------------------- Kryosleeper · 2023-10-26 14:36 I'd say, start with a need and not with a tool. Tools are much easier to understand once you know the problem you have to solve. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:36 My problem starts WAY before I try to install anything ---------------------------------------------------------------------- Wixely · 2023-10-26 14:37 The hosts file works pretty much the same as it does on linux. Requires admin to edit it so very rarely gets touched unless you do it yourself. But obviously not as good as a locally hosted DNS server. Be mindful, as too with local DNS, that if you make changes your OS and your browser will both have DNS addresses cached. You can clear the OS cache with ipconfig /flushdns You can clear chrome cache with chrome://net-internals/#dns ---------------------------------------------------------------------- PTwolfy · 2023-10-26 14:38 I feel like Self-hosting is like playing the game of life on the hardest mode possible. The easiest is always to pay for services or let others have the responsibility. That's the Easy Mode. When Self-hosting, mind blowing fucked up things will happen, your head will sometimes explode because you will doubt everything you learned, things that always worked will stop working for no apparent logical reason. Maybe you'll understand that some data was corrupted, maybe you backed your servers up, maybe you didn't, maybe you can recover data, maybe you can't. Sometimes you will feel amazing for your achievements, sometimes things just stop working even for things you won't control. ( Energy and Internet Suppliers ). Most people even with high curriculum and studies opt for the easy mode. And that's okay. Others enjoy the struggle and the amount of knowledge that comes with that. Choose your path, and stick with it. Pros of Self-hosting: self enlightenment, joy of achieving great things, freedom, saving money, power. Cons of Self-hosting: Time consuming, stress, dangerous if not done with responsibility, requires patience and perseverance, may be actually more expensive on your personal and professional time, if your time can be indeed valuable. You may be able to actually sell more, or be more profitable if you don't lose time thinking about the hosting itself. ---------------------------------------------------------------------- Ashragnorok · 2023-10-26 14:39 That is a great thing, never stop learning. I have seen this response before here but YouTube, blogs, subreddits, etc are available. You are not alone in this frustration, but that also means that someone else also had the same problem(s). My advice, start hosting your own DNS. Dont be afraid to break things in the lab, its where you will learn what to and what not to do. Last thing, document EVERYTHING **NOW**. Make sure its readable if things break. Please, save yourself a headache in the future. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:40 Haha, fair enough. I guess I want to self-host to get experience with it rather than to solve an issue. What I self host is actually quite simple and mostly made by me ---------------------------------------------------------------------- CrunchCrisps · 2023-10-26 14:41 Hmm okay, what's the problem? Usually you just tell your package manager to install a package ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:45 What should I be documenting? And how should I not be worried to break things in my own home network? If I’m compromised it could be a long time before I actually realize and my other devices could be screwed by then ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:46 Basically, the problem starts at knowing what package to tell the package manager to install. I just can’t find what my options are in the first place, without digging through a mountain of useless things everyone claims is a must ---------------------------------------------------------------------- mynumberistwentynine · 2023-10-26 14:46 >I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up. This is big part of why I, even as an IT professional, don't do much homelabbing or selfhosting. So often I'll be scrolling through these subs and see something someone has done and I'll decide I want to do that too, but so often I end up abandoning whatever it is because doing x requires googling for y and z because without y and z x doesn't work. And after a day of dealing with issues and tinkering on work stuff, do I want to spend more time troubleshooting? Often not. ---------------------------------------------------------------------- NSMike · 2023-10-26 14:49 To be honest, I stumbled into it a few years out of college, when a recruiter noticed that I had a computer science minor on my transcript alongside my BA in English. What was supposed to be a CS degree turned into an English degree when I realized I didn't want to do CS. There are books out there about career building and the like within technical writing. If you want some really dry reading, you can get an idea of the kind of writing you'd be doing by poking around the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style- guide/welcome/), which is honestly probably one of, if not the most widely adopted set of standards in the industry. If you have any writing experience at all, you've got to break yourself of certain habits - get ok with repeating yourself, because the goal here is not to worry about "pretty" writing, but clear, concise communication. Get comfortable with second person. Squash passive voice out of your writing style like an ugly insect. Make friends with UX people. ---------------------------------------------------------------------- CrunchCrisps · 2023-10-26 14:52 This pretty much depends on your goals. A reverse proxy via nginx is what at least I consider the basis for most of self hosting stuff, but other people might like accessing their services via vpn instead. After that it is really dependent on what services you want to use and how they should interact. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 14:53 Thanks! Really valuable advice here. Honestly looking more and more attractive the more I look into it ---------------------------------------------------------------------- Dairalir · 2023-10-26 14:54 Yup, you have to tackle something hard, slam your head at it for a while, tease out solutions for small problems your having along the way, and just brute force it sometimes. Then you’ll come out having learnt the thing and the next thing will become easier. Watch a video or read an article about the new thing so you get an idea of what it can do. Look at the docs, there’s usually a quick start guide. Try that, then as you fool around with it, try implementing the actual thing you want to do. You’ll hit roadblocks but these will be actual searchable problems that other people have probably ran into and solved, rather than “how do I implement this new tech end-to-end”. ---------------------------------------------------------------------- Ashragnorok · 2023-10-26 14:56 Document configuration in general to be able to explain the 5 Ws (Who, What, When, Where, How). This is just for your future sanity, how much information will you need to to ensure you can fix the widget you are hosting in the future. In my mind, when dipping your toe into selfhosting your own services you would start with only internally facing services before adding anything external. With a old pc and an install of pihole you can get comfortable hosting a service without having to tangle with external bad actors. ---------------------------------------------------------------------- revereddesecration · 2023-10-26 14:59 None of the code that comprises PiHole is ARM-specific. It just compiles to ARM instructions, and can be compiled to x86 just the same. ---------------------------------------------------------------------- Psychological_Try559 · 2023-10-26 15:19 The first thing to remember is that our hobby is literally a career (admittedly a number of people on this subreddit actually do)...so yeah, it's not simple. Also, there's a perception problem where you forget that things used to be hard now that they're easy. For instance years ago a friend & I spent months trying to get reverse proxies to work. And they never did, now we both see them as trivial to setup. We have a saying that they "went from impossible to obvious", and that happens a lot in this field. This is a result of there being so many different ways to do things. Hardware, OS, virtualization, network topology, router/switches, etc...each decision makes some aspects easier while limiting you in other ways. There's no best way to do this or it would be the standard. ---------------------------------------------------------------------- Reasonable-Bear5084 · 2023-10-26 15:21 Self hosting, right now, is for people with lots of knowledge. It's madness, every app has different requirements. Some work behind a reverse proxy, others are incompatible or require complex configurations, and so on. I stopped self hosting because of the complexity, maintenance and backups. ---------------------------------------------------------------------- Realestaste · 2023-10-26 15:23 Enjoy it man! All your problems can be solved with enough effort. THis is the best part, the part where you are forced to learn a bunch of new stuff to move up in your skills :) Best of luck! I love this shit! ---------------------------------------------------------------------- beje_ro · 2023-10-26 15:26 We cannot afford to have customized solutions. For this we should pay, and not just a little. That is why these general solutions: they cover main needs, are easy to maintain and deliver but add to the complexity... ---------------------------------------------------------------------- BicycleParking5977 · 2023-10-26 15:27 Use bind. It's fairly easy. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux /8/html/managing_networking_infrastructure_services/assembly_setting- up-and-configuring-a-bind-dns-server_networking-infrastructure- services ---------------------------------------------------------------------- UTOPROVIA · 2023-10-26 15:28 About gpt, I assume you mean you know better because you see mistakes in code that it generates. It is a tool and not a magic portal of answers that is "dumb" a lot of the time. ---------------------------------------------------------------------- spanky34 · 2023-10-26 15:29 > Jeff Geelong Do you mean /u/geerlingguy ? AKA, Jeff Geerling? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 15:36 Fuck no, you’re wrong. If it runs on Docker it runs on Linux. Why go through all the trouble to get something running just so it can only be run inside a container? I’m not asking for customized solutions, I’m literally asking for the most barebones experience. I’m asking for the bare minimum imho. Running things should be the default, virtualization, orchestration and whatnot is the extra shit ---------------------------------------------------------------------- Jolly_Molasses6566 · 2023-10-26 15:36 I understand what you’re saying, I’m quite new to selfhosting but as a software dev you’re getting used to reading documentation sooner than later. So how everything works should be explained in most docs - I just find it hard to find the connecting the dots pieces in hosting. It’s a lot of information, but I guess that’s what happens if security, access and roles suddenly play a role which often is an afterthought in software development ---------------------------------------------------------------------- Ieris19 · 2023-10-26 15:38 I love Codex and constantly use it (through Github Copilot) as a tool to help me code. But arguably, I’m knowledgeable enough to catch fuckups and understand what Codex is doing and correct it when I need to. Learning something I don’t know anything about through any sort of GPT? Hell no. Also I’m banned for some reason ---------------------------------------------------------------------- SmellsLikeHerpesToMe · 2023-10-26 15:40 Typically you’ll just end up falling into these items as you self host. Starts with hosting something locally, then wanting to access it from other devices from your home, then wanting to access them remotely. Over time it just makes sense to do it the way you want, but taking extra precautions when you get to the remote access bits. ---------------------------------------------------------------------- unidentified_sp · 2023-10-26 15:40 I’d just install Docker and add a CloudFlare Tunnel to securely host your stuff. No need to open any ports, your WAN IP remains hidden and you have the benefits of DDoS protection. It’s free as well; all you need is a domain name for which you can configure the nameservers. ---------------------------------------------------------------------- KarmaPoliceT2 · 2023-10-26 15:40 It will make you a better engineer because you'll be exposing yourself to more stuff and soaking in more knowledge (personal experience here)... Not to mention, soooo many of the homelabber tools are open source, and as a software engineer you are capable and should be motivated to pick a few projects and contribute to them. I would also say, there are things to focus on, and things to follow tutorials on... One of the most important pieces of homelabing (imho) is having a test and prod env separated... This way you can tinker without blowing up your home setup (doubly important - maybe life critical - if you have a spouse/significant other who don't appreciate things 'suddenly not working' :) )... Use the test env to run through tutorials, gain understanding at your own pace, then use production for the things you've vetted as useful and have importance to them... This will ease your suffering and "trial and error" cycles substantially! ---------------------------------------------------------------------- emitlinks · 2023-10-26 15:44 Once you understand the concept of what you are trying to setup it will depend on how well it's documented. Most stuff out there have a documentation to help you set them up but you might still need to have prior knowledge of system and networking to fully understand what you are doing. I am self hosting a lot of stuff myself (media server, ldap directory, single sign-on, game servers, dns, mail, reverse proxy, nextcloud, etc) and It took me some time to set all of this up, it's not something you set up without doing some reading (learning about everything you can configure and how, comparing to other similar solution), testing and tinkering (not everything will do what you exactly want). It helps a lot that I have a degree in system and networking administration and that my job relate to this without that knowledge It would take me a lot more time to understand what I'm working with. I am still learning a lot when setting stuff up, and I can use what I learn on my free to apply it at work at the same time as I am having fun with it. ---------------------------------------------------------------------- BobbyTables829 · 2023-10-26 15:47 I'm a software developer and I think system administration is so much harder. It requires so much learning of really complicated concepts ---------------------------------------------------------------------- cardboard-kansio · 2023-10-26 15:49 Then maybe start by hosting things you don't really need. Host a single media file, but so so with every single service you can think of. Can you access the file internally? Externally? On browser? Dedicated app? Via SSH? With a VPN? Did you host your own VPN? You just need to learn to approach practical problems that are common, and then adapt what you learned to your other needs. These are how most of us have grown and learned. ---------------------------------------------------------------------- indykoning · 2023-10-26 15:54 Even if it might suck when you run into things. Start how you want, run into these problems. Solve these problems. You learn most from solving the problems instead of starting at the top. I'm running Proxmox too with docker, Portainer and docker compose which is what I like and solves things for ME. But most applications come without docker as well, and a reverse proxy is only interesting if you want to put multiple services public via the same port. Heck you can probably run many homeserver things on Windows if you like. Everyone has their own journey to getting things how they like, and understanding what they have. But people only share their end result ---------------------------------------------------------------------- lestrenched · 2023-10-26 15:56 Do you have a purpose to host whatever you happened to name here? There is a reason to host reverse-proxies, however, one can do without them in a self-hosted environments. First, one needs to understand the point behind reverse-proxies (this is an example btw, you might very well know the how and why behind them), and only then would the instructions to set it up start to make sense. ---------------------------------------------------------------------- lestrenched · 2023-10-26 15:59 > When I say at a theoretical level I mean I’m familiar with it from University lectures And here's something you need to keep in mind: most of what you are taught in education is absolute rubbish. Note that I said most: familiarity with concepts in Docker help. You seem to be an undergraduate student, which makes me think that you haven't actually dipped into the core networking of Docker (not that I'd expect one to do so at such a stage). It's not that hard. Understand the why, look up tutorials on the how, which will make sense since you now know the reason behind the steps. ---------------------------------------------------------------------- Ivancittoo · 2023-10-26 15:59 We can set up a zoom if you want my help. ---------------------------------------------------------------------- lestrenched · 2023-10-26 16:00 > I have probably never managed to follow a tutorial This hobby might not be the best since instructions can be long and tedious sometimes. If you lose interest midway then walking away is usually the best idea. ---------------------------------------------------------------------- lestrenched · 2023-10-26 16:02 > you’ll be bombarded with a thousand words that barely mean anything You look them up on your favourite search engine and carry on. If you have an interest in this, inside a month or so of serious studying you will be able to pick up a lot of what you need. Again, you need to know your purpose. ---------------------------------------------------------------------- Wixely · 2023-10-26 16:05 Yes ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:07 Putting a server in front of my own defeats the whole purpose of self- hosting for me. I didn’t say CloudFlare “bullshit” so aggressively for no reason. I want to learn, because I feel like I should know how to deploy stuff and my uni is not teaching me. ---------------------------------------------------------------------- unidentified_sp · 2023-10-26 16:10 With CloudFlare Tunnels, you’re still hosting yourself. You just prevent a lot of headaches. But I understand your point; I used to just forward ports and use Let’s Encrypt etc. to host my own SSL- secured stuff. ---------------------------------------------------------------------- VexingRaven · 2023-10-26 16:10 You can save 99% of the hassle by just using a VPS or some cloud host or some sort. I know, I know, "self hosted!" but hear me out. Self hosting is about having control of the services you use. You still have vastly more control using hosted infrastructure than using something Google or Microsoft service where you're the product. Most of the complexity in hosting this stuff comes from securely hosting it all behind your home connection. If you take that out of the equation, it becomes a lot simpler and more approachable. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:12 I don’t have a need but I do have a goal for the things I want to setup. I got downvoted to oblivion for saying I didn’t even find what kind of software I could use to make an internal authoritative DNS service for example, where I want to create a custom internal TLD for my VPN. But apparently people took offense I’d never heard of bind and assumed PiHole was proprietary… ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:13 I’ve actually made my own images and even hosted some on Google Cloud for a school project (and some on my own server for myself). But thanks for the advice, I also don’t think I’m too deep down the Docker rabbit hole hehe ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:14 It’s more about tutorials revolving around magic configs and basic instructions and I need more complex instructions to tickle my fancy. But fair enough ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:18 My purpose with selfhosting is not control but learning. So I will stay far away from sticking another server in front of mine. The only server I’ll put in front of mine is DNS, because really, I can’t replace it. I would if I could haha ---------------------------------------------------------------------- TerminalFoo · 2023-10-26 16:22 Things are daunting because you’re viewing everything as one giant block. Instead, break things down into pieces. It’s then less daunting and more digestible and easier to tackle and google search. ---------------------------------------------------------------------- lestrenched · 2023-10-26 16:27 > assumed PiHole was proprietary Did it look proprietary on their website? Personally, I wouldn't even run `bind` in my homelab. I don't see the point. `dnsmasq` is good enough for me. But if you'd like to run it, go ahead! Creating a new TLD for a VPN with an authoritative DNS for your local network isn't the hardest thing, I'm sure you'll find documentation on how to do so BIND's website. If you don't understand something just ask your search engine or ChatGPT/Bing (do not rely on LLMs for factual information, but they're good at summarising information, from Wikipedia for example). Maybe get in the habit to RTFM, it does help. BTW here's a good list of internal TLDs one might want to use: https://serverfault.com/questions/17255/top-level-domain-domain- suffix-for-private-network ---------------------------------------------------------------------- lestrenched · 2023-10-26 16:28 Define "magic" configs? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:30 Sounds like me haha. I’m constantly chasing the shiny new thing haha. But that’s pretty much what I get from this thread. I need to learn to search better and break up content into more manageable chunks ---------------------------------------------------------------------- Deses · 2023-10-26 16:31 Sure, I'd display that somewhere nice. BTW, to add something constructive to our exchange: If you use Unraid you should check IBRACORP's tutorials. They are very helpful and touch many different topics to selfhost and reverse proxy safely. ---------------------------------------------------------------------- VexingRaven · 2023-10-26 16:33 All the more reason IMO to start with using a VPS. Learn the basics. Learn a single service on a single server. Then add another service. Then if you're brave add another server with another service that relies on something on the other system. The reason people find self-hosting so unapproachable is because the community tells them to jump straight into reverse proxies and and port forwarding and tunnels and all kinds of crazy stuff right from the start. A single server hosted directly on the internet is a so much simpler way to start learning. Just don't put anything too sensitive on there in case it's compromised, but that applies to anything you use for learning anyway. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:36 I just thought it’s a Raspberry Pi thing, and since I don’t own one or want one I just never looked into it at all. I did admit that was on me but still people got salty. I dislike learning from GPT, I use it to help when I can fact check it, but never with new stuff, I’ve seen it be plain wrong too many times. Surprisingly, I mostly have an idea of how to make an internal TLD, just was missing the software to run a DNS in. Bind came up elsewhere in this thread, same for PiHole, but I’m open to other suggestions if anything sounds better. As for what the TLD, I thought of using my username haha, might be cheasy but it’s my online identity so I thought it’d be cool to have something like “whatever.ieris19” as my URL hehe ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:37 “If you write this and that, download these dependencies and run that command, Voilà! It works just fine” and no further information ---------------------------------------------------------------------- virtualadept · 2023-10-26 16:40 A lot of the stuff assumes that you want to build this massive, complex piece of machinery all at once. You don't have to. Reverse proxying, VPN access, Cloudflare, and stuff like that are all nice to have and fit specific use cases, which you may not have. Docker, et al are nice if you have a lot of stuff that you want to stand up all at once, and you might need to rebuild it from scratch at the drop of a hat, but the additional complexity might not do you any good. If you're just learning how to do stuff it definitely doesn't make life any easier (or more fun). HTTPS is essential on the public Net, but you don't necessarily need it if it's a single board computer sitting next to you on your desk at home. It's okay to start small and simple. In fact, it's probably the best way to learn. If there's an application that you want to host written in Python, and it listens on port 8888, IT'S OKAY to let it run that way for a while, and it's okay to only access it from [http://twinkie:8888/](http://twinkie:8888/) from home. Just like it's okay to install and configure Nginx six months later to sit in front of it "like a real webapp." Self hosting isn't a $whatever measuring contest online. It's a thing that some people like to do for fun, some people do because they need it, and some people do because they're learning how to run servers. If you're having a great deal of trouble getting stuff running, it's more or less normal. It's a sign that you maybe need to scale back a bit, take your time, and hack around a little bit. If don't have a lot of stuff running at this moment, consider ripping it down (yes, I said "rip it down," I've done it a lot of times in the past) to start over, but start over with one thing that you're going to use (maybe a wiki or a notepad or something) on a box that's in a known good state (maybe a brand-new install, freshly patched up). Don't read any of the huge tutorials that tell you to stand up Kubernetes, Cloudflare, a VPN, blah blah blah... follow the application's directions to set it up (if there are any - that's getting to be a not-common thing these days) and mess with it for a few days. Tinkering is good for you. It teaches you how to solve problems. Trying to troubleshoot a fairly complex setup before you've troubleshot a fairly simple setup is an exercise in frustration and futility (ask me how I know...) ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:42 What about hosting it on my own home network? I’m comfortable with for example my game servers being hosted for a few weeks intermittently without any added security, but as far as doing anything more long term I’d be worried my network will be compromised and I’ll be non the wiser ---------------------------------------------------------------------- virtualadept · 2023-10-26 16:42 A lot of people forget what it's like to just start out. We have a saying elsewhere on the Net: ABN. Always be n00bin'. It's okay to be new at things. It's okay to not know and to ask questions to learn about what you don't know. Nobody here - not a single one - was born knowing everything there is to know about system administration like Athena sprung from the forehead of Zeus. It's also not like search engine results are worth a tinker's dam these days, either. ---------------------------------------------------------------------- virtualadept · 2023-10-26 16:43 "Impossible to obvious" is a really good way of describing it. Do you mind if I quote you on that? ---------------------------------------------------------------------- mulletarian · 2023-10-26 16:48 Jerf Greeloong ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:54 Perhaps one of the best answers in this thread. Thanks so much for taking the time, and while I don't have much to say in response, I wanted to let you know this answer is really helpful haha ---------------------------------------------------------------------- Ieris19 · 2023-10-26 16:56 I swear, Google lately sucks more and more, maybe I'm just not good at searching but I find myself adding site:reddit.com or stackoverflow to my queries more and more (only because the search tools for those websites sucks more than Google's haha). The crap Google feeds me unless I'm searching something specific is hardly ever useful beyond the first result, and maybe the second ---------------------------------------------------------------------- ItsMelodyy · 2023-10-26 16:58 The things of "how to set up" are usually covered by software's documentation, further supplemented by googling certain specific questions and going on various forums. Never just use the first page you find but crossreference information as well. ​ Having gone into self-hosting as a hobby with the intention of supplementing knowledge in my career path (currently still studying), it can become quite daunting quite fast, especially if you try looking at a big picture, rather than taking it step-by-step. The best thing to do would be to follow a tutorial within the documentation of something you're trying to host, then expanding per step as far as dependencies goes, and then go from there. Compartmentalizing is a very effective way to not get lost. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:00 I struggle with searching and compartimentalizing as I’m learning from this thread. But honestly, when I get to the docs I’m mostly done and sorted. Finding what docs (or to be precise, what tool) to use is where I most often get stuck ---------------------------------------------------------------------- VexingRaven · 2023-10-26 17:03 Depends on the level of security you want. You *can* generally just port forward a game server and be fine. Vulnerabilities that compromise the host machine are relatively uncommon, especially if you are not running any mods or third-party integrations. That being said, they *do* happen and I don't want to take that chance, so I have a separate VLAN I host my game servers (and other internet-accessible services) from. But again, that's adding complexity and increasing the barrier to entry. You need a router capable of VLANs and custom firewall rules, you need to know how to configure it. There's another element to consider here: I don't really think game servers are a great learning tool. There's generally not much you'll learn that will apply more broadly, other than extremely basic stuff like how to port forward and how to start and stop a service. Combine the fact that their high performance requirements generally prevent you from using a cheap VPS as a learning tool for them, and I think they're a bad learning tool. Fun as a side project if you're gamer, but overall not great for learning. ---------------------------------------------------------------------- Erikt311 · 2023-10-26 17:19 Keep in mind that you are reading subreddit populated by people who are deep in the weeds on this stuff with years of experience. Self hosting doesn’t have to be any more complicated than setting up a spare laptop. ---------------------------------------------------------------------- makanimike · 2023-10-26 17:33 On the one hand, I feel you. On the other, man, I am so happy you are suffering! lol If you, a Software Engineering student are struggling, then it does not make me feel quite as clueless anymore as a normie :) ---------------------------------------------------------------------- Osni01 · 2023-10-26 17:33 Based on your OP and a few of your answers, it looks like you're struggling with setting up too many things together, when in reality some of them are meant to be a foundation and /or are required to move on to the next sparky thing in the awesome list. You mentioned "Reverse proxy and VPN bullshit" in your OP. Start with setting up your internal DNS (Pi-hole and Adguard are good examples that work mostly out of the box). Once you're confortable with managing internal DNS, and only then, start messing around with reverse proxy. And like you realized a few posts up, pretty much "every" Pi-related software can be run in a regular computer. If you're not yet familiar with virtual machines, do some research on Proxmox and the likes and it will benefit you immensely. ---------------------------------------------------------------------- muirthemne · 2023-10-26 17:36 I think This is going to be a super unpopular opinion here, but you don't necessarily have to go with what "everybody else" is using just because they're the big shiny pennies that everybody else is using. Years ago, "everyone" was using Windows and Internet Explorer. Does that mean that nothing else was worth running? As to VPN, Docker, Cloudflare, Traefik, Crowdsec, Authentik... this definitely isn't a "don't use those things" post. If you want to learn Cloudfare, and want to use Cloudflare, then by all means, do so. This isn't an anti-Docker, anti-Cloudflare, or anti-anything post. But that being said, I *started* self-hosting with a Dell laptop running XAMPP with a single, tiny, intranet web portal on port 80. It was a couple of clicks to get a functioning LAMP stack, despite me being a complete noob. It was incredibly insecure, and not the best, newest, fanciest choice to host my little website. But it allowed me to learn incrementally, and quickly, rather than feeling like I was thrown into the deep end way over my head with -- as an example -- Docker and virtualization. - I got sick of XAMPP pretty fast, and learned how to host my own database and Apache on bare metal, instance outside of XAMPP - then I learned how to buy and set up an actual domain name - then I learned how to install SSL certificates for my domain - then I learned about hardening Apache, and things like honeypots and fail2ban - later, I acquired several more small servers with their own services, all behind the same router, and I learned about subdomains, virtual hosts, and proxies - even later, with all of that foundation, I learned about moving individual services into Docker. I only had to learn Docker. I already knew the rest of what I needed. - etc. I wanted to self-host my own intranet-only DNS server, but I found BIND to be overwhelmingly daunting at the time, with poor information for how to actually set it all up and get it working. So I found dnsmasq and installed that instead. The configuration was incredibly simple, and it worked almost out-of-the-box. Am I going to have pretentious snobs telling me how much dnsmasq sucks and how I'm stuck in the 90s if I still use it? Probably. Does it serve my own, personal, use case for the features *I* needed? Yes, it's been great and simple to maintain for years. My "dynamic DNS service" is a bash script running from a cron job, accessing a webhook that my domain registrar offers, to update my DNS records automatically. Surely, there is some great piece of software with a catchy name and a GUI dashboard that I could be using for this purpose. But I don't feel that I *have* to. My laughably basic bash script has worked without touching it for about 10 years now. For my VPN, for a long time, I had a PPTP daemon running on my server. That, too, had been very easy to set up for a young sysadmin who knew nothing about anything. It wasn't the most secure, I used it long past PPTP fell out of favor, but it worked at the time, and there was ample documentation that was easy to follow to set it up. I've moved on from most of the things I started out with when I was learning the basics, but all of that to say: Sometimes, it's okay for something to be "good enough". It's okay if your server isn't PCI-DSS- level secure and protected from absolutely everything. It's okay to do the best you can, get your setup working... and then don't stop there. Continue learning how to make it better, more secure, more reliable, more scaleable. But just because the popular kids are going to insist that you should never consider anything less than Nginx in a Docker container, managed by Portainer, behind Wireguard, behind Cloudflare... that setup can be your end goal, but you don't have to start there and learn everything all at once. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:37 Thanks! Yeah, I’m reaching much of the same conclusion here myself. I mentioned VPN and Reverse Proxy bullshit because everyone on the internet tends to call these things “essential” and whatnot and I know they’re overkill for me haha. Most I’ve ever done is actually just setup Tailscale VPN and some apps on bare Linux metal haha. But trying to setup something, just to end up bouncing around between VM, Docker, Promox, Portainer and 3 other layers of extra bullshit just to realize I only really needed to hit run and it would’ve worked make it frustrating ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:41 I’ve got a pretty advanced router that supports 4 networks at once. But I don’t think that includes cables vs wifi nor do I think that’s anything other than 4 Wifi signals to the same network. There’s probably a different unrelated option for separate networks if I’m not wrong. But I did get a 40$ router at a liquidation sale about a year ago, plugged it to my ISPs and set the ISP’s router to pass-through mode. I’ve actually got 3 routers at home cause the ISP forgot they had another in the apartment already (my landlord had these installed in every unit since they’re newly built). It just happened to be unplugged by the previous tenant so the ISP couldn’t reach it haha. Maybe I should plug it into my pass through and use that second network as my public facing one? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:42 That’s exactly what I’m doing, and exactly why it frustrates me that there’s so few resources for beginners like me ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:43 Haha, stay on the grind, consensus from this post is sysadmin is harder than developing software haha ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:47 I resonate with this so much. My DynDNS service for a Minecraft server with friends 2 years ago was a Discord Bot that would reply with the current IP when called. Oh those were the days (I was starting uni in September after that summer). I feel like on the one hand, I want to setup something nice and simple, but somehow it’s better documented how to set things up on a very complex infrastructure like Docker or VMs… ---------------------------------------------------------------------- lestrenched · 2023-10-26 17:47 That is to be expected. RTFM is the culture of the community. You have other options: https://github.com/awesome-foss/awesome- sysadmin#dns---servers (NSD is a fun one, used in the industry, if you're interested). Just make sure no one has bought a global TLD with `ieris19`, because if they have then your internal traffic might go haywire. ---------------------------------------------------------------------- lestrenched · 2023-10-26 17:48 Well, it's up to you to understand said commands and the reason behind the dependencies. This is why one bookmarks the Gentoo, Void, Debian and Arch wikis, because they have a lot of very good documentation on many packages and other tit-bits. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:50 Well, I don’t think ICANN has registered a TLD called ieris19 sooo, I think I’m safe (and that’s partly the reason why I want to use something so unique as a TLD). There is not that many TLD available anyway, and they’re highly regulated on the open web ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:50 That’s why I just avoid the tutorial and go straight to the wiki/docs and poke around those configs and commands directly ---------------------------------------------------------------------- InspectionLong5000 · 2023-10-26 17:51 I have a very modest setup, comprising of a few applications hosted on a PC, with IIS acting as a reverse proxy using URL rewriting. Ports 443 and 80 are forwarded to the PC, along with 34200 for Plex. I have a self issued SSL cert using LetsEncrypt. In terms of security... How secure/insecure would you say this setup is? ---------------------------------------------------------------------- ScuttlingLizard · 2023-10-26 17:54 You are trying to add too many things at the same time. Start simple. You don't need to do everything all at once. Pick your number one priority and get it running smoothly locally first. Then add in more as you need it and over time. I started in a similar situation to you. All I built was a basic NAS I could download files to and I could mount the drive in XBMC. These days we have Jellyfin, Emby or Plex but it was just a single service. As things got pushed out I added couchpotato and sickbeard. These days I am running the various `arr` stack of services to download linux isos. I then added in Home Assistant and followed the docs for getting it secure. I eventually added in Frigate NVR with local security cameras. I added off site backups and local raid arrays to add redundancy. I added Immich to replace Google Photos. I added wireguard to avoid needing to expose all services to the internet. I did this over 12 years. I didn't just plop into a full setup or try to do it all at once. That is crazy overwhelming. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:55 Yeah, I think I’m shooting myself in the foot here trying to do so many things at once haha ---------------------------------------------------------------------- weeklygamingrecap · 2023-10-26 17:56 I'm confused why you say you can't find information on how to set up home lab projects. There are hours and hours of content on YouTube that also do sometimes point back to blogs and GitHub repos. I will say that the problem I have is once they show you how to setup say a reverse proxy there's generally not follow through in either deep diving into all the additional settings or more advanced setups. If there is the other problem you'll run into is that it can become outdated quickly so it's a double edged sword. But setup and getting things running in a home lab there shouldn't be any shortage of info to consune. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 17:59 The problem is how everyone has this crazy infrastructure and those videos and blogs are either too simple or too complex. There seems to be nothing out there to really get through the middle stages of being a technical person who knows what they’re doing around Software, yet a beginner who doesn’t know about self-hosting much. I either have to watch/read painful hours of stuff I already know for a chance of something new, or get in way above my head and stumble until I figure it out. But the consensus in this post seems to be that I just gotta deal with the latter ---------------------------------------------------------------------- atheken · 2023-10-26 18:03 TBH, I don't think Software Engineering, especially just 2 years of experience, is going to really help you understand network architectures/distributed systems. You can probably build some small apps, but designing larger systems is a skill that requires intentional practice. The best thing you can do is figure out how to containerize the stuff you want to run and store the configurations in source control. Figure out how to isolate your data and back it up, and then experiment with configuration changes to see how they change system behavior. A few specific things to learn/practice. - Learn how to break down a larger problem into distinct components with specific responsibilities. - Learn about docker, what problem it solves. - Learn how to experiment with the tools to find out what they can do, and how you can configure their behavior (docker is very helpful here, because you can spin up a temporary sandbox to figure things out without risk of breaking a "live" system). ---------------------------------------------------------------------- muirthemne · 2023-10-26 18:04 Well -- arguably, if you're struggling with setting them up and can't get them to work properly, that doesn't sound like "better documented" to me, lol. In any case, I think that's my point: it's not really a binary choice that you have to make. If you do get the complex infrastructure working by brute force, it tends be a huge headache to maintain, update, or make changes to. And again, I'm not saying "give up on VMs". But since you're frustrated and overwhelmed with just trying to get things to actually *work* in the first place, you might find it less daunting and more rewarding to start with nice-and-simple first, get your setup working, and then fight with complexity later. And sometimes, like with my DynDNS script, there is such a thing as "over- engineering" for something that doesn't actually need to be more complex. What services are you looking to self-host? The other mistake you might be making (I haven't read every single one of your replies here) is trying to host too many things at once out-of-the-gate. ---------------------------------------------------------------------- adamshand · 2023-10-26 18:06 I also do this for a living, and yes it's complicated and yes there is always more to learn. I burnt out and ran away from it all after 20 years and had to go do stuff away from computers for a while. But now I'm back at it and having a pretty good time again. :-) My recommendation would actually be to start small. Pick a single app you want to selfhost (I often recommend something like AdGuardHome as a reasonable starting point). And then slowly work your way through everything required to get that to work. Then pick something else (maybe Navidrome?) and do the same thing. Each time you build (or rebuild) something you learn. Just keep at it and after a while you'll be amazed at how much you've learned. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 18:10 I’ve written pretty decently sized distributed systems and hosted them through Docker and Google Cloud (so Kubernetes, technically) Sure it’s not enterprise-level stuff, but my last school project was a fullstack webapp to serve data from an embedded device, built over 6 months with 12 people in my year, web deployed through Github Pages, backend microservices dockerized and setup in Google Cloud and an embedded device running its own custom code (teachers were nice enough to give us drivers though). I wouldn’t call myself a beginner, and 6 months of banging my head against a wall definitely taught me a lot about Docker, Distributed Systems and some networking. I also had a course on Networking and whatnot. I am far from an expert, much left to learn, but I know enough beginner tutorials are absolutely self evident to me. Yet more advanced tutorials are way too much to handle ---------------------------------------------------------------------- elementfortyseven · 2023-10-26 18:11 networks are black magic and printers are voodoo. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 18:12 That’s certainly how they’re presented and it annoys me to no end ---------------------------------------------------------------------- A9-EE-78-6A-C8-9F · 2023-10-26 18:24 Start small Set up a nextcloud or something then use NGINX on another VM to connect to it as a reverse proxy That'll help you understand cf a little more You can't just drive in and be an expert, it'll take years to get good ---------------------------------------------------------------------- throwaway234f32423df · 2023-10-26 18:24 Use a decent VPS instead of trying to host off a residential internet connection behind 47 layers of NAT and you don't have to worry about 90% of that stuff i.e. use the right tool for the job ---------------------------------------------------------------------- Psychological_Try559 · 2023-10-26 18:26 Please do! ---------------------------------------------------------------------- junialter · 2023-10-26 18:29 I strongly suggest to not only read articles on the internet but get decent books and read them carefully. It will fill in plenty of gaps you have now and in a blink of an eye self-hosting will become a smooth and fun experience. I have been self hosting for like 20 years and I must say it has never been easier to bring up a service and make it secure as well. Good network understanding is key, forget about IPv4 it's dead. If you engineer new stuff, concentrate on IPv6. Also a good book or two about Linux, it's the platform to go for the next decade. If you're still hungry dive into containers and container orchestration. Most importantly, keep experimenting. I'd say 80%+ of my empirical data comes from my own experience. ---------------------------------------------------------------------- odaman8213 · 2023-10-26 18:29 Depends on what you're trying to do.. Plex/Jellyfin server on a Pi with Zerotier for remote access? Super easy and you can get running in 20 minutes with zero previous knowledge. Self hosted email and reverse proxy with auto SSL and best security practices? Well that's a rabbit hole... ---------------------------------------------------------------------- guptaxpn · 2023-10-26 18:41 I think the reason is that there's a preferred method for doing things for each person. Like take reverse proxies for instance. You can use nginx proxy manager if you're comfortable with docker but uncomfortable with CLI or text file configs. Or you can just use straight nginx. Or if you want you can use Caddy, which is what I personally do, because I dig how well Caddy does automatic management of letsencrypt certs. But that means being comfortable grokking how to write a Caddyfile (the config), and how to launch a daemon on some sort of Linux. So if you want to take my route you've got even more choices, you can run it on a Linux VM, a Linux container, or let docker manage your Linux container. Or you can use someone's Caddy docker container and config that for yourself. Personally I hand configured a Linux LXC container on proxmox and am using that for all the reverse proxying on my proxmox NUC server. It's just so many variables now. I remember back in the day you just installed a LAMP server and ran everything with PHP. I'm not sure which is better. But I've got to say, the sheer amount of abstractions and ways to orchestrate a system are mind numbing. Like you can do proxmox or truenas or truenas scale or Synology or rancher or bare metal. There are so many things to decide on now. And I've been running Linux for the past 18 years or so. The biggest thing for me is that you should decide on the outset if you want a simple setup, or if you want to play with bleeding edge enterprise stuff. One is relatively plug and play and somewhat tends to be less secure. The other can be fun for some or job training for others. I find the people who find it fun to be a bit weird, like they are basically LARPing being a system administrator. But it might be just dressing up for the job they want instead of the job they've got, or they just like it and that's fine for them. It's not really for me. I look for simplicity, but also security. Luckily I'm a fan of minimal installs and am happy with operating on a CLI. OpenBSD is a pretty solid base to build on. It makes you think about everything you add and how it's a possible security hole. Anyway, enjoy reading manuals. That's most of what we do here. Join a forum or two (a real one). Find a supportive community there, ask good questions, contribute good answers. It's computer karma and by helping others you're broadening and solidifying your own comfort levels. Don't give opinions as fact. They are opinions. This comment is clearly riddled with opinions. YMMV. Good luck with your goals. ---------------------------------------------------------------------- muirthemne · 2023-10-26 18:49 I just saw your post where you said: > Why go through all the trouble to get something running just so it can only be run inside a container? > I’m asking for the bare minimum imho. > Running things should be the default, virtualization, orchestration and whatnot is the extra shit This is basically my philosophy. It is true that pretty much all popular "apps" these days are made for Docker, with no option to install outside Docker. But there is no inherent requirement to use Docker if the services you're trying to self-host are just, for example, a Nextcloud instance, a media server, and a music player. So what you use ultimately depends on what it is you need to host. I don't actually have any containers in my home self-hosted setup, and only work with them in the enterprise environment. But if there's a problem on my home server, I like being able to SSH in on my phone and issue a few basic commands on a tiny touchscreen terminal, without layers of chaining services to troubleshoot in between, or any inherent mandate for a GUI. Upgrades are simple, maintenance is simple, backups are simple, and config is in one place. Most of my services are web-based. This isn't for everyone, and I'm definitely not saying it's the best approach. But the entire point of self-hosting is *self*. Just for you. If everybody keeps saying that a VPN and Cloudflare are absolute must-have requirements, but you don't really see why you need to use them -- then don't. I actually sometimes feel like an alien in this sub because I don't have a dashboard with 20 containers in it for all the most popular apps, and prefer to write my own software, or find off-the-beaten path options. ---------------------------------------------------------------------- LegendEater · 2023-10-26 18:52 The fact that you're even seeing sponsored links is the first place shows that you aren't as good at Googling as you think you are ---------------------------------------------------------------------- LegendEater · 2023-10-26 18:52 It also doesn't necessarily use Raspbian. There are many distros. ---------------------------------------------------------------------- NeverMindToday · 2023-10-26 18:56 I've worked all the way across system administration, web development, product development, devops, cloud infrastructure, management etc - and I've worked with engineers that could use tools or abstractions without understanding what they were doing, and with engineers that understood what their tools were doing under the hood. The latter group who understood things at a lower level (even if they didn't operate at that level 95% of the time) were so much better to work with and had better career prospects as they are more sought after. Curiosity and caring about these things was one of the defining differences between a good engineer (or one with potential) and a mediocre one. Mediocre engineers in mediocre workplaces work by rote and staying within the lines defined by someone else - the better paid ones work in better environments where they are either building abstractions themselves or using lower level knowledge to solve unusual problems the mediocre ones can't. This applies all over too - even with things as seemingly non engineering related like CSS. A front end dev who understands the underlying layout models and has even read the specs operates so much better and more enjoyably than those that just seem to prod randomly with random Stack Overflow answers. Or a back end dev that understands the output of a SQL EXPLAIN, or an infra engineer that can tell you what a container is in Linux kernel terms rather than Docker commands etc. ---------------------------------------------------------------------- belibebond · 2023-10-26 18:57 This is true for any thing in life. Especially technical stuff. It helps that you have some understanding to start with, but you should learn how to start small and organically grow. You can't setup all and everything in one go. I would do in this order to make self hosting bite size, 1. Get docker working, with basic docker stuff. 2. Host one service and use it over local lan. 3. More services, but still strictly internal lan only. Get comfortable. 4. Get VPN and try accessing services through VPN 5. Now start research on how to expose, tools to use etc etc. ---------------------------------------------------------------------- NeverMindToday · 2023-10-26 18:58 I've worked all the way across system administration, web development, product development, devops, cloud infrastructure, management etc - and I've worked with engineers that could use tools or abstractions without understanding what they were doing, and with engineers that understood what their tools were doing under the hood. The latter group who understood things at a lower level (even if they didn't operate at that level 95% of the time) were so much better to work with and had better career prospects as they are more sought after. Curiosity and caring about these things was one of the defining differences between a good engineer (or one with potential) and a mediocre one. Mediocre engineers in mediocre workplaces work by rote and staying within the lines defined by someone else - the better paid ones work in better environments where they are either building abstractions themselves or using lower level knowledge to solve unusual problems the mediocre ones can't. This applies all over too - even with things as seemingly non engineering related like CSS. A front end dev who understands the underlying layout models and has even read the specs operates so much better and more enjoyably than those that just seem to prod randomly with random Stack Overflow answers. Or a back end dev that understands the output of a SQL EXPLAIN, or an infra engineer that can tell you what a container is in Linux kernel terms rather than Docker commands etc. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:01 Except using a VPS defeats the purpose of trying to learn how to deploy my own apps… ---------------------------------------------------------------------- NuK3DoOM · 2023-10-26 19:03 Maybe my path can help you. I don’t have any educational / professional background in tech, but it is my hobby. I started simple, wanting a NAS for my home. I downloaded TrueNAS and reading the docs and using a lot of YouTube videos (from multiple creators) I managed to get it up and running. Then I tried sharing but using permissions. It took me weeks to understand ACL and NFS shares. It took me a week to figure out that I need to share both parent and child dataset to access a child via NFS. I had to goggle a lot, but it was such a stupid mistake that there was no information about, so I resort to try and error. After I had it setup I thought it would cool to install pinhole. Then I had to learn VM on trueNas, Linux server, package managers and so on. I had zero experience in Linux. I managed to install docker and portainer via a tutorial but then I learned docker compose files and it made everything easier. I destroyed and created that vm multiple times. Tried Debian, Ubuntu server, fedora server … The pattern goes on, home lab will be about reading a lot and not afraid to try and break stuff. It is better to try when you don’t have a lot of services. Always backup your vm, if something breaks, nuke it and start from the backup. Keep it a piece each time. Don’t try to setup all at once. Have an idea about the end state but solve a small problem at time. They say the best engineers are the ones that are able to break complex problems in really small ones. Doing that I have a vm on true nas running more than 20 containers, a redundant pinhole on a separated server, everything accessible via VPN. Now I’m learning about podman and kubernetes, probably I will spin a second vm, try to migrate what I have a nuke docker. It never ends while you are curious. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:05 I’ve always struggled to find good books. And as a broke student in college, the ones I find are either too expensive or unavailable in my region. It’s also highly illegal and actually prosecuted to do piracy where I live, so I don’t want to do that… ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:07 I’m not interested in Plex/Jellyfin or any other media service that this sub seems to be obsessed with. I mostly deploy my own applications so I don’t get shortcuts and need to work most of the tools directly. That’s perhaps some of my frustration ---------------------------------------------------------------------- junialter · 2023-10-26 19:10 What does highly illegal mean? What happens with someone who \*copies\* a book from a friend where you live? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:10 SSH and DNS are gonna be my focus now. Never really got around to figuring out how to setup SSH because everything else was overwhelming, but this post has me motivated and focused on small areas of knowledge, on starting small and growing from there. And that’s what I’ll do ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:12 No, it means Google is shoving a lot of “sponsored” ads on top of everything regardless of what I search. They’re not always there, but sometimes crazy specific queries have them, while other generic ones don’t. I actually see them a lot less now that I use Opera’s built in adblocker, but I used to see them a lot more back when I used Vivaldi. I just mentioned them to make clear I know they exist and avoid them whenever available ---------------------------------------------------------------------- LegendEater · 2023-10-26 19:14 Haha, you still don't get it. If you're seeing them, you aren't even ad blocking properly. Learn to use the PC before trying to self host anything. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:16 It means Lawyers monitor illegal websites and will then harass you and threaten to sue unless you pay, and while it’s likely never gonna go to court, they’ve got plenty ways to make you waste time money and energy, which I’m just not willing to. I don’t even think your IP activity is legally your responsibility, but also don’t want the harassment, especially when legal proceedings against me could cost me my residency status here ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:19 That’s why I specifically mentioned the adblocker I use is built-in. I’m NOT actually adblocking, it’s a feature my browser has, added convinience, sure, does it occasionally slip up? Also yes. I’d worry when I’m actually trying to adblock lol. I can say I haven’t seem them at all in a very long time, but I’ve got smth like 8 flavors of chromium installed rn, sometimes I’ll use a non-blocked browser or the browser will not be up to date. It’s no big deal lol. There’s a reason adBlockers are constantly updating, they need to keep up blocking the new shit companies come up with to show ads (which I don’t even mind lol) ---------------------------------------------------------------------- Legitimate_Farm_3095 · 2023-10-26 19:26 > so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. no offense but maybe you just aren't using the right search terms. ---------------------------------------------------------------------- atheken · 2023-10-26 19:26 I wasn't trying to trivialize your experience, but my point was that you're just getting your feet wet. I've been doing software for 20 years, and a lot of this stuff takes time to learn and model how all the pieces fit together. It's really hard to learn it all at the same time, and I was trying to give you a blueprint for how to experiement and build understanding incrementally. Take it or leave it. ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:28 Maybe? That’s something I’m getting from this thread. But then again, just because people say VPN that doesn’t immediately tell me what kind of software I need to install and run. Once I know to run Wireguard, Tailscale or some alternative its easier to dive into the docs or search more specific stuff ---------------------------------------------------------------------- VexingRaven · 2023-10-26 19:39 Your router probably just supports 4 wifi SSIDs, maybe with some isolation between then, but nothing that would help with this. You also definitely would not want to put your public network nested like that as that is accomplishing the exact opposite: You'd be protecting your public network from your private one while giving your public network full access to your private one. I suppose you could do the reverse and out your private network on a second router, but I imagine that would break some stuff that relies on UPnP or NAT holepunching. The only way I know of to really do this right is with a router that supports VLANs. I use a Mikrotik router, you can get them relatively cheap and they can do all kinds of fancy stuff. You can also install something like OpenWRT on your router if it's supported. Another option is to brew your own with OpnSense. ---------------------------------------------------------------------- FruitieX · 2023-10-26 19:41 Yeah I agree it can be quite daunting, I recommend starting out small and trying to understand and getting one piece working at a time. That is the approach I've taken while migrating my workloads from manually deployed services on old PC hardware into a fully declarative Kubernetes setup on a more energy efficient mini PC. Sounds scary but it's been a lot of fun actually! I tried writing down some help on getting started, as well as short explanations on most of the infrastructure here in my homelab git repo: [https://github.com/FruitieX/homelab/blob/main/docs/configuratio n.md](https://github.com/FruitieX/homelab/blob/main/docs/configuration .md) Hopefully that's helpful if you decide to try setting up something similar! 😎 ---------------------------------------------------------------------- EsotericJahanism_ · 2023-10-26 19:43 Well I mean there's a reason why these skills are in such high demand. ---------------------------------------------------------------------- -SHINSTER007 · 2023-10-26 19:45 just chip away at it, I'm slowly learning SWAG/reverse proxy. Its complicated for me but its rewarding and eventually becomes easy. Such is the way with linux from my experience ---------------------------------------------------------------------- daYMAN007 · 2023-10-26 19:49 People are overcomplicating stuff because they fear for security. Move fast and break stuff, and just start somewhere. You will not die if you run your setup without reverse proxy for a week. Just go step for step till everything is nice and secure. Also the thing about understanding everything is a habit that you get in school. But in bigger software projects it's not feasible at least in my experience. ---------------------------------------------------------------------- -SHINSTER007 · 2023-10-26 19:50 these are just copyright trolls reporting your IP from a swarm to your ISP -- how would they know what you're doing if you use a proxy or VPN? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:51 So I was right about the wifi thing. I could create subnets, but I am curious now as to the nesting thing. Currently, I’ve got my ISPs router plugged into my home router’s wan port and set to pass through. This is necessary because the ISPs router is also what “authenticates” me within the ISP and allows internet traffic. How would plugging another output from this “bridge” into a router (so basically, being a sibling to my home network in the “tree of routers” I’d be setting up) protect my public net from the private one and not just create two whole subnets using the same WAN address, since I assume as far as the “bridge” is concerned, these two routers plugged are just another “device”. Am I wrong? ---------------------------------------------------------------------- Ieris19 · 2023-10-26 19:54 They probably wouldn’t, but my VPN is selfhosted so the VPN’s address is still under my name, just redirects my traffic through my home network rather than whatever I’m connected to atm. They’re more than trolls though, it’s basically like debt collectors, they will harass you until you pay, and actually send you physical mail, not report you to the ISP. It’s a grey area legally and they use it to extort people for money ---------------------------------------------------------------------- AlexHurts · 2023-10-26 20:07 It is daunting! There is so much out there. I constantly have to remind myself that I don't have the time/energy to build everything I can imagine because each step takes so much learning and troubleshooting. It can be hard to figure out what step 1 is when you need multiple things to work together. I've had so many nights where I think "ah, easy day at work finished, I've got the energy to make xyz work now." Then in order to troubleshoot step 4 or whatever I have to completely learn some new skill/topic and I go to bed past my bed time with no visible progress.... Definitely daunting. But I have to disagree with you on there not being info out there. There is so much on YouTube, and unless it's a really obscure service, lots of how to blogs and reddit posts on how to use it. ---------------------------------------------------------------------- hostetcl · 2023-10-26 20:07 Take a step back for a second. There are “so many names” because there are so so SO many concepts, and they are all very *complicated*. It is 1000% okay to not know these things and it’s even more okay to be confused by them. This stuff is not easy - most concepts are abstract, convoluted, poorly communicated, and very very specific. As an engineer, this never goes away - we just get better at navigating. IMO you’re at a point in your journey where you’d benefit from a technical mentor that has a strong grasp of these things already so they can help you navigate around the confusing bits. You were right to reach out here and ask all the questions you’ve asked, but I don’t think this will be as effective as having someone that will let you pick their brain in real time. Do you have someone in your circle that does this sort of thing professionally? ---------------------------------------------------------------------- throwaway234f32423df · 2023-10-26 20:12 you know that apps still have to be deployed to the VPS, right? ---------------------------------------------------------------------- Do_TheEvolution · 2023-10-26 20:14 >yet I can barely find info on HOW to set up this things [This repo should help.](https://github.com/DoTheEvo/selfhosted-apps- docker/) Its core is very detailed guide on how to setup [caddy](https://github.com/DoTheEvo/selfhosted-apps- docker/tree/master/caddy_v2) as a reverse proxy and then the typical posted stuff from nextcloud, vaultwarden, bookstack, uptimekuma, minecraft, jellyfin, wireguard vpn, syncthing... shitload of stuff there.. ---------------------------------------------------------------------- talentedfingers · 2023-10-26 20:17 Agreed with start small. Trying to implement everything in one go would be too overwhelming to troubleshoot. Keep lots of backups of working configurations in case you need to revert any changes. I highly highly recommend SmartHomeBeginner https://www.smarthomebeginner.com/, which is how I learned to setup mine. They have many articles that explain step by step what they are doing and why, and also have a discord where you could ask for help. ---------------------------------------------------------------------- disguy2k · 2023-10-26 20:22 I usually type in " guide" in Google and get the basics of what I need. The most important first step is to flesh out the structure of your network. What services you want to, and what the best way to serve them is. Once you determine the structure you can start to implement it. Start with the foundation OS, hypervisor, container systems. From there you add your services. Read what others have done, and more importantly what worked for them. Eventually you'll end up with something that can keep itself running for a few weeks at a time without you needing to babysit it, or have it crash completely and need to be rebuilt from scratch. ---------------------------------------------------------------------- itsbentheboy · 2023-10-26 20:24 Its the complexity. Realistically, self hosting is the same as running a mini infrastructure. In a regular company, different teams or people handle each different aspect. At home in the lab though, you wear all the hats. you're the full stack administrator from hardware to edge networking. on top of that, you're usually doing things outside best practices, or on unverified platforms, just because you are using what you have available, and not the "standard deployment" recommended by any manufacturer. Workarounds, hacks, and compromises make following any Official guide more challenging, because it rarely fits neatly into what you have to work with. --- On information gathering: This is also a struggle, because you are right, a lot of content out there is just a brief overview, telling you what something is and not how to deploy it. Its a 2 part issue, the first being that a ton of content creators out there dont actually know any more about the products than they put in their videos or blogs. There is an entire industry of people that make light intro content, but have never actually deployed the software they talk about more than a proof of concept or demo installation. Much less actually running it as a stable component of their own infrastructure. Their content is meant for clicks and ad revenue, not to be an instructional guide. The ones that do go more into depth are often just presenting their specific workarounds that work in their specific labs. These are the better channels, but for each one of these practical enthusiasts, there's hundreds of "check out this new tech" bloggers that have little to no experience. some of them aren't even humans, but repost-bots and AI generated content. The second part has to do with the current issues in many projects, and that is a lack of good documentation. Plenty of developers, companies, or organizations are putting less effort into documentation these days. Its a trend seen across the industry. the assumption of pre-existing knowledge, or that the "tools document themselves", but what the outcome ends up being is half finished examples with no practical use case, and thin documentation that barely covers the basics. there is many reasons for this, some being a lack of budget to hire technical writers, project culture, or pace of development. Slow development can lead to slow documentation, and rapid development can lead to a lack of it as information stays relevant for shorter periods of time. Also, as the complexity increases, making documentation for every potential environment becomes an impossible task. --- My 2 cents, is that the entire industry needs to take a look at improving some of the basics. Big projects with massive success are built because of their documentation, and many promising projects fail due to a lack of it. This is a problem that the entire industry needs to address, because as things keep growing in complexity and users coming into the field lack the assumed traditional experience of old-school admin work, the collective understanding of these systems is degrading towards Warhammer 40k levels of understanding the technology around us. Its apparent in the self hosting world because we have to do each part of the deployments in our labs ourselves, but its not a problem exclusive to us. ---------------------------------------------------------------------- gjvnq1 · 2023-10-26 20:24 As someone who has been self-hosting stuff on and off since middle school, I feel like nowadays the choice paralysis is much much worse. Oh, and the sheer amount of outdated advice and info is a big issue. ---------------------------------------------------------------------- fishypants · 2023-10-26 20:35 For me and I'm not sure if it's been mentioned here already or not, but I don't really have any programming skills or super geek out on this stuff. But I like the tools, right? So teaching myself some of this is hard, especially when there is an attitude with most of this stuff that you should know everything about docker containers and weird networking rules, etc. Sometimes people are helpful, more often than not, it's a super short answer you have to interpret and decipher to figure out. ---------------------------------------------------------------------- zer04ll · 2023-10-26 20:36 Because security isn't easy. Know you know why system admins have a problem with programmers that just expect things to work. ---------------------------------------------------------------------- ProfessionalAd3026 · 2023-10-26 20:50 If those applications are running on your Windows machine natively, pray for them not to be compromised. TLS doesn't secure your application from attacks, and IIS isn't something I'd expose to the internet if the Windows beneath isn't hardened for DMZ operation. Honestly, all my career, I avoided exposing Windows to the internet if possible. To be fair, I left the Windows world 7 years ago and a lot has changed (and a lot not). ---------------------------------------------------------------------- ProfessionalAd3026 · 2023-10-26 21:07 I guess you’ll realize your tone is off when rereading it. Aside from that. The reason why people tend to hand out containers is that it becomes tedious to install 3rd party software correctly. If you don't want to run it in a container, use docker history to figure out how it was built. You could redo it on bare metal. For years, I've been self-hosting on bare metal. You'd be amazed at how many services you can run in parallel on a single Linux server without using any container. The bigger the arsenal of software running, the harder it will be to keep dependencies aligned as others said. Start simple. Server, dnsmasq for DNS and DHCP, add an application Add valid certs Add access via VPN At some point, you might want to enter virtualization in the form of Docker or VMs. ---------------------------------------------------------------------- Aurailious · 2023-10-26 21:24 I self host for the same reason I play Factorio, it's a constant stream of solving problems and adding new ways to do things. There is a lot to do, but you don't need to do everything right away. Like a lot of people said, just break it up. I use Vikunja to create a list of things I want to and then break those things down into hour or weekend projects. There is a challenge in the "bootstrap" process. But once you get something going then its easier to keep adding. Stuff like "Reverse proxy, VPN, Cloudfare bullshit" is not super necessary at first. Start with just getting an app going at an ip address and port that's only accessible on you LAN, then add the other stuff. ---------------------------------------------------------------------- lupercal93 · 2023-10-26 21:25 Haha yes gotta love autocorrect ---------------------------------------------------------------------- kitanokikori · 2023-10-26 21:26 If you don't want to deal with these things and it's just you or maybe a few others, here's the easy way: * Define the services for each host (i.e. physical + virtual machine) in a single Docker Compose file. You can even put all of these docker- compose files in a single Git repo * Install Tailscale on all your machines - this is really easy and will let you securely access everything without having to forward ports or worry about the attack surface of the public web * Set up something like Heimdall or https://github.com/tailscale/golink to have easy to use shortcuts for your services This gets you out of SSO signin, certificates, reverse proxy setups, all the things that are (understandably!) complex and annoying to set up ---------------------------------------------------------------------- MoogleStiltzkin · 2023-10-26 21:26 are u sure u have been looking hard enough? you can go youtube type "remote homelab docker vpn" or some variation of that. And you will get some results that would lead to things like from the sources i linked. anyhow, just watch these youtube channels which may help you setup remote access if that's what you need. if you don't, you don't have to setup remote access. some homelabs are lan only. Nothing is wrong with that if that's all you need, not to mention that is probably the most secure setup. Only once you start trying to setup remote access do you have to accept some level of risk. Hence why they suggested to setup vpn or cloudflare tunnels to do this safer. [https://www.youtube.com/@christianlempa](https://www.youtube.com/@chr istianlempa) [https://www.youtube.com/@TechnoTim](https://www.youtube.com/@TechnoTi m) [https://www.youtube.com/@DBTechYT](https://www.youtube.com/@DBTechYT) [https://www.youtube.com/@WolfgangsChannel](https://www.youtube.com/@W olfgangsChannel) [https://www.youtube.com/@AwesomeOpenSource](https://www.youtube.com/@ AwesomeOpenSource) [https://www.youtube.com/@LAWRENCESYSTEMS](https://www.youtube.com/@LA WRENCESYSTEMS) ​ Look up vpn cloudflare tunnels [https://www.youtube.com/watch?v=ZvIdFs3M5ic](https://www.youtube.com/ watch?v=ZvIdFs3M5ic) ​ or tailscale vpn. installing the package on pfsense router, then immediately you can go straight to configuring in the UI [https://www.youtube.com/watch?v=P-q- 8R67OPY](https://www.youtube.com/watch?v=P-q-8R67OPY) ​ or you setup vpn the usual method [https://www.youtube.com/watch?v=PgielyUFGeQ](https://www.youtube.com/ watch?v=PgielyUFGeQ) ​ but what you should not be doing is simply just exposing your network online without any one of the vpn methods i mentioned. thats how you get wrecked. and nobody to blame but yourselves if you do that by throwing caution to the wind. ---------------------------------------------------------------------- mdcbldr · 2023-10-26 21:44 Selfhosting has a steep learning curve. You have to know hardware, networking, servers, front end, backend, VMs, docker/kubernettes, etc There is a ton of stuff to learn. The concepts are not difficult, the implementation is non-trivial. ---------------------------------------------------------------------- virtualadept · 2023-10-26 21:52 Thank you. Going into my .plan file, going out in the next push, whenever that is. ---------------------------------------------------------------------- virtualadept · 2023-10-26 21:53 It really does. I keep looking for Terraform related stuff for work, and I keep finding crap blog posts written with ChatGPT that talk about Orion's Arm and stuff like that. ---------------------------------------------------------------------- virtualadept · 2023-10-26 21:53 You are very welcome. ---------------------------------------------------------------------- mrcaptncrunch · 2023-10-26 22:30 You need to know the tech side, but also how app X works and how it does things. So, if you want to host X app, you need to know what dependencies it needs to run, you need to know what ports it needs, if it needs communication with something else, you need to know the ports/subnet. If when you upgrade, you need to run some command, you might want to run that on startup every time, etc. So, it’s not just hardware or theory, but also what that software is doing, how it’s doing, protocols it might implement, and that’s for it and other tooling it depends on. ---------------------------------------------------------------------- Beastmind · 2023-10-26 22:34 Also instead of waiting to host everything you see here or on the web, start with just what you really need as in needed "right now". You'll expand naturally once you start anyway ---------------------------------------------------------------------- JaegerBourne · 2023-10-26 23:00 I hate the fact you gave your honest opinion and othets have to downvote you. I myself rarely use ChatGPT to find a *direct answer*, instead I use it to give me resources to find the answers for the questions I didn't think about asking or knew existed. More like thinking outside the box or brainstorming. And always do your due diligence to verify any answer it gives. ---------------------------------------------------------------------- SchmeepyDooDoo · 2023-10-26 23:09 Networking is extremely dense and extremely hard. Its also a space filled with professionals who do it for a job. Many on this sub are current or ex networking or security professionals who understand this shit like the back of their hand. Im no idiot with IT - I’ve had pc’s my whole life and work in close proxiimity to IT. But networking, security and the clusterfuck of everything associated with it is honestly about as hard to get into as actuarial science. It requires such a heavy base of fundamental concepts that if you dont start at the bottom and work up, you have no clue wtf is going on. ---------------------------------------------------------------------- Wixely · 2023-10-26 23:57 I use it on x86 Debian machine ---------------------------------------------------------------------- pattymcfly · 2023-10-27 00:11 Learning how to learn fast is the best skill. ---------------------------------------------------------------------- lvlint67 · 2023-10-27 00:16 > I can’t get anything actually running! Most of my time in my sysadmin work is sitting there staring at the screen reading error logs about why some new product isn't working. make sure you ACTUALLY have a solid grasp of networking. dig into the osi model or the 3 layer dod model. Go further than "yup. encapsulation. got it". Look at the actual headers in layer 2 and layer 3. Figure out how packets get routed and how frames get forwarded. Look at how NAT actually mangles packets. Understanding hos the traffic flows will make troubleshooting much easier. ---------------------------------------------------------------------- lvlint67 · 2023-10-27 00:19 > looking at the apps that are only documented on how to run in Docker I will say... **IF** you can find their actual docker files things can get a lot easier. There should generally be a "FROM" line telling you what base system they started with... and then a bunch of sequential commands that build and run the system. Some developers are better about this than others... ---------------------------------------------------------------------- Adenn76 · 2023-10-27 00:21 "self hosted DNS server" is what I searched for, but I also knew what I was looking for "Unbound" with PiHole is what I use. Also, Windows has a host file that it uses. %windir%\System32\Drivers\etc\hosts %windir% is the environment variable for your windows install directory. Usually C:\Windows You can add your entries in there, just like Linux, though I am sure the syntax might be slightly different. I really haven't played with host files on Linux, yet. ---------------------------------------------------------------------- machstem · 2023-10-27 00:32 If you've never gone and built your own system, or started a network from scratch without a container based system, and you've relied on stacking your own environment for a long time, you recognize it as you would any skill. I'm sure I could build a shed, for E.g. I know to get the wood, the panels, the cement, and all the things I'll have all documented in front of me. Then I will start, realize I forgot I also needed a permit, which I then found out I might also need to run power to it, so now shed has become something that doesn't feel impossible, it feels daunting Why? Well, I don't build sheds, I don't keep building things that would allow me to have all the framework, project of getting my shed built. So, as with a homelab, take it one step at a time, and if rhe shed isn't the house, you're allowed to skip a few things and stick with what works for you. I use homelab and self hosting as a passion, but I also have 25 years of networking and systems administration experience so docker and containers just made my normal, tedious job of building up complex network infrastructure, feel like a job, and my homelab and self host happens to be a hobby Treat your homelab as a passion or simple hobby, or use it as a skill building tool for work Overall it's just time, experience and expectations that either drive you forward or set you back ---------------------------------------------------------------------- edthesmokebeard · 2023-10-27 00:49 Begging the question. ---------------------------------------------------------------------- mpw-linux · 2023-10-27 00:56 Decide what you want to self-host. Keep it simple at first then work your way up as need be. You could first self-host a simple web site. You could use Tailscale free vpn with 100.x.x.x ip's to test your setup from another computer anywhere. ---------------------------------------------------------------------- John_Mason · 2023-10-27 01:05 This guy has the best guides I’ve seen. Hope they help! https://drfrankenstein.co.uk/ ---------------------------------------------------------------------- rocket1420 · 2023-10-27 01:43 I used to be like that. I loathed using programming libraries, for example, because why should I figure out how someone else's code works when I can just write my own? I'm 41 now and I'm definitely past that attitude. It helps since things are far more involved now than when I went to school. There are many blogs and YouTubers and such, but I definitely feel like it's helpful to watch and read as much as you can, because not everyone is, well, right, or covers the aspect in which you're interested, things change, avenues are opened that you didn't consider, etc. It's not necessarily a simple and straightforward field, but it's very rewarding. ---------------------------------------------------------------------- elingeniero · 2023-10-27 02:10 You don't have to do it, 99.9999%^([citation needed]) of the population don't. If it doesn't interest or excite you then it's not worth it. ---------------------------------------------------------------------- Candle1ight · 2023-10-27 02:37 One piece at a time. There's so many moving parts, trying to do it all at once is a great way to burn out. My setup started as a few game servers behind a firewall, as new apps caught my attention and requirements grew I added or revamped piece by piece until I ended up with that I have now. Most people here have been doing this for years and/or went to school for it, don't feel discouraged if your setup doesn't look the same off the bat. ---------------------------------------------------------------------- celticchrys · 2023-10-27 02:49 Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you'll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That's a more realistic set of expectations. I've done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn't exist yet. VPNs weren't a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn't you being inadequate, it's you trying to do multiple jobs that each require constant professional development. ---------------------------------------------------------------------- celticchrys · 2023-10-27 03:02 What are your biggest needs? Like, what need for yourself as a user do you need an app to fulfill? Make a list. Rank them from most important to least important. Then, try to come up with the simplest way to do the first one. Do that for each one. This is before you install or setup anything. Just brainstorm. What is the need? What is the simplest way to provide that? For example, if your biggest/most desired need was to watch movies/listen to music from your collection on any device in the house, well, you could set up Plex, port forwarding, VPN, blah, blah, or you could set up an SMB network share or three on a box in your house, and then just play directly from it on any computer/phone/tablet/etc. in your house, using a media player on each device. Now, if you need to access that media collection as you travel the world, then that's a different need. If you need to share it with a bunch of other people outside your LAN, then that is a different need. Be as discrete as possible in defining your problems. You don't have to do a thing just because other people do it. I was playing with virtual machines before Docker existed, but I'm not currently running any, because I don't need to. My needs are modest at present. If you're doing so much that you can't enjoy experimenting and exploring, then you're ruining the fun of self hosting for yourself. ---------------------------------------------------------------------- celticchrys · 2023-10-27 03:12 This is the way. ---------------------------------------------------------------------- geerlingguy · 2023-10-27 03:47 [Jeeeeeoooorrrbbbb!](https://youtu.be/8C4ayBHTES0?t=102) ---------------------------------------------------------------------- rustin-dustin · 2023-10-27 04:47 NetworkChuck on utube ---------------------------------------------------------------------- rustin-dustin · 2023-10-27 04:53 What happens when you start crawling, and ur feet don't work on this floor... So one spontaneously combusts, whilest loosing all ur data fluid, as a baby, looking everywhere on the webbernets, trying to find a replacement foot, that is free, your ADHD kicks in, you then have 11teen feet , and u are trying to figure out how to crawl with all of them..... Except they all attach with different connectors, cords, protocols, and color thread... Oh yeah, all the how to cartoons are in different languages and using differing terminologies... Do you just keep on makin ur FrankenBaby do the best he can, and give him some delicious boob juice every now and then??? ooooor, put him in the Microwave with his gold fish, turn it on ZipZap, click Abort and Retry, because you know what the Ignore button does never, and try again??? ---------------------------------------------------------------------- TheCaptain53 · 2023-10-27 05:30 The best thing I could advise is to just start. When I started my home server setup, I researched it a bunch beforehand, but it was ultimately to be able to achieve a goal: download content and deliver it. You make a start - eventually you start refining the processes, maybe a software package you picked out actually isn't great, or maybe you need to add another software package in there too. First thing to do is to determine what are you trying to achieve. If your goal is the homelab itself, then try to build a setup that gives you the most flexibility. This would typically be something like ProxMox (the point is not learning ProxMox, it's the flexibility it gives you), but I'd prefer to just install straight Debian on a laptop and run Docker instead. If it was a server, then I'd consider the use of ProxMox or another virtualisation layer. Once you've made a start, you just keep playing and tweaking your setup until you've got something that works for you. And remember - if it doesn't work, you can always trash it and start again! You've already worked through a bunch of kinks, so getting to the point you did previously is now way faster. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:37 Sadly no such thing, I wish I had someone who could mentor me… I’m searching for an internship in development at the time too so ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:39 Absolutely not! I can deploy apps to very real not virtual private servers. There’s no need for virtualization, but it is nice. And most people who talk VPS talk about external services… ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:40 I usually struggle to find what solves “problem” which is probably down to my Google skills ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:43 I can’t imagine not being a technical person in IT. If you’re running anything already color me impressed, because like you said, diving through all these things is hell, and I kinda know what I’m supposed to be doing ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:47 I’m a Software Engineering student, I understand the Software WAY more than I understand networking or hardware around it ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:51 I’ve got an exam on this shit January. Idk why people keep assuming I don’t know about these things after I’ve repeatedly mentioned I’m a student and I’ve got a theoretical grasp of networking. But knowing about packages or just network traffic in general doesn’t help when say, I can’t get a DNS server running. And in all honesty, most of my issues stem from poor resources out there that simply don’t have a “solution” (as in, what software would solve the issue) but rather, explanations that I already know about what things are ---------------------------------------------------------------------- Ieris19 · 2023-10-27 05:54 That’s basically the problem, yet almost everywhere people make these look so essential and necessary ---------------------------------------------------------------------- TikTak9k1 · 2023-10-27 06:37 It depends really. Are you exposing web services to the outside? Then a reverse proxy can be helpful for serving all SSL requests. Don't want to expose them? Having a VPN would be useful. ---------------------------------------------------------------------- gagalkannm · 2023-10-27 08:22 This repo is worth checking out. ---------------------------------------------------------------------- Jealy · 2023-10-27 08:29 Not sure how you can say you hate ChatGPT and then state you don't know how to modify the hosts file on Windows! ChatGPT returned VERY good instructions on how to do this (including flushing the DNS cache). ---------------------------------------------------------------------- Ieris19 · 2023-10-27 08:32 Because ChatGPT is straight up wrong more often than not. I use Codex daily, and maybe 9/10 answers are quite correct, but that 1/10 is wildly wrong, and I’m not willing to risk it when learning something new. I’d rather read docs or something more concrete. Plus, I’ve mentioned many times in this thread I’m banned from GPT for no apparent reason. I use Codex, because it helps me, and if it fucks up, I know how to correct it (and I can realize it fucked up). If I don’t know something, there’s no way to catch the mistake and it’ll just be a headache wondering why something broke down the line… ---------------------------------------------------------------------- maidenmaan · 2023-10-27 08:33 Level up your engineering skills! ---------------------------------------------------------------------- Ieris19 · 2023-10-27 08:34 How do you suggest I go about doing that? ---------------------------------------------------------------------- Ieris19 · 2023-10-27 08:55 What repo? ---------------------------------------------------------------------- hilarino · 2023-10-27 10:02 Well, chatGPT works very well to answer questions regarding self- hosting and Linux questions in general. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:03 Well, it’s almost like you missed a point about me being banned for no reason… ---------------------------------------------------------------------- hilarino · 2023-10-27 10:05 I'm sure there is a reason. I would try registering again with other email. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:07 No, I could one day, tried like a month later and couldn’t send messages, now I can’t even login. Didn’t ever do anything weird, no asking any weird questions, nothing beyond discussing my CV, reports or student council documents. I mostly used it for rephrasing… ---------------------------------------------------------------------- hilarino · 2023-10-27 10:09 Try registering again with a different email. It is so worth it, specially for sysadmin questions and small scripts. ---------------------------------------------------------------------- hilarino · 2023-10-27 10:10 Maybe it didn't like accessing personal information? No idea, just a guess ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:11 I won’t be learning from any GPT model, I use Codex daily, a LOT and when it’s wrong it’s very convincingly so. I only use it when I have a pair of experienced eyes that can notice mistakes. Else I’m just setting myself up to miserably wonder what’s wrong in the future ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:15 I didn’t even give it personal info. Mostly just told it to rewrite the bits about my experience which are hardly personal (things like, worked at a restaurant, but more detailed) ---------------------------------------------------------------------- lvlint67 · 2023-10-27 10:24 If you want a DNS server and you don't want a lot of extra fluff.. bind has been powering the Internet for about forever now. ---------------------------------------------------------------------- delimitesvd · 2023-10-27 10:25 my love here :D ---------------------------------------------------------------------- hilarino · 2023-10-27 10:26 So far, on the rare occasion when GPT4 is mistaken on a sysadmin question, I notice when the thing doesn't work or produces an error. I copy/paste the error or explain the issue and GPT4 corrects itself and then provides a correct answer. So you would notice the few errors this way, by testing things after implementing. Imagine a person a 100 years ago saying that doesn't want to learn from books because sometimes there are errors, and only wanted to learn from masters. You are disregarding what several persons with experience are telling you is a good tool for the job, for no good reason. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:31 No, I’m disregarding it because I already use it extensively, I know it’s strengths and limitations and I know a little about how it works, predicting the likeliest next word within a context is not quite a teaching or learning tool. AI is a great tool, don’t get me wrong, but this is not what it’s for This is basically the opposite of what you described. I’m avoiding learning from other people and trying to get information from the source, because AI, much like people, is inaccurate and prone to mistakes. Most people here have advised the opposite, that I actually go read the docs. Or RTFM as the community likes to call it. Which is and was already my preferred approach anyway. Just asked GPT with a different account to setup a DNS and the instructions where vaguely more detailed than what I already knew/has been mentioned elsewhere in this thread. And I assure you no one’s been writing tutorials over here… ---------------------------------------------------------------------- seabuckasa · 2023-10-27 10:41 That's not bad ---------------------------------------------------------------------- hilarino · 2023-10-27 10:42 Agree that reading the appropriate docs is the best approach to learn. What GPT is excellent for is to answer questions quickly when you don't want to read the docs, the docs are too daunting or you don't even know where to start. As an example: I never used a windows machine as a server. If I wanted to install a DNS server on windows for some reason, I would ask GPT4 for step by step instructions, like this: https://chat.openai.com/share/666c3428-624b-4c44-a9a7-5a1f24ac970a ---------------------------------------------------------------------- Ieris19 · 2023-10-27 10:54 I don’t bother paying for GPT-4 I’ll get Copilot X eventually but I don’t think that’s out. I rarely use ChatGPT so I think it’s just a waste ---------------------------------------------------------------------- Jealy · 2023-10-27 11:01 > My issue was more so that I had the impression Windows didn’t have host files… My point was only that ChatGPT would quickly answer that question quickly and accurately, saving you the entire headache. I wouldn't say it's wrong more often than not. ---------------------------------------------------------------------- mrobo11 · 2023-10-27 11:03 The greatest advice my teachers in university told me was to tackle everything in life with the foundation/philosophy of K.I.S.S. = Keep It Simple Stupid. It's more of a code to live by; to which you should ask yourself the question first "why do I need this thing?" What problem is it solving, and is there a simpler method? Usually, most labbers don't even need 10% of what we think we need. We introduce complexity for the sake of complexity. Just be comfortable with an ecosystem or make your own that aligns with your values/principles. Hone your senses by questioning the conceptual integrity of others (everyone has an opinion, don't take their opinion unless they're a trusted person that you respect). If you want to emulate someone based on their own system that matches yours, then do exactly to the letter what they have done and mimic their approach. Most times, you find that once you've concluded your journey you often feel it wasn't worth the journey in the end and you take those hard lessons and sometimes failures and it bridges your understanding further to the truth of a matter. ---------------------------------------------------------------------- FierceDeity_ · 2023-10-27 11:32 Because it's a complicated topic, you're just feeling the actual responsibilities of doing it right that many are ignoring now. ---------------------------------------------------------------------- FierceDeity_ · 2023-10-27 11:33 Please keep doing that, it seems people blindly follow tutorials too much and then think they're experts. You're on the right track to become an expert by doing things slowly and from scratch. Bottom-up > Top-down ---------------------------------------------------------------------- Ieris19 · 2023-10-27 11:42 Honestly, I feel like DFT of knowledge is both a boon and a curse ---------------------------------------------------------------------- FierceDeity_ · 2023-10-27 11:45 It is a curse. You look at projects and get a bad feeling, ending up not using them because you know too much. You do start severely limiting what you "can" use responsibly ---------------------------------------------------------------------- PovilasID · 2023-10-27 12:08 I got hooked by Youtube videos from TechnoTim Christian Lempa and others. Videos include screen capture what they are doing so it bridges some things that may be missing from text guide because it may be 'obvious' if you are doing it all the time. I keep to principal that I have two 'sections' in my lab 'production' and 'testing' Testing I am mucking about testing stuff it works and production has some stuff that I rely on. ---------------------------------------------------------------------- l0c0dantes · 2023-10-27 12:25 On a certain level, most of those things don't matter. Yes, reverse proxying and being able to point a domain is nice A VPN more secure Cloudflare (either tunnels, DNS, or DDoS protection) is also good You can always do better with more "best practices" but if you want to stream a movie to some friends? Throwing up a server, opening a port and providing an IP just works as well. The better you get the more fancy you will be. Its very much a tinkering hobby. Find a reason you want to tinker, and if you enjoy the tinkering you will keep up with it. My current dashboard points to a direct IP. Most of my services are reverse proxied but for some reason I can't get CORS to work well with dashy. At some point I'll get around to fixing it, but it gets the job done now. ---------------------------------------------------------------------- hostetcl · 2023-10-27 12:59 You’ll probably find some people with shared interests when you get into the work force that could help. Stay hungry! ---------------------------------------------------------------------- Legitimate_Farm_3095 · 2023-10-27 13:09 > But then again, just because people say VPN that doesn’t immediately tell me what kind of software I need to install and run not everyone should be expected to tell you everything or hold your hand. the information **is** out there and some of you just aren;t looking hard enough, using the right terms, or looking at all. this sub is a great example of people who come here looking for not help, but step by step hand holding. ---------------------------------------------------------------------- Nealiumj · 2023-10-27 13:32 As a computer science student you should know: just break it down into small chunks. 1. Get service to run locally (IP / port) 2. port forwarding, access remotely 3. Set up a free DuckDNS domain, Lets Encrypt, use for a bit 4. Buy a cloudflare domain, set up dynamic dns, Lets Encrypt (again), swap away from DuckDNS 5. Set up Nginx reverse proxy, port forward to that instead, proxy to service’s internal IP + port 6. Run more things! That’s generally what I did.. I’ve been self hosting for about a year now and I’ve got 6 services going now.. once you get going, it’s quite easy! ---------------------------------------------------------------------- Ieris19 · 2023-10-27 13:43 Well, the information *is* out there. But the whole point of this post is that when I try to research how to setup something, most results tell what that something is without any indication of how to set it up. Maybe it’s my fault, I don’t expect any hand holding, but sometimes being pointed to a list of tools to read the docs of would be welcome ---------------------------------------------------------------------- mpw-linux · 2023-10-27 14:07 Making web apps using the Go programming language is fairly easy. You don't need overly complicated frameworks. You could create a simple front-end in Go to call various Grpc services that might call a database to reply with some info about your system. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:12 Just as easy as making them in Java, C# or Python. That doesn’t make me despise Web Development any less. The decades of poorly stacked standards, shortcuts and workarounds needed to run “modern” websites just makes me plain uncomfortable. Simple as that, ---------------------------------------------------------------------- mpw-linux · 2023-10-27 14:15 What programming languages do you know? Are you using Linux? Do you know Linux? If you are going to Self-Host you need away to expose your applications to the user so that requires some front-end - very simple or as complex as you want it. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:16 I know Java mostly, but also a little Ruby, Python, C# and bare minimum C ---------------------------------------------------------------------- mpw-linux · 2023-10-27 14:30 How about this: forget about docker,containers, cloudfare /etc. Install Nginx,php, create a simple page that displays php-info. Get 3 cheap thinkPads, install Linux on them , Install Tailscale on each ThinkPads then you can communicate between each machine. Learn the basics of networking from a higher level. You could then program a simple client/server app in any language you choose that will teach you basic networking. When it comes down to it everything is networked. Once you have a client/server setup you then could monitor it by accessing the log files to see the connections from each machine. If any of this does not appeal to you then maybe Self-Hosting is not for you and that is fine - no big deal. Do something that you enjoy. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:34 This is bullshit advice. First, I already have developed many distributed systems. I understand how they work and know how netoworks work. Ditching Virtualization and Cloudfare is the only advice I think is valid. I don’t need to buy any fucking hardware, I got a dedicated server and a daily use machine, which is plenty. None of that appeals to me, but for example, developing my own databases, with extremely simple front-ends. My own selfhosted Git repos and VPN/DNS do appeal to me. Comments like yours all over the internet are precisely what annoy me from the internet. Everyone just assumes what worked for you works for everyone and if it doesn’t maybe this just isn’t for you… ---------------------------------------------------------------------- SwizzleTizzle · 2023-10-27 14:34 > documentation around it sucks No it doesn't. You wrote that you wanted to know how volumes work, have you read this page and the pages it links to?: https://docs.docker.com/storage/volumes/ ---------------------------------------------------------------------- mpw-linux · 2023-10-27 14:37 Just get some books forget about watching all that YouTube stuff. Learn programming and have fun. If you are really serious then take some college courses in programming and networking. The whole fun of it is experimentation as the software is free and the hardware is cheap not like the old days. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:38 You missed the point. I know how to create a volume. That’s not my point. Docker does a very poor job at explaining how and mostly just presents itself as magic. “It just works”. Containers are explained as “kinda like VMs but they share a kernel with the host” and that’s about it. It doesn’t explain what they are for example, and rarely you’ll see anywhere that does. Sometimes I’m curious as to how things work ---------------------------------------------------------------------- mpw-linux · 2023-10-27 14:47 Maybe you have to go to a 4 year university where they really teach you. A 2 year It degree is from where - might not be enough to get you started. I remember taking lots of programming courses at the Harvard Extention school while working as a programming during the day. I would spend nights and weekends at the School's lab programming my assignments. You gotta get your feet wet. Find a really good mentor or teacher that will guide you as you seem a little bit lost with all the complexity. Practice the KISS principle. ---------------------------------------------------------------------- SwizzleTizzle · 2023-10-27 14:48 They don't though. Their documentation is full of information. Want to know the architecture? Read here: https://docs.docker.com/get-started/overview/ Want to know about the namespaces they reference in the architecture? Read here: https://man7.org/linux/man-pages/man7/namespaces.7.html All your responses in this thread is you whinging about how documentation sucks and you can't find anything, yet it's all right there for you - if you *actually* read it IMO you don't know anywhere near as much as you keep claiming to. ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:49 I don’t yet have a degree, which is why I called myself a student. I’ve got probably the best uni in this country, a very practice oriented degree with a good measure of theory still and I’m still half way through it. But all I’ve got left is electives, internship and bachelor sooo ---------------------------------------------------------------------- Ieris19 · 2023-10-27 14:53 That architecture page still doesn’t say a word about WHAT a container is, just that the Daemon does the heavy lifting, which still doesn’t tell me jack shit. This post is about me whining about how inaccessible documentation is, because it’s either too abstracted or too intense, and there seems to be no in between ---------------------------------------------------------------------- SwizzleTizzle · 2023-10-27 14:59 Literally quoted from that page: """ The underlying technology Docker is written in the Go programming language and takes advantage of several features of the Linux kernel to deliver its functionality. Docker uses a technology called namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container. These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace """ So go read about namespaces, the kernel provides them. Or just keep bitchin' about how your medium-article-of-the-day doesn't cover namespaces & cgroups ---------------------------------------------------------------------- Ieris19 · 2023-10-27 15:03 I’ve read the docker docs in depth. Oh wow, a SINGLE sentence in a sea of bullshit marketing, pardon me missing it. It’s not even clear whether namespaces are a Linux thing or a Docker thing in that sentence and there’s no link. So pardon me from assuming that wasn’t really an explanation… Everyone here acts like I only read fucking articles and it pisses me off. Just because there’s a single sentence on it doesn’t mean Docker isn’t presenting itself as magic in 80% of cases… ---------------------------------------------------------------------- SwizzleTizzle · 2023-10-27 15:16 It's not bullshit marketing. You have the architecture definition, then a paragraph explaining what underlying technology is used. If you want to know more about the underlying tech, then **search it yourself** (or, you know, read the manpage I linked you on namespaces) Stop claiming documentation is shitty when it's not. Toodles. ---------------------------------------------------------------------- InspectionLong5000 · 2023-10-27 16:55 Right, but that's what my question was about. Based on having ports 443 and 80 open and forwarded to the windows machine, how vulnerable is it? Just praying they don't get compromised doesn't give me any sort of indication other than if they do then = bad. ---------------------------------------------------------------------- TBT_TBT · 2023-10-27 18:38 You might know about concepts in theory, that however doesn’t at all mean that you „know“ SysOps. And being a „2 year student“ doesn’t mean anything in this field, which is way bigger than that. ---------------------------------------------------------------------- xh43k_ · 2023-10-27 19:23 Senior DevOps Engineer here with 15+ years of experience in system engineering now, even with my skillset and experience I am improving my setup piece by piece. When I look at my old docker-compose versions and general setup, I sometimes laugh how I could've been so dumb. You don't have to and won't learn and set everything up at once, ever. ---------------------------------------------------------------------- soupdiver23 · 2023-10-27 20:00 >don’t know if it’ll make me a better engineer haha it will :) ---------------------------------------------------------------------- mpw-linux · 2023-10-27 20:55 sounds great - so you are all set! ---------------------------------------------------------------------- mpw-linux · 2023-10-27 20:59 Then don't take the advice and just do what you want to do. Everyone is giving you their own opinions on what might help you. if want to "selfhosted Git repos and VPN/DNS" then do it - case closed. ---------------------------------------------------------------------- Mr_Evil_Sir · 2023-10-27 21:58 More that this is not just an onion level challenge (layers of learning + knowledge + doing), it's that per thing. In other words, your self set task is peeling all the onions in the onion field to get every onion layer. I know for mine right now in early stages I need to accept that security is something I just do not understand well enough. What that means is no public facing services for me. It is also something I am forced to accept will be a knowledge gap for some time (it's not related to my real job or my future job just now). So I'm playing and learning trying to solve my local use cases and accept that's good enough for now. ---------------------------------------------------------------------- Tropaia · 2023-10-27 21:59 Well, I can somehow understand you, but I didn't have so many problems until now. Especially reverse proxy is in my opinion pretty well documented. I got everything running what I wanted, except LDAP. I can't seem to understand this shit and there aren't any good guides either :D. ---------------------------------------------------------------------- VexingRaven · 2023-10-28 01:29 Without knowing how your router is set up, I can't say for sure, but I see 2 possibilities here: First, the "bridge" port just has every port forwarded to it. If you attach something there, you are opening it up to the internet essentially. The thing you attach there is on your network and has full access to everything within it. However, if you attach a router there then that router is still blocking traffic for whatever is plugged into the 2nd router. Thus, your 2nd network has full access to your first network, which is not at all what you want if you plan on putting insecure stuff there. The other option is that... Well, actually after reading again I think I need a diagram because I don't understand what you're proposing. ---------------------------------------------------------------------- TBT_TBT · 2023-10-28 11:27 @mooglestilzkin has already posted the url for Christian Lempa‘s YT channel. CL has just started a series „made for you“ about Docker: https://youtu.be/Nm1tfmZDqo8?si=g_L8bmm9qcFx9wve ---------------------------------------------------------------------- Ieris19 · 2023-10-28 16:34 So, my ISP requires that the router they provide is plugged to the fiber optic “output” for what I assume it’s either decoding or authenticating that I indeed pay for the service. Their routers have a “passthrough bridge mode” which basically disables every feature on the router and allows it to simply act as the fiber optic output. An ethernet cable is then plugged from an ethernet output on ISP router, to my personal router’s WAN input. But I guess I gotta poke at my router and see if the mask is a subnet of my local net or just my local net forwarded ---------------------------------------------------------------------- VexingRaven · 2023-10-28 17:13 Usually how bridge works in a situation like that is it just passes a single public IP to whatever's plugged into it. You can plug one router into it, and nothing else. ---------------------------------------------------------------------- Ieris19 · 2023-10-28 17:36 Maybe haha, I haven’t tried ---------------------------------------------------------------------- Training-Swan-6379 · 2023-10-28 18:19 Back in the day you had to find a book if you didn't know people ---------------------------------------------------------------------- CallTheDutch · 2023-10-28 19:55 Its a job on it's own for a reason :) I'm an avid debian user. when i need to know something i google "debian 12 how to setup openvpn" or "debian 12 nginx how to setup cloudflare" and so on, which usualy gets me where i want to go in 2-3 results... "Learn to search and enable yourself to do almost anything.." And, it will still be like a jigsaw. Especialy those things you do once and only return to a year later to change something or set up a new system...That's kinda part of the fun aint it ? :) ---------------------------------------------------------------------- weeklygamingrecap · 2023-10-31 14:07 So that is true like I said, there's lots of begin here do this cool thing and then a huge drop off after that or where you find people skipping around but when they come back to something you find they've changed a lot of things and it doesn't line up if you were following their journey. Even paid courses from back in the day sometimes had this issue. A lot of it comes down to research and being able to learn and troubleshoot problems. You may just want to break things down into smaller chunks that hopefully build on each other, "I want to setup a hypervisor", "I want to build a Ubuntu VM", "I want to build a Ubuntu VM with ansible", "I want to use docker to run pi-hole". That way you can just focus on each new thing instead of getting bogged down on the minutia of a huge setup and trying to do 3 things at once and not knowing what is broken. Somethings take longer and somethings start as 1 step but need to be broken down further something like setting up Bitwarden/Vaultwarden also needs a reverse proxy so in those cases you break it down to even small chunks. A lot of us gained this knowledge over years, working on stuff at home and at work in different environments and picking stuff up as we went. So there can be a lot of assumed knowledge that I don't have a good answer for how to acquire besides learning to search and keeping your own documentation and bookmarks.