Run the reverse proxy on the router, or punch a port through the router to a full-fat server running the proxy? reddit / selfhosted ---------------------------------------------------------------------- rekabis · 2023-09-29 02:52 Trying to get a handle on this. I have been looking at Squid and Caddy in particular, and I am getting a bit confused as to a generally- accepted ideal and robust setup. I have a router running OpenWRT 22.03.2. I can also set up a separate 1L PC with 512Gb RAID-1 and up to 16GB RAM, with OpenBSD as the OS. I will be running various other Windows servers with Hyper-V VMs with all sorts of operating systems and serving up all sorts of Internet content. This is what I am wanting to proxy, because I need to make Port-80/443/587/993 services across various machines and VMs available to world+dog. Would it be better to host the reverse proxy on the router, or better to install it on the custom 1L PC and just redirect the router’s Port 80 (and others) to that PC? I also don’t work with docker, and will not be working with it in the short term. I am old-skool VM based. ---------------------------------------------------------------------- theTrebleClef · 2023-09-29 03:24 I personally ran into several issues running a reverse proxy on my pfSense router (which is a VM). I wanted to run it there for the perception of protection, thinking being in the firewall and router would make a difference. In the end I decided not to expose ANY services without VPN. I chose to go with WireGuard for that purpose. And once I was behind VPN, the reverse proxy didn't make sense to put in the firewall, so I ran a VM as a docker host and am running nginx in a docker container. It could easily have been installed to the VM itself. My logic was... I'm doing this at home. Even if I was doing it as a business, today, it's maybe not the safest idea to expose my local systems to the Internet in this way. Maybe some cloud hosted ones, so if something got screwed up and someone got in, it's not MY network. Maybe I just don't trust myself enough! ---------------------------------------------------------------------- ericesev · 2023-09-29 03:55 I run the reverse proxy (Traefik) on my router along with around 30 other Docker services. It's definitely not a generally-accepted ideal and robust setup, but it works great for my use-case and I'm comfortable maintaining it. ---------------------------------------------------------------------- koolboy145 · 2023-09-29 03:59 I tried the process of setting up haproxy on pfsense. Also tried the process of hosting NPM as a docker instance on one of my VM’s. I feel it is better to separate components that run major components of your infrastructure. Keeping the networking component of pfsense separate from the proxy is probably easier for troubleshooting if you run into issues later. That way, you could just redeploy the instance via ansible or an alternative, if something breaks ---------------------------------------------------------------------- mpw-linux · 2023-09-29 04:05 If was me a would put the reverse proxy on one of your PC's then redirect to other servers. I use OpenWrt on a GL.inet router that communicates to the main router via wifi. ---------------------------------------------------------------------- ElevenNotes · 2023-09-29 04:58 WAN :443 > router/NAT > proxy:8443 > target. Don't run proxy on your router. ---------------------------------------------------------------------- zim1985 · 2023-09-29 05:36 Personally I would keep my router doing router things and have another server doing my web hosting/other server things. ---------------------------------------------------------------------- kon_dev · 2023-09-29 09:13 I agree, most routers already do a bunch of things besides routing, I would not run it on the same hardware, if you have other options. It's also a risk assessment task, if someone exploits your reverse proxy, it's possible to access a random VM, vs the central router of your network which could also intercept your whole network traffic... ---------------------------------------------------------------------- boggie26 · 2023-09-29 09:54 Look at Cloudflare tunnels as well. They work great and don’t require any exposed ports. ---------------------------------------------------------------------- verticalfuzz · 2023-09-29 12:00 If you are using a vpn, why do you need nginx as well? ---------------------------------------------------------------------- theTrebleClef · 2023-09-29 12:05 I have several docker containers running at home in a variety of ports. Nginx as a reverse proxy makes it possible to have an easy lookup that points to both the right IP and Port. So I can type "homeassistant/" or "homepage/" and it takes me to the right destination. ---------------------------------------------------------------------- dopey_se · 2023-09-29 12:56 Everyone has their own risk tolerance, biases, and expertise. For me, I just cannot get over the idea of running applications/containers on my router. To me, it inherently opens many more attack vectors that would otherwise not exists. I'd always only poke holes (dstnat) to a specific IP/port and have the proxy running there. It blows my mind seeing mikrotik nearly promoting containers on their routerOS. To be fair; I can absolutely see how this is 'perfectly safe' in a everything is properly configured, no exploits, etc, etc. I guess for me, adding such complexity on the physical device between you and the internet at minimum opens one up to far more likely make human error. Perhaps a bit old school, but the idea of a physical device doing one job well (FW/Router on perimeter) that quickly requires seldom if any config changes is likely to stay more secure. ---------------------------------------------------------------------- schklom · 2023-09-29 13:07 Reverse-proxy on router has many supporters. But adding other non network critical services to the router is asking for trouble: if one of your containers crashes or takes a lot of resources you lose Internet so you can't even figure out what is going on. And you also have 30 points of failures & danger to worry about, because if any gets breached, they can access your entire network. ---------------------------------------------------------------------- pielman · 2023-09-29 13:48 Neither, no ports exposed but use cloudflare tunnel ---------------------------------------------------------------------- ericesev · 2023-09-29 14:32 Absolutely agree. It requires some planning. >But adding other non network critical services to the router is asking for trouble Working through those troubling scenarios is part of the fun/learning part of the homelab for me. I see this not as my router, but as my homelab. Router/firewall/DNS/DHCP are just some service it provides. >if one of your containers crashes or takes a lot of resources you lose Internet I've definitely seen this happen. Code server is a memory hog, as is Home Assistant & AppDaemon. Memory & CPU limits can be added to Docker containers to prevent this. And the containers can be configured to auto-restart. >so you can't even figure out what is going on Logging, monitoring, and alerting is important to me. I use the Promtail/Loki/Prometheus/Grafana stack. If something crashes, I'll get alerted. If CPU usage is high, I'll get alerted. Even if one of the systemd services isn't running, I'll get alerted. Worst case, I can login to the PiKVM and investigate from the console. >And you also have 30 points of failures & danger to worry about Exactly 2 are accessible from the internet. Those 2, plus one more for DHCP/DNS, are accessible from the LAN. Both require a 2FA logins using Yubikeys. Both have AppArmor profiles to constrain them. And alerting is setup for weird behavior: process crashes, AppArmor violations, many successful logins, etc. Nothing but authenticated/authorized traffic makes it past those 2. >because if any gets breached, they can access your entire network. I don't use a flat network architecture for this reason. Breaching a container service is not the same as breaching the host. I have the containers isolated on separate bridge networks. Getting into one container doesn't provide access to others (with some exceptions like Home Assistant). I also have written AppArmor profiles for several of them (those that process data from the internet). AppArmor is running in the mode where the process/container is killed if the policy is violated. I've modified some backend services (Home Assistant, Guacamole) to support JWTs for SSO. For those services, the reverse proxy cannot access them on its own because reverse proxy can't generate the JWTs. It could steal the JWT as it is being used, but the JWT is bound to each backend so it would require stealing one for each backend. I'd expect that AppArmor would have killed the reverse proxy before it got to that point though. Do I recommend this setup for others? Absolutely not. But I like working through the constraints/challenges. Constantly tinkering to improve it is part of the fun of the homelab for me and I enjoy learning from it. This single device setup also fits well with my hobby of being self powered too. It uses only 15 watts. Our whole house is able to be completely self powered, solar & battery, 9 months of the year. [https://imgur.com/a/XRWTchM](https://imgur.com/a/XRWTchM) It's one of the reasons it works great for my use-case. ---------------------------------------------------------------------- its-nex · 2023-09-29 16:16 Sounds like your server is running a router service more than your router is running the others. Mostly semantics ---------------------------------------------------------------------- rekabis · 2023-09-29 17:11 > Look at Cloudflare tunnels as well. Sorry, no. I avoid CloudFlare like the plague for moral and ethical reasons. They are actively damaging the open Internet, and are deeply hostile to it. ---------------------------------------------------------------------- rekabis · 2023-09-29 17:12 > Neither, no ports exposed but use cloudflare tunnel Sorry, no. I avoid CloudFlare like the plague for moral and ethical reasons. They are actively damaging the open Internet, and are deeply hostile to it. ---------------------------------------------------------------------- pielman · 2023-09-29 17:19 It‘s the first time I hear someone being so negative against cloudflare. Can you provide some explanation and examples? Why you are so emotional? It’s just technology. ---------------------------------------------------------------------- McQueen2063 · 2023-09-29 18:26 no, its not. the internet was envisioned to be decentral. everybody using cloudflare is centralizing the internet. it makes it less resilient. plus the security nightmare of them being able to see all of our traffic. and remember, if the product is free, then you are the product! ---------------------------------------------------------------------- pielman · 2023-09-29 19:17 Well it’s your opinion without any facts. Very emotional it’s disgusting pathetic. ---------------------------------------------------------------------- McQueen2063 · 2023-09-29 19:22 wtf??? :-)) its not my opinion, its a matter of facts. yes, throw in aws as well, which adds to centralizing. and since you don’t seem to do your own research, here is one: https://www.crn.com/news/security/cloudflare-outage-causes-major- global-internet-disruptions centralizing internet infrastructure is not how the internet was supposed to work. after 30 years in the industry, I’m certainly not alone with that assessment ;) ---------------------------------------------------------------------- L0g4in · 2023-09-29 19:57 Honestly I would never want to run my rProxy mixed with anything else. It is on its own. ---------------------------------------------------------------------- jivanyatra · 2023-09-29 22:30 I'm not the user you're replying to, but I just wanted to say that I appreciate that you took the time to politely refute those points with the details of your setup. I definitely got a lot out of your setup, primarily in inspo. Great homelab setup, and I agree, I do it to tinker and learn. ---------------------------------------------------------------------- ericesev · 2023-09-29 23:56 Thanks for the kind words. Just dumping this here in case it helps others. Custom AppArmor profiles (traefik, loki, promtail, grafana): [https:// gist.github.com/esev/c5af1618bb70caad4aed7e94b6c53066](https://gist.gi thub.com/esev/c5af1618bb70caad4aed7e94b6c53066) Promtail config: [https://gist.github.com/esev/5ae688a5253a943df5921b2 961a7bacb](https://gist.github.com/esev/5ae688a5253a943df5921b2961a7ba cb) Logs/Promtail based alerts: [https://gist.github.com/esev/36e7727bec49 b194613f200c54d90986](https://gist.github.com/esev/36e7727bec49b194613 f200c54d90986) Host based alerts: [https://gist.github.com/esev/779747dce58052b8241ff 3fe650ab991](https://gist.github.com/esev/779747dce58052b8241ff3fe650a b991) JWT auth provider for Home Assistant: [https://gist.github.com/esev/ac dab7e5258d6cbe0d60b46b2b7c43cb](https://gist.github.com/esev/acdab7e52 58d6cbe0d60b46b2b7c43cb) ---------------------------------------------------------------------- schklom · 2023-09-30 00:15 You convinced me: if you have the time to plan this carefully, it is fine. I would rather not spend that much time to plan all these scenarios though, but that's a preference :P > alerting is setup for weird behavior: process crashes, AppArmor violations, many successful logins, etc > I also have written AppArmor profiles for several of them (those that process data from the internet). AppArmor is running in the mode where the process/container is killed if the policy is violated. I'm almost jealous of you for going that far in making your homelab so safe. I was starting to feel content with what I did, but this is next level. > It uses only 15 watts I have around 40 containers on a Pi 4 including Home-Assistant, Nextcloud, and *arr, but I'm the only user. What device do you use, and how many people use the services, if you don't mind me asking? :) ---------------------------------------------------------------------- schklom · 2023-09-30 00:16 Wow, thank you for the documentation! Definitely bookmarking this post :) ---------------------------------------------------------------------- ericesev · 2023-09-30 01:04 40 containers on a Pi. That's impressive! >What device do you use, and how many people use the services, if you don't mind me asking? [NUC11ATKPE](https://www.intel.com/content/www/us/en/products/sku/2176 68/intel-nuc-11-essential-kit-nuc11atkpe/specifications.html): * RAM: 32G * Disk: 2x 4T NVMe - ZFS mirrored root There are 4 of us in my family that use it. It's mostly idle except for Home Assistant. HA has a few cameras streaming to it so there is always a 10 second buffer of video prior to detecting motion and that uses about 20% of one CPU core. Code server is the other big memory hog, but only when I'm compiling something. Container stats: https://imgur.com/a/LlRtKlV ---------------------------------------------------------------------- Aufgeiga89100860 · 2023-09-30 08:53 You sound technically illiterate. What are you doing on this sub?