Diving into Docker: Proxy Puzzles and Cloudflare Conundrums reddit / selfhosted ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-10-07 06:37 --I picked proxy for the flair on this post, because that is my main issue right now, but aside from that this is about Docker Management and Remote Access to say the least. But I digress.-- After several attempts over the years, I have finally dove headfirst into the world of Docker containers. Each prior venture left me baffled, but armed with my BaS in Cybersecurity, I embarked on what turned into a week-long Docker deep-dive, with nearly every free hour I had aside from work consuming me (I barely slept). I never even played a single game of Starfield in the last week. The journey, though laden with challenges, felt progressively rewarding. But then it seemed every time I made something work, I was hit with obstacles. I watched a NetworkChuck video discussing the Cloudflare Tunnels, and I thought that it sounded amazing, I have had a nice domain sitting around for a few months that I've been to lazy to make a website for, so I thought "What better time to get into it?". I got the cloud tunnels working fine, it took me a few days to figure out, as I messed up and fiddled with the DNS records, thinking I had to be the one to link each subdomain with the cloud tunnel. Until I read that if it's a remote configured tunnel that Cloudflare automatically manages the it for you. And that's why I kept getting confused, because rules and configs DID just keep getting changed when I wasn't looking lol. So aside from getting the tunnels working, I wanted to get a "Reverse Proxy" set up, on the sl;did I not need to open any ports on my router, if I ended up somehow getting broken into, they would only be looking at the one machine, my Rev-Prox, and not my entire bank of services. I felt like I was making real progress when I figured out the cloud tunnel thing and got all of my services working at once from outside my local network! I thought I had it dwn... Cut to yesterday, and I was reading some forums, watching videos, etc, and somehow through the knnowledge I was gaining, I realized that I don't think I have been proxying at all!! All of my subdomains were pointing at different machines, my rev-prox was just being one big useless papaerweight! Don't ask me why I thought it was actually doing something, I don't know. After that, I told myself I was going to take a break, and play some Starfield and relax when I got home instead of stressing myself out more. Lo and behold, I ended up fiddling with it all day ( I work graveyard and am there now), and I ended up shortly before leaving for work tonight completely uninstalling Docker, WSL, and all of my distros. (I guess I should have mentioned I've been trying to do this all in this weird hybrid Windows/Linux environment that I feel is just dragging me down at this point!) Several times, my containers have disappeared, stopped working seemingly randomly, such as NGINX Proxy Manager out of nowhere started spamming 30-40 lines of "Starting server..." per second along with issues saying the path for the cert didn't exist, when I had just gotten them they day before and they were supposed to be there for a while. I really am gonna take a break today when I get off, but I know my next steps are to PLAN everything out AHEAD of time, decide what containers to actually compose together, and what ones to leave standalone. I'm trying to take it as a learning opportunity, but man, I am FRUSTRATED. It makes me feel so stupid when I can't figure these things out that so many people use, and have incredible videos and guides on how to do. So really, after this long winded rant I needed to get off my chest, should I even bother doing a reverse proxy? Cloudflare Tunnels route me to where I want to go, they protect my IP, like I'm trying to think if there is anything I'm really missing without one. I need some help and guidance, not only on this issue but I feel like I have a many with no one to talk to about it. ---------------------------------------------------------------------- NikStalwart · 2023-10-07 06:58 A reverse proxy is good for, among other things, the following: * Centralized TLS termination; * Access control (kinda...) * Reusing HTTP(s) ports across multiple services by using subdomains or paths. I use nginx for all three reasons even on private networks and localhost. Cloudflare or no cloudflare. > Several times, my containers have disappeared, stopped working seemingly randomly, such as NGINX Proxy Manager out of nowhere started spamming 30-40 lines of "Starting server..." per second along with issues saying the path for the cert didn't exist, when I had just gotten them they day before and they were supposed to be there for a while. This is why I tell people to never use point-n-click solutions that promise to 'work out of the box'. You have no fdisking clue what is going on and why. Some practical advice: don't watch youtube videos. Don't copy-paste commands and expect them to work. Install (if you don't already have) man-db. Before running any commands, type 'man ' and read the manual page for it to understand what it is going to do. Don't use UIs (like nginx-proxy-manager), instead, learn to read and edit the underlying config files. Usually, these are well documented and well-commented, and you learn a lot of the underling technology by dealing with it directly. Incidentally, over my years, I have discovered an interesting correlation between people "watching videos" and not understanding the technology they are trying to use. I think this is because videos are made for the sole purpose of being a "content creator" (i.e. view / ad farming) and lack the depth of content that will come from reading manual pages, documentation and stack overflow questions. ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-10-07 09:21 That's definitely true, but I try not to just blindly copy and paste things without having some kind of idea. I'm not really a developer, so I don't really know code. I was about to start editing the config files so I could write my own ingress rules, and just set the DNS rules manually on the cloudflare side so I had more control. But then the damn NGINX stopped working lol. You gotta start somewhere right? You may be able to answer this, I saw many times people talking about how you just comment out some lines of code ragarding the cert, or running this command "to" the container, and that's where I'm lost, like am I supposed to get a CLI within the container? While it's like spazzing out? Or am I supposed to log into the Linux distro from my WSL? Because I tried that and the folders that were supposed to be there weren't. When the containers are running I can click on the files tab, but once they get shut off they're gone. I feel like I'm on the verge of being able to figure everything out, but that's how I've felt the last week lmao. Thanks for the response! I am trying to learn! ---------------------------------------------------------------------- NikStalwart · 2023-10-07 09:45 I cannot help you because I have zero insight into the way your environment is organized. The best I can do is tell you what I use: * I have 10 different distros in WSL. My primary distro (for legacy reasons) is Ubuntu. It runs NGINX natively  —  not in a container. * I have docker running on my host. It uses the WSL backend, but it is not installed *inside* a WSL distro (that is, I used a Windows installer not the linux one). * I have done some TCP fuckery to forward host ports to WSL ports, but this is no longer necessary as of the latest builds of WSL. * I do not use and will likely never use cloudflare tunnels on my local machine, because my local machine is just that  —  local. I don't want a stupid security misconfiguration or new zero-day to compromise my tax and medical documents, for instance. I think you should start small and build upwards. First get nginx to work within your LAN, then consider running cloudflare etc. ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-10-07 10:36 I had it working perfectly within my LAN. And then at one point I had it working perfectly with the Cloud Tunnels. Some of what you mentioned is what confuses me. I also installed it using the Windows installer. But it uses the WSL2 backend. So I am not sure where I am supposed to go when people say to give commands to the containers themselves, or mess around with their file system, as we are unable to see inside of their volumes? Correct? I'll give you a little context into my setup, I originally was mostly trying to rant and get some suggestions on what to do first to kinda start understanding what's going on better. But since you brought it up :) At the most complex I had the following containers: \- NGINX Proxy Manager - Pihole \- Lidarr-on-steroids, which basically was just lidarr with a deemix plugin, it worked fantastically. \- Was starting to install Heimdal and \- Portainer extension on Docker Desktop and the portainer-ee container, those also worked well until somehow the extension locked me out, the password I knew it was set to didnt work, so I imagine I must have somehow gotten rid of the wrong volume at some point \- A few other extensions such as Log Viewer, Disk Space something or another I quickly got tired of trying to figure out all the networking shit AND pihole DNS at the same time, so I took that one off to simplify it, and it seemed to work better, until trying to "do more" fucked it up lol. Oh yeah, I was trying to implement Authentication on Cloudflare, as NGINX PM had its own, as did Portainer, but Lidarr/Deemix did not. That's what really caused me to start messin with stuff and mess it up. I already knew, before my next stab at this I'm going to diagram it out first, start small, keep everything segmented, unlike before when I put NGINX along with Portainer and Lidarr-on-steroids in one compose file, that screwed me royally. I thought it was a waste to have a "whole compose file" and only call one container, but why not? No harm no foul. What do you think I should do next considering what I said my plan is. Should I start building the dockerfiles from scratch so I can really understand the inner workings of them? Final question, so you use a 'native' Ubuntu NGINX, and use it to route traffic on your physical IRL network? How did you implement that? ---------------------------------------------------------------------- TBT_TBT · 2023-10-07 11:29 I kinda understand the problems you are facing, because Docke can indeed be quite daunting at the beginning. I would suggest the following: - Docker is a Linux thing. Get a Linux machine to run it. This could either be a separate computer or a virtual machine on an always running server. Normally you would like to run everything all of the time, so make sure it is on a device that is on 24/7. Docker on Windows is confusing and no howto really fits, because all are done for Linux. Don‘t do it. - Learn Linux if not yet familiar (enough). It is absolutely essential to know your way around in your sleep with a „BaS in Cybersecurity“. - Learn how to start/stop/handle containers on Linux. Make yourself familiar with docker-compose files and how they are structured. Set up containers exclusively with docker-compose, not with „docker run“. - Set up a >>private<< Github repository and push/sync your docker- compose files there after changes. This way you have a backup and different versions, so that you can go back to an earlier one if things don’t work and you don’t know your way back anymore. - Set up a concept for yourself with structured directories for your docker-compose files and the volumes of a container. - Some of your points seem to suggest that you don’t have understood that pulling/recreating a container will set it back to default except if volumes (or folders in the host OS which are mounted in the container) are defined with persistent data. Familiarize yourself with how volumes work and how to use them. - Start with everything locally and only after that works add the Cloudflare tunnel. - Nginx Proxy Manager will be using ports 80 and 443 on the host. Normally you would forward those ports in your router to the host and therefore NPM. No other container is allowed to use those ports, so set all other containers to different ports (they normally are) in the docker-compose file. - Manually create a „proxy_network“ using the docker command. Use „external network“ in docker-compose files to tell docker-compose to not put the containers in their own network but in the „proxy_network“. This way you can setup a proxy with „http“ „name of container“ „8080“ (= HTTP port of the web service). - To be able to use SSL, setup a Let‘s Encrypt wildcard certificate (*.domain.tld) in NPM by using the API token you get in your Cloudflare account. You can then use that certificate in all service.domain.tld URLs you want to use. For this to work, a CNAME to a DynDNS name needs to exist in the DNS server (should be Cloudflare for LE to work) with that exact DNS name. I would start with NOT using CF tunnels but a DynDNS service. This way you learn it „the proper way“ and still can change it in the future. There is a lot more to learn and do. But the steps I described are quite a lot in the first place. Don’t expect to learn everything in 2 days. Rather take your time and learn everything properly (have a look at the Docker and other documentations). Don’t just do stuff written somewhere without knowing exactly what every single line will to. ---------------------------------------------------------------------- clintkev251 · 2023-10-07 13:22 I think part of the reason that you're confused is that Cloudflare tunnels double as a reverse proxy, so you don't technically need one. Now many people will still use a reverse proxy that their tunnel terminates to, myself included. My main reasons for doing so are to centralize authorization, apply additional access controls, and allow for direct local access. But if you're happy with your configuration as it stands, there's nothing wrong with it. ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-10-07 15:07 Well, I deleted my entire config because it was fucking up, so I'm starting over. What you mentioned are the reasons I want one too, I just configured it wrong. But I'm gonna start small now, one piece at a time and make sure I can get everything smooth. ---------------------------------------------------------------------- PowerfulAttorney3780 · 2023-10-07 15:19 Really good advice! Thank you. I have actually done a lot of what you suggest, but also some of what you said you hit the nail on the head. I am pretty familiar with Linux, I just don't have a dedicated machine for it. I have my gaming PC, and a miniPC I've been using as a server. Well, and a laptop, but I'm not running any of thise kinda stuff on it. I think I'm going to make a dedicated Linux VM, using half of the resources of my miniPC "server". Additionally, you were right about me not understanding the volumes. I thought it seemed like a good idea to have everything on persistent volumes, but it just confuses me because we can't really access those volumes. Also, I have probably read every page in Docker's documentation multiple times, it's just so much info and hard to wrap my mind around fully. Like I think I get it, I always try to use compose, but then sometimes I spend half an hour chasing down why it wont let me compile this compose.yaml and it was like one thing out of order or something, or indented at the wrong level. So somehow I need to get a better understanding of how their file structure works, I imagine after I start doing it mostly fully in Linux itll get less confusing. So do you think it will still be just as effective if I use compose, but only have one container per compose? Also, what is a good resource for me to understand how to interact with the containers better? I kniow how to turn them on and off obviously, but other than deleting and redeploying from scratch, or rebooting, I dont really know how to interact with them. Also, I did have a \*.domain.tld cert from LetsEncrypt, but that's one of the things that fucked up, even tho it didnt expire for a month my container was caught in a loop of trying to access it or renew it or something.,. Should I install "Docker Desktop" on Linux, or just the daemons? ---------------------------------------------------------------------- TBT_TBT · 2023-10-07 21:17 Yep, a true Linux environment will help. Concerning volumes: the container creators will tell you which folders inside the container will need to be handled to have persistent data. You have 2 options: 1. Using a volume: `volumes:` `- myapp:/home/node/app` Those volumes can be found in /var/lib/docker/volumes and managed by Docker. The "myapp" is the name of the volume outside the container, /home/node/app is the folder inside the container. 2. Matching / mounting a folder of the file system to a folder inside the container: `volumes:` `- /dockerdata/app/folder:/home/node/app` Both can be used kind of interchangably. I tend to use method 2 more, because I have a folder in which I have subfolders for the apps. In those I mount the needed folders for the app. If you have trouble chasing the yml bug, try [https://www.yamllint.com/](https://www.yamllint.com/) to squash it. Yes, the levels are very important. Before Docker, it took time to install an app following some howto. Now it takes time to create a suitable docker-compose file. ;) The advantage of the latter is that once you have it, you have it everywhere, you can use it once, twice or a thousand times, the result will always be the same. I use 1 docker-compose file for 1 app, no matter if that app is only 1 container, 1 app container and a database or several containers (there are some very complicated docker-compose files out there, just have a look at the compose file of Mailcow, a mailserver AIO: [https://github.com/mailcow/mailcow-dockerized/blob/master/docker- compose.yml](https://github.com/mailcow/mailcow- dockerized/blob/master/docker-compose.yml) ) Resources? Hm. I would learn by doing stuff and selfhosting. By just not caring if it takes longer to create a running and well organized compose file. By reading in the docs if you don't understand something. Believe me: once you get it, you will never set up stuff differently. Installation: DO NOT use Docker Desktop. Do not even install a GUI on the Linux VM. Install it like this: [https://docs.docker.com/engine/in stall/debian/](https://docs.docker.com/engine/install/debian/) (if you use Debian or Ubuntu). Then create a folder structure: /dockerdata here you can create subfolders for the persistent data of your containers. /dockerbuild here you put all your docker-compose files. I tend to name them something useful, like npm-compose.yml or nextcloud-compose.yml . Push that folder to a PRIVATE Github repo to have a backup of the compose files and to be able to pull them quickly to another server. Care about the LE cert later on. You could use [https://nginxproxymanager.com/](https://nginxproxymanager.com/) for that. ---------------------------------------------------------------------- [deleted] · 2023-10-08 05:24 [removed]