Port logic for containers? reddit / selfhosted ---------------------------------------------------------------------- rudeer_poke · 2023-12-25 11:12 I am running 75 containers in my homelab and I want to consolidate the ports they are using as I hate them being random as they are now, but I am confused how to set this up properly. I am thinking something like this: \- ports 6xxx dedicated to media management tools (Plex, Jellyfinn, \*arr) \- 6110 - Plex \- 6120 - Jellyinn \- 6210 - Radarr .... \-ports 7xxx dedicatet to Home Automation, like Home Assistant, ESPHome, MQTT server, etc ​ However where I am confused: \- how to sort the containers? simply by alphabet, by importance, or attempt to create some kind of hierarchy/dependency? \- how to react on replacements in the infrastructure, when i phase out a solution because I have no use for it anymore or is no longer developed \- some containers seem to be quite picky about the ports their are running on, and changing it may be complicated I think this would need some general guidelance and homelab governance rules, as right now its a mess. ​ ---------------------------------------------------------------------- GherkinP · 2023-12-25 11:21 If you’re running a front end proxy what does it matter the port you’re using? ---------------------------------------------------------------------- stupv · 2023-12-25 11:40 If these are docker containers, you can simply rebind the ports in your compose files - what is currently 8080:8080 can be [anything]:8080 and you don't need to touch app config at all ---------------------------------------------------------------------- joyfulmarvin · 2023-12-25 12:08 I've set up a traefik in front of them, made a \*.mydomain.com wildcard certificate and same wildcard domain resolution entry, so that [whatever.mydomain.com](https://whatever.mydomain.com) and [somethingelse.mydomain.com](https://somethingelse.mydomain.com) point to the same address (of my docker container host). Adding a new app now is as simple as deploying a docker and putting the app name in the label section of a docker config of the newly deployed app. Everything plain works by opening [https://mynewappname.domain.com](https://mynewappname.domain.com) ---------------------------------------------------------------------- Kaleodis · 2023-12-25 12:11 Honestly i wouldn't bother. Here's two ways I would do this that don't require changing (or remembering!) ports: a) setup a reverse proxy. use a local domain name (or a rented one, easier ssl, point dns to local reverse proxy) and put all services behind descriptive subdomains (e.g. plex.my.local or radarr.my.local). Please note that, depending on the service, you might have to forward headers in a specific way. This is usually described in the documentation of the service (under reverse proxy/nginx/etc). Also note that you \*can\* use anything for a TLD, but you shouldn't. ".local" is reserved for local domains and should be used, unless you own a public one and can point certain subdomains to your proxy with local DNS (e.g. pihole). "Nginx proxy manager" is popular for this purpose. b) use a jump page. i set up homepage (the gethomepage one) for this. every service has a small widget, clicking on it takes me to the page of e.g. radarr. A lot of services also expose a bunch of info to this page, so it's nice as a status overview. if you absolutely have to change port numbers, doing this for containers is easy and straight-forward. I won't explain it here, since you probably should know by now (75+ containers lol) how port binding works for docker/compose. Changing ports for bare-metal services might be harder to achieve or even straight-up impossible (i.e. hard-coded) - a reverse proxy might help here too (which means a service will be exposed on TWO port numbers now)! ---------------------------------------------------------------------- CmdrLightoller · 2023-12-25 12:32 I'm basically doing the same thing. Everything I need is behind a reverse proxy, so the port numbers don't really matter day-to-day, but having a system makes it easier to create new containers and update the proxy configuration. My approach is to inventory all the ports, the container name, the protocol (http, ssh, database, other, etc.), and the proxy hostname into a spreadsheet. I also make note of any dependent services using that port or hostname. I'm then just assigning any port that can be arbitrarily assigned a sequential port number. There are a few services that have preferred port numbers that I don't move, and a few services that have custom network configs to map to their own IP addresses, and I record those as well. I track any open ports that the docker host uses for non-containerized services in the same spreadsheet. I intentionally don't care that they are sorted in any particular way so that when I make changes, I don't need to reshuffle anything. The only important thing for me is that I know which numbers have already been assigned and to maintain a reference, so that when I add/remove/change a service I can update the proxy files and docker files accordingly. I've had a similar philosophy around IP assignments and have always found that unless it really matters (like an address must be in a specific subnet), it's best to not worry too much about the numbers having a specific sorting order since things will invariably change. ---------------------------------------------------------------------- PleasantDifficulty · 2023-12-25 13:00 Reverse proxy and only changing the port numbers when it’s something that has a conflict (8080 is usually the one). Installed Homarr to test yesterday and the default port is 7575 and that was good for me. ---------------------------------------------------------------------- Korkman · 2023-12-25 13:27 If I had the ambition to organize ports, I'd think from a firewall perspective and put stuff into ranges which might be treated differently in the future. Like 20000-29999 for privileged stuff (admin interfaces), 30000-39999 for non-privileged internal, 40000-49999 for publicly accessible. Then make a counter you keep incrementing. Like IDs for your containers, to be used only once. That way any forgotten obscure firewall hack applied to a port will not affect a new project. Container ID 1 would get port 20001, and have 30001 and 40001 reserved in case during evaluation you notice it belongs more into those other groups and it changes scope. You could then work with ranges in your firewall, with the notable security implication that applications open random high ports for outgoing connections, which is a concern for UDP based stuff. Organize your ranges and sysctl net.ipv4.ip_local_port_range accordingly so they don't overlap (my examples are a bit high and broad, you hopefully don't consume 10000 IDs in the forseeable future) ---------------------------------------------------------------------- arnemcnuggets · 2023-12-25 13:55 I'm usually trying to 1337 the apps running the containers. Can get wild tho ---------------------------------------------------------------------- Agile_Ad_2073 · 2023-12-25 14:09 Forget ports altogether and use a domain + reverse proxy. That's the way to go. ---------------------------------------------------------------------- ericesev · 2023-12-25 15:43 I try to use 8080 on all the containers. That keeps the firewall rules simple. I don't forward the ports to the host. ---------------------------------------------------------------------- primevaldark · 2023-12-25 16:42 Your energy is much better spent on setting up DNS and proxy. In my docker compose files ports are only “exposed” to the proxy network and not mapped and proxy (traefik in my case, maps them to container and port). No port assignment headaches. ---------------------------------------------------------------------- Rorixrebel · 2023-12-25 16:44 Damn this is overly complex.. use a proxy mate ---------------------------------------------------------------------- tatanpoker09 · 2023-12-25 17:47 As a further question to this, would just adding dns records in wherever you purchased your domain be considered insecure? Or whats the reason behind suggesting local dns resolver with pihole ---------------------------------------------------------------------- Kaleodis · 2023-12-25 17:58 i have a public domain and it points to my VPS (which proxies a few select services that i allow on the Internet). i don't want to buy another domain, so i'd just override the dns resolve locally - but just for the subdomains. this means my dns record is already set to something else. also i want to expose as little info online as possible. please note that I'm still planning to do this - i haven't built this solution yet. ---------------------------------------------------------------------- tatanpoker09 · 2023-12-25 18:03 I see, I understand it from the security pov, cause at the end of the day you could still point each subdomain to the exact same IP but the reverse proxy could resolve each to each service. I do understand the benefit in exposing less information (such as which services you’re running) ---------------------------------------------------------------------- LynxesExe · 2023-12-25 18:17 NGINX as a reverse proxy will be your savior. Having a bunch of random ports for stuff is not great, you will never have enough containers to run out of ports, but if you want to have SSL you need to bind the app to port 443 (unless you want the browser to complain, and you have only one of those), and in general remembering on what port something is bound is just... unnecessary? Just have a DNS and NGINX to redirect you to the desired container basing off of the hostname the request is sent with, might as well make NGINX an SSL endpoint since you're at it; I mean, why not? https://plex.home.local is better than http://192.168.1.34:9887 ---------------------------------------------------------------------- blackstar2043 · 2023-12-25 19:22 Any port that falls within the range of 1024 and 49151. It's up to you to decide how to structure these ports. ---------------------------------------------------------------------- nitsky416 · 2023-12-25 20:18 Using ports like that ends up being a pain in the ass. Redirecting subdomains and using a reverse proxy is way way easier than managing a pile of port forwarding and trying to wrangle password managers that don't want to auto detect your site properly and family and friends who think it's sketchy or forget to leave the port on the end. Trust me. ---------------------------------------------------------------------- tamatea · 2023-12-25 21:54 Same here. Specific port + reverse proxy. You can also export all the used port with a commande like : `docker ps -a --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}" | (read -r; printf "%s\n" "$REPLY"; sort -k 2 )` ---------------------------------------------------------------------- emprahsFury · 2023-12-25 22:58 This is the whole point of docker. Docker's job is to handle networking, if you're not publishing the ports to the host they can all have the same port. They should be addressed by container name not a socket like it's 1991 again. ---------------------------------------------------------------------- rudeer_poke · 2023-12-25 23:59 of course i am using reverse proxy with subdomain names. but that does not solve my problem. whenever I am spinning up a new container a still need to assign it a port and need some clear logic what to choose ---------------------------------------------------------------------- Reasonable-Ladder300 · 2023-12-26 00:03 Why assign a port?, just point traefik label to the port the container uses and you do not need to expose or remember any ports. ---------------------------------------------------------------------- rudeer_poke · 2023-12-26 00:10 i am not faimiliar with traefik (i am using Swag container from LinuxServer as a reverse proxy) but I still need to map the containers port to a local port on the machine, no? Otherwise I would have conflicting ports, as a lot of containers come with ports like 8080 by default ---------------------------------------------------------------------- nitsky416 · 2023-12-26 00:29 You can do some fun auto configuration by tagging your containers in your compose file. Then the services aren't accessible outside the docker network without going through the proxy, you don't use any ports on the host, etc. Otherwise, if you're dead set on using mapped ports, for most things just use the default and make sure you've got authentication turned on. You only have to change if you have a conflict. ---------------------------------------------------------------------- throwawayacc201711 · 2023-12-26 02:00 You could even have compose read an env var file so one you can name the ports (great if you ever wanted to grep on it) and then even remove any personally sensitive data from the compose file and then check it into source control or make it part of an ansible playbook. ---------------------------------------------------------------------- MoonTimber · 2023-12-26 04:37 - You can using ‘—label’ command to labeling the container. Then you can list all running containers and filters them by label. - On podman you can group containers by creating a pod. This is the great way to categorized your container which should also help you check status for each group. - For picky port container, using separate pod should also solve that on the condition that each pod should not have application on the same port. - using ‘—sort’ to sort when listing containers. You can create a command alias to save your option. - Normally, I tag the container as legacy when I plan to phase it out from my infrastructure. So, I easily list if I have any legacy services left. Hope this helps. ---------------------------------------------------------------------- Reverent · 2023-12-26 09:09 create a docker network: `docker network create reverseproxy-nw`. Define that network for the reverse proxy (assuming it also lives in docker) and then you can refer to the container and port internally. https://blog.gurucomputing.com.au/Reverse%20Proxies%20with%20Caddy/Adv anced%20Usage%20of%20Caddy/#using-docker-networks ---------------------------------------------------------------------- Reasonable-Ladder300 · 2023-12-26 12:21 With traefik this is not an issue, you do not have to expose any ports. But you add a label pointing to the container port. This makes it possible for multiple containers using the same port internally. I recommend using traefik with docker since the problem you describe in your post is the one traefik solves. ---------------------------------------------------------------------- SoulRedSquid · 2023-12-26 13:15 .local won't work from iOS, it's not recommended for home usage especially if you have Apple devices. See RFC6762/app-G: https://datatracker.ietf.org/doc/html/rfc6762#appendix-G Use .lan or .home if you want a short one. ---------------------------------------------------------------------- Kaleodis · 2023-12-26 13:35 Thanks for the addendum. ---------------------------------------------------------------------- rudeer_poke · 2023-12-26 20:13 just looked into this a bit deeper and it looks awesome, however i need to manage and expose at least internally containers from 3 different machines and that introduces some additional complications as they are on different docker networks. looks like easiest to solve with a docker swarm, but that also introduces additional complexity. oh, and during a quick test run i wasn't able to acquire letsencrypt certs with traefik, so there's that too ---------------------------------------------------------------------- PassiveLemon · 2023-12-27 04:53 How could I get this to work across hosts? I host larger things like Jellyfin and stuff on my desktop but my reverse proxy lives on my RPi which also runs Pi-hole. Currently I do a similar thing of mapping out ports and just using IP + port in Nginx PM since I couldn’t reasonably figure out Docker networking across hosts. ---------------------------------------------------------------------- Reasonable-Ladder300 · 2023-12-27 09:31 I use docker swarm for this.