Better Docker Stack reddit / selfhosted ---------------------------------------------------------------------- RuthlessJailer · 2023-10-06 02:44 Hello, I have been running a couple of Docker stacks for the past few months; one on my PC, hosting servarr apps / DL clients, and one on my RPi running Plex, PiHole, and number of microservices. I've set them up using docker-compose and it's really nice because I can just do docker-compose restart and they all boot up in the right order (specified by depends, etc.). However, I don't like the fact that everything is jammed into one single compose file, and especially since I named both of them "selfhosted" for lack of a better name. Does anyone have any suggestions on how to better organize my containers? I like compose, and I tried moving them to separate directories but ended up having to manually go to each folder to manage them. Any feedback is appreciated --Thanks ---------------------------------------------------------------------- NortySpock · 2023-10-06 02:59 Currently I have been using just different directories and not minding very much. But you could use the `--file` option to specify differently-named docker compose files. $ sudo docker-compose help Define and run multi-container applications with Docker. Usage: docker-compose [-f ...] [options] [COMMAND] [ARGS...] docker-compose -h|--help Options: -f, --file FILE Specify an alternate compose file ---------------------------------------------------------------------- thekrautboy · 2023-10-06 04:50 Use a seperate folder and one compose for each stack, as in a group of services that absolutely need to be together. ---------------------------------------------------------------------- revereddesecration · 2023-10-06 04:59 Alternatively, have a configs folder that contains your compose files in their own folders, but separate the volume mounted directories into a data folder. That way you can version control your compose files and back up your data separately. ---------------------------------------------------------------------- NikStalwart · 2023-10-06 05:01 > I don't like the fact that everything is jammed into one single compose file That is because everything should not be jammed into one compose file. Just as the idea behind containers is 'one container = one service', so too should one compose file correspond to one *project*. A project can be something as small as a web app + db container, or something as packed as a a matrix homeserver + a dozen appservices. The best (and proper) way to do this is to group services thematically and place them each in their individual compose files. Then, as /u/nortyspock suggested, use the -f option to specify which compose file you want to operate on from the 'parent' directory. How exactly you organize your directory structure is up to you. You can have one folder containing differently-named compose files, or you can have multiple subfolders each with its own service stack. I prefer the second option because I sometimes mount config directories into the containers, and this arrangement makes things much cleaner and clearer (not to mention more portable). You can write yourself a shell wrapper function for docker-compose to remove some redundant typing you need to do. How you do this depends on your directory structure and other personal preferences, but something I'd do is write a simple function that, as a first argument, takes the name of the folder/service stack you want to operate on, and then passes all remaining arguments directly to docker-compose. That way, you can do something like `dc matrix up -d`... ---------------------------------------------------------------------- thekrautboy · 2023-10-06 06:23 True thats a option too. But imo you can version control your compose files regardless if they are stored with data or not, simple enough to use gitignore or similar. I prefer to have all the data of one stack within on folder and subfolders of it, nothing spread around. I know exactly where everything required is, and that makes moving it to another system and backups/restores very simple. ---------------------------------------------------------------------- lilolalu · 2023-10-06 07:48 Still, having all the *arr services in one stack makes sense especially if you have custom network stuff to route them over a vpn ---------------------------------------------------------------------- IsThisNameGoodEnough · 2023-10-06 08:21 Yep, I combine services into a single docker compose based on whether there are dependencies. For instance, I have one compose file that handles my rclone, VPN, and arr programs since start-up order is important. ---------------------------------------------------------------------- duskit0 · 2023-10-06 09:24 IMO it's easier to use symlinks for the docker-compose.yml files. I use something like this: docker ├── _compose-files/ │ ├── .git/ │ ├── $(hostname)_$(stack/folder-name).yml │ └── docker1_nextcloud.yml ├── nextcloud/ │ ├── [docker-compose.yml] → ../_compose- files/docker1_nextcloud.yml │ ├── data │ └── config This way I have version control on the compose-files without having to deal with exceptions. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 09:38 Eh, whatever works for you :) ---------------------------------------------------------------------- PaulEngineer-89 · 2023-10-06 10:18 One other point…docker-compose stacks can only be uploaded or stopped/started/removed. You can’t recover the Docker-compose.yml. Why this matters is that Portainer needs the actual yml to manage your stacks. Once you tear down everything and copy to Portainer (as individual stacks) maintenance is a lot easier. ---------------------------------------------------------------------- mar_floof · 2023-10-06 11:02 Kubernetes my good man. Let it handle the mess, and if you add a second rpi in the future it can also handle the what runs where, also the auto-restarts, etc. ---------------------------------------------------------------------- lorax-06 · 2023-10-06 11:20 I have a separate compose for each, locating config and cache dirs in with the compose something like this. I keep notes for each with any specific setup information and instructions. Then I backup the root dir with all the configs nightly. ├── certbot │   ├── cleanup-cron.sh │   ├── docker-compose.yml │   ├── drun-generate.sh │   ├── nginx.conf.exa │   ├── notes.md │   └── proxyLE-verification.conf ├── immich │   ├── docker-compose.yml │   ├── if-500-error.md │   └── upgrade.sh ├── mealie │   ├── docker-compose.yml │   └── upgrade.sh ├── mqtt │   ├── conf │   │   ├── mosquitto.conf │   │   └── mosquitto.passwd │   ├── data │   ├── docker-compose.yml │   └── log ---------------------------------------------------------------------- lorax-06 · 2023-10-06 11:37 I keep them all separate, with notes and config / cache files. Something like this \`\`\` ├── certbot │   ├── cleanup-cron.sh │   ├── docker-compose.yml │   ├── drun-generate.sh │   ├── nginx.conf.exa │   ├── notes.md │   └── proxyLE-verification.conf ├── docker │   └── get\_docker.sh ├── immich │   ├── docker-compose.yml │   ├── if-500-error.md │   └── upgrade.sh ├── mealie │   ├── docker-compose.yml │   └── upgrade.sh ├── mqtt │   ├── conf │   │   ├── mosquitto.conf │   │   └── mosquitto.passwd │   ├── data │   ├── docker-compose.yml │   └── log \`\`\` ---------------------------------------------------------------------- chronop · 2023-10-06 11:41 you could use something like portainer, swarmpit, kubernetes, etc ---------------------------------------------------------------------- EndlessHiway · 2023-10-06 11:43 Thanks for the detailed and well reasoned write up. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 11:47 Youre welcome :) ---------------------------------------------------------------------- TheCaptain53 · 2023-10-06 11:54 I've got all of my services sitting in a single compose as I've either got a lot of interdependencies, or I access them via a single reverse proxy. As I've only got a single IPv4 address publicly, I can only realistically have 1 reverse proxy. ---------------------------------------------------------------------- leonhart83 · 2023-10-06 12:01 Am I the only idiot that uses portainer? I feel like I am the only idiot ---------------------------------------------------------------------- OneToCrowOn · 2023-10-06 12:12 No, for some reason it doesn't feel "cool" to use easy tools. I use portainer and it works for everything except the raspberry pi zero w. I have some music streamers throughout the house and I use docker to run them. ---------------------------------------------------------------------- fuken33 · 2023-10-06 12:12 I have a folder for every stack, and inside each folder I use a .env file that tells docker compose which files to load (I don't remember the exact name of the env var, is in the docker compose docs). Then, inside each folder I have files with names like jellyfin.yml, sonarr.yml and so on. Every bind mount is outside of these folders, in the storage unit under a /config/ folder ---------------------------------------------------------------------- ZeeroMX · 2023-10-06 12:33 Your docker-compose commands really run without the dash? ---------------------------------------------------------------------- HNO_ · 2023-10-06 13:20 I usually don't bother about manually organizing it as since I have started using cosmos-cloud.io opensource project.. it's really awesome and takes out lot of stress and highly flexible as well.. ---------------------------------------------------------------------- hussei10 · 2023-10-06 13:31 Yea using the file arg is the way to go. It gets better if you create aliases for your specific docker compose stacks so that 1 command will reference the right file automatically For example alias dcarr = docker-compose --file docker-compose-arr.yml ---------------------------------------------------------------------- TheMcRibReturneth · 2023-10-06 13:48 Good lord, people need to stop putting all of their containers into one compose file. A compose file is designed to be ONE "stack", that does ONE thing. You want to host an image server, cool, make the "image server" stack with it's db, application, and whatever else it needs. You want to host your legal iso download stack, awesome. Make a single compose file with all your \*arr apps and your torrent app in one network that you run through a vpn. Do one thing and do it well. Don't do everything poorly. ---------------------------------------------------------------------- TheMcRibReturneth · 2023-10-06 13:49 I use portainer whenever I get annoyed with unraid or whatever other tool I'm using. It's a crazy powerful tool, especially after you get all the volume mount functionality figured out. ---------------------------------------------------------------------- happymellon · 2023-10-06 13:54 Compose has been an argument for docker for a while now. You don't need the docker-compose script. ---------------------------------------------------------------------- tbrummell2 · 2023-10-06 14:00 >Your docker-compose commands really run without the dash? Yes, latest version if Docker & Compose do this. ---------------------------------------------------------------------- TBT_TBT · 2023-10-06 14:20 You don't have to have all containers in one docker-compose file to have it also in the same network as your proxy. You can just use the external network feature in all other composes. So you tell Docker to put it into the proxy network. Many people have a frontend and a backend network to which they attach the containers as needed. ---------------------------------------------------------------------- Dairy8469 · 2023-10-06 14:21 you can use a reverse proxy without a monolithic docker compose file. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 14:21 You mean `docker-compose` vs `docker compose`? That has changed quite a while ago. If your setup is still using the dash, its time to update :) ---------------------------------------------------------------------- TBT_TBT · 2023-10-06 14:22 What do you mean? If there is a docker-compose file, you can stop the stack, edit it and then start the stack again from it. No need to use Portainer for that. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 14:24 Not exactly. Docker compose is still a separate thing, its a plugin to Docker itself now. If it were just a argument for Docker, then it wouldnt have its own version numbering etc. See `docker version` and `docker compose version` for example. ---------------------------------------------------------------------- PatochiDesu · 2023-10-06 14:25 i have my stuff organized in git repos and use ci/cd to roll out changes to my environment as "releases" so all changes can be reviewed and potentially undone and also the environment recreated somewhere else. if you like compose you can still use it. I personally like use a single for example mysql for multiple services. it is treated as an independent component in my environment. this makes it possible for me to move each "service" in its own git repo and assign each service their own ci/cd processes to get them on the target environment. I personally moved away from plain docker after i was also unhappy with my docker composes that were flying around. i was long afraid to use kubernetes (k3s on baremetal alpine) because of its complexity. On my opinion it comes with better management possibilities but also adds costs (learning, trial and error etc) but there are good guides on the net. if I look back I dont regret the invested time because it enabled me to also use workloads that are only documented/designed for kubernetes. ---------------------------------------------------------------------- SnooPeppers2758 · 2023-10-06 14:38 I recently started breaking my monolith up like this, and so far I'm happy. I've heard rumblings newer versions of Compose will allow referencing other compose files (and thus hopefully having to cd into each directory). What really allowed me to split them up, efficiently, was this script. I keep a `.env` in the root directory which is the master: every container has its variables in this file. I then symlink it into each stack's directory. The script automates this for any new directories (aka Stacks) in the `./services`. So a Stack might be located at `/opt/docker/your-project/services/immich/` yet reference `/opt/docker/your-project/.env`. Overall, this helps keep shared directories (ie. config directories) easy to maintain and ports clear of conflicts. Repo [here](https://github.com/jgwehr/homelab- docker/tree/maintenance/94-docker-organization). ```bash varMainDir=/opt/docker/homelab find $varMainDir/services -mindepth 1 -maxdepth 1 -type d -exec ln -sbf $varMainDir/.env {}/.env \; # Now, remove the .env~ files find $varMainDir/services -mindepth 1 -maxdepth 1 -type d -exec rm -fI {}/.env~ \; `` ---------------------------------------------------------------------- lilolalu · 2023-10-06 14:44 You can use stacks with portainer. Or without portainer, or both. I have it installed but rarely use it, sometimes I just check if after a reboot everything went up again since I find that overview better than in CLI. ---------------------------------------------------------------------- TheCaptain53 · 2023-10-06 14:54 Oh shit, TIL. I will check that out. *Although being able to update all of my containers with a single easy command is nice.* ---------------------------------------------------------------------- daYMAN007 · 2023-10-06 14:58 For me using portainer is just slower. But in the end it's all preference. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 14:58 No, youre not :) I use Portainer too. But i dont deploy my stacks with it. I use plain folders and compose files to set things up, i prefer it that way and in some (rare) cases Portainer failed for me with that. So i stick to the "direct" approach for deploying. But i do use Portainer to manage the host, to check on things, start/stop stuff. But since i very often have a terminal to that host open anyway, i tend to just do a lot of Docker stuff there too, instead of switching to browser, opening a tab, loading Portainer and clicking through the menus to get what i want. There is absolutely nothing wrong with using a tool like Portainer, its especially good for Docker beginners. Enjoy it! But i have seen people who only used Portainer and never learned how Docker actually works, and when confronted with a problem that Portainer was unable to solve, or even made worse because of it, they had no idea what to do. So imo its a good idea to also spend a little time to learn the very basics of Docker, even when relying on Portainer for day-to-day usage. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 15:01 > You can’t recover the Docker-compose.yml Why would one need to recover it? It doesnt disappear once deployed. Do you mean to restore it in case it gets deleted by accident? I dont see that as a good argument. Anything can get deleted by accident and its not Dockers or Portainers duty to restore a deleted file... Compose files are supposed to be kept and containers get spawned based on them. Why would they be removed? ---------------------------------------------------------------------- thekrautboy · 2023-10-06 15:04 Yeah, shoutout to /u/azukaar and their project [Cosmos- Server](https://github.com/azukaar/Cosmos-Server) ---------------------------------------------------------------------- TBT_TBT · 2023-10-06 15:59 I let [https://containrrr.dev/watchtower/](https://containrrr.dev/watc htower/) update my containers automatically and get info about new versions and the successful update process via [https://hub.docker.com/r/poma/docker-telegram- notifier](https://hub.docker.com/r/poma/docker-telegram-notifier) . So it really doesn't matter if one command updates them. I rather prefer not needing to stop the containers I am not working on. I don't want to stop my Teamspeak container e.g. if I don't want to work on it (so almost never). ---------------------------------------------------------------------- redoubledit · 2023-10-06 16:01 I have a portainer instance up and running for quick testing. But as soon as I know, I want to keep something running long term, I will put it in a compose file in my git repo. ---------------------------------------------------------------------- azukaar · 2023-10-06 16:26 Thanks both <3 ---------------------------------------------------------------------- Dairy8469 · 2023-10-06 17:18 I haven't used portainer, does it work with version control? The way I (and I suspect many others) do things is by putting them in git and deploying them, this way if something goes wrong the code can be redeployed from git which should be correct. Maybe that can be done with portainer but having a gui doesn't offer me personally an advantage. that said, if it works for you, it works and that doesn't make you an idiot. ---------------------------------------------------------------------- Dairy8469 · 2023-10-06 17:20 kubernetes offers an absolute ton of complexity so the benefits may not weigh out the advantages for a lot of users. ---------------------------------------------------------------------- MalcolmY · 2023-10-06 17:27 I can't live without Portainer. ---------------------------------------------------------------------- RuthlessJailer · 2023-10-06 17:53 Wow! I wasn't expecting such an indepth reply! You mentioned networking, and I think that is the reason I have ended up globing them. On the PC, I run all my download clients and servarr apps behind gluetun. I know people advise running eg sonarr without VPN, but I don't want to have prowlarr talking to sketchy indexer sites from my ip, and they have to see each other to work (same net). I also want to avoid creating multiple gluetun networks if possible so as not to fill up my mullvad slots. How would you recommend I go about splitting them up? Since all the stuff is pretty closely related, do you think it's probably fine as is? Thanks again Additionally, I think you've cleared up how i'm going to organize the pi's stack. So thanks for that ---------------------------------------------------------------------- mar_floof · 2023-10-06 18:04 It absolutely can, no doubt there. But spin up something like k3s and it just handles 90% of the mess for you. If you can write a docker file, you can k3s a service or 12 ---------------------------------------------------------------------- itxtoledo · 2023-10-06 19:26 If I understand correctly, you can try something like Caprover or Easypanel, search and reply if it is useful to you ---------------------------------------------------------------------- icebear80 · 2023-10-06 20:07 Yes of course! You can do a full GitOps workflow. Honestly, why would you store your compose files anywhere else than GitHub and then automatically deploy with something like Portainer? That’s the way! 😉 ---------------------------------------------------------------------- literal_garbage_man · 2023-10-06 20:17 Any suggestions on maintaining a single .env file for use in each docker compose? E.g. if I want to define my homelab domain, or a common credential, in one spot and reference it in each docker compose folder? If it’s too much fuss, duplication isn’t the end of the world ---------------------------------------------------------------------- thekrautboy · 2023-10-06 21:03 Uhm if youre deadset on a single file to manage, i guess the option to have one .env somewhere and then symlink it to each compose stack folder would work... ---------------------------------------------------------------------- thekrautboy · 2023-10-06 21:09 > You mentioned networking, and I think that is the reason I have ended up globing them. You can easily have containers be part of the same networks if you want even if they are in separate compose files (stacks). Its not hard to look up in the Docker documentation. Also /r/Docker exists > On the PC, I run all my download clients and servarr apps behind gluetun. I know people advise running eg sonarr without VPN, but I don't want to have prowlarr talking to sketchy indexer sites from my ip, and they have to see each other to work (same net). I also want to avoid creating multiple gluetun networks if possible so as not to fill up my mullvad slots. Again, thats fine and perfectly possible even if you divide services into separate compose files. You can create the gluetun network from the Docker commandline for example and define it as attachable, then any other container that are separate can join it. > Since all the stuff is pretty closely related, do you think it's probably fine as is? Honestly that comes down to a more personal choice. Of course the *arr containers are all related and a lot of them want to talk to each other directly, so having them together makes a lot of sense. However, if you easily want to update just one of them, or restart one of them, then it becomes more complicated. > Additionally, I think you've cleared up how i'm going to organize the pi's stack. So thanks for that Youre welcome. ---------------------------------------------------------------------- tenekev · 2023-10-06 21:39 I use Portainer as part of my CI stack. Compose files are version controlled and when a change is pushed, Drone tests it, and deploys it as a Portainer stack (created and managed by Portainer). Via a custom Drone plugin. The fastest thing to figure out whats not working is Portainer. ---------------------------------------------------------------------- weeklygamingrecap · 2023-10-06 22:57 To answer your question, yes, a lot of YouTubers and guides for that matter mention a single compose file to bring up and down all the containers quickly. There's a bunch of info that kinda gets spread around that you later find out is more for ease of use than best or even just good practices. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 23:18 > To answer your question, yes, a lot of YouTubers and guides for that matter mention a single compose file to bring up and down all the containers quickly. Well honestly, fuck them! Feel free to name them. This is a horrible thing to teach newcomers. ---------------------------------------------------------------------- Nnyan · 2023-10-06 23:26 Looks cool going to check it out. ---------------------------------------------------------------------- EndlessHiway · 2023-10-06 23:28 TraSh guides for the arr apps. ---------------------------------------------------------------------- thekrautboy · 2023-10-06 23:31 I actually know the "trash" guy and i sort of doubt they would recommend such "trash"... but i will look into it, thanks. ---------------------------------------------------------------------- arcoast · 2023-10-06 23:35 >appreciated Docker compose can already reference other compose files. Here's a snippet from my current `docker-compose.yml` file ```yaml version: '3.9' include: - ${CONFIG}/audiobookshelf/audiobookshelf.yml - ${CONFIG}/authelia/authelia.yml ``` And my `authelia.yml` then looks like this: ```yaml networks: authelia: external: false name: authelia services: authelia: image: authelia/authelia:master container_name: authelia networks: - authelia - ldap - traefik # ports: # - 9091:9091 volumes: - ${CONFIG}/authelia/authelia:/config restart: unless-stopped labels: - "traefik.enable=true" - "traefik.docker.network=traefik" - "traefik.http.services.authelia.loadbalancer.server.port=9091" - "traefik.http.routers.authelia.entrypoints=https" - "traefik.http.routers.authelia.rule=Host(`login.${TRAEFIK_DOMAIN}`)" - "traefik.http.routers.authelia.middlewares=securityHeaders@file" authelia-redis: image: redis:alpine container_name: authelia-redis networks: - authelia # ports: # - 6379:6379 volumes: - ${CONFIG}/authelia/authelia-redis:/data environment: - TZ=${TZ} restart: unless-stopped ``` You can read about `include` [here](https://docs.docker.com/compose/multiple-compose- files/include/) ---------------------------------------------------------------------- EndlessHiway · 2023-10-06 23:36 [Their docker compose file is on this page.](https://trash- guides.info/Hardlinks/How-to-setup-for/Docker/) ---------------------------------------------------------------------- thekrautboy · 2023-10-06 23:39 ffs, this doesnt suggest to put **everything** into one single compose file. It simple shows most of the *arr containers as one stack. ---------------------------------------------------------------------- EndlessHiway · 2023-10-06 23:45 Okay, guess I misunderstood. Sorry. ---------------------------------------------------------------------- Singers_Laundry · 2023-10-06 23:53 Docker compose is intended to be used for groups of containers that very closely depend on each other. There isn't much you can't do by putting your regular docker commands into a bash script for each container. These are services running on servers. Are there times you need to manually stop and start "all" containers? If so, a script can be used for that. ---------------------------------------------------------------------- DotDamo · 2023-10-07 00:46 I use systemd, and natively thought everybody used it (or init). Maybe it’s only Podman people who use it. Is that an option for Docker people? How does docker-compose handle auto-starting/stopping, is it called by init/systemd? ---------------------------------------------------------------------- RuthlessJailer · 2023-10-07 01:31 Compose has a restart block. I have it set to restart: unless-stopped that way it will launch as soon as dockerd launches. idk the intrinsics, but I know that my services launch without fail unless I've manually done docker-compose stop or docker-compose down. ---------------------------------------------------------------------- GrumpyCat79 · 2023-10-07 01:59 Just migrated from Docker Swarm to Kubernetes with k3s and man, it's not as easy as writing a docker-compose file (in fact, it could if you use kompose to convert them). Having to declare PersistentVolumes, PersistentVolumeClaims and Service in addition to the deployment isn't as easy. I am usually doing good with most of this stuff, but I banged my head a few time when trying to mount external shares for instance Luckily, I am using k3s so it wasn't hard to setup! ---------------------------------------------------------------------- Crytograf · 2023-10-07 09:17 I have about seven containers defined in compose file. There are no dependencies. You suggest splitting them up to seven compose files in seven different directories. I don't know. It seems harder to manage. ---------------------------------------------------------------------- thekrautboy · 2023-10-07 09:18 Do whatever you want. ---------------------------------------------------------------------- Toorms · 2023-10-08 12:42 When your database for your whole system is down, all applications are down at the same time. If just one database is down, so just one application is down in the best case. For this example my way it’s to use a single instance on every microservice. ---------------------------------------------------------------------- phillibl · 2023-10-10 11:09 Nope I use Portainer stacks for everything, because if that I have my yml files if I ever feel the need to directly deploy through compose. Seems like only a very small subset of users would need what Portainer can't do ---------------------------------------------------------------------- SnooPeppers2758 · 2023-10-10 14:01 ah, thank you! I heard about it a few months ago on the selfhosted.show podcast. Lost track of the timing. Thanks! ---------------------------------------------------------------------- OLLCommander · 2023-10-24 16:28 Would it be advisable to take the network isolation one step further by also separating out each frontend app into it's own proxy network, rather all frontend apps sharing the same proxy network? E.g. for your instance, remove the `traefik` network, and then define an `authelia-proxy` network and join only Traefik and Authelia to that? ---------------------------------------------------------------------- arcoast · 2023-10-24 16:30 I mean I guess I could, but my Traefik container would probably end up with about 30+ different networks which isn't really practical.