best practices for multiple docker compose files? reddit / selfhosted ---------------------------------------------------------------------- flying_unicorn · 2023-12-28 00:10 I currently have everything in 1 compose file and i'm trying to redo with a separate compose file for each "app". If an app depends on multiple microservices they should all be in 1 compose file. Each app with it's own directory. What i'm struggling with it, shared .env variables. For example I may want to define my timezone in a .env file that each compose.yaml should be able to reference, or my secrets dir. Also an easy way to start/stop multiple services at once, without having to cd into each project directory. So what i currently have is: /opt/docker/ as my root dir. /opt/docker/.env (i'd like to have this as a global .env file, but i don't think docker supports this). /opt/docker/secrets /opt/docker/stacks/project1/compose.yaml /opt/docker/stacks/project2/compose.yaml Some apps also define common networks, like traefik, and docker socket proxy. I feel like i'm probably missing something basic, like docker compose is supposed to work like "X" and my brain has it all backwards and is trying to make it do "Y" ---------------------------------------------------------------------- EnumeratedArray · 2023-12-28 00:36 Splitting out your docker compose file like that isn't really what docker compose is designed for. If you want to keep it separate in that way, you may as well just run docker containers individually. If you're just wanting to group services, you could assign profiles in your big docker compose file and keep similar services within the same profile ---------------------------------------------------------------------- dadarkgtprince · 2023-12-28 00:53 I grouped mine by ones that have dependencies on each other in one, and my testing/QoL stuff in another. Save me from having to manage each separately ---------------------------------------------------------------------- Senkyou · 2023-12-28 02:41 It's my understanding (based on documentation and personal experience) that splitting out compose files into different projects is actually best practice. ---------------------------------------------------------------------- fossilsforall · 2023-12-28 05:28 If you're already making individual directories for each of them why not just copy and paste the .env file too. Down the line if you have any changes you could just automate synching them. ---------------------------------------------------------------------- GodsAsshole666 · 2023-12-28 05:40 You could use symlinks to have all the files reference the main file and not need to worry about keeping them in sync ---------------------------------------------------------------------- fossilsforall · 2023-12-28 05:47 Oh fuck yeah I didn't even think of that ---------------------------------------------------------------------- primevaldark · 2023-12-28 06:15 1. For sharing values between compose files you can use environment variables and then refer to them in compose files using ${VARIABLENAME} syntax. 2. You can use networks from different compose files, you just need to declare them in top-level ‘networks’ section with ‘external: true’ flag. Just be careful with names. If traefik compose file creates a ‘proxy’ network, by default it is going to be known as ‘traefik_proxy’ in other files as docker compose tries to make them unique and prefixes network name with “project” name. ---------------------------------------------------------------------- geekierone · 2023-12-28 06:25 I have gone a slightly different way and am using a tool called dockge (see https://github.com/louislam/dockge) to separate each compose file in its own directory and use the Dockge UI to start/stop service independently ---------------------------------------------------------------------- mattssn · 2023-12-28 07:44 Beat me to it! DockGE is amazing and should do most things you need. ---------------------------------------------------------------------- MiakiCho · 2023-12-28 11:01 I recommend switching over to ansible. It is a much more robust system for sharing variables, defining dependencies, tagging containers, etc etc. ---------------------------------------------------------------------- TehBeast · 2023-12-28 16:41 This is possible with the [include](https://docs.docker.com/compose/compose-file/14-include/) element added in Docker Compose 2.20. I set up my containers such that I have one global .env file, services separated by folder, and each service may also have a local .env file (for variables specific only to that service). I run docker compose commands in the context of the root level docker-compose.yml. Example file structure: .env docker-compose.yml service1/.env service1/docker-compose.yml service2/.env service2/docker-compose.yml .env (root level) TZ=America/Los_Angeles docker-compose.yml (root level) include: - path: 'service1/docker-compose.yml' - path: 'service2/docker-compose.yml' ---------------------------------------------------------------------- flying_unicorn · 2023-12-28 23:23 Thank you! This is working exactly the way i was trying to make it work! ---------------------------------------------------------------------- flying_unicorn · 2023-12-28 23:23 I actually played around with dockge, it's a handy tool. I've found a few bugs with it though. Frankly i prefer to work through the cli. ---------------------------------------------------------------------- geekierone · 2023-12-29 00:57 That is fair. I used to have the CLI as the primary option with directories to separate the different compose directories. I was looking at Portainer and Yacht to control which containers were running, as my use case was to turn on and off GPU containers in order to not have GPU memory locked by the container. As such I was looking at a WebUI that I could put behind a reverse proxy and Dockge gave me all this. Another recently posted option (GUI, still) is [https://www.reddit.com /r/selfhosted/comments/18kc20h/dokemon\_is\_open\_source\_now/](https: //www.reddit.com/r/selfhosted/comments/18kc20h/dokemon_is_open_source_ now/) ---------------------------------------------------------------------- tys203831 · 2023-12-29 10:34 May not be the best practices, but that's how I host multiple website apps with multiple docker-compose file and multiple .env files: https://tanyongsheng.net/blog/how-to-host-multiple-websites- incl-n8n-with-docker-traefik-on-one-server/