Docker compose or swarm/kube for 2 nodes? reddit / selfhosted ---------------------------------------------------------------------- RikkelM · 2023-12-31 08:17 I'm self hosting with currently 2 mini pcs that run all the time. One runs pretty much everything (Plex/Bitwarden/Traefik/adguard/prometheus etc), the other runs Nextcloud with AIO. I wanted to separate them because Nextcloud can be resource intensive especially with recognize AI and Plex is the number one priority with about 10 users. They are using docker compose and the main reason is that it's how most applications document it. It's getting a bit annoying to deploy stuff in both machines as well as manage secrets. I have a mono repo that contains two folders with 2 docker-compose, and i deploy them both from the Plex machine with the use of docker ssh context. There are some stuff that are deployed in both machines (gitlab runner, node exporter...), and some secrets must be copied in both .env files. So my main issues are: - Cannot use docker secret easily since most stuff are injected by env var (knowing kube and configmap it would have been easy to solve) - if i decide to move a container to the other server, it's way more complex - Traefik configuration is not pretty, labels all over the place and hard coded IP addresses. There are no shared storage. Would i benefit from using docker swarm or is docker compose good enough for my use case? ---------------------------------------------------------------------- ComprehensiveGap144 · 2023-12-31 08:32 If you would like to go for Kubernetes then add a third node for HA. But i would like to introduce a third options: caprover. It looks like it fits your needs. It has docker swarm under the hood and some extra benefits. ---------------------------------------------------------------------- RikkelM · 2023-12-31 08:38 I checked about it, it seems to use nginx under the hood. Would it mean that I would need to replace everything that's using Traefik? (Not like there's a lot to do tho except maybe some crowdsec stuff) ---------------------------------------------------------------------- ComprehensiveGap144 · 2023-12-31 09:18 Yeah you are right, it has no traefik support unfortunately. Then maybe just docker swarm with swarmpit for easy management? ---------------------------------------------------------------------- NiftyLogic · 2023-12-31 11:36 You could have a look at Nomad/Consul/Vault. Much easier than setting up a Kubernetes cluster. And agree with an earlier comment, a third node would be great for HA. Maybe a Raspberry PI or a VM on a NAS. ---------------------------------------------------------------------- prime_1996 · 2023-12-31 14:06 I had a similar setup, but in LXC containers in Proxmox. Where I had 1 LXC container with my data folder, running docker apps like nextcloud, photoprism etc. The other would have no data access and run docker apps like speedtest, librespeed etc. This meant I had 2 portainer stacks, 2 traefik stacks, 2 LXC containers to manage etc. I have recently pulled the trigger and moved everything to docker swarm, I was looking at kubernets, it was a good learning opportunity, but way too complex for my home lab. Now I only need one manager, where I keep all the compose files and manage my stacks. I can have any number of worker nodes, and all I need to do is install docker and add it to the swarm and it's ready to have containers. I also have the ability to drain a node, if needed, and all stacks will be moved to another node. I use Portainer for easy visualization, but still deploy stacks using CLI. Since I am using LXC, storage is way easier, since I can map my docker data folder to all my LXC nodes, meaning I can easily backup all my data. But since you are using baremetal, might be worth looking at glusterFS. You could also install Proxmox on both PCs, and have a proxmox cluster, then docker VM/LXC containers. ---------------------------------------------------------------------- RikkelM · 2023-12-31 19:09 Yeah my issue was that i started with one windows machine where i installed everything manually with no containers... Then i gradually improved from there, moved to ubuntu then container, added a new machine etc. Each machine has an external USB hard drive. What you describe looks very nice and versatile Im just worried about the data volume part ---------------------------------------------------------------------- RikkelM · 2023-12-31 19:15 I read that Nomad is way easier to setup than kube. I'll take a look at it! Vault would solve part of the secrets issues !