Proxmox Cluster or K3S for HA reddit / selfhosted ---------------------------------------------------------------------- SillyLilBear · 2023-11-04 19:04 I want to setup HA for my docker containers. I was originally looking at K3S but I'm actually now thinking maybe Proxmox cluster would be simplier and more effective. I have no experience with either, but I have used Proxmox for a while without clustering and I like it a lot. From what I have looked at for K8 and K3 it seems Proxmox clustering is actually less headache and would allow me to cluster VMs if I needed to run one (I'm thinking about going VM route for Home Assistant). Either way, it looks like I will need 3 nodes regardless. I have a server I was using for Proxmox with virtualized TrueNAS that I am considering just making a dedicated TrueNAS box that can offer NFS shared storage to either solution. ---------------------------------------------------------------------- Reverent · 2023-11-04 19:39 Use k3s if you want to learn kubernetes. Otherwise yeah, a HA VM running Docker is a legitimate and simpler choice. ---------------------------------------------------------------------- Ok_World_4148 · 2023-11-04 19:42 Proxmox doesn't natively support docker containers, it does LXC though. If you're talking about installing docker in a VM in proxmox, well you could of course do that, and same obviously for kubernetes. Kubernetes is one huge beast to operate and learn. it's an overkill for 99% of homelabs, unless your goal IS to learn k8s and/or you're masochistic. ---------------------------------------------------------------------- SillyLilBear · 2023-11-04 19:53 >Kubernetes is one huge beast to operate and learn. it's an overkill for 99% of homelabs, unless your goal IS to learn k8s and/or you're masochistic. I really don't need or want to learn it. I just would like to have redundency. I never have downtime, but I would like to not have everything fail at once. ---------------------------------------------------------------------- anyOtherBusiness · 2023-11-04 22:09 Dou you actually need k3s? Or do you want to learn? If you just want to host your docker services think about plain docker with docker compose. If you consider adding VM services like HA, go the Proxmox route and host docker in a VM. ---------------------------------------------------------------------- SillyLilBear · 2023-11-04 22:10 I am currently using docker compose but it’s a single host for all the containers. ---------------------------------------------------------------------- calan89 · 2023-11-04 22:21 I was going to say 'Why not both?' but if you're looking for simplicity and not trying to learn Kubernetes, then yeah, use what Proxmox supports natively. ---------------------------------------------------------------------- colin_colout · 2023-11-04 22:24 I do k3s because I wanted to mess around with Kubernetes and have many decades of experience. If you're not sure where to start, I suggest leaning Proxmox to get comfortable with virtualization first. You can run your containers in a simple way inside the VMs as a start. As you figure out what you're needs are, you can start working toward orchestrators. My only suggestion is to save k3s until your ready. It's called a private cloud for a reason. ---------------------------------------------------------------------- colin_colout · 2023-11-04 22:58 You can have proxmox restore VM to another host automatically if one dies. ---------------------------------------------------------------------- [deleted] · 2023-11-04 23:13 [deleted] ---------------------------------------------------------------------- morbidpete84 · 2023-11-04 23:13 “Many decades of experience” how’s that? Came out like 2014/15 unless you are I cling borg ---------------------------------------------------------------------- onedr0p · 2023-11-05 15:18 Have you considered buying cheap SFFPCs and just doing bare metal kubernetes? With proxmox there's a lot of overhead if you want a stable cluster when using Kubernetes. https://onedr0p.github.io/home-ops/notes/proxmox-considerations.html ---------------------------------------------------------------------- colin_colout · 2023-11-05 15:19 Thanks for the note. I clarified in an edit. I have decades of experience in *nix systems professionally and as a home yet. Started with Redhat 5.2 in the late 90's (Solaris and Irix before that). ---------------------------------------------------------------------- morbidpete84 · 2023-11-05 16:00 Nice…old school. I remember messing with Red hat in the 90’s right before their IPO. Didn’t do anything serious with *nix until 2010ish. Thanks for the clarification. Tried to bring it up without sounding hostile on the K8’s 😬 ---------------------------------------------------------------------- Professional-Swim-69 · 2023-11-05 17:34 When I started all the rage was Slackware, freshmeat.org was the repo site Then Redhat with the use of RPM changed the entire way you did Linux, although tarballs was still needed for many packages which weren't bundled I remember suffering installing dependencies and glibc incompatibility, what a pain ---------------------------------------------------------------------- SillyLilBear · 2023-11-05 17:51 If I used proxmox, I wouldn't use Kubernetes. I'd use Proxmox clustering. It's kind of the direction I am leaning. ---------------------------------------------------------------------- blind_guardian23 · 2023-11-05 19:07 than dont use it. if you dont need HA: dont activate it. you need a cluster file system (like ceph) anyway, just activate HA rules and VM starts again on another node. even docker swarm is possible. would recommend replication with ZFS instead, latency is also much better. ---------------------------------------------------------------------- SillyLilBear · 2023-11-05 19:26 Is there a way to have a second proxmox server replicating zfs via send/receive and firing up vms automatically if primary goes down? ---------------------------------------------------------------------- blind_guardian23 · 2023-11-05 19:58 https://pve.proxmox.com/wiki/Storage_Replication automatic would mean proxmox throws 1min worth of data (which IT does not). Automatic failover requires cluster filesystem.