Looking for recommendations on how to set it up
reddit / selfhosted

----------------------------------------------------------------------
apperrault · 2023-10-12 15:04

hey all

I am thinking of rebuilding my infrastructure and am looking for
recommendations.  I have 3 decent processor, 32GB RAM mini-computers
(2 think center and 1 Dell SFF).  I am currently running Ubuntu Server
and Docker on each of the systems.  I would like to maybe load balance
some of this out without having to move things manually.

Right now all of my storage is external USB drives.  Right now is not
the time to upgrade to NAS.  I can mount the drives to one of the
systems and then NFS share to all of the systems.

Would you recommend Kubernetes, Swarm (i have no experience with
either, but have been an Admin for 25 years so am technically
proficient) or something else?

let me know your thoughts.

Right now, two of the systems are not doing much so I could get the
new infrastructure setup with these two and then add the 3rd one once
I migrate the containers to the new setup

thanks

app

----------------------------------------------------------------------
eXgam3 · 2023-10-12 15:44

My first question would be, are you hosting something important that
it needs load balancinng/high availability?

----------------------------------------------------------------------
apperrault · 2023-10-12 15:46

I'm running things like Plex, pihole, authentika, and stuff like that.
It isn't "critical" but people don't really like downtime

----------------------------------------------------------------------
eXgam3 · 2023-10-12 15:53

I'm currently running around 40 containers at 3% CPU usage, 7th gen
Intel. Only adguard/NPM is running on two nodes, but  I have shared
storage so in dissaster I would only have to run docker-compose on a
different node. It is simple, requires no maintenance.

----------------------------------------------------------------------
eXgam3 · 2023-10-12 15:58

I work with k8s on daily basis, but never felt like its a good choice
for home usage. You can try it, spend time on learning, but at home I
want it to be simple as possible. Everyone's different and thats just
my opinion.

----------------------------------------------------------------------
dazchad · 2023-10-12 16:14

Unless you want to play with k8s and HA, there’s absolutely no need to
use it. You are better off investing on a UPS and a script to auto
restart services if they come down, and Docker can do this for you
already.

If you want a better isolation, put proxmox and create a VM to put
your docker apps. This way you can create a snapshot of the system in
a working state, and if things go haywire you can simply restore the
snapshot. Be mindful to store the data on an external volume heh

And backups!

----------------------------------------------------------------------
sk1nT7 · 2023-10-12 16:33

HA and load balancing is most often overkill for a homelab. Also
running K3S or K8S productively. However, it's your choice and
infrastructure.

I personally run about 60 docker containers and multiple LXCs and VMs
on a proxmox machine. It is a Minisforum HM90 NUC, which barely
notices the load.

So running a subset of those containers on multiple cluster nodes will
only increase complexity but will not make it really better.
Especially in homelabs, a power or ISP Internet outage is more likely
than a single node crashing randomly and the others not.

Also running Swarm, K3S, K8S and such will require some form of shared
storage. Not sure whether this will be nice using USBs. I'm talking
glusterFS, Ceph etc.

So my recommendation would be a single, beefy server that runs
Proxmox. Then create your VMs/LXCs as needed and run your docker
containers. You can even spawn a few LXCs and try out K3S if you want.
Totally fine for learning purposes.

----------------------------------------------------------------------
dazchad · 2023-10-12 18:50

K8s is not a good choice for the vast majority of use cases. It shines
when you have dozens/hundreds of teams in an organization and need to
make the most of your available hardware.

----------------------------------------------------------------------
apperrault · 2023-10-13 21:27

Thanks y'all. That's what I thought, about it being overkill. I think
one of the reasons I was thinking about something like swarm was for
docker secrets, which from what I understand are only available in
swarm.

Thanks..

App