ZFS voes
reddit / selfhosted

----------------------------------------------------------------------
ProfessionalAd3026 · 2023-10-16 22:07

So, I bought a nice Zotac ZBOX with 2TB local storage and 16 GB
memory. Installed Debian with k3s and partitioned the disc to have a
ZFS pool for the containers. I've set up the ZFS provisioner.

I'm migrating away from an RPi4 with 4GB of memory and docker-compose

I run out of memory and have my pods evicted while not having all my
things migrated. Okay, I've got two new things: ArgoCD and Navidrome.
But what the heck?

Now I've updated k3s according to the manual (just download the binary
and restart). It wouldn't start. Ran the installer instead. During
boot-up, two of my ZFS volumes got deleted by the provisioner. F\*ck.
Luckily no important data on there, just extra work to redo the
migration. That's the second time the provisioner kissed a volume
goodbye. The first time, I was to blame because I messed up a commit
in my ArgoCD repo. Data was faster gone than I could say bye-bye.

I work with Kubernetes daily. K3s seemed a logical choice for managing
a more significant environment. I looked into Proxmox, but docker
containers are not supported.

I spent every free minute for the past 14 days migrating, but I have
my doubts about the correct path. Eventually, I need a stable system.

ZFS was supposed to save me from some of those stupid mistakes. I run
Btrfs so far, and snapshots are a lifesaver. Can't wrap my head around
them in ZFS. Unfortunately, is the Btrfs provisioner not very
widespread/stable.

Any advice or encouragement?

PS: Velero is still on the todo list... :(

----------------------------------------------------------------------
JzJad12 · 2023-10-16 23:31

Set the max ark size to something like 2GB if that's what's eating the
RAM

----------------------------------------------------------------------
phein4242 · 2023-10-17 12:54

This. You need to tune the zfs memory, esp if the box is shared with
other applications.