Docker on Proxmox reddit / selfhosted ---------------------------------------------------------------------- Frosty-Paperclip · 2023-11-02 09:38 I have several docker containers, now I setup a PC with Proxmox. Which is the best approach to run containers on Proxmox? Reading online there are two approaches VM Vs LXC (Ubuntu I prefer). What do you suggest? Thanks ---------------------------------------------------------------------- thekrautboy · 2023-11-02 09:45 Why not ask /r/Proxmox? In general its not recommended to run Docker inside a LXC, especially for a beginner. However plenty of people (including myself) are doing it, you just might run into a few hurdles. If you plan on having mostly a single Docker host with your containers running, then i would suggest a VM for that. If you want to run multiple hosts separately, then the overhead from each VM adds up and i would recommend separate LXC instead. This gets asked often, you could simply search for existing threads. ---------------------------------------------------------------------- ElevenNotes · 2023-11-02 09:49 If you want to easily backup all your data from all containers at runtime with no downtime: **VM**. ---------------------------------------------------------------------- Frosty-Paperclip · 2023-11-02 09:53 Thanks for the hint on the community! I'll run several containers and in fact my initial idea was to use a dedicated VM. Today I read about the LXC so I would like to choose the best. I think I'll use the VM also for the ease of backup. ---------------------------------------------------------------------- thekrautboy · 2023-11-02 09:55 There is no "the best". Since youre getting started with Proxmox, also look at running Proxmox Backup Server (PBS), it can schedule and backup all your VM and LXC. ---------------------------------------------------------------------- Frosty-Paperclip · 2023-11-02 09:58 Currently I make VM backup and store them on Nas. I've heard of PBS but haven't looked into it yet. Does it require a dedicated machine? ---------------------------------------------------------------------- thekrautboy · 2023-11-02 10:01 It doesnt require it, but it makes sense. But plenty of people are fine running it in a VM by itself. ---------------------------------------------------------------------- Frosty-Paperclip · 2023-11-02 10:06 Nice! I'll keep that in mind! Using a VM becomes much more convenient. ---------------------------------------------------------------------- MainstreamedDog · 2023-11-02 11:20 Just take the tteck script for alpine docker and portainer, very lightweight. I even use one for each docker service and then just backup/restore on the full Proxmox container level. ---------------------------------------------------------------------- c_one · 2023-11-02 12:03 If for example he wnats 1 vm with all docker things inside wouldnt it be better to just install drbian and rin docker inside. Instead of running a vm in promix that runs docker? ---------------------------------------------------------------------- thekrautboy · 2023-11-02 12:07 Yes of course, if the longterm plan is to have only a single VM and nothing else, then Proxmox is nearly useless to use as a host and plain Debian or Ubuntu with QEMU/KVM whatever as VM solution would be more logical. But if that is also easier for a beginner, im not sure. Proxmox makes it very simple to use. If they only want to run containers, then even more so, no actual need for Proxmox. Sure it does LXC nicely, but its not required. Could run plain Debian for example with Cockpit to manage LXC/LXD etc. But OP says they already setup their Proxmox, so that train has left the station, sort of. They are asking VM vs LXC, so thats where we are with that :) ---------------------------------------------------------------------- OddMulberry3107 · 2023-11-02 12:25 PBS can also be installed directly on the host machine and will run alongside PVE, this way you have even less overhead ---------------------------------------------------------------------- xXfreshXx · 2023-11-02 12:33 What about monitoring like netdata? Is this recommended to run in a host docker? It's my only docker container running on host... ---------------------------------------------------------------------- Spaceman_Splff · 2023-11-02 12:35 I have all my docker containers in a vm and then anything not docker in a lxc. For a while I didn’t have issues with docker containers in lxc and then they all stopped worked every time there as an update so I moved them to vms. ---------------------------------------------------------------------- thekrautboy · 2023-11-02 13:09 Yeah but i have heard some not so great stories about that in /r/Proxmox and so, i run my own in a VM and that works well too. With half-decent hardware the overhead that a single Debian-based VM causes is minimal, im happy to pay that price for sleeping a little better at night ;) ---------------------------------------------------------------------- OddMulberry3107 · 2023-11-02 13:13 I didn’t know this, thanks for pointing it out. For what it’s worth it’s been about a year since I installed it and I haven’t encountered any issues so far ---------------------------------------------------------------------- thekrautboy · 2023-11-02 13:23 Depends what you want Netdata to be able to monitor. In a container you need to give it access to specific things on the host. If you run it directly thats not required. ---------------------------------------------------------------------- thekrautboy · 2023-11-02 13:23 Nice :) ---------------------------------------------------------------------- joost00719 · 2023-11-02 13:35 Create a VM and install Docker on there. It's the only correct way. I tried LXC before, but it has limitations and you can break stuff when you migrate file system (doesn't work great on ZFS for example). If disk space is a concern, you can use thin provisioning on your file system. ---------------------------------------------------------------------- Frosty-Paperclip · 2023-11-02 13:35 Thank you for this information. I'm setting up a VM for my docker containers ---------------------------------------------------------------------- ManiacMog · 2023-11-02 14:08 For a long time I ran a proxmox VM as a Portainer host. At first it was just a try out Portainer but it gradually grew to be the host that ran the bulk of my self-hosted homelab services. Eventually I decided to scale back my Proxmox cluster by one host and I converted that physical host to a dedicated Portainer machine. I also moved all storage to local storage. The performance boost was measurable. I think the largest reason was using local storage, but I think some credit goes to running natively on the physical host rather than VM. If you just want to try out Portainer I recommend you first spin it up as a VM and determine if it's the service you want to use. If you're convinced and you want more performance consider doing what I did. ---------------------------------------------------------------------- ManiacMog · 2023-11-02 14:12 I also see some comments here about backing up your data. The way I do this for my Portainer host, as well as my other VMs, is I use Kopia. I have a Jenkins host run daily snapshots using Ansible to invoke Kopia. As for running Docker native on VMs on Proxmox, I do this as well for services that don't make sense to run on my Portainer host. For that I also use ansible to deploy and use docker compose to bring up those special containers. Having this hybrid approach gives me greater flexibility on what I can do in my homelab. ---------------------------------------------------------------------- xXfreshXx · 2023-11-02 14:47 So your recommendation is to install it on host without docker? ---------------------------------------------------------------------- thekrautboy · 2023-11-02 15:47 No, thats not what i said. ---------------------------------------------------------------------- xXfreshXx · 2023-11-02 15:56 OK. I see two options to monitor proxmox host: 1. Install it on host, 2. install docker on host and run in docker container. You see a third option? ---------------------------------------------------------------------- thekrautboy · 2023-11-02 16:00 Good luck smartass :) Maybe you should just go back to watching Montanablack etc. ---------------------------------------------------------------------- Cvalin21 · 2023-11-02 18:02 Hands down LXC. Here is the easiest way to get started https://tteck.github.io/Proxmox/ ---------------------------------------------------------------------- mdirks225 · 2023-11-02 21:32 I just used a VM and installed docker on that, works out pretty well. ---------------------------------------------------------------------- Exzellius2 · 2023-11-02 23:01 Am curious: what are the „not so great stories“ you heard? ---------------------------------------------------------------------- thesnizzles · 2023-11-02 23:15 Avoid docker in lxc...it will *work* but will give you headaches. ---------------------------------------------------------------------- thekrautboy · 2023-11-03 02:15 Just a bunch of weird problems it could cause apparently. Easy enough for you to simply search /r/Proxmox or make a post there and get feedback :) ---------------------------------------------------------------------- itguy327 · 2023-11-03 02:30 Why is LXC not recommended? I've been running this way and curious if I'm going to hit some wall or have some security issue Thank you kindly for your insight ---------------------------------------------------------------------- Jealy · 2023-11-03 16:44 There's the argument for how simple VM backup & restore is vs baremetal OS though to be fair, so even with just 1 VM if you can afford the overhead it's not a bad idea. ---------------------------------------------------------------------- thekrautboy · 2023-11-03 16:46 In general yes, true. But with Proxmox and Proxmox Backup Server making a backup or just a snapshot of a VM is the same effort as doing it of a LXC. And when someone uses Proxmox they really should also use PBS for this. ---------------------------------------------------------------------- maximus459 · 2023-11-04 18:11 Agreed with this.. Single docker hosts are good in that there isn't a hard limit on resources, essentially each container has the whole service at it's disposal* Buuut.. you really don't want to 'put all your eggs into one basket' (in case something goes wrong), esp if some of those eggs are experimental (learned that the hard way) Anyways, it's not a big deal to set up and manage with tools like Portainer.. Some services realty do need to be in their own, esp when they, - need specific pets like 80/443 - run scans - DNS and reverse proxies Also, while you can map ports in containers (like guacamole, lbrespeed), doing so well make it look like all incoming IP's are from the gateway IP of the docker subnet. The audit function is useless unless you run it with the network mode set to "host" Tldr: - Use ProxMox+multiple docker hosts of you have many VMs and a complex environment - single docker host for simple setups/you only have one device - either use ProxMox with VMs/lxc, or single Ubuntu server docker host I use the first in production, and the second for learning/testing