What do people typically host on Raspberry Pi clusters? reddit / selfhosted ---------------------------------------------------------------------- enricojr · 2023-10-01 10:35 I'm using a spare laptop (Lenovo E570) running Ubuntu as a remote Docker host, and I do personal web dev on it - I run things like Flask / Django web apps (which are basically small, single-user HTTP servers), MongoDB, Redis, sometimes MySQL. I'd like to add paperless- ngx to all that, but that's a problem for later for later. How do Raspberry Pi clusters stack up against spare laptops in terms of Docker performance? If I were to build one out, how many would I need to match the performance of my old laptop, if at all possible? I know people run Kubernetes on these things, but is K8S really necessary? Or can I just use straight Docker (maybe Docker Swarm)? What kinds of things are RPI clusters suited for? ---------------------------------------------------------------------- fm2606 · 2023-10-01 11:52 I have an email server so that I can send documents scanned from an older laser printer and be able to retrieve them. Setting an email server was the only way since drivers are no longer availble or windows based. What ever website project I am working on. I tunnel from a VPS to my home network. Git repo. ---------------------------------------------------------------------- enricojr · 2023-10-01 12:22 > Git repo. As in, self-hosted git? Or something like Bitbucket / Bitea / etc ---------------------------------------------------------------------- fm2606 · 2023-10-01 12:43 Yep self hosted. I tried gitea for a while and didnt like the UI. I found gitbucket and it as a look and feel similar to githib/bitbucket IMO. I have external hard drives for majority of files not used directly by the RPi OS. It isnt the best setup as everytime I have power flickers some of my drives wont remount automatically when power is restored. None of it is critical so I just deal with it. I recently bought 2 Dell Micro computers with quad I5s and will eventually be migrating everything to them. RPi's have served me well for quite a few years and taught me a lot about networking and remote administration which has helped me in my jobs just in the fact that I am comfortable using CLI, not that I am anywhere near admin level. ---------------------------------------------------------------------- enricojr · 2023-10-01 12:58 Neat,. how many RPis in your cluster? ---------------------------------------------------------------------- EndlessHiway · 2023-10-01 14:10 Stuff they want to host. ---------------------------------------------------------------------- Howdanrocks · 2023-10-01 14:55 I don't think singular RPis are a good choice for selfhosting, let alone clusters of them. The $60+ you'd spend on a single RPi 4 4gb could get you a used optiplex or elitedesk with a fat 6th or 7th gen i5 and double the memory. Unless you need the GPIO pins why not get 7x the performance, PCIe, and expandable memory/storage? ---------------------------------------------------------------------- enricojr · 2023-10-01 15:06 > Unless you need the GPIO pins why not get 7x the performance, PCIe, and expandable memory/storage? It's just curiosity driving my question, really. I'm getting tons of youtube vids about RPi clusters and how great they are for learning Kubernetes and stuff, but they don't really go into performance comparisons so I can't tell if a cluster would be better or not. I'm happy to stick to laptops if it comes down to it. ---------------------------------------------------------------------- puppetjazz · 2023-10-01 15:25 I use mine as a NFS service acting as an extension for the storage on my NUC running jellyfin ---------------------------------------------------------------------- lilolalu · 2023-10-01 16:10 I think people learn kubernetes administration with mini clusters, that's what they make sense for. ---------------------------------------------------------------------- lucassou · 2023-10-01 16:28 I don't have a rasp clusters but a single rasperry 4 8Gb where i run docker with all my services. Which includes : plex, immich, nextcloud, memos, linkding, froodle pdf, mealie, pihole, qbitorrent and a few more. To be honest it runs better than I expected, it's not perfect but clearly good enough, i'm still looking for a new machine because clearly some services could run a little bit faster but if you only have this available it's alright. I did store everything on a external HDD (even docker files) because sd cards have garbage speeds. ---------------------------------------------------------------------- _nix-addict · 2023-10-01 16:50 This depends on the laptop you're comparing and the Raspberry Pi model as well. If you're trying to recreate a 16 gig laptop with Raspberry Pi Zeros then you'll be in for quite the ride. The real problem with Raspberry pis for real server loads is their lack of native support for SSD storage and any kind of vertical scaling needs. ---------------------------------------------------------------------- readit-on-reddit · 2023-10-01 17:20 This seems to be the narrative spreading on Reddit now. But an optiplex it's just not comparable. 1. The power draw is very different. 2. The Pi is smaller. Almost invisible in a home. 3. An used optiplex is still at least twice the price. Looking at random posts on Ebay right now it's realistically around 3 times the price of a Pi. ---------------------------------------------------------------------- soulless_ape · 2023-10-01 17:32 I used to buy raspberry pies I now buy used hp, dell and Lenovo mini pc instead they are still relatively low power but are true x86-64 cpu and you can virtualize full systems or run lxc containers. Those systems paired with proxmox make for great functional and scalable home labs. ---------------------------------------------------------------------- soulless_ape · 2023-10-01 17:34 eBay is more expensive, I get them on Amazon for 90+ bucks. ---------------------------------------------------------------------- opensrcdev · 2023-10-01 17:45 Your spare laptops are much better self-hosting hardware than RPis. I have a bunch of RPis, and they're too weak to run very much. They're CPU and memory limited for all but the lightest tasks. On my x86 Linux servers, I primarily run LXD + Docker on them, so I can spin up container services and Linux virtual machines. ---------------------------------------------------------------------- ChickenMcRibs · 2023-10-01 18:06 I use an rpi 4 for self hosting. Been about 4 years now. I run deluge, sonarr,radar,bazarr, Plex+Plex debrid, wireguard, nextcloud. Also run frp on it to get ports on a second raspberry pi that is behind a cgnat. No issues so far. ---------------------------------------------------------------------- -quakeguy- · 2023-10-01 18:11 1. This was an argument to make when Pis were a new thing, but not today. Figuratively speaking, nobody cares about 4-5 times higher total system power draw when you get 10-20 times the performance from it. 2. If a single Pi serves all your needs, sure. Problem is, very few people’s needs are served by a single one. When you have 3-4 of them, with their drives, with their switch and network and power cables… the size and sight isn’t pretty AT ALL. ---------------------------------------------------------------------- readit-on-reddit · 2023-10-01 18:33 > Figuratively speaking, nobody cares about 4-5 times higher total system power draw when you get 10-20 times the performance from it Absolutely not true. With energy costs on the rise why would someone have something that generates more heat and consumes more power 24/7 if a raspberry Pi does the job just fine? Imagine someone wants to run Pi-hole and a few other lightweight jobs. It would be extremely silly to buy something that is more powerful but more power hungry if you're not going to use that power anyway. What you mention is only true if someone is looking to run more demanding applications like multiple docker containers and/or a media server that transcodes on the fly. > When you have 3-4 of them, with their drives, with their switch and network and power cables… the size and sight isn’t pretty AT ALL. If you are going to have multiple Pis that could be replaced by a single x86 based server then sure. But to me, a Pi was always a device people bought when they were not fully committed to be system admins in their own homes. It's more for people that have one application in mind, like Pi-hole and Home Assistant. ---------------------------------------------------------------------- SelectCase · 2023-10-01 18:37 I started out on a Raspberry Pi cluster with self hosting. Depending on the age of your laptop, a Raspberry Pi 4 might actually out compete it. The major advantages of a Pi cluster are: * Extremely low power consumption. Idle Pis use 2-3 watts, loaded pis use 7-8 watts. Running a Pi under 100% load 24/7 will only eat about 10$ of power annually. * Fantastic for learning. Since they run on SDs you can swap the entire operating system every day if you want to. And since flashing a new card takes like 5 minutes, you can do all the changes you want with basically no risk of causing an extended downtime. * Space. My big boy box is full sized desktop. installing a DR box means I have to have another big boy desktop. With a pi cluster, you can have 12 Pis easily in the space of one desktop. Or just two mirrored pis in about the space a box of crackers takes up. The main disadvantage is the lack of processing power. Jellyfin/other big services will run on a Raspberry Pi, but it's going to cry if more than two people need stuff transcoded at the same time, and it's going to cry harder if it's trying to handle 4k video. I've also noticed that sites hosted on the Pi aren't quite as responsive of sites on a my current main server, but its' a fantastic place to start. As far as what you can run on a raspberry pi, event though I do have a big boy server now, I offload services that are lower-power to the Pi, so my big box can spend it's time transcoding video on demand and handling streaming/video chat. I currently run these on my 8gb Pi to free up my main server for more resource intensive stuff: * My dashboard. (I use Homarr for myself and envlinks for my friends) * Status monitor (uptime-kuma) * Gitea (for my version control and code base) * A container with LibreOffice/VNC for remote office access * Password manager (vaultwarden) * code-server (vs code in a container) * A personal relationship manager (Monica) * A pdf swiss army knife (sitrling-pdf) * and a seperate paperless instance for work related paperwork. * My custom container and replication manager. (I call it Shittiernetes) * Pi-hole * Dynamic DNS updater (ddclient) Note that it can handle way more. All of these services consume under 3gb of memory. ---------------------------------------------------------------------- obeythelobster · 2023-10-01 18:38 Whats VPS do you use? I am looking for a VPS solution to access my private network, but I seems every VPS nowadays is focused on browsing the web anonymously. I not sure if they work for my case. Any recommendations? ---------------------------------------------------------------------- TXAGZ16 · 2023-10-01 18:46 Can’t you boot from an SSD? ---------------------------------------------------------------------- Howdanrocks · 2023-10-01 18:58 1. The power draw is not "very different". We're talking a 20 watt delta between the two. Even in places with high electricity costs that won't be noticed on your bill. 2. Do you live in a literal shoebox? SSF/mini/micro elitedesks and optiplexes are very small. The cable management is also nicer since power and ethernet are both on the back as opposed to the RPi. 3. You're bad at eBay searching. I'm staring at multiple listings for G2&G3 elitedesks for under $65 right now. I'd also check Facebook marketplace as my area is flooded with them there. ---------------------------------------------------------------------- lucassou · 2023-10-01 19:10 To be honest I had read a while ago that this was not possible but this is appartly possible. I don't have an external SSD available but it's clearly a better idea than my current setup. ---------------------------------------------------------------------- -quakeguy- · 2023-10-01 19:43 In both of your counterarguments, you actually circled back to my original point - a Pi serves well a person whose entire needs are served by a single unit. As soon as that’s not enough (which doesn’t take much), it becomes a garbage choice. But sure, I am certain that there are people out there whose needs are entirely satisfied by a single Pi. ---------------------------------------------------------------------- readit-on-reddit · 2023-10-01 19:52 Well, I rephrased it and expanded on it since you started off with "nobody cares about power draw if one of the machines is much more powerful". So I specified why it's extremely common for a Pi to be enough. Even two Pi's might still be a better choice than an used Desktop. Plus, this was also created for education and young people. I imagine is harder for parents to accept a desktop running 24/7 as opposed to a credit card sized computer running 24/7. ---------------------------------------------------------------------- Howdanrocks · 2023-10-01 20:04 If you want a cluster for learning purposes you could check out used thin clients. Something like a Dell Wyse 3040 with an Atom x5-Z8350 is as powerful as a RPi 4 but you can get them for less than half the price (~$25-30). ---------------------------------------------------------------------- majordingdong · 2023-10-01 20:22 Yes. By flashing the EEPROM. Very easy to do and with plenty of guides online. ---------------------------------------------------------------------- Aurailious · 2023-10-01 20:39 Right now DNS, DHCP, and K8 control plane. ---------------------------------------------------------------------- fm2606 · 2023-10-01 21:01 4 are on the network but only 3 do something. ---------------------------------------------------------------------- fm2606 · 2023-10-01 21:03 I use kamatera for $5 a month. They had a month free and it is enough for what I need. I think Digital Ocean still has $4 or $5 VPS. ---------------------------------------------------------------------- MalcolmY · 2023-10-02 02:13 I used to run a raspi 32bit with pihole, which was absolutely fine. Then I got docker and containers and I started seeing a bit of lag, understandable. Then I noticed plenty of containers are not available for 32 bit, so I installed 64 bit OS. The unresponsiveness and the lags and were just unbearable. I fixed all the by buying an old used Dell PC with 4670 CPU or something from that era, which was infinitely better than a raspi. It costed me more but gave a me a peace of mind, and better reliability and uptime. ---------------------------------------------------------------------- lalcaraz · 2023-10-02 02:35 I have 4 boards: docker swarm (because I hated k3s) for bunch of public facing websites, my local network env (firewall, router, hotspot, dns, dhcp, proxy, vpn terminator), and some internal apps (Jellyfin, family chat, email relay service, voip service, etc) ---------------------------------------------------------------------- Nagashitw · 2023-10-02 04:51 Shittiernetes. I love it. Hahaha ---------------------------------------------------------------------- keepcalmandmoomore · 2023-10-02 05:05 I'm curious, would you run a firewall on it too? Or use a seperate pi for that? ---------------------------------------------------------------------- SelectCase · 2023-10-02 06:35 My inspiration was "what if kubernetes was written only in bash scripts?" ---------------------------------------------------------------------- VAI_YR · 2023-10-02 06:44 Hah, ever heard of processor architecture? Even your applications and containers aren't the same. You'll never reach the computing power of a full sized laptop, even when strapping 50 zeros together. At least in a single application. When running big monolithic applications you'll simply need more than 4GB of RAM, so rpi>=4 with 8GB of RAM and that will get expensive. If you aren't in microservices or any other architectural paradigm you will reach a border. For the big things I currently have three old 1U servers with dual xeon and 32gb each.. Sums up at 24cores and 96gb, but they take ~200watts each in idle and can reach up to 600w. But I just run them when I have to so it's ~2$/month. My pi runs all the time and consumes around the same. And that's the point, for the most simple applications they just fit and are extremely efficient. So why not strap some of em together and get started simply with some clusters before getting your hands on cloud machines or industrial hardware. Get started slow with what you have! ---------------------------------------------------------------------- SelectCase · 2023-10-02 06:48 Depends how much traffic you're going to run through it. I think the raspberry pi 4 is still basically using the USB controller as combined NIC. I'd use it as a firewall to learn to build one and mess with IP tables, but definitely not going to put it between a gigabit Internet connection and a home network. ---------------------------------------------------------------------- FierceDeity_ · 2023-10-02 09:46 Won't be noticed? 20 w is 480w per day, let's say 500w. 100 days, 50kw. A Year, 175 kw or so. At 0.30 per kwh that's 50 currency. On the monthly that's like a 5 increase almost. it's probably noticeable ---------------------------------------------------------------------- Wolf-yuan · 2023-10-02 10:30 Have you tried to update your firmware (aka EEPROM) on your pi? ---------------------------------------------------------------------- MalcolmY · 2023-10-02 11:30 No. Is that something that should be done when moving to 64 bit OS? ---------------------------------------------------------------------- _nix-addict · 2023-10-02 11:58 Youre still limited by the read/write of an SD card as well as it's lifespan. Otherwise you are passing SSD over USB which is not exactly native SSD support. Yes they are energy efficient. Learn to use a paragraph ---------------------------------------------------------------------- Howdanrocks · 2023-10-02 13:18 Yes, I'd classify less than $4.50/month as unnoticeable on a monthly electric bill. ---------------------------------------------------------------------- FierceDeity_ · 2023-10-02 13:48 Speak for yourself, I'm a student and a fiver could mean something. IT enthusiasts wanna eat, heh. ---------------------------------------------------------------------- Howdanrocks · 2023-10-02 13:56 If you're in a dorm you're not paying for power. Also, see my edit. ---------------------------------------------------------------------- TheMcRibReturneth · 2023-10-02 14:22 Because they are insanely low TDP and the barrier for entry for a cluster is super cheap. They don't take up much space, they run off of an SD card, and they get everything done pretty well. ---------------------------------------------------------------------- FierceDeity_ · 2023-10-02 14:52 I'm not in a dorm, no. I have to pay my power bill myself and the power prices are like 38 cent per kwh for me right now ---------------------------------------------------------------------- Kerboq · 2023-10-02 15:23 How does PleX perform on an RPi4? Do you have many concurrent streams/what quality do you generally stream? ---------------------------------------------------------------------- herrjonk · 2023-10-02 16:25 Not op but it depends if you direct stream or not. I would imagine it cannot handle many transcodes, but probably multiple direct streams ---------------------------------------------------------------------- lucassou · 2023-10-02 16:51 I always stream original quality, this is not an issue for me because I almost never want a lower quality than what I originally downloaded so performances is quite great, i'm pretty sure it would not be an issue to have a few concurrent original quality streams. You can kiss goodbye to transcoding tho, it might work for 1 stream for a small file with a small resolution but don't count on that, it might be a downside for some people but personnally I really don't care about it. ---------------------------------------------------------------------- Kerboq · 2023-10-02 20:04 interesting. I run my plex on a nas, but I need the power for transcodes + multiple streams. I have an RPi 4 tho, it's good to now I could use it as a back up should my main server experience some problems. ---------------------------------------------------------------------- obeythelobster · 2023-10-02 22:17 I know you said VPS but I was actually thinking about VPN 😬, my bad ---------------------------------------------------------------------- Wolf-yuan · 2023-10-02 22:35 No, but sometimes updating it will help. ---------------------------------------------------------------------- GrumpyCat79 · 2023-10-02 22:38 It wasn't initially supported when the Pi4 came out, but it was possible to put GRUB on a SD card and point it to the SSD. They since updated their firmware and it's now possible to boot directly on an external drive. It's been supported for at leaat a year or two now ---------------------------------------------------------------------- fm2606 · 2023-10-03 00:54 I also have a bpn and I use PiVpn