What scenarios does a home server need a GPU? reddit / selfhosted ---------------------------------------------------------------------- TaCoDoS2 · 2023-11-27 21:16 I want to host security cameras and a plex server. Does this mean that my server needs a GPU? (Or would benefit from one). I heard plex does fine with just a CPU ---------------------------------------------------------------------- whereswally007 · 2023-11-27 21:34 AI projects Stale Diffusion for example ---------------------------------------------------------------------- FelisCantabrigiensis · 2023-11-27 21:35 My home server has a GPU because neither the motherboard nor the CPU have any builtin graphics, so I have a low-end passively-cooled GPU to display the BIOS setup and boot messages. ---------------------------------------------------------------------- Acrobatic_Assist_662 · 2023-11-27 21:36 If you want ANY additional video processing power you’ll benefit from a gpu. Plex may work okay with just the cpu but this also relies heavily on the format of the video files and whatever service or player you’re using to access/watch them. For security cameras I don’t have too much experience but if you want access to facial recognition and so on, it will also benefit/need a gpu. Your cpu igpu can usually do this and may circumvent the need for a dedicated gpu but your use may vary ---------------------------------------------------------------------- Kaikidan · 2023-11-27 21:46 I use a T600 4gb for plex trasncoding, could use the quicksync but 6th gen QS is not the best, besides trasncoding I also use it for: * Subgen: generate subtitles for video using whisper. * Immich: helps reducing cpu load when processing photos. * Unmmanic: trascode video files to smaller sizes. * DVR: CUDA can be used for objetct detection and recognition. * AI: You can run a instance of stable difusion or Llama GPT but it needs a really powerfull GPU and your powerbill wil sktyrocket. Those are some examples of uses for a GPU. Of all of those, I use Subgen the most, with the T600 4gb I can handle a single transcription at a time using the large v2 model. Ideally it needs at least 6gb but.. it is what it is... ---------------------------------------------------------------------- floppydiet · 2023-11-27 21:48 Self hosted LLMs ---------------------------------------------------------------------- releenc · 2023-11-27 22:00 Plex can only use the GPU if you haver the paid version. Those features are disabled in the free version. I run my plex server on Windows Server 2016 with an I7-3770K CPU @ 3.5GHz and 16GB RAM, with no dedicated graphics card, just the Intel on CPU graphics. I have no performance problems with transcoding videos using the free version of plex. ---------------------------------------------------------------------- Firm_Nothing_2223 · 2023-11-27 22:14 So my Lenovo P520 Thinkstation with 6 core Xeon W2135 and a P2000 GPU has 8 eight 2k cameras transcoding with Frigate sits at 20% CPU load. If I do one HEVC 1080 stream with Emby to Firefox the CPU load jumps to 90%. So if you don't have Quicksync cpu and want to do anything with video where the client may not transcode I highly suggest one. And well I despise Nvidia, CUDA seems to be the go to for a widest variety of projects. I assume this will change as Intel Arc gets better Linux support. ---------------------------------------------------------------------- jerwong · 2023-11-27 22:17 I use Jellyfin which is similar to Plex. I have it on a Raspberry Pi 4 8 GB. It's perfectly fine if I'm sending H264 but most modern browsers do not support H265 so it forces the server to transcode. That will consume almost all processing power if it's CPU-only and is a very slow process. ---------------------------------------------------------------------- LavaCreeperBOSSB · 2023-11-27 22:36 If you use a third party app that can play directly, like **Infuse**, or your CPU has Intel QuickSync, then you'll likely be fine. For security cameras unless you need any sort of facial recognition or object detection just a CPU will still be fine. If you need anything more advanced then a GPU is necessary ---------------------------------------------------------------------- pak-ma-ndryshe · 2023-11-27 22:38 With all the SaaS monthly subscriptions adding up, I see every office in the future opting for self hosting with the added benefit of running unlimited unrestricted local LLM for their teams. I just wish, open source project took design more seriously. Nextcloud could become gigantic if they implemented some monday.com features for example. I am doing just that currently, installing self hosted solutions in some offices and a client bought up the idea of hosting ChatGPT in the server. ---------------------------------------------------------------------- I_want_pudim · 2023-11-27 23:09 If you want your own ChatGPT, LlamaGPT. ---------------------------------------------------------------------- Powerstream · 2023-11-27 23:11 Bought an Intel A750 for transcoding on my Jellyfin server. The CPU did fine for just 1 or 2 people watching at the same time. It just pegged out the CPU at 100%. Plus had limited support for what types could be transcoded. With the A750 can easily handle several people watching at the same time and can transcode any format. With little CPU usages, so everything else running stays fast. ---------------------------------------------------------------------- ArgoPanoptes · 2023-11-27 23:13 If you want object detection with your ip cameras, you can use Frigate, and to have good performance, you can buy a Google Coral to perform the object detection part. ---------------------------------------------------------------------- SchindlersIgnoreList · 2023-11-27 23:22 Transcoding ---------------------------------------------------------------------- Still-Snow-3743 · 2023-11-27 23:41 I have found transcoding to work noticeably better when using quicksync (the intel chip native encoder) rather than a GPU. At this point, I think the only real reason you would want a GPU is for LLMs. ---------------------------------------------------------------------- MrClickstoomuch · 2023-11-27 23:47 Yeah, and while ROCM for AMD is catching up, it still is buggy for a lot of projects. LLM / AI is especially a problem for some tools. I'd generally lean towards AMD versus an Arc GPU b/c even with the better encoding methods on Intel, I think the extra performance for most and GPUs is better than Intel Arc. Though I do know they've been getting better quickly with driver updates. ---------------------------------------------------------------------- justan0therusername1 · 2023-11-28 00:38 Yup. If you run any LLMs for any bit of time saas charges get steep ---------------------------------------------------------------------- ZaxLofful · 2023-11-28 00:40 Plex server or running your own LLM ---------------------------------------------------------------------- hypercyanate · 2023-11-28 00:47 For facial recognition/object detection a coral device or something similar is much more appropriate and cost effective. They aren't much use for anything else other than something relying on ML ---------------------------------------------------------------------- Acrobatic_Assist_662 · 2023-11-28 00:59 Oooooh. I wasn’t sure and I see places advertise gpus in NVRs all the time so I thought that was the overlap ---------------------------------------------------------------------- fliberdygibits · 2023-11-28 01:04 Just about anything Machine Learning or AI, Transcoding for a media server, render farm for something like blender perhaps? ---------------------------------------------------------------------- hypercyanate · 2023-11-28 01:11 GPUs were the go to for a long time, these coral devices are fairly new and may not be supported by some NVRs yet as well, they were also really hard to get for a while. ---------------------------------------------------------------------- Kltpzyxmm · 2023-11-28 01:18 Get an Intel cup with iGPU and you’ll be good to go for anything that you’re doing there. ---------------------------------------------------------------------- jimheim · 2023-11-28 01:19 A decent, recent Nvidia GPU is going to beat most CPUs. I wouldn't shell out extra for a GPU just for transcoding, though. Good enough is good enough. ---------------------------------------------------------------------- Still-Snow-3743 · 2023-11-28 01:23 8th gen, so not even that recent intel chip can do 24 similtaneous transcodes: [https://www.reddit.com/r/PleX/comments/hrpuhf/wow\_quicksync\_on\_new er\_gen\_intels\_are\_transcode/](https://www.reddit.com/r/PleX/commen ts/hrpuhf/wow_quicksync_on_newer_gen_intels_are_transcode/) A RTX 4090 can do... 5: https://www.guru3d.com/story/nvidia-removes-encoding-limitations-on- consumer-gpus-allowing-up-to-5-simultaneous-streams/#:\~:text=NVIDIA%2 0has%20made%20a%20quiet,limitation%20of%20three%20NVENC%20encodes. ---------------------------------------------------------------------- ggezboye · 2023-11-28 01:32 If you use Shinobi CCTV you can use the dGPU for encoding/decoding (will also work for media center application) and even some opencv stuff. ---------------------------------------------------------------------- levogevo · 2023-11-28 02:02 Jellyfin transcoding. ---------------------------------------------------------------------- DearJudge · 2023-11-28 02:14 You won't need a GPU for what you want to do. A GPU is more for heavy transcoding loads, or when you really need compute (i.e. training and deploying your own ML models, I'm assuming you won't be running your own weather simulation). A reasonable rule for this stuff is that if you have to ask, you probably don't need a GPU. It's fairly unlikely that you'll ever set something up without it being very obvious ahead of time that a GPU is required. ---------------------------------------------------------------------- Frostlike2816 · 2023-11-28 03:06 Oh snap, I can use GPU for immich?! ---------------------------------------------------------------------- Frostlike2816 · 2023-11-28 03:10 That transcoding limit gets A LOT bigger with https://github.com/keylase/nvidia-patch I have a GTX 1650 Super, and can do like 15-20. Can't even imagine how much a 4090 could do. ---------------------------------------------------------------------- Still-Snow-3743 · 2023-11-28 03:15 I've used that patch on a 1050 ti to do my plex transcodes and yeah it worked, but I had to write a lot of supervisor programs to detect when the GPU crashed, reinstall the drivers, and reapply the patch. It was so much effort and it sucked. Then a friend of mine said "just use quicksync" and everything was easy again. Hardware transcoding is done on its own dedicated circuitry specifically designed to encode and decode x264 /x265 streams and doesn't actually use the compute resources of the GPU. I would be surprised if the 4090 GPU actually had any more transcoding power over a 1650, it's just a feature of the architecture and not a feature of its processing power. ---------------------------------------------------------------------- xardoniak · 2023-11-28 04:11 I use a p4000 to transcode Plex content. The GPU uses less power than maxing out my CPU. I also have the GPU available in Kasm though I'm unsure if that works as I mostly use kasm as a SSH jumpbox or access to an unfiltered Chrome session at work. When I upgrade my CPU, I plan to put my spare 1650 in my host and spin up a windows gaming VM for guests and remote play. ---------------------------------------------------------------------- dal8moc · 2023-11-28 04:26 Given that the op wants to host security cameraS (!) a gpu is pretty much needed for video decoding. Especially when nowadays the cameras aim for 4K. Of course a cpu with integrated graphics would be enough. But it still is a gpu. The other part of the equation would be a coral accelerator which greatly reduces CPU load for object recognition in apps like frigate. Plex or Jellyfin isn’t really a concern for a normal household imho as most end devices can decode on the fly. So the transcoding capability is seldom used. ---------------------------------------------------------------------- ITXEnjoyer · 2023-11-28 04:58 I added a cheap Quadro p400 to my Unraid server as the old girl (4790s) couldn’t handle h265 on the igpu. Far cheaper and easier than a whole server upgrade just for a little more capability when transcoding. Interested in the arc a380 just for transcoding when the kernel is updated to support the cards in Unraid natively when they eventually do but in no rush to do so. ---------------------------------------------------------------------- UNITY_NP · 2023-11-28 05:23 Consider any low power gpu for media hardware transcoding ---------------------------------------------------------------------- RileyGoneRogue · 2023-11-28 05:33 I stream games from my server PC using steam-headless. ---------------------------------------------------------------------- SeLiKa · 2023-11-28 05:42 You still need Plex Pass to use hardware encode from your integrated graphics. ---------------------------------------------------------------------- JaggedPolygon1900 · 2023-11-28 05:55 If you run an AI server or crypto miner. Mostly ---------------------------------------------------------------------- I_EAT_THE_RICH · 2023-11-28 06:16 Yes, GPU for transcoding is reason enough, but object detection using frigate, or AI stuff is nice also. Buy a nvidia. ---------------------------------------------------------------------- crysisnotaverted · 2023-11-28 06:47 There is significant overlap, but AI accelerators are probably 'better' for video analysis watt-for-watt and GPUs have onboard video encoding hardware + GPU cores. ---------------------------------------------------------------------- micalm · 2023-11-28 07:34 Using their desktop client is a solution. ---------------------------------------------------------------------- metyaz · 2023-11-28 08:08 > steam-headless That just gave me an idea to install Steam on cloud and play from local. Not sure about the performance though. ---------------------------------------------------------------------- sulylunat · 2023-11-28 08:15 Will their CPU even support quicksync? Pretty sure it wasn’t available that early on anyway. ---------------------------------------------------------------------- Suterusu_San · 2023-11-28 08:15 I'm using a spare desktop that uses an 11th gen intel i5, but it doesn't have an iGPU, and the board won't post without a GPU inserted, so I had to install an old 970 I had lying around. ---------------------------------------------------------------------- SeLiKa · 2023-11-28 08:19 It does support quicksync, not that he is going to be using it anyway without Plex Pass 🤷‍♂️ ---------------------------------------------------------------------- RelevantFoundation14 · 2023-11-28 08:51 Man, I didn’t realize I can run whisper locally! I’ve been paying for API calls. ---------------------------------------------------------------------- sphereatmos · 2023-11-28 09:27 With plex, an intel quick sync igpu will be fine unless you plan to do >10 transcodes at a time ---------------------------------------------------------------------- Krieg · 2023-11-28 10:14 As long as you have Plex Pass, hardware transcoding is extremely good with moder QuickSync Intel processors, and specially good if you run Linux. ---------------------------------------------------------------------- stupv · 2023-11-28 10:23 GPU accelerated cloud isn't cheap either ---------------------------------------------------------------------- agent_kater · 2023-11-28 10:34 > That will consume almost all processing power if it's CPU-only and is a very slow process. This is a complicated topic and the terminology is a bit ambiguous. Yes, non-hardware-accelerated transcoding is slow and will consume the CPU. However, you don't necessarily need an external GPU to do hardware- accelerated transcoding. When you use Intel QuickSync for example, the codec hardware is part if the CPU. On the other hand it is only in CPUs that have integrated graphics, so you could still say the transcoding is done "by the GPU", just not the additional one that you put in. In fact, putting in a dedicated graphics card often disables the integrated graphics and you have to use tricks to re-enable it before you can use it for transcoding again. ---------------------------------------------------------------------- agent_kater · 2023-11-28 10:37 In fact, putting in a dedicated graphics card often disables the integrated graphics including QuickSync and you might have to set up a virtual screen to re-enable it. ---------------------------------------------------------------------- voprosy · 2023-11-28 10:49 What self hosted solutions are your installing for your clients? Genuinely curious. ---------------------------------------------------------------------- voprosy · 2023-11-28 10:54 Transcoding is only necessary if the client can't run the video natively. With modern clients, you'll never run into a situation where transcoding is needed. Is this correct? ---------------------------------------------------------------------- Im_Caster · 2023-11-28 10:56 What is LLM?? ---------------------------------------------------------------------- jewbasaur · 2023-11-28 10:59 My main server has a 3070 and I use it to stream games through moonlight to all my tvs and computers around the house. That way I get the most value from the card instead of it being locked into one machine ---------------------------------------------------------------------- datenresilienz · 2023-11-28 11:06 https://en.m.wikipedia.org/wiki/Large_language_model ---------------------------------------------------------------------- DarkKnyt · 2023-11-28 11:43 I have two GPUs in a single tower. A GTX 750 to that I share with my LXCs. It does jellyfin transcode, frigate nvr for 3 cameras, kasm accelerated desktops, xfce4 pve host acceleration, Jupyter tensorflow, ersatz tv transcode, and I plan to use it for immich. At most it is taxed about 25 percent but I plan to have a lot more nvr and jellyfin streams. I also have a 1660 ti passed to windows 11 VM for my gaming VM. I use sunshine and moonlight for remote gaming but I also roll easy diffusion for some image generation. I had an LLM but (https://github.com/oobabooga/text-generation-webui) but it was too slow for what I'm used to - I just use bing chat and now meta on whatsapp for my personal and an LLM I have access to at work. ---------------------------------------------------------------------- how_now_brown_cow · 2023-11-28 12:10 This plus Plex transcoding ---------------------------------------------------------------------- Cylian91460 · 2023-11-28 12:39 Ai, encoding ---------------------------------------------------------------------- the-berik · 2023-11-28 12:56 Also wondering whether it could be interesting to run the LLM training, especially on moments their is overcapacity of solar energy, for example. Also just either more heavier scripts which don't need to run on exactly set periods, I can imagine to start running GPUs more heavily the moment there is overcapacity of energy, by starting cron jobs on those moments. ---------------------------------------------------------------------- davepage_mcr · 2023-11-28 13:56 There's the possibility of self hosted speech recognition for use with Mycroft or other personal assistants. Saves you from having Amazon listen to your every word. ---------------------------------------------------------------------- autogyrophilia · 2023-11-28 13:57 Assuming that you have videos in codecs that are streamable. Does Plex support AV1 yet? If you have videos on WMVx or MPEGx those need to be transcoded. ---------------------------------------------------------------------- Frostlike2816 · 2023-11-28 14:13 Fair enough. I've only had to reapply the patch when the driver updates. What OS were you on? ---------------------------------------------------------------------- siddhesh_s7 · 2023-11-28 17:13 Plex will benefit with GPU and for security cameras you can have something like Frigate for object recognitions ---------------------------------------------------------------------- mrdeadbeef · 2023-11-28 19:19 plex would benifit from using a gpu ---------------------------------------------------------------------- lilolalu · 2023-11-28 20:28 You mean like this? https://nextcloud.com/blog/ai-in-nextcloud-what-why-and-how/ ---------------------------------------------------------------------- lilolalu · 2023-11-28 20:37 Here is a little quote for you: "Inflection AI has released the Massive 175B Parameter Model- Inflection-2. It is the latest language model developed by Inflection, aiming to create a personal AI for everyone. It has been trained on 5K NVIDIA H100 GPUs and demonstrates improved factual knowledge, stylistic control, and reasoning abilities compared to its predecessor, Inflection-1." So, as a starter get 5000 Nvidia h100 for roughly 175.000.000€, then gather a dataset with 175 Billion Parameters to train the LLM (or 1.76 trillion if you want to compete with chatgpt-4) then a couple of months of running the training process and you are almost there :) ---------------------------------------------------------------------- releenc · 2023-11-28 21:04 Agreed. The point I was making is that without the use of a GPU I still have good performance. ---------------------------------------------------------------------- releenc · 2023-11-28 21:07 That is generally correct. If you use a Plex client, there is no need to transcode. ---------------------------------------------------------------------- Kaikidan · 2023-11-29 00:41 yes, but only for photo video conversion, I know that it's possible to use it for facial recognition and those smart functions, someone made it work, but I don't see it in main branch, that would be fantastic. ---------------------------------------------------------------------- the-berik · 2023-11-29 00:52 Hmm. I'd rather see it in a more closed, specialized domain. There's a lot of domain specific business knowledge ChatGPT would never have access to to learn from. Quality data, policies, procedures, etc. (business related). I would see the purpose of training an LLM on this to get a "smart" internal "search engine." On the other hand, I realize most of Googles Dell based GSA's ended up in homelabs. ---------------------------------------------------------------------- Jedkea · 2023-11-29 04:39 I’ve thought about this too. I think you would still need a ton of “normal” data however. I don’t think you would have near enough internal data to create a basic LLM ---------------------------------------------------------------------- lilolalu · 2023-11-29 06:09 I think from my limited knowledge of AI technologies, I think what you are looking for is not training a LLM. Which is near impossible for a "normal" person without a research team, massive financial and computational resources me so on. Adding information about your data to an existing LLM on the other hand is a different story. There are several existing projects running LLMs over "external" data to make their information available to the algorithm: https://github.com/arc53/DocsGPT Nexctloud is allowing integration of AI tools already: https://nextcloud.com/de/blog/introducing-hub-5-first-to-deliver-self- hosted-ai-powered-digital-workspace/ So, I think that's already doable. ---------------------------------------------------------------------- RickRotHut · 2023-11-30 17:22 [Nextcloud](https://docs.nextcloud.com/server/latest/admin_manual/ai/i ndex.html) with [Local AI](https://localai.io/) and some LLMs installed If you use StableDiffusionXL for image generation and LLAma as a Text- AI, you can use CUDA with your Nvidia GPU for example (if you make the GPU available to the Local AI container). ---------------------------------------------------------------------- TaCoDoS2 · 2023-11-30 22:20 Will AMD integrated graphics be okay? ---------------------------------------------------------------------- TaCoDoS2 · 2023-11-30 22:23 Would AMD integrated graphics also work? ---------------------------------------------------------------------- releenc · 2023-11-30 22:34 I would assume so.