There are jobs where the company wants to hear about your homelab, here is one example. reddit / selfhosted ---------------------------------------------------------------------- Nebakanezzer · 2023-10-12 22:02 I had a guy sitting in his basement next to his rack during a video interview. He got the job. It's definitely not required, but a plus ---------------------------------------------------------------------- r3dk0w · 2023-10-12 22:06 I interviewed someone that mentioned they had a homelab. I started asking ad-hoc questions about what they do with it, what hypervisor, etc and it was very apparent that the person interviewing didn't actually have a homelab. He didn't get a call back. ---------------------------------------------------------------------- BloodyIron · 2023-10-12 22:08 LOL yikes D: ---------------------------------------------------------------------- BloodyIron · 2023-10-12 22:09 How loud was it?... I SAID HOW LOUD WAS IT? ;P really though, I guess not all that loud? ---------------------------------------------------------------------- Nebakanezzer · 2023-10-12 22:29 It wasn't bad actually. And he wasn't wearing a headset either... Maybe he had some noise cancelling on his mic. ---------------------------------------------------------------------- FuriousRageSE · 2023-10-12 22:36 Today i updated drives and stuff on my new AMD mainboard.. It supprised me that they actually have an AI dual way noise cancelling functions. ---------------------------------------------------------------------- oxamide96 · 2023-10-12 23:11 I have homelab and I would blank at your hypervisor question ---------------------------------------------------------------------- flecom · 2023-10-13 00:00 would have to attach a vizio, no way I could put it all in one text box ---------------------------------------------------------------------- paradoxally · 2023-10-13 00:24 I'd just say "What hypervisor? I just use Docker" lol ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 00:43 Exactly! What year is this? We don’t use hypervisors anymore bro, that VMware stuff was so 2013. Nowadays I’ve got a k3s install with argocd and cluster autoscaler and actual api’s to automate against ---------------------------------------------------------------------- Nebakanezzer · 2023-10-13 01:04 That is pretty dope. I am jealous tbh. Over 20 years ago when I started doing music as a hobby, before professionally, and way before transitioning into IT, I had to forgo buying a car to buy all the analog equipment just to be able to scratch some records and record some crap quality songs. I eventually moved to the PC, which at that time was fruity loops (no studio version yet) 3.0 and "cool edit" which eventually became adobe audition. Still needed to drop thousands into a good external soundcard/daw, mic, monitors (speakers), etc.. Then I still had to treat the room since it was in my apartment. now you can grab a scarlett off amazon, a mic shield, headphones, and a decent mic for under $500 and the AI effects will take care of any noise the lack of treatment creates, and bam, you're on soundcloud. ---------------------------------------------------------------------- bripod · 2023-10-13 01:06 What if you run Hyper-V? ---------------------------------------------------------------------- InasFreeman · 2023-10-13 01:10 AWS says huh? :) ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:12 Well in this particular case it was actually for a Linux-centric role/job, so... yeah ;P ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:13 Due to lack of experience with hypervisors, or...? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:16 Running a k8s cluster node bare metal only goes so far. With the rational justification of the 110-ish pod limit by default per k8s node, even R720's will run out of # of pods before they run out of CPU/RAM/etc. There's substantial value in a hypervisor between metal and a k8s node. Not just for the capacity aspect I just mentioned, but for things like backups, live-migration of k8s nodes to avoid outages, and all kinds of stuff like that. And what dinosaur still uses VMWare? Slap Proxmox VE on that and get with the times bruh ;P Also, there's plenty of "things"/systems/software one might want to use that doesn't really have a container image already made, or isn't really feasible to containerise. ---------------------------------------------------------------------- NikStalwart · 2023-10-13 01:17 include it as a data: url ;-) ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:18 I wouldn't even bother with a $500 mic, the Antlion USB wired modmic is GORGEOUS when you apply some filters to it, as I've done in OBS. Forget this moment which filters, but it blows how other content creators use their mics out of the water massively, lol. Don't even need AI stuff either ;P (unless I guess you're accounting for background rando noise?) ---------------------------------------------------------------------- FamousSuccess · 2023-10-13 01:18 I think the term homelab implies that you're doing enterprise level things on your own hardware at home. But I think thats not necessarily the case, as there's a crap ton of people on here that run a linux distro with docker on it hosting applications. Hell I'd even bet a decent chunk of people on here are nearly terrified of CLI. I run proxmox on one machine, Truenas baremetal on another, and a linux distro on another. They all do different things and allows me to learn different technologies based on different environments. I don't think what I'm doing is a "common" thing to do ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:22 There is absolutely, positively, without any doubt, NOT _just one way_ to do homelabbing. For real. Like if you're working on networking centric stuff, you might not even have any servers at all! Or if you're focusing on storage more than compute, well yeah... Homelabbing doesn't actually imply "enterprise level" things at all. Sure, it's possible, but there's no requirement that that is the way you have to do it! As for whether what you're doing is common or not... so what? Do you have fun? Is it satisfying? Are you learning stuff? Isn't THAT more important? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:22 HAH see if they have any CSP at all, nice ;D ---------------------------------------------------------------------- Dairy8469 · 2023-10-13 01:24 that is probably fine for home use but if someone said that in a technical interview I'd be concerned with their understanding because virtualization absolutely still has a place in the world. ---------------------------------------------------------------------- AlfredoOf98 · 2023-10-13 01:34 or maybe it was just a fake façade showing some lights (like a Star Trek bridge) ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:35 They also didn't account for default pod limits per node ;) ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 01:35 Sure it does. Most sysadmins still only know VMware, hypervisors and the like. But in startup world where I’m at everyone uses kubernetes these days. Just better at orchestration and more extensible (and more complicated). So if someone said they only knew hypervisors in my world I would be concerned in the opposite direction haha ---------------------------------------------------------------------- jared252016 · 2023-10-13 01:36 So does capitalization. :) ---------------------------------------------------------------------- Nebakanezzer · 2023-10-13 01:40 Oh I meant $500 in total for all of that. There are $100 mics now that rival those of the 300-500 range from years ago ---------------------------------------------------------------------- Morkai · 2023-10-13 01:45 > Slap Proxmox VE on that and get with the times I currently have Unraid running, but I want better VM support so I've been considering setting up Proxmox and virtualising Unraid within a VM... Haven't actually pulled the trigger yet however. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:46 👍 ---------------------------------------------------------------------- BloodyIron · 2023-10-13 01:52 Virtualising storage _can_ work, but whether it's unraid or TrueNAS, it really can be a pain in the ass and a pig if things go wrong. You're also battling for resources between functions. A dedicated system for storage (NAS/wtvr) really is the way to go. As for Proxmox VE, dooo eeeettt ;P ---------------------------------------------------------------------- spanish4dummies · 2023-10-13 02:00 Today while in class I used remmina on a ubuntu wsl2 instance on my laptop to remote into a windows 11 pro mini computer in my apartment after installing tailscale on each of them it was rad ---------------------------------------------------------------------- spanish4dummies · 2023-10-13 02:11 if he built it, still impressive ---------------------------------------------------------------------- jonchaka · 2023-10-13 02:48 That question got me my current job. We ended up going over time discussing what we both do. Left the interview with my start date. ---------------------------------------------------------------------- ratcodes · 2023-10-13 02:51 i mentioned mine in an interview and they were pissed i said i didnt feel comfortable doing devops. this was for frontend 😭 i'd rather keep it a secret than have it be something they use to leverage more labor out of me. every time it's come up it's only been used against me, though it might be because im in dev and not IT? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 03:04 Man isn't that just the fucking best? I love when interviews go over, like honestly, because it's usually a super strong sign you got it! And I haven't really had it happen in many years, but when they tell you start date, OH MAN. Satisfaction rains upon. Good job! I bet you felt like a million bucks after that. :) ---------------------------------------------------------------------- BloodyIron · 2023-10-13 03:07 If they want you to do more, and you're interested in doing more, then have them pay you more. If you don't want to do DevOps or don't like it, whether it's for software dev or infrastructure, then so be it. But this is a two-way street. If you actually might enjoy that stuff, well you can make a LOT of money doing it! As for if you want to avoid over-extending yourself into an area you aren't interested in, well there's things like that for me too bud. For example, I do my own PHP/JS/CSS dev stuff with Concrete CMS, but it's like styling/templating of certain things _my_ way. I don't want to do that for a job, so I don't tell anyone that I do that (and that I've taught myself on how to do it). But I talk about lots of other things that I _do_ want to do that is going on in my homeDC, and that does help me get sweet jobs and fat wads. Regain control bud. Don't let others walk all over you. You can do it. ---------------------------------------------------------------------- ILikeBumblebees · 2023-10-13 03:16 How do you attach a television set to a job application? ---------------------------------------------------------------------- PuckSR · 2023-10-13 03:16 Quantify “terrified of CLI” I’m always terrified I’m going to type out some long-ass folder location or complex command arguments and then need to spend 5 minutes just to figure out if I typoed or if the command is actually invalid. So, I default to GUIs if available. But I don’t have any actual fear of typing something simple like; chmod -R 777 ---------------------------------------------------------------------- CryptographerOdd6143 · 2023-10-13 03:46 Much less so compared to a decade ago. Depends on what the interviewer is asking of course. ---------------------------------------------------------------------- Morkai · 2023-10-13 03:53 I do have a Dell USFF PC sitting spare. Unraid is currently on a HP ML10 tower server... Do you think having proxmox on the ML10 with the storage and VMs etc, then Unraid (with docker containers etc) on the Dell USFF accessing storage over the network would be viable? It's all local, all connected to the same gig switch... Hmmm, this feels like a fun project but also feels like I could be adding a bunch of hurdles into what currently works reasonably well. ---------------------------------------------------------------------- sexyshingle · 2023-10-13 04:07 > chmod -R 777 ironically, that one command in particular, you should be a bit afraid of lol ---------------------------------------------------------------------- SomeSysadminGuy · 2023-10-13 04:09 With the power of KubeVirt and Rook, there's really no reason not to run k8s on bare metal! Using something like OKD will bootstrap KubeVirt mostly automatically and get you 250 pods per node by default. For backups, the only valuable things are the PVCs and the etcd data which have plenty of typically integrated backup solutions. Maintenance is easy with draining and tuning of replicas and rollout strategies. I do have a couple things launched in KubeVirt, so even in my aggressively container focused setup, there is some need for VMs. But they're arguably easier to manage via KubeVirt than Proxmox IMO. ---------------------------------------------------------------------- ratcodes · 2023-10-13 04:13 i have a couple offers on the table rn and like to keep it strategic because devops isn't fun for me, but i get what you mean ---------------------------------------------------------------------- VexingRaven · 2023-10-13 04:50 For folders, tab-complete everything all the time. ---------------------------------------------------------------------- VexingRaven · 2023-10-13 04:50 Modern laptops have some pretty impressive microphone arrays these days. ---------------------------------------------------------------------- NanoDrivee · 2023-10-13 05:07 What positions do you apply for that mention this? So far I've only had one chance encounter with a manager at an IT company and he's the only person I've been able to mention my homelab to. I love working on my homelab and I'd love to use that time invested to help me acquire a better job (in terms of pay, interest etc) ---------------------------------------------------------------------- BloodyIron · 2023-10-13 05:28 > HP ML10 I'm not sure if I'm reading the right spec sheets but is that seriously a dual-core Pentium system? A NAS connecting to another system for storage over the network... you're not coming at this from a good angle at all, as that really increases the points of failure to an extremely concerning degree. You want all the disks (HDDs or SSDs or whatever) DIRECTLY attached to whatever is the NAS, be it internal connections, external SAS, or whatever. Like, if you want to test it for proof of concept, fuckin' giv'r, but in no way is this topology reliable, so don't start using it beyond testing it, you're going to create a minefield. Also, consider liquidating and getting some more competent kit for this (there's ways to pinch pennies in this kind of stuff). ---------------------------------------------------------------------- PuckSR · 2023-10-13 05:37 Nah, it fixes everything and has no negative repercussions I can determine ---------------------------------------------------------------------- BloodyIron · 2023-10-13 05:45 > With the power of KubeVirt and Rook, there's really no reason not to run k8s on bare metal So you've found a magical way to run more than 110-pods on a node without it exploding? Because the rationale is that each node is a /24, and when a pod terminates, the internal IP it was using is still held while the replacement is spun up. So if 110 pods terminate, that's now 220 IPs in a /24 that are instantly "in-use" and you have only 34-ish remaining IPs for if any more terminations happen. Now let's imagine you hit that 254 number and still have terminations going on, or you decided to go above the 110 pod number... well you now have traffic going to the wrong pods because pods are holding IPs at times you REALLY do not want them to. Sure, you can up the number to 250. But that doesn't make it a good idea. Even still, let's assume somehow 250 pods is workable enough. That still probably is not anywhere near enough to exhaust the kinds of resources you can stuff in the example R720, let alone much newer models. Proxmox isn't hard. Running k8s on top of Proxmox in VMs isn't hard either. I myself use Rancher and can provision a brand new k8s cluster rapidly in fresh VMs. Yumm fresh baked in the oven! It would actually take me more time to do the same with bare metal. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 05:46 Hey no worries there bud! Just trying to give you some thoughts to chew on is all :) hope it actually helps though, even if just a teeeentsy bit ;) ---------------------------------------------------------------------- MrFibs · 2023-10-13 05:48 You can aftermarket swap out the fans on servers and net gear. Typically OEM chassis are built without decibels in mind, so the focus goes exclusively to air throughput (factoring in air circulating in a full and enclosed rack might be limiting), which of course makes things loud. But if you're in a home environment, and you're going to be near the rack or servers, by all means take a look at the thermals of your devices, airflow around them available, and A/C for where they're located, and get some new (significantly quieter/efficient) fans for the hardware. There's also lining (which'll likely have an insulating affect as well) for rack enclosures you can look in to for dampening the noise as well. You just need to make sure you're realistically evaluating your thermals, airflow, and future growth and plan accordingly. ---------------------------------------------------------------------- Morkai · 2023-10-13 05:49 > I'm not sure if I'm reading the right spec sheets but is that seriously a dual-core Pentium system? That's one of the configs I think, but I currently have a Xeon 1265L v3 in it, which is 4c/8t. My main objective is I want to start playing with Ansible, and I don't really think the 4c/8t will cope running half a dozen VMs on top of what it's already doing. I rescued a workstation from my office out of e-waste, which is (currently) a dual socket Xeon 2680 v4 with 64GB DDR4 ECC, of which each CPU is 14c/28t (I'm almost certainly just going to dump one of them and make a christmas bauble or a keychain or something, I don't need a total of 56t) so this was likely going to be the basis of a new box if I decide to go down this route. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 05:50 For job application forms asking about homelab? I think in the last 6 years I've maybe seen 3. HOWEVER it does come up in interviews with companies that are actually worth working for, pretty often. I think I intentionally actually bring up my homelab in most later stage interviews, as what I've done and what I'm doing is very noteworthy, even if not directly relevant in some regards. So you don't necessarily need to wait for someone to bring it up, you can try to find a good time to segue into it. It takes some practice but once you get the hang of it, it can come naturally. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 05:55 Use that workstation and don't dump the CPU you dingus. That's worth keeping, even if right now you don't use it. Slap Proxmox VE on the workstation, and can you get me a proper data sheet on the HP ML10 you _do_ have? I want to make sure I'm looking at the relevant storage options, before I say good/bad for it being a NAS. ---------------------------------------------------------------------- SomeSysadminGuy · 2023-10-13 06:31 Not really magical, it's out of the box for OKD. In its case and for all else, you can adjust the node IP pool size. My nodes have a /23 for their container networks. RedHat has also validated using a /22 with 500 pods per node as well. 250 pods is a fairly significant workload for a node. My cluster runs some heavy applications, so I'll certainly be out of memory on my R630s before I'm exhausting the 250 pod limit. Clearly not all workloads are the same, but databases, OSDs, and java apps are common and usually consume a couple gigs of memory each. Promox isn't hard, I'm not saying it's a bad idea, but it's not the objective right answer. ---------------------------------------------------------------------- Morkai · 2023-10-13 07:06 Not sure I have a data sheet per se, but I can log into the ILO controller when I get home and get whatever you need. ---------------------------------------------------------------------- flecom · 2023-10-13 07:23 hah ya thanks autocorrect, but I could find a way ---------------------------------------------------------------------- Croatwink · 2023-10-13 07:27 When I was interviewing people I always asked about their home network/homelab, it was always a delight to hear about. I hired one guy on the spot because he pulled out his phone and showed me a homemade android app to control his home automations. Of course he quit within 6 months because he was way overqualified, you live and you learn. ---------------------------------------------------------------------- sexyshingle · 2023-10-13 08:14 > Nah, it fixes everything and has no negative repercussions I can determine You say that until your machine gets hacked and used to mine crypto or in some botnet. But hey, you do you. ---------------------------------------------------------------------- opensrcdev · 2023-10-13 08:36 Run Linux VMs on Hyper-V. I do it all the time. ---------------------------------------------------------------------- AnApexBread · 2023-10-13 10:28 Please explain to me how you're doing a VDI solution with Kubernetes ---------------------------------------------------------------------- AnApexBread · 2023-10-13 10:32 >But I think thats not necessarily the case, as there's a crap ton of people on here that run a linux distro with docker on it hosting applications This! This sub seems to forget that homelabing can be more than just running a Plex server with Transmission and Radarr. Docker is great for applications, but it's not the only thing people do in home labs. ---------------------------------------------------------------------- LawfulMuffin · 2023-10-13 10:39 A Shure SM57 or even the knockoff (~$20) are honestly hard to beat. ---------------------------------------------------------------------- AnApexBread · 2023-10-13 10:42 If you answered like that in an interview I wouldn't hire you. Your answer would make me think you're pretty short sighted with pretty narrow definitions of what "right" is. ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 10:48 Guess we can just not hire each other then lol ---------------------------------------------------------------------- Pl4nty · 2023-10-13 11:02 there are a few container-based VDI solutions eg [Kasm](https://kasmweb.com/), and k8s can orchestrate VMs via [KubeVirt](https://kubevirt.io/). but virtualisation still has value in cloud-native stacks imo - especially with security architectures like gVisor ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 11:03 Well, you aren’t, so that is one use case for a hypervisor still. It’s not totally obsolete technology. There are containerized os solutions out there, but you saying vdi makes me think of the IT role of managing employee machines which feels a bit different than management and orchestration of application server farms which is typically what I might use k8s for. ---------------------------------------------------------------------- Pl4nty · 2023-10-13 11:08 you can increase the node CIDR and pods per node, /24 and 110 are just defaults not technical limits ---------------------------------------------------------------------- AnApexBread · 2023-10-13 11:24 Kasm isn't really a VDI solution. At least not in a traditional sense. It does provide VDIs but they're ephemeral so they get torn down KASMs VDIs are more designed around "I was to test this thing" , or" I need a VDI for one very specific purpose." rather than a workstation employees will be using all day. ---------------------------------------------------------------------- dazchad · 2023-10-13 11:26 I have worked at and known companies that make a lot of money and have worse infra than my non-enterprise homelab. For me homelab is playing/studying with stuff on your home, hence home+laboratory ---------------------------------------------------------------------- Pl4nty · 2023-10-13 11:27 Kasm supports a few [persistence mechanisms](https://kasmweb.com/docs/ latest/guide/persistent_data.html), ephemeral containers are just the default behaviour ---------------------------------------------------------------------- FamousSuccess · 2023-10-13 11:29 Hard to quantify that. I think fear is proportional to the nature of what you're doing, along with the alternative methods of getting it done. If you're doing something via CLI that has 0 alternatives, and can't be done by GUI, I think a lot of people get nervous real quick. And potentially avoid it. If you're doing something simple like changing directories and installing packages, that (to me) seems fairly straight forward. But theres absolutely a decent part of the group that would prefer downloading a .deb, installing it that way, and avoiding "sudo apt install xyz" ---------------------------------------------------------------------- AnApexBread · 2023-10-13 11:30 >but you saying vdi makes me think of the IT role of managing employee machines which feels a bit different than management and orchestration of application server farms which is typically what I might use k8s for. Because it is different. But you said there's no reason to use virtualiztion because we have docker and K8s. That's fine if you're looking at a very narrow view of "I only want to run applications". But to claim that virtualiztion is obsolete because you're only looking at it through the lense of running an application is short sighted. ---------------------------------------------------------------------- FamousSuccess · 2023-10-13 11:33 Nice. When I leverage my Guacamole setup to hop onto one of my VM's at work I get a lot of "whoa" looks from people. They're not sure what the hell i'm doing but think it's cool nonetheless. ---------------------------------------------------------------------- AnApexBread · 2023-10-13 11:34 Looks like I need to dive into Kasm documentation a bit more. I only really use them for my research purposes. I'm a malware and forensics analyst by trade so having an ephemeral web browser I can go to pull malware with is really useful. ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 11:42 In the days of ChatGPT that fear is basically irrelevant these days anyway. You can literally type what you want to do in ChatGPT and it will give you a correct shell command faster than any gui based approach that you would want. Doesn’t even matter if it’s completely illegible because you can always just turn right back around and paste that same command back into ChatGPT and it will give you a detailed breakdown of what that illegible pile of shell is doing ---------------------------------------------------------------------- FamousSuccess · 2023-10-13 11:50 Fair point. But that doesn't discredit the fact this person is adverse to CLI by their own doing. ---------------------------------------------------------------------- PintSizeMe · 2023-10-13 11:54 On applications it is pretty rare, though I don't fill out detailed applications anymore, if Linked-In or my resume doc isn't adequate for the majority of it, then the company has broken processes. Companies that want you to retype your job history into their form don't use that data in the future, so why bother? As for home tech interest I bring it up in interviews all the time on both sides of the interview, it's a great conversation point and I like working with the geeks and it helps identify the geeks. I really like having team members that like to learn and someone who has a good home setup is someone who will keep learning. I don't care if they learn about stuff directly related to work or not, but everything you learn helps you learn more later, have greater context, etc. ---------------------------------------------------------------------- Preisschild · 2023-10-13 12:15 You can just incrase the pod limit. I use kubernetes directly on bare metal on my homelab and would also do in prod. ---------------------------------------------------------------------- AnApexBread · 2023-10-13 12:35 You're still acting like all businesses (or homelabers) do is run applications. ​ That's certainly a part of it, but it's not the ONLY part. Docker and Virtualization are very different technologies for very different purposes. You're replies very much indicate that just because all you do is host applications that means all anyone else should do is also host applications. ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 12:45 What homelab needs a VDI though? Everyone here is hosting jellyfin, nextcloud, etc. Applications. I'm sure some people actually need these virtualization aspects that something like a container orchestration tool don't offer, but those are quickly becoming the minority of use- cases. In fact VDI is probably the only use-case in this thread being brought up that doesn't really have a suitable container-like replacement I'm not saying virtualization is not important. Your docker/OCI pods are almost certainly running on some virtual machine and not baremetal. But the provisioning of that machine is better handled by a proper orchestration tool when hosting applications vs a hypervisor. Hypervisor way encourages pets, not cattle. In my experience a lot of sysadmins use hypervisor technologies for things like application hosting simply because that is all that they know and don't know about newer, better tools to handle that kind of thing. ---------------------------------------------------------------------- SigmoidGrindset · 2023-10-13 12:48 Even "enterprise level" doesn't necessarily require a hypervisor. I spend > $1m / month in AWS for my day job, yet I've never had a reason to mess around with hypervisors myself (yeah I realise I'm running on Nitro in EC2, but it's still something I don't have to think about). I've run various "enterprise level" services on my homelab machines (e.g. Elastic, Cassandra, Solr, Hadoop etc), but I've always been able to get away with docker (+compose) running on bare metal linux. I've just never felt the need to bring a hypervisor into the mix. ---------------------------------------------------------------------- Mr_Lifewater · 2023-10-13 12:49 I’m curious what you run on your VMs. At my job we’ve moved from VMs to containers, and containers to pods (where possible). VMs are usually a last resort at this point. For example something not cloud native or with one-off outrageous burst requirements. And for those we tend to go bare metal. Or if a true desktop experience is needed sometimes Citrix. Just interested in what other people use VMs for at work nowadays. I’m kinda isolated in my DevOps niche, where they chase any new shiny tech ---------------------------------------------------------------------- AnApexBread · 2023-10-13 12:59 I run a few things in VMs. * I'm a malware analyst by trade and practice so I run 2 dedicated VMs for analyzing malware. One is a REMNUX VM for doing static analysis and the other is a Windows VM for doing dynamic analysis. I know Remnux is on KASM as well but static analysis can take days to weeks to complete so I need something that is persistent. * I run my kids computers as windows VMs from my homelab. They have a RPI3B+ as the thin client and it RDPs into the Windows VM for them to do school work. * I teach cyber security at a local highschool for the Cyber Patriot program. So I host a lot of the training images in my homelab for the students to access and learn in. Cyber Patriot teaches them things like system administration and basic system hardening. It's all scored to so they need actual machines with the scoring engine set up to practice with. * I also teach cyber security at work (not running on my home lab). I build a lot of virtualized networks with VMs running in them to simulate real world networks. That way the security teams can pratice attacking and defending actual networks. * We also build a lot of niche cyber security tools, and while you can build in Docker its frequently easier to just build it in a VM and then dockerize it later. And that's just from my cybersecurity hat. My work is full VDI so all our computers are actually just thin clients which pull a VM. ---------------------------------------------------------------------- lvlint67 · 2023-10-13 12:59 /shrug shiny shot is cool... But when you're doing real administration work you rapidly find that a lot of what you need to do just isn't going to work well in a container. Going container first is a fine strategy. We run Kubernetes in VMs... We also have a ton of vendor crapware that we need to support that won't run in containerized environments. Your response would raise red flags in any interview, so you'd better come prepared with a solution for products that don't run in containers. ---------------------------------------------------------------------- lvlint67 · 2023-10-13 13:01 I mean there are definitely people here that can't install software unless it has included dockerfiles... Never mind that they could open the dockerfile and manually run the commands they find... But... /Shrug ---------------------------------------------------------------------- Preisschild · 2023-10-13 13:03 To be fair every dev should do devops. The guy developing the software knows best how to deploy it. Thats the whole concept. ---------------------------------------------------------------------- lvlint67 · 2023-10-13 13:04 You're happy in frontend? **No one** is happy in frontend. Everyone wants out. ---------------------------------------------------------------------- AnApexBread · 2023-10-13 13:04 >What homelab needs a VDI though? I do. >Everyone here is hostin jellyfin, nextcloud, etc. Applications. No, not everyone. And some people who host applications also host VDIs. Again, just because that's all YOU choose to use your homelab for does not mean that's all a Homelab can be used for. >In my experience And that's your problem. You're baseing everything off your experience, and even when people tell you that they have needs that aren't the same as yours you reject that because it's not what you've experienced. Virtualization and Containerization both have their uses and its important to understand both. ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 13:10 Hmm, do you have examples of these kind of things? Kind of curious what kind of "crapware" you have to deal with that won't run in containers. A container is literally just a small linux machine. Do you mean you're running a lot of vendor software that calls out to things like systemd? FWIW i do plenty of infra/ops interviews, doing rounds right now actually, and people in my line of work on both sides are not asking about running stuff outside of containers ---------------------------------------------------------------------- eightcupsofcoffee · 2023-10-13 13:15 well duh, but do you have a use case other than VDI's that containers don't solve? I guess that was my point, and I agree with yours that virtualization and containerization both have their uses and its important to understand both. My point is that you shouldn't be using hypervisors to manage your farm of hosted virtual machines for the vast majority of use cases in 2023. I haven't seen convincing use cases yet otherwise outside of VDI ---------------------------------------------------------------------- jasonweiser · 2023-10-13 13:30 That box isn't nearly big enough. ---------------------------------------------------------------------- Mr_Lifewater · 2023-10-13 13:41 Yep those are all super good uses for VMs. Sometimes I forget how useful they are just because of the industry I’m now in ---------------------------------------------------------------------- PuckSR · 2023-10-13 13:44 You realize I picked that specific example because I was joking, right? ---------------------------------------------------------------------- lvlint67 · 2023-10-13 14:23 Many enterprise grade solutions will not run in containers. Some that might .. won't be supported by the vendor if you put it in a container. Some companies can get away with not relying on the older enterprise stuff and can develop any in-house functionality for containers. Other companies, for various reasons, are going to be stuck supporting crap that won't run in containers. The admins in those situations need to see the situation for what it is and make appropriate accommodations. ---------------------------------------------------------------------- ratcodes · 2023-10-13 14:46 i like it :) ---------------------------------------------------------------------- ratcodes · 2023-10-13 14:47 i agree... i don't know why i struggle to find it interesting enough to get more than just basic skill in it, though ---------------------------------------------------------------------- Janewaykicksass · 2023-10-13 15:36 I'm one of those weird nerds. I have a 25U rack, Cisco router, switch, and R720 at home. I run Ubuntu with Docker on Proxmox. I have a Samba/AD experiment I'm working on. Stuff that I read about and want to try but has no use in my current enterprise. My home lab put me over the top and got me a job. ---------------------------------------------------------------------- afinita · 2023-10-13 16:21 Same. No IT experience, no degree and no certs, but discussed my Hyper-V cluster and various applications I’ve installed and clustered for 30 minutes and got a position. A few years later and now I’m the senior! ---------------------------------------------------------------------- spanish4dummies · 2023-10-13 16:23 i've succeeded in installing guacamole before but then i'm like "what do i do?" I need to read through the internets to figure out how to get the proxies running ---------------------------------------------------------------------- AnApexBread · 2023-10-13 17:48 I'm not against docker by any means. I use it heavily in my homelab for a lot of the application specific stuff like hosting my blog, my owncloud, omada controller, and more. I just get irritated when people claim that there's no reason to use VMs because we have docker. It's like saying no one should eat apples because we have bananas. They're both fruit sure, but very different. ---------------------------------------------------------------------- vivkkrishnan2005 · 2023-10-13 17:54 I always ask for this as part of interviews. This tells me how much the person is ready to experiment. Can swing both ways, but usually I take it always positively. ---------------------------------------------------------------------- Mr_Lifewater · 2023-10-13 17:56 Yep I agree, its more of a "use the right tool for the job" situation. They aren't evolutions but tools for different situations ---------------------------------------------------------------------- Mr_Lifewater · 2023-10-13 18:04 Kinda makes sense to me, if your looking add someone to your company and they've passed the initial screening, its generally a good sign if they have a homelab and can speak about it, or even whiteboard the setup. While it might not be directly related to the role, its a good sign when you have technical people doing technical things outside of work ---------------------------------------------------------------------- Famous-Atmosphere816 · 2023-10-13 18:07 Most "home labs" that I have seen are very small. They are generally a single machine with a multi-core CPU running bare metal or using Proxmox. I have seen a few Proxmox setups that are interesting, but they are generally just for play. What you are doing is uncommon based on what I have seen in other home labs. My friend has a home lab server with an 8 core/16 thread CPU and triple SATA SSDs. He is running Proxmox on that and he is definitely not comfortable on the CLI which ends up with him calling me quite often. I don't know how much RAM he has. Another home lab that I saw, which is now in a commercial building, went the direction of cyber-currency mining. The last time I saw that setup it had 15 nodes with each node running 12 GPUs (It was pretty big). I got involved there because their lab kept crashing every night (someone gave the guy my name and phone number). I found the trouble and stabilized it for him. The trouble was a network card that would intermittently throw garbage into the network and force all machines in the cluster to go offline. I threw an enterprise switch in to replace the dumb switch they were using and set it to disable any port that received bad data. That caused any machine with intermittent NIC failure to go "offlline" without affecting the cluster's communications. It made diagnosing intermittent NIC failures a trivial problem after that. My home lab (I do have servers in data centers as well) is a cluster of three 1U nodes with 192 cores (384 threads), 36 NVMe drives (12 x 3) each in a zraid setup and 1TB of RAM/node (I can put 4TB RAM/node if needed). Each node has dual 800 watt power supplies. I can add an additional 12 drives, (4 x 3) internally to each node. I will be doing this soon in order to add internal backup and staging space in the cluster. Each node also runs a GlusterFS cluster which is RAIDed across the three nodes. After I built the cluster, I priced the hardware out on Dell's website. I built my home lab for under $70,000. Dell's price was just shy of half a million dollars for comparably speced equipment. Clearly building servers yourself offers some substantial savings. All nodes run Ubuntu Server with LXD and are strickly CLI. I also have an AI node running 128 threads on the CPU, 1TB RAM, 32768 Cuda cores and 1024 Tensor Cores, 48GB VRAM and a 1650 watt PSU. That machine is NOT data center ready as I had to Frankenstein the chassis (metal cutting and fabrication was required). I run modified versions of GTP-NeoX and StableDiffusion in containers on the AI node. The next iteration of that machine will most likely be based on nVidia A100 cards. They have slightly less Cuda and Tensor cores. The difference is that each card has 100GB of RAM versus the 24GB that each of my current cards have while dropping the RT cores and video capabilities. Testing shows that slightly less cores, the higher RAM and removal of unneeded circuitry results in a 300%+ boost in performance while using significantly less electricity. Apparently AI workloads are constrained not only by available cores but also available RAM and the presence of unneeded video and RT circuitry. My development workstation is a 16 core/32 thread machine with 64 GB RAM, NVMe main drives and dual large HDDs for data storage as well as quad 27" IPS displays. It runs Ubuntu server with a desktop installed. It also runs both KVM and LXD virtualization. I can run massive workloads and parallel workloads in my home cluster. LXD containerization provides me with essentially bare metal speed on my virtualizations. The only roadblock that I have run into with my home lab is saturation of the LAN. The next upgrade is to go to infiniband network fabric. There are some good deals on ebay for infiniband switches and NICs that have been pulled from data centers. With the exception of my development machine, all nodes are in a server rack. My home lab is for testing and development of technology that will make it into my data center machines in the future. I do realize that, since I own a technology company, my home lab is not your average setup. ---------------------------------------------------------------------- Famous-Atmosphere816 · 2023-10-13 18:16 Since long command lines are often run more than once, I simply put them into a shell script and run the script. It creates a nice audit trail, removes much of the failures that come from fat fingering the CLI and lets you create repeatable results for tasks that are done more than once. As for long folder locations, the easiest way to do that without mistakes is to type the first couple characters of each folder and then hit tab to let it auto complete. If you make a mistake in the folder path, you will see it immediately. ---------------------------------------------------------------------- originalodz · 2023-10-13 18:26 I get questions about it all the time. When they ask about it and seem interested I know we're past first base. I interview mostly for *Ops positions though so it's rather relevant to show my interest. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:11 Hmm seems I need to read even further into these aspects of kubernetes, I might have missed stuff.... hmmmm I appreciate you sharing the insights! Also, not sure what you mean by OSDs as an abbreviation, not really sticking out as self-indicative this moment 🤔 As for if Proxmox is the objective "right answer", is there ever such a thing in IT? ;P ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:12 Well I appreciate trying to dig up more info, but the spec sheet was of interest to me so I could see if it has any hot-swap bays, on-board SAS/SATA/other related connectors, how much PCIe slots (and their parameters), etc, to speculate on possible appropriate options. :P ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:14 > I always asked about their home network/homelab From a percentage perspective, how many of them became more excited when you asked about it? > I hired one guy on the spot because he pulled out his phone and showed me a homemade android app to control his home automations. SICK! (was this relevant to the position?) > Of course he quit within 6 months because he was way overqualified, you live and you learn. OOF, regrets? D: Did he give notice or just 0day? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:16 🤔 ---------------------------------------------------------------------- Alleexx_ · 2023-10-13 19:22 Actually I am just waiting for the question, but I never got to mention my Homelab 👀 ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:23 > I don't fill out detailed applications anymore, if Linked-In or my resume doc isn't adequate for the majority of it, then the company has broken processes. Companies that want you to retype your job history into their form don't use that data in the future, so why bother? Agreed. Time is already so precious for me, when I see stuff like that I just go to the next job listing. I laugh plenty thinking about the stupid HR folks who think that kind of a workflow actually is worthwhile to any degree. Also, nice for your insights on the topic! Thanks! :D ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:24 Agreed. ---------------------------------------------------------------------- Alleexx_ · 2023-10-13 19:24 Same here. Just started with GPU passthrough vms and I kind of love it, and count it towards my Homelab? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:25 > Samba/AD experiment I'm working on Oh? I can tell you right now Samba AD DCs are production ready and have been since about... uhh Nov 2012 if memory serves me correctly. (depending on which features your environment actually needs) > My home lab put me over the top and got me a job. Yay! ---------------------------------------------------------------------- Croatwink · 2023-10-13 19:28 I interviewed maybe 6 people, during a period when we were looking for a backup for me so I was allowed to participate with my manager. I also rewrote the ad because my hapless manager wanted to use a template from HR that was completely irrelevant. If you want something done right you gotta do it yourself, this was my backup after all. I'd say it was 50/50, and that was also about the percentage of good applicants vs. useless ones. If they said they had never done anything interesting with their home network, then they usually also didn't display any other good qualities. So it went hand in hand. No it was not relevant at all, it was just a sysadmin with focus on Linux, Unix. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:34 What ways have you seen it swing not-positively? ---------------------------------------------------------------------- BloodyIron · 2023-10-13 19:37 Ahh thanks for sharing! :) ---------------------------------------------------------------------- Morkai · 2023-10-13 20:08 Understood, I'll see what I can dig up. ---------------------------------------------------------------------- BloodyIron · 2023-10-13 20:10 👍 ---------------------------------------------------------------------- hannsr · 2023-10-13 21:06 I got my job entirely based on my homelab experience. Ok and I knew a guy. But still. My resume was 100% not server/hosting related, but I talked to my boss about my lab for an hour, what I do there, and I got the job. ---------------------------------------------------------------------- vivkkrishnan2005 · 2023-10-13 22:00 Seen it several times at ex-company. Was HOD for IT. Most people reply with running Linux and ESXi at home. For me the main key point is the initiative. I deep dive into queries and ask abstract questions and twisters which makes them question their answers - if they are bluffing. Those who come clean are usually the ones to take, and have gone to much higher achievements. These are your first responders and can be trusted to do the job. Those who dont come clean and still try to pull a fast one - one chap was full on bluff - could not do anything for political reasons and one was bureaucratic in approach, left as soon as they got another job elsewhere. These are those who will rise not by skill but by virtue of experience on paper. And usually should not be taken for IT. ---------------------------------------------------------------------- BloodyIron · 2023-10-14 01:10 Yikes! Yeah good insights indeed. Thanks for sharing! :) ---------------------------------------------------------------------- sloaam · 2023-10-14 03:03 Lack of /s, so he might not have. ---------------------------------------------------------------------- sloaam · 2023-10-14 03:06 Oh my zsh on my laptop makes me very upset when I ssh into servers that I would never install a self updating git package like Omz... Because I really love just bashing the tab key until I see what I want. ---------------------------------------------------------------------- sloaam · 2023-10-14 03:08 Or C-x, C-e into a buffer (in vim obvi) to copy and paste commands into, edit as needed quickly, and then move on without having a random script hanging around. For one offs and installs of course. ---------------------------------------------------------------------- sloaam · 2023-10-14 03:14 This is why I work in healthcare lol ---------------------------------------------------------------------- PuckSR · 2023-10-14 03:31 I wrote it, so I think I can speak fairly definitively on the topic Chmod 777 is literally a known joke of something you shouldn't be doing ---------------------------------------------------------------------- spanish4dummies · 2023-10-15 01:28 >Ok and I knew a guy. homelab and networking ---------------------------------------------------------------------- spanish4dummies · 2023-10-15 01:34 I feel like I'm in the same realm as vegans and crossfitters: bringing up my homelab in the most scant points of conversation with people, haha ---------------------------------------------------------------------- oxamide96 · 2023-10-15 20:21 I work as a software engineer and never used a hypervisor. I exaggerated when I said that I would blank, but I've never had to use it because the modern cloud takes care of all of that. The fact that the original commenter expected that was strange to me. ---------------------------------------------------------------------- oxamide96 · 2023-10-15 20:22 Yeah I mean I never used them, and I am not only a homelabber, I work as a software engineer. ---------------------------------------------------------------------- BloodyIron · 2023-10-15 20:41 You run everything bare metal? Why not run a hypervisor? ---------------------------------------------------------------------- Morkai · 2023-10-15 21:52 I believe this should be the correct one, just like I mentioned, I currently have the Xeon 1265L v3 installed (that's what it came with when I bought it off a friend) https://www.hpe.com/psnow/doc/c04567801?jumpid=in_lit-psnow-red ---------------------------------------------------------------------- BloodyIron · 2023-10-16 00:03 Use this as a head unit for a TrueNAS system. Get a SAS2 SAS HBA card and connect to a SAS2 disk shelf (depending on what parameters you're looking to do, but don't use Z1). IMO That's the best application for this system.