[HN Gopher] My Overkill Home Network
___________________________________________________________________
My Overkill Home Network
Author : monstermunch
Score : 531 points
Date : 2023-08-10 21:05 UTC (1 days ago)
(HTM) web link (blog.networkprofile.org)
(TXT) w3m dump (blog.networkprofile.org)
| femto wrote:
| I used to have a home network, until lightning hit nearby.
|
| It came up the power connection and the copper phone connection.
| The modem/router/switch fried, then it spread out though the
| twisted pair Ethernet and fried every interface connected to the
| network.
|
| These days I use WiFi unless I really need low latency / high
| reliability in a particular location. Optical would be nice, but
| WiFi is the poor man's lightning safe network.
| monstermunch wrote:
| Yep, first thing I did was cut the phone line and Coax off the
| house!
| jamespo wrote:
| The APC 850 UPS has ports to surge protect the data
| connectivity.
| jacoblambda wrote:
| Yeah this is why I like fiber optic cables. Lightning can't
| surge the modem if there's no electrical connection.
|
| And then standard power isolation and surge protection will
| handle all the power connections.
|
| I've yet to loose equipment to lightning from the power cables
| but I literally can't count the amount of equipment that has
| been killed by lightning striking a copper data line and then
| surging through my network.
| pottertheotter wrote:
| All I can get is cable internet so this worries me.
|
| What's the best way to implement power isolation and surge
| protection?
| Dah00n wrote:
| If you want to isolate the internet connection with fiber
| you can use an ethernet to fiber box.
|
| Something as simple and cheap as this would do:
|
| https://www.tp-link.com/us/business-
| networking/accessory/mc2...
| theshrike79 wrote:
| +1 for the TP-link, I've got one running fiber to my AV
| equipment :)
|
| It was a lot easier to run a pair of fiber-optic cables
| there than it would've been to do the same with cat6.
| Also I got to play around with fiber for the first time.
|
| Never thought of isolating my modem from the rest of my
| gear though, I need to look into that.
| Dah00n wrote:
| Side effect of introducing less noise into AV equipment
| is a nice bonus!
| bombcar wrote:
| You can never get perfect isolation from lightning (a
| direct hit will almost certainly take _something_ out) but
| you can do worse than making sure you have an excellent
| grounding rod setup and a whole-home surge protector, which
| you monitor.
|
| And then you can put surge equipment on any other incoming
| wires.
| kibibyte wrote:
| There are surge protectors that can protect the coax cables
| used for cable internet. Here's one that I use:
| https://www.amazon.com/Tripp-Lite-Right-Angle-Protection-
| TLP....
| teddyh wrote:
| Couldn't lightning just as well fry everything connected to
| power? In that case, WiFi will not save you.
| femto wrote:
| When lightning strikes, the phone network and power network
| are often driven to different electric potentials, meaning a
| current will flow between them. When that happens equipment
| that is connected to both the phone line and the power will
| act as an (expensive) fuse.
|
| With only one connection (power or phone) the whole device
| tends swing at the same potential. Damage can occur if it
| arcs to ground, but that's less likely as in a device the
| isolation to ground is typically higher than between the
| power and data parts of a circuit. (Extra bad news if the arc
| to ground happens via a person.)
|
| Incidentally, if a lightning strike is nearby a typical
| consumer surge protector will improve the odds but probably
| won't prevent damage.
| [deleted]
| RulerOf wrote:
| I had an ISA dialup modem that got hit by lightning. Loud boom
| then my DUN connection hung up.
|
| When it came back, that modem still worked, but it was stuff
| "off the hook" forever. I had to unplug/replug it every time I
| wanted to dial out, and the phone line was permanently busy.
| fastball wrote:
| Is it not possible to surge protect the copper phone line?
| NavinF wrote:
| It's pretty easy, but also unnecessary when everyone uses
| VoIP and fiber is cheaper per ft than copper
| harry8 wrote:
| I couldn't find it, so many details. What's the power draw on all
| that?
|
| A kilowatt?
| asynchronous wrote:
| Way, way too much is the answer. That amount of storage alone
| has got to kill his electricity bill.
| princeb wrote:
| he said solar, which is pretty nice if (1) he is generating
| excess solar in the day (2) he cannot sell it back to the
| grid (3) it's not cost effective to store in batteries for
| night use (4) there's nothing else to spend the energy on.
|
| otherwise, i can imagine it'll still be cheaper to not run
| things than to run things. but it's a hobby, people spend
| energy and resources on more frivolous things, and those
| frivolous things make us human.
| ivolimmen wrote:
| I find this really disturbing. On the one hand this looks really
| nerdy and cool but then I see the immense amount of hardware that
| also relies on power... so much power. This is completely
| unnecessary but then I also see the pictures of an F150.. so yeah
| it makes sense. I have to categorize this as a (un)typical
| American person that does not care about the environment.
|
| Do like the fact that he runs a RIPE probe though.
| matheusmoreira wrote:
| You do realize you're posting this self-righteous comment from
| a computer that was in all likelihood manufactured in China,
| right? I wonder how many tons of coal were burned to
| manufacture it. I wonder how many western developed nation
| comforts you're unwilling to give up while you "categorize"
| people on the internet based on the constructive activities
| they choose to engage in with their own time and money.
| amadsen wrote:
| Sigh. This is a guy who is clearly passionate about his hobby
| and was excited to share it with us. Leave it to hn to shit all
| over it.
|
| Some people fly to Africa and shoot elephants for fun - as far
| as hobbies go, this is fairly benign. If you care about the
| environment, I mean actually care, then you should recognize
| that policing how many computers someone have in their closet
| is counterproductive. I don't buy the "every little bit helps"
| argument because there is always an opportunity cost. Attention
| spans are limited and focusing on this non-issue does nothing
| but draw attention away from things that actually matter like
| meat consumption and fossil fuels.
| true_religion wrote:
| My entire region runs on hydro, so even if I had a home lab x10
| the size of this it wouldn't have the same environmental impact
| as someone who ran a laptop with power from a coal plant.
|
| You can't tell from a blog post if what he's doing is good or
| bad for the environment. And further more, its unfair to tell
| people that they shouldn't improve their niche skills when
| those same skills are what they use for employment.
| bliteben wrote:
| Yeah pics did make it look like he could use a smaller truck
| kriops wrote:
| I find your comment to be at the absolute peak of sanctimony.
| Who are you to decide if it is necessary or not? Furthermore,
| why are you justified in negatively stereotyping Americans - a
| diverse group of individuals?
| rb666 wrote:
| How is it an act? Caring about the climate is clearly morally
| superior than not doing so.
| refurb wrote:
| How do you know OP cares about the environment? Because
| they criticize others?
|
| I have no doubt we could examine OP's life and find plenty
| of "wasteful" use of energy but no doubt OP would say
| "that's different". It's always "the other guy" in these
| cases.
|
| Add on top it's a lazy comment that adds nothing to the
| conversation.
| drexlspivey wrote:
| The "using electricity is bad" meme must be one of the
| stupidest ones in the last few years
| maleldil wrote:
| Why? IMO we should all avoid waste, whether that's
| plastic or energy.
| Qwertious wrote:
| >Why? IMO we should all avoid waste, whether that's
| plastic or energy.
|
| Allow me to respond with a slight tangent:
|
| Suppose you care about reducing plastic in the ocean. So
| you try to reduce your usage of plastic straws. You spend
| a whole lot of time convincing everyone you know to buy
| bamboo straws or such.
|
| Plastic straws make up less than 0.1% of the plastic in
| the ocean. Meanwhile, fishing nets make up ~40%. Plastic
| straws are a _hard_ problem to solve, and solving them
| accomplishes less than 1 /400th of the fishing net
| problem.
|
| And yet, we hear a lot about plastic straws but nothing
| about fishing nets. Why?
|
| Because it's the more consumer-facing solution. It's
| vital that people focus on the solution that _solves the
| problem_ , rather than wthe one that is most easily
| pointed to.
|
| Should we do both? In theory, sure. But in practice
| people have finite time, and you have 30 seconds max
| before they start tuning you out, so you're better off
| giving your elevator pitch on fishing nets.
| drexlspivey wrote:
| Then you must hate that almost all of sun's energy
| hitting earth is wasted
| maleldil wrote:
| This is such a bad faith comment. The sun isn't causing
| climate change; we are.
|
| Do you disagree that humans need to minimise waste in
| order to fight climate change?
| drexlspivey wrote:
| Using energy doesn't cause climate change. Producing
| energy from fossil fuels causes climate change. Humans
| don't need to minimise energy usage to fight climate
| change, instead they need to produce energy from non
| fossil sources. I wish we produced 10x energy that we do
| now but from renewable sources.
| globular-toast wrote:
| You're right, in the most meaningless sense of the word.
| I too wish we had 10x the energy from renewable sources,
| but we don't and won't for the foreseeable future. What
| we actually can do is get maybe 1/10 of our energy from
| renewable sources and then, to be sustainable, we need to
| use 1/10 of the energy we currently use.
| a-user-you-like wrote:
| How do you know the man's power is not cleanly generated?
| [deleted]
| antoinec wrote:
| Looks like he is powering at least part of his
| installation through solar:
| https://blog.networkprofile.org/17kw-enphase-solar-
| install/
| uoaei wrote:
| What is the CO2e ROI on producing and installing solar,
| including all the transportation and installation
| equipment being used?
|
| When is it net-negative compared to pulling energy from
| the grid?
| snoman wrote:
| Can you honestly answer that question for literally
| anything you've ever bought, in your life?
| uoaei wrote:
| Would any answer to your question mean mine is not one
| worth asking, or that it's not worth estimating those
| quantities to include in calculations of CO2e-opportunity
| cost?
| antoinec wrote:
| Good question, not sure how I would know the answer
| though. You probably should ask the guy. Given how he
| seems to track everything, I wouldn't be surprised if he
| came up with a way of measuring that as well.
| rimliu wrote:
| The question was not addressed to me, but I have some tongue-
| in-cheek answers: > Who are you to decide
| if it is necessary or not?
|
| Inhabitant of the Earth. The one affected by climate change.
| > Why are you justified in negatively stereotyping Americans
|
| Tell me, what is the best-selling truck in US for 46 years in
| a row?
| refurb wrote:
| It's hilarious how sanctimonious moralizing has gone from
| things like pornography, premarital sex and smoking
| marijuana to now using too much energy.
|
| The people who complained about those issues thought they
| were "on the right side" as well.
|
| But in the end it's all just busybodies trying to force
| other to live their life the way they think it should be
| lived.
| checkyoursudo wrote:
| That particular comment was "the absolute peak of
| sanctimony"? Like, the most sanctimonious comment you have
| _ever_ read on the internet?
| snoman wrote:
| Probably more like a local maxima.
| kriops wrote:
| Yes. This is a prime example of what people refer to when
| they say that someone is being an "asshole". The reason
| being that the OP is being incredibly condescending while
| simultaneously lacking any trace of empathy and knowledge.
|
| I've seen stuff that or more vile etc. of course, but as
| far as sanctimoniousness goes, this is the worst I've seen
| in my several decades on the internet.
| snerbles wrote:
| The satisfaction from serving up self-righteous condemnation
| is one hell of a drug.
| Gibbon1 wrote:
| Giving attitude from 50 degrees north latitude.
| Toutouxc wrote:
| A diverse group of individuals who consume 12 MWh of power
| per capita, while it's around 6 or 7 MWh for modern European
| countries. It's not even a stereotype, just statistics. (got
| my data in two minutes from Wikipedia, so maybe not up-to-
| date)
| thsksbd wrote:
| "who consume 12 MWh of power per capita, while it's around
| 6 or 7 MWh for modern European countries."
|
| Lol, you write that as if the Europeans were consuming half
| as much because they are morally enlightened.
|
| Europeans use 6Mwh/person because thats as much as their
| emasculated societies can steal from Africa. May I remind
| you they didn't leave Africa very willingly in the 50s-70s.
| But they kept on stealing anyway!
|
| Thats right. Steal. Libya, Algeria, Niger. The French
| convertible Franc. The French senate bemoaning _just this
| week_ that Macron "lost" Africa. Lost Africa? Africa is
| not France's to lose!
|
| Unlike, say, Japan or Korea that have more than just a
| token few LNG terminals and pay for their gas with monitors
| and great cars.
| ryan-c wrote:
| I wonder how much of the difference is air conditioning?
| Karrot_Kream wrote:
| I don't really think 2x the power usage comes from AC.
| The US has plenty of areas that only need a bit of AC for
| only part of the year.
| ryan-c wrote:
| It's definitely not all of the difference, but it's gotta
| be some.
| midasuni wrote:
| Presumably those areas don't need heating
| quantumfissure wrote:
| Please don't make assumptions on something you don't know
| anything about.
|
| Even many northern US states (around the same Latitude as
| Madrid, north), on average, are 30-35+ every day during
| the summer and 0C to -10C in the winter. Many Canadians
| have air con.
|
| So yes, heating and air con is a requirement in many
| areas. I live in a northern state, and we regularly have
| full weeks that are 38-40C.
| ryan-c wrote:
| That would be an incorrect assumption.
| Gareth321 wrote:
| In many U.S. states, their primary energy generation is
| nuclear. This is extremely environmentally friendly, meaning no
| issues with OP using a bunch of power.
|
| We really need to stop demonising the use of energy. Energy is
| _amazing._ It is the reason we have seen to much prosperity in
| so little time. It 's the backbone of our economies. Of
| medicine and the internet and EVs and well insulated homes. We
| should be doing everything we can to make energy cheaper,
| enabling more people to use more energy. Few things have been
| as damaging for climate activism than the war on energy. It's
| unnecessary and misses the point of the entire movement: saving
| the environment. Focus on cleaner sources of energy rather than
| demonising people using energy.
| b3orn wrote:
| I looked it up, 5 US states get the majority of their
| electricity from nuclear power, 4 from wind, 3 from hydro.
| The other states use coal, natural gas and petroleum.
|
| Source: https://www.nei.org/resources/statistics/state-
| electricity-g...
| ourmandave wrote:
| Post apocalypse, that Biomass & Other column is going to
| mean something completely different.
| Karrot_Kream wrote:
| It's not about the energy for me. A lot of these posts make
| networking and homelabs seem like a dark art that only folks
| with tons of space, capital, and time can get into.
| Networking is anything but, it's about sending a packet of
| data over a wire! But when we glorify these elaborate,
| expensive setups, we not only lose focus on what the goal of
| a network should be (to satisfy needs for its users), we also
| discourage newcomers from tinkering with networks which to me
| is the bigger crime.
| gwbrooks wrote:
| Or -- hear me out -- you entice newcomers who want a big
| ol' rack of boxes and wires.
|
| Like me.
| Karrot_Kream wrote:
| I guess that wouldn't have enticed me at all. The
| minimalism of networking is what interested me in it in
| the first place decades ago as a kid.
| pprotas wrote:
| Snide categorizations aside, there is a point somewhere in your
| comment. It is a running joke in the /r/homelab community that
| their servers use an unreasonable amount of power that they
| have to hide from their wives.
|
| It really IS unnecessary, as you can tell by the author of the
| article referring to their home network as "overkill".
| fjfuvucucuc wrote:
| [dead]
| cjdell wrote:
| I also have mixed feelings. I stopped home labing because of
| the energy costs. I now have everything I need (HomeAssistant /
| NAS / Router) running on an old Dell Precision laptop. Laptops
| make great servers because they are optimised for low power
| consumption + you get a free console and UPS built-in!
|
| I'm tempted by the new Alder Lake N100 mini PCs that are
| available on Chinese websites. But I need to wait until the
| laptop dies before I can justify it. It's not just about power
| consumption but the (hidden) cost of manufacturing. Make use of
| the stuff you already have before buying new stuff.
|
| At least he had a big solar array...
| maxmalkav wrote:
| I am a big fan of thin clients as servers. It is true you
| lose the "built-in console and UPS", but they usually come in
| very nice and compact form factor, many are fanless and
| consume 10~15W tops while being acceptably capable (in
| general much more than a RPi but with much better memory and
| storage specs)
| abwizz wrote:
| > Laptops make great servers because they are optimised for
| low power consumption + you get a free console and UPS built-
| in
|
| > Make use of the stuff you already have before buying new
| stuff.
|
| this is the way!
| bliteben wrote:
| I would have disagreed on heat, but would love to hear what
| laptops that isn't an issue on.
| thfuran wrote:
| What do you mean?
| bliteben wrote:
| Many laptops have bad thermals in general under load and
| often even worse with case closed.
| abwizz wrote:
| true but mostly irrelevant as there rarely is any load to
| speak of.
|
| if you need a heavy duty machine, use it, but most of the
| time you don't, that's the point.
|
| (for comparison; skylake laptop uses about 20W under max
| load, my 12-gen desktop uses 40W while idle)
| norman784 wrote:
| Maybe you could consider an old framework motherboards? I
| personally don't own one, but is tempting if you own a
| framework laptop or are planning on buy one, it seems to be
| one of the most sustainable options out there.
| Borg3 wrote:
| Laptops are great servers? Hmm I beg to differ. They are very
| compressed, so they heat up a lot.
|
| You need low power server? Go for Atom like CPU with good
| case. My home server running D2550 and 2x 1TB HDD is less
| than 40W. PSU is cold.
| varjag wrote:
| Run the same load you do on your Atom on a laptop and it'll
| be cold as Thatcher gaze.
| abwizz wrote:
| typical laptop goes below 10W while serving a gbps of samba
| ivolimmen wrote:
| I run a server at home. It's a PINE64... uses 10w or
| something. My NAS uses more but I only turn that one when I
| need to backup stuff.
| cjdell wrote:
| Well it depends what you're doing with it. In my case my
| CPU is averaging about 1% with the odd burst here and
| there. As a precaution I propped it up to allow increased
| airflow underneath.
|
| You do have a point in that certain devices can suffer from
| overcharging and bulging batteries. This issue is largely
| addressed in newer devices. I can see that the BMS in my
| laptop server is holding at 90% charge. Definitely worth
| checking if you plan to use an old phone/laptop/tablet as a
| 24/7 device.
| vladvasiliu wrote:
| My main problem with labbing is the noise (I live in an
| apartment). His servers are mostly small PCs, so I think the
| most power draw is from the disk drives. If you need the
| storage, say for the cameras and whatnot, I'm not sure that
| running a laptop instead would save _that_ much power.
| Cthulhu_ wrote:
| I dunno, a lot of people on here use more power without
| noticing it, because it's on whatever cloud provider they use.
| Every time I push some commits, an array of servers spins up in
| an AWS datacenter somewhere to run all of my verifications and
| do a build.
|
| Second, this person seems to have a very large house anyway. My
| house manages with a modem, a wifi router and two IP cameras,
| but that can just scale up linearly.
|
| When I grow up I want a switch rack like this in a basement or
| closet, wired network everywhere and PoE security cameras.
| Bigger house also means I can get an array of solar panels and
| become net positive.
| Karrot_Kream wrote:
| > I dunno, a lot of people on here use more power without
| noticing it, because it's on whatever cloud provider they
| use. Every time I push some commits, an array of servers
| spins up in an AWS datacenter somewhere to run all of my
| verifications and do a build.
|
| Most AWS (and generally cloud) instances are already
| virtualized from bigger machines. Power, heating, and cooling
| in these large racks is running much more efficiently than
| running these things in your home on dedicated compute that
| is only being used for your purposes. Even more efficient is
| using something like serverless compute or something like
| Cloudflare workers. While cloud compute probably is not
| running at low idle power, it's probably maximizing it's
| power utilization as much as possible.
|
| > Bigger house also means I can get an array of solar panels
| and become net positive.
|
| There's a lot of hidden costs associated with becoming "net
| positive", everything from adding more peak load during
| inclement events to issues associated with grid fragility.
|
| Look I don't think it's good form to roast OP on what's
| really standard behavior on the internet: showing off your
| cool digs. But I also think we should be intentional about
| what it is: showing off your cool digs.
| SanderNL wrote:
| I find it very interesting how awareness of climate change (and
| let's not forget: the price of energy) has had so much impact
| on my and others' judgement in a relatively short time.
|
| 5-10 years ago I would think this is perfectly fine. I believe
| I was not alone in this, but maybe I was. The energy would have
| cost pennies too and why whine about it?
|
| Now I think it's exorbitant as well, for me. The F150 etc, it's
| telling a different story now.
|
| I do think it's kind of below the belt to snipe any single
| culture for this. It's too easy to target Americans. I don't
| know about this guy's life. Maybe he has a ranch with 20 square
| miles of renewables and I don't care actually. It's just
| personal reflection.
| [deleted]
| alex3305 wrote:
| > 5-10 years ago I would think this is perfectly fine. I
| believe I was not alone in this, but maybe I was. The energy
| would have cost pennies too and why whine about it?
|
| When I moved into my own home (8 years ago) I brought my
| 'homeserver' with me. Which was just a simple i5-2600 build
| with some shucked drives in it. I never thought about
| electricity prices when I lived with my parents. But that
| changed rather fast. With the server gobbling up a constant
| 90W, I quickly realized that, even back than, it would cost
| me 15 euro's a month on electricity alone.
|
| I than proceeded to put a Pi next to it, that would listen to
| incoming Plex requests and would start up and shut down the
| server with WoL. That only reduced costs by about a third.
| The next couple of years I would move on to a NUC with a NAS
| that would only consume about 29W/h on average. Which was
| decent, but not great considering the poor performance of
| both machines (J4105 and J1800).
|
| Last month I have gone back to the DIY route. Now with a
| i5-13500. I'm still in the process of optimizing it for power
| efficiency. Although much more stressful than the prebuilts,
| I love the hunt for the last watt.
|
| Anyway, what I wanted to say is that I notice that family and
| friends don't really care about saving power in general. They
| mostly just pay for it and there's that. While my house runs
| 100% on electricity and I'm really proud if I can get
| 9kWh/day on average. Even when I see that (for example) the
| 8-bit guy uses 100kWh/day on average [1]. Which seems out of
| this world for me.
|
| 1. https://www.youtube.com/watch?v=bXd-aP06lug&t=45s
| bjornasm wrote:
| >I than proceeded to put a Pi next to it, that would listen
| to incoming Plex requests and would start up and shut down
| the server with WoL.
|
| How hard is this to configure? I have a server at home I
| use to run a database and computational heavy code, however
| I am the only user so realistically it is only in use 8
| hours a day and some weekends etc. However in the fear of
| forgetting to turn it on before I go to work (or if I
| suddenly find time to work while away) I find that I
| default to leave it on. Being able to control it would be
| fantastic.
| MayeulC wrote:
| If it's just that, you can have a Pi next to it and just
| ssh in to send a WoL command. Basically nothing to
| configure.
|
| You can make it simpler to use by making an alias in your
| shell, or a button on your phone (with one of the
| countless "ssh button" apps). Or even make a web page for
| it (some php or python that just calls the WoL function).
|
| OP describes a more transparent (and complex) setup where
| the Pi presumably acts as a reverse proxy. I'd be curious
| to know the exact setup too, one of the simplest ways
| would be to use wake on unicast:
| https://news.ycombinator.com/item?id=35627107
|
| Other ways include wrapping some scripts around socat,
| writing your own proxy, systemd socket activation, etc.
| djhn wrote:
| Hold on a second - how large of a household are we talking?
| House or apartment? What consumes all that power? Electric
| vehicle? Work from home?
|
| I'm environmentally conscious (0 cars, 0 pets) but I
| haven't spent any time measuring and optimising electricity
| usage.
|
| I've had a look at my own electricity usage every once in a
| while, now averaging 48 kWh/month (between 38 and 68) for a
| larger than average apartment for two people.
| Karrot_Kream wrote:
| 48 kWh / month is amazing! That's 4 kWh / day. Just
| cooking on our induction burner and instant pot can use
| up 1 kWh on a day with lots of meal prep.
| djhn wrote:
| Interesting. Maybe our brand new kitchen appliances _are_
| more energy efficient?
| miked85 wrote:
| Funny, just seems like a hobby to me.
| Karrot_Kream wrote:
| I do a lot of similar stuff in my home network with a _lot_
| less equipment. I have a trunk switch that sits behind my
| router, Ethernet running into each of my rooms (came with the
| house), and small point switches that sit in front of each
| patch. I have 3 APs in my house. I have a UPS which connects to
| our router, trunk switch, and a single AP that uses PoE which
| is good enough if we need Wifi when the power goes down. To
| much shock and horror, I just bought discount Cat6 cabling and
| didn 't bother buying a roll and crimping and making my cabling
| look nice (but I did test every point in my network
| exhaustively to make sure I don't have high amounts of latency
| or jitter anywhere.)
|
| The rest of the services that OP runs I run on a combination of
| a 12 year old SBC and a Raspberry Pi. Modern computers (read <
| 15 years old) when used as server appliances are very capable
| and can packet switch very quickly, even while running a kernel
| and a usermode. Doubt me? Run your own iperf tests and check
| where things are bottlenecking.
|
| My IP cameras are all Raspberry Pis that use the camera module
| and stream using RTSP back to the SBC. Instead of managing
| VLANs (I do offer a guest VLAN but my partner and friends found
| it a pain so it doesn't get much use _shrug_ ), I run Zerotier
| through the house. My IP cameras sit on a single Zerotier
| network. All other services I run internally use Zerotier and
| if I do accept external traffic, I expose a port on a VPS (like
| OP) and have some nftables rules which NAT the traffic through
| Zerotier. I've thought about getting a failover ISP (our area
| has 2), but our primary ISP offers us fantastic peering (I've
| exhaustively tested the bandwidth and jitter to the VPS I proxy
| traffic through) and good enough uptime for me to not bother.
|
| Like a lot of other hobbies, people can get very consumeristic.
| This is what a consumeristic version of homelabbing looks like.
| This shouldn't stop you or anyone else from building a humble
| homelab that can do almost all of what OPs home network does.
| The key is to test where your network has bottlenecks. I've
| iperf tested my network exhaustively and frequently look at
| kernel counters and other things to make sure that my network
| is performing up to spec, but that's a lot less fun than buying
| lots of cool networking gear. Much like you don't need an F-150
| to get around day-to-day, maybe even just a bike :)
| adrian_b wrote:
| > I do a lot of similar stuff in my home network with a lot
| less equipment.
|
| This is also my case. For Internet access, I use a router
| which has 4 2.5 GbE ports for the internal LAN, which was
| made several years ago with an Intel NUC and 4 USB Ethernet
| dongles, but which could be done today with a small and cheap
| fanless router using an "Alder Lake N" Intel N100 or N200
| CPU. There are also such small routers with more 2.5 GbE
| ports for the internal LAN, e.g. 6 or 8. That router
| implements a lot of network services, including DNS servers
| for my domain and my own e-mail server.
|
| I also have 5 servers that are interconnected by a 10 GbE
| subnetwork, but that is done without external switches. Each
| server has a dual-port 10 Gb/s NIC and they are
| interconnected directly, in a ring. The servers are normally
| powered off. Whenever needed, I power on the first of them
| through Wake-on-Ethernet. Through it, I can power on, also
| using Wake-on-Ethernet, as many of the others as needed.
| Karrot_Kream wrote:
| Huh the ring topology is a really cool idea. With the right
| routing rules it could open up some ports on my trunk
| switch, though I'd need to upgrade my NICs. Great idea for
| whenever I decide to retire the SBC though!
| adrian_b wrote:
| When all the servers of the ring are powered on, the
| packets are distributed on both links, so the throughput
| of the network doubles.
|
| When a part of the servers are powered off, the topology
| of the subnetwork becomes linear and the throughput
| becomes limited to 10 Gb/s, but with less active servers
| the traffic is also less.
|
| Greater performance can be obtained by using two Ethernet
| cards with dual 10 Gb/s ports in each computer. With 4
| ports per computer total interconnection can be achieved
| up to 5 nodes and above that each node can be connected
| to two intersecting rings, which is a topology very
| frequently used in HPC.
|
| An alternative is to use a single card with dual 25 Gb/s
| ports, per computer. Such cards are not much more
| expensive than the 10 Gb/s NICs, but the 25 Gb/s switches
| are very expensive, so a ring topology without switches
| is even more attractive.
| m463 wrote:
| He said in another post he has 17kw of solar, so...
| irusensei wrote:
| It's never about how green is the source of energy. It's
| about people doing things with their energy in which they
| don't approve. I mean... just read their xenophobic remarks
| about a giant country of diverse people.
|
| Don't bother appeasing climate hystericals.
| pietro72ohboy wrote:
| There's nothing hysterical about the OP's argument that the
| setup is overkill and power hungry. It's important to note
| here that OP is merely criticising the setup. It's their
| right to speak freely about it just as you have the right
| to call them a climate hysterical. Stop shaming people for
| speaking their minds.
|
| And my opinion is that the setup is indeed overkill. The
| author(s) literally admit so themselves.
| imiric wrote:
| If all this is running on renewable energy sources,
| what's the environmental harm exactly?
|
| The point that this is overkill and power hungry is
| completely irrelevant. OP's comment is coming from their
| high horse.
| kristjank wrote:
| I hope you're trolling. Power can be generated without an
| environmental impact. There are companies (and F150s) dumping
| kgbjillions more CO2 than this person's homelab is. Let people
| learn something and have some fun.
| ptman wrote:
| It cannot be generated without an environmental impact.
| Equipment has an impact and a limited life.
| maeln wrote:
| Power cannot be generated without an environmental impact.
| Even solar panel and wind turbine generate a non negligible
| amount of pollution when produced, and a non negligible
| amount of waste when they have to be decommissioned. Also, it
| is highly unlikely that he is using his own grid, he probably
| pulling most of his power from the U.S grid, which (like most
| grid to be fair) contains a fair amount of CO2 heavy sources.
| And that is not even talking about the amount of pollution
| and waste used to produced all that hardware he is running.
|
| I get it, it looks cool and damn fun. I dream of this kinds
| of setup. but brushing off the environmental impact is also
| wrong.
|
| Now admittedly, he is but a drop in vast ocean when it come
| to pollution and waste, and there is definitely far worse
| offender. But keeping our environmental impact in mind should
| be a necessity in this era.
| Cthulhu_ wrote:
| There, it's in mind; what can we do now to stop the big
| companies from making things worse?
|
| Individual accountability is important, but self-
| flagellation or (vicarious) shame because forces bigger
| than us are causing problems is a waste of energy (ha!).
| komali2 wrote:
| This is the argument that holds the most weight with me. It's
| why, as much as I approve of Extinction Rebellion and their
| kin, I think it's counterproductive to deflate SUV tires,
| when the environment harm caused by SUVs is a fraction of
| that caused by the companies profiting off oil and gas
| extraction and refining. If you're gonna do a crime anyway...
|
| It's similar to when there's hypocrisy charges levied against
| "champagne socialists." Nice virtue signalling for worker's
| rights for someone with a 6 figure income... meanwhile that 6
| figure salary accounts for 0.01% the net worth of a
| billionaire.
|
| Go for the high stakes first, then we can worry about
| homelabs pulling more power than they should.
| ur-whale wrote:
| > I have to categorize this as a (un)typical American person
|
| Holier-than-thou much?
| maleldil wrote:
| Americans tend to drive larger cars than other nationalities.
| These cars use more fuel. Most of these people do not need
| large cars and would be fine with smaller ones that use less
| fuel. Therefore, by choosing larger cars, they're using more
| fuel than necessary. An explanation is that only care about
| their own comfort, not the environment.
| abwizz wrote:
| tbf a lot of murricans live in rural environments where a
| truck does indeed make sense because of the lots of unpaved
| roads.
|
| city dwellers not so much
| patmorgan23 wrote:
| Not really, American is pretty urbanized. Less than 20%
| of the population lives in rural areas. Yet the most
| popular vehicle for the past few years has been the ford
| F150. It's not being bought by rural or working class
| people, it's a luxury status symbol now.
| abwizz wrote:
| > American is pretty urbanized. Less than 20% of the
| population lives in rural areas
|
| i'd like that number put into perspective, but it does
| not feel like on the low end.
|
| > Yet the most popular vehicle for the past few years has
| been the ford F150. It's not being bought by rural or
| working class people, it's a luxury status symbol now.
|
| thanks for the insight! but i struggle with the
| combination of "most popular" and "luxury status symbol"
| ryan-c wrote:
| I am not running serious network gear right now due to power,
| which is ruinously expensive in the UK to the extent that I
| find myself longing for PG&E's rates in California.
|
| Currently having a house remodeled, which I'm putting as much
| solar (and battery) on as possible, and the gas is being
| removed entirely. Going to try not to use more than the system
| can generate, though I am not sure I'll be successful.
| TacticalCoder wrote:
| > The generator is a 27kw Generator which powers everything in
| the house. This means the UPS's only need at most 10 seconds of
| runtime, as that's how long it takes for the generator to start
| and switch
|
| That Generac generator though: it powers everything but _only in
| case there 's a power outage_ right?
|
| P.S: there's a slight typo in TFA: it's Mellanox, not Mollonox.
| monstermunch wrote:
| Correct, only if there is an outage
|
| Thanks, I'll correct that!
| bombcar wrote:
| Do you regularly switch to the generator to test it? Is that
| automated?
| trustingtrust wrote:
| Biggest problems I have faced for home networks : ISPs are poorly
| configured and do very shady things sometimes like DNS hijacking.
| More often than not poor ONU firmware or hardware means you will
| see weird things like latency spikes or straight up packet drops.
| Sometimes IPv6 won't work if the ONU is in bridge mode etc. etc.
|
| This post is overkill but what mostly works for home networks:
|
| SQM - like cake sqm in openwrt - will work wonders if don't have
| a great ISP. I have seen as much as 5% packet loss on one ISP who
| said thats acceptable for a home network. Just reduce the load
| until a point where packet loss is really low. Then use a DNS
| resolver. DNS resolver will be much more reliable like unbound
| for your local network as packet loss has cause really problems
| with DNS for me. Cheap home network equipment dns is not
| reliable. You could really just buy a cheap Pi and run a local
| resolver. It would work a lot better than your home router. You
| may also want to consider replacing the ONU. This is rather
| easier than you think. The ISP ONU are really bad. A cheap NOKIA
| SFP GPON is like 20$ on eBay and if you can figure out the PLOAM
| password and Serial and Mac and VPN ID of your ISP, you will be
| able to simply swap it for a more stable link.
|
| All in all I would suggest for a simple home setup: get a cheap
| SFP GPON and replace your ONU. Get a cheap router (either router
| on stick with Rpi4 for <500mbit or x86 box with SFP for hitting
| 1gbps with the SFP GPON) and run your own unbound resolver. Get
| some APs and either wire them or get a cheap mesh router 3 pack
| on amazon (you can get a wifi 6 3 pack for under 150$ these days
| on sale) and run it in AP mode.
| iptrans wrote:
| Obvious caveats:
|
| GPON networks are often vendor locked. This means you cannot
| just buy any random SFP ONU and expect it work.
|
| Even if you do get it to work, no ISP will offer you any kind
| of support if you do this. Some might even take to it poorly,
| if they find out.
|
| Generally, the best option is to ask your ISP if they offer
| other ONU models and choose from those.
| trustingtrust wrote:
| Almost all ISPs use Serial and (optional) PLOAM to
| authenticate your ONU. Nokia GPON SFPs allow you to change
| almost everything that is exposed to the ISP for
| authentication. ITU standards mean that ISPs have to work
| according to the spec.
|
| ISP ONUs are locked for easier management for them. They will
| give you 1/10th of the speed promised if it helps them deal
| with support calls remotely rather than visiting your place
| to fix it. Support calls where people complain for things
| like 'I forgot my wifi password' cost ISPs a lot of money. So
| they can basically just login to your ONU at any time
| remotely and change settings for you like your wifi password.
| They do remote firmware updates and what not remotely. This
| is the biggest reason why ISPs love such modem router combos.
| Support can be guaranteed with a phone call to fix your wifi
| for the average Joe.
|
| You will not get ISP support if you use your own ONU but if
| you are using your own ONU then you are already at that point
| where you know what you are doing. As far as signal issues
| are concerned, like I said ITU specs mean they can see the
| signal strength remotely. Everything else they don't have
| access to but you don't need them to have that access. You
| can always swap out the ISP provided box to troubleshoot.
|
| If you actually swap out the ONU to a better one, chances are
| you'll never need to call your ISP unless there is a fiber
| cut or some serious signal loss somewhere.
| Karrot_Kream wrote:
| Why would you use a local DNS resolver instead of using a
| remote well-known resolver like 1.1.1.1 or 8.8.8.8 in a home
| networking situation? Your systems will cache DNS entries to
| paper over unavailability issues, and as long as you use one as
| your primary and the other as a secondary, there's a very low
| likelihood (probably < 0.0001%) that you'll ever have issues
| resolving DNS.
| trustingtrust wrote:
| If you look at systems like Deco (and a bunch of others I've
| noticed recently like linksys) the devices have stopped being
| dns forwarders themselves to clients and instead they assign
| your ISP (or the ones you've set for your ISP) DNS directly
| to clients. This means your device now resolves dns over wifi
| directly. This means each device is now relying directly on
| google and cloudflare instead of relying on the gateway
| (which being wired will experience the lowest latency
| especially when it comes to redirects and should ideally have
| a robust DNS server).
|
| A local network resolver instead of forwarder on a client
| seems to work a lot better in my experience especially when I
| use root DNS on unbound. Yes a root DNS is a lot slower than
| something like 8.8.8.8 but you could use an upstream dns with
| unbound if you really want something faster. Either way, a
| local DNS will be lot less problematic compared to if you
| devices has to handle all dns over wifi including the
| redirects (which happen a lot).
|
| Almost every time I see a network problem, it's always a DNS
| problem. In a typical home network setup these days:
|
| client DNS cache -> upstream DNS -> root DNS. client DNS
| cache -> local DNS - > upstream DNS -> root DNS.
|
| My setup:
|
| client DNS cache -> local DNS -> root DNS.
|
| The first one works better if the network is wired. Second
| and third one works better on wireless.
|
| One alternative that works 'okay' is if the client directly
| uses DoT or DoH for DNS. TCP works little more reliably than
| UDP (wink) but neither Google nor Cloudflare offer signed
| profiles for iOS and macOS (third party is there). Stubby
| works good on linux and android and windows have a built in
| solution.
| abwizz wrote:
| because those 3rd-party resolvers are not under your control,
| which is usually interesting for someone running a homelab,
| and lie in responses.
|
| "it's always dns" should ring a bell :)
| Karrot_Kream wrote:
| For a homelab yes, but I figured GP was referring to a
| robust home network, not necessarily a homelab. "it's
| always dns" goes both ways as I've had local resolvers add
| huge amounts of latency to connection establishment.
| abwizz wrote:
| yea. i was also advocating against local resolvers in the
| name of robustness, until my isp was starting to block
| certain domains.
|
| not really the fact that some domains i don't care much
| about are blocked, but this is a dangerous precedent in
| deviation from facts.
| m3kw9 wrote:
| I could do all that with a single ddwrt router behind my tv
| mgaunard wrote:
| I like how the small cupboard used to host the rack is bigger
| than most bedrooms in European city apartments.
| pm2222 wrote:
| I didn't read in details. Why esxi is not incorporating whatever
| that's on the rpis?
| greggyb wrote:
| The Pis are using GPS receivers to serve as NTP servers on the
| local network. The specific appear to use GPIO pins. You would
| not be able to wire these to the ESXI machines as readily. You
| typically also want time-sensitive workloads running on metal,
| as there are all sorts of potential complications with
| virtualized time.
|
| I am sure there are PCIe solutions that could easily go into a
| more standard form-factor machine, but perhaps these are more
| expensive or less readily available; this is just conjecture.
| The ESXi machines have minimal PCIe connectivity. Perhaps such
| a card could be put into one of the Supermicro chassis.
| frankreyes wrote:
| GPS modules use SPI which you can get a standard USB serial
| adapter for $2 on eBay. As the title suggests, it's overkill.
| Basically it's using the Rpi as SPI adapter.
|
| Edit: USB to TTL costs like $3.5, it's like a standard RS232
| serial port but much simpler and faster.
| greggyb wrote:
| Ah, cool. I didn't know that about the connection. Thanks
| for sharing.
| monstermunch wrote:
| While you could use serial through to to a VM, I need another
| pin to get the PPS signal in. Not sure how to do that
|
| I also like having them bare metal
| 8fingerlouie wrote:
| I used to have an overkill home network, complete with self
| hosting everything and 10G backbone.
|
| These days i'm more pragmatic. I have exactly 2 wired machines,
| and everything else runs Wifi. The network "rack" is simply a
| firewall and a 16 port POE switch, everything else network
| related has been retired to the closet.
|
| My NAS is gone. With electricity frequently going above EUR1/kWh
| last winter, it seemed like a bad idea to have a NAS sitting
| there, consuming ~35 kWh/month. Instead i spent about EUR20/month
| on cloud storage, and simply upload everything to the cloud
| (encrypted with Cryptomator), and i have redundant cloud storage
| (different provider) included in the cost which is where my
| backup goes.
|
| All that's left at home is a small ARM based machine that handles
| making daily backups of my cloud data locally, as well as run a
| small Plex library on a single USB3 external drive.
|
| Besides saving about half my previous electricity cost (as well
| as all the hardware cost!), i have gained SO MUCH spare time. My
| firewall is completely closed (save a wireguard VPN), and i no
| longer have to maintain anything except the normal patch
| routines.
| [deleted]
| perfopt wrote:
| Could you share how you set up the cloud backup? Also, what
| cloud providers you use
| 8fingerlouie wrote:
| My cloud backup is rather simple, i use Arq backup with
| Family365. Fits nicely that each user has 1TB space in their
| personal OneDrive, and nobody here uses OneDrive.
|
| My local backup is also done with Arq backup to a local USB
| drive, though i have a long time test running with Kopia
| backing up to Minio on a Raspberry Pi.
| bdavbdav wrote:
| Ugh. Agree with this.
|
| My Synology NAS was failing. Saw that as a great opportunity to
| set up a proxmox box on a reasonably powerful used HP SFF, run
| TrueNAS on it. Plenty of time to get proxmox, TrueNAS, Gitea, a
| docker host, ... just right.
|
| It's sat there with 4/16TB used, no real data transfer apart
| from TimeMachine from my Mac and my git server, sucking power
| at European rates.
|
| Wish I hadn't bothered, and had just gone with another
| Synology.
| sangnoir wrote:
| > It's sat there with 4/16TB used, no real data transfer
| apart from TimeMachine from my Mac and my git server, sucking
| power at European rates
|
| I have a similar setup with SFF running backups & ProxMox.
| However, I have smart plugs for my SFF and have a script to
| turn it on/off using cron (or on demand) from an always-on
| Pi. Wake on LAN would have worked too.
|
| I'm considering making my Raspberry Pi the gateway
| proxy/"Load Balancer" to the SFF; turning it on if needed.
| It's trivial to proxy HTTP (Pi works as a Caddy-based proxy
| with long timeouts, powers on SFF in background if off.
| Things got trickier for proxying NFS & iSCSI)
| bdavbdav wrote:
| This is a genius idea. I may try similar.
|
| The box is on another routed subnet - I wonder if I could
| firewall rule a "start" trigger.
| kotaKat wrote:
| I went the same way as... honestly, my IT burnout kicked in. I
| went from an overly complex setup down to... well, a Ruckus AP
| running as an Unleashed gateway directly hooked to a FiOS ONT,
| its second port feeding a QNAP.
|
| That's it, the entire network. Just enough storage to enjoy
| some media, the QNAP runs HomeAssistant (literally just to
| bridge some stuff to HomeKit) and Tailscale, and that's all I
| really need.
| wasksrhh wrote:
| I recommend you checkout Rclone
| (https://github.com/rclone/rclone) instead of Cryptomater, much
| more robust and powerful software. I used to use Cryptomator as
| well, but after my data exceeded certain size, it became slow
| and difficult to operate. Plus, once I went to pull my data out
| of it, I realised that it had reset file metadata on all my
| files (I didn't know what picture was taken when anymore, for
| example)
| 8fingerlouie wrote:
| I do use Rclone as well, but Cryptomator has one feature that
| rclone does not, it allows me to access my data "on the go"
| from my phone.
|
| Personally i have never experienced any of the issues you
| mention, but my Cryptomator vaults are also mostly below 1TB
| in size.
| matrss wrote:
| If you are using android there is RCX
| (https://github.com/x0b/rcx) as a frontend to rclone.
| Rclone can also be used in termux. Not sure about apple.
| mattlondon wrote:
| Amen to this.
|
| I moved house recently and the only things I did were running
| CAT6 from where the internet enters the house, up to a PoE
| switch in the loft that then has a few Wifi6 APs connected to
| that.
|
| Everything is wifi (all cameras, smart switches etc, office
| laptop, a single RPi4 for "projects" and so on) and all seems
| fine. The only device that is directly connected via ethernet
| is a headless gaming desktop that lives in the loft that I
| occasionally use for streaming games to a laptop.
|
| But kudos for this guy if that is what he wants to do.
| Personally I did _not_ enjoy doing the cable runs (especially
| for external-grade CAT6).
| 8fingerlouie wrote:
| My cameras are still POE (UniFi Protect), but i had already
| run the cables for it, so not much point i replacing them
| just for the sake of moving to WiFi.
|
| My "server" is also wired, as is my Hue Bridge, Tado Bridge
| and other "IoT" devices that doesn't support wifi, but when
| it comes to user accessible machines there is exactly one,
| which is my "go to" machine in case the router/switch dies
| and i can't get on the WiFi.
| nuker wrote:
| > My NAS ... consuming ~35 kWh/month
|
| Translates to 50W.
| earthling8118 wrote:
| Thank you. I spent a good few moments trying to wrap my head
| around this measurement.
| AussieWog93 wrote:
| Honestly, I thought it made more sense to measure it that
| way if you're trying to work out the electricity bill.
| missjellyfish wrote:
| I still have my local NAS and will in fact be upgrading it
| sometime this or next year. I don't trust cloud computing with
| my data, even tough I'd rather forgo the complexity and
| electricity cost.
|
| But that's my price of freedom.
| [deleted]
| willis936 wrote:
| Looks like every homelabber's dream.
|
| My main feedback is give those fibers a bit more breathing room.
| The bend radius is a little tight.
| monstermunch wrote:
| All the fiber is specced as 20/10D, so should be well within
| tolerance
| jamesbfb wrote:
| I love this so much! For anyone asking why, the simple answer is:
| because they can! Hats off to you, Sir.
|
| I wonder if this person works in IT? I'm a dev lead, I do silly
| amounts of hours and when I actually have a decent amount of free
| free time up my sleeve, the last thing I want to do is tinker
| with my homelab.
| jamespo wrote:
| of course he does!
| Dagger2 wrote:
| All that and no mention of IPv6? At least get the basics down
| first.
| toast0 wrote:
| Looks like they've got dual-wan (at&t fiber + verizon 5G
| backup). It's pretty easy to do ipv4 failover with NAT as long
| as you've got an indication that it needs doing; ipv6 failover
| is a lot harder; unless you're going to NAT the v6 traffic too.
| I don't know about Apple devices, but I couldn't get Android or
| Windows to play nice with two v6 ranges in radvd. They wouldn't
| listen to priority, and they would do derpy things like pick an
| address from one router's advertisement and then send the
| traffic to the other router.
|
| In some magic happy land where you can use your IPv6 prefix
| over both WANs, then yeah, things can work, but that doesn't
| work for residences, generally.
| hqsolomo wrote:
| Lol I get sym 1G/1G from my ISP and a free static IP address
| for less than Comcrash offers for their 25/10 service in my
| area...
|
| The catch? _No IPv6 support whatsoever_
| systems_glitch wrote:
| Yeah, even with Comcast/Xfinity I run a HE.net tunnel for
| IPv6, just so I can have truly static allocations and not
| goof around with their DHCPv6 prefix request system.
| hqsolomo wrote:
| Dude, got any links to where I can read more? I see what
| you have here and I want it for me!
|
| I wasn't even aware you could use HE's services without
| being a customer!
| systems_glitch wrote:
| https://tunnelbroker.net/
|
| It's been around for a long time! If you do the IPv6
| training certification, they do eventually send the
| t-shirt :P
|
| EDIT: They also provide free DNS for up to 50 domains at
| dns.he.net which is also worth looking at if you don't
| want to run your own for every little thing.
| hqsolomo wrote:
| THANK YOU! This is perfect!
| mrpippy wrote:
| I haven't used it in a few years, but one downside of an
| HE tunnel was that Netflix blocked traffic from it (since
| people were using it like a VPN to avoid geoblocks).
|
| I had to run a DNS proxy to filter out AAAA records for
| any Netflix domains.
| wahern wrote:
| HE and Cogent are still feuding over IPv6 peering, too.
| Using HE for regular web browsing will work fine, but if
| you want to _host_ through HE your services might not be
| reachable to visitors from a Cogent network, e.g. office
| buildings.
|
| The norm seems to be for hosting services to contract
| with _both_ HE and Cogent if they decide to go with one
| of them. This is why for typical client browsing you
| wouldn 't notice--the other side has ensured you can
| reach them. HE seems to be the cheaper option, so people
| will route most of their traffic through HE, and then
| separately contract with Cogent for the smaller amount of
| traffic originating from/to Cogent networks.
| Alternatively, you contract with another network provider
| who handles that idiocy for you.
| systems_glitch wrote:
| That is true, and a bunch of folks (including me) were
| experiencing issues with Google-related sites blocking
| with a 403 FORBIDDEN on the initial /64 static allocation
| from HE.net. Oddly enough, any subnet out of the optional
| routed /48 you can enable worked fine.
|
| For sites that use HE.net and have more than one IPv6
| subnet, I use the initial /64 for the DMZ, so that didn't
| matter on most of them. Small sites had just been using
| the /64 though, and I had to renumber those. Total pain.
| jonathantf2 wrote:
| Probably not supplied, I've got no IPv6 on any connection I
| have access to other than a few Azure VMs (and it's a PITA to
| set up so we haven't bothered)
| monstermunch wrote:
| I have ipv6 on both WAN's and ipv6 on LAN, but not any other
| VLAN's
|
| To be completely honest, I find ipv6 wildly alloying, but I
| guess that means I need to learn more!
| m463 wrote:
| Do you need ipv6 on a home network?
|
| Honestly I just use ipv4 and turn off ipv6 everywhere. Then I
| have just one configuration and one set of firewall rules (in
| and out).
| systems_glitch wrote:
| To say "couldn't live without it" would be an exaggeration,
| but we do use it extensively for work, and having it at home
| makes WFH much easier. End-to-end connectivity, like in the
| old days, is very nice!
| tomerbd wrote:
| I always get the cheapest plan from my ISP. So far all great.
| snailtrail wrote:
| [dead]
| lxe wrote:
| I'm curious what the total cost of all this is. My reservation of
| rack mounted setup has been cost of hardware.
| dheera wrote:
| It can be reasonable if you wait for good 2nd hand deals on
| equipment, especially from startups shutting down.
|
| My 10G switch with PoE and 4 SFP+ ports cost only $100, for
| example, while many new products with similar specs cost $600+.
| hqsolomo wrote:
| Can I guess what you're running? It sounds so much like my
| switch- Brocade ICX series?
|
| Best $60 purchase I ever made!
| dheera wrote:
| Aruba S2500-24P
| hqsolomo wrote:
| Oh, still very nice!
| vdfs wrote:
| Cost of electricity is an other factor to consider
| wredue wrote:
| It's a black hole for your wallet. Once you start. You can't
| stop.
|
| But I also now have a bunch of electrician tools for running
| wires, some private cameras that aren't google or Amazon spying
| on everyone that walks by, and a network that isn't bogged down
| by a couple people watching Netflix.
|
| And I have plans for MORE.
| KnobbleMcKnees wrote:
| What cameras do you use?
| megraf wrote:
| Axis communications, exclusively.
| wredue wrote:
| It's all ubiquiti. I use the Bullet G4 pro and doorbell g4.
|
| I'm not in any position to state anything about the
| competition.
|
| I am glad to have went the PoE route, even if running those
| wires sucked ass.
|
| I only did UI cameras cause everything else I have is ui.
| There wasn't much more thought than that.
| KnobbleMcKnees wrote:
| I have ubiquity APs and a USG but the camera are so
| expensive. I'll have to stick with my Chinese crapware
| and a DNS blocker for the meantime.
| hqsolomo wrote:
| If you're crafty you can work around this- three of my servers
| (2 NAS boxes and a Proxmox box) were custom built from
| commodity hardware. I literally bought plain ol uATX and ITX
| decommed business gear from my local uni surplus store and a
| P-Link chassis for each used from eBay. I spent more on drives
| than I did machines (Y'ALL WANNA KNOW HOW MUCH IT COSTS TO BUY
| 16TB WORTH OF SSDS FOR ZFS MIRRORS? TOO. DAMN. MUCH.)
|
| You really gotta do some due diligence to make sure you're not
| buying lemon parts but it's very possible to get a beefy
| homelab at fair prices!
| monstermunch wrote:
| I make sure to never count the cost!
|
| But, its all used and all over a number of years, so not as
| much as you'd think. If you went out and tried to get it all at
| once, it would be crazy. Most things I spent a good while
| waiting for the right deal to come up
|
| Like the UPS alone, I saved probably close to $3000 off MSRP
| meitham wrote:
| This is beautiful and inspirational! I am about to redo my messy
| homelab and have two questions: 1. I don't see any labels
| attached to the cables, how do you know which-is-what? 2. I am on
| pfsense too but I was considering a move to OPNSense, would you
| explain why you choose pfsense over opnsense? Thanks
| rb666 wrote:
| Stick to Opnsense, it is better software that is better managed
| with a clean historical sheet.
| dikaio wrote:
| All I could think while reading the post is, I want one.
| yobbo wrote:
| It seems it scratches the same itch as a model railway.
| monstermunch wrote:
| You know, I'd LOVE a model railway...
| physhster wrote:
| ...and I thought mine was overkill! Great job!
|
| You might however want to read about the grocer's apostrophe:
| https://www.grammar-monster.com/lessons/apostrophe_error_wit...
| ojbyrne wrote:
| That page includes the sentence "A word that ends in a vowel is
| more likely to attracting this mistake."
|
| I wouldn't bring it up but... glass houses.
| monstermunch wrote:
| I'll have to go over the whole post, I'm terrible at writing!
| Quite a few errors have been pointed out. But I can just edit
| them and pretend they never happened
| physhster wrote:
| Basically, you don't want to pluralize acronyms with 's. The
| more you know :)
| drpixie wrote:
| Very impressive - but it would be terrible if they have to move
| house!
| monstermunch wrote:
| Honestly there is a few things I would do different now, so it
| would be welcome to have a do-over
| bombcar wrote:
| One of the main temptations to having a house built (probably
| partially by me) is that I would be able to do all the wiring
| _correctly_ (not just conduit in the usual places, but
| decorative trim in each room that easily removes to reveal
| the wiring cavity in the wall behind it, etc).
| Thlom wrote:
| >The Hubitat hub has now almost entirely been replaced by a
| HomeAssistant virtual machine. The only reason it is here is to
| connect the Z-Wave and Zigbee devices I have. However, I replace
| those devices with WiFi versions wherever possible
|
| There's cheap and reliable ZigBee/Z-Wave/433MHz USB dongled
| antennas you can replace the Hubitat with.
| iAm25626 wrote:
| what no 100g or IB? Those price point have come to a point where
| for home lab is bearable. Well we are talking about overkill. If
| one want to play around with network do it virtually is a much
| cheaper way to go: https://www.gns3.com/
| selectodude wrote:
| I run GNS3 on a three node HA proxmox cluster :)
| hqsolomo wrote:
| Infiniband? Don't tease me with a good time
|
| JK, wife would kill me
| s09dfhks wrote:
| I'll take that Lenovo M73 Tiny off your hands ;)
| monstermunch wrote:
| You in Houston?
| WaitWaitWha wrote:
| Did I miss the logical and physical diagrams?
|
| I appreciate your hobby of a well installed home network. Add the
| diagrams and you will increase your readers' comprehension
| several fold.
| monstermunch wrote:
| Good idea, I'll work on some
| tristor wrote:
| This doesn't seem that overkill to me other than using fiber
| optics, although I've done that in the past, unless you're doing
| exceptionally long runs copper cabling is sufficient even at
| 10Gbit within a home scenario. Seems like a decent setup.
| monstermunch wrote:
| My main reason for using Fiber or DAC's is because of the power
| consumption. Running 10G-BaseT uses about 3x the amount of
| power
|
| Plus, I always think fiber is cool
| toast0 wrote:
| Surplus 10G switches at good prices are likely to be sfp+ and
| not 10g-baseT at this point in time, too. DAC for same rack
| is a no brainer at that point. To the rest of your house
| depends on what you've got in the wall and how easy it is to
| put new stuff, I guess. I'm running 10G-baseT between my two
| network cabinets, because there's no economical way to pull
| anything else, and there's no reason to go for more than 1g
| other than fun.
| tristor wrote:
| True. That said, fiber can be painful to deal with.
| Electrical isolation is one big benefit, as is power savings,
| and of course forward-compatibility. I still prefer running
| copper for residential, especially because you don't have to
| be as careful about bend radius and such.
|
| Within my rack I use DACs to connect all the gear, which
| works well since it allows maximum performance over short
| runs with minimal effort.
|
| Nice setup though, my comment wasn't meant to criticize, I
| think having a proper network at home is a big boost to
| quality of life that most folks don't realize because they've
| never experienced it.
| frankreyes wrote:
| I need a silent ECC server. Doesn't need to be super powerful but
| I'd rather have at most one fan. Any advice?
| willis936 wrote:
| I've been wanting to make an X570D4U-2L2T + CS381B build for a
| year, but haven't been able to justify the price.
|
| The Latte Panda Sigma looks pretty amazing too if in-band ECC
| is sufficient.
| greggyb wrote:
| I'd start with something like this[0]. The only search term I
| used was "xeon-d". You can easily find other form factors and
| combinations. This line of CPUs is lower power (the linked one
| is rated at 45W), so should be trivially cool-able with a very
| quiet system. They support ECC RAM. You're likely to find
| mostly Mini-ITX and uATX boards, so they will fit in just about
| any case you want.
|
| [0]
| https://www.ebay.com/itm/166190039675?epid=17034031881&hash=...
| favorited wrote:
| I've done this a few times, and the killer has always been
| the RAM. By the time you're buying decommissioned enterprise
| hardware, it's hard (or just expensive) to source RAM from
| its QVL list - even harder so when you want ECC, which not
| all of the QVL'd SKUs will be... And, in my painful
| experience, if it's not QVL RAM there's a good chance it just
| won't post.
| bombcar wrote:
| I've had no problem with getting the RAM (either new or
| just more eBay stuff) - and even got a replacement
| mainboard for relatively cheap when one burned up (it was
| easier to swap the mainboard than reconfigure everything
| for a new server).
|
| The killer is the power consumption. It's better now
| compared to old enterprise gear of 20 years ago, but you
| can sometimes still pay for a brand new low-power system
| just in power savings alone.
|
| Of course, if you have solar onsite that doesn't matter.
| greggyb wrote:
| The power is part of why I prefer the embedded boards
| like the one I linked.
|
| Especially putting one of these _not_ in a rackmount
| chassis, you can opt for quiet cooling and fans that do
| not draw 10s of watts each. The idle loads can get fairly
| low, depending on your additional peripherals.
|
| For very low power, I actually prefer small form factor
| machines, either used 1L SFF machines from the big OEMs
| (see STH's Project TinyMiniMicro[0]) or one of the many
| AliExpress mini PCs[1]. These sorts of machines tend to
| use low-power CPUs and a lot of the AliExpress ones are
| laptop parts.
|
| As with all things, there are always tradeoffs. If you
| enjoy researching and bargain hunting for hardware, then
| this is a viable path. If you just want a machine that
| works, then this is onerous work.
|
| [0] https://www.servethehome.com/introducing-project-
| tinyminimic...
|
| [1] https://www.aliexpress.com/w/wholesale-mini-
| pc.html?catId=0&...
| greggyb wrote:
| The top spec 32GB RAM on the qualified list for the
| specific board I linked earlier runs $40-$41 on ebay. The
| same speed and capacity for generic DDR4 runs in the mid
| $30s. The board has 4 RAM slots. So the premium for QVL RAM
| for that linked board is $20 over generic if you want to
| max out at 128GB.
|
| Supermicro did have RAM issues in the early teens.
|
| This sort of research is the non-dollar price of finding
| deals on server-class hardware.
|
| If you want to get something pre-validated, and usually
| with a warranty, it can be worth it to look for resellers
| for the major OEMs that also offer refurbished hardware.
| It's more expensive than the typical options available on
| eBay, but still much cheaper than new hardware.
| rjsw wrote:
| I have a HP MicroServer with ECC RAM.
| hqsolomo wrote:
| Some Ryzen chips (I think- might be TR) often support ECC- see
| if you can find a compatible mobo that supports ECC modules and
| Ryzen sand! When I last checked there were a few products out
| there
| frankreyes wrote:
| I've just checked and there's some mobos for $60 with ECC. Is
| that for real? Supermicro costs like $1000, half that for
| refurbished.
| hqsolomo wrote:
| Without looking at the listing I can't earnestly say what
| you're seeing isn't fake but I know back when I had the
| same question you're asking here I saw similar offerings.
| Ultimately I bought a used ITX supermicro mobo+CPU combo
| unit from eBay for like $200
| monstermunch wrote:
| Do you really need ECC? Most people don't
| bombcar wrote:
| Many people got scared by the ZFS/ECC howling years ago, and
| didn't really think it through (because they're running
| without unverified backups).
|
| https://www.klennet.com/notes/2022-11-25-zfs-and-ecc-
| rant.as...
|
| IF you can get ECC cheap, get it, otherwise, meh.
| monstermunch wrote:
| Yeah, I think you and me are on the same page there. It has
| ECC? Great! It doesn't oh well
| kuon wrote:
| Here you would need a breaker after the UPS before going into the
| wall because UPS is not considered up to code for breaking
| leaking current and all outlets must have protection. That's why
| I ended up not having a central UPS. How is it in USA?
| monstermunch wrote:
| Allowed in the USA per code NEC 400.7
|
| But, It would have been better if I fed a small sub-panel from
| the UPS, and then went out from there. Shoulda-coulda-woulda!
| thecosas wrote:
| "This is also how you are probably getting to this blog, which is
| hosted at home."
|
| Curious to know how hitting HN impacted your setup (if at all)!
| monstermunch wrote:
| Not much, because a lot of it is cached by CloudFlare
|
| Total bandwidth over the past 5 hours is 253.37 GB. But total
| cached and served by CloudFlare is 237.82 GB. Uncached is just
| 15.54 GB
|
| Here is CPU load on the VM:
| https://files.networkprofile.org/apps/files_sharing/publicpr...
|
| Networking on the VM:
| https://files.networkprofile.org/apps/files_sharing/publicpr...
| jancsika wrote:
| Looking at this setup makes me think it would be interesting to
| have a public-facing chat service on a 30-second clock where the
| user has to pad tweet-sized messages with a pre-determined number
| of zeros and complete sending them off before the deadline is up:
|
| 1g lounge: must pad message with 1gig of zeros
|
| 10g lounge: must pad with 10 gigs of zeros
|
| 100g lounge: must pad with 100gigs of zeros
|
| 1t lounge: must pad with 1 terabyte of zeros
|
| 100t lounge: intelligence community meet-and-greet
|
| Edit: relevance :)
| tdhz77 wrote:
| Reading this post makes me happy we are in the world of cloud
| providers, but realize they don't magically work. People build
| them. I'm glad I don't.
| wredue wrote:
| It's really not all that complicated. Although I still haven't
| figured out if there's some secret way to properly creating
| working Ethernet cables.
|
| I get blisters on blisters on my finger tips when making lots
| of cables.
| monstermunch wrote:
| Your time is too valuable to make ethernet cables. Either
| punch down to a keystone jack or patch panel, or buy patch
| cables
|
| Gave up on that years ago
| bmitc wrote:
| Do you use a crimper?
| efitz wrote:
| You can get an Aeotech USB Zwave stick and plug it into your
| Homeassistant RPi if you want to get rid of Hubitat.
| mhb wrote:
| Are there any building code considerations (e.g., chimney effect
| for a fire) regarding a multi-floor wiring chase? Isn't this why
| laundry chutes are no longer permitted in some places?
| monstermunch wrote:
| Its a single story home, however it is from 1968, so its
| possible that in new code its outlawed or something
| mindslight wrote:
| I've also been curious about this in the context of making a
| dumbwaiter, but not enough to do the research. I would think
| having passive automatically closing fire rated doors would
| take care of it, but I don't know.
| bombcar wrote:
| You could look into it but I think the reason laundry chutes
| aren't made anymore has more to do with kids getting stuck in
| them and less to do with fire code (after all, stairways are
| chutes connecting floors).
| gnopgnip wrote:
| Riser or plenum rated cables are required by code if you go
| between floors, except in plenum rated conduit. Conduit is
| usually more expensive though.
|
| Non rated cable more easily catches fire and makes a lot of
| smoke. Plenum rated cable is about twice the price, but
| ethernet is pretty cheap for a single home.
| LouisvilleGeek wrote:
| Really nice setup! the pfSense part made me smile!
| m463 wrote:
| as long as you're good that pfsense is not open and phones
| home.
| systems_glitch wrote:
| Plus the domain squatting with hitler and goatse thing... :/
| monstermunch wrote:
| I've thought about switching to OPNSense, but honestly I'm
| just set in my ways with PFSENSE now. Maybe when they really
| piss me off I'll switch
| hqsolomo wrote:
| I'm sorry- when did they start doing this? I've been mulling
| making the switch to OPNSense. If pfSense is phoning home
| aside from the update check I might have a busy weekend ahead
| of me
| comprev wrote:
| Have a quick search about the bad activity undertaken by
| pfSense against OPNSense and you'll soon change your
| opinion on the company.
| hqsolomo wrote:
| Welp, that was... A thing they did. Reminds me of middle
| school antics, except these are grown folks. Guess it's
| time to jump ship after all!
| woleium wrote:
| and that BSD still cannot do line speed gigabit on an Intel
| gigabit nic.
| hqsolomo wrote:
| I dunno if this is entirely true- my current pfSense router
| gets as close to line speed as I'll ever be able to get
| with my setup (~900 both ways to a test server in Chicago
| and on LAN) and I'm running an (overkill) i5 in my generic
| Chinese minipc. It's possible that I don't have Intel NICs
| but I swear I do
|
| Got anything more I can read? Dunno if I'm just blind but I
| couldn't find anything on this. I'd like to learn more!
| systems_glitch wrote:
| [citation needed]
|
| We had no problem routing gigabit on an Intel Atom D525
| years and years ago at a previous job. Consumer Mini-ITX
| board, onboard gigabit NIC, and an Intel gigabit NIC in the
| one expansion slot. It did require minor tuning, but
| nothing that couldn't be done thru the web UI.
|
| Everything I have to manage runs plain OpenBSD managed with
| Ansible now, so I don't know what the current state of
| pfSense/OPNsense throughput is.
| monstermunch wrote:
| Mine certainly can
| ThatPlayer wrote:
| What are the plans for the Meshastic mesh? I think having one
| hooked up to a server to do responses could be interesting. Or
| have you actually convinced other people to carry one around for
| messages?
| qwerty456127 wrote:
| I wish every apartment would have at least some units of at least
| half-depth of rack space. Fitting anything (even a basic setup of
| a PaspberryPi + the PON modem + a minuscule switch + a PoE
| inhector) into the tiny space behind hatch where the cables from
| all the wall sockets end in my condo apartment is extremely hard.
| And I am still looking for a 5-port switch which would be
| manageable, support vlans, handle gigabit ethernet and inject PoE
| if possible while being really small.
| tosie wrote:
| When powered adequately, the Ubiquiti Flex (USW-Flex) [1] might
| fit your needs.
|
| [1] https://eu.store.ui.com/eu/en/collections/unifi-switching-
| ut...
| qwerty456127 wrote:
| Looks great but I woud prefer to avoid Ubiquiti because I
| don't want to involve a 3-rd party cloud to manage it.
|
| But I really appreciate your suggestion, perhaps I am going
| stick to it as I've seen no alternative so far and my
| previous experience with Ubiquiti makes me confident it will
| do the actual job great.
| jamespo wrote:
| How about MikroTik hEX S 5 Port Router ?
| qwerty456127 wrote:
| This one seems suiting my preferences perfectly. I even
| wonder how did I overlook it. Now I'm looking for a WiFi6
| access point which would work on the passive PoE it would
| feed.
|
| You really helped me a lot! I wish you could feel how my
| mood has improved. Now I feel empowered to implement the
| set-up I wanted. This is a minor home improvement task
| but it sitted in my to-do list (after numerous attempts)
| for over a year.
| jamespo wrote:
| No problem, I only became aware of this recently and it
| seems to offer a lot of bang for the buck.
| ianburrell wrote:
| I wish 10" half-width racks had taken off. They are available
| in Europe but not the US. They would be good size for home
| networking gear, if anything supported it, and gear would fit
| in normal rack with bracket.
|
| Is there a place nearby for cabinet, table, or shelves? Those
| are good for hiding network gear.
| whalesalad wrote:
| I keep meaning to produce a post like this. My setup is not
| nearly as pretty though. Love to see the cannabis grow operation
| exhaust fan, haha. Very well done!
| f-securus wrote:
| I didn't see a filter on the exhaust fan. He is so thorough I'm
| sure he thought of it but I think he needs a filter to keep the
| fan running long term unless he is filtering the air into the
| room and that is good enough?
| monstermunch wrote:
| Nope, its ALL completely full of cat hair 24/7/365
| monstermunch wrote:
| I hope you do, we need more like it!
| sillysaurusx wrote:
| Thank you for encouraging others. You've earned a loyal fan.
|
| And thanks for sharing your knowledge. Infrastructure has
| been an off and on hobby, but it's hard to know how to take
| it to the next level. Or absurd levels (in the best possible
| way), in your case.
| adr1an wrote:
| To all the comments on energy usage, check this out! this person
| is using solar power, https://blog.networkprofile.org/17kw-
| enphase-solar-install/
| brandhout wrote:
| Yeah but the sun doesn't shine during the night.
| bcohen123 wrote:
| https://en.wikipedia.org/wiki/Battery
| brandhout wrote:
| I know but lead-acid and batteries aren't stellar for the
| environment either.
| saberience wrote:
| But why? I looked at the page but could find no understanding of
| what he gained from doing all this? I have a single wifi router
| and power everything through that and still have much faster
| bandwidth than the author does. It seems like a lot of energy,
| time, and effort (especially for maintenance) for something with
| minimal gains. I get 1 Gbit down and 500 mbit up and even in the
| worst signal rooms it only drops to 600 mbit down and 93 mbit
| up). Printers etc all work through wifi and backups are done to
| several redundant cloud services. No wires required, no expensive
| and energy guzzling servers needed, no patching or maintenance
| needed.
| snoman wrote:
| Still cheaper than golfing.
| orloffm wrote:
| And now add the noise question to the equation.
| chambored wrote:
| OP presumably enjoys it. There's your why.
| [deleted]
| 10g1k wrote:
| My home network: Wifi router. Laptop. External hard drive,
| encrypted. Ready to move at any time.
| hqsolomo wrote:
| Sync that data to cloud-backed storage and you would literally
| need nothing more than an Internet connection!
|
| Granted, that means you NEED an Internet connection... But it's
| as close to "walking data center" as anyone can get.
| darkwater wrote:
| When I worked with racks in datacenters, we usually put the
| network gear facing the other side of the servers, so there was
| no need to cross cables for the rack depth.
| jolux wrote:
| I just bought a Ubiquiti Dream Machine SE along with a U6
| Enterprise a few weeks ago and so far it's my favorite tech
| purchase ever. The management interface is years if not decades
| ahead of everything else I've used before.
|
| My only complaint is that there's no public API and thus no
| official Terraform providers.
| syntaxing wrote:
| I actually moved away from ubiquiti stuff to OPNsense + TP link
| AP. The firewalls rules on OPNsense makes a ton more sense and
| the plugins are pretty awesome.
| LeoPanthera wrote:
| You can have both, as I do. I use UniFi switches and APs
| controlled by a Cloud Key, but my router runs OPNsense
| because UniFi routers are pretty bad.
| syntaxing wrote:
| You can run a UniFi hardware controller through a plugin on
| OPNsense. Worked pretty well minus mongoDB conflicts with
| zenarmor. But I switched to omada APs because they had
| better range and speed.
| petepete wrote:
| I love my Dream Machine too. I bought it about six months ago
| and it's been solid and was really easy for me, a novice with
| advanced networking, to get going with vlans and firewall
| rules.
|
| I don't _need_ this setup at all, like I don 't _need_ cat6
| ports in every room, there 's definitely an element of it being
| a hobby. Maybe I just like flashing LEDs.
|
| Sonos gear still using STP was an unwanted pain, all part of
| the fun I guess.
| tw04 wrote:
| I would take Ruckus Unleashed over unifi all day long.
| Ubiquiti, unfortunately, feels like everything is constantly in
| beta, both hardware and software. Wait until they release a UDM
| SE v2 and abandon firmware on the UDM SE.
|
| The UDM in particular is a masterclass in how to upset all your
| customers. (coming from a previous all-in UBNT customer that
| had a first gen UDM Pro).
|
| Ebay Ruckus + OPNsense and my network has never been more
| stable and performant.
| jolux wrote:
| Yeah I've heard a lot of mixed things about their software
| support but I did a lot of research and people seem to say
| it's gotten a lot better over the past couple years. I
| certainly wouldn't say any of it feels like a beta, it's
| currently been up since I bought it two weeks ago with no
| issues.
| lotsofpulp wrote:
| How does Aruba InstantOn compare to Ruckus Unleashed?
| whalesalad wrote:
| The Unifi software is pretty incredible. I am not using their
| router though (well... an ER-4 but it does not share the same
| management mechanism) so I am missing out on a lot of the
| goodies.
| donutshop wrote:
| ERX here. The EdgeMax line is still rock solid and have
| incredible value for the price.
| whalesalad wrote:
| I see absolutely no reason to jump ship, it's one of the
| most solid pieces of infrastructure in my network.
| ErneX wrote:
| I wanted to replace my EdgeRouter Lite with one of these but
| sadly Unifi hasn't reached feature parity with their Edge line.
| The Dream Machines for example do not support multiple VLANs on
| the WAN port, when many ISPs use that configuration to deliver
| internet, VoIP and TV.
|
| I also read some bad stories of the Dream Machines unable to
| restart after a power loss.
| 404mm wrote:
| This is some HW NSFW!! Wow!
|
| I'm in a very similar situation when it comes connections, FW and
| needs. I'm about to get a backup line for internet and it's
| between Verizon (just like you) or Spectrum (same price, 500mbit
| down). What was your reason for VZW? Was ATT your only wired
| option?
| Summertimes wrote:
| What you using for out-of-band in case something goes wrong?
|
| My favorite is ZPE Systems HSR, comes with 2 LTE modems that
| supports GPS and I can run some containers on it too.
| imiric wrote:
| This is inspiring. Great work!
|
| My homelab is a mess of cables, and half-baked deployments
| running on old Supermicro and custom servers, that is way too
| noisy, and probably pulls a lot of power. I wouldn't know, as
| I've yet to track power usage... (One of those projects I never
| get around to ';-D)
|
| Everything's stacked on a Lack rack, even though I have a 24U
| rack in a closet doing nothing. This is partly because of space
| and cabling constraints, but _one day_ it will all be neat and
| tidy, clean and efficient. One of these days...
| monstermunch wrote:
| >My homelab is a mess of cables, and half-baked deployments
| running on old Supermicro and custom servers, that is way too
| noisy, and probably pulls a lot of power.
|
| I dunno man, sounds identical to mine...
| stn8188 wrote:
| Wow, and my wife says my network is complicated! :)
|
| In all seriousness, thanks for sharing, this is really
| incredible. I see a few similarities (fellow Harbor Freight
| shopper, ADS-B receiver)... but I took the mostly lazy way out
| and just use the TP Link Omada router, controller and access
| points. Works great for well over 50 wireless clients that we had
| at a recent BBQ. I particularly love your note about encrypted
| LoRa networks at the bottom there, I'll be interested in a
| follow-up on that topic. Thanks again!
| xpe wrote:
| Exactly. Those 50 wireless clients must be fed Internet while
| their 50 subjugated humans eat BBQ and prioritize device
| interactions over human ones. (I'm exaggerating of course! At
| least you are having gatherings -- better than many of us!)
| monstermunch wrote:
| Thanks!
|
| I've been very interested in the new TP-Link stuff, they have
| really come a long way. I don't know if I've really ever heard
| of much wrong with the TP-Link stuff
| applied_heat wrote:
| I have the tp-link Omada APs with controller running in
| docker on synology nas and am happy with it
| gchokov wrote:
| My first thought is - imagine hiring this guy to architect your
| corporate network. The cost would probably be 10x for hardware,
| for the "just in case" factor and coolness to the setup...
| Cthulhu_ wrote:
| For a corporate network I would love to have more "just in
| case". We had a day long wifi outage at the office once,
| productivity plummeted and the cost of that far outweighs the
| cost of buying and setting up back-up hardware.
| justapcgamer wrote:
| [dead]
| abwizz wrote:
| it's even more economical to buy more hardware than hire
| somebody to maintain it properly
| envyac wrote:
| Curious why you didn't stack them like...
|
| 1. 24p Patch Panel (Odd ports of switch) 2. 48p Switch 3. 24p
| Patch Panel (Even ports of switch) 4. 1u Cable Mgmt
|
| Then you could have bought 4-6" Cat 5e/6a pre-made patch cables
| to connect in the majority of your patch panel ports. The only
| cables that would traverse through the cable mgmt panel would be
| longer cables to the servers in your rack.
|
| I have a similar Cisco switch as your 24 port. I am not fan of
| how they arranged all the ports at the right side of the
| enclosure. Why not make the 24p one row along the top (or
| bottom), like the 48p with the 2nd row removed? It would be a lot
| easier for cable mgmt.
| monstermunch wrote:
| The reason its arranged like that is because it started as 1
| switch and 1 patch panel, and slowly evolved, so its not ideal,
| but I am fairly happy with the outcome
|
| If I could re-rack it all, I might make some changes. But that
| means turning everything off, and I'm not sure when, if ever, I
| want to do that
| petesergeant wrote:
| > But that means turning everything off, and I'm not sure
| when, if ever, I want to do that
|
| Decent robustness check
| greggyb wrote:
| I noticed this as well. The patch panels seem only to bring
| some wire to the front of the rack, only to immediately send
| the patched wire toward the back of the rack again.
| makach wrote:
| This is not a home network but a home data center. I am envious,
| but also glad that I don't have to deal with the maintenance of
| this setup.
| Fabricio20 wrote:
| This is a pretty cool setup! I'm personally working on getting a
| backup ISP connection going this week as well (two different isps
| with wildly different upstreams), since poorly configured ISP
| network issues have been plaguing me (random packet loss out of
| nowhere, PPPoE issues, scenic routing to companies that have POPs
| on my city, etc..) and I've pretty much exhausted all ISPs in my
| area and they all have some manner of weird issue. So two ISPs it
| is!
|
| My setup currently consists of a single server with an i3-12100F,
| which does all of nas + hosting (9TB array for windows image
| backup of all local machines over smb + Home Assistant +
| AgentDVR) and networking (dual ISP in, single gigabit out, all
| handled by systemd-networkd and ip + firewall), and several
| 1G-CAT6 runs connected to an unmanaged switch. Wireless as
| several random brand APs scattered around the house, all running
| on low power mode and being pretty much room-confined, 5GHz
| everywhere, had mesh before and my experience is it just sucks!
|
| Your post made me interested in those RIPE Atlas boxes.. I might
| see if they are willing to send one down here for me.. with two
| different ISPs on different upstreams I might have some
| interesting data.. and maybe use that raspberry-pi I have for a
| stratum gps server like you did as well... and maybe a weather
| station too.. down the rabbit hole I go again!
| edent wrote:
| Can someone explain what the multiple GPS receivers are for?
|
| I get using one for NTP.
|
| I guess a second for redundancy even though it is picking up the
| same satellites as the first.
|
| And the rest?
| defrost wrote:
| There are 2x Internet connections + racks - one in a house
| closet rack, the other in the garage.
|
| Garage rack has a NAS that mirrors storage of the closet rack.
|
| Garage rack has one Raspberry Pi GPS for time services.
|
| Closet rack has two (different) Raspberry Pi GPS for time
| services.
|
| Looking at it that way, if you accept the 2x Internet
| connections each being independant and acting as cross backups
| in case of ??? then it's just the second GPS on the closet rack
| that's "extra".
|
| That can likely be written up as a home experiment in messing
| about with a different Pi setup.
|
| It's overkill .. but often these home setups are there to keep
| current on new technology and different ways of doing the same
| thing .. in that light it's not uncommon to have multiple ways
| of achieving the same end just to gain knowledge on the pros |
| cons of different approaches.
| znpy wrote:
| Overkill... as the author says in the title.
| godman_8 wrote:
| Nice setup! I have a very similar Homelab minus the Generac (I
| regret not getting one before inflation kicked in, especially
| since I already have LNG to the home.)
|
| My only recommendation would be switching your virtualization
| over to Proxmox (LXC / KVM) and setting up an HA cluster with
| Ceph and MLAG. It's relatively easy and free and will give you a
| lot more features than plain ESXi and even free vSphere/vCenter.
| monstermunch wrote:
| Thanks!
|
| Yeah, the price on this genset I think has gone up around $4000
| since I bought it, not including the install
|
| I've been meaning to try Proxmox, but my day job heavily relies
| on ESXi, so its nice having something to mess with at home. I
| am also running vSphere with an Enterprise licence, so I get
| all the fancy stuff
| zoom6628 wrote:
| Ok seriously jealous of the setup. Have always wanted a rack -
| just because.
|
| I have way too many power cords under the desk but use fused
| plugs and boards. Want a small cabinet just to keep cables and
| devices away from small curious people.
| deepsun wrote:
| SpeedTest.net tests are useless, as ISPs give higher priority to
| traffic to known speed checkers.
|
| They can legally do that as we don't have Net Neutrality anymore.
| bonestamp2 wrote:
| https://speed.cloudflare.com/
|
| I like this one because your speed really depends on what
| you're downloading and this one tests some of those scenarios.
| cardsofinhuman wrote:
| Is this true? Do you have a link?
| hqsolomo wrote:
| Not coming in with proof but I have personally seen this- a
| buddy and I did some testing and the Ookla tests were the
| only ones we did that were much higher than the others.
|
| I'm sorry for not contributing with hard evidence but I've
| seen this behavior (though I never heard about it from anyone
| else before now)
| RajT88 wrote:
| I used to work for a small ISP. The boss specifically wanted
| speed tests to be exempted from traffic shaping.
| smegger001 wrote:
| Not surprised. I have long noted that when my my browser is
| strugeling to load tabs all i have to do is try connecting
| to speedtest.net for all open tabs to suddenly load and i
| have decent speed for the next 15-30 minutes.
| pixl97 wrote:
| Eh kinda... priority traffic shows the maximum potential speed
| of your link. The max speed of your link has nothing to do with
| the speed you'll get on any particular node on the internet.
| For example if the ISP has too little peering bandwidth to some
| other speed test.net host you will still be capped by that
| link.
|
| These days I rarely need large bandwidth to any single host,
| but have a lot of bandwidth use between many hosts including
| streaming applications. Bufferbloat is far more apt to be
| problematic under this workload than ISP priority, unless there
| is a bandwidth shortage on the local links.
| nfriedly wrote:
| That's the reason fast.com comes from the same servers as
| Netflix content - it's difficult to prioritize one without
| prioritizing the other.
| NoPicklez wrote:
| I believe that's why Netflix introduced the fast.com speedtest,
| which runs through Netflix servers. Reducing the likelihood
| that the test gets shaped.
| judge2020 wrote:
| The link - https://fast.com
|
| Cloudflare's - https://speed.cloudflare.com
| SushiHippie wrote:
| fast.com is the only browser based speedtest that shows the
| accurate results for me. The results from there resembles
| the download speeds I get when downloading files from
| different sites or installing upgrades through my package
| manager.
|
| Cloudflare's speedtest often performs worse for me.
|
| iperf3 - https://iperf.fr Is the most accurate CLI speed
| test I've found (depending on which server you choose).
| judge2020 wrote:
| What's "accurate" is going to be different for every
| website / destination network anyways. Unless you suspect
| there's some issue with the speedtest software running on
| the browser / target, it's showing you the accurate speed
| you get to Cloudflare or whatever server you're trying to
| pull from. If you just want to see what your ISP is
| limiting your bandwidth to, sure, you need to find a
| server that can hit that.
|
| The best speed test is to download some large linux ISOs
| with 100+ peers, so you max out your connection to tons
| of networks.
| SushiHippie wrote:
| "accurate" is for me the speed I have on average
| throughout the day no matter what endpoint and not the
| max througput. If I'm doing a speedtest, most of the time
| I just want to know if I'm now downloading file $x from
| $y what download duration can I expect.
|
| Downloading packages with $PACKAGE_MANAGER, downloading
| files from $CLOUD_STORAGE, downloading videos from
| youtube, downloading linux ISOs, receiving files through
| $INSTANT_MESSENGER, ... they all are very close to the
| speed I get with fast.com.
| tibbydudeza wrote:
| Overkill indeed - got just a single router but it is so flexible
| (TP Link Omada) and runs various VPN tunnels without rebooting or
| hanging.
|
| No fancy CPU just a dual core MIPS based MediaTek 7621AT - the
| key difference is the software, stable and well designed and not
| some hacked together version of OpenWRT and some old version of
| Linux from some random OEM that never get's updated after
| release.
| riley_dog wrote:
| This person is pretty average over on /r/homelab.
| monstermunch wrote:
| I've stopped posting there as much, when my colo post got
| removed for not being "in my house" i knew it was going
| downhill
| teddyh wrote:
| If you think a colo is a "homelab", then maybe _you've_ gone
| downhill.
| bazmattaz wrote:
| This is incredible. I still can't believe some people get 1gb
| symmetrical in their home. In the UK I'm stuck with 70mbs down
| and 6mbps up. Pitiful
| toast0 wrote:
| I had 1g symmetric in the bay area (thanks at&t) and it was
| nice, but am now around your speeds (85m/13m) and it's clearly
| worse, but not really terrible. Certainly not terrible enough
| to pay $50k+ install to get munifiber, even though I'd enjoy it
| a lot. Maybe if one of the ISPs on munifiber starts offering
| 10g to residences. Not that I need it, but it'd be fun.
| rjsw wrote:
| I'm in the UK, getting 500MB symmetrical installed tomorrow,
| could have ordered 900MB for PS2 per month more.
| stwr wrote:
| I'm in Spain and get 10Gbit symmetric which comes down to about
| 7000mbps up and down on speedtests to my airbnb (in a small
| town 30min away from a touristic hotspot) for 30 euro's. My
| home in Barcelona gets 1gbit symmetric for 25 euro.
|
| Competition here has gotten pretty good with lots of virtual
| carrier being allowed by law to use telefonicas ftth which has
| driven prices down.
|
| I'm the opposite to you where I'm always surprised when I hear
| people in fairly big towns/cities in US/UK etc are still on
| less then 100mbit symmetric!
| vel0city wrote:
| A chunk of AT&T's residential fiber actually supports 5Gbit
| symmetrical for ~$110/mo in the US.
| throwaway1777 wrote:
| I'm really hoping they upgrade my area some day but not
| counting on it.
| bashy wrote:
| Maybe search around. I got YouFibre[0] few months ago and it's
| 1000/1000.
|
| [0] https://www.youfibre.com
| monstermunch wrote:
| When I bought this house, a requirement was fiber internet. Not
| dealing with garbage cable!
| tdfirth wrote:
| I'm in the UK (Oxfordshire) and have a 1gb symmetrical
| connection at home. It's provided by Gigaclear - there's a
| handful of other similar operators that do fibre in more
| "rural" areas. It costs PS79 a month, so it's not cheap to be
| honest, but I love it.
| jonathantf2 wrote:
| Ring em up and tell them you're leaving, they'll drop you
| down to the new customer pricing. I've got the 1G up and down
| for something like PS38 a month now?
| PaulKeeble wrote:
| Should hopefully change in the next few years, about 54% of
| houses have fibre to the property and the plan is over the next
| five years to expand that to nearly 100%. I have had 100/1000
| for a while now but symmetric is still a rare product that only
| smaller competing fibre companies are rolling out.
| mpsprd wrote:
| The single thing I am most envious in this setup is the little
| box giving symmetrical 1Gbps from the ISP. I wish fiber could
| grow in remote areas...
| megraf wrote:
| I've been fascinated by home labbing for a while now, and
| maintain something with comical simplicity.
|
| ESXI 8.0.1 worked out of the box with the 7090 I was given.
|
| I'm migrating it from 6.7u3, which is running in 14 year old
| hardware (injecting the drivers was easy btw - finding the
| drivers was difficult).
|
| ESXi is not managed with vCenter, it's a standalone host. I'm
| hosting a handful of VM's that do everything I need.
|
| Critical items are never sorted solely on the server; they're
| stored on three remote sources, with integrity checks , and mock
| restorations.
|
| Fortigate 140d PoE, which handles a phone system and all of the
| PoE I need.
|
| Backups are simple. No RAID, again, critical stuff is not solely
| on the server.
|
| I use enterprise disks, and SMART - I'll get a notification with
| ntfy if something gets wonky.
|
| Server is consuming 60-160w. 16gb of RAM, non ECC, 5th gen i5,
| 12TB of storage.
|
| Homebridge, home assistant, scrypted, Asterisk, Samba,
| Timemachine backups (all containers configured with compose,
| checked into Git) windows 10 and 7 VM's
|
| All of the hardware is second hand, with exception to the disks.
| Ambient in the closet is 29, no dedicated cooling.
|
| UPS with cyberpower.
|
| Glad to see you're using solar, and that your hardware too is
| used. When I worked in the industry they'd toss stuff I could
| never afford...
|
| FWIW: Proxmox seemed like something new to learn entirely, ESXi
| seemed similar for me; I don't mind the RAID stance, or the
| licensing...
| fastball wrote:
| I don't know if it was just a lucky guess, but as soon as I saw
| that first pic of the house exterior with the BMW, I said "that
| looks like it is in Houston", and sure enough...
|
| Can someone with more knowledge comment on whether that
| particular design is more common in Houston or was it really just
| a lucky guess? I kinda feel like that carport-style design is
| very Houston, haven't seen it as much in other cities.
| bombcar wrote:
| Houston or Southern California was my guess, because I've seen
| things like that there, too.
|
| But I suspect it was popular in the south as a whole for a
| short time, that time being when major sections of Houston were
| built.
| rbc wrote:
| Much nicer than my setup. I have three small form factor PCs
| running OpenBSD tied together with a small Netgear gigabit
| switch. Each has an external drive for backups using rsync and
| dump. Copies of the backups are made to another drive for offsite
| storage. One computer is a workstation, the other two are
| servers. One server is for internal use, the other is for
| external services. WireGuard is used to tie the outward-facing
| server to an Internet hosted server so services can be passed
| between them. There's no UPS though. I didn't want to deal with
| battery maintenance... Power outages are taken when they occur.
| One thing about that, the computers bios are set to leave the
| computers off if the power blips. The whole thing sits on a
| single shelf of the bookcase.
| bmitc wrote:
| Is there a post that covers the "why?" of all this? As someone
| researching maybe running some Ethernet for more stable WiFi and
| faster Internet access for general stuff, _maybe_ some fiber for
| infotainment other purposes since it can lengthen various cables,
| and hooking up some security and nature cameras, I can 't fathom
| why all of this is needed outside of it just being a hobby or
| running some sort of business (trading?) at home. For example,
| isn't $600 for redundant Internet for like the minutes the
| Internet is out every year worth it? Why is that level of
| connectivity needed?
| Brajeshwar wrote:
| I have been working remotely since about 2005-2006. I have
| always love a good Internet connection with backups (wherever
| possible). I remember befriending the local cable guy so I can
| get 1Mbps in 2000.
|
| Now, I have three Internet connections bonded and balanced. It
| is not about the minutes of disconnection but more about the
| disconnection when I needed it most. I have ample non-internet
| time and personal/family downtime. However, when the time
| comes, I'm happy that the Internet is never a bottleneck in my
| work and play. I have had this setup since around the early
| days of the Pandemic and our home, per se, "never had an
| Internet Outage" since.
|
| I'm not into servers, devops, networking or anything of that
| sorts, but I love tinkering with them. I would love to have an
| "overkill home network" one day.
|
| Internet is super cheap in India and I can afford all three for
| a really decent price. https://www.instagram.com/p/CUWeopdPVOp/
| LeafItAlone wrote:
| Sounds pretty neat!
|
| Just wondering: 1. How often do you actually need the third?
| As in, both primary and secondary are down. 2. Is it that
| important that you always have an internet connection?
| Brajeshwar wrote:
| After the 2nd, it was more of a fun and why-not! It is
| cheap, and comes built-in with a free Netflix subscription
| and quite a few others. I think it even has an unlimited
| voice call if I want to plug in a phone and use it.
| wesapien wrote:
| Most of the time it's the IT Infrastructure, Operations,
| Security guys that love to do this for (WHY) learning, testing
| and as a hobby. Check out r/homelab and r/selfhosted for more
| on this.
| hqsolomo wrote:
| Servethehome forums and smallnetbuilders are also good
| resources, if you're not a fan of Reddit for some reason!
| hqsolomo wrote:
| Speaking personally, it's just another form of tinkering.
| Nobody HAS to do it, much like nobody has to buy and maintain
| their own computer, car, home, etc... We do it because we can
| and it makes us happy.
|
| Extra note- I would NEVER recommend a business use their home
| resources for work unless you know what you're doing (and why
| you're doing it). Even my homelab has vulnerabilities and I'm a
| security professional- more moving parts = much larger attack
| surface. There's a reason why corporations pay big buck$ for
| managed security services. Most homelabs I've seen are mostly
| for fun and personal comfort.
|
| "What good is knowledge that is never applied" is what drives
| me to stuff like this. I can't speak for others but I'm sure
| I'm not the minority here.
| fjfuvucucuc wrote:
| Pretty much this. I wanted to learn about networking and
| server management that I no longer get to do as part of my
| day job.
| bmitc wrote:
| Gotcha. Thanks for the reply! To be clear, it wasn't a
| judgment but just a question to understand there weren't some
| unapparent reasons why. I think it makes total sense as a
| hobby and learning, but perhaps still overkill as the post
| title mentions. Haha.
| hqsolomo wrote:
| I apologize if it seemed like I was irate- I'm not and your
| question is 100% the question we all ask ourselves before
| doing things like this (or for some of us, it's the first
| question on the wife acceptance factor audit). If anyone is
| running a data center in their home for a serious reason
| they either have small loads to justify the power
| consumption (rpi k8s cluster says hi), stacks to blow,
| regulatory pressure, or isn't factoring the costs in and is
| in for a rude awakening. I don't think (but also don't
| know) these labbers are the majority, and us homelabbers
| are already a rarity.
|
| If it makes you feel better, megacorps are getting out of
| the "self-managed data center" industry and embracing the
| cloud, to exemplify your very point.
| bmitc wrote:
| Lol. I was under zero impression that you were irate or
| anywhere close. My question was a little judgemental
| perhaps but not necessarily meant in any way. I was also
| curious if there was some interesting need that had come
| up. My primary thing that I want is a bunch of PoE
| powered nature cameras, buy I'm still figuring that out.
| It will affect whatever comes up though. Oh, and stable
| WiFi coverage.
|
| I certainly have hobbies that go above any need or
| reasonable collection, namely synths and books. Haha.
| hqsolomo wrote:
| It's interesting that you mention PoE nature cams- I
| designed a PoE home surveillance system for a friend that
| involved setting up a solar panel on a 30' pole that fed
| a box with a shitty camera system in it at ground level.
| From there he set the cams around his property-
| particularly where the foxes and coyotes would travel to
| get to his hens. The whole project was apparently about
| $800 aside from the solar panel (I just gave him the
| idea- I didn't help him build it).
|
| He eventually got rid of the cameras because, well...
| They were shitty and only told him the critters were near
| AFTER he popped 'em. I think he's got a for-purpose
| system (in his own words "the new cameras didn't fall off
| a truck") now but it was a fun project!
| resonious wrote:
| I still think your question is reasonable. Even when
| someone is doing something for joy, there's usually a spark
| - an essence to it. There's usually an initial motivation
| that sent them down the rabbit hole, and it's interesting
| to hear what it was.
| csomar wrote:
| There is no "why". Some people like boats. Some people like
| servers. People spend money and time on things they like.
| comprev wrote:
| For the same reason some people cycle around the world instead
| of just a casual ride on Sunday morning like many cyclists.
|
| Some tech people enjoy "homelabbing" and happily throw money at
| their hobby.
| frankreyes wrote:
| I'd say similar like sport sailing, but yeah there's some
| bikes that cost more than a sailboat.
| comprev wrote:
| Being out on the water is such a peaceful experience.
| thombles wrote:
| Take too long to winch in the jib and it's another matter
| though. :)
| j45 wrote:
| It's a good question, as long as the why is not a burden to be
| met.
|
| First it's a useful skill set that helps troubleshoot problems
| in code and apps that run on networks by knowing how such
| things work.
|
| Second, try to think of it as a private cloud instead of a
| network. Because it uses a lot of the same types of software
| cloud provider do. Proxmox is one example that is a self hosted
| vps provider that is tremendous. So, it's a private hybrid
| cloud that can push from your cloud to the other voids (or
| back). Build using a private hybrid cloud and you can push
| to,or between manu clouds
|
| Wifi is for convenience, wired is for reliability. When wifi
| gets jammed, spotty, interfered with which can happen more
| often than imagined, especially during break ins, wired wins.
| Transferring files? Wired wins. 4K streaming ther doesn't cut
| out when multiple devices are doing it at once? Wired wins.
|
| Self-hosting is much easier than it was 5,10,15 years ago.
| Tools like proxmox loaded with yunohost running on a 1L
| usff/mini/tiny pc with 64 gb of ram and mirrored ssds ther
| maybe mirror to another i de oval box can sip power but power
| production grade apps for you personally. If you buy big beefy
| servers it will cost a bit more electricity.
|
| Still, hosting locally can quickly pile up the savings on saas
| not spent.
|
| Imagine being able to keep up all your test projects you might
| spin up a paid VPS for and run for way too long. They might not
| be production grade, but there is something valuable about
| having them around.
|
| Data backups - the cloud is just someone else's computer sold
| as convenient but not secure. Having your own local copy when
| the internet goes down means not as much of your life or work
| goes down. Remember it's not a backup if there is only one copy
| of it.
|
| Backing up your computers in the cloud are only so helpful when
| they are down, a lot of time to download them. Local backups
| win again.
|
| Multiple connections can matter for people who wfh and can't be
| down, or have spotty internet that is up.
|
| If you like smart home tech, it's a ticking clock until the
| cloud based supports for it that are usually free dissapear
| leaving perfectly good gear unusable. Instead you can run a
| local instance of home assistant, etc.
|
| I used to host morethan I wanted ina. Data centre much like
| this rack. I was hesitant to come back to self hosting or
| homelabs but j have realized a home server that runs like an
| appliance (in between that wiring) capturing the sum of all my
| data and worn as services come and was pretty much unavoidable.
| Luckily it's getting easier and easier to do.
|
| Hope that helps.
| Emjayen wrote:
| Although not as elaborate as this, I have what one would call a
| "homelab" and it's for, well, testing and experimenting -- I
| write a lot of high-performance server/network-centric software
| (e.g, saturating a 100G link with 64B frames is a common test)
| and virtual machines just aren't suitable/capable most of the
| time.
| theossuary wrote:
| There's a reason other than "because it's fun to tinker." I do
| it because I love the capabilities it provides. I don't love
| managing Ceph & Proxmox & whatnot, but I love being able to
| deploy whatever I want into a beefy cluster with 10gb without
| having to worry about cloudspend. If I wanted to replicate what
| I run in my home infra (I don't consider it a lab), it'd cost
| 200-300 in compute and easily another 300 in storage a month.
| Instead I spent ~2k on hardware and ~35/month in electricity.
|
| It's the same thing with 3d printers, some like them because
| they want to install clipper and tune the best/fastest benchy.
| I do it because they love being able to CAD something and have
| it in my hands as soon as possible.
|
| That said, nobody goes as far as this guy without really
| enjoying the tinkering
| hqsolomo wrote:
| Very well said- good catch! That's a very compelling reason!
| bmitc wrote:
| What do you run on your cluster? CAD compute?
| theossuary wrote:
| No, but that'd be neat! I don't have GPUs in my cluster
| though.
|
| My big services are a Ceph cluster, a VPN, Borg (backups)
| and a K8s cluster (using kubespray).
|
| In K8s I have the main stuff: Plex, Gitlab, Gitlab runner
| (for CI), vaultwarden (passwords), miniflux (RSS, might be
| moving to freshrss though), rust desk (remote-desktop),
| home assistant (smarthome).
|
| My next project is to stream metrics into grafana. I have
| soil moisture sensors in my garden connected to stm32
| boards, I just need to setup the receiving side and I can
| control my drip lines (they're using opensprinkler) with
| soil moisture information along with weather info.
| heavyset_go wrote:
| It's a hobby that can pay dividends in multiple ways, don't
| really have to look deeper than that.
| gumby wrote:
| Why not? I had a pretty elaborate setup in the house I sold
| during covid (fortunately the purchaser was enthusiastic about
| it) with a wiring closet in each wing and a fibre spine running
| between them. Not at all as elaborate an external connection as
| the author but I had a lot of machines and wanted to keep
| wireless bandwidth for things that moved around.
|
| One handy trick: I had at least one drop in each room, often
| one for each wall of the room. Behind the wallplate was a NEMA
| box with a conduit running straight down to the crawlspace (6'
| high in my house so hardly "crawl"). That made it easy to pull
| cable, not only initially but if I found I needed an extra run,
| which hardly ever happened. Instead of trying to run through a
| rat's nest of conduit the conduit was a straight shot and then
| the cables could easily be managed.
| cortesoft wrote:
| This is clearly the guys hobby. He does it for the act itself.
| m463 wrote:
| Setting up a home network, especially with cisco gear, can
| easily pay for itself 1000x in sysadmin or network engineer
| employment opportunities.
|
| This is exactly the kind of nuts and bolts guy that is
| indispensable when keeping a cloud running.
| lostlogin wrote:
| Another personal anecdote.
|
| Wifi is rock solid, networking is fast, backups good and
| outages are almost nonexistent and are always due to me messing
| with things. I have Netflix, Apple TV+ (is that what it's
| called?) and a few others. Particularly with Netflix, the
| quality is junk so I watch off Plex.
|
| Other benefits are pretty neat too. POE just works. You can
| power cycle things remotely if you want (have never needed to)
| and the abundant local storage makes everything easy.
|
| Enterprise stuff that is getting old is also dirt cheap.
| Converting sections of the network to 10gbe was very
| inexpensive.
|
| It's all rather addictive...
| syntaxing wrote:
| I mean, "need" is a pretty strong word. No one needs a MacBook
| Pro, you can do everything on a Raspberry Pi just slower
| (exaggerating but you get the point). Personally, having
| reliable internet has been a significant increase in quality of
| life but admittedly, mesh routers are getting closer and
| closer. The biggest draw are VLANs. When even the FBI
| recommends separate networks, it shows how prevalent these
| issues are. I know a bunch of people that got affected by
| cryptolocker. While having good practice is probably what helps
| me a lot, a hardened network helps tremendously.
| matthew-wegner wrote:
| I built a pretty serious "homelab" over a few years, in the
| sense of building out 24/7 services as robustly as possible.
| There is an element of practicality for very long-term
| maintenance if you run i.e. VMs off a different storage layer.
|
| But mostly? It felt exactly like exploring computers in my
| childhood for the first time (I'm 43 now). I was bumping into
| things I had never bumped into it before, and it was really
| satisfying to figure all that stuff out.
|
| I'm a game developer, too, and I realized awhile ago a big part
| of that is I like technical puzzles, and game development is
| very fertile soil. But there is a whole new world of enterprise
| networking/storage/virtualization/etc things that you wouldn't
| normally bump into in the course of software development.
|
| And as a bonus, I have set up a lot of build systems for
| friends; being infrastructure-savvy is like the digital
| equivalent of owning a truck and helping people move all the
| time.
| fzeindl wrote:
| > There is an element of practicality for very long-term
| maintenance if you run i.e. VMs off a different storage
| layer.
|
| That is very good advice. I know about a datacenter
| outage/degradation in a very large company, which all of you
| know.
|
| A couple of network switches got overloaded. The switches
| connected the SAN to the VMs, so it caused all kinds of weird
| problems in different applications.
| nanidin wrote:
| AWS or cloudflare?
| groby_b wrote:
| This depends on your area, and your ISP.
|
| If, say, you use AT&T and you need Internet for your job
| (WFH?), redundant Internet is crucial. It's not exactly
| "minutes" they are out. (Their outages states are often "meh,
| we'll get around to it in a day or so", figuratively)
|
| If you live in a rural area, even more so. You're not high
| priority for fixes in the first place. (And AIUI, the author
| lives somewhere in rural Texas)
|
| Do you need to run it as high tech as that? Probably not. My
| alternate Internet is my cell, tethered. (And if that breaks
| e.g. during travel, yes, I have a second cell with a separate
| provider, because heaven forbid the US had consistent cell
| coverage)
|
| And most of us probably don't give a damn about e.g. 1,000 year
| photo retention. Or many of the other things he's doing. But it
| sure is fun if it appeals to your personality, and you can
| afford to keep it running.
| rsecora wrote:
| Cool
|
| but is astonishing how expensive is the connectivity in other
| countries.
|
| >>> Verizon Gateway sitting on top of the rack. I get pretty good
| signal here and get the rated speed, of around 300Mb/s down and
| 20Mb/s up. This costs $50/mo.
|
| There are offers [0] in the EU of 10Gbps / 25 Euros/mo.
|
| [0] https://www.digimobil.es/fibra-optica/
| ListenLinda wrote:
| I knew this wasn't an overkill network when I saw an SG300 switch
| in the first pic.
| monstermunch wrote:
| Call be crazy, but I dig the SG switches
| systems_glitch wrote:
| Very nice setup, thanks for documenting it! I used to have a much
| more involved at-home network, but since moving the business out
| of the garage, I've moved most of the network stuff with it.
| Still think I'm going to have to pay pole attach fees and string
| my own fiber if I ever want it at the house...
| monstermunch wrote:
| We want to see you run that fiber! Do post about it if you ever
| do it
| NKosmatos wrote:
| Come on, let's admit it fellow HNers... Most of us are
| jealous/envious of this network setup and would love to have the
| money and time needed in order to own something similar. Extra
| points for powering all these from solar panels and also for all
| the "little" gizmos (ADB, NTP, LoRa, TinyPilot, MQTT and
| temp/humidity/power monitoring).
|
| @monstermunch You're living the dream :-)
| doublerabbit wrote:
| I'm more interested in my colocation setup than my home setup.
| I have 2x1u in two different data centres.
|
| Just built one of my new servers (Cisco UCS 220 M5) with 2x 4TB
| Enterprise NVMe drives, 4x 2TB enterprise SAS SSD drives and 4x
| 2TB SSD drives, 256GB of ram and 2x Intel Gold 6230 (20cores).
| I don't even need this, I just want it.
|
| Just deciding on if to keep with FreeBSD and bHyve which I
| love, or go with VMware.
|
| If owning an IP block wasn't EURLOL, I'd complete my monopoly
| set.
|
| (Sorry, had to brag somewhere)
| Qwertious wrote:
| Owning an IP block is baaically free, with IPv6. Pity that
| network effects (pun not intended) make that impractical
| right now.
| [deleted]
| wink wrote:
| I'd guess a good portion of us have been at that point in our
| lives but has moved on and downsized or shudders at the thought
| of keeping that monstrosity of a rack in service or updated :P
| joezydeco wrote:
| And not a single mention of the noise level in this room. Those
| rack servers get insanely loud.
|
| I had a single SuperMicro in the basement one summer and it had
| to go after a month or so. You could hear it everywhere.
| throwawaylinux wrote:
| It's very cool, I just don't know what I would ever do with it.
| And I'm not saying that means therefore I don't see why anybody
| else would need or want it, I would never use it though. I was
| thinking about running an ethernet cable or two around the
| house to hook up a 1gb link but I couldn't be assed in the end
| because wifi is good enough. It's not like I can max out my
| 11mb/1mb home internet. \My new laptop's wifi is a bit flaky so
| maybe I'll do it this year if the drivers and firmware don't
| come good.
|
| Cameras and weather station are nice though.
| saberience wrote:
| I'm definitely not jealous, it just looks like an annoying pain
| in the ass. I have a faster network than the OP and it's
| cheaper and wireless and I don't have any servers or cable
| everywhere nor maintenance to do...
| wil421 wrote:
| I wouldn't want to read a blog about your setup.
| snoman wrote:
| You're running faster than 10Gbps on wireless? How's that?
| ElectricalUnion wrote:
| It's probably running several aggregated 60GHz links, or
| it's own private 5G millimeter wave deployment.
| snoman wrote:
| That's still not going to crack 10Gbps.
| larschdk wrote:
| I'm not, even though I also DIY. A multi-2.5GBe port fanless
| N6005 mini-PC (plus a cheap barebones WiFi AP) does it for
| router, switch, storage, game server (Valheim & Satisfactory),
| timeserver (chrony), DNS (unbound with blocklist), InfluxDB,
| MQTT, and it doesn't even sweat, and consumes less than 20W
| combined.
| kilroy123 wrote:
| I'm definitely jealous. I think it's ridiculous and awesome at
| the same time. I imagine it uses a lot of electricity though.
| cyounkins wrote:
| Anyone else have experiences with Ruckus APs? I have Unifi right
| now.
|
| edit:
|
| RUCKUS R760 - $1800 - Wi-Fi 6E 4x4:4
|
| RUCKUS R560 - $950 - Wi-Fi 6E 2x2:2
|
| RUCKUS R750 - $820 - Wi-Fi 6 4x4:4
|
| RUCKUS R650 - $520 - Wi-Fi 6 4x4:4
| VTimofeenko wrote:
| I have been running Ruckus I got off ebay for quite a while
| now. Flashed it into unleashed mode and it has been rock solid.
| In fact I am typing this comment while being about 100 ft away
| from the AP.
| monstermunch wrote:
| Buy used off eBay 100%! Don't pay those prices
|
| Even a few year old AP will work great. My R510 is pretty old,
| but works flawlessly
| hqsolomo wrote:
| Nice and clean, great work!
|
| If you don't mind me asking, does your energy bill take a huge
| blow because of this? I had a modest homelab set up and had to
| start shutting things off due to how much it costs to keep it
| running
|
| I apologize if I missed this info in the blog!
| pm2222 wrote:
| Perhaps there's solar.
| hqsolomo wrote:
| That would definitely be a big help, lol. I also imagine part
| of my problem is my house wiring- the previous owner had the
| place reno'd in '96 and clearly got the landlord special
| monstermunch wrote:
| Correct, 17kw of solar on the roof
| phito wrote:
| I had bought an used enterprise server (Dell something), but it
| just used too much power and made some much noise. It's not as
| professional or reliable as op's homelab, but I'm now hosting
| most of my stuff on an Intel NUC. It's a much better fit for my
| use case and budget
| dheera wrote:
| Having a rack alone doesn't consume anything, it's what you put
| on the rack.
|
| I have a rack setup and most of the time it consumes around 200
| watts during the daytime and 100 watts at night, but can spike
| upto 600 watts if I put a heavy CPU+GPU load on it.
|
| I also put my desktop into suspend at night, something which I
| think a lot more people with desktops could do. Don't run 24/7
| services (e.g. Home Assistant) on your massive desktop with an
| i7/i9 and a GPU. Run that stuff on a NUC or Pi4 or anything
| that has low power consumption. Then turn your desktop on only
| when you're actually using it.
| hqsolomo wrote:
| I get that but the blog discussed a pretty beefy setup. My
| whole rack still used less power than my gaming PC at load
| but after doing the math I ultimately saved more money by
| going serverless for my apps and dumping (non-critical) data
| into a B2 bucket on paper. In reality I just started shutting
| things off and only turn them on when I need them.
|
| I'm curious as to what others are doing to save costs if
| anything. I love the hobby but we're in a recession, lol!
| CTDOCodebases wrote:
| If you don't require ECC RAM a common setup these days is a
| Proxmox cluster set up on refurbished USFF PCs like the
| Lenovo m720. They sip power at idle.
|
| My personal approach is separate archived storage from
| working storage and keep archive storage offline until I
| need it. Keeping hard drives spinning is costly.
| getwiththeprog wrote:
| MiniITX boards are low idle power, 17 watts for idle with a
| Ryzen 5600 compared to a full size boards 50 watts. I would
| no longer buy anything larger, even for a gaming machine.
| dheera wrote:
| I really wish more Mini-ITX boards came out that allowed
| GPUs to be horizontal so it could fit in a 2U case.
| Unfortunately making it vertical makes it 4U, negating
| the size advantages of Mini-ITX.
| hqsolomo wrote:
| So I actually "repurposed" a riser cable from an old ITX
| build to slap it into my primary proxmox server to play
| around with home-rolled VDI. Surprisingly, it worked. If
| you're feeling bold you could try that route
| hqsolomo wrote:
| Amen on that. The past few rigs I built for myself were
| all ITX machines. I went through and undervolted my CPU,
| RAM, and GPU. Once my current gaming rig dies I'm just
| going to stick with the consoles for gaming until they
| goes to shit as well.
|
| In an era of abundance we seem to be trying to use more
| power to support what we already do, rather than
| rethinking what we do to use less power. It's a shame!
| dheera wrote:
| Honestly I could I could easily solar power all of my
| compute needs (and probably a lot more) from my
| apartment's roof.
|
| But at last, I'm a renter and my property management
| isn't going to allow that.
|
| We're literally wasting clean energy because of a bunch
| of rich turds hogging real estate with shitty rules about
| what you can't do. When my property changed management
| the new management even _banned_ EV charging on the
| property and locked up all the outlets in the parking
| lot. These kind of management are not welcome in Silicon
| Valley but somehow they are here.
| hqsolomo wrote:
| It jumbles my jimmies hearing about outright rejection of
| modern tech by industries that would best benefit from
| said tech.
|
| I'd almost be willing to bet money that your landlord
| would charge a "renewable energy" fee even if they're
| just dumping power back to the grid for $$$ even if they
| had the panels
| monstermunch wrote:
| I have 17kw of solar, so usually very close to $0 or below!
| hqsolomo wrote:
| Holy smokes, living the dream! Kudos!
| momirlan wrote:
| energy bill might go up, but the heating bill goes down ...
| hqsolomo wrote:
| "Chestnuts roasting on an open server rack"
| RajT88 wrote:
| Great for the arctic circle.
| monstermunch wrote:
| We don't do heating here in Houston. Maybe for the 2 days of
| Winter
| ihattendorf wrote:
| > Maybe for the 2 days of Winter
|
| That's what the generator is for.
___________________________________________________________________
(page generated 2023-08-11 23:02 UTC)