[HN Gopher] Making a Linux-managed network switch
___________________________________________________________________
Making a Linux-managed network switch
Author : _Microft
Score : 118 points
Date : 2024-07-03 14:47 UTC (8 hours ago)
(HTM) web link (blog.brixit.nl)
(TXT) w3m dump (blog.brixit.nl)
| teddyh wrote:
| Does it support rSTP?
| MartijnBraam wrote:
| I don't think this support it, only regular STP and apparently
| MSTP
| teddyh wrote:
| > _and apparently MSTP_
|
| Where did you find that?
| gerdesj wrote:
| STP is "slow" STP, and RSTP is a degenerate case of MSTP!
|
| MSTP is Multiple spanning trees ie you can group VLANs and
| prefer paths for those groups of VLANs. That means if you
| have say two links between two bridges (switches) you can
| prefer some to use one link and the rest to use the other,
| that means you are not "wasting" a standby link. They will
| fail over to the surviving link on failure. STP and RSTP
| will only consider one link as a whole, so two ports are
| "wasted" when not in use: in the case of a two bridge, two
| links example.
|
| Old school STP without the Rapid part hasn't really been a
| thing for several decades. I can't think of why you
| wouldn't use RSTP in general but if you need to make best
| use of your forwarding capacity then a 50/50 MSTP may be
| indicated. That's where you look at your traffic flows
| across VLANs and try to bundle them up into a 50/50%
| collection. One lot prefers link A and the rest get link B.
| Obviously you can get really creative as the number of
| VLANs and links mount up. Bear in mind that dot1Q is a
| simple version of QinQ!
|
| Sorry, got a bit carried away there.
|
| For nearly all intents and purposes, RSTP _is_ STP. If you
| plug in a network cable between two devices and it does not
| start working within say five seconds then you are living
| in the 1990s.
| thelastparadise wrote:
| What's the advantage of doing this over plugging multiple gigabit
| adapters into a linux machine and adding them all to a bridge?
|
| I'm guessing performance might be better with the hardware, but I
| don't know --has anyone done tests to show the difference?
| MartijnBraam wrote:
| This is more efficient if most of the traffic will get
| switched, seperate adapters is more efficient if most of it
| needs to be routed.
| simcop2387 wrote:
| It'll be a combination of:
|
| 1. Throughput - say you use usb adapters, in a lot of ways usb
| is a shared bus so you'll run into max bandwidth quickly. This
| is especially because data will have to go in and then out, all
| the way to the cpu
|
| 2. Latency - because you're using software to do switching,
| it'll add time to process each packet and send it back out tne
| right place. You've also go any other interface latency adding
| to it
|
| 3. Power usage - eacj adapter will have it's own full network
| phy and hardware, which will increase the power draw. Combined
| with all the extra processing above and now your power usage is
| even higher. This means you also loae out on hardware
| offloading and other performance enhancememts that generally
| reduce power usage because less of the system is involved in
| mocing packets around
|
| 4. Features (potentially) - this will depend a lot on the
| hardware you choose, some of those cheap gigabit usb adapters
| i've tried didn't work with vlans and other features properly.
| But if you say load up a bunch of nice pcie cards with 1 or
| more ports thst support everything (never had issues with pcie
| ones) then you can now actually get a lot of features that are
| otherwise difficult or impossible on simpler hardware (though
| at that point you're doing routing more tham switching, but
| thay flexibility is why you'd potentially do this).
| Fnoord wrote:
| We're only talking 1 gbit ethernet here, so you can have
| multiple of those ports on PCIe.
|
| I have a PCIe card here with two 2.5 gbit on it (don't
| remember exactly how much it was on Ali but between 20 and 40
| EUR) and I can saturate both with iperf3. Since the example
| only uses 4 ports, it should be easy to make a simple router
| with just two PCIe cards. But there's probably 4x 1 gbit PCIe
| out there, too. And if you use 1 gbit fiber, that wouldn't
| cost much power nor would it need much speed. If your uplink
| is DSL, you could use a VigorNIC 132.
| adrian_b wrote:
| For such a low speed and small number of ports using a hardware
| switch circuit is not necessary.
|
| Nevertheless, if the designing and building effort is
| neglected, I assume that the total cost of the hardware might
| be under $100, which is less than a computer with multiple
| interfaces would cost.
|
| Still, 1 Gb/s networks are rather obsolete. One could make a
| managed network switch that is bigger and faster by using only
| off-the-shelf components for slightly more than $200, e.g. a
| 6-port 2.5 Gb/s switch can be made with an Odroid H4+ having 2
| ports, together with its add-on M.2 card with 4 extra Ethernet
| ports. Another variant is to use a small computer with an N100
| CPU and 4 2.5 Gb/s ports, which can be bought in this price
| range from various Chinese companies. Similar small computers
| with 6 2.5 Gb/s ports are a little more expensive, perhaps
| slightly over $300.
| MartijnBraam wrote:
| $100? I paid roughly $80 for two of these and the price per
| boards will rapidly converge to ~$17 when increasing
| quantity.
| adrian_b wrote:
| The PINE A64-LTS alone is $40 + shipping + taxes, so you
| cannot have a $17 total cost.
|
| I assume that you mean that the card with the switch
| circuit alone could reach $17, if made in a large quantity.
|
| However, you cannot make a managed switch with that card
| alone.
|
| If your current cost has been $40 for a card with the
| switch circuit, adding the PINE A64-LTS + whatever you have
| paid for shipping and taxes must make the total around
| $100, exactly like I have said.
| MartijnBraam wrote:
| I meant the cost of just the switch obviously... It can
| hook up to whatever Linux machine you have running and I
| happened to have this one in the drawer.
|
| The point of this was not to make the fastest or cheapest
| or most featureful switch available, it just had to fit
| in a case and none of the options have one port facing
| backwards so it doesn't require an ugly loop cable on the
| front of the final case.
|
| It's also possible to make a managed switch with that
| card alone, there is a footprint for a NOR flash chip, if
| you load the netgear GS105E firmware on that (which is
| available on the netgear website) then it will be just a
| GS105E without leds and one port on the back.
| mschuster91 wrote:
| > Still, 1 Gb/s networks are rather obsolete.
|
| Are they? Most consumer, even office gear still is maximum
| 1000M/port - your average USB-C network/multi-port laptop
| dongle, most USB-C/TB monitors (and shamefully, Apple's
| Studio Display which only has USB-C/TB ports, no network),
| VoIP phones (hell these are usually 10/100 only, with 1000
| being reserved for top models), printers, virtually all entry
| to mid range NAS systems... the list is endless.
|
| Options for more than that tend to be really niche,
| expensive, or are bring-your-own-module-SFP.
|
| Besides, 1000M is way more than enough for almost all
| consumer and office needs. Only exception is heavy video and
| photo editing, if these workplaces don't already use direct
| Thunderbolt/FC attach.
| bongodongobob wrote:
| They definitely aren't, no idea what parent is talking
| about. They are the standard unless you're talking about a
| SAN or something. If I had to guess, 92% of the switches in
| a typical office are 1000M, 5% are 100M, and the rest are >
| 1000M for switch interconnects or HA server
| interconnects/SANs.
| Asmod4n wrote:
| A raspberry pi 400 can handle around 875 MBit when bridging
| two interfaces. So it's not even fast enough for two ports
| let alone full duplex. I doubt a n100 can handle more than
| three.
| bongodongobob wrote:
| 1000M networks are THE standard, no idea where you got the
| idea they were obsolete.
| KeplerBoy wrote:
| Unfortunately gigabit ethernet is far from obsolete.
|
| Yes, there's 2.5 gigabit on some consumer hardware, but it's
| still kind of rare.
|
| Also who is excited about a 2.5x speedup after 20 years?
| Nobody cares until we need 10 gigabit internet access (which
| will probably never happen).
| toast0 wrote:
| As everyone else is saying, power usage should be less this
| way. Probably less latency to traverse the switch than a
| software bridge, too. Switching should continue to function if
| the host OS crashes, combined with a watchdog and recovery, you
| could have a more available system where maybe some things
| don't work for a brief interval, but much better than a
| software bridge (assuming the switch chip doesn't crash or get
| stuck, anyway).
|
| It depends on what your goals are though. If you want to
| inspect all traffic passing through the switch, having 4
| interfaces is clearly better. If your host based switch is also
| doing a lot of communication, 4 interfaces gives you 4gbps from
| the hkst rather than 1gbps. Etc.
|
| Ex-enterprise quad 1G cards are $15 or less on ebay. I'm
| partial to silicom quad bypass 1g_ PEG4BPI-SD; the bypass
| feature can be fun, and they're cheaper cause they're weird
| (you can mostly configure them to be 'standard nics' once and
| then plug them into anything without much fuss, but getting
| there can be challenging. Early ones come with pci ids set to
| silicom as the vendor _and_ subvendor which makes them harder
| to use; the -SD cards have intel vendor id and silicom
| subvendor, so the normal driver will attach.
|
| 4x10g ports would be more to manage, and you might not have
| enough throughput for software bridging, depending on the host
| system. And quad port 10g cards are harder to find. 2x10G is
| reasonably priced though, if you're patient.
| buccal wrote:
| Interesting project.
|
| For simpler use you can get an OpenWRT capable router which in
| most cases uses a managed switch chip. OpenWRT provides a nice
| interface to configure VLANs and other options.
| tomatocracy wrote:
| OpenWRT these days can also be installed on some switches eg
| the Zyxel GS1900 series (though support for things like PoE and
| 10Gb/SFP+ ports might be limited, I'm not sure where things
| stand there).
| stragies wrote:
| Looking at the OpenWrt forum, work is coming along nicely
| w.r.t supporting FasterThan1G on RTL9X. Many/Most
| configurations seem to be working.
|
| RTL8X is AFAIC done, feature-complete.
| https://svanheule.net/switches/
|
| POE on those devices is mostly two types: Broadcom (well
| supported), and Realteks inhouse solution, which uses a
| 'dialect' of the Broadcom protocol. There is a git branch/PR,
| where the 'dialect' differences have been moved to individual
| modules. But it's not released yet.
| https://github.com/Hurricos/realtek-poe/pull/35
|
| Unfortunately, there seem to be hard problems migrating up
| from Linux 5.15 to 6.1 or 6.6.
| znpy wrote:
| WTF, i had no idea that was possible. I have that exact
| switch (GS1900-24E)... I need to look into that
| stragies wrote:
| Install mosquitto afterwards, add a small script, and the
| ports all show up in HomeAssistant as power-monitoring
| ON/OFF switches. :)
| stragies wrote:
| Nice article, Thank you for the write-up.
|
| First time I saw somebody 'creatively using' an RTL83something
| switch was https://spritesmods.com/?art=rtl8366sb, and there were
| others since then, but yours was the first 'build my own managed
| switch', instead of 'adding an external brain to an unmanaged
| switch'
| MartijnBraam wrote:
| Ah yes the legendary sprite_tm, I've come across this while
| writing my own raspberry pi firmware to manage the switch over
| USB.
___________________________________________________________________
(page generated 2024-07-03 23:00 UTC)