[HN Gopher] Making a Linux-managed network switch
       ___________________________________________________________________
        
       Making a Linux-managed network switch
        
       Author : _Microft
       Score  : 239 points
       Date   : 2024-07-03 14:47 UTC (1 days ago)
        
 (HTM) web link (blog.brixit.nl)
 (TXT) w3m dump (blog.brixit.nl)
        
       | teddyh wrote:
       | Does it support rSTP?
        
         | MartijnBraam wrote:
         | I don't think this support it, only regular STP and apparently
         | MSTP
        
           | teddyh wrote:
           | > _and apparently MSTP_
           | 
           | Where did you find that?
        
             | gerdesj wrote:
             | STP is "slow" STP, and RSTP is a degenerate case of MSTP!
             | 
             | MSTP is Multiple spanning trees ie you can group VLANs and
             | prefer paths for those groups of VLANs. That means if you
             | have say two links between two bridges (switches) you can
             | prefer some to use one link and the rest to use the other,
             | that means you are not "wasting" a standby link. They will
             | fail over to the surviving link on failure. STP and RSTP
             | will only consider one link as a whole, so two ports are
             | "wasted" when not in use: in the case of a two bridge, two
             | links example.
             | 
             | Old school STP without the Rapid part hasn't really been a
             | thing for several decades. I can't think of why you
             | wouldn't use RSTP in general but if you need to make best
             | use of your forwarding capacity then a 50/50 MSTP may be
             | indicated. That's where you look at your traffic flows
             | across VLANs and try to bundle them up into a 50/50%
             | collection. One lot prefers link A and the rest get link B.
             | Obviously you can get really creative as the number of
             | VLANs and links mount up. Bear in mind that dot1Q is a
             | simple version of QinQ!
             | 
             | Sorry, got a bit carried away there.
             | 
             | For nearly all intents and purposes, RSTP _is_ STP. If you
             | plug in a network cable between two devices and it does not
             | start working within say five seconds then you are living
             | in the 1990s.
        
               | mito88 wrote:
               | > If you plug in a network cable between two devices and
               | it does not start working within say five seconds then
               | you are living in the 1990s.
               | 
               | a living in a hub...
               | 
               | :)
        
               | teddyh wrote:
               | I meant, where did you find that this specific Linux-
               | managed network switch supports MSTP? Because the Linux
               | bridge networking code, last time I looked, only
               | supported STP; you had to run a separate daemon to even
               | get RSTP.
        
       | thelastparadise wrote:
       | What's the advantage of doing this over plugging multiple gigabit
       | adapters into a linux machine and adding them all to a bridge?
       | 
       | I'm guessing performance might be better with the hardware, but I
       | don't know --has anyone done tests to show the difference?
        
         | MartijnBraam wrote:
         | This is more efficient if most of the traffic will get
         | switched, seperate adapters is more efficient if most of it
         | needs to be routed.
        
         | simcop2387 wrote:
         | It'll be a combination of:
         | 
         | 1. Throughput - say you use usb adapters, in a lot of ways usb
         | is a shared bus so you'll run into max bandwidth quickly. This
         | is especially because data will have to go in and then out, all
         | the way to the cpu
         | 
         | 2. Latency - because you're using software to do switching,
         | it'll add time to process each packet and send it back out tne
         | right place. You've also go any other interface latency adding
         | to it
         | 
         | 3. Power usage - eacj adapter will have it's own full network
         | phy and hardware, which will increase the power draw. Combined
         | with all the extra processing above and now your power usage is
         | even higher. This means you also loae out on hardware
         | offloading and other performance enhancememts that generally
         | reduce power usage because less of the system is involved in
         | mocing packets around
         | 
         | 4. Features (potentially) - this will depend a lot on the
         | hardware you choose, some of those cheap gigabit usb adapters
         | i've tried didn't work with vlans and other features properly.
         | But if you say load up a bunch of nice pcie cards with 1 or
         | more ports thst support everything (never had issues with pcie
         | ones) then you can now actually get a lot of features that are
         | otherwise difficult or impossible on simpler hardware (though
         | at that point you're doing routing more tham switching, but
         | thay flexibility is why you'd potentially do this).
        
           | Fnoord wrote:
           | We're only talking 1 gbit ethernet here, so you can have
           | multiple of those ports on PCIe.
           | 
           | I have a PCIe card here with two 2.5 gbit on it (don't
           | remember exactly how much it was on Ali but between 20 and 40
           | EUR) and I can saturate both with iperf3. Since the example
           | only uses 4 ports, it should be easy to make a simple router
           | with just two PCIe cards. But there's probably 4x 1 gbit PCIe
           | out there, too. And if you use 1 gbit fiber, that wouldn't
           | cost much power nor would it need much speed. If your uplink
           | is DSL, you could use a VigorNIC 132.
        
         | adrian_b wrote:
         | For such a low speed and small number of ports using a hardware
         | switch circuit is not necessary.
         | 
         | Nevertheless, if the designing and building effort is
         | neglected, I assume that the total cost of the hardware might
         | be under $100, which is less than a computer with multiple
         | interfaces would cost.
         | 
         | Still, 1 Gb/s networks are rather obsolete. One could make a
         | managed network switch that is bigger and faster by using only
         | off-the-shelf components for slightly more than $200, e.g. a
         | 6-port 2.5 Gb/s switch can be made with an Odroid H4+ having 2
         | ports, together with its add-on M.2 card with 4 extra Ethernet
         | ports. Another variant is to use a small computer with an N100
         | CPU and 4 2.5 Gb/s ports, which can be bought in this price
         | range from various Chinese companies. Similar small computers
         | with 6 2.5 Gb/s ports are a little more expensive, perhaps
         | slightly over $300.
        
           | MartijnBraam wrote:
           | $100? I paid roughly $80 for two of these and the price per
           | boards will rapidly converge to ~$17 when increasing
           | quantity.
        
             | adrian_b wrote:
             | The PINE A64-LTS alone is $40 + shipping + taxes, so you
             | cannot have a $17 total cost.
             | 
             | I assume that you mean that the card with the switch
             | circuit alone could reach $17, if made in a large quantity.
             | 
             | However, you cannot make a managed switch with that card
             | alone.
             | 
             | If your current cost has been $40 for a card with the
             | switch circuit, adding the PINE A64-LTS + whatever you have
             | paid for shipping and taxes must make the total around
             | $100, exactly like I have said.
        
               | MartijnBraam wrote:
               | I meant the cost of just the switch obviously... It can
               | hook up to whatever Linux machine you have running and I
               | happened to have this one in the drawer.
               | 
               | The point of this was not to make the fastest or cheapest
               | or most featureful switch available, it just had to fit
               | in a case and none of the options have one port facing
               | backwards so it doesn't require an ugly loop cable on the
               | front of the final case.
               | 
               | It's also possible to make a managed switch with that
               | card alone, there is a footprint for a NOR flash chip, if
               | you load the netgear GS105E firmware on that (which is
               | available on the netgear website) then it will be just a
               | GS105E without leds and one port on the back.
        
           | mschuster91 wrote:
           | > Still, 1 Gb/s networks are rather obsolete.
           | 
           | Are they? Most consumer, even office gear still is maximum
           | 1000M/port - your average USB-C network/multi-port laptop
           | dongle, most USB-C/TB monitors (and shamefully, Apple's
           | Studio Display which only has USB-C/TB ports, no network),
           | VoIP phones (hell these are usually 10/100 only, with 1000
           | being reserved for top models), printers, virtually all entry
           | to mid range NAS systems... the list is endless.
           | 
           | Options for more than that tend to be really niche,
           | expensive, or are bring-your-own-module-SFP.
           | 
           | Besides, 1000M is way more than enough for almost all
           | consumer and office needs. Only exception is heavy video and
           | photo editing, if these workplaces don't already use direct
           | Thunderbolt/FC attach.
        
             | bongodongobob wrote:
             | They definitely aren't, no idea what parent is talking
             | about. They are the standard unless you're talking about a
             | SAN or something. If I had to guess, 92% of the switches in
             | a typical office are 1000M, 5% are 100M, and the rest are >
             | 1000M for switch interconnects or HA server
             | interconnects/SANs.
        
             | TacticalCoder wrote:
             | No they aren't.
             | 
             | You failed to mention the big one: _most_ people by very
             | far have 1 Gb /s network switches to... Connect their
             | machines to this thing called the Internet. Some may have
             | heard of it.
             | 
             | And most people also have not more than 1 Gbit/s up/down.
             | 
             | Do I have switches (with a 's') with 10 Gbit/s SFP+ at
             | home? Yup. Is 1 Gbit/s obsolete: definitely not.
        
               | p_l wrote:
               | 1gig is starting to become a problem - we have increasing
               | deployment of 2gig and 8gig internet to homes in Poland
               | for example, figuring out how to deal with that when most
               | computer gear still comes with 1gig is becoming an issue
        
           | Asmod4n wrote:
           | A raspberry pi 400 can handle around 875 MBit when bridging
           | two interfaces. So it's not even fast enough for two ports
           | let alone full duplex. I doubt a n100 can handle more than
           | three.
        
             | sofixa wrote:
             | An Intel Celeron N5105, on one core, does ~28Gbps locally
             | between two bridged interfaces with iperf.
             | 
             | A Raspberry Pi 4 has a single PCIe lane with a total of
             | 4Gbps bandwith for everything (all USB ports and Ethernet).
             | 
             | Edit: shameless plug, but I _just_ finished writing an
             | article on my new home network, including the router with
             | the above spec /results:
             | https://atodorov.me/2024/07/03/running-a-multi-gig-home-
             | netw...
        
               | Asmod4n wrote:
               | The CPU is the bottleneck. All switching is done in
               | software with off the shelf NICs.
        
               | sofixa wrote:
               | Yes, my Intel Celeron N5105 had a full core pegged at
               | 100% while doing that iperf and getting those numbers.
               | 
               | But there are 4 whole cores, meaning I can get far more
               | traffic switched and routed with the CPU capacity than
               | all the ports combined can sustain.
        
           | bongodongobob wrote:
           | 1000M networks are THE standard, no idea where you got the
           | idea they were obsolete.
        
           | KeplerBoy wrote:
           | Unfortunately gigabit ethernet is far from obsolete.
           | 
           | Yes, there's 2.5 gigabit on some consumer hardware, but it's
           | still kind of rare.
           | 
           | Also who is excited about a 2.5x speedup after 20 years?
           | Nobody cares until we need 10 gigabit internet access (which
           | will probably never happen).
        
             | sofixa wrote:
             | > Nobody cares until we need 10 gigabit internet access
             | (which will probably never happen).
             | 
             | For what it's worth, my ISP gives me 5Gbps down/700Mbps up
             | for ~40 euros a month which includes a bunch of TV channels
             | and discounts for Netflix and Disney+.
             | 
             | They also have an 8Gbps down / 8Gbps up plan for ~60 euros,
             | also including a ton of extra things.
        
               | KeplerBoy wrote:
               | Wow, that's pretty cool. Where is this offered?
               | 
               | But have you invested in 5Gbps+ networking gear to
               | actually take advantage of the offer? 10 Gbps Nics have
               | become super affordable (~20$) by the way.
        
               | p_l wrote:
               | Cat7 cabling is something I encountered at discount
               | stores now
        
               | KeplerBoy wrote:
               | I wouldn't trust a Cat7 cable from a discount store to
               | adhere to the spec though.
        
               | p_l wrote:
               | Used it more as illustration of market penetration - just
               | like with HDMI cables, you're either ripped off on
               | markup, sold under-spec cabling, or both.
        
               | ta1243 wrote:
               | I wouldn't use copper for 10G anyway, use a fibre sfp,
               | far more power efficient.
        
               | KeplerBoy wrote:
               | Doesn't that depend on the run length? Surely copper is
               | more efficient for just a few meters.
        
               | sofixa wrote:
               | It depends.
               | 
               | In my case, all rooms are cabled with a weird electrical
               | standard that should get me 10G Ethernet (and does 2.5G
               | without any issue). I'm not going to drill holes to pull
               | my own fibre all around the place when I have perfectly
               | good Ethernet connectivity.
               | 
               | Also, while 10G SFP+ NICs are vastly more available than
               | Ethernet 10G ones, switches seem to be cheaper with
               | 2.5/5/10G Ethernet ports than full on SFP+, unless you
               | buy recycled Enterprise gear which would blow your power
               | efficiency argument out of the water.
        
               | sofixa wrote:
               | > Wow, that's pretty cool. Where is this offered?
               | 
               | Free in France.
               | 
               | > But have you invested in 5Gbps+ networking gear to
               | actually take advantage of the offer? 10 Gbps Nics have
               | become super affordable (~20$) by the way.
               | 
               | The 8/8 plan comes with a router that has WiFi 7
               | (theoretical max 46Gbps, but who knows in reality, it's
               | not really out yet), an SFP+ port, and 4x2.5Gbps Ethernet
               | ports. The one I have only has one 2.5Gbps Ethernet, so
               | that's the Internet speed at which my home network caps
               | out at.
               | 
               | If you're interested in more details, I recently finished
               | writing an article about my home network:
               | https://atodorov.me/2024/07/03/running-a-multi-gig-home-
               | netw...
               | 
               | And yeah, 10G equipment (in terms of NICs and cables) is
               | quite affordable, but switches still aren't really
               | _super_ affordable.
        
             | ta1243 wrote:
             | The main reason for 2.5g is for digital video (2.5gbit will
             | allow do 2 1080i 2110 streams), and especially the
             | increasing numbers of wifi6e APs that do >1gbit but nowhere
             | near 10g
        
         | toast0 wrote:
         | As everyone else is saying, power usage should be less this
         | way. Probably less latency to traverse the switch than a
         | software bridge, too. Switching should continue to function if
         | the host OS crashes, combined with a watchdog and recovery, you
         | could have a more available system where maybe some things
         | don't work for a brief interval, but much better than a
         | software bridge (assuming the switch chip doesn't crash or get
         | stuck, anyway).
         | 
         | It depends on what your goals are though. If you want to
         | inspect all traffic passing through the switch, having 4
         | interfaces is clearly better. If your host based switch is also
         | doing a lot of communication, 4 interfaces gives you 4gbps from
         | the hkst rather than 1gbps. Etc.
         | 
         | Ex-enterprise quad 1G cards are $15 or less on ebay. I'm
         | partial to silicom quad bypass 1g_ PEG4BPI-SD; the bypass
         | feature can be fun, and they're cheaper cause they're weird
         | (you can mostly configure them to be 'standard nics' once and
         | then plug them into anything without much fuss, but getting
         | there can be challenging. Early ones come with pci ids set to
         | silicom as the vendor _and_ subvendor which makes them harder
         | to use; the -SD cards have intel vendor id and silicom
         | subvendor, so the normal driver will attach.
         | 
         | 4x10g ports would be more to manage, and you might not have
         | enough throughput for software bridging, depending on the host
         | system. And quad port 10g cards are harder to find. 2x10G is
         | reasonably priced though, if you're patient.
        
         | denkmoon wrote:
         | Your CPU will be in the data path there. Switches do their
         | packet switching on ASIC rather than CPU. So it depends how
         | good your CPU is, and generally that's not an effective use of
         | whatever compute power you've got.
        
       | buccal wrote:
       | Interesting project.
       | 
       | For simpler use you can get an OpenWRT capable router which in
       | most cases uses a managed switch chip. OpenWRT provides a nice
       | interface to configure VLANs and other options.
        
         | tomatocracy wrote:
         | OpenWRT these days can also be installed on some switches eg
         | the Zyxel GS1900 series (though support for things like PoE and
         | 10Gb/SFP+ ports might be limited, I'm not sure where things
         | stand there).
        
           | stragies wrote:
           | Looking at the OpenWrt forum, work is coming along nicely
           | w.r.t supporting FasterThan1G on RTL9X. Many/Most
           | configurations seem to be working.
           | 
           | RTL8X is AFAIC done, feature-complete.
           | https://svanheule.net/switches/
           | 
           | POE on those devices is mostly two types: Broadcom (well
           | supported), and Realteks inhouse solution, which uses a
           | 'dialect' of the Broadcom protocol. There is a git branch/PR,
           | where the 'dialect' differences have been moved to individual
           | modules. But it's not released yet.
           | https://github.com/Hurricos/realtek-poe/pull/35
           | 
           | Unfortunately, there seem to be hard problems migrating up
           | from Linux 5.15 to 6.1 or 6.6.
        
           | znpy wrote:
           | WTF, i had no idea that was possible. I have that exact
           | switch (GS1900-24E)... I need to look into that
        
             | stragies wrote:
             | Install mosquitto afterwards, add a small script, and the
             | ports all show up in HomeAssistant as power-monitoring
             | ON/OFF switches. :)
        
               | znpy wrote:
               | Jesus christ that's awesome!
        
       | stragies wrote:
       | Nice article, Thank you for the write-up.
       | 
       | First time I saw somebody 'creatively using' an RTL83something
       | switch was https://spritesmods.com/?art=rtl8366sb, and there were
       | others since then, but yours was the first 'build my own managed
       | switch', instead of 'adding an external brain to an unmanaged
       | switch'
        
         | MartijnBraam wrote:
         | Ah yes the legendary sprite_tm, I've come across this while
         | writing my own raspberry pi firmware to manage the switch over
         | USB.
        
       | gerdesj wrote:
       | "Network switches are simple devices, packets go in, packets go
       | out. Luckily people have figured out how to make it complicated
       | instead and invented managed switches."
       | 
       | Expensive switches involve some pretty fancy ASICs. For example I
       | have a pair of fairly elderly Dell OS9 switches with 48 x 10Gb/s
       | ports and four x 40Gb/s QSFP+ fibre ports. These are "old skool"
       | stacked jobbies. Each switch can shuffle up to 1.28Tb/s (1).
       | That's quite a lot. You can get those for PS1800 including VAT
       | (2) these days and they will last nearly forever.
       | 
       | I love to see efforts like this but do bear in mind that say
       | Netgear will do a eight port 1Gb switch with Power over Ethernet
       | on all ports for about PS125.
       | 
       | If you cost your time at somewhere between PS20-50 per hour when
       | evaluating whether a project is financially viable, then an off
       | the shelf box might be indicated. However, if the actual purpose
       | is the project itself then sod the price!
       | 
       | (1) https://i.dell.com/sites/doccontent/shared-content/data-
       | shee... (2) https://www.etb-tech.com/dell-force10-s4820t-10gbe-
       | switch-os...
        
         | bithead wrote:
         | Ignorance is bliss.
        
         | immibis wrote:
         | I worked with one of these ASICs, from Broadcom. Not 40, but
         | something like 4x10+24x1Gbps + PCIe to CPU. The ASIC cost as
         | much as you would expect (I don't know the actual number - a
         | couple hundred bucks?). The software interface to it was very
         | poorly documented, and was a library that supported all
         | Broadcom switch ASICs, so was a few hundred megabytes .a file
         | and was full of functions that would just return "not supported
         | on this device" errors, which you wouldn't know until you tried
         | it.
        
           | zokier wrote:
           | Not Broadcom, but Microchip has list prices straight on their
           | website (how refreshing!), and yeah few hundred bucks sounds
           | about right; for example this random 128Gbps switch chip is
           | about $120 in single quantities, $80 in volume
           | 
           | https://www.microchip.com/en-us/product/vsc7552#purchase-
           | fro...
           | 
           | It feels bit bonkers that a actual switch based on that
           | probably costs thousands.
           | 
           | The software side is crap as usual. They sell their Linux-
           | based package at $75000, with maintenance (basically updates)
           | at $17500.
           | 
           | So even though the chip is cheap, you need to burn six
           | figures to do anything with them :(
        
             | kijiki wrote:
             | Their Sparx-5 line is supported by stock upstream Linux
             | switchdev.
        
               | stragies wrote:
               | Intrigued by that, I went looking. First commit of
               | support was 2019, for Linux5.2. Yet, 5 years later, there
               | is not one example (that i could find) of OpenWrt running
               | on these Chips. If the platform/FW/BSP package were any
               | good, I'd expect there to be chat about it in the forums,
               | and/or attempts to port to such devices from SMBStaX (The
               | Microchip BSP Linux dist).
        
               | inhumantsar wrote:
               | I suspect it's not worth the effort for most people.
               | People who want to DIY a switch are likely content to
               | brute force it with a decent cpu, multiport network
               | cards, and lots of PCIe lanes. While others will be happy
               | with a used enterprise OEM unit.
               | 
               | Does make me wonder what asics Microtik is using under
               | the hood though...
        
               | MartijnBraam wrote:
               | A lot of them are just networking SoCs with built in
               | switching fabrics or the lower end ones have the low end
               | router SoC from various manufacturers.
               | 
               | Mikrotik is one of the few brands that actually list this
               | in the spec sheets.
        
               | zokier wrote:
               | You are right, the situation is better that what I
               | thought, the commercial smbstax SDK might not be
               | absolutely required.
               | 
               | They even have some BSP stuff freely available which is
               | really positive sign. https://github.com/microchip-
               | ung/bsp-doc
               | 
               | This would be such an enticing project to jump into,
               | there are just enough bits to make it seem doable, even
               | if the realist in me knows its not really feasible.
        
             | teleforce wrote:
             | That's a very cheap price for enterprise layer 3 switching
             | chip. Thanks for the info, how this switch chip has escaped
             | me is very perplexing.
             | 
             | Can you elaborate on their Linux-based package, is the
             | package critical for the development and using the chip
             | with embedded Linux. Can we just use it as switching fabric
             | and use Intel CPU for example N100 as the main controller
             | running Linux?
             | 
             | The sibling comment also mentioned that Sparx-5 line is
             | supported by stock upstream Linux switchdev, and this the
             | same chip as in your link.
        
       | wesapien wrote:
       | What ever happened to Openflow? Wouldn't this be a perfect device
       | for it? Setting up a lab for it was almost impossible thats why I
       | lost interest.
        
         | wmf wrote:
         | OpenFlow was kind of the wrong solution to the problem. (The
         | right solution is switchdev or SAI.)
        
           | wesapien wrote:
           | I'll look them up. Cheers.
        
           | SSLy wrote:
           | SAI being what?
        
             | wmf wrote:
             | Switch Abstraction Interface
        
           | p_l wrote:
           | OpenFlow is a protocol, switchdev is a driver API for
           | exposing access to essentially the same operation model as
           | OpenFlow targets. You don't replace OpenFlow with switchdev,
           | you might use OpenFlow to integrate a switchdev with an SDN
        
             | wmf wrote:
             | You want to write code that runs on the switch, whether
             | it's a routing protocol or an intent reconciliation loop or
             | whatever. You don't want to send low-level OpenFlow
             | commands over the network because it ends up being slow and
             | chatty.
        
               | p_l wrote:
               | A considerable portion of SDN craze involved pretty much
               | this. Even systems that didn't use OpenFlow were
               | incredibly chatty, creating complex chains of NextHop
               | routes from centralized or at least semi-centralized
               | servers to the actual switches.
               | 
               | Even in more sane setups, it's not actually uncommon to
               | have control plane talk over network to actual forwarding
               | plane - IIRC Cisco Nexus operates this way, with actual
               | forwarding engines talking with control plane over
               | Ethernet link.
               | 
               |  _I like the code to run on the switches directly_ , but
               | SDNs very often eschewed that for dumber forwarding
               | engines controlled OpenFlow way, even if actual protocols
               | used weren't OpenFlow.
        
         | immibis wrote:
         | AFAIK OpenFlow tried to be far too generic, resulting in it not
         | actually mapping onto any actual hardware without lots of
         | abstraction leakage.
        
           | zamadatix wrote:
           | Ironically it could also be said OpenFlow was not generic
           | enough. More and more networking hardware continues to move
           | towards having a fully programmable pipeline but OpenFlow
           | isn't the most flexible way to define a completely arbitrary
           | pipeline.
           | 
           | https://opennetworking.org/news-and-
           | events/blog/clarifying-t...
        
           | p_l wrote:
           | It mapped very well to HW, but only to dumbest part of the
           | hw.
           | 
           | at least from pov of someone dabbling in SDNs around
           | 2014-2017 including deep guts implementation details of one
           | commercial one...
           | 
           | OpenFlow encoded the approach of dumb ethernet switches,
           | combined with Cisco Express Forwarding - which is a common
           | aspect of many switching ASICs, but not entirety of them,
           | even for Cisco.
           | 
           | It never abstracted that historic aspect, if anything the
           | leaking was the internal model of OpenFlow (editing
           | forwarding table) rather than internals HW (which sometimes
           | was a systolic array of CPUs that exposed "tables" as detail
           | of interaction but nothing actually fundamental).
           | 
           | All was fine so long, like CEF, you dumped everything to
           | CPU/northbound API outside "IP with this address should go
           | there, and TCP flow with those parameters go there", in case
           | of early openflow bejbf essentially "packet with dstMAC shall
           | go to port X".
           | 
           | This does not work well with acceleration engines, or in
           | beefier chips the aforementioned systolic arrays or similar
           | arrangements, outside of hardcoding idea of specific
           | accelerators and their interface tables.
        
       | protocolture wrote:
       | No offense intended, I may have misunderstood something here.
       | 
       | You use a routerboard block diagram as your model to demonstrate
       | how a hardware switch is connected to the rest of the system. But
       | then you go on to claim that they are impossible/difficult to
       | work with.
       | 
       | Did you source a routerboard at any point here? Fairly sure that
       | OpenWRT can be built for most routerboards, and the 2011 should
       | be a fairly common device on the second hand market.
       | 
       | Maybe this is a better question. Was the goal always to build
       | from scratch? Or did you discard the concept of using someone
       | elses hardware for a particular reason?
        
         | stragies wrote:
         | The RouterBoard RB2011 is stuck on an ancient release version
         | of OpenWrt, 19.07, see here:
         | https://openwrt.org/toh/mikrotik/rb2011
         | 
         | The problem seems to be somewhat related to the NAND, which is
         | IIRC somehow different from the supported RouterBoards.
         | Somebody proposed a new solution to this problem, but that
         | hasn't landed yet: https://forum.openwrt.org/t/wiki-cleanup-
         | for-mikrotik-rb2011... and other threads.
        
         | MartijnBraam wrote:
         | I have a stack of them here, they are great to work with for
         | their intended purpose but they are not that great if you want
         | to run custom software.
         | 
         | I also grabbed a RB2011 diagram because it's a simple diagram
         | that explains it well I think, the RB1100AHx4 is a better
         | example technically since it's using the same switch chip but
         | it's more confusing because they use the two CPU ports together
         | and claim it's a 2.5Gbps link while it's two 1.25Gbps link and
         | they ignore the encoding overhead.
         | 
         | The reason why I build this from scratch is that it that the
         | expense is reasonable and this should end up in the FOSDEM
         | video recording boxes and it has to fix a few issues specific
         | to that design like it needing to expose 4 network ports to the
         | front panel of the case but also be connected to the internal
         | SBC. There's simply not that much space in the case to put
         | passthroughs in the case to a switch to not have an external
         | loop cable for the SBC and with a dumb switch the system can't
         | be monitored anymore. Since quite a few of these boxes are
         | built this becomes a reasonable solution (if you ignore design
         | time, but this is volunteer work)
        
       | shmerl wrote:
       | From what I've read, it's very hard to make a 10 Gbps switch
       | without relying on some blobs since those accelerator chip makers
       | don't have any open drivers support.
       | 
       | And you supposedly need those chips since switching it on the CPU
       | is very taxing.
        
       ___________________________________________________________________
       (page generated 2024-07-04 23:01 UTC)