[HN Gopher] The Pumpkin Eclipse
___________________________________________________________________
The Pumpkin Eclipse
Author : alexrustic
Score : 268 points
Date : 2024-05-30 15:52 UTC (7 hours ago)
(HTM) web link (blog.lumen.com)
(TXT) w3m dump (blog.lumen.com)
| hcfman wrote:
| Which routers are affected ?
| aschla wrote:
| "We began an investigation after seeing repeated complaints
| mentioning specific ActionTec devices, as a massive number of
| device owners stated that they were not able to access the
| internet beginning on October 25, 2023. A growing number of
| users indicated the outage was common to two different gateway
| models: the ActionTec T3200s and ActionTec T3260s, both
| displaying a static red light."
| Scoundreller wrote:
| "the ActionTec T3200s and ActionTec T3260s, both displaying a
| static red light."
|
| "This included a drop of ~480k devices associated with
| Sagemcom, likely the Sagemcom F5380 as both this model and the
| ActionTec modems were both modems issued by the ISP."
| prophesi wrote:
| And which ISP?
| AzN1337c0d3r wrote:
| Windstream uses T3200 and T3260.
| prophesi wrote:
| Ah, was able to find/validate that from a news outlet
| covering this. Thanks!
|
| https://arstechnica.com/security/2024/05/mystery-malware-
| des...
| Scoundreller wrote:
| > These reports led us to believe the problem was likely a
| firmware issue, as most other issues could be resolved through a
| factory reset.
|
| My dream is to intercept the write-enable lines on the flash
| chips holding these firmwares so I can lock out updates. And
| schedule a daily reboot for any memory-resident-only crap.
|
| That's what we used to do on, ahem, satellite receivers, 20 years
| ago and maybe we all need to treat every device attached to the
| internet as having a similar susceptibility to "electronic
| counter-measures".
|
| Or at least monitor them for updates and light up a light when an
| update happens if it was my own equipment and I'd know if it
| should go off or not.
| iknowstuff wrote:
| Have you never interacted with a person who's not a programmer
| in your entire life? How could you possibly think this is a
| good idea
| whatevaa wrote:
| He just wants to watch the world burn.
| Denvercoder9 wrote:
| Most non-programmers don't give a shit about their router
| beyond "the wifi must work". Something completely stateless
| that can't be broken or messed with actually sounds like
| something they'd want.
| starttoaster wrote:
| What about when the router has a security update that is
| actually useful to stop people from getting owned? Such as
| to address: https://thehackernews.com/2023/07/critical-
| mikrotik-routeros...
|
| Blocking updates seems useful for blocking malicious and
| helpful updates. I wonder which camp the majority of
| updates fall into.
| stacktrust wrote:
| A physical switch can be locally toggled by the device
| owner/admin.
|
| Some motherboards offer a physical jumper for firmware
| updates, including x86 PC Engines APU2 coreboot router.
| starttoaster wrote:
| So here's another problem. Going back to the point made
| earlier in this thread "users don't care much about their
| routers." The average user opens their router page
| exceedingly rarely, which is the benefit of automatic
| updates. I set up my mom's router, and she works in IT
| (more in project management these days, so she's fallen
| into being a mostly non-technical user.) And she still
| texts me every time she needs to get into it to ask me
| what the password I set up was... once every few years.
| The downfall of automatic updates is obviously something
| like the case of the article this discussion is about.
| But weigh up the costs and benefits, in my opinion, the
| scale tips more towards automatic updates when you factor
| in all the critical level vulnerabilities that router
| OS's have accrued over the years.
| stacktrust wrote:
| Some consumer routers (e.g. Amazon eero) have moved to
| cloud/app config and automatic updates.
|
| A read-only partition can bootstrap recovery from cloud,
| if the main firmware or config is damaged.
|
| Some updates are possible with live kernel patching of
| the memory-resident OS, which can also be used by
| malware.
| labcomputer wrote:
| Well, considering that most home users _never_ update
| their router 's firmware, I'm going to go out on a limb
| and suggest that the majority of applied updates are
| malicious.
|
| If you did want to go this route, a simple fix would be
| to have the write enable line gated by a hardware one-
| shot timer (think like a 555 timer) triggered by a
| physical button on the front (and the button would also
| reboot the router).
|
| The firmware update sequence would go like: Router
| prompts user to update using button -> user presses
| update button -> router reboots to clear malicious
| software -> when the router comes back up, the write
| enable gate remains open for $n minutes (and maybe
| there's a GPIO that can hold the gate open if it is
| already open) -> router performs software update.
|
| The problem is that there's pretty much no way to do this
| without adding a $1-2 to the BOM, and no manufacturer of
| (pro|con)sumer routers will do that.
|
| Edit: Or do what stacktrust wrote and just have a toggle
| switch (update/secure).
| starttoaster wrote:
| > Well, considering that most home users never update
| their router's firmware, I'm going to go out on a limb
| and suggest that the majority of applied updates are
| malicious.
|
| Consumer grade routers often have automatic updates these
| days.
| internetter wrote:
| Yeah I have a pretty nice router and inexplicably, random
| nights, it drops out at 3am for ~3 minutes and then comes
| back on (and has all the markings of a reboot in terms of
| the pattern of request failures). I have to assume
| Synology just decided that nothing's using the internet
| at 3am
| starttoaster wrote:
| I had no idea that Synology did routers too, but I would
| assume this would be a configurable time somewhere in the
| settings. But yes, that sounds like automatic updates to
| me.
| Scoundreller wrote:
| To fix security issues, or to introduce new features
| increasing the attack surface?
| stacktrust wrote:
| OPNsense automatic updates are relatively conservative.
| dymk wrote:
| Bummer then, SSIDs and WiFi passwords are state.
| q0uaur wrote:
| plenty of routers in my country (id go on a limb and say
| the majority) have both written on a sticker, different
| password for each device generated when the firmware
| first gets installed, and people NEVER change it.
|
| _edit_ they also come with a QR code you can scan on
| your phone to connect to the wifi without manually copy
| the password, so people just put up that qr code
| somewhere and connecting is easy enough.
| WhyNotHugo wrote:
| To be fair, most people never change these.
|
| In any case, you can have a separate storage (you need
| less than 1KB) for these two things.
| Scoundreller wrote:
| I think that's usually the case: that will be stored in a
| small 24c or 93c (i2c or SPI) chip that is separate from
| the firmware that may only handle 1000 flashes and takes
| a lot longer to flash (often requiring risky and long
| downtime).
| Denvercoder9 wrote:
| Half of my technical friends haven't changed theirs from
| the factory defaults. None of my non-technical friends
| have changed them. A non-representative sample of SSIDs I
| receive in my apartment gives 9 default ones, and 2
| custom ones (plus mine).
| fiedzia wrote:
| For "wifi to work" you will need updates. Though updating
| all devices at once is a really dumb policy.
| hifromwork wrote:
| Wormable vulnerability in a router is 10x bigger issue in
| practice than attacks that brick a router. By "in practice"
| I mean judging by the real world attacks that I know of.
| Hacked routers are used as residential proxies for
| criminals, for DDoS attacks, fast flux networks, credential
| stuffing attacks, and more. Bricking routers is rare (and
| very loud), that's why it's news on HN.
|
| Many router malware families don't even try to be
| persistent. Even Mirai - arguably the most famous router
| botnet - is not persistent [1]. In case the device is
| rebooted, it just gets infected again in a few minutes.
|
| It's very important that all network connected devices have
| an update mechanism, working automatically in the
| background.
|
| [1] At least the original version. After the code leak
| people were doing all kinds of updates, so there are some
| variants that try to be persistent in some cases.
| stacktrust wrote:
| _> In case the device is rebooted, it just gets infected
| again in a few minutes._
|
| Can Suricata detect and block known router botnets?
| ghshephard wrote:
| I have a friend who used to do classified military work - and
| a lot of the firmware on munitions is designed in precisely
| this way to avoid countermeasures. Advanced systems have a
| _lot_ of software, and the systems they are shipped on have
| fuses and circuit traces that are melted to avoid any
| possible countermeasures or modification once they 've passed
| acceptance.
| starttoaster wrote:
| When the military finds a flaw in their immutable firmware,
| what do they do with those munitions? Throw them away and
| build new ones? Routers are networked and critical
| vulnerabilities that should be addressed get patched on
| them over time. Is the proposal to just tell the user to
| buy a new router?
| ThrowawayTestr wrote:
| >what do they do with those munitions?
|
| They're sent back to the manufacturer and the boards are
| replaced.
| rvnx wrote:
| Here is the same with the routers that got bricked. They
| probably need to be reflashed. Most likely they are
| leased by the ISP so under their scope of responsibility
| and maintenance.
|
| "internet doesn't work" -> "send me a new router, it
| broke", problem solved.
| Cthulhu_ wrote:
| Given what little I know of military stuff, swapping out
| a hardware board is cheap compared to the cost of the
| whole thing.
|
| Would they be sent back or would an engineer travel to
| wherever stuff is stored to do the job?
| ThrowawayTestr wrote:
| >swapping out a hardware board is cheap
|
| The labor is cheap, the boards are expensive as hell.
| bongodongobob wrote:
| Same type of person who blocks Windows updates, messes with
| their registry, removes system files and processes, and then
| complains when Windows gets flaky on them and breaks in
| strange and unpredictable ways.
| Scoundreller wrote:
| If it's a switch, you could unlock it and allow for the
| provider's (or your own if it's your own equipment) updates
| before re-locking it.
| stacktrust wrote:
| Windows has a stateless mode.
|
| https://learn.microsoft.com/en-us/windows/iot/iot-
| enterprise...
|
| _> Unified Write Filter (UWF) is an optional Windows 10
| feature that helps to protect your drives by intercepting
| and redirecting any writes to the drive (app installations,
| settings changes, saved data) to a virtual overlay. The
| virtual overlay is a temporary location that is cleared
| during a reboot or when a guest user logs off.. Increases
| security and reliability where new apps aren 't frequently
| added. Can be used to reduce wear on solid-state drives and
| other write-sensitive media. Optimizing Application load
| timing on boot - it can be faster to resume from a HORM
| file on every boot rather than reloading the system on each
| boot. UWF replaces the Windows 7 Enhanced Write Filter
| (EWF) and the File Based Write Filter (FBWF)._
| Cthulhu_ wrote:
| Reminds me of some management software we ran on school
| computers some 20 odd years ago, it basically made the
| computer immutable and it'd be back to how it was after a
| reboot. I forgot what it was called and never knew how it
| worked though, at the time I was impressed. It needed an
| admin password to make changes that persisted.
| faefox wrote:
| Deep Freeze, maybe? :)
| dotnet00 wrote:
| They wouldn't have to do that if Windows didn't insist on
| being forceful with updates even for power users, who can
| be trusted to decide when to update. Lately it's not as bad
| since you can delay/disable forced reboots for updates for
| ~2 months at a time, but still it'd be far more ideal to
| just be able to turn off forced reboots entirely.
| bongodongobob wrote:
| Hi, I'm a power user too, and updates don't break
| anything, stop pretending it's 1998. Sincerely, every IT
| dept.
| a1369209993 wrote:
| Hi, I'm a linux user, and we never have any technical
| problems that would require updates in the first place.
| Sincerely, someone who is just as capable of lying as you
| apparently are.
| dotnet00 wrote:
| The issue isn't updates breaking stuff, the issue is
| forced reboots when I have something open for which I
| don't just want the OS throwing away the state for. The
| IT dept. comment is pretty accurate though, just as
| dismissive, pointlessly snarky and disconnected from
| doing productive work as the real deal.
| aidenn0 wrote:
| I suppose from the point of view of someone with a black-market
| HU card, DirecTV was an example of an Advanced Persistent
| Threat. Never thought of it that way before.
| Scoundreller wrote:
| Funny thing about directv is that because they allowed for
| many manufacturers to build receivers, directv had little
| control over the receiver firmware, so these counter-counter
| measures weren't necessary at the receiver level.
|
| Other providers that rolled out their own receivers had high
| control over the receiver firmware and once users figured out
| how to protect their cards, the receivers became an effective
| attack vector for the lazy.
|
| But that's where a lot of the public knowledge about JTAGs
| really started coming to light. Awfully nice of them to put
| in a cutout at the bottom of the receiver.
| schmidtleonard wrote:
| > maybe we all need to treat every device attached to the
| internet as having a similar susceptibility to "electronic
| counter-measures"
|
| "First party malware"
| luma wrote:
| I'm not too familiar with customer DSL solutions but for cable
| modems, that firmware and configuration is managed by the CMTS
| because technology and configuration changes on the head end
| may require customer-side changes to ensure continued
| operation. The config is a pretty dynamic thing as frequency
| plans, signal rate, etc change over time as the cable plant and
| head end equipment is upgraded and maintained.
|
| I'd expect that any attempt to lock write enable to the EEPROM
| would eventually result in your modem failing to provision.
| Scoundreller wrote:
| When your provider cuts you off, that's when you know that
| your provider has a legit upgrade you need to take. Take the
| update and then lock stuff up again.
|
| Of course, I don't think you're supposed to make mods to your
| vendor provided equipment...
|
| In the satellite world, this would happen too: old firmware
| would be cut off. That's when you go legit for a while with
| your sub'd card, take the update, and watch your sub'd
| channels until the new update could be reverse engineered.
| And probably have some heroes learn the hard way of taking
| the update and having some negative impacts that are harder
| to reverse.
| luma wrote:
| I'm not sure what such an approach would accomplish. If the
| goal is to prevent the kind of problem seen in the OP
| (which, let's be real - is a rare occurrence) in order to
| avoid an unplanned outage, you've instead created a
| situation where it'll fail to connect far more regularly as
| you're kicked off the network for not correctly handling
| the provisioning process. You're trading a rare unplanned
| outage for a common unplanned outage.
| Scoundreller wrote:
| Depends how often the provider pushes out updates (and
| the purpose/necessity of them).
|
| And it's only that "rare unplanned outage" when a
| malicious update bricks your device. Much worse is a
| malicious update that doesn't result in an outage.
| Probably still rare but that impact though.
|
| Edit: would also add that there's probably a big firmware
| chip that changes infrequently, and frequently changing
| config stored on a separate and smaller chip (like a 24c
| or 93 series eeprom that holds a few kilobytes). That way
| you don't risk bricking your modem by unplugging it at
| the wrong time.
| not2b wrote:
| It's a no-win situation. Sure, disabling firmware updates would
| have prevented this attack, but it would also prevent security
| fixes that keep the routers from being turned into a botnet.
|
| But what I don't get in this case is why it was not possible to
| reset the device to its original state. It seems like a
| misdesign if it's possible to destroy all of the firmware,
| including the backup.
| yjftsjthsd-h wrote:
| > It seems like a misdesign if it's possible to destroy all
| of the firmware, including the backup.
|
| Humor me; how would that work? If anything, I'd expect it to
| be easier to overwrite the inactive slot (assuming an A/B
| setup, ideally with read-only root). If you really wanted,
| you could have a separate chip that was read-only enforced by
| hardware, and I've seen that done for really low level
| firmware (ex. Chromebook boot firmware) but it's usually
| really limited precisely because the inability to update it
| means you get stuck with any bugs so it's usually only used
| to boot to the real (rw) storage.
| sounds wrote:
| It's an interesting challenge because the device is nominally
| "under ISP control" but any device located in a customer's
| home is under the physical control of the customer. The
| mistrust between the ISP and the customer leads to "trusted"
| devices where the firmware, including the backup, can be
| overwritten by the ISP, but then cannot recover if it gets
| corrupted. And believe me, the corrupt firmware scenario
| happens a lot due to incompetence.
|
| This is getting attention because it wasn't incompetence this
| time.
|
| But how does blank, unprovisioned equipment discover a path
| to its provisioning server? Especially in light of the new
| "trusted" push, this is an arms race in a market segment such
| as routers where there isn't any money for high end solutions
| - only the cheapest option is even considered.
|
| tl;dr: a social and economic problem, likely can't be fixed
| with a purely technical solution
| sidewndr46 wrote:
| This was years ago, but I remember getting cable service
| activated somewhere in Florida with Bright House. I handed
| the cable guy some ancient motorola cable modem I had found
| at a discount store. The guy took one look at it and said
| "look dude, if you hacked this thing to get around
| bandwidth caps it is your problem if you get caught". I
| guess apparently that particular modem was pretty easy to
| modify
| Scoundreller wrote:
| Maybe it already was modified!
| cuu508 wrote:
| Technical solution: customer treats ISP's modem/router as
| untrusted, and daisy chains their own router after it.
| Neither malware nor ISP's shenanigans can access the inner
| network.
| Scoundreller wrote:
| That's what I do. Also makes changing providers
| straightforward (though last time I needed to set up some
| custom VLAN stuff on my router but didn't have to fumble
| with any wifi config).
| bippihippi1 wrote:
| the bootloader installs the firmware. if you corrupt the
| bootloader, it can't install anything anymore. you'd need to
| physically access the chip to use an external flashing
| device. Some devices have non-writable bootloaders. They have
| an internal fuse that blows after the first write, so the
| chip's bootloader is locked. That means you can always flash
| a new firmware, but you can't fix any bugs in the bootloader.
| incangold wrote:
| 25 years in tech and I'm still waiting for that free lunch
| dataflow wrote:
| > the bootloader installs the firmware. if you corrupt the
| bootloader, it can't install anything anymore.
|
| That seems like awful design? Can't you have an alternate
| immutable bootloader that can only be enable with a
| physical switch? Or via some alternate port or something?
| That way they can update the live one while still having a
| fallback/downgrade path in case it has issues.
| bastard_op wrote:
| That was likely the point that whoever did it was trying
| to make, that they _were_ an extremely bad device.
|
| 1) The ISP exposed some form of external management they
| used to access them they shoudldn't have 2) The attacker
| overcame whatever security used on said management
| interface 3) Once in, the attacker could simply overwrite
| the first few sectors of the nand to make them unbootable
| without local hardware serial console. 4) There was no
| failsafe recovery mechanism it would seem
|
| An actual "modem" would mostly likely prove
| volatile/immutable by nature, but anything with a
| "router" built into it is far more vulnerable that
| typically run for poorly secured tiny linux systems, and
| subject to Chinese enshittification.
| Scoundreller wrote:
| Or a JTAG interface that the chip has in silicon and
| recovery is always possible from bare-metal. Dunno if
| that's technically in the MCU's bootloader or if the boot
| loader comes after.
|
| Still requires a truck roll but at least you don't need a
| hot air workstation.
| tonyarkles wrote:
| > Or a JTAG interface that the chip has in silicon and
| recovery is always possible from bare-metal. Dunno if
| that's technically in the MCU's bootloader or if the boot
| loader comes after.
|
| If the vendor's actually trying to lock down the platform
| they'll usually burn the JTAG fuses as well. It's hit or
| miss though, I've definitely come across heavily locked
| down devices that still have JTAG/SWD enabled.
|
| Edit: To your question, JTAG is usually physical silicon,
| not part of the bootloader.
| kbenson wrote:
| You could but a base level firmware on ROM, with a hardware
| trigger, and all that does on boot is listen and receive a
| signed firmware to write to the system. It needs a way to be
| triggered through hardware examining traffic and that also
| needs to require the seen command be signed. That recovery
| boot system needs to be as simple and minimal as possibly so
| you can have good assurance that there aren't problems with
| it, and should be written in the safest language you can get
| away with. Guard that signing key with your life, and lock it
| away for a rainy day, only to be used if much of your fleet
| of devices is hosed entirely. It should not be the same as a
| firmware signing key which needs to be pulled out and used
| sometimes.
|
| I think that could work, to a degree. There's always the risk
| that your recovery mechanism itself it exploited, so you need
| to make it as small and hardened a target as possible and
| reduce its complexity to the bare minimum. That doesn't solve
| the problem, which might be inherently unsolvable, but it may
| reduce that likelihood of it to levels where it's not a
| problem until long past the lifecycle of the devices.
| ajross wrote:
| > You could but a base level firmware on ROM, with a
| hardware trigger, and all that does on boot is listen and
| receive a signed firmware to write to the system.
|
| Almost all devices have something like that already in the
| form of a bootloader or SOC bootstrapping mode. But the
| idea breaks down if you want to do it OTA. The full
| storage/kernel/network/UI stack required to make that
| happen isn't ever going to run under "ROM" in the sense of
| truly immutable storage.
|
| The best you get is a read-only backup partition (shipped
| in some form on pretty much all laptops today), but that's
| no less exploitable really.
| stacktrust wrote:
| Apple has a robust recovery mechanism on their laptops,
| via T2 security coprocessor.
|
| https://www.macrumors.com/2020/06/25/apple-silicon-macs-
| new-...
|
| https://support.apple.com/guide/security/
| ajross wrote:
| Which is still running out of mutable storage. The point
| isn't whether you can verify the boot, it's whether you
| can prevent a compromised device (compromised to the
| point of being able to write to its own storage) from
| bricking itself.
|
| Now, as it happens Apple (everyone really, but Apple is a
| leader for sure) has some great protections in place to
| prevent that. And that's great. But if you feel you can
| rely on those protections there's no need to demand the
| ROM recovery demanded upthread.
| stacktrust wrote:
| There's also Apple DFU mode to restore OS with help from
| immutable ROM and external device, without depending on
| installed OS or mutable "Recovery OS" partition,
| https://theapplewiki.com/wiki/DFU_Mode &
| https://support.apple.com/en-us/108900
|
| _> DFU or Device Firmware Upgrade mode allows all
| devices to be restored from any state. It is essentially
| a mode where the BootROM can accept iBSS. DFU is part of
| the SecureROM which is burned into the hardware, so it
| cannot be removed._
| ajross wrote:
| ... right, which as mentioned requires physical access
| and external hardware and doesn't meet the requirements
| above either. And it's not particularly notable either:
| every flashable SOC in every market has something like
| this, at different levels of sophistication. Again it's
| just not a solvable problem at the level of integration
| imagined.
| kbenson wrote:
| > The full storage/kernel/network/UI stack required to
| make that happen isn't ever going to run under "ROM" in
| the sense of truly immutable storage.
|
| Why not? I'm essentially describing a specialized OOB
| system, and it would just use a carved out small chunk of
| system RAM or ship with a minimal amount RAM of its own.
| If you mean actually impossible to change because it's
| physical ROM ("truly immutable"), that's less important
| to the design than there's no mechanism that allows that
| storage area to be written to from the system itself,
| whether that's just the very locked down and minimal
| recovery kernel it houses not allowing it, or a jumper.
| ajross wrote:
| Sure, but now your device needs _two_ eMMC chips or
| whatever to store all that extra junk, and it 's been
| priced out of the market. FWIW: an alternative to designs
| like this is just to ship your customers an extra router
| to keep in a box if the first stops working: it's exactly
| the same principle, and fails for the same reasons.
| stacktrust wrote:
| _> My dream is to intercept the write-enable lines on the flash
| chips holding these firmwares so I can lock out updates. And
| schedule a daily reboot for any memory-resident-only crap._
|
| There was an open hardware project for SD card emulation, where
| the emulator could reject writes,
| https://github.com/racklet/meeting-notes/blob/main/community...
|
| OSS emulation for SPI flash, https://trmm.net/Spispy/
|
| Some USB drives (Kanguru) and SSD enclosures (ElecGear M.2 2230
| NVME) have firmware and physical switch to block writes, useful
| to boot custom "live ISOs" that run from RAM.
| Scoundreller wrote:
| Eventually in the satellite world, card emulators took over
| and only the receiver was a vector of attack, but then the
| receivers started getting simulated too.
|
| The nice thing about emulators is that you could intercept
| calls that you wanted and send your own response while still
| taking any and all updates. Hard to break when you have more
| control than they do.
| russdill wrote:
| Secure boot schemes can already "fix" this. If a boot image is
| programmed that isn't signed, the system boots to a write
| protected backup image. The system can also to some degree
| block the programming images that aren't signed, but presumably
| malware has gained root access.
| ck2 wrote:
| ISPs can send any firmware to a docsis cablemodem, without the
| user knowing or accepting.
|
| Imagine the damage that could be done by a malicious actor via
| the ISPs computers.
|
| Or imagine someone being able to hack the system that does that
| update even without the ISP.
|
| 600K users would be a toy, they could do it to 6 Million.
|
| Doesn't even have to be clever, just brick millions of
| cablemodems.
|
| North Korea or some other government level entity could manage
| the resources to figure that out.
| stacktrust wrote:
| Do ISPs and modem vendors roll their own OTA infrastructure
| and signing key management, or contract it out?
| localfirst wrote:
| this along with other recent security incidents suggest somebody
| is rehearsing for massive campaign tied to another geopolitical
| ambitions.
| waihtis wrote:
| well there is a top cyber offensive power whom is de facto at
| war with the west, hardly surprising
| Crosseye_Jack wrote:
| I would say that the rehearsal has long been over. Just before
| Russian troops entered Ukraine Viasat saw an attack on it's
| network which among other countries serves Ukraine, which saw
| updates getting pushed to modems designed to disable those
| modems.
|
| https://en.wikipedia.org/wiki/Viasat_hack
| dralley wrote:
| Said geopolitical entities have conducted much larger and more
| consequential attacks in the past
|
| https://www.wired.com/story/notpetya-cyberattack-ukraine-rus...
|
| https://www.technologyreview.com/2022/05/10/1051973/russia-h...
|
| It's hard to call this a meaningful rehearsal when previous
| attacks knocked actual core power and economic infrastructure
| offline
| sgtaylor5 wrote:
| related article from Ars Technica:
| https://arstechnica.com/security/2024/05/mystery-malware-des...
| skilled wrote:
| That doesn't count as related as it is a rewrite of the
| original source. Just saying, it adds no details of its own.
| thecosas wrote:
| It does include information which the original article
| specifically excluded from mentioning: the ISP involved.
|
| "Windstream" is mentioned in the first paragraph of the Ars
| article, while the Lumen post makes references to "a rural
| ISP" throughout the post.
| skilled wrote:
| So say _that_ instead of _related_ and make people waste
| their time reading the same information.
| thecosas wrote:
| Fair point! That's part of why I added my comment above
| :-)
| jeffbee wrote:
| "Router" being used to mean customer premises equipment, it
| seems.
| TheJoeMan wrote:
| Here I was hoping someone was "encouraging" an ISP to upgrade
| their infra.
| ronnier wrote:
| For a few years now I only buy a small x86 box with dual nics and
| run OpenWRT. I love it. It's open source, lots of support, good
| community. It supports wireguard. Latest version allows you to
| even run docker containers.
| hedora wrote:
| I've got an old PC Engines board with openbsd on it. It's been
| remarkably trouble free for something like 8 years.
| ziml77 wrote:
| The PC Engines boards are great for that. I've got mine
| running OPNSense (FreeBSD) and it's not required any fuss.
| glitchcrab wrote:
| Another very happy PC Engines user here, I've been running
| pfsense and more lately opnsense on one for over 10 years
| now. It has never missed a beat.
| rpcope1 wrote:
| It's huge shame Pascal basically stopped building those
| boards since AMD and Intel wouldn't play ball. I'd really
| like to have something like an APU with 10G connectivity with
| an x86 processor that was not built and designed in China
| running open firmware. With PC Engines gone now, I think
| you're basically out of luck.
| jeffbee wrote:
| These are DSL modems, though. At some point there has to be
| some interface between the WAN side, be it DSL or coax or
| fiber, and your network. Even DSL adapters for PCIe slots are
| just systems on a stick, coming with all the features and bugs
| of a "router" but without the enclosure.
| ronnier wrote:
| You can tell I didn't read the article :)
| fckgw wrote:
| Since it were modems that were affected, OpenWRT would do
| nothing to protect you.
| ronnier wrote:
| Ah, that's what I get for not reading the article.
| nisa wrote:
| OpenWrt works for some modems pretty fine. It's not straight
| forward as the VDSL firmware can't often be distributed but
| poeple use it on avm Fritzbox devices. Also LTE devices are
| supported. Not sure about cable modems, probably not. It's
| probably involved and not straight forward so for most users,
| even technical ones it's no alternative.
| londons_explore wrote:
| > Lumen identified over 330,000 unique IP addresses that
| communicated with one of 75 observed C2 nodes
|
| How does Black Lotus Labs global telemetry know which IP
| communicated with which other IP if they have control of neither
| endpoint? Who/what is keeping traffic logs?
|
| If these guys can do it, remind me again how Tor is secure
| because nobody could possibly be able to follow packets from your
| machine, through the onion hops, to the exit node where the same
| packet is available unencrypted...
| hedora wrote:
| This is reasonably standard functionality for backbone routers.
| They have to parse the TCP headers in hardware anyway, and can
| track common endpoints with O(1) state.
|
| Of course, on the other end of the spectrum, the NSA has tapped
| into core internet links, is recording everything it possibly
| can, and is keeping it forever.
| Hikikomori wrote:
| Yup. Pretty much all ISPs collect sflow/netflow from their
| devices to be able to debug problems or detect ddos.
| sebzim4500 wrote:
| Is that actually feasible with their budget?
|
| If we are generous and assume there a zettabyte of data a
| year that they want to store.
|
| At consumer prices, you would have to pay $10B per year just
| buying hard drives yet alone the operational
| costs/redundancy.
|
| The budget for all of the US intelligence services is ~$65B.
| I think if they wanted to actually do what you are describing
| it would be the single biggest intelligence expense they have
| and I don't see how you hide that.
| oasisbob wrote:
| > They have to parse the TCP headers in hardware anyway
|
| Backbone routers have no need to implement stateful TCP
| inspection or deal with the transport layer for TCP, dealing
| with IP is enough.
| luma wrote:
| Presumably, Windstream is logging customer traffic as a matter
| of course. It might just be metadata (NetFlow/sFlow/IPFIX/etc),
| but one way or the other the only way they have this
| information is if they are recording and retaining it.
|
| Hopefully this is made clear in Windstream's contract terms.
| londons_explore wrote:
| These aren't likely 'top flows', since the C&C data will
| probably only be a few kilobytes.
|
| So to capture this, you at a minimum need to be logging every
| TCP connection's SRC IP and DST IP.
|
| And they seem pretty confident in their worldwide map and
| fairly exact counts, so I would guess they must have probes
| covering most of the world doing this monitoring, and it
| likely isn't just 1-in-a-million sampling either...
| luma wrote:
| For whatever it's worth, what you describe above is
| specifically what IPFIX/NetFlow etc does. Not full-take,
| just metadata for each flow such as the time, src/dst
| ip/port, tcp sequence #, octets sent, etc.
|
| This is common in datacenters for traffic and flow analysis
| for troubleshooting, capacity planning, and the occasional
| incident response.
|
| More details:
| https://en.wikipedia.org/wiki/IP_Flow_Information_Export
| shrubble wrote:
| Lumen (merger of Level3 and CenturyLink) sells services to a
| large part of the Internet and may provide a lot of the
| backhaul for Windstream. In which case they would be in the
| path for monitoring.
| vieinfernale wrote:
| I'm quite disenchanted here. So this means that it is
| practically impossible to avoid IP fingerprints in any way ?
| Even with Tor, VMs, etc ? You'll always be at the mercy of
| whoever runs the show unless you own the physical servers
| miohtama wrote:
| The physical servers do not matter. Someone owns the physical
| cable.
| codexon wrote:
| Lumen is a tier 1 network so a lot of traffic passes through
| them. They can man-in-the-middle the traffic and see the TCP
| packets going through their network.
| perlgeek wrote:
| "They can man-in-the-middle the traffic" could be interpreted
| as them having to actively do something to become the man in
| the middle, when they already are.
|
| It's likely they just do sampling (think netflow) to get some
| statistics over the data that's already transiting their
| network.
| rpcope1 wrote:
| I have a friend who works at Black Lotus (and who may have
| written this blog post, who knows). Black Lotus is part of
| Lumen which is Level3 and CenturyLink and is one of the biggest
| (if not the biggest) backbone traffic provider in the world,
| with a huge percentage of the worlds traffic transiting their
| network, and thus I think they get direct insight into the
| traffic including metrics on it.
| nisa wrote:
| Article is light on the interesting details. How did they came
| in? Do these routers have open ports and services by default and
| answer to the Internet in a meaningful way?
|
| Couldn't someone grab different firmware versions and compare
| them?
|
| Looks like they are doing what everyone else is doing and using
| OpenWrt with a vendor SDK: https://forum.openwrt.org/t/openwrt-
| support-for-actiontec-t3...
|
| What's interesting here is speculated the vendor send a
| malicious/broken update:
| https://www.reddit.com/r/Windstream/comments/17g9qdu/solid_r...
|
| So why is there no official statement from the ISP? If it was an
| attack shouldn't there be an investigation?
|
| I'm not familiar with how this is handled in the USA but this
| looks really strange.
|
| Maybe these machines were bot infested _and_ the vendor pushed an
| update that broke everything?
|
| Maybe it's like in the article and it was a coordinated attack
| maybe involving ransom and everyone got told it's a faulty
| firmware update, keep calm?
|
| which is also kind of bad, as the customer I'd like to know if
| there security incidents.
|
| Has anyone links to firmware images for these devices? Or any
| more details?
| xacky wrote:
| Reminds me of the CIH virus. It's only a matter of time for
| ransomware authors to start using firmware blanking as a new
| technique.
| pragma_x wrote:
| For anyone else that was confused by the headline, this is about
| the destruction of 600,000 individual (small) routers. Not
| routers that are worth $600,000 (each or combined).
| steelframe wrote:
| For my home network I've purchased a networking appliance form-
| factor computer, which is basically a regular old an i3 with VT-x
| support in a fanless case and 4 2.5GiB NICs. I've installed my
| favorite stable Linux distro that gets regular automated security
| updates in both host and a VM, and I've device-mapped 3 of the
| NICs into that VM. The remaining NIC remains unattached to
| anything unless I want to SSH in to the host. I'm running UFW and
| Shorewall in the VM to perform firewall and routing tasks. If I
| want to tweak anything I just SSH in to that VM. I have a
| snapshot of the VM disk in case I mess something up so I can
| trivially roll back to something that I know works.
|
| I've purchased a couple of cheaper commercial WiFi access points,
| and I've placed them in my house with channels set up to minimize
| interference.
|
| Prior to this I've gone through several iterations of network
| products from the likes of Apple, Google, and ASUS, and they all
| had issues with performance and reliability. For example
| infuriating random periods of 3-5 seconds of dropped packets in
| the middle of Zoom conferences and what not.
|
| Since I've rolled my own I've had zero issues, and I have a
| higher degree of confidence that it's configured securely and is
| getting relevant security updates. In short, my home network
| doesn't have a problem unless some significant chunk of the world
| that's running the same well-known stable Linux distro also has a
| problem.
| tmoertel wrote:
| Out of curiosity, which networking appliance form-factor
| computer did you purchase?
| steelframe wrote:
| It's a HUNSN RJ36. It came preloaded with pfSense, as many of
| them do, but I immediately made a full disk backup and then
| wiped and installed with a Linux distro because, well, "This
| is Linux. I know this." You're going to find a lot of people
| who strongly prefer one over the other, and you may find you
| prefer pfSense over a "do-everything-yourself" Linux distro
| if you give it a shot. There are also Linux distros that are
| targeted for network appliances, and setting them up
| (correctly) can be easier if the distro is built for the
| task.
|
| There are quite a few machines in this category, and what's
| in stock at any given time tends to rotate relatively
| quickly. I think the one I bought might still be available,
| but you will want to check to see if there is something with
| specs that will work better for your use case.
| fckgw wrote:
| This attack affected modems so even with all that fancy
| hardware you would still be dead in the water.
| Kiboneu wrote:
| Well if you backdoor 600k routers and introduce a firmware bug
| with one of your patches, this is what happens.
|
| Can't they just stage their updates? Surely, malware authors and
| users must be too cool for adopting standard prod practices.
| perlgeek wrote:
| > Surely, malware authors and users must be too cool for
| adopting standard prod practices.
|
| Their economic pressures are just different, it's not their own
| hardware that they're bricking, nor are likely to be held
| liable for it.
| its-summertime wrote:
| Is the >2x increase in other devices addressed in any form?
| bostonpete wrote:
| What is the significance of the article/post title...?
| thamer wrote:
| This attack happened a few days before Halloween 2023
| (pumpkins), with a large drop in the number of devices
| connected to the Internet - like how an eclipse suddenly brings
| a period of darkness, maybe?
|
| This is just my interpretation, I also found it cryptic.
___________________________________________________________________
(page generated 2024-05-30 23:00 UTC)