[HN Gopher] CrowdStrike's impact on aviation
___________________________________________________________________
CrowdStrike's impact on aviation
Author : jjwiseman
Score : 184 points
Date : 2024-07-29 19:41 UTC (3 hours ago)
(HTM) web link (heavymeta.org)
(TXT) w3m dump (heavymeta.org)
| feyman_r wrote:
| >> Why were other airlines able to get back to normal so much
| faster than Delta?
|
| I read somewhere that their crew tracking software was hit hard
| and took time to recover. Will look for source on that.
|
| (Edited) source: https://news.delta.com/update-delta-customers-
| ceo-ed-bastian
|
| "... and in particular one of our crew tracking-related tools was
| affected and unable to effectively process the unprecedented
| number of changes triggered by the system shutdown..."
| Onavo wrote:
| Because they used Windows 3.1
| ZeWaka wrote:
| In the article it says Southwest used 3.1, not Delta (though,
| that's apparently incorrect according to other posters).
| Someone1234 wrote:
| And Southwest had two crew-management outages in 2022[0],
| so let's not sing their praises for escaping the
| CrowdStrike disruption. Southwest has been widely critized
| for under-investment in technology, Delta on the other hand
| purchased one of the best security products on the market
| and that backfired.
|
| [0] https://en.wikipedia.org/wiki/2022_Southwest_Airlines_s
| chedu...
| chgs wrote:
| Delta put all their eggs in one basket and had no DR
| capability
| Someone1234 wrote:
| What basis do you have for saying that? It is likely
| their DR was running on a mirror of their production
| systems, and was similarly impacted by the Crowdstrike
| outage. So they fell back to Windows Servers similarly
| stuck in a boot-loop.
|
| Keep in mind there was no way to opt out or delay CS
| Channel updates.
| chgs wrote:
| If your DR system is susceptible to the same faults as
| your main system it's not a DR system.
|
| It would be like claiming raid1 is a backup.
| freeopinion wrote:
| > Keep in mind there was no way to opt out or delay CS
| Channel updates.
|
| Do CS updates somehow work over airgaps? You know, the
| kind that production systems have to prevent any access
| to or from external networks? Well... some production
| systems anyway.
| shagie wrote:
| I chased through this chain the other day...
|
| https://www.tomshardware.com/software/windows/windows-31-sav.
| ..
|
| https://www.forbes.com/sites/tedreed/2024/07/20/meltdown-
| wha...
|
| > A story on the website govtech.com on Friday asked the
| question, "Why isn't Southwest affected by the
| CrowdStrike/Microsoft outage?
|
| > "That's because major portions of the airline's computer
| systems are still using Windows 3.1, a 32-year-old version of
| Microsoft's computer operating software," the website said.
| "It's so old that the CrowdStrike issue doesn't affect it so
| Southwest is still operating as normal. It's typically not a
| good idea to wait so long to update, but in this one instance
| Southwest has done itself a favor."
|
| The govetech.com article is https://www.govtech.com/question-
| of-the-day/why-isnt-southwe...
|
| which linked to
| https://www.digitaltrends.com/computing/southwest-
| cloudstrik...
|
| which linked to an earlier Forbes article -
| https://www.forbes.com/sites/hershshefrin/2022/12/31/can-
| sou...
|
| > The December 2022 scheduling fiasco was the result of
| skimping on information technology. I am old enough to
| remember when Microsoft introduced a new operating system
| called Windows 95, to replace its predecessor operating
| system Windows 3.1. The 95 in Windows 95 refers to the year
| of its introduction: 1995. By some accounts, major portions
| of Southwest's scheduling system for pilots and flight
| attendants is built on the Windows 95 platform. That platform
| is now more than 25 years old.
| JumpCrisscross wrote:
| Southwest does not run Windows 3.1:
|
| "That's it. That's where all these stories can trace their
| origin to. These few paragraphs do not say that Southwest
| is still using ancient Windows versions; it just states
| that the systems they developed internally, SkySolver and
| Crew Web Access, look 'historic like they were designed on
| Windows 95'."
|
| https://www.osnews.com/story/140301/no-southwest-airlines-
| is...
| shagie wrote:
| The other day, I saw a screen capture from Tom's Hardware
| and so chased the series of links and quotes to try to
| find the earliest one that had reporting on it that was
| the source. That was the chain that I found.
|
| I am not claiming that they run Windows 3.1 or Windows 95
| ... but rather "this is where that story was sourced
| from" because everyone kept linking to somewhere else.
| The relevant XKCD is https://xkcd.com/978/
| Modified3019 wrote:
| Funny enough, this cycle is close to what the Russian
| disinformation machine does deliberately to spread
| bullshit.
| smileysteve wrote:
| Re Delta
|
| It's not so much a severity as "hard"; but with the hub and
| spoke model that Delta uses, scheduling being down (at all on
| Friday), combined with FAA hour limits. It becomes
| exponentially difficult to reschedule flights.
|
| Put more plainly, on Friday, your scheduling software is down
| for 4 hours in the morning, so you "borrow" any replacements
| you need for employees that are late or sick. This ruins the
| availability for the next flights, at which time you hope the
| system is up again; but if it's not, you borrow from the
| evening flights. Combine this with each flight that was
| late/cancelled as you were hoping to fill now affects the hours
| available for the employees that were available. Finally, as
| you've cascaded this, you head into a weekend trying to catalog
| how many hours each crew member did or did not log, and you're
| not sure how to get them back in time.
| inferiorhuman wrote:
| Except for Southwest the other legacy airlines (United,
| American) also use a hub and spoke model. So does jetBlue.
| crazytony wrote:
| Funny you should mention WN. Delta's meltdown is the exact
| same scenario as Southwest. Crew scheduling is messed up,
| they don't have a way of tracking where employees are, if
| the employee is legal, etc and so the operation grinds to a
| halt
| sidewndr46 wrote:
| wouldn't this imply either an upper bound on down time
| (airline simply folds as it never catches up) or an upper
| bound on the duration of the impact ?
| reaperducer wrote:
| _> > Why were other airlines able to get back to normal so much
| faster than Delta?
|
| I read somewhere that their crew tracking software was hit hard
| and took time to recover. Will look for source on that._
|
| I heard on the radio (maybe NPR, not sure) it wasn't about the
| computers, it was about Delta's response.
|
| According to the report, the other airlines delayed flights,
| while Delta cancelled them outright. That left Delta with more
| people and planes in the wrong places, making it harder to
| recover.
| crazytony wrote:
| One other compounding problem is that Delta's headquarters and
| main traffic patterns are on the east coast. Crowdstrike
| affected all the airlines at roughly the same time. This gave
| them roughly one to two fewer hours to respond before they hit
| their morning peak flights.
|
| As someone else pointed out, they probably weren't ready by the
| time they needed their systems for the morning rush so they
| went to their business continuity strategy (manual). This has a
| throughput and recovery time penalty and obviously it compounds
| the longer they are in that mode.
|
| I think what we're finding with the Southwest meltdown and now
| the Delta meltdown is that the big airlines just don't have the
| manpower or scheduling slack to accommodate going into business
| continuity. I do think this should be investigated. Hopefully
| financial penalties incentivize action but time will tell.
| firtoz wrote:
| Is there a similar global analysis?
| fullspectrumdev wrote:
| I'd love to have some solid numbers of "global cancellations
| due to" - I heard a bunch of varying figures so far.
| jjwiseman wrote:
| Maybe I'll do a Part 2: The World.
| ssivark wrote:
| > _Apparently Southwest Airlines' ingenious strategy of never
| upgrading from Windows 3.1 allowed it to remain unscathed._
|
| OMFG, does this mean we need to be prepared for a (juicy) "IT
| failure" that brings down Southwest at some point?
| ryanmcbride wrote:
| You don't even have to wait it's been happening
| Someone1234 wrote:
| Southwest had two of these recently. It was widely reported:
|
| https://en.wikipedia.org/wiki/2022_Southwest_Airlines_schedu...
| toast0 wrote:
| Southwest experienced this kind of scheduling issue in 2021
| [1], and again in 2022 [2]. Honestly, if they're running win
| 3.1 or win 95 as suggested, I think that puts them in a better
| place tech wise than keeping up with the Joneses on the upgrade
| treadmill --- although they should consider updating to windows
| 3.11, because they have a workgroup :P and the microsoft hearts
| network is pretty cool; but they have historically done poorly
| on scheduling after a significant disruption. An article from
| last year [3] says they updated their crew assignment software
| as well as increased staffing in colder airports and in general
| and got more deicing equipment. We won't really be able to tell
| if it works, until they experience another disruption.
|
| [1] https://www.cnbc.com/2021/10/12/southwest-airlines-
| reduces-c...
|
| [2] https://www.npr.org/2022/12/26/1145536902/southwest-
| flight-c...
|
| [3] https://www.npr.org/2023/11/09/1211064462/southwest-
| airlines...
| tracerbulletx wrote:
| This isn't true, and that should have been obvious to technical
| people. It's so sad that we have a tech media that doesn't give
| a damn about making things up.
| bedobi wrote:
| > Apparently Southwest Airlines' ingenious strategy of never
| upgrading from Windows 3.1 allowed it to remain unscathed.
|
| this is pretty damning both ways
|
| on the one hand, it's insane, unfathomable and inconceivable that
| anyone can run anything critical on windows 3.1 (!!!)
|
| on the other hand, it's equally insane, unfathomable and
| inconceivable that those who do are actually _better_ off - 30
| years of "progress" is actually just bs? what are we as an
| industry "even doing here"???? is computing actually a solved
| problem and we're really just mostly reinventing the wheel and
| enshittifying perfectly already working systems?
| TonyTrapp wrote:
| From my memory, this wildly circulating Windows 3.1 quote is
| inaccurate. The software they were running was compared to
| running something like Windows 3.1, but it wasn't actually
| running on Windows 3.1, as far as I understand.
|
| Edit: https://kotaku.com/southwest-airlines-windows-3-1-blue-
| scree...
| Swizec wrote:
| > is computing actually a solved problem and we're really just
| mostly reinventing the wheel and enshittifying perfectly
| already working systems?
|
| 80% of the work is json bureaucracy
|
| The other 80% is adapting to new requirements
|
| And if you're lucky maybe 0.1% of the time you get to build
| something new.
|
| Fear not, a lot of this stuff was perfectly solved with pen and
| paper long before us computer nerds came to play in the big boy
| sandbox
| nightpool wrote:
| Obviously Southwest is not using Windows 3.1, and you should
| probably be thinking hard about the trustworthiness of any
| outlet or article that repeats that claim:
| https://www.osnews.com/story/140301/no-southwest-airlines-is...
| m3kw9 wrote:
| It's like a calculator, if you need to do basic math, you can
| use an old calc.
| jandrese wrote:
| In the long run the hardware that can still run Window 3.1 will
| become harder and harder to find and they'll be forced to
| upgrade, but currently enjoying the benefits of "if it ain't
| broke don't fix it". Plus, there were literally millions of
| systems made that can run Windows 3.1 so it will be many many
| years before the hardware is too hard to find.
|
| We're talking about a problem on the scale of 4,000 flights per
| day. Assuming you avoid O^2 complexity computations that's the
| sort of thing even 90s computers could handle easily.
| arsome wrote:
| You can basically run Windows 3.1 in dosbox on a potato now,
| so the hardware really isn't even a problem. If any of this
| was actually true...
| burningChrome wrote:
| >> Plus, there were literally millions of systems made that
| can run Windows 3.1 so it will be many many years before the
| hardware is too hard to find.
|
| Two of my first contractor roles I had as a developer really
| opened my eyes to a lot of this.
|
| We were building an inventory management system for a large
| company that built farm equipment. We started building it and
| once we got to the browser and mobile requirements, one of
| the VPs spoke up and asked if it would run on IE6 since they
| had not one, but THREE of their inventory legacy systems that
| still ran on Win98. This was in 2014, a full 6 years after
| many companies had stopped supporting it. And another 4 years
| since websites stopped supporting it.
|
| The other one was for a very large, regional construction
| company. Same thing, we were building a web app for them and
| in one of the conference calls, one of the VP's was asking
| how this would run on Windows95 for the same reason. They had
| several legacy ERP systems that were running on Win95 and had
| specific requirements for stuff to run on that OS.
|
| As a developer who was used to working with somewhat current
| tech - it was a real eye opener. It was crazy to think how
| many massive companies just didn't have the constitution to
| upgrade their stuff, and then by not doing so, had now dug
| themselves into an even deeper hole.
|
| Once I started hearing stories about the details of this
| scenario, it made perfect sense to me since I had seen it
| multiple times. And not from little companies who didn't have
| the money or resources to upgrade, but massive Fortune 500
| companies who just neglected their stuff until was too late.
| shagie wrote:
| Thirty years of progress is still progress for managing the
| additional complexity that modern software needs.
|
| A lot of software doesn't need that additional complexity.
| Having thirty year old software, if properly sequestered (since
| there are security holes large enough to fly a 737 through),
| means that this is software that has been working for three
| _decades_. It has issues (as the mess they had previously
| showed), but Southwest appears to be able to be able to manage
| that to some degree without needing to incur the additional
| complexity of managing a modern software stack for application
| software that doesn 't need it.
|
| The ability to play minesweeper on critical computing equipment
| without impacting it isn't necessarily a desirable feature.
| Having the computer boot in five seconds and run the desired
| application _is_.
|
| And there are a number of ways to handle that ... running old
| operating systems is one of the ways. Space Force S02E07 is not
| a desirable situation https://youtu.be/xDLvUqhwHZc . You could
| also have a kuberentes cluster with multiple replicas and load
| balancing and all of that additional complexity that takes more
| people to be able to manage without any real gains in what the
| application itself is doing.
| Wytwwww wrote:
| > Having the computer boot in five seconds
|
| There are certainly more modern options that allow that and
| it's highly doubtful that specifically is particularly
| relevant for Southwest.
|
| Not that there is any evidence that they're actually using
| 3.1 for anything?
|
| > that this is software that has been working for three
| decades
|
| Or it's so buggy or designed (or more likely updated) so
| poorly that everyone is afraid to touch it. e.g. I doubt
| there are many (even any?) practical reasons for airlines to
| use GDS besides the cost and complexity involved in designing
| an entirely new system and somehow forcing all other airlines
| to switch to it?
| shagie wrote:
| > Not that there is any evidence that they're actually
| using 3.1 for anything?
|
| Windows 3.1? No. I'd even say there's no evidence that
| windows 95 is being used but rather that they've got what
| appears to be some old software with older design.
|
| https://www.dallasnews.com/business/airlines/2022/12/30/wha
| t...
|
| > 2. The crew scheduling system is the main culprit.
|
| > Southwest uses internally built and maintained systems
| called SkySolver and Crew Web Access for pilots and flight
| attendants. They can sign on to those systems to pick
| flights and then make changes when flights are canceled or
| delayed or when there is an illness.
|
| > "Southwest has generated systems internally themselves
| instead of using more standard programs that others have
| used," Montgomery said. "Some systems even look historic
| like they were designed on Windows 95."
|
| Screen shots of this are in http://www3.alpa.org/LinkClick.
| aspx?fileticket=IO7kd%2Bfm2Do...
|
| Unfortunately, I don't know the nuances of Microsoft
| Windows UI well enough to be able to pick out which OS
| version is running the software in those screen shots.
|
| ---
|
| > Or it's so buggy or designed (or more likely updated) so
| poorly that everyone is afraid to touch it.
|
| That is a _very_ common occurrence (I 'm dealing with that
| now ... a .jar file that hasn't been rebuilt in 15 years).
| The big rewrite is something that comes with one part
| excitement (I can do it right this time!) and dread (oh my,
| that's how much code that I need to retest?!).
|
| I was involved in the tail end of a 3 year project at one
| company with some software that replaced previously running
| DOS (and yes, it was DOS - they had an C and assembly guru
| employed who's job it was to remove / optimize code in the
| binary to get it to fit into 640k) to a Java Web Start
| (which was a neat technology) and the millions of lines of
| software that monstrosity had and needed to be debugged and
| fixed.
|
| While they're in a better spot now (can use modern
| hardware), and its something that they can build in house
| (a major part of the reason to do it was to drop the
| external contractor who didn't like maintaining the C code)
| ... but that also came with the added complexity of the
| software that they licensed and the maintenance and
| deployment of that software. Before they could put the
| software on a floppy and have it shipped to each location
| ... now its a big bigger and more complex of a deployment
| (that was built with duct tape and chewing gum one night to
| do diff deployments of specific class files rather than
| trying to push the entirety down the pipe for each
| location).
|
| My rambling point is that we are moving forward with
| complexity - and that allows us to manage more complex
| situations, but it comes at the cost of managing that
| additional complexity of the infrastructure and software it
| needs and that cost is ongoing and not always taken into
| account.
| jwagenet wrote:
| Those screenshots look more like WinXP to me with the
| rounded and shaded button elements. It's the boring grey
| and buttons people presumably associate with the 90s.
| spookie wrote:
| Most times its less about a system being poorly designed
| and more about it being able to solve very hard problems
| which most existing employees today haven't even heard of.
| Institutional knowledge plays big time on these decisions.
| aftbit wrote:
| Well all you'd really need to do to avoid this outage is not
| run auto-updating proprietary kernel modules in the early anti-
| malware environment. Bare Windows 11 would have been fine - the
| problem was Crowdstrike.
| jakub_g wrote:
| You don't want to know what OS Sabre (backend for 30% of
| world's airlines) is using on their mainframes.
| bedobi wrote:
| actually, I do :) is it DOS? some IBM mainframe OS? do tell
| Wytwwww wrote:
| Was DOS actually every used on mainframes/servers on a
| significant scale? (genuine question, not saying it wasn't)
| wrs wrote:
| A mainframe OS called DOS was in fact quite popular, but
| it's not the same thing as the DOS that was in PCs.
| (There were others, too, like Apple ][ DOS. As soon as
| your computer gets the capability of attaching a disk
| drive, somebody has to write a Disk Operating System.)
| kayodelycaon wrote:
| I do actually. My last job had a mainframe team maintaining
| (and adding to) an AS/400 application. They still had
| punchcard programs.
|
| They had json apis. Each one had some variation on parsing
| http from a raw tcp connection with IBM RPG. I had to do some
| unspeakable things to a ruby library so I could control the
| order of the headers.
| fourteenfour wrote:
| Looks like it was IBM System/360 mainframes but they've
| recently migrated to google hosted services.
| numpad0 wrote:
| It is BS. Continuous updates for security notion, especially
| so. That said, the barrier to entry for programmers did come
| down significantly.
| alliao wrote:
| productivity wise 70's and 80's peaked with those thin
| terminals with every action carried out by keyboard... workers
| tapped at light speed due to muscle memory, didn't look fancy,
| but it got the job done. GUI is sexy but like short videos
| ultimately did nothing for the user
| JumpCrisscross wrote:
| > _productivity wise 70 's and 80's peaked with those thin
| terminals with every action carried out by keyboard_
|
| Computers only started showing up in nationwide productivity
| figures in the 80s to 90s.
| Wytwwww wrote:
| > GUI is sexy but like short videos ultimately did nothing
| for the user
|
| That's a stretch.
|
| I guess it increased the productivity (measured in amount of
| "work" done, not necessarily something useful) expectations
| for most workers which effectively did nothing for them
| because they still need to work as much even if modern
| software allows them to accomplish much more in the same
| amount of time. So t might make sense in that regard.
|
| > workers tapped at light speed due to muscle memory
|
| That's great if we're mainly talking about robotic tasks than
| can be mostly automated to only require a fraction of those
| clicks but wasn't for some unclear reasons.
| ToucanLoucan wrote:
| Almost every industrial embedded system I've ever used runs
| Windows XP at the absolute _newest,_ and it is not uncommon in
| the slightest to see stuff as old as 95 /3.1. These are
| computers that operate machinery that costs 6 or 7 figures. If
| it ain't broke, don't fix it.
|
| Don't get me wrong, my Macbook is an absolute beast for all of
| my work tasks, and my gaming PC is an utter joy to use for my
| recreation time, but at the end of the day, for a ton of
| applications, a computer doesn't need to do shit beyond sending
| a lot of signals out of a parallel/RS232 port to control
| systems to operate... I mean Christ, anything. CNC mills,
| building lighting/security systems, packing machines, or to do
| things like issue tickets to people parking in a ramp. Like...
| a lot of this stuff just does not benefit at all from a modern
| software stack. Stick a crappy PC inside instead, load it up
| with the same image it had before which includes firewall rules
| that shut down every last port and connection apart from
| whatever needs to manage it, and you're done.
|
| Don't fix what ain't broke.
| josephg wrote:
| > is computing actually a solved problem and we're really just
| mostly reinventing the wheel and enshittifying perfectly
| already working systems?
|
| On a whim I tried playing Solitaire on windows the other day.
| You know, that game that's shipped with windows since forever.
| Well, it's horrible now. When I tried firing it up, it first
| spent several minutes downloading software updates. Then it
| loaded in some horrible "casual games bundle" app which felt
| laggy like a web app - complete with Xbox cloud sync for my
| progress, and daily achievements and other junk.
|
| The game used to run flawlessly on my old 486. My computer now
| is orders of magnitude more powerful - but solitaire feels
| laggy. I bet the entirety of windows XP is smaller than the
| "update" it performed to install solitaire.
|
| I have a personal theory that there's always something that
| gets the attention of the best engineers. Decades ago it was
| human interface guidelines and UI toolkits. Today it's LLMs and
| AAA game engines. Most of the rest of the software in the world
| is worked on by the B team. And they don't blink an eye at the
| idea of rewriting solitaire for windows on top of electron. If
| JavaScript is all their team knows, so be it. Heaven forbid we
| have to learn how to properly build software for windows.
| AnimalMuppet wrote:
| Well, what actual new features does Windows 11 give you
| compared to Windows 3.1?
|
| It will support a huge number of new chips, new peripherals,
| more memory, and so on.
|
| If I'm running Southwest's crew scheduling software, how much
| of that do I care about? Do I care that it will now support the
| latest Bluetooth? Do I care that it now has the same UI as
| tablets? Do I care that it has better ads to display on the
| start menu? No, no, and _no_.
|
| The only thing might be more memory. (I mean, the UI might not
| look like it belonged in the Stone Age, so that's something, I
| guess...)
|
| There hasn't been a real fundamental improvement in the
| _functionality_ of OSes since Windows 3.1. It 's all been
| device support (including new classes of devices), new CPU
| support, and new UI styles. (The security improvements in
| Windows were a legitimately big deal, but those were fixing
| what was broken, not adding new functionality.)
|
| And I'm sure that, having said this, someone is going to point
| out something really important that I forgot...
| Wytwwww wrote:
| > If I'm running Southwest's crew scheduling software, how
| much of that do I care about?
|
| Not a lot of if you can't/don't want to upgrade or replace
| that software to make sure it runs on modern OSes. I'm sure
| that for the most part that software is causing various
| unnecessary issues and decreasing potential productivity at
| least to some extent. Just look at GDS, they love to get rid
| of that, but that would require a coordinated effort and
| extensive collaboration between all major airlines which is
| somewhat tricky.
|
| > There hasn't been a real fundamental improvement in the
| functionality of OSes since Windows 3.1.
|
| Multitasking? A massive amount of other important features
| that matter if you want to build new software or
| significantly improve what you're using now.
|
| Also you seem to be downplaying security a but too much?
| Those devices would need to be carefully isolated from
| everything else (not that as I understand there is any
| evidence that Southwest Airlines is actually using 3.1?).
|
| > but those were fixing what was broken, not adding new
| functionality
|
| It's like saying that every new feature in introduced in any
| type of software that wasn't entirely novel was actually
| fixing stuff that was broken and wasn't "new". I guess some
| would apply to the claim GUI/desktop wasn't something new but
| it was just "fixing" (inherently "broken") command line
| interfaces?
|
| Being able to design significantly objectively better (based
| on how much it could increase productivity) is I guess is not
| strictly tied to the OS at least in some cases. But it
| certainly make it a lot cheaper/easier.
| AnimalMuppet wrote:
| Multitasking. Yeah, I'll give you that. That is actually a
| huge step up.
|
| I'm presuming that Southwest's internal software backends
| are not internet exposed, which is why I'm downplaying
| security.
| qingcharles wrote:
| It was a "troll tweet" says the guy who started the rumor:
|
| https://x.com/ArtemR/status/1815408553131426179
| jujube3 wrote:
| Sounds like we saved a lot of tons of CO2.
| more_corn wrote:
| Always look on the bright side!
| aflag wrote:
| Hard to say, it could actually increased emissions. As when the
| timings of things don't align correctly it's common to cause an
| increase of resource usage. Eg. people travelling less optimal
| route, extra commutes back and to the airport. People having to
| physically travel to datacentres in order to fix things, just
| rebooting the machine without need will use more CPU.
| mbreese wrote:
| Or planes taking less efficient routes or flying at faster
| speeds to "make up time".
| systemtest wrote:
| In my country, companies are required by law to keep track of
| their CO2 emissions.
|
| In the case of CrowdStrike, they would have been able to deduct
| this event from their emissions for decades.
| aftbit wrote:
| One interesting feature of this outage was that "PROD" was
| generally fine, on account of mostly running on Linux and/or
| ancient proprietary software, while "CORP" was generally wrecked,
| on account of mostly running Windows. In other words, the bank
| systems responsible for moving money mostly worked, while the
| systems responsible for allowing humans to interact with them (to
| issue approvals, change configuration, or other ops things) often
| did not.
| brazzy wrote:
| In the original thread there were some reports of people having
| their Linux systems taken down by Crowdstrike as well. At
| separate times, of course, and I supposed the greater
| heterogeneity of Linux distros prevents events of this
| magnitude. But that would be little consolation when it takes
| down your systems.
| foobarchu wrote:
| Those should be considered coincidence until proven
| otherwise. Crowdstrike is intended to bring down systems when
| it believes there was an intrusion, after all.
| 7thaccount wrote:
| Same thing for a lot of industries actually. PROD runs on Linux
| and probably has some delay to prevent this. Corp gets hosed.
| LeifCarrotson wrote:
| Yep, here in manufacturing production/OT PLCs run on Wind
| River VxWorks from Rockwell, Siemens, and others. The HMI
| (human-machine interface, basically a touchscreen used to
| display status and enter setpoints and other data) and
| SCADA/ERP systems run on Windows. Sometimes, this is an
| industrial fanless PC with eg. Ignition (Java+Python)
| software, other times it's a Rockwell Panelview which
| actually still run Windows CE 6.0.
|
| This gets to be a problem when IT wants to get their hooks
| into OT networks. The PLC is meant to be left alone, and will
| happily send its Ethernet packet to that servo drive or
| digital IO card every 10ms for literal decades. There is no
| reason to update its firmware ever, just don't expose it to
| the Internet. But corporate wants everything on the Internet.
|
| The PLC will reliably run its sequence when you close the
| contacts on the physical "Cycle Start" pushbutton. But if
| corporate is down, you can't know what part number you're
| supposed to make or how many of them, or get a serial number
| from and report test results to the traceability database.
| ks1723 wrote:
| I found it quite interesting, that crowdstrike actually exclude a
| bunch of services explicitly. They also basically say, don't use,
| if it needs to be reliable. I don't know if this is standard for
| software, but for me this was quite surprising.
|
| From crowdstrike terms and services [1]: [...] THERE IS NO
| WARRANTY THAT THE OFFERINGS OR CROWDSTRIKE TOOLS WILL BE ERROR
| FREE, OR THAT THEY WILL OPERATE WITHOUT INTERRUPTION OR WILL
| FULFILL ANY OF CUSTOMER'S PARTICULAR PURPOSES OR NEEDS. THE
| OFFERINGS AND CROWDSTRIKE TOOLS ARE NOT FAULT-TOLERANT AND ARE
| NOT DESIGNED OR INTENDED FOR USE IN ANY HAZARDOUS ENVIRONMENT
| REQUIRING FAIL-SAFE PERFORMANCE OR OPERATION. NEITHER THE
| OFFERINGS NOR CROWDSTRIKE TOOLS ARE FOR USE IN THE OPERATION OF
| AIRCRAFT NAVIGATION, NUCLEAR FACILITIES, COMMUNICATION SYSTEMS,
| WEAPONS SYSTEMS, DIRECT OR INDIRECT LIFE-SUPPORT SYSTEMS, AIR
| TRAFFIC CONTROL, OR ANY APPLICATION OR INSTALLATION WHERE FAILURE
| COULD RESULT IN DEATH, SEVERE PHYSICAL INJURY, OR PROPERTY
| DAMAGE. Customer agrees that it is Customer's responsibility to
| ensure safe use of an Offering and the CrowdStrike Tools in such
| applications and installations. CROWDSTRIKE DOES NOT WARRANT ANY
| THIRD PARTY PRODUCTS OR SERVICES.
|
| [1] section 8.6 of https://www.crowdstrike.com/terms-conditions/
| objclxt wrote:
| > I don't know if this is standard for software
|
| This is pretty standard. There is almost identical language in
| the Windows and macOS EULAs, for example.
| ale42 wrote:
| Same for datasheets of most electronic components. The
| manufacturers don't want the responsibility to avoid possible
| multi-million lawsuits.
| SoftTalker wrote:
| So how does it get installed on all the endpoints in 911
| dispatch centers?
| nemonemo wrote:
| What is the alternative? Have you considered a possibility
| that those could be the best out there for 911 despite
| their imperfections?
| wrs wrote:
| Because no endpoint protection software exists that doesn't
| have the same disclaimer clause. So you install this one
| and accept the lack of vendor liability.
|
| (If such a thing did exist, it would cost a lot more!)
| EvanAnderson wrote:
| Because FBI CJIS requirements, adopted by state law
| enforcement bodies, require it. I support a Public Safety
| Answering Point (PSAP, aka a 911 call center) and I push
| back on as many of the inane requirements as I can with
| compensating controls.
|
| Example: As of right now I am still required to expire
| passwords every 90 days. My state is considering the
| current guidance from NIST but FBI CJIS policy still
| mandates the expirations.
| sidewndr46 wrote:
| My experience has been better legal counsel has the relevant
| terms struck before the deal is signed. In this case it would
| have been the terms around Aircraft and aviation
| mjevans wrote:
| Outsourcing a core business competency and surely also cutting
| the contracts to the bone as well to pocket the savings
| embrittled Delta and I seriously hope the compensation to
| customers costs more than any savings or profits they made in the
| interim. It MUST be painful enough that they do not repeat this
| mistake again.
|
| The article quotes
| https://www.reddit.com/r/delta/comments/1edtfbh/why_did_delt...
| (with improper attribution)
|
| topgun966Platinum wrote on Reddit """ These "experts" are
| completely wrong. The core issue was Delta did NOT have a proper
| DR plan ready and did NOT have a proper IT business continuity
| plan ready. UA, AA, and F9 recovered so fast because they had
| plans on stand-by and engaged them immediately. After the SWA IT
| problem, UA and AA put in robust DR plans staged everywhere from
| the server farms, to cloud solutions, to end-user stations at
| airports. They had plans on how to recover systems. DL outsources
| a lot of their IT. UA and AA engaged those plans quickly. They
| did not hold back paying OT for staff. UA and AA have just as
| much reliance on Windows as Delta. AA was recovered by end of
| data Friday and resumed normal operations Saturday. UA was about
| 12 hours behind them having it resolved by Saturday morning
| resuming normal schedules Saturday afternoon. The ONUS is 100% on
| DL C+ level in their IT decisions. The problem is that the lower
| level IT staff is going to get the brunt of the blame and the
| consequences. """
| rdtsc wrote:
| From the included link:
| https://www.techradar.com/pro/security/southwest-airlines-av...
|
| > To give you an idea of just how outdated this operating system
| is, Windows 3.1 was originally launched in 1992, and Microsoft
| ended support for it on December 31, 2001, except for the
| embedded version, which was officially retired in 2008.
|
| I keep hearing the Windows 3.1 story repeated. I mean here it
| comes from TechRadar and even has the "Pro" in the name, they
| can't possibly make stuff up, right? But still don't quite
| believe it.
|
| Can anyone working at Southwest confirm that their main
| scheduling system is running on Windows 3.1?
| JumpCrisscross wrote:
| > _keep hearing the Windows 3.1 story repeated_
|
| It's wrong [1] and serves as a litmus test for whether an
| outlet independently verifies its claims.
|
| ("The systems [Southwest] developed internally, SkySolver and
| Crew Web Access, look 'historic like they were designed on
| Windows 95'." That got mangled into they run 3.1.)
|
| [1] https://www.osnews.com/story/140301/no-southwest-airlines-
| is...
| jjwiseman wrote:
| Thanks, I updated the post.
| kragen wrote:
| i miss the lemonodor blog
| xp84 wrote:
| Wow, that's even more frustrating considering it's conflating
| an unfashionable UI (which I'd argue is a good thing, since
| all modern UI trends are towards slick, minimalism-worshiping
| messes which hide everything from users) and old, provably-
| flawed technological foundations (like a 16-bit system
| without things like filesystem access control or memory
| protection).
|
| I knew this story was false immediately though because no
| company ever even in 1993 had production server systems which
| ran a desktop OS like Win 3.1. It just wasn't up to the task.
| They would have used NT if anything.
| cjbprime wrote:
| Windows 95 is an "unfashionable" OS which has not received
| any security updates since 2001.
| andrewxdiamond wrote:
| Yes and the fact that my software's UI looks like Windows
| 95 makes it vulnerable to all the same security
| vulnerabilities.
|
| /s
|
| The systems don't run on W95, they look like W95
| btown wrote:
| http://www3.alpa.org/LinkClick.aspx?fileticket=IO7kd%2Bfm2D
| o... shows the system as of 2020. To the parent's point,
| it's actually quite a reasonable UX, with colored outputs,
| filter banks, and just enough abbreviations and whitespace
| to balance density with intuitiveness.
|
| But that doesn't mean this is the only modern design system
| that meets those requirements. And conflating all modern UI
| with consumer design trends is an equally frustratingly
| broad statement.
| quotemstr wrote:
| Broken link
| veggieroll wrote:
| Link worked for me but took a long time to load. It just
| seems like their server is overloaded.
| qingcharles wrote:
| OK, this is definitely unfashionable looking if your main
| exposure to apps is the latest doodah on your phone that
| was literally updated yesterday.
|
| Very standard looking legacy Win32 looking app. Which,
| admittedly, would have probably look very similar had it
| been on Windows 3, but is probably running on LTSC
| Windows 10 or something in reality.
| spookie wrote:
| Being blasted by media for running your own software,
| incredible. As others have commented, just a single tweet was
| enough to propagate this story. Quite concerning how easy it
| is to fake reality nowadays.
| madeofpalk wrote:
| This is the same as the "Olympic cardboard beds are anti-sex"
| fake story that persisted. Anyone who publishes it
| demonstrates they don't actually research.
| zitterbewegung wrote:
| I know this is a hot take but companies have to figure out if
| modernization of a UI will be worth it to retrain everyone in
| the new UI. Many people were involved with its creation and
| maintenance and due to its age the UI may have a large amount
| of glue code that can't be separated unless you build an API
| around the other software. Especially if there is some kind
| of change in the system that moving off the old one is
| meaningless. Southwest is also making changes to their
| operations so they probably might be in maintenance mode for
| the software especially when the outage of their current
| software was done since they will have to not have anyone
| choose any seat at this time. [1]
|
| [1] https://www.cnn.com/2024/07/25/investing/southwest-
| airlines-...
| stavros wrote:
| I don't know, I like the classic Windows UI. I don't think
| modern UIs are an improvement on that.
| hn_throwaway_99 wrote:
| The "Southwest uses Windows 3.1" claim is false, and is a great
| example of how bullshit can spread on the Internet once some
| semi "reputable" organizations repeat the false rumor:
|
| https://kotaku.com/southwest-airlines-windows-3-1-blue-scree...
| dsr_ wrote:
| Tech Radar quotes Tom's Hardware; Tom's Hardware quotes a
| tweet.
|
| Not a tweet from Southwest, mind you. Not even a tweet from
| someone who says that they used to work for Southwest. Just...
| a tweet.
| shombaboor wrote:
| I just wish there was some type of identifiable credit /
| penalty system for writing accurately as a news source. And
| this would include quotes / retweets. Never been a better
| time to be wrong about everything.
| JumpCrisscross wrote:
| > _wish there was some type of identifiable credit /
| penalty system for writing accurately as a news source_
|
| Good starting point is if the news is free. A shocking
| fraction of people get their news from solely free sources.
| mewpmewp2 wrote:
| And why would someone put in effort for free?
| kspacewalk2 wrote:
| What's "solely free"? Does the ad-driven model count as
| free? Why do you think an outlet that works for you will
| necessarily deliver better quality news that the one that
| works for advertisers? There are obvious bias downsides
| to both.
| sxg wrote:
| The ad-driven model does count as free, and it's far less
| likely to deliver better quality news than a subscription
| service users pay for. The core metric for ad-driven news
| sites is maximizing views--it doesn't matter how you get
| views as long as you get them. This means free sites are
| heavily incentivized to be the first to break a news
| story even if the details are wrong or sparse. Sure,
| they'll issue corrections and updates later, but only a
| small percentage of the initial viewers will ever see
| these, and there's essentially zero cost for having made
| the mistake.
|
| The core metric for subscription news sites is minimizing
| churn. A mistake will cost a subscription site
| subscribers who have a massive lifetime value. These
| sites are heavily incentivized to report high quality,
| accurate news even if they're not the first to break the
| story.
| Analemma_ wrote:
| Your cure is worse than the disease. The second such a
| system existed, it would be gamed to hell and back, and
| nobody would believe it anyway since they'd all angrily
| insist that "you shouldn't have counted X" or "you
| should've counted Y more" and it would just turn into a war
| over who got to control the system and use it to deplatform
| their enemies.
| andrewflnr wrote:
| It doesn't have to, and indeed shouldn't, be a single
| system. We'd rather have a handful of independent news
| checker orgs, maybe some topic-specific ones. Funding
| remains an exercise for the reader.
| treflop wrote:
| There just isn't. You just have to read enough of one
| source to determine your own opinion.
|
| Just like with anyone you meet: you are the judge if they
| are trustworthy, nice, mean, funny, etc.
|
| That said, I think tech journalism is the bottom of the
| barrel. I just feel like they focus more on tech than
| journalism.
| mardifoufs wrote:
| Community notes on twitter is the closest thing to what
| you're describing I've seen yet. It's been very helpful too
| imo
| thereddaikon wrote:
| A great example of why people don't trust journalists
| anymore. They don't even perform a basic amount of fact
| checking before publishing.
| colechristensen wrote:
| Articles from the likes of Tech Radar or Toms Hardware I
| would trust to a higher standard than a random tweet, but
| really I wouldn't label them as "real journalists"
|
| I question the ethics and standards of the New York Times
| at least a little at this point so it's not like great
| journalism is common.
| jxy wrote:
| I don't trust any kind of generalization like this, which
| only serves further disinformation and misinformation.
|
| There are bad journalists (if they can be called
| journalists at all) and good journalists. At this point in
| history, our only hope lies with diligent reporters from
| reputable publishers.
| Dalewyn wrote:
| As far as I'm concerned at this point, journalists are
| cancers upon society without exception. The world would
| be a lot more peaceful were it not for journalists'
| constant sensationalizing of peoples' fear and anger.
|
| I will not bat an eye when most of them are eventually
| replaced by "AI" and other forms of automation. It
| doesn't take entire payrolls of full time humans to spew
| sensational bullshit, certainly not anymore in this day
| and age.
|
| And yes, I'm jaded. Very much so. Been taken for many
| rides over the past 20 years if not more and I'm
| sincerely sick of it. Screw journalism.
| Terr_ wrote:
| It's kind of depressing to think that we have had this world-
| spanning system of knowledge and "hyperlinks" for decades
| now, individual pieces that _should 've_ enabled an easy
| chain of attribution/citation...
| Y_Y wrote:
| And encourage the reader to move away from your site!? No
| self respecting PHB could condone such a thing.
| nostromo wrote:
| I've started seeing this on Wikipedia.
|
| Wikipedia sources an article from a semi-legit source. That
| semi-legit source either just says "sources" or points to
| something less-legit, like a Tweet.
|
| You can bring new "facts" into existence by just laundering
| them from lower- and lower-quality sources.
| umvi wrote:
| > Can anyone working at Southwest confirm that their main
| scheduling system is running on Windows 3.1?
|
| I can't confirm that, but I can certainly confirm lots of
| hospital equipment is still running Windows XP and lots of
| hospital personnel browse the internet with Internet Explorer.
| ponector wrote:
| This story is another example how hallucinations from LLM can
| successfully replace many "news" portals.
| qingcharles wrote:
| The guy that started it all said it was just a "troll tweet":
|
| https://x.com/ArtemR/status/1815408553131426179
| brianpan wrote:
| The San Francisco subway runs off of 5-inch floppy disks.
|
| https://sfstandard.com/2023/02/02/sfs-market-street-subway-r...
|
| That article links to an (only slightly older) article about
| British Airways loading navigation updates every month off of
| the fancy new 3.5-inch floppy disks.
| Zigurd wrote:
| I would like to know if a solid, up to date, well-rehearsed
| disaster recovery plan saved anyone's butt, or if we're all just
| raw dogging our machines whether IT is paying for backup and
| recovery or not?
| Zigurd wrote:
| I see just moments after I posted, someone posted this:
| https://news.ycombinator.com/item?id=41103486
|
| So, yeah, lack of DR is why Delta was so screwed.
| ta1243 wrote:
| Our systems worked fine, we expect things to fail - including
| software like sentinal one, crowdstrike, etc, and have DR
| systems which can keep us limping along. We have DR systems
| which will work should other things happen - say the Thames
| barrier fails (i.e. no docklands)
|
| Unfortunately some of our outsourced suppliers didn't have such
| attitudes.
| xyst wrote:
| > Apparently Southwest Airlines' ingenious strategy of never
| upgrading from Windows 3.1 allowed it to remain unscathed.
|
| The "ingenious" strategy saved them from a weeks worth of
| downtime this year. But that same "ingenious" strategy was the
| primary reason for their meltdown in 2022
|
| [1] https://www.npr.org/2022/12/30/1146377342/5-things-to-
| know-a...
|
| [2] https://www.nytimes.com/2022/12/28/travel/southwest-
| airlines...
| skrebbel wrote:
| It's also not true
| skrebbel wrote:
| I love that "CrowdStrike" is now a synonym for "global outage".
| Not some cute hihi name like "heartbleed", just the name of the
| company that did the screwup. Seems fair.
| jraph wrote:
| Not sure it's fair, but I am certainly waiting for it to become
| a verb or a noun. crowdstrike. n. 1.
| A set of major disruptions caused by an update that was not
| tested enough, pushed to many devices across the globe.
| 2. The name of such an update. 3. (by extension) a
| joke so bad it causes major disruptions. For
| instance: - Congrats for your crowdstrike! Now my
| weekend is ruined as I'll be the one who'll be asked to fix
| this mess. crowdstrike. v. (simple past
| crowdstruck or crowdstriked1, past participle crowdstricken, or
| crowdstruck, or (obsolete, regionalism) crowdstroke2)
| 1. Action of pushing an update to many devices that causes a
| global outage or major disruptions in various sectors.
| For instance: - We've been crowdstruck. Again.
| crowdstrike. adj. 1. Qualifies an update that, when
| pushed to many devices across the world, causes major
| disruptions across the globe. 2. Qualifies such a (set
| of) event(s). For instance: - We are
| sorry for the crowdstrike event we caused. We gently remind our
| kind customers and their end users that per our ToS, we will
| issue no refund, and that no liability can be held against us.
| Customers who don't try to contact us in the following month
| will get a discount for their next contract renewal. You will
| hear us speak before the Congress, who nicely invited us for
| some comedy in the hope it will appease you all. Make sure you
| like the related videos on the various online platforms. We
| wish you a nice end of the week and nice, relaxing summer
| holidays.
|
| 1 people have differing but strong opinions on which simple
| past form is correct, mainly due to regional differences. Some
| avoid saying crowdstrike and say crowdhit instead.
|
| 2 some people have tried to push crowdstricken, which first
| caught on in some areas or particular contexts. The idea that
| this form likens the qualified subject to the bearer of some
| sickness has eventually seduced a critical mass of people after
| some initial push back. Please also see the usage notes for
| strike for other, rarer, alternative forms [*].
|
| [*] https://en.wiktionary.org/wiki/strike#Usage%20notes
|
| (Thanks to the contributors in this thread)
| skrebbel wrote:
| "The intern crowdstruck half the customers"
| jraph wrote:
| Exactly, by the way I added the irregular inflections and
| fixed the example for the verb. Thanks for your
| contribution.
| LeifCarrotson wrote:
| I disagree, I think that the simple past should be
| "crowdstruck" but the participle should be
| "crowdstricken", as might apply to someone afflicted by
| an illness:
|
| "The update wasn't tested, so the servers are all
| crowdstricken."
| jraph wrote:
| Thanks, I added the documentation for this form, and
| added a second usage note. I initially wanted to tease
| you by documenting that people with bad taste tried to
| push for this form, but I really like this illness idea.
| arrakeen wrote:
| since nothing will happen to them except a slap on the wrist,
| and all our employers will continue to force this crapware on
| our machines, i think we should make a point to start using
| their name as a pejorative (similar to the 'santorum'
| neologism). any when they inevitably try to rebrand, use that
| term too
| stana wrote:
| Rebranding project coming up at CrowdStrike?
| jraph wrote:
| That would be a shame, the name is so fitting, more than
| ever!
|
| They struck a very big crowd real bad.
| knappe wrote:
| For everyone flabbergasted by Southwest running ~Windows 3.1~ old
| software, I have bad news about the telecom industry. I worked at
| Ericsson at an R&D branch and one of the projects in the works
| was to move one of the main pieces of routing equipment that
| handled millions of telephony operatorations a day away from an
| ancient version of Windows.
|
| A lot of code lives on much longer than you think. The general
| attitude we took was that most of the code we were writing would
| be running for at least 30 years. And that was the attitude at an
| R&D branch, arguably a side of that industry where we were
| working on the new tech.
|
| Edit: Win 3.1 or something else, the point still stands. There is
| a lot of old software running out there that will continue to run
| our core services. Legacy software doesn't just mean v1 versus
| v2, it can mean v1 versus v41.
| llm_nerd wrote:
| >For everyone flabbergasted by Southwest running Windows 3.1
|
| Southwest isn't running Windows 3.1, though. That's some rather
| lame, but predictable, truth-through-repeated-assertion thing
| on social media.
|
| Not everyone uses CrowdStrike, and in this case SW was the
| lucky one that didn't.
| wrboyce wrote:
| I find this more surprising, even if the Southwest & Win3.1
| claims were true, I would expect most Ericsson systems to be
| Erlang based and thus happily chugging along on a (perhaps
| ancient) Linux box.
| knappe wrote:
| I did too. But I knew of only 1 project that was using
| Erlang. I have always wanted to use it.
|
| Instead I saw a lot of MML and (happily) TCL.
|
| https://en.m.wikipedia.org/wiki/MML_(programming_language)
| ketchupdebugger wrote:
| Good thing a lot of our banking still runs on mainframes, will
| never be taken out by crowdstrike
| fckgw wrote:
| Southwest does not use Windows 3.1. Why does not one read the
| article?
|
| Southwest wasn't affected because they don't use Crowdstrike.
| That's it.
| knappe wrote:
| I did read the article. It links to
| https://www.techradar.com/pro/security/southwest-airlines-
| av... perhaps you might want to read it again?
|
| Notably, crowdstrike won't run on 3.1, and thus you're kinda
| right.
| philipwhiuk wrote:
| Yes and that article is wrong.
| knappe wrote:
| Win 3.1 or not, the point still stands. There is a lot of
| software out there that has been running for a really
| long time and will continue to do so.
|
| Relatedly, it is nice to provide a source for your
| claims. I did see this [0] which would have been an
| appropriate thing to link
|
| [0] https://kotaku.com/southwest-airlines-
| windows-3-1-blue-scree...
| otterley wrote:
| Dude, take the L. But for that false story being recited,
| you wouldn't have made that point in the first place.
|
| > For everyone flabbergasted by Southwest running
| ~Windows 3.1~ old software
|
| You said it; own it. You could have said "For everyone
| who was tricked by the joke that Southwest runs ~Windows
| 3.1~ old software" but didn't.
| knappe wrote:
| That is fair. I wasn't aware of the kerfuffle around
| whether Southwest was using 3.1 or not. I took the source
| and linked source at face value. It would have been nice
| to have had someone do more than "nope" and instead link
| to a reputable source. This is how you counter
| disinformation.
| kayodelycaon wrote:
| No one runs servers with Windows 3.1. They would have used
| Windows NT.
|
| The really damning bit is Windows 3.1 did not have
| preemptive multitasking. It barely had networking. You
| couldn't run a server with it if you wanted to.
| SoftTalker wrote:
| > They would have used Windows NT.
|
| Or, at the time, OS/2
| otterley wrote:
| It blows my mind how many people actually believed the claim --
| clearly in the obvious-joke category -- that SWA is running their
| mission critical flight systems on Windows 3.1. (Yes, Southwest
| runs a lot of old tech in their stack, but that claim is patently
| hyperbolic.)
|
| People need to stop believing everything they read on the
| Internet and have a little bit of skepticism.
| beambot wrote:
| Lawsuits inbound. Delta appears to be gearing up for one already:
|
| https://finance.yahoo.com/news/delta-air-lines-seek-compensa...
| azinman2 wrote:
| What this also tells me is there are a lot of computers connected
| to the internet that probably shouldn't be.
| nostromo wrote:
| It's insane to me that CrowdStrike's stock is still up 66% year-
| over-year.
|
| With all of the angry customers, lots of incoming lawsuits, and
| the fact that their "protection" is provably more costly than no
| protection at all now - I can't imagine why investors aren't
| dumping it like mad.
| parmenidean wrote:
| Good news then! You can short it and make a ton of money if
| you're confident this share price increase is a mistake.
| johndhi wrote:
| My guesses: -no one really cancels their security vendors since
| security budgets don't shrink -they have a big moat so their
| customers won't be able to leave them
| MattGaiser wrote:
| 1. Compliance. No protection at all isn't a contractual option
| in many cases.
|
| 2. Companies react slowly. When has a vendor paid a high price
| for failure? Boeing can kill people and fail time after time
| still sell planes.
|
| Catastrophe always changes less than anticipated.
___________________________________________________________________
(page generated 2024-07-29 23:00 UTC)