[HN Gopher] SFMTA's train system running on floppy disks; city f...
       ___________________________________________________________________
        
       SFMTA's train system running on floppy disks; city fears
       'catastrophic failure'
        
       Author : sidlls
       Score  : 71 points
       Date   : 2024-04-07 03:20 UTC (3 days ago)
        
 (HTM) web link (abc7news.com)
 (TXT) w3m dump (abc7news.com)
        
       | abeppu wrote:
       | Frustratingly, though the article goes into how
       | replacing/upgrading the whole control system will be an big,
       | expensive project, they don't attempt to say why they can't just
       | update the "read from floppy" part of the system to read the same
       | info from a modern component.
        
         | inferiorhuman wrote:
         | Floppies aren't the problem, the main problem is lack of
         | support. The MTA moved off of OS/2 back in 2008. Yeah, halfway
         | into its intended lifespan Alcatel/Thales basically stopped
         | supporting the whole mess. If I had to guess they're using an
         | equally unsupported version of Windows now.
        
       | sevenseventen wrote:
       | Can someone just ask Foone or lcamtuf to help them get it running
       | off an internet-connected toothbrush or something? It seems like
       | there are so many people doing complex reverse-engineering of
       | ancient stuff _just for kicks_ that this floppy-disk issue just
       | shouldn't require a massive project.
       | 
       | yes, I know safety-critical systems are different. I also expect
       | that the floppy-disk issue is just the easiest problem to explain
       | of a long chain of terrible legacy lock-ins. However, if they're
       | literally holding their breath every morning when it's time to
       | IPL the system off a floppy...that part sounds solvable.
        
         | gojomo wrote:
         | The hypercompetence of tech enthusiasts and the mind-numbing
         | idiocy & dishonesty of politial governance don't mix, so cities
         | have to spend tens of millions with parasitic contractors for
         | what a few clever hackers could do as a side-project.
        
       | WaitWaitWha wrote:
       | >Turns out that in 1998, SFMTA had the latest cutting edge
       | technology when they installed their automatic train control
       | system.
       | 
       | > "We were the first agency in the U.S. to adopt this particular
       | technology but it was from an era that computers didn't have a
       | hard drive so you have to load the software from floppy disks on
       | to the computer,"
       | 
       | In 1998, most personal computers already had hard drives [0].
       | From Wikipedia "The IBM PC/XT in 1983 included an internal 10 MB
       | HDD, and soon thereafter, internal HDDs proliferated on personal
       | computers."
       | 
       | The 3.5" floppy is from the mid 80's, again from Wiki [1] "In the
       | early 1980s, many manufacturers introduced smaller floppy drives
       | and media in various formats. A consortium of 21 companies
       | eventually settled on a 31/2-inch design..."
       | 
       | [0] https://en.wikipedia.org/wiki/Hard_disk_drive
       | 
       | [1] https://en.wikipedia.org/wiki/Floppy_disk
       | 
       | Why do I have to do this research instead of the "journalist"?
        
         | abeppu wrote:
         | Given that the agency seems not to be able to cope with change
         | at a reasonable pace, I wonder if 1998 was just when the
         | control system project finished, having been planned and
         | started several years prior.
        
           | HumblyTossed wrote:
           | > I wonder if 1998 was just when the control system project
           | finished, having been planned and started several years
           | prior.
           | 
           | Probably this. Probably they got the plans worked up in 1990,
           | but didn't finish everything until 1998.
        
             | rootusrootus wrote:
             | I'd say they must have started closer to 1980. By the time
             | we had the 286, most PCs came with a hard drive, and 1990
             | was the era of the 486.
        
         | compuguy wrote:
         | I have this feeling that they meant _1988_ instead of
         | _1998_....but I could be wrong?
         | 
         | Edit: Even the news article hints at 5 inch floppy
         | disks...which makes 1998 make absolutely no sense to me at
         | least.
         | 
         | https://abc7news.com/san-francisco-train-system-has-been-
         | run....
        
           | deathanatos wrote:
           | I'm not sure moving it back a whole decade saves them?
           | 
           | My family's first PC in 1987 had a hard disk. The Wikipedia
           | quote they provide lines up with that, and provides a more
           | authoritative point of when it was introduced.
           | 
           | And yeah, the "5 inch floppy" quote paired with a photo of a
           | diskette. God only knows what actual hardware the system
           | uses. But point being ... the journalist doesn't seem to have
           | found out.
        
             | gsk22 wrote:
             | There's a big difference when choosing technologies for a
             | home computer versus a safety-critical system that will run
             | for decades. Seems reasonable they would've chosen a
             | proven, if slightly outdated, tech over a newer one that
             | had less data to back up its long-term reliability.
             | 
             | And of course we can get into the discussion of cost:
             | floppies are way cheaper than hard drives, especially for
             | the presumably-small amounts of data that are needed for
             | such a control system. A 500MB hard drive was probably
             | overkill.
             | 
             | I am of course speculating, but I don't know why we should
             | assume that the engineers working on the project originally
             | made a silly decision. They almost certainly weighed the
             | available options to deliver the project within the defined
             | constraints of the system.
        
               | JoyousAbandon wrote:
               | Hard drives weren't new in 1998. I don't even know where
               | you'd have gotten a computer that lacked one by then; and
               | most would also have had a CD-ROM drive.
        
               | prewett wrote:
               | The Mac SE had a 20 MB hard disk in 1987! And there was
               | no way Windows 95--which everyone had by 1998--was
               | fitting on a floppy of any size.
        
         | inferiorhuman wrote:
         | Why do I have to do this research instead of the "journalist"?
         | 
         | Ratchet the snark back. The journalist was referring to the
         | train control system not home computers in someone's basement.
         | And, yes, twenty five years ago SelTrac was cutting edge.
         | Moving block systems were basically unheard of back then.
        
           | JoyousAbandon wrote:
           | The only snark here is yours. What are you talking about with
           | "computers in someone's basement?"
           | 
           | He pointed out glaring factual errors in the story, which
           | should not have made it through any kind of editorial review.
           | For example: 25 years ago pretty much every computer had a
           | hard drive. And the disk depicted in the article is obviously
           | a 3.5", not a "five-inch floppy."
        
             | inferiorhuman wrote:
             | Whether the computer in your basement had a hard drive or
             | not has no bearing on whether or not SelTrac was cutting
             | edge at the time (it was). That there were even computers
             | with microprocessors put SelTrac decades ahead of what was
             | in use elsewhere at the time.
        
         | jcgrillo wrote:
         | Why would a hard drive be better? If the floppy fails just grab
         | another off the shelf and try it (surely they have more than
         | one copy). Downtime measured in seconds.
         | 
         | The other good thing about the floppy is it can't hold very
         | much code. So the system has a tight upper bound on how bloated
         | and complex it can get. Simpler systems are more maintainable.
         | 
         | These things seem like great assets for maintaining critical
         | infrastructure.
         | 
         | EDIT: Another great thing is such a system will be stateless.
         | No disks, no filesystems, no databases. Sign me up.
        
           | deathanatos wrote:
           | While nothing you've said is wrong, that's not their point.
           | 
           | (These days, you could also replace the floppy with a USB
           | drive: they make adapters/emulators.)
        
           | JoyousAbandon wrote:
           | He never said it was better. He's pointing out glaring
           | factual errors in the story.
        
             | jcgrillo wrote:
             | Where is the factual error? This passage is the one we're
             | taking about right?
             | 
             | > Turns out that in 1998, SFMTA had the latest cutting edge
             | technology when they installed their automatic train
             | control system.
             | 
             | > "We were the first agency in the U.S. to adopt this
             | particular technology but it was from an era that computers
             | didn't have a hard drive so you have to load the software
             | from floppy disks on to the computer,"
             | 
             | The fact of when hard drives became commonplace in personal
             | computers (EDIT: or when 3.5" floppies were introduced) has
             | no bearing whatsoever on whether this transit control
             | system was cutting edge in 1998. This statement is not, at
             | least not _obviously_ , factually incorrect. So if you're
             | going to claim that, show your work.
        
       | t1c wrote:
       | "If it ain't broke, don't fix it" definitely has its practical
       | limits
        
       | nvahalik wrote:
       | I don't understand.
       | 
       | The retro community has proven reliably that a simple Raspberry
       | PI can easily bit-bang floppy controllers. We have myriad floppy-
       | to-SD card adapters.
       | 
       | Surely a plug-and-play solution that removes the area of most
       | concern (reliance on the media itself) should be easily
       | achievable in a few months?
        
         | jwells89 wrote:
         | Speaking naively (I have no insight into the situation), I
         | would guess that this is probably the result of the officials
         | in charge not having technical understanding and contracting it
         | all out. Contractors aren't going to be choosing the quick or
         | cost-effective option, because that'd undercut their profits.
        
           | deweller wrote:
           | Also, it is difficult to sell a solution of "well - we aren't
           | really modernizing the system, we are just putting a fix in
           | place that pretends like it is a floppy drive."
           | 
           | I would guess higher ups don't want to spend money on a band-
           | aid without really fixing the problem.
        
             | krallja wrote:
             | > higher ups don't want to spend money on a band-aid
             | without really fixing the problem
             | 
             | I have literally never had this problem in my career.
             | Higher-ups LOVE to push the problem off to after the next
             | budget cycle.
        
         | jdblair wrote:
         | You are correct that this would be much simpler if the only
         | goal was to replace the floppy disk part of the system.
         | 
         | The floppy disk angle is there to make a good headline. The
         | article makes it clear this is a much bigger project than just
         | replacing the floppy disks.                 "The detail[ed]
         | project schedule will be finalized once we have a contractor
         | onboard. This is effectively a multi-phase decade long project
         | that starts with pieces of market street subway and pieces in
         | the surface. Ultimately our goal is to have a single train
         | control system for the entire rail system," said Tumlin.
        
           | renewiltord wrote:
           | SF has to make everything a big bang project because every
           | project has equal large fixed cost. So whether you do a small
           | thing or a big thing you do a big fixed cost thing first.
           | 
           | So SF always does the big thing because otherwise the
           | overhead dominates.
        
         | luxuryballs wrote:
         | I think the government is involved which means the task must
         | primarily be an effort to funnel large amounts of money around
         | to just the right people while facilitating all the best
         | changes to whatever they can find within N degrees of
         | separation from the project itself, (where N approaches
         | infinity.)
        
         | lepus wrote:
         | There are other concerns that need to be addressed where a hack
         | wouldn't help:
         | 
         | >The transportation body says the train control system was
         | built to last for just 20 to 25 years, meaning it surpassed its
         | expected lifetime in 2023. In 2020, the Muni Reliability
         | Working Group, said to be composed of local and national
         | transit experts, recommended replacing the transit control
         | system within five to seven years. [...] "We have to maintain
         | programmers who are experts in the programming languages of the
         | '90s in order to keep running our current system, so we have a
         | technical debt that stretches back many decades," Tumlin told
         | San Francisco's KQED in February 2023.
        
         | ammo1662 wrote:
         | It is not a technical problem. Who will be responsible for this
         | migration?
         | 
         | Clearly no one wants to do that.
        
         | pessimizer wrote:
         | > However, budget challenges put the project's timeline into
         | question. The SFMTA's train upgrade project isn't just a
         | migration off of floppy disks but also a "complete overhaul of
         | the current train control system and all its components,
         | including the onboard computers, central and local servers, and
         | communications infrastructure," Roccaforte said.
         | 
         | > Much more critical than the dated use of floppy disks is the
         | system's loop cable, which transmits data between the central
         | servers and the trains and, according to Roccaforte, "has less
         | bandwidth than an old AOL dial-up modem."
         | 
         | Also, they're already late as hell, what's 5 more years? 5 1/4
         | isn't going to become more obsolete than it is at this point.
        
           | inferiorhuman wrote:
           | You don't need more "bandwidth than an old AOL dial-up modem"
           | for train control. What Roccaforte left out is that the MTA
           | spent over a decade trying to keep the trains from slicing up
           | the inductive loop. No idea how the new trains are working
           | out, but I'm pretty sure they never found a permanent fix for
           | the old ones.
        
         | snakeyjake wrote:
         | Playing video games on an old Apple II via a floppy emulator
         | has a different set of requirements than a safety-critical
         | application.
         | 
         | Lawyers will probably spend a year wrangling liability
         | concerns.
         | 
         | I could probably replace a floppy drive on any system within a
         | couple of weeks. I would not accept legal or financial
         | liability for any such solution without an extremely thorough
         | and slow review of all aspects of the project. edit: and an
         | astronomically high paycheck.
        
         | cortesoft wrote:
         | I don't think that is actually a concern. The article is
         | written in a way to imply that it is, but I doubt the people
         | who are running the system actually think that is the risk.
         | 
         | I am sure they have copies of the floppy disk, and likely have
         | images on other machines that can be used to create a new
         | floppy even if all the copies are lost.
         | 
         | I feel like the SFMTA is mainly using the existence of the
         | floppy disks as a marketing point for their desire to get
         | funding to update the system. It is something that the average
         | person will be able to look at and know it is out of date in a
         | visceral way.
         | 
         | The reporter seems to read this as if we are one bad floppy
         | away from failure, but that is not the actual case.
        
       | sourcecodeplz wrote:
       | Didn't it used to be that they broke often when you would keep
       | them in the sun, even for a little?
        
       | nerdjon wrote:
       | Well... I feel slightly better about Boston constantly pushing
       | back being able to use our phones as tickets after seeing the
       | timeline for the transition and how just that they are still
       | doing this.
       | 
       | What is it about public transit in the US that it is so... bad?
       | Inadequate funding seems to be the easy one, but the MBTA
       | (Boston) doesn't even handle the funds it has well. Yeah it needs
       | more funding but there is also just a core issue to how it's run.
       | 
       | It is sad to see the state of public transit in this country,
       | particularly in dense urban areas where we should be discouraging
       | Car use as much as possible.
       | 
       | I am very curious what other countries are doing that we are not.
        
         | asdff wrote:
         | I think about this all the time every time I see an lcd screen
         | in my local train stations. If people who designed them
         | actually used transit, maybe they'd have the signs you can only
         | see as you walk out of the station say something about the bus
         | lines you'd transfer to, instead of when the train behind you
         | that just departed from is set to arrive. At some stations the
         | screens they installed don't display anything useful at all,
         | not when the next train will come, just the date and time as if
         | everyone doesn't have that in their pocket, and this needs to
         | be displayed every 25 feet on the platform.
         | 
         | A lot of transit could be fixed by just taking a regular
         | routine user, empowering them to become a dictator for a week
         | and point out all the friction points they hit actually using
         | the system. But then that would make the entire bureaucratic
         | system that is the transit agency look like idiots who don't
         | understand their own jobs, so it will unfortunately never be
         | done.
        
           | xcrunner529 wrote:
           | Well and lots that don't even take the transit they manage.
           | It's just so infuriating.
        
           | nerdjon wrote:
           | I feel like almost every screen I just ignore, either it was
           | just put there for ads or it is wrong/broken.
           | 
           | It is now a fairly regular occurrence to have a train show up
           | that the screen had no reference too existing a minute or two
           | before.
           | 
           | But yeah it feels like they were not designed or setup by
           | people that actually use the trains and were setup by a
           | comity thinking they know best.
           | 
           | Maybe that is the big difference, in other countries the
           | people using it are also the ones managing it?
        
             | rtkwe wrote:
             | The big issue is putting those screens in isn't just a
             | simple task, you need the entire support and switching
             | network to be able to track individual trains which is a
             | relatively new thing in terms of the age of many of the
             | subway swtiching and train tacking (where those even exist)
             | systems in the US.
        
           | wpietri wrote:
           | For sure. I think a lot of this is just about how heavy the
           | planning cycles are. A lot of stuff like that is developed
           | through big, waterfall planning efforts and large outsourcing
           | contracts. Most of the choices were made before they had
           | their first user, and once it's all installed any revision
           | would be a) expensive, and b) a black eye for the people in
           | charge.
           | 
           | And in some ways I don't blame people for doing this, because
           | you need really supportive stakeholders to work in an
           | iterative fashion. Otherwise you get a ton of nitpicking that
           | amounts to a lot of "why didn't you guess 100 perfectly
           | everything up front" and "how can we start if you can't give
           | me a full plan and a firm price?" In a blame-hungry
           | environment, waterfall is the safest choice for the people
           | doing the project.
        
         | roody15 wrote:
         | I am an elected official (12 years and elected six times in
         | Illinois) unfortunately seeing this process work over 12 years
         | is discouraging. For example President Obama allocated 8
         | billion dollars for high sped passenger rail service
         | improvements. By 2017 almost 2 billion of this money had been
         | spent on railroad improvements / railroad crossings and line
         | inspections. Fast forward to 2024 no high speed rails have
         | actually been built. For example Chicago Illinois was to get a
         | line from Iowa City to Quad Cities to Chicago (straight line).
         | This project was fully funded at one point but Iowa and
         | Illinois had (still have) a disagreement on bridge repairs
         | connecting the states.
         | 
         | Now Governor Pritzer has authorized a new high speed rail
         | commission to try and get this project going again.
         | 
         | In a nutshell .. Rail project between Iowa City and Chicago was
         | fully funded as of 2016... all that actually happened is some
         | rail road crossing were improved.. lines were inspected and no
         | actual new lines constructed. Instead 1 out of the 2 billion
         | was spend on "engineering" costs and compliance paperwork ...
         | which now has expired and need to be redone if the project is
         | to be completed.
         | 
         | The amount of money spent on compliance paperwork and
         | "engineering" is staggering. Many six figure salaries depend on
         | slightly altering existing engineered projects to meet
         | compliance requirements for projects that are never built. From
         | waste treatment upgrades, water treatment, road improvement,
         | traffic studies, on and on. The amount of money spent on
         | services that are not finished or lead to an actual project is
         | absolutely staggering ... entire industries depend on this
         | inefficiency and lobby effectively to keep things "obtuse".
        
           | baggy_trough wrote:
           | Wasting money on favored constituents is the point of these
           | projects.
        
             | harpiaharpyja wrote:
             | Wasting money on special interests would be more accurate.
        
           | nerdjon wrote:
           | I feel like Boston has had similar, we have contracts with
           | companies to complete a thing by a specific time, they don't
           | but we are stuck paying them even more to complete a project
           | so it ends up being over budget, late, and possibly not done
           | well as is evident by our green line extension that opened
           | last year that was shut down to redo it because the tracks
           | are the wrong width.
           | 
           | It just feels like the money is going to the wrong places
           | like you said.
           | 
           | I would probably argue that we still need more funding even
           | if we fixed how we used the money, but we need to fix how we
           | use it first.
        
             | nomel wrote:
             | This is what happens when the repercussions for poor
             | performance/failure is more money. Privatization, with some
             | regulations, and reduction of monopolies, seems like a good
             | idea. But, the system will never choose to harm itself.
        
               | simfree wrote:
               | The current engineering ecosystem is largely private, and
               | it's poorly functioning, costing us way more money than
               | just hiring engineers as public employees rather than
               | paying a contracting firm to do initial and final design
               | on each rail system segment.
               | 
               | Just go look at the RFPs and responses each municipality
               | has four expansion of their current system. It is all
               | outsource to private industry, at great cost compared to
               | doing it inside government.
        
               | nomel wrote:
               | I would claim that the problem is that they're doing it
               | for government, which is what makes the chain of
               | accountability lead to nowhere.
        
           | TheJoeMan wrote:
           | From someone who's seen the other side... every time the
           | engineer has to quabble over email with the permitting
           | departments over matters such as "you sent it to the wrong
           | bureaucrat so I'm ignoring instead of forwarding it" or "the
           | permit admin wrote the wrong 1 of 50 Comcast subsidiaries on
           | the paperwork so now I have to get a skip-manager's signoff"
           | it's billed at $400/hr.
        
         | electric_mayhem wrote:
         | > What is it about public transit in the US that it is so...
         | bad?
         | 
         | Massive money and entrenched behind preserving a car-based
         | life.
         | 
         | Car companies, auto workers unions, railroads can siphon off
         | funds from promised improvements to cover deferred maintenance
         | and give themselves bonuses for being so clever... Hell, AAA
         | which one might expect to spend its income on serving its
         | members instead diverts money to proactively lobbying against
         | improvements in public transit in an ongoing display of cynical
         | self-preservation
         | 
         | It's basically hopeless.
        
           | KerrAvon wrote:
           | Nonsense. There is working transit in the US and a huge
           | appetite for more. But it takes decades to do this stuff.
           | "China did massive infrastructure project in two years" --
           | China is also a communist dictatorship (for some value of
           | those words), and we can't do that here. We could streamline
           | things a bit more, of course, but it's never going to be
           | cheap and easy in the US.
           | 
           | If you want to help, vote. Or even run for something on a
           | pro-transit, YIMBY platform.
        
         | rangestransform wrote:
         | other countries don't treat public transit as a jobs program
         | 
         | other cities (in the east, with the best public transit) don't
         | let unions grab the entire city by the balls
         | 
         | government employees in other cities aren't as hellbent on
         | extracting their pound of flesh from the taxpayer
        
           | xxpor wrote:
           | NYC is grabbed by the balls plenty. They still don't have
           | OPTO for example.
        
         | wpietri wrote:
         | I think there are a bunch of things causing this problem. Off
         | the top of my head, some factors:
         | 
         | - chronic underfunding - we all know how problems build up when
         | maintenance is deferred
         | 
         | - waterfall planning - Mary Poppendieck has a nice talk on how
         | much trouble this causes:
         | https://www.infoq.com/presentations/tyranny-of-plan/
         | 
         | - political point-scoring - blame-oriented cultures discourage
         | experimentation and incremental improvement
         | 
         | - political polarization - one-party areas can more easily
         | slide into cronyism, and fighting between parties makes it hard
         | to compromise even on things like fixing infrastructure
         | 
         | - classism - in a lot of places, transit is for the poors
         | 
         | - racism - many don't want transit bringing Those People around
         | 
         | - manager culture, not engineer culture - as we see with
         | Boeing, standard MBA thinking doesn't work well for long-term
         | safety and reliability; the focus on short term metrics, mostly
         | financial ones, leads to underinvestment and decay of
         | infrastructure
        
           | J_Shelby_J wrote:
           | I'm a liberal democrat, but it's clear that one party
           | politics is not tenable. There has to be an option to the
           | status quo that you can vote for to incentivize competency in
           | government. But with the state of national politics it would
           | be almost impossible for a republican to win in most of urban
           | America. So that leaves just primary challengers from within
           | the Democratic Party which is very difficult. So urban
           | political machines have become absolutely rotten with
           | complacency and now the richest zip codes on the planet have
           | trains running on floppy disks.
           | 
           | One option is ranked choice voting.
        
             | wpietri wrote:
             | I totally agree we need more actual competition, and also
             | agree that national polarization plus a first-past-the-post
             | system makes that unlikely in the near term.
             | 
             | I would also love to see widespread use of RCV, but as a
             | San Franciscan I don't think it has done tons to fix the
             | problem of complacent political machines here, so we're
             | going to have to do more.
        
       | therealmarv wrote:
       | Upgrade to 3.5 inch ones!
        
         | yonran wrote:
         | They do use 3.5 inch disks; this is a misstatement: "SFMTA's
         | train control system relies every morning on 5 inch floppy
         | disks." The SFMTA Board presentation justifying the >$600M
         | upgrade shows a photo of the real 3.5 inch floppy disks in use
         | that were written in 2021.
         | https://www.sfmta.com/sites/default/files/reports-and-docume...
        
       | anigbrowl wrote:
       | They haven't even settled on a contractor yet? Maybe the problem
       | here is that they're trying to write one check to fix all their
       | problems at once instead of taking an incremental approach.
        
         | wongarsu wrote:
         | That seems to be common in US infrastructure projects. If it
         | isn't a megaproject, is it even worth attempting? /s
        
           | rtkwe wrote:
           | I get it though, it's a nightmare getting a project through
           | approvals so by the time you have an issue big enough to push
           | an overhaul through it's a major project and you want to pile
           | as many things onto it as possible because it might be a
           | decade before you get another decent funding injection to do
           | what should be regular maintenance.
        
         | maxvt wrote:
         | This takes deep expertise and a system oriented, long term view
         | with a very precise eye towards the details of what's
         | technically possible at every individual step.
         | 
         | This kind of knowledge and experience doesn't come cheap, and
         | we all know how much US city governments pay. I was at one
         | point, very briefly, motivated to apply to a city job before I
         | learned the pay is approximately 1/3 to 1/4 of what the private
         | sector pays. The USDS routinely posts on "who wants to get
         | hired" and the comments on that and other thread also mention a
         | 66% to 75% salary reduction from baseline.
         | 
         | This is even before we get to legal liability and political
         | risk shifting that can happen when there is a fully responsible
         | contracted company involved.
        
       | _trampeltier wrote:
       | Floppy disks seems common in the us. Just since 2019, they don't
       | use 8" floppys for the nuclear rockets.
       | 
       | https://www.theverge.com/2019/10/25/20931800/usa-nuclear-8-i...
        
         | rootusrootus wrote:
         | Hardly specific to the US. Lots of stodgy old institutions
         | across the world still run on elderly hardware. If it works, it
         | works.
        
       | m3kw9 wrote:
       | If it works don't fix it
        
       | FpUser wrote:
       | >""The system is currently working just fine, but we know that
       | with each increasing year, risk of data degradation on the floppy
       | disks increases and that at some point there will be a
       | catastrophic failure," Tumlin told ABC7."
       | 
       | Do they know that floppy can be backed up?
        
       | zitterbewegung wrote:
       | Japan is doing away with 3 1/2 floppies .
       | https://arstechnica.com/gadgets/2024/01/floppy-disk-requirem...
        
         | wongarsu wrote:
         | German high speed trains used 3.5" floppies to update seat
         | reservations until ~2016.
        
         | cortesoft wrote:
         | The article picture is of a 3.5" floppy, but the article says
         | they are using 5.25"
        
       | HumblyTossed wrote:
       | > This system was designed to last 20 to 25 years. SFMTA's
       | director Jeffrey Tumlin said upgrading the system will take
       | another decade and cost hundreds of millions of dollars.
       | 
       | This is the problem, not that they're using a floppy. This isn't
       | web dev where you get to rewrite everything every 6 mos. Systems
       | have to have decades long life cycles BUT THEY EVENTUALLY NEED TO
       | BE REPLACED and that's not happening quickly enough here.
       | 
       | Edit: It was last updated in 1998, so it's due now not a decade
       | from now.
        
         | markus_zhang wrote:
         | I'd like to see a breakdown of that "hundreds of millions of
         | dollars" and which companies provided the quotes, plus who owns
         | the companies and who are their
         | husbands/wives/uncles/nephews/nieces/etc.
        
           | kion wrote:
           | I looked into this last year, a significant part of the cost
           | is in new hardware, both on the vehicles, but also sensors
           | spread through the system. They all need commercial grade RF
           | systems that allow for 2 way signaling communication and
           | integration into the vehicles control systems.
        
             | bobthepanda wrote:
             | also they need to be rated for the operating environment.
             | you can't stick any random part from newegg in there.
             | 
             | Tesla used non-automotive grade touchscreens in some cars
             | and so they died early:
             | https://www.thedrive.com/tech/27989/teslas-screen-saga-
             | shows...
             | 
             | Automotive grade has very high requirements:
             | 
             | > Grade 0: High Temperature Operating Life (HTOL): +175C
             | for 408 hours. High Temperature Storage Life (HTSL): +175C
             | for 1,000 hours. Temperature Cycling: -65C tp +175C for 500
             | cycles
             | 
             | And generally speaking transit agencies aspire to buy
             | higher-reliability, longer-lasting equipment than your
             | average consumer car.
        
               | krallja wrote:
               | I've noticed that my car's touchscreen operates at a very
               | high temperature at all times - almost hot enough to burn
               | skin, which seems like a weird choice for something
               | designed to be touched by skin.
        
           | jl6 wrote:
           | It's probably not overt corruption. It's probably because in
           | this kind of large system, there is a strong demand for
           | validation and assurance at all levels, which is usually
           | achieved by hiring experts* to write reviews,
           | recommendations, and reports that hold up in court.
           | 
           | * They must be experts, otherwise how could they get away
           | with charging so much, right?
        
           | strangattractor wrote:
           | The cost is distribute across all the PowerPoint, Word, and
           | Excel spreadsheets created to give the illusion that people
           | are doing actual work. Go read a couple of the SFMTA PDF's
           | linked to by a commenter. They are totally enlightening
           | justifications like - reduced congestion blah - better
           | monitoring blah. No meat whatsoever. No analysis backing
           | anything up - just pretty pictures.
           | 
           | I rode Muni for 10 years almost every working day. Do you
           | want to know what reduced congestion the most. Buying new
           | muni cars because the old muni cars door mechanism had been
           | repaired so much they did not open or close %50 of the time
           | and people had to use other doors to get on and off a train
           | on a daily basis.
        
         | wongarsu wrote:
         | Still, when you have a 20-25 year replacement cycle being 10
         | years behind is pretty significant. And that's assuming their
         | estimate of needing another decade is correct. Which seems
         | doubtful considering they don't have a contractor, they don't
         | have the money, and the previous system probably took a decade
         | to develop considering their choice of 5.25" floppies (3.5"
         | floppies overtook 5.25" floppies in sales in 1988).
        
           | aidenn0 wrote:
           | Right, if they think it will take 10 years, 15-20 is probably
           | more realistic, which means you need to start the upgrade
           | process 5-10 years after buying your system: seems very
           | expensive.
        
         | bdamm wrote:
         | Nah.
         | 
         | What's really happening is going to be: Emerging new use-cases
         | and rising costs from specialized vendors for replacement parts
         | that match the existing system are driving a desire for the
         | agency to replace the control system. The floppy system bit can
         | 100% be replaced with a solution that isn't a floppy, but that
         | wouldn't help them with the rest of what they want.
         | 
         | "because life cycles" is a lazy description.
        
         | dblohm7 wrote:
         | I think it's worth pointing out that 5.25" floppies were
         | already well out their way out in 1998, already.
        
       | Aloha wrote:
       | I dont really see the issue?
       | 
       | The technology works, there is a replacement outlined, there is
       | no shortage of floppy disks - even 5 1/4 ones.
        
         | perihelions wrote:
         | Yeah, the article reads as if the real problem is a lack of a
         | tested system of backups. Or I don't know how else to parse
         | this line:
         | 
         | - _" It's a question of risk. The system is currently working
         | just fine but we know that with each increasing year risk of
         | data degradation on the floppy disks increases and that at some
         | point there will be a catastrophic failure."_
         | 
         | "Data degradation on the floppies" should not, by itself, cause
         | any sort of "catastrophic failure" in a sound system. If one
         | software disk fails, you should have five identical copies in a
         | drawer, five more in a different room, ten more off-site, plus
         | a disk image on cloud storage that you can write fresh floppy
         | disks from.
         | 
         | I mean, it's probably a good idea to change the storage medium
         | too--but that's not the root problem.
        
           | jcgrillo wrote:
           | checksums are a whole thing
        
       | shrubble wrote:
       | Does no one at the organization know what a GoTek FlashFloppy is?
       | 
       | Sounds like they are using the floppy as an excuse to push for an
       | upgrade that has nothing to do with the floppy drives.
        
         | rtkwe wrote:
         | The entire system has reached the end of it's design life and
         | the floppy is just the shocking bite to get people to click
         | through. Really this is the regular replacement of an old
         | system with a newer control system that will make the trains
         | run better and more reliably.
        
           | nelsondev wrote:
           | Why will a new system make the trains run better? It could
           | also make it run worse, if they don't cover all the current
           | cases.
           | 
           | I've ridden Muni in SF for years, drivers control most of the
           | decision making.
           | 
           | There's a handful of single track tunnels/sensors/necessary
           | software-based coordination, but as another commenter pointed
           | out, the doors cause more issues than signal problems.
        
             | inferiorhuman wrote:
             | I've ridden Muni in SF for years, drivers control most of
             | the decision making.
             | 
             | All of the tunnels are under automatic operation, and the
             | majority of the switches (street or tunnel) are computer
             | controlled.                 the doors cause more issues
             | than signal problems.
             | 
             | While doors are a real common failure/abuse point, I'd want
             | to see some actual numbers on that. Train control failure
             | may not always be self-evident to a rider, especially if
             | the remedy is to run things manually (such that things
             | aren't ever stopped completely).
             | 
             | Historically the cab signaling system would glitch in wet
             | or damp weather. Transponders for the switches had a near
             | 100% failure rate. Trains chewed up the inductive loops,
             | forcing emergency braking. That was common enough that they
             | had to dial back the emergency deceleration. Twenty years
             | on they've mostly worked out the kinks but it hasn't always
             | been that way.
        
       | VyseofArcadia wrote:
       | > "It's like if you lose your memory overnight, and every
       | morning, somebody has to tell you hey 'this is who you are and
       | what your purpose is what you have to do today,'" said Maguire.
       | 
       | Yeah, that's called a cold boot. Moving to not-floppies doesn't
       | mean you can avoid this. Clearly it's off of floppies instead of
       | ROM so you can more easily update the software, but I am
       | wondering how often that ended up happening. Maybe EEPROMs would
       | have been better.
       | 
       | > Luz Pena: "How dire is it to change the system to upgrade it
       | from a floppy disk to a wireless system?"
       | 
       | I agree that floppies aren't the peak of reliability, but "a
       | wireless system" also sounds like a disaster. I don't want
       | critical urban infrastructure running on extremely hackable OTA
       | updates. For the love of god, SF, you can avoid pretty much all
       | potential cybersecurity problems by just _not putting your trains
       | online_.
       | 
       | I feel like neither the interviewer nor the interviewee really
       | had the technical expertise to speak to this. This entire piece
       | is just, "oooooo, floppies are old. Old bad! Why not new yet? New
       | good."
        
         | JTbane wrote:
         | Couldn't the city set up a private WAN for the trains, avoiding
         | the security issues of being connected to the internet?
        
           | VyseofArcadia wrote:
           | That doesn't avoid security issues inherent in wireless, but
           | at least you have to be somewhat near the train instead of
           | the other side of the planet.
        
       | ChrisArchitect wrote:
       | The reporter waving around a 3.5" floppy because most viewers
       | wouldn't even know what the 5.25" was
        
         | rainbowzootsuit wrote:
         | The 8" are reserved for the ICBM targeting systems and
         | financial institutions.
        
       | sitkack wrote:
       | So, replace the floppies with a high reliability flash device
       | that emulates a floppy drive, should have happened years ago.
        
       | jcgrillo wrote:
       | > "Wow. I mean I thought we were moving on to AI. So why are we
       | doing floppy disk," said Katie Guillen, SFMTA passenger.
       | 
       | This naivety is _not_ Katie 's fault. We who work in tech are to
       | blame for constantly pushing our half-baked experimental garbage
       | as if it was "engineering" on par with civil or aeronautical
       | systems. We can't blame people for occasionally believing the
       | lies.
       | 
       | The tech-washed version of this quote might go something like
       | "wow I thought everything was moving to the multi-cloud
       | serverless kooberneetus now, why is it still running on a
       | computer?"
       | 
       | > It is easy to run a secure computer system. You merely have to
       | disconnect all dial-up connections and permit only direct-wired
       | terminals, put the machine and its terminals in a shielded room,
       | and post a guard at the door[1].
       | 
       | This is the kind of thing I want running the trains. Give it ECC
       | RAM too, please.
       | 
       | [1]
       | https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer...
        
       | iancmceachern wrote:
       | I emailed them. I live I'm SF. This is the sake, solved, problem
       | we have in the machine tool world (milling machines, lathes,
       | etc). Similar to old keyboards, you can just buy and use a floppy
       | emulator.
       | 
       | They emailed me back, they said that the floppy thing makes a
       | good headline but is really just the tip of the iceberg. It's
       | really the whole system that's like this at every layer, it needs
       | replacing they say.
        
         | boringg wrote:
         | Protected from hackers though!
        
           | jmb99 wrote:
           | Unless said hacker has a big magnet.
        
         | Hrnrurj wrote:
         | Here is 2017 article from NY to get an idea.
         | 
         | https://www.businessinsider.com/nyc-mta-subway-delay-2017-6
        
         | akira2501 wrote:
         | > it needs replacing they say.
         | 
         | It sounds like they let this problem fester until it has
         | reached this existential end. They clamor for new technology,
         | which they get, then they use the most risk adverse management
         | strategy and never upgrade or change it, until it reaches this
         | problem state.
         | 
         | They either need significant third party help deploying and
         | managing this system, or they should go back a few generations
         | of technology and use the simplest possible system that meets
         | their needs. Pen and paper should be considered if it can be
         | made more efficient.
         | 
         | I guess the people who work on public transit aren't interested
         | in having the best public transit system available. They're
         | only interested in keeping it running for as long as possible
         | with zero changes or responsibility.
         | 
         | This is why I'm strongly doubtful on public transportation in
         | the US. Our bureaucracy can't handle it.
        
           | arrosenberg wrote:
           | > I guess the people who work on public transit aren't
           | interested in having the best public transit system
           | available. They're only interested in keeping it running for
           | as long as possible with zero changes or responsibility.
           | 
           | Let me guess, you also think public servants are lazy,
           | overpaid and unskilled? Comments like this belie how many
           | people form opinions of public agencies without having any
           | real experience with government.
           | 
           | The problem is the public and the politicians - transit
           | employees largely do the best they can with the resources and
           | constraints they are given. In order to succeed transit needs
           | municipalities and states to: pay more taxes; accept that
           | transit won't generate a profit, but does generate non-
           | monetary public value; and gather their resolve to deal with
           | NIMBYs.
        
             | akira2501 wrote:
             | > Let me guess, you also think public servants are lazy,
             | overpaid and unskilled?
             | 
             | Do you not see this article as evidence of this fact? If
             | you walked into any other business reliant on "floppy
             | disks" just to take your money, would you praise their
             | practices and efficiency?
             | 
             | > Comments like this belie how many people form opinions of
             | public agencies without having any real experience with
             | government.
             | 
             | I form opinions because I try to use their services and am
             | disappointed and often disgusted. Do you really think
             | "experience with public transport" is particularly hard to
             | come by? That this is all too esoteric for the "common man"
             | to opine on? Please...
             | 
             | > transit employees largely do the best they can with the
             | resources and constraints they are given
             | 
             | Perhaps they should earn those resources competitively
             | instead of being given them? Otherwise, this seems like a
             | perennial chicken and the egg problem that swallows tax
             | dollars with no worthwhile result.
        
               | arrosenberg wrote:
               | I don't think the article is a reflection on efficacy of
               | the public agency. If their budget for new software is
               | $0, how exactly are they supposed to upgrade it? It's
               | certainly possible that the agency has been utter shit
               | for 25 years, but this article doesn't give the
               | information to conclude that.
               | 
               | > Perhaps they should earn those resources competitively
               | instead of being given them?
               | 
               | How? They're a public agency. There is no profit motive -
               | if the public wants nicer stuff, they need to pay for it.
               | That includes both the software costs and the higher
               | salary costs to hire people capable of those types of
               | projects.
               | 
               | Again, you come off as someone running their mouth with
               | no actual experience interacting with public agencies.
               | It's very frustrating to know how to fix a problem
               | without being given the resources or permission to do so,
               | which is often the case in transit. Add to that most
               | transit agencies have a tiny budget for IT, and I'm not
               | sure how you expect this to get solved without a big
               | public push.
        
             | madaxe_again wrote:
             | Transit employees are not the same people as, nor have the
             | same motivations as the politicians who manage transport
             | agencies.
             | 
             | Of course the employees work hard. It's the vocation
             | they've chosen. I'm always amazed by the diligence of TFL
             | and rail employees in the U.K., for instance.
             | 
             | No, the guiding principle for the folks at the top is "so,
             | how can I make money for myself out of this before I fail
             | upwards?".
             | 
             | I'd wager the SF situation is just the same old looting by
             | the Very Important People who run the show, leaving
             | employees and actual managers with a shoestring and two
             | peanuts to make it all work, while the head honchos bathe
             | in gravy.
        
               | arrosenberg wrote:
               | > No, the guiding principle for the folks at the top is
               | "so, how can I make money for myself out of this before I
               | fail upwards?".
               | 
               | Who are "the folks at the top"? Transit agency executives
               | in California make less than software engineers (though
               | if you put in your time, you get a nice pension to make
               | up for it) - how are these people getting rich? If you
               | mean the politicians, sure, I've already said they're
               | half the problem.
        
               | bhawks wrote:
               | They make money by exploiting their position not through
               | their salary.
               | 
               | These people are around in SF - see Mohammed Nuru,
               | Rodrigo Santos and so on.
        
               | stackskipton wrote:
               | There is always stable "Go work as consultant at company
               | that does a ton of business with former agency. This
               | position is in no way related to any agency work they
               | did. :wink:"
        
               | inferiorhuman wrote:
               | Nah, Jeffrey Tumlin (current SFMTA head) makes about
               | $500k total comp.
        
             | skrbjc wrote:
             | >Let me guess, you also think public servants are lazy,
             | overpaid and unskilled? Comments like this belie how many
             | people form opinions of public agencies without having any
             | real experience with government.
             | 
             | I've worked in the public sector for several years, and
             | there are definitely a decently-sized group of people in
             | many different positions that are riding out their time
             | until retirement trying to fly under the radar and do as
             | little as possible. Not to say there aren't people that
             | aren't competent, but as a competent person you have to
             | work around the people that don't care and often actively
             | don't want to see more work come their way to get anything
             | done. It becomes tiring and eventually the private sector
             | is too enticing and the competent people leave.
        
           | DowagerDave wrote:
           | or they're not immune to the same urge all SW devs have to
           | rewrite - it's a lot easier to start from scratch than (a)
           | figure out how an existing system works, (b) figure out how
           | to upgrade or modernize
        
           | whyenot wrote:
           | It reads like you are conflating newer with better.
           | 
           | It was more than a few years ago, but I remember the posts on
           | Slashdot where tech people were wringing their hands over the
           | fact that election offices were not adopting touchscreen
           | ballots fast enough. Some posters were also promoting things
           | like internet voting. How did that work out?
           | 
           | Sometimes it actually is better to move slowly instead of
           | adopting something new just because it is new. With old
           | equipment and older technology, we know the failure modes, we
           | have the procedures in place for addressing them. Whether
           | it's floppy disks, paper ballots, (or human beings,) older
           | does not necessarily mean worse.
        
           | bongodongobob wrote:
           | The problem is "lower my taxes" single issue voters. People
           | vote for whomever lowers taxes and then complain about
           | potholes. It's always the same people. GOVERNMENT IS
           | INCOMPETENT BLAH BLAH. Yeah, cause you refuse to fund it and
           | everything goes to shit, then is 10x more expensive to fix it
           | compared to properly funding maintenance. You end up paying
           | more in the end because these people can't see past the
           | couple hundred dollars they saved on their property taxes and
           | wonder why everything is turning to shit.
        
             | aorloff wrote:
             | You have some evidence that government is not incompetent ?
        
               | afavour wrote:
               | It's weird that people have this notion about government
               | and _only_ government. There are things governments do
               | well and things they do badly. You could say the exact
               | same thing about private business: businesses go bankrupt
               | every day, wasting time and resources. But no one argues
               | that the very concept of business is "incompetent".
        
               | asveikau wrote:
               | The US government created the network you used to write
               | that comment.
        
         | JohnTHaller wrote:
         | Same thought I had immediately when I read the story. Worth
         | fixing that failure point at least.
        
         | jandrese wrote:
         | This was my thought. Floppy emulators are a dime a dozen thanks
         | to the retrocomputing community. Emulators for the custom
         | boards full of discrete logic, opaque ROM chips, PLAs, and so
         | on from long dead companies are going to be a much bigger
         | challenge.
         | 
         | If they had good schematics for all of the parts it might be
         | possible to keep the the system running for a long time with a
         | couple of smart EEs who are comfortable with the scope and
         | soldering iron, but eventually they're going to run out of some
         | obscure part and be up a creek.
         | 
         | Or maybe they could replace entire boards with home designed
         | versions condense all of the old logic down to one chip and a
         | handful of support components and start in-place upgrading
         | without a total system revamp. Still an expensive process, and
         | one that requires some hard to find engineers on staff, but
         | theoretically spreads out the upgrade process over many years.
         | It also loses out on functional improvement opportunities while
         | your system is made up of a hodgepodge of old and new hardware.
        
           | inferiorhuman wrote:
           | SelTrac was designed to use commercial off the shelf (COTS)
           | parts so there shouldn't be much in terms of needing custom
           | chips, etc. The problem is that Alcatel (now Thales)
           | basically doesn't support any of this (and hasn't for a long
           | time).
           | 
           | The computers in question are, I believe, just thin clients.
        
         | andbberger wrote:
         | the state-of-good-repair grift will continue until the morale
         | improves! SelTrac is fine. and if it ever actually breaks
         | (dubious) they can fall back to block signaling which
         | comfortably handles more trains than muni is capable of
         | running. the MBTA does just fine with it on the green line, and
         | when the MBTA is making you look incompetent you've got to
         | question where it all went wrong.
         | 
         | this is the agency that brought you the central fucking subway,
         | a $300M sewer project masquerading as some red paint on van
         | ness, that picked the already-mostly-grade-separated M Ocean
         | over the N Judah to subway-ify and that is incapable of
         | flicking the traffic preemption switch on the rest of the T to
         | the on position without a decades long pedestrian detection
         | LIDAR project for the unique in the world needs of san
         | francisco.
        
         | fdr wrote:
         | A once a generation refresh seems okay in my book. The Breda
         | trains that came in 1996 were being phased out about twenty
         | years into their tenure, most generally seem to think that is
         | about reasonable a time to do something like that. I don't see
         | software related regimes as inherently that different.
         | 
         | Since I experienced 5.25 inch floppy disk era...and even the
         | occasional bernoulli disk...we could simply say: the system had
         | its run, replacement is reasonable. A lot of stuff had changed
         | since then, and not just in storage media.
        
           | jtbayly wrote:
           | Wait, you're suggesting replacing trains because the
           | computers inside them are old?
        
             | asveikau wrote:
             | I think they're just saying parts should be periodically
             | replaced across the board on that timeframe, and using the
             | fact that the trains are being swapped out as an example.
             | The Breda trains are still running some percentage of the
             | fleet however. I rode in one today.
        
               | fdr wrote:
               | That's right. System refreshes are a fact of life,
               | especially if one is parsimonious in using what works
               | until obsolescence. There are downsides of doing things
               | that way, but I don't see it inherently more defective
               | than other approaches, such as some kind of continuous
               | refresh that has a conceit of keeping a system "modern"
               | in perpetuity.
        
               | thsksbd wrote:
               | But twenty years for a train seems very low. My daily
               | driver is 12 years old, I thrash it every morning and
               | never gives me problems. I live in the snowbelt.
               | 
               | By comparison a train is vastly over-engineered and has
               | an electric drive train. Sure stuff will need replacing
               | but a train should last 40 years at least
        
               | jcgrillo wrote:
               | https://www.wbur.org/morningedition/2016/03/07/mbta-
               | signal-m...
        
         | SllX wrote:
         | What are they still running OS/2?
         | 
         | (they probably are...)
        
         | kazinator wrote:
         | > _like this at every layer_
         | 
         | It may feel emotionally like this at every layer, but the
         | layers that are not floppy discs are completely different from
         | floppy discs.
         | 
         | Other than electrolytics caps (easy replacement items), old
         | electronics is reliable.
         | 
         | Moving parts need service; that's life.
         | 
         | > _needs replacement_
         | 
         | Not believable without detailed justification.
         | 
         | In the present article, someone is literally quoted as
         | everything working just fine.
        
         | yonran wrote:
         | The real scope of the project is a mind-boggling $600 million
         | to track the light rail trains inside and outside the tunnels
         | and to control the trains inside the tunnels. Divided by the
         | 250 light rail vehicles (apparently does not include historic
         | streetcars), it works out to $2.6 million per car. Plus $36M
         | for consulting (2023-11-07 board presentation from the
         | documents that I linked earlier
         | https://www.sfmta.com/sites/default/files/reports-and-
         | docume...).
        
       | gojomo wrote:
       | Remember when, less than 3 years ago, SF projected a $108M
       | _surplus_ for the next two years?
       | 
       | Woulda been a nice time to clean up some of this technical debt!
       | 
       | Or how about the SF Emergency Sirens, taken offline in late 2019
       | for a "2 year" upgrade plan that officeholders implied was
       | already in place?
       | 
       | In August 2023, with _no_ progress whatsover, with the Maui fire
       | disaster fresh on their minds, Mayor Breed  & Supervisors
       | President Peskin touted they'd finally funded a plan to return
       | them to service soon: https://www.sf.gov/news/mayor-breed-and-
       | board-president-pesk...
       | 
       | In that same August 2023 timeframe, Peskin said the plan would
       | bring this "need to have" system "up and running" & to "state of
       | the art" by end of 2024, for $5.5M:
       | https://www.cbsnews.com/sanfrancisco/news/san-francisco-city...
       | 
       | Of course, this was just more blatant self-exonerating bullshit
       | from our local political machines immune from any real
       | accountability for incompetence in basic public functions.
       | 
       | A mere 6 months later in February 2024, nothing's been started,
       | Peskin admitted "we don't even have a plan", the department is
       | still waiting until "funding is identified", and the cost
       | estimate has ballooned to $20.5m: https://abc7news.com/san-
       | francisco-sirens-emergency-911-aler...
       | 
       | That works out to $170K+ for each of 119 units - units that each
       | could probably just be a weatherized consumer-grade handheld
       | device with multiple mobile/packet/sat radios, & a simple
       | authenticated-playback app, mounted on existing poles that
       | presumably already have power and even loudspeakers.
        
         | paradox460 wrote:
         | Sirens don't even need to be that complicated. A motor, a noise
         | generator wheel (basically just a big spinning squirrelcage
         | fan), and a horn. You could toggle them on/off with a simple
         | relay/contactor.
        
           | gojomo wrote:
           | These "sirens" actually need to be able to play spoken
           | announcements - something they did easily since about 2005,
           | before the fumbled 2019-2024 shutdown/upgrade.
           | 
           | More history, through January 2022, via JWZ's blog:
           | 
           | https://www.jwz.org/blog/2022/01/the-reason-the-tuesday-
           | noon...
        
             | paradox460 wrote:
             | Now that you mention it, I do remember hearing the spoken
             | message every Tuesday.
             | 
             | Still, a big loudspeaker is even simpler than a classical
             | motorized siren. No moving parts!
        
             | pastureofplenty wrote:
             | JWZ doesn't like HN and doesn't want his site being linked
             | to on here so your URL redirects to a crude image.
        
       | Workaccount2 wrote:
       | My company provides new compute modules for another major global
       | city's infrastructure that still relies on Intel CPU's from the
       | early 80's. That is, we are building them brand new boards
       | populated with chips that are over 40 years old.
       | 
       | They haven't shown any interest in updating the system. It works,
       | they can get service, and get "new" replacements for things that
       | go bad.
       | 
       | What they might not know though is that there is basically just
       | one engineer we have (and probably the only one on Earth) who
       | knows how to work on these things. He's getting old, and
       | obviously none of the younger engineers really have an interest
       | in learning ancient forgotten systems.
        
         | strangattractor wrote:
         | LOL:) this story seems to resurface every couple of years. My
         | favorite part of the article:
         | 
         | "Jeffrey Tumlin: "It's a question of risk. The system is
         | currently working just fine but we know that with each
         | increasing year risk of data degradation on the floppy disks
         | increases and that at some point there will be a catastrophic
         | failure."
         | 
         | This seems to imply they have been using the exact same disk
         | for the past 20 years (absurd), they have absolutely no idea
         | what is written on the disk and how it can be safely backed up
         | or restored. This would be a problem regardless of the medium
         | used.
         | 
         | Although I hold the line at using paper tape there is nothing
         | wrong with using floppies other than it seems antiquated. It
         | certainly is reliable and cheap. Maybe the only thing that
         | needs replacing is the people running the Muni.
        
           | asveikau wrote:
           | You make a good point. However, even with the disks backed up
           | to a better storage medium, detecting data loss on the floppy
           | and getting a replacement written and deployed to the correct
           | location might take a while. During that time the system may
           | face unacceptable transit delays.
        
             | jcgrillo wrote:
             | Wait, what? If this thing boots from a floppy, and that
             | floppy has a checksum on it, and the boot sequence involves
             | loading the checksum and the rest of the floppy contents
             | into RAM and computing the checksum, corruption detection
             | would be near instantaneous and the remediation would be as
             | quick as it takes to eject the corrupted floppy, grab
             | another floppy copy from the bin, plug it in, and flip the
             | on switch. So the delay would be (roughly) zero.
        
               | asveikau wrote:
               | Let's say it's not just booting from it, but constantly
               | reading.
               | 
               | Let's say no one implemented the checksum. Edit: I forgot
               | that floppies have a crc check, my bad.
               | 
               | Let's say the machine that can write the new disk is
               | physically far away from the machines that read them. (Or
               | the guy with a stack of new disks -- which are harder to
               | find every year -- his office is some distance from the
               | control machine).
        
               | jcgrillo wrote:
               | I guess I assumed it was only reading from the floppy,
               | not also writing to it.. that doesn't seem like it would
               | last very long at all without encountering errors (at
               | least based on my memories of reusing floppies until they
               | were completely worn out)
        
           | nneonneo wrote:
           | Every sizable manufacturer of floppy disks has exited the
           | market. There are no more (major) suppliers for the tech.
           | They've likely been depending on a dwindling supply of
           | functional disks; if at some point they find themselves
           | without enough working disks to operate the system, they will
           | indeed be screwed.
           | 
           | There's also the chance that they take a disk that's on the
           | verge of failure, plug it into the system, and some corrupted
           | commands get loaded into the system. That could easily result
           | in a "catastrophic failure".
           | 
           | Floppy disks are not reliable or cheap. They physically
           | degrade over time, and at this point are nowhere near cheap
           | for "new" disks.
        
         | delfinom wrote:
         | Nah getting you fee engineer interested isn't that hard,
         | there's always a fun challenge in mastering something. You just
         | have to give them time to figure it out.
         | 
         | At my workplace, we still ship devices with AMD 8086 processors
         | to the military. And AMD still makes us 8086 processors on
         | special order.
         | 
         | We got a few guys under 40 that work on that project once in a
         | blue moon when a change is required.
        
         | culopatin wrote:
         | Are you hiring young engineers who want to learn the old stuff?
        
           | LeifCarrotson wrote:
           | Where are you going to find a young engineer who wants to
           | build a dead-end career? I'm sure that if you pay them
           | enough, someone will show up, but most people would rather
           | build skills that will continue to be useful rather than
           | skills that will be useless once an inevitable, sorely needed
           | refresh happens.
        
             | asveikau wrote:
             | I think you overestimate how much working on obsolete tech
             | is a dead end career. Skills are still transferable.
             | Someone who can sling 8086 assembly can figure out your
             | dumb web app.
        
             | culopatin wrote:
             | I'd do it. I'd like to be the expert that can help them
             | choose or maybe build the replacement.
        
         | gopher_space wrote:
         | One of my older jobs relied on ancient hardware, and the life
         | of the company was measured by parts in the warehouse and
         | vendors who still cared. From their perspective they'd have a
         | modern system as long as your company delivers.
         | 
         | > hey haven't shown any interest in updating the system.
         | 
         | So expensive to update that there's a calculated end-of-life to
         | the system. They'd _love_ to know about your engineer
         | situation. That 'd trigger plans put in place a while ago.
         | 
         | Your company could do a last big batch for the city and send
         | the old guy out with a nice bonus.
        
       | throwaway74432 wrote:
       | >SFMTA's train control system relies every morning on 5 inch
       | floppy disks.
       | 
       | That's not the 3.5" floppy disk in the video. This is the _old_
       | floppy disks[1]
       | 
       | 1. https://www.digitaltreasures.ca/img/level2_floppy_525.jpg
        
         | prewett wrote:
         | And floppy floppies wasn't "cutting edge technology [in 1998]",
         | either.
         | 
         | The OG floppy disks were 8 inches, though, 5 inch floppies were
         | the small ones (hence why 3.5" disks were "microfloppies"):
         | 
         | https://images.techhive.com/images/article/2015/12/floppy_di...
        
       | yonran wrote:
       | I believe that the article is about this RFP for "Contract No.
       | SFMTA-2022-40 FTA" to upgrade the Communications-Based Train
       | Control System (CBTC) or Advanced Train Control System (ATCS).
       | The resolution to create the RFP for the supplier was approved
       | 2023-01-17 (https://www.sfmta.com/reports/1-17-23-mtab-
       | item-14-communica...), and a $36M resolution to create an RFP for
       | a consultant was approved 2023-11-07
       | (https://www.sfmta.com/reports/11-7-23-mtab-item-11-train-
       | con...). I'm not sure where the actual RFPs are pubished though.
        
       | time4tea wrote:
       | Raspberry pi controller $30, couple of people to make it work -
       | let's say $500k. Sure... there are loads of things that probably
       | need attention, but maybe you don't need to fix them all at once?
        
       | ericmcer wrote:
       | SF is a joke. 20 years of being one of the richest cities in the
       | world (probably in the history of human beings) and it comes out
       | worse than it was before.
        
         | Smoosh wrote:
         | That assessment can probably be scaled up to the USA as a
         | whole.
        
       | aluminum96 wrote:
       | SF Voters rejected Proposition A in 2022 [1], which would have
       | included funding to upgrade Muni's control systems (among many
       | other projects). We'll eventually have to find the money
       | somewhere else when the system fails.
       | 
       | [1] https://www.sfchronicle.com/sf/article/S-F-voters-
       | narrowly-r...
        
       | fifteen1506 wrote:
       | Probably just a ploy to privatize the whole thing. Nothing to see
       | here, move along.
        
       | JoyousAbandon wrote:
       | Absurd errors in this article. Starting with:
       | 
       | "it was from an era that computers didn't have a hard drive"
       | 
       | Absolute BS. Pretty much every computer had a hard drive in 1998,
       | and most had CD-ROM.
       | 
       | Then they referred to the 3.5" disk as a "5-inch floppy."
       | 
       | <sigh>
        
       | asveikau wrote:
       | Just a few short hours ago I was hearing some Muni workers at
       | West Portal station talk about this story. One said to the other,
       | "young people don't know what a floppy disk is anymore".
        
       | ornornor wrote:
       | > SFMTA's train control system relies every morning on 5 inch
       | floppy disks.
       | 
       | And in the video, she says "on 3x 5 inch floppy disks like this
       | one <shows a 3.5 inch floppy>"
        
       | kazinator wrote:
       | > _It 's a question of risk. The system is currently working just
       | fine but we know that with each increasing year risk of data
       | degradation on the floppy disks increases and that at some point
       | there will be a catastrophic failure._
       | 
       | The key words in the sentence are: "it's working just fine".
       | 
       | Data degradation of floppy discs is easy: just copy them to fresh
       | ones, and verify that you have a good copy. The images should be
       | safely backed up so they can be regenerated. (Plus there are
       | emulators; a topic covered elsewhere under this submission.)
       | 
       | I mean, are they really using the same 30 year old floppy discs
       | over and over again until they degrade?
        
       ___________________________________________________________________
       (page generated 2024-04-10 23:00 UTC)