[HN Gopher] Due to failure in the IT system, it is not possible ...
___________________________________________________________________
Due to failure in the IT system, it is not possible to run any
trains today
Author : ar0
Score : 614 points
Date : 2022-04-03 21:44 UTC (1 days ago)
(HTM) web link (www.ns.nl)
(TXT) w3m dump (www.ns.nl)
| lifty wrote:
| Most likely their Kubernetes cluster went down. (not a joke)
| mostertoaster wrote:
| Probably Putins fault.
|
| Conspiracy theories are so fun.
| qwezxcrty wrote:
| Based in my knowledge as a public transportation enthusiastist in
| China, I got really surprised to see a IT problem bring the
| entire system down. There should usually be some "more manual"
| backup mechanism that allows downgrade and operation of trains
| without computerized signaling system. By kinds of token
| signaling it will still be possible to run trains as long as the
| physical infrastructure is still there. The efficiency will be
| much lower and more prone to (potentialy deadly) errors, but no
| more trains today sounds unlikely.
| apexalpha wrote:
| The Dutch rail network is top 3 busiest in the world. Many
| lines have 10+ trains in each direction, every hour.
|
| The usage of certain lines at a certain time is calculated in
| periods of _6 seconds_!
|
| We're waaay beyond normal human coordination capacity.
| rdxm wrote:
| HNHatesUsers wrote:
| gridspy wrote:
| tl;dr : Restarting train schedule mid-day is too hard and unsafe.
| So we're sending them back to depot instead.
| amelius wrote:
| Why "unsafe"? Isn't safety supposed to be handled at a
| different abstraction level (in hardware)?
| coder543 wrote:
| Defense in depth? With this safety feature inoperable, the
| margin of safety is reduced.
|
| Other systems that might normally provide safety critical
| redundancy could be providing the sole measure of safety,
| with no other redundancy available in case one of those
| fails.
|
| "Unsafe" is always defined based on context.
| throwaway0a5e wrote:
| What's the point of having redundancy if you can't use it
| to avoid an impact to service?
|
| Edit: and to be clear, we're not talking about holding down
| a resetable circuit breaker to avoid being late, we're
| talking about the rail network of an entire nation being
| inoperable for a day.
| throwawayboise wrote:
| A passenger aircraft can still fly with a failed engine,
| but it must land at the nearest suitable airport. Because
| at that point, another failure could not be accomodated.
| This sort of redundancy is about preventing disaster, not
| about avoiding service disruptions or keeping to a
| schedule.
| coder543 wrote:
| The point is to still have it when you didn't know you
| needed it. That's the purpose of all redundant safety
| systems. The redundancy isn't there to keep the service
| operating; it's to keep people from dying.
|
| You can _never_ know if the primary safety system is
| functioning perfectly, so you need other systems to be
| there to step in when the primary fails _unexpectedly_.
|
| If you detect the primary system has failed, isn't it
| reasonable that you should stop operation as quickly and
| safely as possible, and be thankful nothing bad happened
| while you lacked redundancy? Any SPoF could be fatal for
| hundreds of people.
| throwaway0a5e wrote:
| I'm not going to write the wall of text that's required
| to properly describe the situation but that's just not
| how these systems work. Industrial automation has all
| sorts of feedback loops so they people operating it know
| if they can trust it. The people using it are pretty much
| always trained in how to run the various systems manually
| so that an errant sensor doesn't turn into a clogged up
| rail line or waste whatever is in your process equipment
| that happens to be mid-cycle.
|
| This isn't some consumer appliance where you have to
| stupid proof every inch of it. These systems are bespoke
| and their architecture is mostly a matter of business
| decisions and not at all a matter of the internet peanut
| gallery trying to figure out how safe they can make it.
| coder543 wrote:
| If you want to claim public transit operators are happy
| to take on liability for potential deaths by continuing
| to operate when they know about reduced (or no) safety
| margin, that's an interesting idea.
|
| Calling me a "clipboard warrior" isn't furthering your
| argument, though.
|
| This comment provides an excellent example along the
| lines I was talking about:
| https://news.ycombinator.com/item?id=30902016
| throwaway0a5e wrote:
| spullara wrote:
| I think that trains can travel in both directions on the same
| track and the schedule keeps them from hitting each other
| head-on.
| blibble wrote:
| scheduling hasn't been used for preventing conflicting
| movements since the 1830s
|
| https://en.wikipedia.org/wiki/Railway_signalling#History_of
| _...
| makeitdouble wrote:
| Isn't way too late if the conflict is detected at signal
| level ?
|
| The worst will have been avoided, but at least one of the
| train will have to back down to the nearest junction, and
| depending on how packed it is they might have to back
| down a few more train as well. I'd assume in reality it
| would deadlock from there.
| denkmoon wrote:
| That's why they have fancy software to route incoming
| trains around delayed ones.
| makeitdouble wrote:
| I suppose it relies on some slack in the system and
| everything else moving as expected. Which could also be
| the reason why they don't want to restart a schedule at
| mid-day with every active train in a non standard state.
| lmm wrote:
| The worst case is the trains stop running for a while,
| which it sounds like is what they pre-emptively did?
| throwawayEuropa wrote:
| It still sometimes is. There was a big crash in Czech
| republic recently, where two passenger trains (motor
| units) collided head on between stations.
|
| Some local routes with few trains a day, where there is
| no longer any track-side staff, operate in a simplified
| rules (there is a whole code for that). There, the train
| will arrive to a station and depending on a symbol in the
| timetable, it will either continue to the next station
| without any signals or permissions needed or will wait
| for crossing train in oposite direction (the train driver
| must also unlock the station office and call the
| dispatcher on phone in some cases). Switches are either
| operated by train crew or they are self-returning (one
| direction always go to track 1 in station, the other
| direction goes to track 2, and the switches can be
| travelled through in the oposite direction without
| setting them first (this is an accident on standard
| switches) and they return back.
|
| So, the problem was that the crossing was there on some
| days and wasn't on some other days, driver forgot or
| misread the day and the train left into oncoming train...
|
| Was a big discussion after that to do anything, even an
| uncertified mobile app to shout "train coming" at the
| driver... Otherwise Czech rep. has a very modern European
| rail infra, just some local branch routes run like this
| saving them from being cancelled completely, but the
| rules were simified maybe too much.
| throwawayboise wrote:
| There is also the token-based control of bidirectional
| travel on single-track, in which physical posession of a
| token governs permission to enter a track block. Still
| used in various places.
|
| https://en.wikipedia.org/wiki/Token_(railway_signalling)
| jrockway wrote:
| That is not strictly true. The LIRR Main Line east of
| Ronkonkoma uses timetable/train order scheduling:
| https://en.wikipedia.org/wiki/Train_order_operation
|
| What is more interesting is how they get away with this
| when PTC is now mandatory (a lot of special operating
| rules/restrictions).
| bigDinosaur wrote:
| It'd make sense that a line famous for being terrible
| around the world (in transit circles) would still use
| such a system.
| denkmoon wrote:
| What if they are late?
| gridspy wrote:
| Here is a theoretical solution:
|
| Lets say there is a north-south train line that is
| shared.
|
| The schedule can create a number of north slots and south
| slots into which a train of a given speed and length can
| fit. A variety of different trains could fit into the
| same slot.
|
| If a train is late, that train is moved into a later
| slot. That leaves a free slot where the train was going
| to go. Perhaps every later train is then delayed into a
| later slot also. Lets call this a rippling delay.
|
| You would probably schedule some spare slots to prevent
| delays rippling through the entire day's schedule. Some
| slots might be long enough to permit several trains going
| in the same direction to travel (it's more efficient if
| 10 north trains go then 10 south trains rather than 10
| north/south interleaved).
| makeitdouble wrote:
| Might not be your point, but I wouldn't be surprised if
| having people stuck in wagons in the middle of a section for
| half an hour because of conflicting schedules would be seen
| as safety risks for the travellers.
| jrockway wrote:
| It sounds like it was a crew scheduling issue. So if you
| implement some manual replacement for the automated system,
| it's possible that train crews will "time out" before they
| reach the destination, stranding passengers in the middle of
| nowhere. Or, will make a tired crew operate the trains, and
| tired people aren't very good at safety sensitive tasks.
|
| There was an incident a few years ago where a LIRR track crew
| member stepped in front of a train going 78mph and died.
|
| https://www.ntsb.gov/investigations/AccidentReports/Reports/.
| ..
|
| The employee that died had been on duty for _38 of the 50
| hours_ leading up to the fatality. One of the NTSB 's
| recommendations is to implement software that avoids worker
| fatigue:
|
| "The FRA encourages the use of certified biomathematical
| models, such as the Fatigue Audit InterDyne Model and the
| Fatigue Avoidance Scheduling Tool (FAST) by railroads to help
| them develop work schedules for safety-sensitive employees
| that align with healthy work-rest scheduling practices;
| however, these safety measures do not apply to roadway
| workers. 14 The work schedules developed through
| biomathematical models avoid many pitfalls causing worker
| fatigue that arise from excessively long work hours, highly
| variable work shift times that disrupt human circadian
| rhythms, and infringement on sleep opportunity times."
|
| TL;DR: it's not about the trains crashing into each other.
| It's about sleepy people making bad decisions that gets them
| or their colleagues killed.
| usr1106 wrote:
| > It sounds like it was a crew scheduling issue.
|
| I'm not from the NL, but I have travelled there by train.
| My feeling is that they have very dense traffic, but in a
| relatively small country most trains don't travel more than
| 2 hours. That means each driver and each conductor needs to
| make 4 different trips a day. And because they need breaks
| they proably switch trains during a working day. If all
| this planning has been computerized for many years it just
| means that nobody knows what to do next if the computer
| screen remains blank. Probably they don't get a printout in
| the morning like it was some decades ago. No idea how
| adaptively the system would react during the day
| dependening on delays and disruptions.
|
| Just guessing, maybe some insider knows more.
| rgj wrote:
| 20 years ago, the NS - the Dutch National Railroad
| company - tried to remove complication from the railroad
| schedule and got a lot of criticism from their employees
| because it would mean they would travel less far from
| their home stations. Train drivers said "so we will only
| be driving around our church". It led to a lot of
| protests and eventually a lot of the plans were revoked.
|
| See https://nl.m.wikipedia.org/wiki/Rondje_om_de_kerk (in
| Dutch, use Google Translate to translate to a language of
| your choice).
| 323 wrote:
| Maybe it might cause a traffic jam - blocked forwards,
| blocked backwards.
| supernova87a wrote:
| While having algorithms and computers dictating every aspect of
| train movements, crew scheduling, resource allocation is
| efficient and in a sense optimal, I think there's something
| compelling to be said about having a system that can survive with
| human-rememberable patterns when such an optimizer fails. That it
| can fall back on (or have at the core) simple patterns that
| people can still operate without algorithmic intervention. I.e.
| the algorithmic tuning enhances at the edges, and isn't at its
| core some tangled non-interpretable system that someone can't
| operate without computer assistance. The idea of a protected set
| of core routes and staffing patterns that operates without having
| to be computer-determined?
|
| I used to be so impressed at European train stations that they
| had a single sheet that would give the entire day's arrivals and
| departures, down to which track. I assume that behind that were
| schedules that weren't so far from being hand-calculated with
| equivalent sheets that told people and equipment where to be at
| what times.
|
| I think we lose something when we jump so far forward that you
| cannot fall back gracefully without a system completely breaking
| down.
| laurent92 wrote:
| > I assume that behind that were schedules that weren't so far
| from being hand-calculated
|
| It's more than that: The trains are negotiated at political
| levels! The "8:18 to Marseilles" (fictitious example) could be
| a headline in the news if the region refuses to fund it, and
| the worker's union may have striked to keep it, while
| inhabitants' HOA has negociated with the city to keep it under
| 12EUR: They are the object of a convergence of fixed interests.
| They run for generations: the train I took as a kid is still
| arriving on the same track today (and in one stop, it stops at
| track E; tracks A-D were dismantled but never renumbered due to
| this legacy).
|
| The good thing with rails is that they aren't going away, and
| trains can be negotiated for decades, they're far from being
| scheduled on-the-fly.
| pjmlp wrote:
| There is an Isac Asimov book about this.
|
| A crew is stranded in space, because of a computer malfunction
| and no one on board remembers how they used to be programmed.
| KineticLensman wrote:
| In at least one Arthur C. Clarke short story the problem
| doesn't arise because they still have sliderules
| at-fates-hands wrote:
| Is that "The Last Question"?
|
| https://en.wikipedia.org/wiki/The_Last_Question
| Tenoke wrote:
| Are you posting it reflexively because 9 out of 10 times
| Asimov is mentioned that's the story people talk about?
| Because that's definitely not the plot of it.
| andybak wrote:
| 7/10 times.
|
| The other three are the Carthage time-viewing story.
| vages wrote:
| I work within the train system of a European country (with
| ticketing, so I'm on the top and not the bottom of things).
| From my experience with station and onboard crew, I am very
| certain that the schedules and patterns are still very much
| human rememberable, at least collectively: People have to show
| up for work, drive the train and many other things.
|
| My guess is that the failure is related to signalling or some
| other knowledge that exists "between" the trains (and brains).
| Having systems that can negotiate right of way is necessary to
| increase the number of trains and the speed of each individual
| train beyond their pre-computer-era limits.
|
| Having all trains stop for one day is probably a judgment call
| to lower the overall duration: Having all trains stop for a day
| is a pain in the neck, but you know when you're done. If you
| could not wait, you could probably have (quite) a few trains
| run without problems. However, you would have to work for a
| very long time to get all the trains back to where they were to
| resume the regular time table. This would be the preferable
| option during a time of crisis.
| mikebos wrote:
| The problem was in the scheduling of personnel and trains.
| The train schedule in the Netherlands is not human memorable.
| This is something they tried to correct in the past but was
| met with protest from employees because then they would only
| have one route on a day which would be boring.
| t0mas88 wrote:
| The Dutch railways said (in Dutch) themselves that it was a
| problem with the scheduling software which also has a role in
| safety. Apparently this is also used to route around for
| example a stopped train, so when it's not working it's not
| safe to operate anything but a handful of trains.
|
| There was a lot of extra impact because this system also
| feeds the passenger information so their app, website and
| information screens at the stations were not working either
| leaving passengers in the dark.
| belter wrote:
| There is no correlation between the schedule you see at
| Dutch train stations monitor screens, and the actual
| schedule of Dutch trains:-)
|
| You might see this is an ironic comment, but it is as much
| that, and as much a technical statement of fact. Sometimes
| the trains arrive at the correct time ( +/- 10 min) so they
| correlate to what you are seeing on the screen.
|
| But unless it's a delay of 1 hour or more, neither trains
| or tracks will be updated on the screens. It's just happen
| they sometimes coincide, so they might correlate, but only
| by coincidence. Trains schedule and train reporting on the
| train station are, I was told..."Run by different
| companies!"
|
| -> Source: Conversation with platform manager at Amsterdam
| Central Station a few years ago. ( not that many years...)
| Denvercoder9 wrote:
| You either misunderstood, misremembered or were
| misinformed. As long as I've been taking the trains (20
| years or so) here, the monitors have always updated with
| schedule changes. Even when the information screens were
| still analogue, they updated with delays and track
| changes.
|
| It might be that what you heard only applied to
| international trains, those have generally lagged behind
| technology-wise (though nowadays there's certainly also
| real-time schedule information for them).
| belter wrote:
| Not true. It's not if the screens get updated. They do
| not get updated based on the actual train on the track
| but from information info you can infer from an pre-
| established schedule.
|
| Let me tell more about what prompted the conversation
| with the platform manager. I was staying at a hotel in
| Amsterdam while doing an IT project in the north of the
| country. One of the days took the wrong train. At that
| time, I just attributed it to not really carefully
| reading the platform indicator. The next day had to take
| the same train, at the same time, carefully checked the
| platform monitor, but as I had been burned the previous
| day....Decided to also ask the train ticket controller at
| the train door. He laughed and pointed me to another
| different indicator some trains have, and that happened
| to show a different destination for that specific train.
|
| The cognitive dissonance that I experienced was so
| strong, I decided right there to also be late that day,
| look for the train station manager, for a polite but
| frustrated conversation on my part, and it was then he
| clarified this.
|
| Also any Internet search will show others with similar
| experiences.
|
| Edit: Looking at this example from 6 months
| ago...Continues to happen:-)
|
| https://community.ns.nl/dienstregeling-59/geen-sprinter-
| maar...
| Denvercoder9 wrote:
| > Not true. It's not if the screens get updated. They do
| not get updated based on the actual train on the track
| but from information info you can infer from an pre-
| established schedule.
|
| Not true in general. The screens do actually get updated
| based on the actual schedule when there are track changes
| and/or delays. They don't stubbornly stick to the pre-
| planned schedule. I've seen that happen countless times.
| That said, occasionally I've seen the screens lag behind
| reality a bit, usually during and around big
| disturbances.
|
| > Edit: Looking at this example from 6 months
| ago...Continues to happen:-)
|
| > https://community.ns.nl/dienstregeling-59/geen-
| sprinter-maar...
|
| This example doesn't support your argument: there's not
| even a claim there that the screens weren't correct.
| belter wrote:
| Translating only one part of the long thread it says
| exactly what I was referring to:
|
| "...Ik zeg "als het goed was" want soms gebeurt het bij
| vertragingen wel eens dat de schermen een verkeerde trein
| aangeven..."
|
| "...I say "if it was good" because sometimes it happens
| with delays that the screens indicate the wrong train..."
| markus92 wrote:
| They got GPS tracking on the trains. Any delay over 5
| minutes will show up automatically, less than 5 minutes
| you can see on the app. The train operator took over the
| information screens, so it's all part of the same company
| now.
| belter wrote:
| Do you know as from which date?
| mikebos wrote:
| It has nothing to do with safety it's pure a who is where
| system.
| jtwaleson wrote:
| Curious why "who is where" is unrelated to safety? You
| mean it doesn't affect the signaling?
| mikebos wrote:
| There are several systems all with specific functions.
|
| ProRail the infra manager has all the systems related to
| safety and long term planning. Think air traffic control
| to keep it simple.
|
| NS has all the systems for communication with passengers
| and personnel and train movement (planning). Think
| airliner.
| mpol wrote:
| Signaling is on a separate system. There is ATB-EG/ATB-NG
| for max speed and there is ATB-Vv for stopping for red
| lights.
|
| Part of train history in the Netherlands is the train
| disaster at Harmelen in 1962 where 93 people died. It
| spurred the installation of these systems. Every time
| there is even a minor accident with a train, the national
| governement is asking questions, up to the highest level.
| For contrast, every time a child dies in traffic, only
| the local government (gemeente) asks questions. It is
| considered completely unacceptable for a train accident
| to involve more than 2 deaths, it seems.
| https://en.wikipedia.org/wiki/Harmelen_train_disaster
| sofixa wrote:
| Which on a train system is paramount for safety,
| otherwise you'd have trains crashing into each other
| because they're on the same track.
| markus92 wrote:
| Different system, that'd be the switches/signalling.
| Freight and international trains were running just fine,
| safely. This was literally just planning which train
| (rolling stock) is going to operate which train (ride)
| with which personnel.
|
| Safety could be related to duty times, or making sure the
| rolling stock is fit for a certain trajectory (maximum
| platform length, weight restrictions etc.)
| HowTheStoryEnds wrote:
| No the section a train is on is automatically and
| mechanically secured by the lights on the section ahead
| and afterwards. (2 sections if you count the yellow
| warning lights)
| VBprogrammer wrote:
| Unless they are operating a dynamic block system like the
| channel tunnel lines and other high speed lines across
| Europe. Anything that runs over 125mph (or has to mix
| with traffic which does) isn't using block signalling.
| iggldiggl wrote:
| True dynamic block ("moving block") still isn't that
| widespread, at least as far main line railways are
| concerned. High-speed lines are still by far and large
| using fixed block signalling with conventional train
| detection (track circuits or axle counters), it's just
| that the movement authority is given via some sort of cab
| signalling.
| Foobar8568 wrote:
| Try to run the pricing system in Denmark if it hasn't been
| changed over the years.... It was "deem" "impossible" for a
| human to calculate the pricing within a reasonable amount of
| time (with all the associated bonus).
|
| Do not forget that there might several companies running on
| the same track, cargoa and passenger trains etc.
|
| Managing a busy track is hell.
| vanderZwan wrote:
| > _It was "deem" "impossible" for a human to calculate_
|
| I live in Malmo so you have to understand I'm obliged to
| make a "well no wonder given the Danish counting system"
| joke here
|
| > _Fifty is halvtreds, which stands for halvtredje-sinds-
| tyve, meaning "third half times twenty", or "two scores
| plus half of the third score" [21/2_ 20]*
|
| https://www.languagesandnumbers.com/how-to-count-in-
| danish/e...
| gsliepen wrote:
| Indeed it is a very weird way of counting. It was a
| question on Code Golf a few years ago:
|
| https://codegolf.stackexchange.com/questions/92657/count-
| to-...
|
| What is remarkable here is that usually there are a lot
| of answers in dedicated golfing languages that can solve
| seemingly very complex tasks in just a few tens of bytes,
| here there is only one such answer, and it takes 115
| bytes. So perhaps one could say it's even hard for
| computers?
| tromp wrote:
| That is not 115 bytes, but 115 wide (16-bit) characters,
| or 230 bytes.
| xigoi wrote:
| 05AB1E has a custom codepage where all 256 characters are
| one byte.
| Aperocky wrote:
| Commercial Flights can still be flown manually, without many
| automated sensing system.
|
| But if they don't work you don't take off, it's simply not
| worth the risk.
| lbriner wrote:
| You also have to ask at what cost to maintain the backup system
| and how easy it is to test the Disaster Recovery process. If
| once in a while, you have a few hours delay (or in this extreme
| case, a whole day) you might ask if it would be worth it,
| especially if you could only test it by adding random failures
| to the main system like Netflix do!
|
| As another poster mentioned, they are many, many independent
| variables in a railway system. In the old days, people
| recovered from problems by gut feel and experience was OK
| although there is usually no "right" answer. The newer systems
| apply some statistical anlysis and try and make the best
| decision usually with humans doing the last part.
|
| I'm not sure about "because safety" though since signalling
| systems protect trains from each other and they only have
| automation in the sense of first train that arrives gets it
| route sets, the system cannot make trains crash unless there is
| a critical bug.
| pmontra wrote:
| The train schedule as seen by passengers stays the same for
| years, often to the rail number in stations. What changes are
| the crews, wagons, locomotives, possibly routes on the railway
| (switches and reactions to delays and failures.) A friend of
| mine worked at an app used by a railway's personnel to know
| where to go next. The app was not particularly complicated but
| the backend, which has been running for ages, took a long time
| to run. I don't know the details.
| djcapelis wrote:
| As a general response not specific to this system failure:
| Train systems are designed to fail safe when not in state and
| significant safety engineering is done to ensure that if a rail
| system remains in a known good state over multiple decades
| almost every potential threat to human life caused by various
| equipment failure possibilities is quantified and reduced to a
| level of risk that's acceptable. (Zero typically, for any known
| non-adversarial risks.) Many of the pathways in the fault tree
| involve shutting something down if it's out of the correct
| state. So you will see train networks fail safe rather than run
| in a regressed state since these vehicles with up to a thousand
| human lives on board drive themselves to some degree in every
| modern system and we tend to hold them with a level for safety
| incompatible with easily winging it when something is wrong.
| This absolutely makes the system more brittle and less
| resilient, but for the most part it's the tradeoff we accept in
| most modern rail engineering projects which are designed with
| the goal of reducing human loss of live to zero over the entire
| span of the system's operational lifetime.
|
| That said, train systems themselves usually maintain a ton of
| active operational procedures that remain part of staff
| training which would theoretically allow them to be more
| resilient and these procedures are kept current and you will
| often see them used in emergencies. I.E. on automated systems,
| trackway signaling often exists in the "not lit" off state and
| if the automated train control system failures, is often
| capable of lighting up for human controlled movement (usually
| required to be done at reduced speeds) of the trains during
| irregular operations like bringing stuck trains to the end of
| their lines and letting the people in them out at the next
| station. And any trains that still have a human conductor in a
| cab (or a cab for a human conductor at all) have procedures for
| that human to operate the train directly, even on segments of
| track where automated train control otherwise control all train
| movements.
|
| We could try and run these systems in these degraded states
| under these emergency procedures, but most modern systems have
| safety analysis and engineering which focus on bringing the
| system to a safe halt state while fixing the underlying issue
| and returning to full operational state rather than messing
| around with rolling along in degraded states with unknown but
| assumed to be increased potential for catastrophic engineering
| failures with outcomes including loss of life.
|
| (In the US it is generally illegal for a human being to operate
| a train unaided by some form of train control above 79mph. Most
| other systems have similar rules. We no longer trust humans
| themselves to operate these vehicles just because so many human
| lives can be at stake when they fail.)
|
| Source: I did a stint working on system security and a little
| bit of electromagnetic compatibility safety engineering for
| rail systems. It was enlightening to see how the capital E
| engineers I was working alongside handle these concepts and
| design risk out of modern systems. The bulk of my work was in
| the US but the parts that weren't were on systems that spanned
| multiple continents and/or train cars which were distributed
| worldwide. (To my knowledge none of my work has ever related to
| system discussed here.)
| ncmncm wrote:
| It wasn't the IT failure that kept the trains from running. It
| was management failure, not ensuring they could run without the
| IT system.
| system2 wrote:
| Do you know anything about NL train management? Just curious,
| how do you even know this is a management fault.
| regularfry wrote:
| It's both. Even with a system you could run without it, having
| to fail over would still be a negative event.
| lucasyvas wrote:
| Now we're getting somewhere!
| j0057 wrote:
| They do have some kind of crisis mode where trains run back and
| forth between three stations, but maybe they had no way to
| trigger even that.
| Tepix wrote:
| That sounds unrealistic given the size of the problem.
| Aeolun wrote:
| They did it before the whole thing was automated. Surely they
| can do it again.
| CyanBird wrote:
| Sure, * _but not on a single day without any warning what
| 's ever*_
| Aeolun wrote:
| I'm sure _nobody_ raised this issue before. They just
| didn't care "Don't worry, the system will always work".
|
| But here we are.
| gpvos wrote:
| That was with about a third of the current number of
| trains.[0] Expect complexity to go up exponentially as you
| increase the train frequency.
|
| [0] Extremely rough off-the-cuff estimate. They were
| obviously automating as they went along during the years.
| innocenat wrote:
| The automation allowed running more trains than ever before
| using less personnel than ever before.
|
| You want the train company to keep 2x-3x personnel on hand
| just for days like this? Not to mentioned transition time.
| ncmncm wrote:
| So, run fewer trains. Fewer is better than none.
| innocenat wrote:
| I am not familiar with European train, but it seems that
| they did?
|
| > The international trains are not affected by this
| failure.
|
| > The night trains can still run.
| akvadrako wrote:
| That isn't useful for 99% of passengers. He obviously
| meant run a reasonable percentage of trains, like 25-50%.
| innocenat wrote:
| Maybe we have different definition of fewer trains,
| because I wouldn't expect more than 10% at most,
| realistically 2-3%. And those trains are probably already
| 2-3%.
| gpvos wrote:
| Many international trains were actually not running. I
| haven't read up on the details yet as I was away this
| weekend, but I think it was only short cross-border
| routes that still worked, but not much more. The night
| trains only started up past midnight. And anyway, even if
| more than that had run, night trains and international
| trains only run on a small number of Dutch train routes.
| The three daytime long-distance international train
| routes are: Amsterdam-Rotterdam-Antwerpen and onwards,
| Amsterdam-Arnhem-Duisburg and onwards, Amsterdam-Hengelo-
| Osnabruck and onwards (several intermediate stations
| omitted). The only domestic night service is Utrecht-
| Amsterdam-The Hague-Rotterdam. That's it. You can't fit
| any serious amount of domestic travel into those more
| than they already normally carry.
| Agentlien wrote:
| This reminds me of the story one of my university teachers used
| to introduce himself. He was the most brilliant educator I've
| had, really passionate about making sure people could actually
| follow and so humble. The course was called "Digital and Computer
| Technology" and it was a first year course teaching you to build
| a CPU starting from logic gates and then program it with
| assembly.
|
| He said that when he had recently graduated and just moved from
| Norway to Sweden his first job here was as a technician for the
| local railway. One of his first assignments was a call where the
| computer controlling all the track switches had stopped working.
| Luckily there was a backup, but they needed him to fix it
| immediately.
|
| He arrived, took a look at the computer and its backup running
| next to it. He started out by measuring voltages on both,
| comparing to see what the difference was. After a while some men
| in suits came in and asked how things were going. He said it was
| all good, they said it wasn't because now all trains had stopped.
| Apparently, he had short circuited the backup.
|
| "And that's when I decided to go for a theoretical career!" My
| teacher happily concluded. The classroom was left in a stunned
| silence.
| Saser wrote:
| Rolf Snedsbol? :-) Agreed that he was one of the best educators
| (if not _the_ best) I had
| Agentlien wrote:
| Yes, Rolf Snedsbol!
|
| I absolutely loved his classes. I bumped into him a few years
| ago when I was holding a recruitment thing for Ghost Games
| and it happened to be just after his lecture. I told him how
| much I enjoyed his courses.
| RonaldOlzheim wrote:
| Did he wrote a book? Do you have a link?
| valvar wrote:
| An absolute legend! My favourite of his jokes: "when I moved
| to Sweden from Norway, the average IQ went up in both
| countries".
| Agentlien wrote:
| My favorite story about him was when he caught a student
| sleeping in his class. He paused, mid-sentence, and asked
| the student if they were too tired. The student looked up
| sheepishly and confirmed.
|
| "Did it get a bit late last night? Can't keep your eyes
| open?"
|
| Again, the student agreed. Rolf flashed a huge smile and in
| a singsong voice said "then I have something for you!". He
| walked over to his bag, pulled out a stuffed animal and a
| tiny embroidered pillow. (I have a grainy photo of them
| somewhere).
|
| Rolf tiptoed up to the student, put the pillow in front of
| him, and as the student rested his head on the pillow Rolf
| gave him a pat on the head.
|
| After this Rolf went back, said that if you're that tired
| it's better to sleep in your own bed. He resumed the
| lecture, picking up in the middle of the sentence where
| he'd stopped. No one slept in his class for the rest of the
| course.
| valvar wrote:
| I have a similar anecdote which completely stunned us,
| but which may possibly be part of his standard routine,
| and thus something he does for every set of students:
|
| He always wore the same striped sweater. At one point, he
| took a break to drink some water and remove his sweater.
| But beneath the sweater was another identical sweater!
|
| I assume everything he did was very well-planned and part
| of his pedagogic system and routine, but it was often
| hard to tell if he was acting, or if he was actually
| being serious but eccentric.
| Agentlien wrote:
| I don't remember that but a lot of his antics seemed to
| be planned and part of his way of keeping people awake
| and paying attention. If it is part of his routine I
| wonder when he added it. I took his courses in 2006-2007.
| valvar wrote:
| I took that course in 2015 or 2016, so yeah, I guess
| that's about a decade in which he could have added it. I
| just browsed around Chalmers' course pages and am very
| happy to see that he's still teaching that course.
| alexose wrote:
| My Norwegian grandfather used to tell a lot of these jokes.
| "What do you call a Swedish golfer with an IQ of 120? A
| foursome!"
| ngcc_hk wrote:
| Take a bit time to understand this joke. Crazy. Especially
| the new host would not be very appreciated of that la.
| diarrhea wrote:
| To be fair, a great joke, but not an original one (unless
| he _is_ the original author -- I 've heard it before in
| different contexts).
| aloisdg wrote:
| Read it when new zealander move to australia
| dm319 wrote:
| It's a known statistical phenonema (Will Rogers
| phenomenon) [1] used to illustrate the problems when
| adjustments are made to clusters.
|
| [1] https://en.wikipedia.org/wiki/Will_Rogers_phenomenon
| nojs wrote:
| Concrete example from that article: with improved
| detection of cancer, it's true that "people with cancer
| live longer on average" and also that "people without
| cancer live longer on average" even if there's no change
| to the treatment or anyone's lifespan.
| joenathanone wrote:
| 'Those who can, do; those who can't, teach'
|
| -George Bernard Shaw
| paranoidrobot wrote:
| We extended that at a technical college I went to for a while
| because so many of the teachers were downright incompetent.
|
| "Those who can, do; those who can't, teach; those who can't
| teach work here."
| tristor wrote:
| I heard from a group of teachers (my mother was a teacher)
| that were at our house once. "Those who can, do; those who
| can't, teach; those who can't teach, administrate."
| devoutsalsa wrote:
| I had a professor in college teaching computer systems and
| design. Somehow he got to talking about the NSA. He said
| that back in the 60s, the NSA had computers that were
| running at 1000. "1000?" I asked, wondering if he'd
| forgotten the units. He replied "Yes, 1000."
| Hendrikto wrote:
| And nowadays they are over 9000. How time flies...
| ChuckNorris89 wrote:
| I've heard this joke before on HN.
| devoutsalsa wrote:
| I've told this once or twice before I think. True story.
| Sebb767 wrote:
| I'm not sure I got the pointe. Could you explain?
| mitchdoogle wrote:
| The professor is incompetent because he doesn't actually
| know how to talk about the performance of computers. It's
| not really a "joke", more of an anecdote
| LoveMortuus wrote:
| In the original it's actually over 8000, which is
| interesting because numbers usually don't cause
| translation issues.
| ilaksh wrote:
| Or maybe he was just smart enough to go for a steady job with
| less stress and humble enough to make a self-deprecating
| joke.
| blowski wrote:
| TIL that quote is _actually_ from George Bernard Shaw.
| https://www.phrases.org.uk/meanings/176500.html
| laerus wrote:
| and I thought it was Jack Black
| 2143 wrote:
| And I thought it was by Khaled Hosseini.
|
| (He wrote The Kite Runner (among others), which is a
| heartbreaking novel).
| Dyac wrote:
| There's a common extension to this about teaching colleges -
|
| 'Those who can, do; those who can't, teach; those who can't
| teach, teach teachers.'
| acbart wrote:
| Maybe if we paid those who teach better, we'd attract the
| best of the best to teach. Instead we drive good people out
| by making the job unbearable.
| panxyh wrote:
| We should pay better those who teach better, not everyone.
| Otherwise we just make the market bigger with even more
| room for incompetence. Look at the software industry.
| southerntofu wrote:
| And what kind of metrics can you use to determine whether
| someone teaches well? Whatever metrics you use, there's
| probably ways to game the system without providing better
| services.
| panxyh wrote:
| Exactly, and this would be just as true, only on bigger
| scale, if we just raised wages and tried to filter out
| 'the best of the best' from a now larger pool of
| candidates. I should write "if anything" at the start of
| my previous comment as I don't really believe a good
| solution based solely on financial incentives exists.
|
| Your other comment efficiently expresses my intuition on
| the topic.
| [deleted]
| southerntofu wrote:
| > Maybe if we paid those who teach better, we'd attract the
| best of the best to teach.
|
| I agree struggling for basic survival as a public servant
| is not ideal, but be careful not to turn education into a
| lucrative opportunity where people come for money not
| passion.
|
| Money is certainly not the only way to attract talent
| (though teachers still need to pay their bills) and in most
| schools i've attended the talented teachers were just
| ridiculously hindered by the bureaucrats defining their
| program and agenda. David Graeber's talk "Manageurial
| feudalism and the revolt of the caring classes" comes to
| mind.
| nisegami wrote:
| I believe that we should be making whatever changes we
| need to ensure that some of the most capable minds in our
| field are doing something more useful than ad tech.
| Speaking from experience, seeing where my classmates went
| to after we graduated a few years ago.
| JaimeThompson wrote:
| Maybe they outsourced running the rail system to the people that
| run the Texas power grid?
| bmitc wrote:
| More and more we are seeing "the computer says no" in our lives.
| We've replaced human activity and decision making with
| computerized versions, which are nothing more than the automation
| of stored human knowledge and routines.
| sofixa wrote:
| A much more efficient automation.
|
| For instance, running high speed rail without computer
| signalling would be a huge risk there's no sense in taking (
| due to the speeds involved, by the time you see anything of
| issue like a train or damage in the tracks in front, it's too
| late to stop).
| bmitc wrote:
| I think that's a bit of a leap from what I was saying. I
| didn't imply that all software and computation is bad.
| Obviously, a human cannot live handle any control system that
| needs even remotely fast operation. The point is that due to
| scaling issues, we're removing humans from systems where
| humans excel or could serve as backups in the event of
| failure (and actually vice versa).
|
| Scale creates problems that are partially solved via software
| systems, which only handle a subset of the existing variety
| and new variety introduced by scaling. This makes the systems
| leaky with regards to unhandled variety and incredibly rigid
| (i.e., not robust) against exceptions, exceptions that could
| often be readily be handled by humans. By software being a
| snapshot of some small group of humans' knowledge of a
| system, it is completely unable to respond to live
| operational exceptions. Stafford Beer's work is interesting
| background on this type of thing.
|
| > A much more efficient automation.
|
| There are many, many examples where that is both not true and
| true. For the not true cases, just see literally any customer
| service interaction with a large company in the past half
| decade. I suppose this could be arguable though if one asks
| the question "efficient for who?".
| tsimionescu wrote:
| > There are many, many examples where that is both not true
| and true. For the not true cases, just see literally any
| customer service interaction with a large company in the
| past half decade. I suppose this could be arguable though
| if one asks the question "efficient for who?".
|
| Another example I love, that is even less ambiguous, is
| restaurants (traditional, not fast food) that require
| waiters to take orders on touchscreen PoS systems, instead
| of taking the orders on a notepad and inputting them later.
| The touch screens are incredibly slower on a bigger party
| ordering appetizers, a main course, drinks and desert - it
| takes forever to navigate back and forth and find the right
| item, and usually has to be done again and again for each
| diner. By comparison, a notepad allows the waiter to write
| down exactly what they hear, often just making small extra
| notes to indicate duplicate items.
|
| Of course, this particular one could be improved by
| arranging the menus more naturally for someone taking
| orders, and several other conveniences (such as always
| keeping the current order on screen, to easily tap to add
| "oh, and another beer for me", "I'll have the same desert
| as her" etc).
| burntoutfire wrote:
| A couple years ago, politicians in Poland wanted to do a quick
| change to the country's universal pension system. The change
| had to be quick to score some kind of political win. They wrote
| the new law and went on with quickly pushing it through
| parliament. To their surprise, sometime along the legislative
| process , the contractor responsible for maintenance and
| changes in the pension system informed them that they can
| indeed implement the change - by June next year. The
| politicians learned that, in the mordern computerized world,
| just passing a new law does not change how the state bureucracy
| operates.
| throwaway48375 wrote:
| Could they not make it retroactive?
| burntoutfire wrote:
| Yes possibly, but the whole point was for people to get
| more money into their pockets ASAP (as opposed of just a
| promise of more money in the future, which has lesser
| impact), so that they feel gratitude come elections day.
| lqet wrote:
| Seems like trains are up and running again today as announced,
| with only little delays:
|
| https://travic.app/?z=9&x=628581.0&y=6814491.0
| woudsma wrote:
| I didn't know about Travic, thanks!
| NWoodsman wrote:
| This post is a fascinating use of grammatic passive voice. The
| hallmark is gratuitous use of "being" verbs, i.e. "is,was".
| Passive voice is a way to avoid using active voice, which is an
| implicit way to avoid responsibility for an error.
|
| "Due to the enormous impact of the failure in the IT system, it
| is unfortunately not possible to run any trains today." // =>
| "Due to the enormous impact of the failure in the IT system, NS
| cannot run trains today."
|
| "Although the cause of the failure has now been resolved, the
| impact is considerable. To be able to start up reliably, systems
| must be updated and trains must be brought to the right place.
| That takes time. For our passengers, this is extremely unpleasant
| news." // => "Although we have fixed the cause of the failure,
| the failure impacted most of our customers. To fix this failure
| so that trains would start reliably, we needed to return all
| trains to a central depot, which took nearly 12 hours. We
| apologize for the extremely unpleasant interruption."
|
| "The expectation is that tomorrow morning the normal timetable
| can largely be resumed. The night trains can still run." // => We
| expect to resume minor operation with night trains first and then
| major operations tomorrow morning.
|
| "The IT failure occurred at the end of the morning. It affected
| the system that generates up-to-date schedules for trains and
| staff. This system is important for safe and scheduled
| operations: if there is an incident somewhere, the system adjusts
| itself accordingly. This was not possible due to the failure." //
| => The IT failure occured at 11am in the scheduling system, a
| major requirement for safe operation; if train operations are
| interrupted at one location, the system normally reroutes other
| trains to prevent collisions. This system failed.
|
| "The international trains are not affected by this failure. For
| information about the timetables of other transporters,
| passengers can consult the websites of these transporters." // =>
| The failure did not affect international trains. If you have an
| inquiry about other transportation services, please consult those
| services.
|
| "The journey planner is updated." // => We updated the journey
| planner accordingly to account for the interruptions.
|
| // Generally the use of "passive voice" indicates poorly-taught
| grammar, however the use could also indicate desire to avoid
| responsibility.
| ivraatiems wrote:
| I think this is a translation thing? I'm not sure they're doing
| it on purpose since a lot of it isn't proper idiomatic English.
| NWoodsman wrote:
| The "about us" section of the webpage :
|
| "Who are we? NS is active in the public transportation sector.
| We encourage the use of public transportation and keep the
| Netherlands moving. Our travellers are our 1st, 2nd and 3rd
| priority in all of our activities, and we do our utmost to make
| their trip as pleasant and sustainable as possible from door to
| door."
|
| This is also fascinating. I would expect to find "NS
| (Netherlands Service) provides rail services for the entire
| country". Instead they choose vague grammar, more the idea of
| NS rather than what NS actually is.
|
| "I am a programmer" => "I encourage the use of computers to
| provide solutions for common business problems". Just a strange
| use of grammar.
| samspenc wrote:
| I do hope they have a more detailed RCA at some point, right now
| this is the only helpful paragraph in the article about what
| happened:
|
| > The IT failure occurred at the end of the morning. It affected
| the system that generates up-to-date schedules for trains and
| staff. This system is important for safe and scheduled
| operations: if there is an incident somewhere, the system adjusts
| itself accordingly. This was not possible due to the failure.
| qiskit wrote:
| I was expecting more as well. Not much information to glean
| from a barebones press release.
|
| If I had to guess, it probably was an issue with
| scheduling/timetable software rather than anything to do with
| the trains, rails, etc. Nothing exceedingly seriously or
| difficult to correct.
| XorNot wrote:
| Keep in mind that train schedules control what trains are on
| what tracks when, so while their are literally fail-safes in
| the system, this is not exactly far removed from being
| "safety critical".
| TylerE wrote:
| That hasn't been true in Europe for over a hundred years,
| and exceedingly rare in the US these days.
| benjaminwootton wrote:
| Isn't this exactly what the press release states? " if
| there is an incident somewhere, the system adjusts itself
| accordingly. This was not possible due to the failure."
| TylerE wrote:
| No, it's talking about crew scheduling/allocation , not
| signaling.
|
| The signaling system is totally fail safe. Even if that
| system totally went out, drivers are required to know
| where all the signals are, and to treat an unlit signal
| as a red.
| throwawayboise wrote:
| What if the failure lit all the signals in green?
| TylerE wrote:
| It can't, they're not wired like that. They're engineered
| so any fault shows either red or nothing.
| innocenat wrote:
| The train schedule system data aren't fed into the
| Automatic Route Setting system in Europe and US?
| TylerE wrote:
| Sure, but failing that everything will just stop, not
| crash into other like GP implied
| AndyMcConachie wrote:
| The fact that it happened on a Sunday suggests to me that this is
| the result of a service outage gone awry. Realize that the trains
| failing on a Sunday is really not a big deal. However, if the
| trains don't run for Monday morning rush hour that is a big deal.
| I bet they made a change Saturday night and something broke
| Sunday morning.
|
| It's now Monday morning here and the trains are back to normal in
| time for rush hour. So it's not really a big deal.
| regularfry wrote:
| I have to say that I do find this framing which says "well, it
| didn't affect business, so no big deal" to be somewhat
| distasteful. It's completely dehumanising. Why run the trains
| at all at the weekend, if a weekend outage is no big deal?
| tsimionescu wrote:
| That's not the framing. For the majority of travelers,
| missing a weekend trip is an annoyance. In contrast, missing
| their work is a major headache, for them personally. It also
| cascades pretty badly if it happens on a lathe scale -
| meetings get pushed and overlap others, suplliers are not
| able to deliver, stalling production, financial operations
| get delayed, potentially affecting deadlines - it's a huge
| mess in everyone's life that won't just go away when trains
| come back. It's also as true for non-profits, state
| organizations and schools as it is for businesses.
| stuckindoors wrote:
| There was a massive southwest airlines IT failure compounded by
| weather issues over the weekend. This cancelled numerous flights
| but also snowballed into most of the southwest flights being
| delayed. We were lucky to return home. The news is only reporting
| weather delays yet for those at the airport this weekend...it was
| clearly larger.
|
| If it is true that Netherland's and Italy's transport
| infrastructure were also down I wonder if Russian supported
| hackers are to blame.
| wsc981 wrote:
| _> If it is true that Netherland 's and Italy's transport
| infrastructure were also down I wonder if Russian supported
| hackers are to blame._
|
| It's been Russian hackers so many times last few years (Hunter
| Biden's laptop, Trump elections, etc...), it's a sure bet this
| time it's the same.
|
| /s
|
| Nobody ever got fired for blaming Russians.
| BlueTemplar wrote:
| Their bad reputation is their own damn fault.
|
| See also : assassinations - maybe the death of the top polish
| government & military and Total's CEO in plane crashes were
| just freak accidents, but a nagging suspicion that Russia was
| involved will always remain...
| selimthegrim wrote:
| Southwest has put something out about it so it's in the
| press/Twitter for sure.
| yummybear wrote:
| Failed OTA update?
| throw7 wrote:
| Whatever happened, I actually kinda like the wording in this
| notice in that it did not have the word "glitch".
| qwertox wrote:
| The short sentences were what got me.
| dt3ft wrote:
| There is a swiss company based in Zurich that develops this kind
| of software. I believe it was a c#, wpf application, the last
| time I spoke with them. The schedules that their software
| produces is calculated and set for several years ahead. That
| baffled me! The tracks and infrastructure is booked up so far
| ahead! I think they mentioned that the german railway uses their
| software, nut only the sbb.
| vanviegen wrote:
| From what I understand, the failing system is a bit different,
| in that its primary purpose is to deal with outages and other
| unexpected events.
| chevman wrote:
| It's always DNS!
| comprev wrote:
| Does Not Schedule? :-)
| rossdavidh wrote:
| As a developer, any day in which you don't see a national
| headline about how your f*#(up has just made a lot of other
| people sad and/or angry, is a day that could have been worse.
| user_7832 wrote:
| I was in the station for a different reason when I heard the
| announcement. The people entering and hearing the announcement
| were unsurprisingly very crestfallen, but it appears that twitter
| helped organize a lot of travel, with people posting where they
| were/going and picking up other travellers.
|
| Also probably worth noting is that trains here are used fairly
| frequently, however unfortunately this isn't the first time
| trains are stopping - for example snow is a common reason for
| delayed/reduced service. (Snow isn't very common here in the NLs)
| mushyhammer wrote:
| > with people posting where they were/going and picking up
| other travellers.
|
| That's ludicrous, I wish people helping other people this way
| was more common.
| wolverine876 wrote:
| Imagine if we did that all the time! An online ride board.
| mark-r wrote:
| ... which would stop working if the computer went down.
| shapefrog wrote:
| You could even install rails that go along regular
| routes, and put on scheduled carridges that people could
| all jump into at designated places along the route.
|
| The computer could generate a schedule and post it
| publicly so people knew when they could hop on.
| wolverine876 wrote:
| You really don't see a difference between a public
| rideshare and public mass transit?
| BozeWolf wrote:
| Finally a use case for the blockchain ;-)
| CannisterFlux wrote:
| Sounds similar to a problem that happened a couple of weeks ago
| in Madrid, although that was not the whole network it pretty much
| shut down the trains for the day. That was an IT failure too.
| https://www.20minutos.es/noticia/4973632/0/graves-retrasos-e...
| (in Spanish)
| tuukkah wrote:
| Wow, talk about a single point of failure when an issue in the
| systems of one station can stop all but one train line.
| Interestingly, it seems to be a station where tracks from six
| directions meet so a backup mode of trains simply passing
| through might be impossible:
| https://en.wikipedia.org/wiki/Madrid_Chamart%C3%ADn_railway_...
| [deleted]
| comprev wrote:
| I was walking up to Amsterdam Centraal when I noticed floods of
| people all exiting the building... within minutes an Uber was
| booked to Schiphol before prices rocketed due to demand.
| reilly3000 wrote:
| I think it's worth mentioning that while the IT system couldn't
| coordinate the trains safely, it was a management decision to
| stop them from running. Coordination can happen myriad ways. For
| example, trains could operate according to their normal schedule,
| and if one needed to deviate they could radio in, and an operator
| could let downstream trains know to proceed accordingly. Maybe
| simple cell-phone GPS pings could suffice. Perhaps the state of
| the system was in such chaos that such orchestration wasn't
| possible with the staff on hand. That said, a truly resilient
| system should have multiple coordination channels to fall back
| upon.
| mnumber wrote:
| Computers are great at ensuring your workforce cannot function
| without them. This is a small price to pay.
| throwaway48375 wrote:
| The cost of a single mishap is too high. It's just not worth
| it.
| gpvos wrote:
| The drivers and conductors would like to end up at home at the
| end of the day. Even if you just run the normal schedule, there
| will always be disruptions, and there needs to be a system to
| adjust the planning so people and things end up at the right
| place at the end of the day. That was the system that failed.
| Have you any idea how frequent the trains in the Netherlands
| are and how complex the schedule? "Simply" running the existing
| schedule is an absolute impossibility.
|
| About the only thing they could maybe have done was to fall
| back on back-and-forth shuttle trains between major node
| stations. But that would only provide a very basic service,
| with incorrect train materiel being used for the normally
| expected number of passengers. It would be just enough to get
| passengers and personnel home. I would like to see something
| like a pen-and-paper schedule that they can fall back on, but
| they don't have it. (They did something like that at the first
| corona lockdowns, although with more planning involved.)
| j0e1 wrote:
| What jumps out at me after reading this notice is how it
| completely lacks any apology. I am not prescribing what it should
| say but merely observing that having lived in North America for
| sometime now, it seems hard for me to imagine for such a notice
| to not sound profusely apologetic here.
|
| Note: I do not consider words like 'unfortunately' and 'extremely
| unpleasant' apologetic. Also, I understand there are cultural
| differences between the Netherlands and the USA- no joke.
| mnumber wrote:
| "Due to the enormous impact of the failure in the IT system, it
| is unfortunately not possible to run any trains today." I think
| the writer is more pointing the finger at IT than feeling
| apologetic. Imagine getting called an enormous failure by your
| adjacent department, that writer wants you to know who screwed
| up instead of apologize for their screw up. Seems more useful
| than an apology in my opinion.
| withinboredom wrote:
| Why would they be sorry? They're the ones who are missing
| millions? of Euros in revenue due to this issue while the
| passengers just had to take a bus or bike somewhere. They
| probably would tell you that if you had to bike, it's healthier
| that way. A little bit of rain won't make you melt.
|
| /s
| fwr wrote:
| This is very consistent with customer service in The
| Netherlands - nothing is ever the fault of service providers,
| and you manage to convince them otherwise - you'll get nothing
| more than "unfortunately (...) not possible"
| coolgoose wrote:
| I don't mind the tone, I got tired of US style politically
| correctness just for the sake of saying it.
|
| How would "We are trully sorry for this incident" make it any
| better ?
|
| I do mind the fact that they don't pay the people their money
| back if they try to find alternatives to get to their
| destination. That's 100% unacceptable.
| Vinnl wrote:
| > I do mind the fact that they don't pay the people their
| money back if they try to find alternatives to get to their
| destination.
|
| Is there any reason to assume that this is a fact? Usually in
| these situations, NS is pretty good about paying people back.
| shankr wrote:
| When is being apologetic became politically correctness? Are
| we soon going to call every decent human like behavior in
| public space as being politically correct?
| shapefrog wrote:
| > When is being apologetic became politically correctness?
|
| When it became insincere or boarderline fradulent.
|
| "We deeply regret this one incident" - you regret getting
| caught and it has happened many times before and will
| happen many times again.
|
| "We are sorry, we have learnt the lessons and it will never
| happen again" - no you havent and yes it will.
|
| "Thank you so much this is delicious" - umm try tasting the
| food before lavishing your praise.
|
| 5* uber ratings for everything - need I say anymore?
| coolgoose wrote:
| Exactly this.
|
| Actions are more important than words, otherwise it's
| just political fluff.
|
| Again they should have refunded people, not care about
| their language, they didn't so that's a big no-no.
|
| Every company is saying 'sorry', every politician is
| saying 'sorry' but there are no actual follow ups and
| ways of fixing it.
| Tepix wrote:
| The cynicism runs deep. Did you know that cynicism is
| linked to depression?
|
| " _The study was conducted on a group of 3,399
| individuals, all from London and aged between 35 and 55.
| The personality traits that were observed included
| cynicism, suspicion, resentment and distrust. After the
| study period was completed, it was found that the
| majority of individuals who expressed these particular
| personality traits that were monitored at the start of
| the study had developed depression during the 19 year-
| period._ "
|
| https://www.bphope.com/blog/how-cynicism-is-linked-to-
| depres...
| shapefrog wrote:
| > from London and aged between 35 and 55
|
| Check and check.
|
| > cynicism, suspicion, resentment and distrust
|
| Its london - if you are not suspicious and distrusting
| your phone will be removed from your possession as a you
| walk down the street. Resentment - from the council
| estates to mayfair.
|
| Cynical, I would say I am 100%, however, as per your
| article _Being cynical essentially means you are unable
| to trust the intensions of those around you_. I trust
| their intentions, I dont trust what they say their
| intentions are.
|
| The intention of the politician (or child) who gets
| caught lying is not to appologise and make amends, its to
| get out of trouble. The intention of the diner who says
| this is delicious before tasting a dish is to ingratiate
| themselves to the host or appear the "polite one" at the
| dinner party - otherwise they would find out how the food
| actually tasted before claiming it is the finest they
| have ever tasted.
|
| As for the 5* uber rating - _The bottom line is if you
| give your driver less than 5 stars, you have screwed him
| over. So, unless they fuck up, give 5 stars._
|
| Take care not to slide so far to the other side where
| there is another trait - toxic positivity ... _With toxic
| positivity, negative emotions are seen as inherently bad.
| Instead, positivity and happiness are compulsively
| pushed, and authentic human emotional experiences are
| denied, minimized, or invalidated._
| ricardobeat wrote:
| > The bottom line is if you give your driver less than 5
| stars, you have screwed him over
|
| The bottom line is that the system is messed up, isn't
| it? Same with performance reviews at big co. Why have
| reviews at all if it's only useful as a weapon.
| 323 wrote:
| In 2019 there was a power blackout in London. When power was
| restored, a number of new trains refused to start without a
| technician physically coming over with a diagnostics laptop to
| them because power fluctuated outside allowed bounds or something
| before the blackout.
| gridspy wrote:
| It's likely the same power fault could occur in a single train
| in other circumstances. When an electric train cannot access
| the grid reliably it would become far more dangerous - no more
| regenerating brakes, no reliable acceleration ...
|
| Better to force the operators to fix the boom arm / whatever
| other cause of power glitches with fault codes like this. Even
| if they false-trip when there are grid level outages.
| Symbiote wrote:
| No, it was considered a design fault with these particular
| trains, and fixed soon afterwards.
|
| Being unable to accelerate or unable to use regenerative
| braking is not in any way a safety problem. Either situation
| can happen at any time. (Electric trains have banks of
| resistors used to dump the unwanted kinetic energy when it's
| not possible to return it to the power supply, and this was
| the normal method for decades before regenerative braking
| became common.)
| iggldiggl wrote:
| > Electric trains have banks of resistors used to dump the
| unwanted kinetic energy when it's not possible to return it
| to the power supply
|
| On AC-electrified railways (especially those with their own
| independent power supply network like
| Germany/Austria/Switzerland), resistors are actually rather
| uncommon on stock capable of regenerative braking, because
| the OHLE not being able to accept regenerated power is a
| relatively rare occurrence. Though of course in that case
| the conventional friction brakes are of course fully rated
| to bring the train to halt, only with a bit more wear and
| tear.
|
| DC-electrified railways are a different matter -
| transmission losses are higher, power supply sections are
| smaller and substations often not capable of returning any
| surplus power back into higher network layers of the power
| supply systems, so the OHLE/third rail not being able to
| accept the power generated by regenerative braking happens
| much more frequently and so trains continue carrying
| braking resistors as a fallback system.
| CaptainZapp wrote:
| Reminds me of when the entire Swiss train network shut down in
| 2005[0]
|
| This was, however, not due to a computer glitch, but problems in
| SBB's electrical grid, which lead to overload of some of the
| lines and then to a complete shutdown of the entire grid.
|
| Warnings were displayed on control consoles, but were drowned in
| 1000s of other meaningless warnings (there's a lesson for an UI
| designer here).
|
| I was in the ICE from Basel to Zurich at the time and after
| fuming for a few minutes I figured: "What can you do?" and moved
| to the bar wagon. There we had something of an impromptu party
| until there was no more power for the cash register and they
| stopped sales.
|
| In the end it was a (relatively) funny adventure for most and a
| _huge_ embarrassment for SBB. I don 't think they like to be
| reminded of that faithful day.
|
| [0] https://www.swissinfo.ch/eng/swiss-train-network-shuts-
| down/...
| Scoundreller wrote:
| > There we had something of an impromptu party until there was
| no more power for the cash register and they stopped sales.
|
| Funny how the entire system has ground to a halt and the
| bartender is still worried that their cash register receipts
| and inventory won't reconcile and shuts that part down too.
| Tabular-Iceberg wrote:
| I'm puzzled by the lack of resilience in retail sales. I
| thought "always be closing" was supposed to apply to all
| types of sales. A receipt pad and a credit card imprinter
| under the counter would be a very cheap backup. It would be
| slow, but at least they would be closing.
|
| It can even be considered a national security issue in the
| case of food, medicines and other vital services.
| [deleted]
| Freak_NL wrote:
| This is Europe. Credit card imprinters are rare if they
| were ever used at all. I've never seen one.
|
| The most resilient back up (especially in the face of
| national crises) is cash of course, but a growing group of
| people have stopped carrying any cash at all, and a lot of
| people in power are doing their best to make sure a
| cashless future is nearer each day. I don't mind paying
| with a debit card; I just want to have a backup in place in
| an increasingly interconnected world where a glitch can
| take out any nationwide system.
| Tabular-Iceberg wrote:
| I'm in Europe too, and I have never seen an imprinter
| either, but I keep being issued cards with embossed
| numbers, so I figure they must be good for something, if
| not imprinting, what else?
|
| I'm concerned about the phasing out of cash too as a
| separate issue, I just don't think the cashless options
| should have to be subject to artificial limitations like
| requiring electricity or a network connection to be able
| to be used. I guess online transactions with chip and pin
| are much more secure and should be the default, but
| retailers could keep a log of outages that they are
| required to submit along with the imprinted slips. They
| could be corroborated with the logs of the utility
| companies as well as other businesses in the area, and if
| something doesn't make sense it could trigger an audit.
| GekkePrutser wrote:
| Some EU countries don't issue credit/debit cards by
| default but they have their own system. The Netherlands
| is one of them. Such cards don't have embossing. You can
| get a MasterCard/Visa compatible card but it's not
| standard, though I believe this will change soon.
|
| But of course the imprinting system is ancient and
| extremely prone to fraud. As is the magstripe. I wish we
| had a choice not to have the latter anymore. I'd gladly
| give up the ability to pay in places where magstripes are
| still used in return for more security against skimming.
| Here in Europe the magstripe is just not used anymore
| except for supporting foreign cards.
| niels8472 wrote:
| The Dutch debit cards are Maestro (Mastercard) or V Pay
| (Visa) cards which are a pretty standard systems. There
| is no point in embossing because transactions need to be
| authorized immediately by the issuing bank for obvious
| reasons.
| GekkePrutser wrote:
| They are pretty standard but not accepted where credit
| cards normally are. In particular Maestro, I'm not sure
| about V Pay as I've never had it.
|
| But when I lived in the Netherlands I had to get a credit
| card to do stuff like book flights or order on foreign
| sites.
| gpvos wrote:
| They are pretty non-standard as they don't display a
| number that can be used where credit card numbers are
| normally accepted (they do have such a number internally
| to work with the Maestro/V-Pay system).
| gpvos wrote:
| I've seen an imprinter used in Germany I think, but that
| was more than 30 years ago.
| Maken wrote:
| My lastest debit card (issued last year) was the first
| one I got without embossed numbers. I guess
| retrocompatibility is no longer a priority. Not that it
| changes anything for me, virtually every shop has already
| phased out even the magstripes and use either chip cards
| or NFCs.
| regularfry wrote:
| I like the embossing because it gives my thumb something
| to grip when I'm pulling the card out of my wallet.
| botzi2001 wrote:
| My one and only chance to use a card imprinter was on a
| train from London to York. It was the first time that I
| realized why the embossing exist
| klausjensen wrote:
| I am in Europe as well, but I am probably older than you,
| so I actually have seen the imprinters in Denmark.
|
| However, that was probably 20+ years ago. :)
| Thlom wrote:
| Same in Norway. I used to go out for beers to this one
| bar that had an imprinter when my bank account were empty
| in the early 2000s. good times.
| d1sxeyes wrote:
| UK too, which is arguably still part of Europe. I
| remember them being used at less professionally organised
| events (street parties, church outings etc.).
|
| We had one at work as well 15 years ago as part of a
| contingency strategy in case of a power outage, but don't
| remember it ever being used. I left retail so no idea if
| they still have it.
| sideshowb wrote:
| We did have them in the 1990s. But my most recent debit
| card is no longer embossed
| sofixa wrote:
| Even for a cash payment the register still needs to work
| for the payment to be... well, registered as a sale. You
| need to get a receipt with tax paid, and all that.
| Tabular-Iceberg wrote:
| That's easily done with a writing pad with carbon paper
| between each sheet. You write the items, prices and tax,
| rip off one sheet to give to the customer and another
| sheet for the books. I checked, these seem to still be
| sold at office supply places, so there is no excuse.
| tsimionescu wrote:
| Not sure about the Netherlands, but that's illegal in
| some parts. There are laws that sales MUST be recorded
| electronically.
| Tabular-Iceberg wrote:
| Cash invoices will probably still work in most
| jurisdictions, to be recorded electronically after the
| fact. It's even more work than a handwritten receipt
| since the customer would have to be identified, not just
| what is being sold and for how much, so I wouldn't fault
| a merchant for taking out a service surcharge for it.
| tazjin wrote:
| Most credit or debit cards support offline transactions,
| so a shop can keep a battery powered terminal that
| reconciles when it comes online.
|
| Imprinters don't exist in Europe to my knowledge - the
| first time I saw one was on a visit to NYC a few years
| ago and I thought they were pranking me.
| traceroute66 wrote:
| > Most credit or debit cards support offline
| transactions, so a shop can keep a battery powered
| terminal that reconciles when it comes online.
|
| Almost....
|
| Its been a while since I have been involved in POS
| payments, but IIRC there is a clear protocol which goes
| something like this: 1. If the card
| machine is online and working, use that 2. If
| the card machine is not working *AND* the business is
| willing to take on the risk, record an offline
| transaction for later reconciliation 3. If the
| card machine is not working *AND* the business *IS NOT*
| willing to take on the risk, call bank $phone_number for
| manual transaction validation and record the validation
| code provided at the end of the call.
| anticensor wrote:
| 4. If the card machine is not working and your business
| requires automated immediate finalisation of payment,
| refuse all cards altogether
| unnah wrote:
| There are a lot of different cultural practices in
| Europe, and unification is a slow process. I for one have
| seen many imprinters in use in Europe, although it has
| been years since I last saw one. An online terminal used
| to be significantly more expensive than an imprinter, so
| small shops would prefer the imprinter, especially in
| rural locations with bad connectivity.
| Symbiote wrote:
| My employer has (or had) one. They used it a couple of
| years ago when they sold some stuff to staff (old
| furniture etc).
|
| The usual fraud risks don't really apply in this case.
| Other than that, it must be over 20 years since I saw my
| parents pay using one, and that was unusual enough that I
| remember it.
| jaclaz wrote:
| I have some experience related to cash/registers/POS.
|
| Card imprinters did exist in Europe (Italy) just fine, but
| they are not used since (at least) thirty years.
|
| Most cards that are presented to the cash now have no
| embossed numbers (most people have the national payment
| circuit called Bancomat/Pagobancomat, sometimes coupled -
| on the same not-embossed card - with a Visa or Mastercard
| debit).
|
| So in 99% of cases there is no way to "extract" money from
| cards without using the card reader and if electricity or
| landline/internet connection are down, it is a problem,
| though there are a number of battery powered and
| GSM/cellular connected POS/terminals that may give a few
| hours of autonomy.
|
| Since several years sales to the public MUST be recorded on
| an electronic cash register[1], there is something like a
| "special emergency exception" where you can annotate
| manually on a paper registry sales, but as often happens
| the exact procedure is not largely known and I believe most
| people would not risk a (hefty) fine should they do any
| mistake in these cases and prefer to close the shop.
|
| [1] which is BTW connected via internet to the Agenzia
| delle Entrate, the governement agency that deals with
| taxes, and data from the cash register is transmitted daily
| to them automatically, though there is a few days time
| tolerance in case of connection issues
| ajb wrote:
| Hmm, the last time I heard of one being used in the UK I
| can track to a date in 2002. My uncle bought a commuter
| from Dan Computers the day they went bust. The credit
| readers stopped working, because the credit company
| pulled access; but the retail staff did not realise this,
| so they broke out the imprint machines and kept going.
| Presumably the receiver didn't notice the flimsies,
| because they never either finally got the payment
| processed through the banks, nor tried to collect the
| payment any other way.
|
| Originally they were the main means of taking card
| payments in the UK, though.
| avar wrote:
| > So in 99% of cases there is no way to "extract" money
| from cards without [...]
|
| The card has the customer's IBAN, which you can use along
| with their ID and have them sign a contract to allow you
| to deduct the purchase via direct debit, the system is
| routinely used for point of sale, but is mostly known for
| regular recurring billing.
|
| It's too much of a hassle and deviation from standard
| business practices for shops to use it in practice, so
| they just wait until the network is up.
|
| But it's not because there's "no way", embossed cards
| were a hassle too. We've just gotten used to less manual
| paperwork.
| jaclaz wrote:
| >The card has the customer's IBAN
|
| No, not really, most cards have an "own" number, not
| connected to the IBAN (though a few do have it), it
| depends on the bank or organization issuing the card and
| on the specific type of card, I would say (without any
| proof of it) that the ratio is maybe 80% without IBAN and
| 20% with IBAN.
|
| And pre-paid cards are also very common, without a
| connection you have no way to know if the money is there.
| avar wrote:
| Are you sure about that? I'm pretty sure IBAN is
| universal (or almost so) for all EU bank accounts at this
| point. Thus consumer debit cards have in IBAN associated
| with them.
|
| Now, it may not be the fully formatted ISO IBAN number,
| but some shorter variant you need to prefix with e.g. the
| country's IBAN prefix. E.g. in The Netherlands all Debit
| cards have the full number, but Iceland uses a suffixed
| system, those are just two systems I'm personally
| familiar with.
|
| I quickly searched for what Italy's largest bank is, and
| found pictures of their debit cards, which seem to have
| 4x4 numeric digit on the front. This page then seems to
| explain how to convert it to IBAN:
| https://help.finecobank.com/it/il-conto-fineco/conto-
| corrent...
|
| In any case, I was just using IBAN as a shorthand for "a
| unique account number suitable for a direct debit
| contract".
|
| If Italy really uses proprietary bank account numbering
| system it's probably still true that people there can pay
| e.g. their electricity via direct debit, which will
| eventually lead back to a contract where they said such-
| and-such can withdraw money from such-and-such account.
|
| The store would then print out that contract template to
| have on hand.
|
| Except that they wouldn't, because you're not going to
| train your point-of-sale people for the 10 minutes per
| year the phone system is fully down, or have accounting
| deal with the mountain of resulting paperwork and manual
| one-off payment processing.
|
| But that doesn't mean that offline payments are literally
| impossible if you have neither card nor credit card, it's
| just become so exceptional that people have forgotten
| how, even to the point of claiming it can't be done :)
| jaclaz wrote:
| Yes, I am sure, at least in Italy.
|
| Some Bancomat/Pagobancomat ones (but again not all of
| them) are connected to the IBAN (amd they sport both the
| IBAN and the name of the holder).
|
| Many Bancomat/Pagobancomat are connected to Mastercard or
| Visa (or Maestro) debit cards and they have "their own"
| number, unrelated to the IBAN (and they don't even have
| the name of the holder).
|
| Some have the name of the holder, but not the IBAN.
|
| As said it depends on the issuer, I have an account at a
| bank that issues the Bancomat card with the IBAN number
| (and my name on it and you have to wait for the card by
| mail for replacement), have another two accounts in two
| different banks where you can go to the teller, and
| he/she fishes a new card from a drawer and "couples" it
| to your account, the card is "blank", no name, nor IBAN,
| just the card number.
|
| Fineco is I believe third or fourth bank in size, the two
| largest ones should be Unicredit (which casually is the
| one from which I have the IBAN connected card) and Intesa
| san Paolo (which is one of the two I have the
| "unconnected" card from).
|
| My current experience is with some 30 payments daily, or
| 200 per week.
|
| I would say that (from memory and very roughly) what
| happens here:
|
| American Express or other "real" credit card: 2-10
|
| Other embossed numbers credit or debit cards: 25-30
|
| Bancomat/PagoBancomat (some with IBAN, many without):
| 90-100
|
| Not embossed number Visa/Mastercard (as said sometimes
| they are the same as the Bancomat ones): 40-50
|
| Prepaid or similar (not embossed numbers): 20-50
| Robin_Message wrote:
| Aside: in case that's an eggcorn and not a typo, the day would
| be "fateful", as in, something momentous happened, rather than
| "faithful", showing loyalty.
| CaptainZapp wrote:
| Thanks!
|
| Not a native English speaker. So it's easy for me to confuse
| the terms. You're right, of course.
| xiphias2 wrote:
| That software bug made a lot of Hungarians miss elections today
| in Netherlands :(
| csomar wrote:
| So you are saying it's a plot to change the election's outcome?
|
| edit: It was a joke. :/
| synicalx wrote:
| You must never joke on HN, this is a SERIOUS site where
| SERIOUS 10x people talk about SERIOUS things like....
| uhhh.... playing Doom in Grafana.
| xiphias2 wrote:
| I don't think so. Anyways the same dictatorship (Viktor
| Orban) got full control again of the country.
| mardifoufs wrote:
| Were the elections illegitimate ? I'm a complete outsider
| to hungarian politics so I'm not sure if this is
| hyperbole... but talking about how a dictatorship has been
| reelected again doesn't make sense to me. Unless elections
| in Hungary are just theather, and results are just faked
| like in Syria for example.
| Timwi wrote:
| Outright dictatorships like North Korea are getting
| rarer. It's more common nowadays for de-facto
| dictatorships to maintain an appearance of democracy by
| actually holding elections but either having no viable
| candidates other than the incumbent (e.g. Russia), or by
| systematically banning or even murdering all opposition
| (e.g. Belarus). Hungary isn't quite that far gone yet. It
| still has elections that technically meet the
| requirements of EU membership. I don't know any more
| details though.
| boomboomsubban wrote:
| It's a relatively free election, there's a strong
| incumbent bias that undoubtedly helps him win but there's
| no evidence the entire thing is rigged. Dictators can be
| popular, the best example is Napoleon.
| xiphias2 wrote:
| Each opposition party gets 5 minutes per 4 year election
| cycle by the state controlled media (and a lot of other
| media was bought from stolen EU money).
|
| I dare you to watch 5 minutes of news on the government
| media channel (the only channels that can be accessed in
| the free TV tear, that poor people are able to watch) and
| say that it's not rigged.
|
| 4 year ago the joke party used that 5 minutes for being a
| chicken to show the absurdity of the situation:
| https://www.youtube.com/watch?v=ZcpuwIl-610&t=28s
|
| If Napoleon, who ruled hundreds of years ago is the best
| example you know, there must be something in not allowing
| any person to stay for 16 years in power (especially
| because he just had 9 years)
| boomboomsubban wrote:
| All your saying can be described as a "strong incumbent
| bias that undoubtedly helps him win." International poll
| watchers agree, the elections themselves are fair but the
| blurred separation of party and state leads to an undue
| advantage. 2014, but I don't see evidence of the
| situation having changed. http://www.osce.org/odihr/elect
| ions/hungary/121098?download=...
|
| Maybe I should have said "fully rigged." Real opposition
| candidates do get to run, they just have almost no shot
| of victory. And it's possible that if they did have a
| shot, the elections would be outright rigged.
|
| Napoleon was my example of a popular dictator, I didn't
| want to use other examples as people will debate whether
| they're dictators. I never said that's a good thing,
| Napoleon was horrible despite remaining popular in
| France.
| randomchars wrote:
| > I dare you to watch 5 minutes of news on the government
| media channel (the only channels that can be accessed in
| the free TV tear, that poor people are able to watch) and
| say that it's not rigged.
|
| This is not true. While most of the channels are
| government media, you also get RTL, which is very much
| opposition media. Source: https://mindigtv.hu/muszaki-
| segitseg-1/csatornak/csatornakio...
| xiphias2 wrote:
| Thanks, I didn't know that.
| usr1106 wrote:
| The government made extra payments to their typical
| supporters like pensioners and rural areas. Economically
| strong regions and younger professionals pay the bill and
| are free to vote against, but they remain a minority.
| adhesive_wombat wrote:
| Yes, but the question was about Hungary, not the UK.
|
| Though since there's no election for a while and they
| don't apparently get elected based on promises delivered,
| just promised made, they haven't shied away from breaking
| the pensioner-bait promises anyway for this year and
| removing the triple lock (pensions rise at least as fast
| as the largest of inflation, earnings or 2.5%).
| shapefrog wrote:
| > they haven't shied away from breaking -- promises
| anyway for this year
|
| Non-election year actions, while unironically promising
| that they are going to give away money in the upcoming
| election year.
| orblivion wrote:
| For those outside of Europe, could you explain how this works?
| Are these Hungarian citizens living abroad who are able to
| somehow vote from polling places accessible by train within the
| Netherlands? Is this an EU organized thing?
| xiphias2 wrote:
| Only in Amsterdam inside Netherlands after waiting a long
| line. At the same time Hungarians in Romania could vote by
| mail and the government was using Fidesz to collect the votes
| and threw out votes of the opposition.
|
| The soft dictatorship works very well, Hungary will go even
| more down in the future. I'll probably leave Hungary before
| Hungary leaves EU/NATO and gets into a war.
| throwawaylinux wrote:
| Voting by mail is insecure? This is very troubling.
| xiphias2 wrote:
| It's not simply voting by mail, it's voting by mail by
| not even using official the post office in Romania.
|
| I know that you're probably refering to US, but I don't
| make any comment about it, because I don't live there, so
| I don't know what really happened in US.
|
| In Romania some people found partially burned ballots
| with only opposition votes next to a garbage bin before
| the elections ended. The proofs of voter fraud couldn't
| have been clearer. It's supposed to be a scandal, but
| nothing will happen about it.
|
| https://hungarytoday.hu/discarded-burnt-postal-ballots-
| scand...
| throwawaylinux wrote:
| Well that is just an isolated case. There is simply no
| evidence to support the baseless conspiracy theory that
| postal voting results in widespread fraud. This has been
| well established by the experts and journalists.
| rendall wrote:
| > _There is simply no evidence to support the baseless
| conspiracy theory that postal voting results in
| widespread fraud._
|
| You're applying US culture-war stuff to another country
| and culture.
| bigDinosaur wrote:
| Clearly the quality of the audit matters and that differs
| massively between countries. Nobody disputes the validity
| and integrity of postal voting in Australia for example.
| aetherson wrote:
| Who exactly would Hungary get into a war with? It borders
| Slovakia (EU member), Austria (EU member), Croatia (EU
| member), Serbia, Romania (EU member), Ukraine, and Poland
| (EU member). Is it your contention that it's going to
| invade the EU? Or Serbia? Or Ukraine? Or is Hungary going
| to use its awesome force projection capabilities to get
| into a war with a country that it doesn't border?
| mastermedo wrote:
| "Beat them with sarcasm, not arguments" -- James Looney
| detaro wrote:
| They need to travel to the embassy to vote.
| flak48 wrote:
| Most countries (even non EU ones) allow its citizens residing
| in foreign countries to vote in national elections at its
| embassies.
|
| So Hungarians residing in Netherlands could have been able to
| vote at Hungarian embassies/consulates in Netherlands (if
| they made it to the embassy).
| snorkledoodle wrote:
| Sometimes even local stuff. U.S. states, our provincial
| government level, have processes in place to account for
| Americans living abroad to vote in state and local
| elections. It's guaranteed by law, I think. Our states
| handle all voting on behalf of the federal government
| anyway (registration, voting day operations, counts, all
| often under the misleadingly named office of a Secretary of
| State), so it's a natural consequence of our structure. It
| is nice to live abroad as if you were still at your last
| physical address for voting purposes, though, and usually
| not notice a big difference. I think it's implied that
| those of us who have done it intend to come back, in the
| system's thinking, but I think everyone understands the
| relevance of their vote according to their situation and
| beliefs.
|
| We're of course all too familiar with this process from
| several consequential court cases deliberating its efficacy
| and merit in generational memory. Whether it's sensible has
| been argued with elections on the line.
| bourgeoismedia wrote:
| A shocking thing for me is that if you're an American
| living abroad registered to vote in California you're
| allowed to vote by email (well technically fax, but
| there's a DoD email alias you can send your scanned
| ballot PDF to and someone will print it out and fax it,
| which counts). Oh, and if you vote any ballot this way
| other than a write-in for federal offices you might also
| get a residency audit and have to pay California's
| personal income tax (which doesn't recognize any US tax
| treaties). You're not allowed to vote by fax or email if
| you don't have a non-US address.
| withinboredom wrote:
| In SC, you vote in a web app if you live out of the
| country.
| [deleted]
| miggol wrote:
| My curiosity is killing me as to what exactly went wrong. And as
| someone living in The Netherlands I'm also kind of mad at the
| apparent fragility of this huge chunk of our infrastructure. If
| this had happened on a work day, it would have been a real
| national emergency, rather than just a huge national
| inconvenience.
|
| I would appreciate one of those post-outage "what happened"
| reports like with the AWS and Facebook outages last year. But
| outside of IT I don't think anyone really expects those over
| here. And there might be national security considerations
| preventing such disclosure until any chance of a repeat has been
| engineered away, at which point everyone will have forgotten.
| quartz wrote:
| From the notice:
|
| > It affected the system that generates up-to-date schedules
| for trains and staff.
|
| ...boy oh boy did trying to look into what NS uses for crew
| scheduling ever send me down a rabbit hole.
|
| I don't know if the systems have changed but while poking
| around online I found this[1] doc about the Netherlands'
| timetable revamp around 2006 and it talks about the complexity
| of TURNI-- their on-the-fly crew scheduling system.
|
| > A typical workday at NS includes approximately 15,000 trips
| for drivers and 18,000 for conductors. The resulting number of
| duties is approximately 1,000 for drivers and 1,300 for
| conductors. This leads to extremely difficult crew scheduling
| instances. Nevertheless, because of the highly sophisticated
| applied algorithms, TURNI solves these cases in 24hours of
| computing time on a personal computer. Therefore, we can
| construct all crew schedules for all days of the week within
| just a few days.
|
| Then I found more detail about TURNI's implementation in
| this[2] paper about optimizing crew scheduling for timetables.
|
| > In the railway industry the sizes of the crew scheduling
| instances are, in general, a magnitude larger than in the
| airline industry. Moreover, crew can be relieved during the
| drive of a train resulting in much more trips per duty than
| typical in airlines. In other words, the combinatorial
| explosion is much higher. The latter has made the application
| of these models in the railway industry prohibitive until
| recently.
|
| Cool stuff.
|
| Finally, gleaning from ns.nl's careers page[3] everything else
| in their IT land outside this system runs off SAP (likely
| including the actual distribution of the output of crew
| scheduling) so if I had to gamble I'd say the failure happened
| somewhere in the integration between them.
|
| [1] https://homepages.cwi.nl/~lex/files/Interfaces.pdf
|
| [2] https://repub.eur.nl/pub/11701/ei200803.pdf
|
| [3] https://werkenbijns.nl/werkgebieden/it/sap-specialist-bij-
| ns...
|
| sidenote: If anyone out there is an SAP specialist ns.nl looks
| like a pretty great place to work: 36hr week, 5 weeks vacation,
| pension, and free unlimited 2nd class + low cost 1st class
| train travel.
| sofixa wrote:
| > 36hr week, 5 weeks vacation, pension
|
| 1 hour more than the standard French contract, and there are
| also far more holidays here..
|
| > free unlimited 2nd class + low cost 1st class train travel.
|
| .. okay that's actually a great perk. IIRC the French
| national railway company, SNCF, provide only a limited number
| of tickets per employee (like 10 per year? someone please
| correct me if I'm wrong). Their UX is also atrocious though
| (websites are just shit at workflows, apps are weird and
| break easily, ticket machines are OK but with really bad
| touch screens, and train announcement screens are each
| running Windows XP(with literally hundreds, if not thousands
| of them on big train stations), while from what i recall NS
| was acceptable.
| thibautg wrote:
| I work for the Belgian railways and I have a free unlimited
| first class pass for Benelux (Belgium, Netherlands and
| Luxembourg) + lots of international free tickets, also in
| 1st class. But no company car, obviously.
|
| There is no UX: it's just a small sheet of paper.
| HowTheStoryEnds wrote:
| The workers don't get that though, they get 2nd class.
| rgj wrote:
| > sidenote: If anyone out there is an SAP specialist ns.nl
| looks like a pretty great place to work: 36hr week, 5 weeks
| vacation, pension, and free unlimited 2nd class + low cost
| 1st class train travel.
|
| Those are pretty standard terms here in .NL. 36 hours is
| considered fulltime by most employers, including government
| and semi-government. Pension is offered almost everywhere and
| at least 4 weeks vacation is the legal minimum.
|
| Maybe except for the _unlimited_ train travel but most
| employers do offer free train travel between home and work.
|
| This also means that if an IT system goes down on a Sunday, a
| lot of employees won't even pick up the phone until Monday 9
| AM.
| rocqua wrote:
| Note that a pension is essentially a legal requirement.
| There might be ways around this, but it would be really
| hard to offer employment without offering a pension in NL.
| consp wrote:
| Only AOW (state pension) is a requirement by law and for
| full-time salaried employees there is no way around that.
| Private pension (what is meant here) is not required by
| law though financially very lucrative compared to doing
| it yourself. It can be made required by making a CAO
| (collective bargaining agreement) "algemeen bindend" for
| all employees in a sector/company but this is not always
| the case in all sectors (e.g. IT). Note that this is way
| more complex and this is just a simplification.
| rocqua wrote:
| Really, I thought pensions were required. I guess it's
| just so fiscally attractive that essentially every
| employer offers it. Combined with the pervasiveness of
| CAOs, especially among the more stingy sectors.
| roeles wrote:
| I worked about a year at NS as a consultant. I wrote python
| software that broadcasts the audio messages in the
| passenger compartment, steers the outside lcd screens and
| such.
|
| It is a great place to work. A nice atmosphere, good coffee
| and they embraced change (in this case Scaled Agile) better
| than most places I have seen. They are often listed as
| top-5 best places to work and I can see why.
| t0mas88 wrote:
| Indeed pretty standard, even in non-government it would be
| 40 hours but all other terms the same and most likely a
| company car + private use of it instead of the train.
|
| And I'm also sure they have a significant on-call setup of
| the IT teams for any out of office hours issues, just like
| every other large 24x7 kind of organization.
| rgj wrote:
| Of course they do. But the real specialists will most
| likely have been able to keep themselves out of the on
| call schedule.
| ChuckNorris89 wrote:
| _> 36 hours is considered fulltime by most employers_
|
| How standard is that in the private sector or non former
| state owned enterprises? I haven't looked at the NL market
| in a while but most tech jobs I saw back then were
| 40h/week.
| fjfaase wrote:
| I have been working between 30 and 24 hours as a Senior
| Software Developer in the past 20 years. I have had
| several colleagues who work a similar amount of hours. I
| have been telling employers that I am only productive for
| about six hours per workng day on average and that never
| has been a deal breaker. The number of productive hours
| does vary from day to day, but I know that I push myself
| beyond that limit, I will run in trouble after some
| months. I have heard about studies that the average
| employee is only really productive for about two or three
| hours per day. With me working six hours per day, I might
| just as productive as someone being eight hours in the
| office.
| AndyMcConachie wrote:
| Most private sector tech jobs in NL are 40 hours per
| week. But many private sector jobs are also flexible in
| ways that I've never seen in the USA. For example,
| allowing an employee to work 40 hours in 4 days instead
| of 5.
| dtech wrote:
| 40 hours in 4 days - 4x10 - is unusual, but 4x9 for 90%
| part-time is common.
| rgj wrote:
| It's hard to find actual numbers. Tech jobs, smaller
| companies ("MKB") and work in the agricultural sector
| tend to have a higher percentage of 40 hour work weeks
| than other sectors.
|
| Only 3% of the people in the Netherlands work 41 hours
| per week or more. The average Dutchman works 31 hours per
| week. (source https://www.cbs.nl/nl-
| nl/nieuws/2020/08/meer-dan-de-helft-we...)
| yourusername wrote:
| Not very standard but if you want to work 36 or 32 hours
| that is possible almost everywhere (you'll just get
| 10-20% less pay).
| lultimouomo wrote:
| > sidenote: If anyone out there is an SAP specialist ns.nl
| looks like a pretty great place to work: 36hr week, 5 weeks
| vacation, pension, and free unlimited 2nd class + low cost
| 1st class train travel.
|
| And we can assume that a position in the IT department just
| opened up...
| sofixa wrote:
| Blameless culture! Firing people for (one-off) mistakes or
| issues is much more a thing in the US than in most EU
| countries.
| gpvos wrote:
| tbh a few similar outages have happened in recent years,
| they do seem to have a problem and maybe someone does
| need to be fired.
| tannhaeuser wrote:
| I have zero knowledge of how the Dutch railway scheduling
| system works, nor of the nature of this failure, but SAP has
| been blogging recently about using Prolog for combinatorical
| optimization tasks such as the one that may be in use here or
| in SAP's Advanced Planning [1]. There are also well-known
| Prolog proponents from NL.
|
| [1]: https://blogs.sap.com/2022/03/11/what-you-may-not-know-
| of-pr...
| samstave wrote:
| I love deep diving on public info to determine the internals
| (even, not only SOPs, policies, but also *governing culture*
| can be easily gleaned from reading what is placed in public
| view.
|
| For example, The Salvation Army is one of the biggest jokes
| in all of charitable history.
|
| I wont go into details here, but I would NEVER give a dime to
| the Salvation "army" (of free slave labor under the guise of
| "salvation" (its a tax and money fraud to its core (I went
| through a SHIT ton of financial documents to see what their
| true operatin is disguised as. AVOID)
|
| TL;DR: TSA uses its place as a church-like service to get
| special tax incentives, even the buildings in SF were donated
| to them. They run a for-profit-grey-market of boutiques they
| supply with "donations" they have running trades with China-
| Town in SF. They take subsidies, donations and other
| benefits. They require all that they "help" sign up for state
| benefits, sign the benefits over to their org, take the
| benefits, then feed donated & expired processed fod to their
| labor force.
|
| They only take in able bodied. Force them to go to church
| each sunday. and make them use external addiction help, such
| as AA and make it clear they are not a treatment organiztion.
|
| They maintain a very small amount of actual employees, and
| play on the egos of those in their "program" using military
| style rankings to appease the egos of the ignorant...
|
| They then profit like mad, corruption is rampant and they
| feed all their "good" donatins to their side hustle of
| boutiques and other grey market.
|
| Its a fn racket.
|
| =---
|
| The point being that if you DD a company/org, you can often
| find out a lot from financial public info, comments from even
| just a few employees etc..
| nicolas_t wrote:
| Please go into details, I always think it's important to
| call out charities that are fraudulent and going into
| details helps with detecting if other charities are
| fraudulent or not.
|
| What lead you to look into this? How did you originally
| suspect them of being fraudulent? What clues did you see in
| their financial documents?
|
| For example, I tend to look at Charity Navigator and for
| Salvation Army, they mostly get dinged by not having an
| independent board (which to be fair is a clear red flag)
| but not much more, so as someone who does want any
| contribution I make to charities to go toward their cause,
| I'm very interested in any methodology around researching a
| charity.
| wolverine876 wrote:
| Do you have any evidence of these claims? A news article,
| for example?
|
| I knew someone who worked briefly at the Salvation Army.
| They said it was shockingly efficient and effective at its
| charitable mission.
| incompatible wrote:
| If they are scheduling days in advance, then why would a
| failure of the scheduling system cause an instant outage?
| anticensor wrote:
| Why not fall back to old-school time and distance interval
| running?
| jonny_eh wrote:
| Maybe a mistake was detected and they needed to
| recalculate?
| tyingq wrote:
| It said for "schedules for trains and staff". Crew
| scheduling is complicated and changes constantly.
| naikrovek wrote:
| because accidents and delays happen anytime, and if the
| schedule can't adapt, you can't run the trains.
|
| people seem to believe that running trains is some simple
| thing. it is the exact opposite of a simple thing, even for
| small systems with only a few dozen trains
| xyst wrote:
| I have seen "SAP" tossed around in this industry and still
| have no clue what they do to earn money. Are they the
| equivalence of "salesforce"?
| liamwire wrote:
| In a word: ERP.
| cerved wrote:
| think Uracle
| drumhead wrote:
| The OG ERP
| marban wrote:
| You might look up the running gag for the german
| abbreviation.
| acomjean wrote:
| Basically SAP is a large database/software package that
| runs all aspects of your business. From accounting to
| billing. Your business probably needs to change a lot if
| you're going to use it. When I did a time card application
| for a big company we interfaced our app with a backend some
| sap engineers made available to us.
|
| I have no idea how it works, but apparently it does..
| sometimes...
|
| https://www.computerweekly.com/news/252435884/Chances-of-
| suc...
| chii wrote:
| > that runs all aspects of your business.
|
| i think this is under-emphasized for people unfamiliar
| with them.
|
| I think SAP and related modules are "solidified business
| processes", and their consultants and sales persuade your
| CxO level people that switching business processes will
| improve overall efficiency (and somewhat hint at
| standardization, so that the CxO will have experience at
| that domain when switching jobs!).
|
| But of course, it's not really true - best practice
| business processes are an illusion imho, and it will turn
| out that the subtle differences have to be implemented
| individually.
| AdrianB1 wrote:
| SAP is mainly an ERP system, but it has various modules
| dealing with stuff from Purchase Orders and Materials
| Management to HR. It can be very large and extremely
| expensive, like over $1 billion for one company in ~ 20
| years. It can be customized in many ways, good consultants
| are very expensive and the cheap ones are usually a waste
| of money. Without customization very few companies can map
| their processes on out of the box SAP.
|
| It was developed a very long time ago, it has a huge
| adoption in large companies and a few years ago when they
| migrated from R3 to Hana one of their own people told us in
| an architecture review meeting that they have over 300,000
| built in reports to migrate.
|
| In many cases companies are adopting SAP because "nobody
| was ever fired for buying IBM". Anecdotical, the company
| where my brother works is implementing SAP and changed all
| existing networking gear with Cisco because "this is what
| big companies do". They threw away brand new equipment and
| spent several millions on the SAP implementation that is
| still not working, forecasting they will pay even more
| millions to make it work then millions per year to use it.
| samstave wrote:
| Think Oracle, with business logic on top. (JDEdwards,
| Oracle, SAP) were the OGs of BI.
|
| (and bloat, and McKinsey infiltration into many a large
| F100+)
|
| --
|
| They are old school, entrenched, but yu would jump off a
| cliff supporting any of this legacy stuff. It will persist
| as the inertia on huge systems is heavy.
|
| But yeah - I wonder if they are signing any NEW business...
| aside from military bunker grade stuff (which will cost as
| much as the bunker to implement) I dont see a reason to
| implement (not because I am bashing, but because I dont
| know the modern business case reqs that would call for SAP)
| rendall wrote:
| > _McKinsey infiltration into..._
|
| What is _McKinsey infiltration_?
| n3k5 wrote:
| I don't know about "McKinsey infiltration into many a
| large F100+" first-hand, but from context I'd guess it's
| simply getting consultants from McKinsey & Company[0]
| jobs in the 'infiltrated' firm, on the basis that it has
| no in-house employees capable of properly setting up /
| running the bizarrely complex and idiosyncratic SCM-ERP-
| HCM-BI-etc. software in question.
|
| Basically: You have a problem; Oracle/SAP says they'll
| solve it for you. Now you have a second problem; they say
| McKinsey'll solve it for you. Now you have ...
|
| [0] https://mckinsey.com
| samstave wrote:
| Its the cancer that permeates your C-Suite. Meta-stasize-
| ing,
|
| God I love etymology:
|
| "-Stasize" **To stand on a hilltop
| once. To be, this time, on advance. To reach the highest
| clouds. To touch the limits of space. To out-grow
| skyscrapers. And make them look at my face.**
|
| ---
|
| So FB is becoming a meta-stacious being..
|
| That which wishes to control from a vantage, or t corrupt
| from within the minds... The light house.
| sveme wrote:
| Google switched to SAP two years or so ago.
| cerved wrote:
| switched what?
| RedNifre wrote:
| The company.
| anticensor wrote:
| I would have expected something like ABC CRM, tailored to
| Google.
| belter wrote:
| Imagine an Octopus like Goldman Sachs but for Enterprise
| Software.
| lmpdev wrote:
| SAP are more known for ERP than CRM
|
| They're absolutely colossal in size
|
| My ex-girlfiend's father is an engineer there in Baden-
| Wurttemberg - I'd characterise them as closer to something
| like Oracle rather than Salesforce
| nine_k wrote:
| They are so colossal that switching to SAP usually means
| reworking company's processes to match SAP's, not the
| other way around.
| tonyedgecombe wrote:
| One of my customers was taken over during their SAP
| implementation. The general feeling among management was
| the work had weakened them so much it made them
| vulnerable to takeover.
| shdon wrote:
| This was true even for Royal Dutch Shell, one of the
| largest companies in the world. A few years before he
| retired, my dad led a team responsible for coordinating
| implementation of SAP and for training thousands or even
| tens of thousands of employees to work with the new
| system.
| GekkePrutser wrote:
| To be fair, that is the case for most ERPs. It's not
| really about the size of the product but rather the
| important role it plays in a company.
|
| Most ERPs can be quite flexible too, there's a ton of
| customisation possible. And a whole cottage industry of
| addons and integrations for every imaginable integration
| or business model. But the problem is you really screw
| yourself with added complexity when it's time to update.
| Then you're really condemning yourself to spending
| millions on consultants at a time when you might not have
| the resources. And postponement leads to technical debt.
|
| Hence the idea of adapting the business to the system.
| Making sense too if you think about it this way: you're
| not really just buying a system but a methodology.
|
| And another thing is that it makes integration with other
| companies easier. Makes the company more easy to sell or
| too acquire other companies and integrate them.
| kovacs_ wrote:
| Heh, you described my work situation to a T.
|
| They decided to modify a number of core functions to work
| the same way as their old terminal system instead of the
| default ERP process. Now our job for the next couple
| years is decoupling the custom add-ons so we can safely
| update to a modern ERP system.
| manmal wrote:
| I read recently that not using SAP - or heavily
| customizing it - in markets where it's prevalent can
| break big(ish) companies. Apparently the interface to
| other companies (from which to buy or sell) is so much
| easier to setup when all have the same system, that using
| any other system is prohibitive. If you need to interface
| with hundreds of enterprises that all implement standard
| SAP, you become the expensive outsider if you don't.
| Integration projects can take years and then fail, with
| nothing to show for - which can threaten a company's
| existence. Better to use standard SAP ,,off the shelf"
| and do as little customization as possible.
| regularfry wrote:
| Without necessarily wishing to call into question your
| source on this particular occasion, it is of course in
| SAP's interest that as many as possible believe this to
| be true.
| ant6n wrote:
| > I'd characterise them [SAP] as closer to something like
| Oracle rather than Salesforce
|
| They feel more like IBM, but more crusty, more
| conservative and more corporate.
|
| Working for a railway on anything touching SAP sounds
| like an absolute nightmare of wasting decades threading
| water with nothing to show for it.
|
| (I saw a bit of SAP inside DB... nope nope nope)
| miggol wrote:
| Thank you for your informative reply! These are such good
| reads
| MetaWhirledPeas wrote:
| > everything else in their IT land outside this system runs
| off SAP (likely including the actual distribution of the
| output of crew scheduling) so if I had to gamble I'd say the
| failure happened somewhere in the integration between them.
|
| I have a reflex repulsion for off-the-shelf "enterprise"
| software. You pay millions of dollars to subscribe to a suite
| of (seemingly) bad, bloated software that takes more work to
| customize and integrate than it would to fully replace. An
| army of developers trying to keep a business running with the
| world's largest Swiss Army knife.
|
| My gut tells me that big names like SAP and Sales Force are
| easier to sell to executives, and it makes a company seem
| like it knows what it's doing. And maybe feeling pressure to
| choose a "respected" brand leads to fewer alternatives? Or
| maybe all the alternatives are quickly bought out by the
| behemoths. I really don't know. Maybe someone can enlighten
| me. It just seems like an enormous turd of inefficiency to
| me.
| ArnoVW wrote:
| Have done consulting for 15 years, understand your
| feelings. But I'd like to offer some alternative
| viewpoints.
|
| Buying an off the shelve package also buys you an industry-
| standard business process. The idea is that part of value
| comes from the process that you have to shoe-horn into.
| Some integration is expected (and provided for) but if you
| have to modify heavily, You're Not Using It Correctly.
|
| Also, using an OTS package means less development risk (not
| every IT dept has 10x developers coming out the wazoo) and
| it also means that you have a huge reservoir of skilled
| labour if you need it. Not just for development but also on
| the business side.
|
| Finally, if you're a big, publicly owned company, you
| likely have many financial / reporting constraints. The
| scope of your non-negotiable requirements may be bigger
| than you imagine.
|
| I've seen people implement SAP in a couple of months. It's
| possible. It's still expensive and clunky though :-)
| regularfry wrote:
| Industry standards shouldn't be single-vendor though.
| That's what leads to expense and clunkiness.
| eatingcrowe wrote:
| > I've seen people implement SAP in a couple of months.
| It's possible. It's still expensive and clunky though :-)
|
| Never worked with SAP, but still coming off my burn-out
| hangover from $OLD_JOB - worked to migrate the nexus of
| the firm's accounting systems from COBOL to MS Dynamics.
| Unto itself, I don't really have a problem with Dynamics
| - I do have a problem with MS and their piss-poor
| available materials on the topic. While I understand
| their business sells training and certifications, this
| does ass all for folks pulled in as a pinch hitter during
| a death march. Can scarcely fly out to Las Vegas for a
| week long vacation while I'm working 16 hour days to beat
| the clock on our last true COBOL coder's retirement date.
|
| The greater issue I have in enterprise systems - often
| the consultant / contractor firms[0] involved in the
| process. The classic fly-out of classy sales-engineers
| replaced with low-rent off-shore coders for the actual
| work. Slick-as-shit documentation that does not match
| reality in even the simplest of cases. Counter-Party
| contacts that neither care nor hide their lack of caring
| for the outcome of the project. Pressing for aggressive
| timelines that do not meet reality
|
| I would assume there is some correlation, however,
| between buying $ENTERPRISE_TOOL and an unwillingness to
| hold consultants feet to the fire on their own failings,
| structuring a contract that keeps them honest.
|
| [0] - Not to state `ArnoVW is one, only stating a pattern
| I've run into all too often.
| ArnoVW wrote:
| Thanks for the footnote =) And I concur. Outsourcing your
| IT replaces one type of work with another. If you can't
| manage your externalized projects, you're gonna get eaten
| alive by IT services suppliers.
|
| In fact my line of work was managing 'complicated IT
| projects', generally involving formalizing things and
| chasing said suppliers (ie they outsourced the
| outsourcing)
| belter wrote:
| Implement SAP only in a couple of months? Surely an Agile
| project...
| ArnoVW wrote:
| It was a young company and they only had the bare minimum
| in modules (accounting, HR)
| madaxe_again wrote:
| I've never seen a SAP install that wasn't modified
| heavily, usually to the tune of millions of dollars of
| development - which can't help but make me think that if
| _nobody_ can use it "correctly", then perhaps it _can't_
| be used in its unmodified form.
|
| In addition, a common occurrence seems to be businesses
| paying for extensive custom developments (such as an
| integration with a third party system), and footing the
| entire development cost - despite their "custom"
| development being 100% identical to that deployed at
| other businesses. They're then trapped in a world where
| changing the URI for an API endpoint costs EUR150k.
|
| Honestly, I see it as just being a big corrupt hole
| fuelled by interpersonal relationships rather than any
| actual rationale.
| PennRobotics wrote:
| You see, when your Foxpro guy retires in 10 years, either
| you've been training your new hires how to program Foxpro
| (a waste of time) or you'll pay a specialist to move all
| your business data and operation macros to a new platform
| (a waste of money).
|
| SAP lets you stop kicking that can down the road by
| wasting your time and money today.
| aliswe wrote:
| It's interesting that the word "bespoke" is derogatory in
| SAP land.
| resonious wrote:
| My previous boss recently tried to pick up an enterprise
| solution. He runs a non-software company that has a lot of
| custom internal apps - 90% built/maintained by interns. Of
| course the intern-built custom apps are super buggy but the
| bugs get fixed immediately, and the system is tailored to
| the company's way of doing things. The company spent an
| obscene amount of money just demoing a full-fledged SAP-
| type thing to potentially replace the custom stuff, but
| they ultimately found the rag-tag web apps better and
| cancelled the SAP contract.
|
| That whole situation convinced me pretty thoroughly that
| "enterprise" software is indeed a form of large-scale snake
| oil. I think most businesses can go a long way with just
| spreadsheets and email.
| sebmellen wrote:
| This is one of the reasons I'm bullish on no-code doc-
| apps like Coda [0]. I recently started consulting with
| someone who calls himself "The Coda Guy" and pulls
| together truly wizard-like apps out of Coda [1]. (Coda is
| effectively a hybrid of Excel, Google Docs, and Notion.)
|
| I think the "Excel+" category should receive a lot more
| attention than it does now. I can easily see "no-code"
| tools like Coda taking over Salesforce and effectively
| filling the niche you describe.
|
| [0]: https://coda.io
|
| [1]: https://coda.io/@simpladocs/cycling-specific-
| template / https://coda.io/@simpladocs/crm-interaction-
| tracker
| kwhitefoot wrote:
| Agreed, except for the spreadsheets. Please get people to
| use server based databases instead. They are no longer
| difficult or expensive to install or use and make life so
| much better even for 'interns' and it will reduce their
| workload dramatically as well as increasing the quality
| of the solution..
| theshrike79 wrote:
| First rule of SAP integration:
|
| You do not modify SAP to fit your company, you modify
| your company to fit the SAP model.
|
| You can try it the other way, but the project will always
| cost millions and it will eventually fail. It always has
| and it always will.
| shapefrog wrote:
| Nothing gets me as excited as a lowly footsoldier in a
| company as when the c-suite decide to spend $100mil doing
| a custom install of a big name software stack that will
| "totally change the way we work and at the same time be
| fully customised to the way we work here".
| sjamaan wrote:
| > My gut tells me that big names like SAP and Sales Force
| are easier to sell to executives.
|
| I've heard it said that SAP is sold on the golf course, not
| in the IT boardroom.
| belter wrote:
| Nowadays it's more business trips to Prague...
| [deleted]
| weq wrote:
| I worked in defence on systems that did similiar and i
| remmember the one my company inherited and wow and wow it was
| the most complex, nested peice of SQL stored procs i have
| ever witnessed. It did the job, but it took HOURS to run. We
| had to re-write it completely to make any changes. But thats
| what happens when you let companies like Accenture at a
| system of yours! You pay them the big bux to be sueable, and
| they milk you like a cow.
| quartz wrote:
| A lifetime ago I did a stint as an Oracle consultant and
| 100% this. A gigantic sql query was always the answer no
| matter the question. I remember using sql queries to
| generate pdf and ps file output that would stream directly
| to printer feeds. So many "concat"s in that massive blob
| and nearly impossible to maintain for the next poor soul
| who had to own the code.
|
| Learned a lot about db best practices to make these systems
| resilient and reportable but the whole "the database is the
| system" angle was very much a "every problem is a nail to
| this hammer" situation.
| jokoon wrote:
| > And there might be national security considerations
|
| That especially. I wonder how expensive that thing is going to
| cost.
| gonzo41 wrote:
| I bet their schedule once ran on a commodore 64, and then they
| upgraded to a 386, and then ignored the IT department for 30
| years. Hence today, chickens are coming home to roost.
| postingposts wrote:
| My _very good guess_ is that you had a database failure caused
| by a crypto virus. They will not wish to announce that this was
| the root cause until A. A plan of action to avoid a future
| infection can be enumerated. B. They can be certain they will
| restore the functionality of the trains.
|
| The other part of my guess is that they have a SQL db which is
| stored on /likely/ a windows server instance. I'd even surmise
| that this instance may be hosted on Azure but that's
| speculation, not a _good guess_.
| gpvos wrote:
| Do you have _any_ way of backing up that guess related to NS,
| or are you just wildly speculating?
| slim wrote:
| My guess for the attacker : conti ransomware group. They've
| been pretty active since Ukraine war. Last weekend they
| targeted the central bank of Tunisia
| jollybean wrote:
| What would lead us to be inclined to believe that this is a
| 'very good guess'?
| glitchcrab wrote:
| People are very good at putting 2 and 2 together and
| getting 655.
| lucb1e wrote:
| This is one of my leading theories as well, although I
| thought they usually hit on Friday night in order to catch
| admins off-guard and encrypt as much as possible before being
| stopped? "The IT failure occurred at the end of [Sunday]
| morning".
|
| On Sunday there is also not as much pressure to get things
| running as there would be on Monday. Or perhaps this is the
| incentive to pay now and get things fixed on time for the
| work week? In that case Friday night would again have made
| more sense unless the attackers have some very specific
| insight into how much slower restoring without paying is.
|
| My alternative theory is an expired certificate that makes
| some core systems just not talk to each other anymore. The
| announcements on the stations, for example, were also out,
| and they lost control of the mobile apps (couldn't make the
| apps show that trains didn't run, the in-app scheduler showed
| all was A-OK), and that sounds quite dissimilar from the core
| train operation service, making me think it's more of an
| infrastructure than a specific system's problem. On the other
| hand, once the app could be controlled again (assuming this
| singular underlying cause), you'd think they could then also
| start putting train service back in place and that didn't
| happen for hours still.
|
| I can't really make the pieces fit together for any theory,
| so then presumably something multi-faceted (one thing
| tripping one or two other things so it escalated from
| restarting one component to not being able to restart the
| trains anymore the whole day).
| withinboredom wrote:
| Certificates make the most sense to me. The delay can
| possibly be due to cached certificates, having to find the
| person to sign the CSR (assuming in-house PKI or even just
| someone with access to the right email address for external
| PKI), and/or a CA that isn't valid on the machines.
| rocqua wrote:
| > On the other hand, once the app could be controlled again
| (assuming this singular underlying cause), you'd think they
| could then also start putting train service back in place
| and that didn't happen for hours still.
|
| Before you can get the actual service up and running, you
| need to get trains and personnel to the right locations. So
| I don't think this discredits your theory as much as you
| think.
| postingposts wrote:
| It takes time to roll the backups out and define the level
| of affected systems before doing so. That's your time
| window.
| skeeter2020 wrote:
| >> but that's speculation, not a good guess.
|
| Well, as long as you're clearly stating _this_ part of your
| otherwise validated and highly likely pre-post-mortem is
| speculation... </s>
| tjr225 wrote:
| Physical infrastructure can be fragile/fail catastrophically,
| as well. I recall starting a job near Seattle, 40 miles from my
| home in Olympia, WA and a passenger train derailed onto the
| highway killing a few people and backing up traffic for days.
| That was not a pleasant first day!
|
| https://en.wikipedia.org/wiki/2017_Washington_train_derailme...
| eepp wrote:
| Damn this made me read a lot about this specific derailment,
| but also about Puget Sound, Salish Sea, Positive train
| control and all this 220-MHz stuff.
|
| Thanks!
| Wowfunhappy wrote:
| I assume, however, that this didn't ground every single train
| in the state!
| tjr225 wrote:
| No but the failure caused a halt to all motor traffic on I5
| in both directions which is possibly even more catastrophic
| than stopping the entire train system.
| noizejoy wrote:
| I believe most trains in the world are grounded to this
| day.
|
| (Mixed metaphors always make me smile.)
| axiolite wrote:
| Electrified trains have a lot of potential...
| mark-r wrote:
| But they make it impossible to run short trains...
| totetsu wrote:
| If it happed on a work day I wonder if peoples pandemic logic
| would kick in and you would start hearing, "It's not worth
| closing down the whole economy just because of the risk of a
| few people dying in a train crash"
| spac wrote:
| I wonder if this is connected to a similar incident in Italy.
| https://news.yahoo.com/italys-state-railway-may-target-16341...
| mushyhammer wrote:
| Considering the state of Italian railways IT, I wonder if the
| attack was accomplished by entering _admin /admin_ in some
| login box.
| denysvitali wrote:
| I think it's more likely that they didn't have any password
| at all :)
___________________________________________________________________
(page generated 2022-04-04 23:02 UTC)