[HN Gopher] After Alaska Airlines planes bump runway, a scramble...
___________________________________________________________________
After Alaska Airlines planes bump runway, a scramble to 'pull the
plug'
Author : dlgeek
Score : 195 points
Date : 2023-02-18 15:25 UTC (7 hours ago)
(HTM) web link (www.seattletimes.com)
(TXT) w3m dump (www.seattletimes.com)
| OliverJones wrote:
| A weight discrepancy of ten to fifteen TONS????
|
| That's no small mistake. It's not a rounding error on the weight
| of the soda cans loaded into the galley for the flight.
|
| That's most of the passengers. It's most of the freight. It's
| heavy enough to be a spare engine in the hold. It's a goodsize
| fraction of the fuel needed to fly from Seattle to Honolulu. It's
| certainly enough to foul up the mandatory weight-and-balance
| computation the pilot in command is required to do.
|
| Somehow the input to this software package missed something big.
| It would be interesting to know exactly what was missed.
| sudhirj wrote:
| > Peyton added that even though the update to the DynamicSource
| software had been tested over an extended period, the bug was
| missed because it only presented when many aircraft at the same
| time were using the system.
|
| That seems horribly wrong to me. I can understand software being
| slow under load, but being _wrong_ under load sounds like a
| horrible internal architecture problem.
| dilyevsky wrote:
| From the description it sounds like they tried to give a non-
| technical explanation of a concurrency bug
| ericmcer wrote:
| If you view software as a bunch of contracts between different
| services/libraries/etc. it is hard to ensure that each piece
| upholds its contract, so we trust them to a degree. I can see
| how a failure in one place due to load would be hard to catch
| or gracefully fail from.
|
| That said they have a pretty explicit cap on maximum activity
| (number of planes) so it is weird they didn't test around this.
| It isn't like 400,000 aircraft suddenly DDOS their system.
| cratermoon wrote:
| Concurrency is hard. Thread unsafe operations that end up
| getting executed in multi-threaded contexts are a huge source
| of heisenbugs
| djanogo wrote:
| This wasn't really a concurrency problem as I understand it,
| the requests for calculations didn't need to share any data
| between them.
|
| "the bug was missed because it only presented when many
| aircraft at the same time were using the system"
| usea wrote:
| I believe any bug that only happens under load is a
| concurrency bug by definition. The shared resource is the
| thing under load. If it weren't shared, then the load from
| one computation would have no effect on another.
| hn_throwaway_99 wrote:
| There is certainly not enough information in that quote to
| say one way or another what the bug was. I've definitely
| seen concurrency bugs under load because data that _wasn
| 't_ supposed to be shared actually was, e.g. I posted this
| serious GitHub bug in a comment above,
| https://github.blog/2021-03-18-how-we-found-and-fixed-a-
| rare....
|
| Obviously sessions _should_ be independent and not sharing
| data, but that 's why it was a bug.
| howinteresting wrote:
| Should have used Rust.
| klyrs wrote:
| A proper transactional database would suffice. You could do
| this safely and performantly on '00s hardware in PHP3 for
| crying out loud, serving thousands of planes per second.
| AdrianB1 wrote:
| Bad coding happen even in transactional databases, see my
| example in the other comment.
| speed_spread wrote:
| I refuse to believe that summing up weights for each of 300
| planes is so hard that it can't be made correct by a team of
| moderately competent developers operating under sane
| management. There might be some complexity, but it's 2023 and
| we have enough knowledge and tools to solve that kind of
| stuff routinely and reliably. Somebody fucked up big time and
| should be fired never to work in software again.
| lcuff wrote:
| I agree that it's hard to see from the outside how
| calculating weight figures for an airplane could be
| software load-dependent. But this all smacks of the point
| that Brooks makes in 'The Mythical Man Month' all those
| years ago. Two guys in a garage can make a program that
| does 'x', but to make a programming system product that
| does 'x' is more than an order of magnitude more work. The
| complexities introduced don't relate to the 'x', but the
| intricacies of large systems. Deciding that 'someone has
| fucked up and should never work again'? Sorry, competent
| and conscientious people regularly make such mistakes.
| Which is why we need tests, software review, engineering
| processes et al to catch those mistakes.
| AdrianB1 wrote:
| I can give you a simple example on how this can happen,
| from a real production system in a different regulated
| environment: all the SQL SELECT statements are WITH
| (NOLOCK) on MS SQL server, resulting in non-committed
| (dirty) reads. Under very light load it works fine, on
| higher loads it reads _some_ dirty data. If you calculate
| the weight of the plane by adding the weight of the checked
| in luggage plus passengers read from a shared database,
| some differences will appear and they will increase with
| the load as more transactions will be in flight and
| possibly read or ignored.
| justinclift wrote:
| Boeing itself (unsure about this company though) was widely
| reported to have offshored their senior engineers a few
| years ago. eg:
|
| https://www.bloomberg.com/news/articles/2019-06-28/boeing-s
| -... (paywalled)
|
| https://archive.is/vdg9S (paywall workaround)
|
| Note that there were several articles about it around the
| time, the above one is just the first reasonable seeming
| one from a quick search.
| exabrial wrote:
| I was just about to post the _Exact_ same thing. Each execution
| of the program should be completely independent from other
| calculations. Something is _Horribly_ wrong with their
| architecture.
| lostlogin wrote:
| I wonder how it works. Obviously you are correct, but the
| number of moving parts must be huge. The number of
| passengers, amount of cargo and the fuel load must be
| changing the whole time.
| basch wrote:
| Maybe a timeout. Some part of a value returned 0 if it didn't
| return in time.
| coin wrote:
| Sounds like poor error handing. 0 is valid value is not a
| substitute for no data or error.
| sudhirj wrote:
| Not according to the article, the values returned were within
| the realm of possibility, but wrong. Giving out zeroes might
| have actually been fine, it's easy enough to realise there's
| a problem if you see your plane weighs 0 pounds.
| basch wrote:
| Hence "some part of a value."
|
| Such as weight=plane+fuel+people+luggage. If one of the
| variables became 0 you'd still have the other 3.
|
| It was a low enough value to have made multiple pilots
| question its value that day, like fuel or passenger load
| was missing completely.
| advisedwang wrote:
| I mean it might be just one part. E.g. luggage gets counted
| as zero but passenger weight, fuel weight and aircraft
| weight correct - that might be pluausible but low.
| stateofinquiry wrote:
| So much I can't understand here. Why would the load matter- is
| this a web app? Why would that possibly be a good idea? This
| seems like software that should run locally, for security,
| assurance, and auditability reasons.
|
| Follow up thought: When software results are this critical, I
| wonder if a totally separate program should be used as well and
| result compared. An independent implementation from another
| vendor.
| jg42 wrote:
| The key here seems to be the word "many." Alaska Airlines has
| 289 airplanes (per wikipedia).
|
| All the other arguments seem to assume a large consumer type of
| load - tens of thousands of users, etc...
|
| I just can't see an undue strain being placed on a well
| designed system from < 300 data points. And I haven't even
| accounted for the distribution of needing to compute takeoff
| data over the course of a day nor how many planes are NOT
| taking off at the same time, etc...
|
| Also, to somewhat change the topic, didn't Alaska Airlines
| disband their QA org a few years ago as part of cost cutting?
| IIRC, they did this to model the software company models (that
| ship bugs regularly to consumers) and seem to be getting some
| data that they need to bring back that org...
| Aloha wrote:
| They also mentioned American was using the same system too.
| So it may have been load from more than one company.
| hn_go_brrrrr wrote:
| Good observation. I had assumed each company had their own
| instance, but that's not necessarily true.
| hn_throwaway_99 wrote:
| > but being wrong under load sounds like a horrible internal
| architecture problem.
|
| Tons of software can be "wrong under load" - things like race
| conditions and memory leaks are common problems, and they don't
| necessarily point to a huge architectural defect. E.g.
| Cloudflare had a bug years ago that caused highly sensitive
| data to leak, but only for a teeny (relatively) portion of
| their site. Similar issue happened to GitHub:
| https://github.blog/2021-03-18-how-we-found-and-fixed-a-rare...
| lupire wrote:
| "huge" is not the same as "horrible". One racy boi can take
| out an airplane.
| hn_throwaway_99 wrote:
| But calling something an "internal architectural problem"
| implies things are fundamentally designed wrong, and the
| code needs a rewrite. My point was the architecture can
| definitely be fine, but you can still have bugs (even
| catastrophic ones) because someone left out a
| `synchronized` keyword somewhere.
| thrashh wrote:
| It could still be
|
| For example, one way to use SQL is to escape your
| parameters and then concat them straight into your SQL
| string. However, if someone leaves out an 'escape'
| somewhere, you have a major security incident
|
| The other way is to pass parameters separately as data to
| your SQL driver. It completely negates the problem
|
| If you've chosen the first way in your project already,
| you've committed to a major internal architectural
| problem. In the same vein, maybe if your code requires
| sprinkling 'synchronized' everywhere, you did it wrong
|
| But that said, we don't know what actually is the problem
| since it's just a marketing statement saying that high
| load was the cause and that doesn't tell us anything
| Ozzie_osman wrote:
| True, but software for domains where people can die is very
| different than Cloudflare. The scale is much higher in the
| latter, and it needs to be less "bulletproof".
|
| If you're writing software for giant metal flying cylinder
| carrying hundreds of people, we can't just brush mistakes off
| the same way we might with web-based software (and yes,
| privacy is important, but it's not life or death).
| AdrianB1 wrote:
| In most industries that are not IT, testing the IT systems
| very well is needed, but rarely achieved. In my area (FDA
| regulated) performance testing is almost never done
| correctly and (for sure) never completely, but as long as
| it works and the auditors never look at these details
| nobody from management is willing to know about it, because
| fixing it costs money. For many years the people signing
| off for all the software were intentionally not business
| people with zero IT knowledge, so they could not be accused
| of knowingly signing off low quality code. That saved tens
| or hundreds of millions of dollars in the past 20 years
| alone.
| hn_throwaway_99 wrote:
| I'm not "brushing mistakes off", at all. The process that
| led to the bugs needs a full review and root-cause
| corrective action, especially enhanced testing scenarios.
| I'm just pointing out that software "being wrong under
| load" is not some sort of crazy, unusual problem that
| points to poor software architecture. These bugs can easily
| be caused by a single line of code that wasn't synchronized
| correctly.
| jki275 wrote:
| That's still an architecture defect.
|
| Software's accuracy being affected by load is completely
| unacceptable.
| hn_throwaway_99 wrote:
| Please let us know where these software magicians are that
| write perfect, bug-free code.
| jki275 wrote:
| Who said anything about that? You're arguing against a
| straw man nobody has set up but yourself..
|
| There's no such thing as perfect software, but there is
| definitely software that doesn't make up a false value if
| it can't work.
| puffoflogic wrote:
| For what it's worth, I had the same reaction upon reading
| this paragraph. It seems there are HN commenters who
| understand software design and those who do not.
|
| To be fair to the other commenters, it's conceivable that
| the practical difference between the current unsound
| architecture and a sound replacement might come down to to
| some small defect. But an attempt to fix the problem by
| fixing just the defect, even some kind of RCA process on
| it, will fail if the architecture itself is not fixed, and
| that requires a change in attitudes and understanding by
| the devs responsible for the system, not just a software
| change. That is where the real flaw lies. But some people
| don't believe that it's possible for the flaw to be in
| these places, outside the code. All the RCAs in the world
| can't help them.
| 015a wrote:
| > Bret Peyton, Alaska's on-duty director of operations,
| immediately ordered no more planes were to take off across the
| airline's network. All Alaska flights not already airborne were
| stopped nationwide.
|
| An absolute pro. There's a hundred variations of this story, to
| varying degrees of criticality and impact; seeing a pattern out
| of two data points, connecting the dots, making the tremendous
| call to immediately pull the plug, to stop the world and give
| engineering time, then diagnosing and triaging the problem in
| less than a half hour; that's world class reliability
| engineering.
| raldi wrote:
| Yes. Even better if there are situations declared in advance to
| be automatic halts so no call needs to be made in the moment.
| hef19898 wrote:
| No matter what complex system it is, you always need humans
| in the loop. Especially for networks and not just single
| nodes.
| raldi wrote:
| I think you misunderstand my comment. I'm saying, for
| instance, that during a calm period when a group of people
| can take their time and review each other's reasoning, a
| rule might be put in effect like, "Two tail strikes within
| X hours of each other means an automatic, immediate halt."
| Humans execute that procedure (or override it at the risk
| of their career) and then decide what to do next, having
| been in the loop the entire time.
|
| Such a rule likely saved an SR-71 once:
| https://theaviationgeekclub.com/the-story-of-
| belmont-86-the-...
|
| _The natural tendency for military air crews is to
| complete the mission if humanly possible. To counter this
| inclination, the Wing Commander had designated certain
| emergencies sufficiently critical to require immediate
| landing. This was one of those emergencies._
| lowbloodsugar wrote:
| My fucking washing machine can sense how big a load is in it.
| Surely the plane could measure how fast it is actually
| accelerating given its thrust and thus derive its actual mass. If
| it doesn't match what it's been told it should alarm.
|
| Also shocking that the system gives an incorrect answer when it
| is under load. Lack of load testing is a problem, sure, but an
| architecture that allows the answer to just be wrong is
| fundamentally flawed.
| Tempest1981 wrote:
| Curious about the actual bug:
|
| > the update to the DynamicSource software had been tested over
| an extended period, the bug was missed because it only presented
| when many aircraft at the same time were using the system
|
| > the data was on the order of 20,000 to 30,000 pounds light.
| With the total weight of those jets at 150,000 to 170,000 pounds,
| the error was enough to skew the engine thrust and speed
| settings.
|
| Multithreading/contention issue? But how would that alter the
| weights?
|
| > The software code was permanently repaired about five hours
| later
|
| That's surprisingly fast, isn't it?
| johntb86 wrote:
| > Multithreading/contention issue? But how would that alter the
| weights?
|
| Maybe they stored something in a global variable, and with two
| planes querying at the same time values for the wrong plane
| could be retrieved.
| renewiltord wrote:
| Concurrency is one source, yes, but integration is another
| possible source. If the vendor software interfaced with
| Alaska's software then the interfacing layer could imperfectly
| handle pagination, assume ordering where there is none, and
| other such data source problems.
|
| A trivial error could be "list all active planes", "list
| current luggage weight for all these plane IDs", and have the
| latter give responses in a different order than the input list.
| Or perhaps "fuel load" returns in a different order. Or
| something.
|
| Concurrency seems like the obvious one, but it's not the only
| source I can imagine here. And ultimately, the max is under 300
| planes. You can do that serially unless the integration layer
| is slow.
| mmcconnell1618 wrote:
| I'm surprised there wasn't any human or general guardrail
| around the calculation that would have flagged such a
| significant weight difference. No one looked at the passenger
| count and the weight calculation and thought "whoa, that
| doesn't look right?"
| hodgesrm wrote:
| > The bug was identified quickly in part because some flight
| crews noticed the weights didn't seem right and asked for
| manual validation of the figures.
|
| Actually there were such guardrails, so the system had
| multiple manual checks built in.
| ww520 wrote:
| The fix was probably just a rollback to the previous version.
| sidlls wrote:
| _Multithreading /contention issue? But how would that alter the
| weights?_
|
| Off the top of my head: failure to sum values correctly,
| misreporting the weight to the target plane (reading values
| from another plane's weight data).
| hansvm wrote:
| Could be something as simple as the first concurrency 101
| example: atomic addition. The += operator behaves poorly on
| concurrent access.
| Jtsummers wrote:
| We had that error in an application at work. Essentially data
| was coming in on multiple threads, but there was a shared
| value between them that was being incremented and
| decremented. Without a lock and without using atomics, the
| tally could be incorrect. But under low load this didn't
| appear because the odds of interrupting +=/-=/++/-- was low.
| Under high load, however, the odds of a thread being
| interrupted mid-operation increased and so it appeared in
| production. At least in our case it was just an annoyance (no
| critical decisions were based on the value, it was merely
| informative).
|
| If there are multiple data streams feeding in weight data per
| aircraft, under a low aircraft load (the scenarios they
| apparently tested) the odds of interruption during an
| operation like += can be low enough to not see the issue.
| Under high load, though, the odds of interruption and
| incorrect tallying increases substantially.
| KennyBlanken wrote:
| In what scenario would you anticipate data for the
| calculation of rotation speed and engine thrust for one
| particular plane to be based on a value that is not only
| shared and writable, but changing each time a plane's
| performance data is calculated?
|
| "multiple data streams feeding in weight data per
| aircraft"? What does that word salad even mean?
|
| Even if what any of what you said was true: these
| developers are writing critical safety software. If they
| can't manage to write code without generating race
| conditions _and testing their system for such conditions_ ,
| they are grossly unqualified to be writing this sort of
| software.
|
| As others have said: this is a huge fuck-up.
| Jtsummers wrote:
| It is a huge fuckup, I didn't say otherwise.
|
| And that's not word salad, though maybe not as clear as I
| could have made it. But to clarify for this hypothetical
| (if this is the issue): Each luggage weigh station
| submits data to the application or database which then
| has to tally the weight. Those are multiple data streams,
| not a hard concept. An airport has many luggage weigh
| stations all potentially submitting data at the same
| time.
|
| If those are being processed concurrently and the
| database or however the data is stored is not properly
| locked then the tally can be incorrect for the same
| reason non-atomic increments/decrements can become
| incorrect in a multithreaded application. But just like
| the problem with non-atomic (unlocked)
| increments/decrements the problem may not manifest
| without high enough load on the system. Low-load testing
| (what the article seems to describe) means that updates
| can happen fast enough (server or application is under
| loaded, less likely to interrupt any update) or submitted
| slow enough that the issue never appears. So you need to
| test with a higher load to detect the problem (if you're
| going to be able detect it in testing at all).
| richwater wrote:
| I fucking cracked up at:
|
| "multiple data streams feeding in weight data per
| aircraft"
|
| it's literally some integers (floats at the worst?).
| ww520 wrote:
| If weight calculation is so crucial for take off, it seems like
| redundancy of the weight calculation is needed - a different
| weight calculation from a different company using a different
| method to verify against the main one.
| tzs wrote:
| Would it be possible to actually weigh the loaded plane to
| check the calculations, say with something like a bigger
| version of truck scales that the plane could taxi over?
| pqdbr wrote:
| Why doesn't the plane weight itself? Couldn't there be
| sensors in the landing gear or something?
| [deleted]
| justinclift wrote:
| With Boeing's recent 737 family history (MAX fiasco, etc), it's
| very concerning to see "software errors" still happening in
| operation. :(
| ceejayoz wrote:
| How does a "SaaS Founder, Senior DevOps Engineer" get the idea
| that software errors are entirely avoidable?
|
| This story is about the system working well. Detected
| immediately via well-trained and alert humans, someone had and
| used their individual authority to ground the entire airline
| for safety reasons, mitigated with a workaround 20 minutes
| later, permanently fixed within five hours, and new tests
| implemented to account for the root cause.
| abudabi123 wrote:
| The SpaceX rockets launch routinely with automation at
| takeoff and landing. There is no need for human heroic drama
| in the loop when the software is AAA grade.
| ceejayoz wrote:
| > The SpaceX rockets launch routinely with automation at
| takeoff and landing.
|
| Sure. And there are bugs in their code, too.
|
| https://www.theverge.com/2019/9/3/20847243/spacex-
| starlink-s...
|
| "SpaceX acknowledges that it failed to communicate due to
| the bug and missed the emails about a higher probability of
| collision. Finally, on Monday morning in Europe, ESA made
| the call and used the thrusters on Aeolus to raise the
| satellite's orbit by about 984 feet (300 meters) without
| waiting for SpaceX to take corrective action."
| sxyuan wrote:
| I see this as a near-miss. At my work, there'd be a post-
| mortem written explaining what went well and what went
| poorly. Just because not every line of defense was breached
| doesn't mean everything's peachy.
|
| The first failure was the bug itself - without knowing more
| about the details it's hard to say more. But one should ask
| what kind of practices exist at DynamicSource - e.g., compile
| time thread safety checks, code review practices, testing
| requirements, rollout procedures - that would prevent bugs
| like this from getting out into production.
|
| The second failure was that the software failed to validate
| its own output and fail noisily or generate a warning of some
| kind. It's safety-critical software - bugs may be excusable,
| but not giving at least a warning for an unusual output is
| poor engineering.
|
| The third failure was that the flight staff failed to notice
| the error until tail strikes happened. Human errors do occur,
| but we need better systems to assist human operators. E.g.,
| perhaps the flight software itself could act as another line
| of defense to prompt the pilot when values appear out of
| normal range.
|
| Yes, in this case the planes weren't anywhere close to
| crashing. But if the assumption is that bugs are unavoidable,
| then there needs to be better systems in place to catch those
| bugs before they cause a major accident. Because the next
| time we might not get so lucky with detecting the issue and
| having someone with integrity calling the shots.
|
| Also, no need to start off your comment with an ad hominem -
| a simple "software errors are unavoidable" would've said just
| as much.
| sopooneo wrote:
| I'm with you that a couple layers of the "safety in depth"
| worked impressively well here. But a bug of this types points
| to egregiously bad process and standards with the software
| vendor. And the airlines or FAA should have audits to catch
| that well upstream of this incident.
| justinclift wrote:
| > How does a "SaaS Founder, Senior DevOps Engineer" get the
| idea that software errors are entirely avoidable?
|
| It's literally a safety critical system, not move-fast-and-
| break-things front end web dev code.
|
| Remember the _extraordinarily_ in depth review of their
| software while their entire fleet was grounded, where _each
| and every individual bug found was reported in mainstream
| media_?
|
| Where they weren't allowed back in the sky until we were all
| assured everything was fixed?
|
| The bug they're reporting here sounds like they didn't do
| reasonable testing. And that's the kind of thing that
| shouldn't be happening with Boeing 737* aircraft especially,
| after their recent problems. :/
| ceejayoz wrote:
| > It's literally a safety critical system, not move-fast-
| and-break-things front end web dev code.
|
| So was the Space Shuttle, whose software team is widely
| regarded as the absolute gold standard in this regard.
|
| https://www.fastcompany.com/28121/they-write-right-stuff
|
| Still, bugs:
|
| "This software is bug-free. It is perfect, as perfect as
| human beings have achieved. Consider these stats : the last
| three versions of the program -- each 420,000 lines long-
| had just one error each. The last 11 versions of this
| software had a total of 17 errors. Commercial programs of
| equivalent complexity would have 5,000 errors."
|
| > Where they weren't allowed back in the sky until we were
| all assured everything was fixed?
|
| Because no aircraft would fly again with that standard.
|
| > And that's the kind of thing that shouldn't be happening
| with Boeing 737* aircraft especially, after their recent
| problems.
|
| The software in question isn't made by Boeing, and it's not
| just for Boeing aircraft. It's a third-party thing, picked
| by individual airlines. https://www.dynamicsource.se/ lists
| support for 13 aircraft across 7 manufacturers.
| drewg123 wrote:
| I don't understand why an airplane being heavier than what was
| estimated by the software would cause the tail to drag. I would
| assume it would cause the plane to not have sufficient thrust to
| take off, or to use more runway. But having the tail drag makes
| it seem like it had _too much_ thrust, which translated to a
| steeper take off than anticipated... Can somebody explain?
| pqdbr wrote:
| Let's say your target rotation angle is 10 degrees, so at the
| specified speed (called VR) the pilots 'rotate' the aircraft
| nose up by 10 degrees.
|
| When they start the 'rotate' maneuver, the nose of the aircraft
| doesn't immediately go to (for instance) 10 degrees. It's a
| progression.
|
| As soon as the nose starts pitching up (by 1 degree for
| instance), the aircraft is already generating lift to pull the
| aircraft up.
|
| If the speed is set right (so you're fast), when you get to
| that 10 degrees angle of attack, your tail has already cleared
| the runway by a very safe margin, because you've been producing
| X lift from 0 to 10 degrees of rotation.
|
| However, if your speed is set too slow, you're not generating
| as much lift, so the aircraft will be pulled up slower (you
| have been producing Y% less lift from 0 to 10 degrees of
| rotation). This means that by the time you reach your
| designated 10 degrees of angle of attack, you haven't cleared
| the tail yet, and it will hit the runway.
| cmurf wrote:
| In order to achieve the required lift force for takeoff, a
| higher angle of attack was needed to compensate for the
| incorrect low rotation airspeed. In effect, pilots raised to a
| higher nose-high attitude to achieve lift off by the expected
| point, and this resulted in a tail strike.
| lcuff wrote:
| Not an expert, but here's my best guess. When I learned to fly
| a small plane, you pulled back the yoke at a specific speed.
| This is called 'rotation' and lifts the nose of the plane off
| the ground, (wheels are still on the ground) and changes the
| attitude of the wings, creating substantially more lift for the
| speed. The plane then lifts quickly off the runway. Presumably
| different possible weights in a larger plane would cause you to
| choose different speeds at which to rotate. I also presume
| rotation is also a more gradual process in a larger plane (a
| few seconds long???), so the 'early' part of the rotation
| occurs, gets the plane off the ground, and allows the space for
| the angle to become steeper in the later part of the rotation.
| Would be happy for an expert to review and comment on this.
| sowbug wrote:
| I believe it's a question of angle of attack (orientation of
| the wings vs the direction of airflow). The more you angle the
| wings, the more upward lift they generate. So when the pilots
| commanded the plane to "rotate" (take off), they had to pull it
| back farther than expected to generate sufficient lift for the
| heavier-than-calcuated and slower-than-calculated plane to
| leave the runway. That means the nose of the plane was pointing
| higher than normal, which means the tail was lower than normal.
|
| (Note that you can't keep increasing AoA indefinitely if you
| need more lift; at a critical speed, the airflow will separate
| from the wing and stop generating any lift. That's an
| aerodynamic stall.)
| bogomipz wrote:
| The article states the following about the software:
|
| >"It delivers a message to the cockpit with crucial weight and
| balance data, including how many people are on board, the jet's
| empty and gross weight and the position of its center of gravity.
|
| >In a cockpit check before takeoff, this data is entered into the
| flight computer to determine how much thrust the engines will
| provide and at what speed the jet will be ready to lift off."
|
| Given that overhead bins are regularly maxed out with carry-on
| luggage now since airlines began charging for checked bags, how
| are they able to accurately account for the weight and balance?
| Airlines seem to almost never weigh customers carry-on at check
| in.
| ceejayoz wrote:
| "Regularly maxed-out" translates to "fairly predictable". If
| the 20 people in first class all have 500 pounds of uranium
| each they've snuck onboard, you have a problem, but otherwise,
| there's plenty of wiggle room to account for minor variability
| of this kind.
|
| They'll have a decent idea of the average weight of the average
| passenger (https://airinsight.com/the-pending-new-faa-weight-
| balance-ru...). If you're chartered by an anorexia treatment
| facility to move 300 patients, you may put a little extra
| attention to the exact values.
|
| > The new FAA standards will increase an average adult
| passenger and carry-on bag weight to 190 pounds in the summer
| and 195 pounds in the winter. Up 12% from 170 pounds and 175
| pounds, respectively. This includes an extra ten pounds for
| winter and five pounds for summer. This also includes 16 pounds
| for personal items, up from ten. Airlines must increase the
| average weight of female passengers and their carry-ons from
| 145 pounds to 179 pounds in summer, and from 150 pounds to 184
| pounds in winter. The average weight for males with carry-ons
| is increased from 185 pounds in summer to 200 pounds, and from
| 190 pounds to 205 pounds in winter.
| jcutrell wrote:
| Seems like something as critical as thrust in this phase of
| flight should have some kind of cross-check validation. Sounds
| like a bad single point of failure to me, but maybe there is a
| cross check I don't know about that isn't covered in the story.
|
| What I do know is that with something like this, a little could
| go a long way. I wonder what the inspection and repair for a tail
| strike is, and whether that cancels out the money saved by
| minimum viable thrust across the fleet. I'm sure someone is
| punching the calculator on this to determine that.
| fwlr wrote:
| "the bug was missed because it only presented when many aircraft
| at the same time were using the system."
|
| The system reports data on number of passengers, weight of cargo,
| plane balance, etc., to the pilots. The calculation is done by
| the plane's flight computer. How can it be off by 20,000 pounds,
| but only under heavy server load?
|
| The explanation that comes to mind is that DynamicSource has a
| subservice for each source of weight and one of those subservices
| crashed under heavy usage. So the top-level aggregate-and-report
| service got an error from one subservice and said "well, guess
| it's zero, lol"?
| AceyMan wrote:
| >The calculation is done by the plane's flight computer.
|
| Not exactly true.
|
| The dispatch release has a flight & performance plan based on
| the load manifest. It's calculated out of band (here, by the
| software vendor) and punched into the airplane configuration
| for departure.
|
| If the plan is wrong, the plane flies wrong. GIGO at its
| finest.
| jcutrell wrote:
| Makes me think this could have been a race condition where
| these aircraft somehow received the load calculations from
| other aircraft being calculated at the same time window. Some
| kind of poorly managed event stream / consumer that assumed no
| parallel jobs would be run? Or maybe the jobs are deduped by
| departure / arrival instead of flight number? Seems possible
| newZWhoDis wrote:
| This is why explicit error handling is so critical. Being
| forced to handle the timeout error would make such a bug
| obvious.
| 1101010010 wrote:
| Lots of blame to pass on the software, but the real culprit is
| spelled out clearly in the middle of the article:
|
| > the computer then calculates just the right amount of engine
| thrust so the pilots don't use more than necessary. "The goal is
| to lower the power used on takeoff," he said. "That reduces
| engine wear and saves money" on fuel and maintenance.
|
| This is not an accident, but rather a feature of capitalism, this
| time it is human lives that are commodified and have costs
| externalized unto.
| almenon wrote:
| Less fuel is better for the environment. That should be
| counterbalanced with safety, which they did - even with a bug
| and a heavy plane the plane still took off safely.
|
| This means they are also leaving a safety margin of fuel, even
| though that costs extra money.
| kfajdsl wrote:
| I don't see how this is an example of that? Saving fuel and
| reducing wear on the parts is a good thing. In this case,
| nobody's lives were in danger either.
| CPLX wrote:
| That's a feature of engineering. Figuring out what performance
| and safety requirements are and then optimizing the resources
| required to meet those requirements is _what engineering is
| for._
|
| The influence of capitalism would be on the requirements. But
| assuming the requirements are well-considered trying to
| minimize fuel and engine usage to meet those requirements is a
| _good_ thing.
| jcutrell wrote:
| I think this is pretty subjective. You could also say that
| engineering should be used to increase the margin of safety
| (what is often labeled over-engineering).
|
| If the engineering requirement is "do something that brings
| you close to the margin of safety in order to save
| resources", is that not an influence of capitalism?
| CPLX wrote:
| No, again the decision on where to draw the line is a
| political decision. It's produced by humans making values
| decisions. There's no absence of risk situations so there
| are always trade offs.
|
| How to hit that line _precisely_ and _with confidence_ is
| where engineering takes over.
| brookst wrote:
| You think it would be more ideal to wear out engines sooner and
| burn more fossil fuels?
|
| If efficiency is the curse capitalism brings us, I'm feeling a
| little better about it.
| 1101010010 wrote:
| If efficiency is the goal, packing people into a standing-
| only flight would also save fuel and reduce wear, I suppose
| you don't see any problem with that either?
|
| https://www.theguardian.com/business/2012/feb/28/ryanair-
| sta...
| brookst wrote:
| Are you saying that unless I'm willing to travel in
| uncomfortable conditions, you see no reason to reduce
| environmental impact of flights?
|
| You're trolling, right? That seems like a really
| implausible world view.
| pxmpxm wrote:
| > feature of capitalism
|
| Software to optimize resource usage while providing a service
| => blame capitalism is che-guevara-tshirt-edgelord level of
| inane.
| 1101010010 wrote:
| The ad hominem attack isn't appreciated, you need to do
| better especially if you're going to sincerely defend the
| virtues of your economic beliefs.
| renewiltord wrote:
| Any fool can build a bridge. You need an engineer to build a
| bridge that can barely stand.
| kube-system wrote:
| It doesn't matter if you used full power on every takeoff, if
| you rotate early, you rotate early. Vr is based on weight (and
| other factors).
| Game_Ender wrote:
| In a competitive industry saving money can mean lower fares for
| customers. It can also mean fewer total planes or engines
| produced reducing environmental side effects of that activity.
| terr-dav wrote:
| It can mean those things, yes, but only as side-effects. The
| primary effect being to increase shareholder value.
|
| There's no need to have market based competition to keep
| costs down. Look at USPS for example. That operation is run
| without a profit motive, serves literally every zip code in
| the US (by definition), and runs healthily year after year
| (except for the part where Congress imposed arbitrary pension
| funding requirements[1] and private capital interests have
| been chomping at the bit to buy up all the real-estate
| holdings).
|
| They also don't have UPS's problem of not putting AC in their
| delivery vehicles.
|
| [1] https://ips-dc.org/how-congress-manufactured-a-postal-
| crisis...
| zaroth wrote:
| Calculating the appropriate amount of liftoff thrust based on
| the weight of the plane is universal for flight ops.
|
| This feature of capitalism caused the money to be invested to
| achieve this efficiency, and also the systems and processes in
| place to ensure it was implemented properly; with sanity
| checks, authoritative oversight to halt ops when something was
| determined to be off, manual backup systems in place and well
| trained on, a fix rapidly deployed, and new testing put in
| place to detect any similar bugs in the future.
|
| Yes indeed the stench of capitalism is strong here! </s>
| jcutrell wrote:
| I'm not sure that thrust calculation is actually incredibly
| common. I'd be curious what the handbook on these aircraft
| say, but common practice for many aircraft is full thrust on
| takeoff - rarely is there a negative safety penalty for too
| much thrust. Just an earlier rotate.
|
| Comment OP isn't wrong entirely - this is the result of an
| economic decision. Not without it's good effects - both can
| be true, but this is not a safety optimization by any means.
|
| Many aviation accidents can be traced back to capitalistic
| pressure - cost saving on inspections, time saving for flow,
| etc. It's appropriate to ask whether this is true here as
| well.
| zaroth wrote:
| "Common practice for many aircraft is full thrust on
| takeoff" - not for the class of aircraft we are discussing.
| What hobbyists do on their Turbo Props is irrelevant to the
| operation of a 737.
|
| There are many considerations to setting takeoff thrust
| like climb speed limits, tire speed limits, elevation,
| temperature, engine-out safety margins, and many more I'm
| sure I don't know about with my limited flight knowledge.
|
| This is before we get into discussing weight load, which is
| primarily a factor of fuel load, which is also not just
| filled to 100% for every flight, but carefully calculated
| based on distance, weight, wind speed, etc. to not carry
| unnecessary fuel, which is also a huge efficiency boost.
|
| In short there are safety reasons to use the appropriate
| (non-maximum) thrust on these types of planes just as much
| as there are efficiency reasons.
|
| Capitalistic market forces are a big reason we have such an
| amazing airline industry with planes that are technological
| marvels of comfort, speed, and efficiency. Capitalism drove
| 120 years of relentless R&D into commercial flight systems,
| from the very first flight in 1903.
|
| Not paying for necessary safety inspections is not
| "capitalistic" in any way. It seems like an ideological
| battle to use the term that way.
| pixl97 wrote:
| I mean yes, would you rather flights are $10,000 or $100,000
| each so the engines are taken apart and rebuilt each time after
| a flight? Reducing the maintenance interval requirements
| 'safely' is good for everyone.
| 1101010010 wrote:
| Let's reduce the intervals even further so flights are only
| $10 then, right?
| pixl97 wrote:
| If we could do it safely, then it would be stupid not to.
| signal_space wrote:
| calculation of total weight ends up being a pde right?
| batch12 wrote:
| This would be an effective side channel attack to use against
| one's enemies. Even if the true source was never known, the chaos
| and distrust in infrastructure after the crash would be nice
| second-order effects.
| Merad wrote:
| I was under the impression that SOP for most aircraft was to take
| off at full power, and if you happen to be lightly loaded you
| just get into the air faster. Was I wrong, or is this a recent
| change? If so, it seems like a concerning way for airlines to cut
| costs. Having excess power available at take off means there's a
| greater margin for error to handle emergencies (engine failure,
| bird strike, etc.).
| sokoloff wrote:
| Jets have been using reduced power takeoffs for many decades,
| with good operational results. There's an AC about it from 35
| years ago:
| https://www.faa.gov/documentLibrary/media/Advisory_Circular/...
|
| Note that these are manufacturer-sanctioned procedures, not
| applicable to small piston aircraft, which should be using full
| power/RPM if the AFM/POH calls for that.
|
| Note also that the full rated power (and sometimes more) is
| available in the case of an abnormal or emergency.
| bsder wrote:
| Airlines are reducing safety margins in order to do cost
| cutting.
|
| As long as you save more than $1.5 billion ($5 million x 300
| passengers) before a plane crashes, no problem.
|
| Until we _completely bankrupt a company_ for killing people
| under its jurisdiction, this will all continue.
| lordnacho wrote:
| I'm surprised too. I would have thought the take-off system
| would incorporate some sort of feedback about how close the
| tail was to touching the ground and either tell the pilot or
| correct for it and tell the pilot.
| coin wrote:
| They use reduced power to extend the life of the engines
| salawat wrote:
| Everyone else's (consumer's/engineer's/pilot's) "safety margin"
| is an executive's "cost to be optimized".
|
| Make no mistake. The Exec/finance class will cut into any
| margin that you aren't willing to make them directly legally
| culpable for.
| lcuff wrote:
| Yes. Chelsea Sullenberger ("Sully") made the same point in an
| interview. He said in the early days of the airlines, the CEO
| was always a pilot, and error margins were set much higher
| than legally required. Now that CEOs tend NOT to be retired
| pilots, new regulations are needed to raise the safety
| margins beyond current regulatory points to what a safety-
| conscious pilot would choose.
| gdmt wrote:
| My understanding is that using less than full power prolongs
| the life span of engine components and makes fatigue-related
| engine failures less likely.
| Animats wrote:
| _" Peyton added that even though the update to the DynamicSource
| software had been tested over an extended period, the bug was
| missed because it only presented when many aircraft at the same
| time were using the system."_
|
| Now that's strange. Anyone have more details? Why should there be
| any connection between the calculations for different aircraft?
| roamerz wrote:
| Seems like any modern jet would have a load sensing system built
| into the landing gear. Something so critical such as takeoff
| weight would be both estimated by a software system and compared
| to an onboard system for redundancy. Any discrepancy beyond a set
| limit would turn on a warning light.
| Aloha wrote:
| This seems like mostly how it ought to work -
|
| * There were two unusual events in very short order
|
| * Someone quickly noticed and gave the order to go to ground stop
|
| * The problem was figured out quickly and a work-around was
| developed
|
| * Flights Resumed after successfully deploying the work around to
| the 'production process'
|
| * A patch was quickly developed and deployed once the underlying
| bug was uncovered.
|
| I think everyone (but perhaps the developers) comes out looking
| like a champ.
|
| Now, do I think telling pilots what thrust they should use to
| take off is wise? I dont know - I'm not a pilot, and I'd want
| pilots feelings on the matter before commenting further, but it
| feels to me like a loss of authority that would make me
| uncomfortable - particularly if I was held to account for
| undetected failures.
|
| The other part that I'd note here, is _process_ is just as
| important as _software_ could process alone have caught this
| failure without the tail strikes? possibly, and its something
| worth looking into further - but only if it doesn 't add an
| unacceptable workload burden to pilots workload that could
| otherwise compromise safety further.
| sidlls wrote:
| This is a failure in the development of the software that good
| process in the purchaser's handbook saved them from.
|
| That there were no tests for this system under high load
| indicates a _poor_ engineering practice at DynamicSource. They
| don't deserve any kudos for a post-hoc patch that a test in
| production discovered the bug for when the system is a safety
| critical system like this.
| Aloha wrote:
| They may have thought their test coverage, covered it
| adequately - who knows, the success was on Alaska's part.
|
| Also process, procedure and more importantly people are
| always the last line of defense against failure.
| lupire wrote:
| Making excuses for the failures in the first line of
| defense is the first domino in catastrophic failure.
| Normalization of deviance.
| sidlls wrote:
| I don't think OP was making excuses. It is true that the
| software vendor's missing test case indicates a poor
| quality development environment _and_ that humans are the
| last line of defense. No development process is perfectly
| resilient against errors, even good quality processes.
| cm2187 wrote:
| The bug is in the weight of the payload of the plane. I don't
| believe pilots have any way to measure the actual weight of the
| payload of the plane, they rely on that sort of estimates to
| determine how much fuel they need, take-off speed, etc. It's
| not really a loss of authority.
|
| Bad weight estimates have resulted in crashes in the past.
| t0mas88 wrote:
| > Now, do I think telling pilots what thrust they should use to
| take off is wise? I dont know - I'm not a pilot, and I'd want
| pilots feelings on the matter before commenting further, but it
| feels to me like a loss of authority that would make me
| uncomfortable
|
| Pilot here :) the journalist got it a bit wrong, performance
| data isn't primarily about how much thrust to use. It is part
| of the calculations, but crews are free to deviate from reduced
| thrust calculations and will do so.
|
| This incident is much more about the calculated speeds to use
| and aircraft trim settings. That's what caused the tail
| strikes, trying to lift off at a speed that is too low for the
| weight.
|
| Those numbers are required by every airliner taking off. And
| getting them from software is standard everywhere. The
| alternative is doing the calculations manually using paper
| charts, every pilot is required to know how to do that, but
| nobody does it that way for commercial operations.
| Aloha wrote:
| Thank you for the thoughtful comment!
|
| I thought I was missing something in the way the story read,
| I'm glad to know I was.
| basch wrote:
| The purpose of telling pilots thrust was to save money and
| fuel. It's a "least viable" optimization.
|
| >"The goal is to lower the power used on takeoff," he said.
| "That reduces engine wear and saves money" on fuel and
| maintenance.
|
| >Flights to Hawaii are typically full, with lots of baggage and
| a full load of fuel for the trip across the ocean. The planes
| are heavy.
|
| > That morning, a software bug in an update to the
| DynamicSource tool caused it to provide seriously undervalued
| weights for the airplanes.
|
| >Peyton added that even though the update to the DynamicSource
| software had been tested over an extended period, the bug was
| missed because it only presented when many aircraft at the same
| time were using the system.
|
| > Both planes headed down the runway with less power and at
| lower speed than they should have. And with the jets judged
| lighter than they actually were, _the pilots rotated too
| early._
|
| >The data "confirms that the airplane was safely airborne with
| runway remaining and at an altitude by the end of the runway
| that was well within regulatory safety margins. Both aircraft
| got airborne well within safety limits despite the lower
| thrust.
|
| It's also probably lucky that two Hawai'i bound flights both
| didn't notice the discrepancy, or two bumps wouldn't have bee
| _as_ suspicious. Interesting that someone cited divorce and
| sleep examples as emotional pleas, reassuring us they are only
| human.
|
| Like all complex failures at least 5 things had to happen.
| Software update that morning, some kind of hard to simulate
| resource issue in processing, overweight flights, pilot not
| pricing plane data appeared light, pilot turning early despite
| low speed, two in a row.
| pxmpxm wrote:
| >Interesting that someone cited divorce and sleep examples as
| _emotional pleas_ , reassuring us they are only human.
|
| Huh? FAA spends a huge amount of time and energy focusing on
| human factors, tasks saturation rates and crew resource
| management. Nothing to do with with argumentum ad passiones,
| just prudent risk mitigation.
| hooverd wrote:
| I thought a lot of safety thinking was built around
| acknowledging that humans don't do everything perfectly all
| the time.
| ddulaney wrote:
| It does, and one class of mitigation is to improve
| automation, which airlines (and their regulators) do all
| the time.
|
| But another class of mitigations is to make the human
| less error-prone in the first place. This is especially
| important in preventing failures that were caused by
| improper management of the automation. Hence the emphasis
| on rest periods, training, and other mitigations on the
| human side.
|
| In the "swiss cheese" model, a disaster needs to slip
| through both the human's defenses and the automation's
| defenses to actually happen. It makes sense to improve
| both of those layers of defense against disaster, while
| acknowledging that humans will never be perfect. But at
| least they can be awake and knowledgable.
| NovemberWhiskey wrote:
| Right. Literally every pilot in the US has been taught the
| IMSAFE mnemonic to assess fitness to fly, even before their
| first solo in a single-engine piston: Illness, Medication,
| Stress, Alcohol, Fatigue and Emotion.
| basch wrote:
| I just don't see how it fit the article. Unless they are
| surmising the pilots poorly sanityized the data going into
| the computer because they were having an off day.
| [deleted]
| mnw21cam wrote:
| > The purpose of telling pilots thrust was to save money and
| fuel.
|
| No, it's more that using lower thrust during take off saves
| on engine wear and noise levels around the airport. It
| doesn't really have that big an impact on fuel use, at least
| that's not the primary purpose.
|
| The article concentrates far too much on the thrust setting.
| The important bit is the speed where the plane should be
| rotated to take off, which is known as V_r. That depends on
| the weight of the plane. On the takeoff roll, the pilots
| watch the airspeed indicators and pull back to lift off when
| that speed has been reached. If they have been told a V_r
| that is lower than it should be, then pulling back will
| rotate the aircraft _without_ it taking off, and that 's when
| you have this danger of the tail hitting the ground. When
| that happens, the plane must be inspected, because a tail
| strike has the potential to weaken the pressurised container
| that is the fuselage, and this could lead to an explosive
| burst and depressurisation when flying at high altitude. See
| https://en.wikipedia.org/wiki/China_Airlines_Flight_611 for
| why this is a Bad Thing.
| phkahler wrote:
| It seems like a slower rotation might be useful, to wait
| until you feel it leave the ground before aiming higher.
| I'm sure there are reasons for the procedure as is.
| jcrawfordor wrote:
| Trying to "force the airplane off the ground" by rotating
| prior to Vr has a few safety downsides. One of them is
| that it makes tailstrikes more likely, as you'll have to
| carefully moderate the angle before lift occurs. Another
| is that it removes loading from the wheels more quickly,
| which extends the "awkward zone" in which the airplane is
| still rolling on the ground but has so little weight on
| the wheels that it will skid very easily. This can lead
| to aircraft being blown sideways off the runway since you
| can't yet roll into the wind.
|
| A somewhat less direct concern but a related one is
| ground effect lift. the aircraft can remain a short
| distance off the ground (roughly a wingspan of altitude)
| at lower speeds than it can actually "fly," due to ground
| effect. Smaller aircraft might routinely spend some time
| in ground effect gaining additional speed before they
| begin climbing, but airliners have so much thrust they
| usually rotate pretty directly to their climb speed. This
| makes it more of an issue though that if rotation occurs
| too far before Vy climb speed the ground effect period
| will be prolonged and increase the amount of time the
| aircraft spends at risk, flying but slow with poor
| control authority and too close to the ground to have
| much of a recovery opportunity. The ground is a pretty
| safe place to be, the sky is a pretty safe place to be,
| but that first couple thousand feet between the ground
| and the sky is rather hazardous and jetliners get out of
| it as quickly as possible.
| AdrianB1 wrote:
| Rotating at a slower speed is a bad idea. Indicating a
| lower weight, resulting in a lower calculated rotating
| speed, is the root cause of the problem. There is no
| "slower rotation", one does not just pull the yoke
| slowly, raise the nose wheel and wait for the plane to
| take off by itself when it will be fast enough, that you
| can do on a light prop plane where rotate speed is so low
| some planes will take off by themselves, but not on an
| airliner where you have a high chance that raising the
| nose too early results in speed not increasing or even
| decreasing.
| dandandan wrote:
| I interpreted "slower" as "later," ie wait longer before
| rotating.
| AdrianB1 wrote:
| That means rotating at a higher speed than calculated Vr,
| this is also bad as you put more wear in the tires and
| use more fuel (rolling on the tires vs flying in the air)
| to take off. The Vr is an optimal speed, below is
| dangerous, above is wasteful.
| basch wrote:
| > saves on engine wear
|
| Which is saving on money
| seabird wrote:
| I'm cracking up at the thought of an airline telling
| their pilots "fuck it, run em however hard you think you
| need to."
|
| There's a difference between dumping toxic waste into a
| river to save money and keeping a plane engine in safe
| operating conditions to save money.
| NotYourLawyer wrote:
| It's software. It's made of bugs.
| fortran77 wrote:
| [flagged]
| JamesonNetworks wrote:
| The article says the reason the weights were wrong was due to
| high load on the DynamicSource software. Seems more like a
| concurrency issue
| [deleted]
| loeg wrote:
| Both aircraft seat about 180 pax. To be 20,000 lbs off in your
| hypothesis, each passenger would have to be 110 lbs heavier
| than modeled, on average. For 30,000 lbs, 165 lbs.
| fortran77 wrote:
| See: https://airinsight.com/the-pending-new-faa-weight-
| balance-ru...
|
| There were recent changes to the weight and balance rules
| because of increased weight. It's certainly possible this was
| a factor.
| loeg wrote:
| > The new FAA standards will increase an average adult
| passenger and carry-on bag weight to 190 pounds in the
| summer and 195 pounds in the winter. Up 12% from 170 pounds
| and 175 pounds, respectively.
|
| This is a lot smaller in magnitude than +110-165 lb.
| orasis wrote:
| Why is there so much aviation disaster porn on here recently?
|
| This is turning into the same sort of high drama news site that
| most of us are trying to get away from.
| sithadmin wrote:
| 737-900s are notoriously tail-heavy. Some airlines _require_ that
| they use tail stands when parked, and some airlines (like United)
| have taken measures to redistribute weight forward of the rear
| landing gear (by moving one of the economy lavatories so that it
| sits directly behind first class /in front of most economy
| seating).
| coin wrote:
| Height of the landing gear is fixed, they've milked the 737
| design by stretching the length making it vulnerable to a tail
| strike.
| hodgesrm wrote:
| That was an impressively quick call based on ability to reason
| about unexpected behavior of the system. It should give pause to
| anyone who wants to take the human out of the loop in safety
| critical software.
| guitarbill wrote:
| Agreed, although it takes an individual with a huge amount of
| integrity or a culture that values and promotes integrity to
| make a call like this:
|
| > "At that point, two in a row like that, that's when I said,
| 'No, we're done,'" said Peyton. "That's when I stopped things."
|
| Hats off to Peyton, and possibly Alaska. I would also love to
| hear about times when something similar happened, and it was
| discovered that nothing was wrong.
| CPLX wrote:
| Amazingly, this really is the approach that has characterized
| all aspects of aviation for decades. It's incredibly
| admirable and worth emulating in many other situations.
|
| It's also fragile, and the fact that it strikes many as an
| uncommon level of integrity for a business decision is
| precisely why aviation needs to be shielded from having the
| prevailing business culture sneak in, like we have seen
| happen at Boeing in a few highly notable instances.
| hodgesrm wrote:
| This type of judgment call by a single person has also
| helped avoid nuclear war on at least two documented
| occasions. [0,1] Again, food for thought.
|
| [0] https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false
| _alar...
|
| [1] https://en.wikipedia.org/wiki/Able_Archer_83
| nop_slide wrote:
| This other article linked near the end of the source article is
| even sketchier!
|
| It basically boils down to "this flight started nosediving,
| almost hit the ocean, and we don't know why".
|
| https://www.nytimes.com/2023/02/13/us/united-maui-flight.htm...
| Max_Horstmann wrote:
| > The Alaska captain said that, as for many things in aviation,
| pilots routinely use an acronym when they do the pre-takeoff
| "sanity check": TLAR, which means "That Looks About Right."
|
| Aviation's LGTM
| neonate wrote:
| https://archive.md/XOgpi
___________________________________________________________________
(page generated 2023-02-18 23:01 UTC)