[HN Gopher] Boeing's 737 MAX Software Outsourced to $9-an-Hour E...
___________________________________________________________________
Boeing's 737 MAX Software Outsourced to $9-an-Hour Engineers (2019)
Author : JazCE
Score : 99 points
Date : 2024-03-13 09:27 UTC (13 hours ago)
(HTM) web link (www.industryweek.com)
(TXT) w3m dump (www.industryweek.com)
| jkic47 wrote:
| The article seems to conflate engineering salaries in a low cost
| of living country with the quality of the software.
|
| As I understand it, the software performed exactly as designed,
| forcing the nose of the plane downwards to counter the upwards
| torque produced by the engines being offset relative to the
| centre of gravity of the plane.
|
| Mentour Pilot had a great episode on the two tragic accidents. It
| appears that Boeing did not expose the way the software was
| designed to behave, and the system silently turned itself back on
| even after the pilots disengaged it.
| silent_cal wrote:
| In my experience there is a correlation there. Low-cost
| outsourced code is generally pretty bad.
| Smeevy wrote:
| Not that you intimated this, but I don't know that it's
| necessarily a problem with the people writing the code. What
| I've seen is that organizations that try to save money by
| offshoring are not providing adequate domain knowledge or
| usable requirements to the developers.
|
| Throw in a massive timezone difference and every
| misunderstanding winds up living in your system.
| silent_cal wrote:
| That could be. However I think the language barrier is also
| a significant factor. But of course there is nothing
| inherently problematic about offshore developers.
| cityofdelusion wrote:
| I've had to deal with offshores my entire career and have
| even flown out to train them up. They are good people,
| but almost universally, they contract for U.S. companies
| via large body shops (you know who they are) that have
| awful pay, poor treatment of developers, and just a
| general culture of half-assing work. I've told various
| great offshore developers personally to leave as fast as
| possible and join literally any other organization for
| way more pay. Good developers have zero business working
| at those body shops that all the U.S. corps utilize.
| pfannkuchen wrote:
| Culture issues do exist. IME different cultures have very
| different approaches to communicating e.g. "I don't know"
| or "I made a mistake". Not everyone in a particular culture
| acts the same, but it definitely affects the average
| outcome.
| initplus wrote:
| It's not a problem with individual offshore engineers. The
| problem is the way the business they are employed by
| operate.
| uuddlrlrbaba wrote:
| Weird take.
|
| The theme throughout has been cost cutting and poor oversight
| for the sole benefit of shareholder profits.
|
| A proper in house team would probably say, hey what happens
| when the sole sensor feeding our software is obstructed by a
| birthday balloon? Maybe we need an exponentially backoff
| instead of every 30 seconds. Maybe nose down actions shouldn't
| be automatically executed at low altitude.
|
| What we see playing out is the result of delegation and "not my
| problem", outsourcing safety critical systems to the lowest
| bidder...
| NullPrefix wrote:
| birthday balloon? In front of a landing plane at 170mph?
| moosedev wrote:
| GP is (quite reasonably) using a deliberately slightly
| silly example as a rhetorical flourish. I would do the
| same. Think of it of a stand-in or totem for all of the
| possibilities (from silly to very real) that leap into
| horrifying life in an engineer's brain when they learn that
| a critical system has a single point of failure.
| neuralRiot wrote:
| In a safety-critical design you have to take in account
| what will happen if what cannot happen happens.
| ElevenLathe wrote:
| I understand that it may be more likely for this to happen
| with an in-house team, but my experience working in large
| companies makes me think it would still be unlikely. "Not my
| problem" is necessary armor at a company with 15 levels of
| management between ICs and the C-suite and a rich weft of
| (official and unofficial) "dotted lines", where every
| interaction outside your immediate team is fraught with
| political implications. Code monkeys, in-house or not, might
| not even know that they are working on software for an
| airplane, let alone a Boeing airplane, let alone in a safety-
| sensitive system. Even if they do, and understand that the
| overall plan they are contributing to is folly, there is
| often no linkage by which they can make that insight known to
| someone who can change things (often there is literally no
| one, including the CEO, who could change course even if they
| want to). Even trying to do so is often a career risk.
|
| Large companies are mechanisms that run on their own. People
| have limited agency in a lot of situations but for big
| decisions ("our company's only real product is fundamentally
| dogshit, lets start over") there literally are no people on
| earth who can actually make them. They are pre-made by the
| structure of the company, capital markets, and historical
| accident.
| hgfghj wrote:
| My experience is that contract work like this ends up being
| done by ordertakers; the contractors execute the design, no
| matter how stupid.
|
| Whereas, the best performing companies hire engineers who are
| empowered to understand the goals, understand the problem, and
| push back when appropriate.
|
| The results are much better, because the people working on each
| system, understand it deeply and care.
|
| A low-cost contractor isn't allowed to care. They aren't even
| allowed to talk to somebody who matters.
|
| This is a management disgrace.
| genman wrote:
| I think that this a fair take, especially about the possible
| power structure.
| gregors wrote:
| And yet we seem to conflate salaries with competent leadership
| positions.
| renewiltord wrote:
| MCAS wasn't outsourced to India LOL. People here are gullible
| idiots hahaha.
| seatac76 wrote:
| Yeah would almost certainly be ITAR controlled no way core
| tech like this gets outsourced.
| rednerrus wrote:
| The biggest problem with offshore engineering teams is their
| dedication to delivering exactly what you ask for. One of the
| qualities of a good software developer is to take the
| requirements and interpret what needs to happen then deliver
| appropriately. Delivering exactly what's written without
| thinking about the implications is a huge problem and one of
| the biggest issues I've run into working with offshore teams.
| struant wrote:
| If just deliver exactly what you are told to, you aren't
| doing any "engineering" at all.
| datavirtue wrote:
| Right,I just spend a little extra time on prompts and get
| much better, faster, work done completely stress free.
| jkic47 wrote:
| I agree with you on the problem with offshoring, having done
| this myself in a previous role in a regulated industry.
|
| However, the offshore team are usually not intended to act as
| domain experts. In fact, they were very likely explicitly
| proscribed from interpreting the specifications handed to
| them to guard against them trying to act as domain experts
| and delivering something different from expectations.
|
| As such, they were (likely) not the ones who specified that
| MCAS should silently turn itself on after a pilot turned it
| off. That misjudgment probably lies with the engineering team
| who made that design decision, and it had tragic results.
| wannacboatmovie wrote:
| > As I understand it, the software performed exactly as
| designed
|
| This is very likely correct.
|
| People on HN don't seem to understand how software is created
| in regulated industries outside the tech bubble.
|
| In a complex system such an aircraft, the behavior of the
| system is modeled and detailed requirements are generated.
|
| These models are created by the system engineers.
|
| The models and requirements are then handed off to the software
| engineers to implement the modeled behaviour in code.
|
| The software is then tested to see if the behaviour matches the
| models.
|
| So it doesn't matter how much the SWEs were paid, if the
| software met the requirements, and implemented the models as
| designed, then the software engineers did their jobs.
| adriancr wrote:
| The software engineers job is also to think about what
| happens in corner cases.
|
| What if sensor malfunctions?, how should software act?, will
| it crash the plane?
|
| If yes, then raise problem and refuse to implement / avoid
| killing people.
|
| Then you have QA which should have the same questions and
| test plan to make sure corner cases are covered. QA is also
| responsible.
|
| Then you have management which is supposed to make sure there
| are people skilled enough doing the above and the initial
| systems parts and encourage this environment.
|
| Then you have business people to ensure sufficient resources
| to hire competent people above.
|
| So what is happening is a massive problem everywhere...
| karmakaze wrote:
| Yep, the problem with the software wasn't the implementation,
| it was the spec: to operate contrary to the expectations and
| experience of pilots.
|
| The problem was that they didn't outsource the design spec. I
| doubt an externally developed design would be so contrary, with
| a flippant we'll document it and include it in the short brush
| up training.
| datavirtue wrote:
| The umpteenth comment trilling on as though MCAS was built by
| contractors when it was explicitly stated in the article that
| it was developed in-house by Boeing employees.
| acchow wrote:
| Experienced engineers don't just build things to spec. They
| question the spec and the human experience of using the
| technology.
| zamfi wrote:
| Is this more, or less, concerning than the growing trend to
| outsource software to $0.03 / 1000 token AIs?
| tledakis wrote:
| Don't worry, with chatgpt the software is going to be safer.
| The plane won't even take off :)
| FirmwareBurner wrote:
| _" To save us time I tasked my AI, Anton, to debug the code,
| and Anton decided that the best way to get rid of all SW bugs
| was to delete the entire codebase, which is technically and
| statistically the correct solution to what I tasked it to
| do."_
|
| -Gilfoyle
| darkhorn wrote:
| Outsourcing is a very bad practice. Those softwares tend to be
| very low in quality.
| Madmallard wrote:
| Wait til Devin makes your next safety critical system. :)
| aslihana wrote:
| > recalled one manager saying at an all-hands meeting that Boeing
| didn't need senior engineers because its products were mature.
|
| Shame. Industrial companies, if you decide as pincher, you will
| be doomed in your little world... as it always has been :)
| FirmwareBurner wrote:
| Most big "industrial-like" product companies are pinchers when
| it comes to SW. They offshore a lot hence why they pay SW devs
| so little. And it mostly works for them.
|
| It works for them, because like Boeing said, they have old
| battles tested processes which usually means big mistakes can
| easily get caught ahead of time. And if big mistakes don't get
| caught they usually can't kill anyone and generate a huge
| public scandal.
| lencastre wrote:
| Fivrrr for airplane safety,... it's like that BM episode,
| flywheel of death...
| fargle wrote:
| > BM episode, flywheel of death
|
| what is this? do you have a link?
| yurishimo wrote:
| They are referencing Black Mirror but a cursory search with
| those keywords does not return an episode. I'm sure with a
| bit more searching that you can find it!
| ulucs wrote:
| AI generated black mirror titles? Seems fitting
| fwungy wrote:
| There's no proof that $9/hr software engineers are not as good as
| $500/hr software engineers. People have different preferences for
| time and money. One of the best software engineers I know hires
| out for $15/hr, and any project would be lucky to get xir.
| methodical wrote:
| Not that you're incorrect in all cases, but generally speaking,
| you do get what you pay for. SWEs who are charging $500/hr can
| charge $500/hr due to significant domain knowledge or overall
| experience, and as such their opinions and work are _worth_
| that amount per hour to the market. SWEs who are charging $9/hr
| are either selling themselves extremely short, lack experience,
| or operate in an organization that doesn't allow them to
| negotiate their rate (in which case they should move to a new
| organization) and thus most likely generate lower quality
| results. To imply there's no relationship between the hourly
| rate and the quality of work is to pretend that there's no
| underlying reason why certain SWEs can charge premium hourly
| rates.
| jbreckmckye wrote:
| If xhe is that good and contracting for $15/hr, then xhe is
| extremely foolish.
|
| There are great engineers in non-Western countries. It's just
| that they realise talent is so rare, they can charge more than
| local rates.
| nottorp wrote:
| > The Max became Boeing's top seller soon after it was offered in
| 2011.
|
| Seriously? I mean, could you still order anything but a Max from
| Boeing in that size? And that being the most useful size, of
| course it became the top seller. Because it was the only
| offering.
___________________________________________________________________
(page generated 2024-03-13 23:02 UTC)