Boeing Outsourced 737 MAX Software Development

If the old saying “you get what you pay for” is to be believed, then Boeing may have gotten just that. Yesterday, reports came out that Boeing has been relying on temporary workers to develop and test software. Those workers, often from countries without significant aerospace background, were making as little as $9 an hour. The outsourcing of work to overseas firms has reportedly led to frustrating miscommunications, an increase in supervision costs, and delays.

The 737 MAX jets have now been on the ground for over three months now with no clear end in sight. Photo: Flickr user Steve Lynes

Extensive reporting by Bloomberg found that overseas contract engineers have had various levels of involvement in the development of the 737 MAX. Connections include:

  • Engineers from Indian HCL Technologies developing and testing the Max’s flight-display software.
  • Employees from another Indian company, Cyient Ltd., having a part in handling software for flight-test equipment.

However, Boeing says it did not rely on engineers from HCL nor Cyient for MCAS. MCAS is the Maneuvering Characteristics Augmentation System and it has been linked to the October 2018 Lion Air crash and the March 2019 Ethiopian Airlines disaster.

Miscommunications, inefficiencies

It should be made absolutely clear that there is nothing inherently wrong with outsourcing work. Sometimes this can be an advantage if a particular firm has decades of expertise on a specific type of project. Furthermore, an external firm may just be better at attracting high calibre candidates. A narrower focus may also lead to a very high standard of training for its employees on the topic.

However, outsourcing of work by Boeing has led to a few problems:

  • While HCL was often designing to Boeing specifications, a former engineer said that it was “controversial because it was far less efficient than Boeing engineers just writing the code [and] it took many rounds going back and forth because the code was not done correctly.”
  • In 2008, one staffer on the 787 project complained about sending drawings back to a team in Russia 18 times. Apparently it took numerous times for them to understand that the smoke detectors must be connected to the electrical system.
  • The 787 made its entry into service three years late and billions of dollars over budget in 2011. This was partially due to confusion introduced by the outsourcing strategy.

“We did have our challenges with the India team…They met the requirements, per se, but you could do it better.” – Charles LoveJoy, former Boeing flight-test instrumentation design engineer

A typical modern jet has millions of parts and millions of lines of code. Photo: Wikimedia Commons

Rushed jobs, and cut corners

Driving down the cost of production seems to be a major factor in many of Boeing’s decisions. With a favorable exchange rate for Boeing, a big part of the attraction for overseas outsourcing was price. Many engineers in India cost $9 or $10 per hour, compared with $35 to $40 for their US counterparts.

But Vance Hilderman, co-founder of TekSci, tells clients the cheaper hourly wage inflates to more like $80 an hour, due to extra supervision required. TekSci is a supplier of aerospace contract engineers, but has lost work to overseas competitors in the past. However, Hilderman says his firm has been winning back some business to fix the mistakes of those same overseas competitors.

Additionally, engineers that worked on the MAX project were complaining about pressure from managers to limit changes that would introduce extra time or cost.

“Boeing was doing all kinds of things, everything you can imagine, to reduce cost, including moving work from Puget Sound, because we’d become very expensive here,” – Rick Ludtke, former Boeing flight controls engineer (laid off in 2017)

The hunt for lower costs also went hand in hand with more orders. The Bloomberg article goes on to say that in exchange for Boeing investing $1.7 billion in Indian companies (such as HCL and Cyient) it landed an $11 billion order in 2005 from Air India.

Boeing’s investment in Indian labor might have helped secure big orders. Photo: Flickr user Liam Allport

Boeing’s defense

According to the Sydney Morning Herald, a Boeing spokesman addressed the controversy by saying:

“Boeing has many decades of experience working with supplier/partners around the world. […] Our primary focus is on always ensuring that our products and services are safe, of the highest quality and comply with all applicable regulations.”

What do you think? Did Boeing make a mistake by outsourcing vital tasks to overseas workers? Let us know your opinion in the comments.

33
Leave a Reply

11 Comment threads
22 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
16 Comment authors
newest oldest most voted
Nigel

“If you pay peanuts, you get monkeys”.

Joanna Bailey

Shall we send Boeing some bananas?

Caroline

If you pay peanuts, you get crashes .
No problem about outsourcing some telecom or design softwares to cheaper Indian engineers but not in the aviation or medical industries.
Airbus aircrafts are more expensive than Boeings’ but Airbus does not do that.

Tony

Heck leave India out of software development please! They’re amazing sales people but software development… I’ve made most of my money fixing their code.
Their management values fast not quality the result is almost always poor coding and spec misinterpretation.

Nigel

A lot of managers don’t seem to realize that cost = hourly rate × NUMBER OF HOURS SPENT. A top-notch engineer might cost $100 per hour, and complete a task perfectly in 10 hours…cost is thus $1000. An amatueur might cost $20 per hour, but take 80 hours to (finally) get the job done acceptably…cost is thus $1600. You’d be amazed how many managers will choose the guy with the lower hourly rate.

Joanna Bailey

Or potentially never get the job done acceptably, and therefore have to go back to the $100 a hour guy to fix it.

Chris Parker

I was involved in outsourcing work to India and it can be beneficial. My experience was that provided things followed a strict process it worked well. As soon as there was any deviation then the problems began.
I would have thought very basic things would be ok. However Boeing is obsessed with shareholder value. The fact that outsourcing is leading to large orders , i suspect there will be more rather than less. Bad news for experienced US workers. The chaos and delays I suspect will get lost in the drive for immediately visible savings.

Joanna Bailey

Absolutely, for things that are straightforward and non critical – it makes sense for companies to get them done cheaper. But when it comes to safety critical processes and complex code, well, you’d think Boeing would have had more sense.

Owen Berkeley-Hill

You hit the nail on the head. The typical MBA (and their numbers are growing) follow the mantra of “maximising shareholder value”. This takes precedence over customer focus and quality. You can get good quality work from outsourcing but how many managers in any American company like Boeing value supplier partnerships over cost, cost cost? The point made about supervisory costs going through the roof, not forgetting telephone costs and time is a good one as I have experienced engineers screaming down the phone trying to get their point across. And then you factor in ten or twelve hour time… Read more »

john PFEFFER

outsourcing is not a mistake.
Not checking the software in a flight simulator is a huge one

Vedant

Very true, John

Nomans Land

No, outsourcing was the mistake. The managers have little to know Software background – so – they can not accurately assess whether or not a Software person has good skills – or – if they are the product of some fly-by-night training and a fabricated resume touting 7 years of experience. They ALWAYS say they can do the impossible. The idiot who hires them does not find this out until large sums of money has been spent and the product either does not work at all (lots of bugs) or works poorly. Years ago, I wrote the Obama about the… Read more »

Marco

Single point of failure on simple objet is not a problem, single point of failure on Aircraft is a dead problem. This is the difference. There are some issue on Max , starting from wrong design ended with the wheels for adjust the trim. The solution is simple do not fligh on Boeing.

Norm

Marco, you areAbsolutely Correct. The issue on the B737 MAX is Not a SOFTWARE issue, it is a Structural Engineering Defect, BOEING tries to Patch, with a so called Software, to partially Remedy the Structural, coupled with Engine Power++ and its Wing placement. This aircraft is already Doomed to Failure from the Flying Public even if FAA, others give their Safety Clearance, sometime in 2020. Won’t see me even close to one of those.

Nomans Land

“The issue on the B737 MAX is Not a SOFTWARE issue,” – while the problem was a structural issue, the resolution (to use software) was ALSO an issue. Just by hearing how the plane operated, one can deduce the faulty code that was created. There program should have had standard parameters (indicating optimal efficiency). If it received any information outside the normal operating parameter set, it should have disengaged. From my understanding, when the pilots pulled up harder, the plane would push down even more. It sounds like a parameter (being used as a guide to determine how much to… Read more »

Nomans Land

Yes, the problem most definitely was a structural issue first and foremost – however – the Software team should also share some of the blame. They should not have said they could successfully perform the task when in reality, that was not the case. The software was designed to work with 1 Sensor – only. That is known as a Single Point of Failure. It is evident that the individuals who designed such a system did not know (or perhaps care) about something that was very much critical. A well-seasoned Software Engineer would never design a Mission Critical system with… Read more »

Owen Berkeley-Hill

How many of you are old enough to remember the rather combustible Ford Pinto if you ran into the back of it? A very expensive lesson for Ford which resulted in Ford importing the aviation methodology, Failure Mode Effects Analysis (FMEA) into its design processes. FMEA was then adopted by the rest of the automotive industry, but that was in the 70s. Not once have I heard anyone, journalist or regulator, question Boeing as to whether they applied FMEA to MCAS. Are we missing something here? Not very investigative! But perhaps we have too much automation and that has led… Read more »

Joanna Bailey

I do remember the pinto… not only did it have a certain fuel issue but also it’s name in Spanish was not at all appealing 😂 But in all seriousness, I agree that at some point Boeing need to admit defeat and just build something new.

Gary

GREED.
Boeing’s priorities are out of synch with their priority for safety, their decision to out source,driven by Greed , the lure for cheaper production costs has turned on them in an ironic twist of karmar.
How long until Boeing file for chapter 11 ?

Joanna Bailey

What’s chapter 11?

Nigel

Chapter 11 is the US term for protection from creditors during bankrupcy. In Europe, it’s sometimes called surséance.
I agree that Boeing is staring Chapter 11 in the face!

Joanna Bailey

Do you not think that Boeing is TBTF? I don’t think the US government would allow them to fail.

Nigel

Well, going into Chapter 11 is a move designed to avoid ultimate collapse: it gives a company breathing space until it manages to re-organize itself. The big three legacy USA airlines have been in Chapter 11 at some stage, and they all rose from the ashes. Sears (the store) went into Chapter 11 last year, but will probably not rise from the ashes. As regards Boeing potentially going into Chapter 11, remember that: – The MAX is supposed to be their biggest cash cow for the coming years. It may never fly again. – They’re heavily bleeding MAX-related costs at… Read more »

Nigel

Perhaps the Simple Flying team should write an article on this?
I’m sure it would precipitate a lot of comments!

Dileep

Gary is absolutely right. Whether something is designed in US, outsourced to India, manufactured in China etc. is irrelevant, if greed and and rush to make profits is allowed to overtake all good sense. Some readers may remember a company named Union Carbide, whose greed resulted in a hundred shortcuts. Result ? Thousands of deaths in Bhopal city, India when poisonous Methyl Iso-Cyanate gas leaked out of their pesticides plant. Something similar happened at Fukushima, Japan where shortcuts in safety resulted in a nuclear disaster. Human race never seems to learn form mistakes of the past!! Sad!! 🙁

Richard

I can not believe that Boeing is trying to blame both crashes as pilot error, those pilots are not at fault whatsoever, Boeing didnt even want pilots to know about the MCAS system, how can a aviation company like Boeing keep this information from the pilots. why are they even adding the MCAS system if there not going to tell pilots about it

Joanna Bailey

The crazy thing is that if you hang around on avgeek forums there are still those out there who think the pilots were to blame.

Nigel

Yes, and I suspect that a lot of those who are still blaming the pilots have firearms and confederate flags in the closet…

Jacque

Hundreds of people perished and not one person held accountable? Boeing needs to clean house and fire every manager associated with the 737 MAX. And change their processes. Unless this is done how else do we know things will be different?

Joanna Bailey

It does seem strange that nobody has taken a hit for what happened. If a car manufacturer designed a vehicle with a flaw that killed 300+ people, you can bet your life the CEO would be fired, if not in prison.

Nigel

I think that, if we’re patient, we’ll soon start to see arrests — lots of them.
I’m sure the DOJ is uncovering lots of savory details at the moment: as soon as their dossiers are thick enough, the public prosecutors will move in.
I’m really looking forward to seeing a lot of executives in handcuffs for this!

Chris

Outsourcing should be illegal. You cant have ppl come to the u.s. to work in an industry that has enough american workers. Why is it okay to outsource software dev when there are enough american devs?

Nomans Land

“Outsourcing should be illegal. ” – the problem is not Outsourcing. The problem is who they are Outsourcing TO. It would not surprise me if some of the developers on that software sported Fraudulent Resumes (displacing the person who had a REAL Resume and REAL experience). Had they hired the person with REAL skills, they would not be staring at the possibility of having to file Chapter 11 now. As someone has said, it is Karma.