1. Business
  2. Tomorrow’s transport
July 25, 2019updated 26 Jul 2019 11:22am

How “software flaws” have had tragic consequences for Boeing

By Ellen Daniel

On Wednesday, the world’s largest aerospace company Boeing reported its biggest ever quarterly loss, as ongoing issues facing its 737-MAX aircraft saw the company lose $2.9bn in the last quarter.

Boeing 737-MAX aircraft were involved in two fatal crashes in 2018 and 2019, which resulted in the tragic death of a total of 346 passengers, with a software issue believed to be behind both incidents.

In April, Boeing confirmed that the aircraft’s Maneuvering Characteristics Augmentation System (MCAS) is thought to have played a role in both accidents. MACS is designed to automatically activate and lower the aircraft’s nose if it is at too steep an angle to prevent it from stalling. However, if the sensors respond incorrectly, the nose may be pushed down when it is not needed. As of March 2019, the 737 MAX is grounded as Boeing works to resolve the software issues and until regulators decide the aircraft is airworthy.

As well as putting passenger lives at risk, the software events have resulted in flights being cut by major airlines and a 37% drop in orders for the 737-MAX. This ongoing situation demonstrates the tragic consequences the deployment of software can have.

“Airplanes increasingly have a lot of software on board”

With the rise of connected cars and increasingly sophisticated aviation software, vehicles are driven by a growing number of interconnected systems, meaning without rigorous testing one glitch can have serious consequences.

Lev Lesokhin, EVP Strategy and Analytics at CAST believes that more needs to be done to prevent “serious safety flaws”. This is particularly crucial in the aviation industry. Lesokhin explains that modern transportation is increasingly software-dependent:

“Airplanes and automobiles have increasingly a lot of software on board. I think the 787 is known to have six and a half or seven million lines of code…a lot of avionics software that’s on board and that’s not even counting all the other software, like the entertainment systems. Cars famously have a lot of software, that’s actually because they’re a little bit more disorganised, in the way they do software development.”

Having been at CAST for the past ten years, Lesokhin sheds some light on what could have gone wrong for Boeing’s software:

“When you have a lot of different components, they may behave a certain way on their own, but then when you put them together into a bigger system, they may behave in unpredictable way… Based on what I’m reading what’s gone on there at least one of the issues that they have is kind of a system level issue, where different components are not interacting the way they thought they would, right.”

He explains that companies sometimes fail to adequately test the system as a whole:

“Companies rely on testing, in order to make sure that their software reliable, and testing is not sufficient because there are so many combinations, and permutations of rare situation, we have to test in order to be sure that you’ve gotten everything so it’s literally impossible to test every combination with every release that you do. And the piece that’s missing is kind of understanding this system level interaction, from a software engineering standpoint.

“Analysing the structure of the whole system and how all the components are interacting with each other and where you may be introducing software engineering flaws. And this is something that I think these companies can do better at in order to control these types of issues from happening.”

“There’s no such regulation around software at all”

Many aircraft now operate using “fly-by-wire”, which has replaced manual flight controls and means that the pilot input data, and the computer system flies the plane. However, according to Lesokhin, despite software now being integral to aviation, it is not not subject to the same scrutiny as other aspects  of a plane:

“When you think about building a jet airplane you think of these really sophisticated factories where you go through all these sophisticated checkpoints that are mandated by the FAA, for the US, they’re heavily regulated. And software is sneaking in as part of that construction process but that sort of level of control and insight about it, is still lagging. From an exec. standpoint, the software issues that we’ve seen Boeing have have a huge impact on valuation on future revenue and profitability. But the level of insight that the executives have into the software that’s running their business is much more limited.”

According to research by IT consultancy Coeus Consulting, a significant lack of a “board-level understanding of the importance of IT” often prevents organisations from taking advantage of cutting-edge technology. Lesokhin believes that there is a lack of awareness of the importance of robust software at all levels of an organisation:

“They tend to believe that things like software are too technical for them to care about. It’s something that developers really should manage…and the aviation industry is probably a bit more sophisticated in that regard than most other industries. And you  have seen the impact of that across retail banks in the UK. Software issues run amok. Totally uncontrolled. So I think executives need to demand more visibility into, you know, into how robust their software is.”

According to Wired, automotive recalls linked to electronic and software failures increased 30% a year between 2012 and 2016. However, despite it software being critical to both the safety of planes and cars, Lesokhin explains that there is a lack of regulation in this area:

“With the physical maintenance of aircraft, there’s a fairly heavy regimen that they undergo with regularity…there’s companies that do regulated checks, so there’s a checklist that you go through with an airplane every several months. So you check that the physical structure of it is intact and that is something that is regulated…There’s no such regulation around software at all.

“You need to check the software, least against the set of minimal industry standards every time that it’s released into the wild,  into live use. Show, so it needs to be something that’s done as part of the release cycle as opposed to just at regular time periods.”

“This puts passengers at risk”

When it comes to aviation, the risks of software malfunctions is heightened. The the US Federal Aviation Administration has warned that the software problems in the Boeing 737 Max pose a “risk to the flying public”.

Lesokhin believes that Boeing has not handled the situation well:

“My feeling is that the company reacted kind of poorly. It took a long time for them to even really admit that there’s a problem. After the first crash with Lion Air, it almost seems like they tried to cover everything up. I think they realised slowly even with the first issue how profound of a problem this is, and I think that they tried to work on it in the dark, and patch it as much as they could before word got out.

“This puts passengers at risk. There’s no doubt.”

He believes that the company could learn from the response of companies from different industries who have faced software issues:

“Recently we had an outage with a cloud service and internet provider called CloudFlare. They communicated the outage right away to all their users put everything online, the whole status of everything that happened, what was going on, what they were doing to fix it, when they were back up and running. I think that in today’s day and age, especially with something as sensitive, as aviation I think it needed to be a bit more transparent. And perhaps what that hints at is that their executives are not really aware of how profound the software problems are.”

“Regulation is something that we’re going to see”

The importance of rigorous testing, and responding correctly when something goes wrong does not just apply to the aviation sector. With 100s of millions of lines of code in modern connected cars, Lesokhin explains that the increasingly important role of software, and the implications when something goes wrong, also extends to cars:

“The car is put together some components that are manufactured such as the break system, the structure, the steering system…Now that those components have software components in them, when you’re assembling all of those, you have a lot more unpredictability, it becomes a system of systems. You have these different software systems that also have to interact with each other, and it becomes a much more complex issue to manage.”

He believes that the ongoing situation Boeing it facing may lead to greater regulation of the software development and testing process:

“As the industry starts to catch up with any of the same level of insight that you have into your manufacturing process that you need to have into your software development process, at the management level. [Regulation] is something that we’re going to see, it’s just a matter of how quickly that’s going to come and how many unfortunate incidents we’re going to see until that happens.”

Read more: IT glitch grounds US flights: “technical debt of the legacy systems” to blame