IEEE Access

Data-driven Reliability Analysis of Boeing 787 Dreamliner

Journal: IEEE Access

Manuscript ID Access-2019-19424

Manuscript Type: Original Manuscript

Date Submitted by the

27-May-2019
Author:

Complete List of Authors: Pandian, Guru; University of Maryland, Mechanical Engineering

Pecht, Michael; Center for Advanced Life Cycle Engineering
Zio, Enrico; Politecnico di Milano, Department of Energy
Hodkiewicz, Melinda; University of Western Australia, School of
Mechanical and Chemical Engineering

Keywords: Reliability, Lithium, Aircraft

Subject Category<br>Please
select at least two subject Aerospace and electronic systems, Components, packaging, and
categories that best reflect manufacturing technology, Industrial electronics, Reliability
the scope of your manuscript:

Additional Manuscript
Keywords:

For Review Only

Page 1 of 28 IEEE Access

1
2 Data-driven Reliability Analysis of Boeing 787 Dreamliner
3
4 (Updated based on feedback received for manuscript # Access-2017-07930)
5 Guru Prasad Pandian and Michael Pecht
6 Center for Advanced Life Cycle Engineering (CALCE)
7 University of Maryland
8 College Park, MD, USA, 20742
9
10 Enrico Zio
11 MINES ParisTech, PSL Research University, Sophia Antipolis, France
12
Department of Energy, Politecnico di Milano, Italy
13
Eminant Scholar, Department of Nuclear Engineering,
14
College of Engineering, Kyung Hee University, Republic of Korea
15
16
17 Melinda Hodkiewicz
18 Faculty of Engineering and Mathematical Sciences
19 University of Western Australia
20 Perth, WA, 6009
21
22 Abstract - The Boeing 787 Dreamliner, launched in 2011, was Electric’s GEnx and Rolls Royce’s Trent 1000) [6].
presented as a game changer in air travel. With the aim of Collectively, these and other design changes were introduced
23 producing an efficient, mid-size, wide-body plane, Boeing
24 to lower operating costs, improve fuel efficiency and
initiated innovations in product and process design, supply
25 cruising speeds, and reduce maintenance costs [7].
chain operation and risk management. Nevertheless, there were
The path to profitability and realization of these aims
26 reliability issues from the start and the plane was grounded by
was tortuous for Boeing [8]. The grounding of the 787 in
27 the U.S. Federal Aviation Administration (FAA) in 2013, due to
safety problems associated with Li-ion battery fires. This paper 2013 focused the spotlight not just on the Li-ion batteries,
28
chronicles events associated with the aircraft’s initial reliability but also on other issues that came to light with the Critical
29 challenges. The manufacturing, supply chain, and Systems Review Team (CSRT) report [9]. Furthermore, the
30 organizational factors that contributed to these problems are recent fatal incidents involving the 737 MAX have refocused
31 assessed based on FAA data. Recommendations and lessons the attention on the reliability of Boeing’s aircraft. In
32 learned are provided for the benefit of engineers and managers particular, the New York Times reported that Boeing has been
33 who will be engaged in future complex systems developments.
fostering a culture of pushing products in the market faster
34 rather than ensuring product quality, especially in the South
35 Keywords: Reliability, Dispatch Reliability, Reliability
Carolina factory where the 787s are manufactured [10]. A
36 Growth, Boeing 787, Dreamliner, Lithium-ion battery
study and interviews conducted with current and former
37 employees of Boeing, ranging from floor technicians to
I. INTRODUCTION
38 quality managers, suggests that the quality of Boeing aircraft
39 has become compromised [10] and quality issues have
40 Boeing, one of the world’s largest aerospace companies, spread to defense aircraft as well. According to CNN, the
41 manufactures products that include commercial aircraft, and U.S. Air Force has been returning some of the delivered
42 defense, space, and security systems. Boeing’s commercial aircraft and has even halted deliveries of aircraft due to the
aircraft business has been in service for nearly 100 years, and
43 ongoing quality problems [10].
its current fleet includes the 737, 747, 767, 777, and 787
44 This paper focuses on failure data released in the CSRT
families [1]. The Boeing 787 Dreamliner was introduced to report [9], NTSB reports [11,12,13], and the journals
45
the market in 2011 as a mid-size, dual-aisle, wide-body referenced in this paper. Using these reports and the incident
46 aircraft. Boeing marketed the 787 as a revolutionary aircraft,
47 reports from the aviation community portals, we have
with an array of new features that would increase fuel collected data to support a reliability growth analysis. This is
48 efficiency by 20% and improve passenger comfort [2]. a first-of-a-kind study of the reliability of the 787 aircraft and
49 Some of the key design innovations in the 787 included provide technical insights into potential contributing factors.
50 the use of composite materials in the wings and fuselage [3]; The paper is organized as follows. Section II is a
51 Li-ion batteries to power up aircraft systems even before the chronology of events before and after the order given by the
52 engine has started, to provide backup to critical loads and FAA to ground the 787 fleet and includes a discussion of the
53 support of battery-only braking; and a no-bleed electrical review conducted by the FAA and Boeing. The development
54 system architecture [4]. It was also the first time that Boeing of the data set to support the reliability growth analysis is
55 replaced the traditional pneumatic system with an electrical described in Section 3. In Section 4, the potential
56 power-generating system for starting the engine, anti-icing contributors to the reliability issues experienced by Boeing
57 the wings, and maintaining cabin pressure [5]. Boeing are identified. Section 5 presents the lessons learned.
58 incorporated the ability to use two types of engines (General
59
60 For Review Only
 IEEE Access Page 2 of 28

1 II. CHRONOLOGY OF EVENTS Table A.1 in the Appendix presents a list of the 787’s
2 technical issues reported to the authorities and/or in the press,
3 The first 787 was shipped in the first quarter of 2011, prior to the 2013 grounding of the plane due to the Li-ion
4 with two to follow in the second quarter. By the end of 2012, battery problem. For each event, the authors have attempted
5 49 aircraft had been delivered, primarily to Japanese Airlines to identify the system and component failure modes. This
6 [14]. Table 1 shows the number of 787 aircraft sold in each information suggests there was a range of different
7 quarter until 2016. component failure modes responsible for the failure events.
8 Furthermore, the 787’s problems persisted even after its
9 Table 1 Boeing 787 Deliveries in Quarters “relaunch”. Operational problems between September 2013
10 and January 2016 are shown in Table A.2 in the Appendix.
11 FY In July 2014, after three months of redesign and
12 Quarter requalification of batteries, Boeing conceded in a press
13 No. Quarter Period Deliveries release that the reliability of the 787 was below their initial
14 3 Jul-Sept 2011 1 expectations and below that of their earlier 777 model. At the
15 same time, they once again attempted to reassure
4 Oct-Dec 2011 2
16 stakeholders that they and their suppliers had already
17 1 Jan-Mar 2012 5 identified suitable corrective actions and initiated or fully
18 2 Apr-Jun 2012 6 implemented them.
19 3 Jul-Sept 2012 12
20 III. RELIABILITY GROWTH ANALYSIS
21 4 Oct-Dec 2012 23
22 1 Jan-Mar 2013 1 Reliability Growth Analysis (RGA) is used in modeling,
23 designing, and improving repairable systems. It is intended
2 Apr-Jun 2013 16
24 to prove the reliability performance of a new or existing
3 Jul-Sept 2013 23 product, component, or system over time. To assess this
25
26 4 Oct-Dec 2013 25 growth, we examined failure events reported in commercial
aircraft journals and the NTSB database (listed in Table A.1
27 1 Jan-Mar 2014 18
28 and Table A.2). As the data on the life (time in service) of
2 Apr-Jun 2014 30 the components responsible for the events of Table A.1 and
29
30 3 Jul-Sept 2014 31 Table A.2 are not available, we use the count of events per
31 4 Oct-Dec 2014 35 month (based on reports in [18][19]) and make the
32 assumption that all defective components are replaced.
1 Jan-Mar 2015 30
33 Based on this, a dataset of the number of events per month
2 Apr-Jun 2015 34 has been created and is shown in Table A.3 in the Appendix.
34
35 3 Jul-Sept 2015 37 To determine the total time on test for the aircraft fleet, the
following additional assumptions were made:
36 4 Oct-Dec 2015 34
1. Aircraft hours is based on number of deliveries by
37 1 Jan-Mar 2016 30 Boeing, as reported in their official orders and
38
2 Apr-Jun 2016 38 deliveries information page. For aircraft delivered
39
40 3 Jul-Sept 2016 36 in one quarter, it is assumed that they do not go into
41 service until the following quarter.
42 In July 2012, the Japanese airline ANA grounded five of 2. For failure events, it is assumed that a failure in the
43 its 787s due to potential corrosion risk in some of the engine quarter occurs at the end of that quarter.
44 parts [15]. This was followed by even bigger problems 3. The operational period of the aircraft is assumed to
45 revealing themselves in the form of fires in 787 aircraft. Two be 50% (half of the number of hours in 90 calendar
46 airplane fires associated with the Li-ion batteries of the plane days), based on the report in [20] and [21].
47 forced the grounding of the worldwide fleet on January 16,
48 2013. The specifics of the battery failure are described in
49 [16]. In congressional hearings, Boeing and its suppliers
50 admitted that despite a significant engineering effort of
200,000 hours, they could not identify the root cause of the
51
problem. Nevertheless, Boeing made changes, including a
52
revision of the internal battery components to minimize the
53
chances of initiating a short circuit, as well as better
54 insulation of the cells and the addition of a new containment
55 and venting system [15]. On March 12, after less than one
56 month of testing, the FAA accepted Boeing’s redesign.
57
58
59
60 For Review Only
Page 3 of 28 IEEE Access

1
in order to improve dispatch reliability. These retrofits are
2 100
changes to the design of the faulty component, as well as
3
Cumulative no. of events
updates made in the practices of manufacturers, airline
4 80 operators, the airports, and regulators.
5 Mathematically, the Crow-AMSAA model is a non-
6 60 homogeneous Poisson process (NHPP), which gives the
7 probability of occurrence of n failures within time T [22],
8 40
9 𝛽
((−𝜆𝑇 𝛽)^𝑛) 𝑒 −𝜆𝑇
10 P(N(T)=n) =
20 𝑛!
11
12
0 where N(T) is the random variable ‘number of failures
13
0 1 2 3 4 occurred up to time T’, and λ and β are parameters to be
14
estimated, based on the available failure event data. The
15 6
Cumulative flight time (x10 hours)
Maximum likelihood estimation (MLE) technique is a robust
16 way to proceed for the estimation of the parameters [23] and
17 Figure 1 Number of flight time hours vs. failure events of has been used also here. By grouping the data, the number
18 the 787 aircraft. of failures in each quarter has been used to conduct the
19 analysis. The results of the analysis are shown in Table 2.
20
21
22 100 Table 2 RGA of 787 post-service re-entry period
23
24
MTBF (x103 hours)

Parameters
25
Model Crow-AMSAA (NHPP)
26
27 Analysis MLE
10
28 Beta 0.70
29 Lambda (hr) 0.002
30 Growth Rate 0.3
31 DMTBF (hr) 70204
32 1 DFI 1E-5
33 5 60 720 8640
34 Statistical Tests
Cumulative flight time (x103 hours)
35 Significance Level 0.1
36 Figure 2 Duane plot of flight time vs. cumulative mean time Chi-Sq Passed
37 between failures (MTBF).
38 The value of β of 0.7 (< 1) in Table 2 suggests a decrease
39 in failure rate over time and is indicative of reliability
Figure 1 is a time-cumulative event plot of event data
40 growth. Examining Figure 2, it can be seen that the reliability
41 from Table A.1 and Table A.2 and other databases of the 787 aircraft (as measured by the MTBF) was
42 mentioned above.. We note that the slope of the plot deteriorating until the grounding and started improving after
43 decreases with increasing time. This is indicative of the aircraft returned to service (fourth data point). Other
increasing reliability. The Duane plot in Figure 2 shows the parameter estimates such as DMTBF (demonstrated or
44
trend of cumulative mean time between failures (MTBF)
45 instantaneous mean time between failures) and DFI
over the flight hours. It can be seen that there is a sharp
46 (demonstrated or instantaneous failure intensity) are also
inflection point at around 115,000–160,000 hours, which
47 corresponds to the period in the first quarter of 2013. The
reported. Their positive values follow from the corrective
48 actions taken by Boeing during the aircraft grounding period,
approximately straight line after the inflection point suggests as well as due to a somewhat natural reduction of those
49 that the data are consistent with an NHPP (non-homogenous
50 problems that typically emerge during the initial stages of the
Poisson process) power law model which allows us to model aircraft operation.
51 the reliability growth using the Crow-AMSAA model [22].
52 The Crow-AMSAA model is generally used to assess Another metric used in aviation to identify component
53 reliability growth during development testing. One of the level reliability is the mean time between unscheduled
54 assumptions of the Crow-AMSAA model is that design removals (MTBUR), which is related to those maintenance
55 changes are applied when failures are found, and thus the activities carried out on an aircraft but that were not part of
56 failure data is also indicative of an updated design the scheduled maintenance:
57 configuration. This is not exactly true in practice, but there
58 are retrofit campaigns that are completed on the entire fleet
59
60 For Review Only
 IEEE Access Page 4 of 28

1
𝑀𝑇𝐵𝑈𝑅 The following sub-sections describe Boeing’s practices
2 𝑁𝑜. 𝑜𝑓 𝑓𝑙𝑖𝑔ℎ𝑡 ℎ𝑜𝑢𝑟𝑠 ∗ 𝑢𝑛𝑖𝑡𝑠 𝑖𝑛𝑠𝑡𝑎𝑙𝑙𝑒𝑑 𝑝𝑒𝑟 𝑎𝑖𝑟𝑐𝑟𝑎𝑓𝑡 in planning and managing the development cycle and supply
3 =
𝑁𝑜. 𝑜𝑓 𝑢𝑛𝑠𝑐ℎ𝑒𝑑𝑢𝑙𝑒𝑑 𝑟𝑒𝑚𝑜𝑣𝑎𝑙𝑠 𝑑𝑢𝑟𝑖𝑛𝑔 𝑡ℎ𝑎𝑡 𝑝𝑒𝑟𝑖𝑜𝑑 chain, the challenges with information sharing with a tiered,
4 globally-dispersed supplier base, with developing a proper
5 diagnostics and prognostics approach, with testing of new
However, we were not able to find any data on the
6 maintenance activities or the components that were technologies, and with oversight of a complex product
7 removed/replaced. In principle, this data should be made development process.
8 available to the public by the airlines and the FAA. These factors are identified as potential causes of the
9 operational problems. Furthermore, these deficiencies are
10 IV. CRITICAL SHORTCOMINGS IN BATTERY CERTIFICATION seldom independent of each other and can have a
11 compounding effect on product reliability.
12 Li-ion batteries (LIBs) were used to power the auxiliary
13 power units and other selected electrical/electronic A. Short Development Cycle and Highly Complex Supply
14 equipment during ground and flight operations to a larger Chains
15 extent in the 787 than in Boeing’s predecessors. Boeing was
16 Boeing intended to reduce to four years the development
required to perform safety assessment for its LIBs as per the period of the 787 (its predecessor 777 was developed in six
17 FAA’s Special Conditions 25-359-SC, “Boeing Model 787- years) and, at the same time, reduce the development costs
18 8 Airplane; Lithium-Ion Battery Installation”. Although from $10 billion to $6 billion [25]. To do so, Boeing decided
19 Boeing did pass all the requirements set by the FAA, there to adopt a new supply chain and product development
20 were shortcomings in the criteria set for failure and guidance structure. This resulted in a new supply chain structure of
21 on assumptions that manufacturers could use in their testing. approximately 50 tier-1 strategic partners, and many more
22 These assumptions were not necessarily supported by tier 2, 3, and 4 suppliers, which they would have little or no
23 engineering rationale and led Boeing to pass the qualification say over. On top of this, 30% of the supply chain was
24 tests. For example, there was an assumption that the internal outsourced to manufacturers outside the USA [25].
25 short circuiting in a cell would only cause that cell to vent The supply chain structure was responsible for the 2.3
26 and not lead to thermal runaway [11]. The battery incident in million parts required to build and assemble the aircraft [27].
27 Japan Airlines showed that Boeing did not put in place The tier-1 partners, such as Alenia Aeronautica (Italy),
28 mitigation strategies to avoid or contain the consequences of Messier-Dowty (France), Rolls-Royce (Britain), and
29 this assumption, were it to prove wrong in practice. Mitsubishi Heavy Industries (Japan), served as integrators
30 A confounding factor was the FAA did not consider responsible for assembling entire subsystems, each having its
thermal runaway to be a potential consequence of cell short
31 own specific supply chain [26].
circuit. Hence, FAA certification engineers did not require
32 The time and cost of production was intended to be
thermal runaway testing as part of compliance
33 reduced by delegating the design, development, and
demonstration. This contributed to a lack of clarity in
34 guidance to certification engineers on translating specific
component manufacturer selection process to sub-system
35 worst-case scenarios to compliance deliverables, such as
suppliers [7]. The tier-1 partners would be responsible for
36 delivery of complete sections of the aircraft to Boeing, who
which test procedure to follow and which test reports to would then perform the final assembly [7].
37 provide in the certification plan. In addition, there were
38 The rationale behind this business strategy was that the
manufacturing defects and absence of thermal management best process skills were increasingly being found outside
39 systems. There were also inconsistencies found in the
40 Boeing factories in the USA, according to Mike Bair, then
Electric Power System (EPS) safety assessment provided by vice-president of the 787 program [6]. This created new
41 Boeing with respect to the compliance with the FAA supplier bases which were either new to Boeing or new to the
42 Advisory Circular (AC) 25.1309, “System Design and aircraft industry as a whole. This included the lithium-ion
43 Analysis” [11]. battery (LIB) manufacturer GS Yuasa, which was selected
44 Eventually, Boeing redesigned the battery system and by Thales Avionics to supply batteries for powering auxiliary
45 had it approved by the FAA. The FAA issued a new devices. As will be discussed later, the inexperience of GS
46 airworthiness directive to install the redesigned batteries on Yuasa in dealing with aircraft products led to inappropriate
47 all 787 airplanes to be returned to service. specification of batteries based on the reliability data from
48 other industrial applications.
49 V. RELIABILITY ASSESSMENT This new supply chain structure was a departure from
50 traditional practice, in which the manufacturer was
51 From the various documents and trends, it can be argued responsible for the assembly of the major subsystems. This
52 Boeing did not adopt an effective Reliability Program Plan tiered system is a complex structure of interacting technical
53 (RPP), where best practice tasks are implemented to produce and organizational artifacts. The new and more complex
54 reliable products [24]. Boeing opted to widen its supplier supply chain led to intricacies in assembling many
55 base and reduce costs by including manufacturers who were components from different suppliers into a large subsystem
56 new to the aircraft development industry. The events that led that was manufactured by a different supplier. For example,
57 to delays during manufacturing and failures during operation Boeing contrived a modular design for the 787 to enable
58 are a testament to Boeing’s flawed practice. engine interchangeability between Rolls-Royce and GE
59
60 For Review Only
Page 5 of 28 IEEE Access

1
engines on the same aircraft. As a result, the interchanging identify the root cause of the thermal runaway event. Many
2 process actually took 15 days against the intended 24 hours, issues such as production quality problems of contamination,
3 because of technical incongruities due to multiple supply electrolyte evaporation, and over-voltage loads were
4 chain participants [9]. Similarly, several “shimming” issues hypothesized, but were not proven to be conclusive [12]. The
5 were found when trying to assemble parts from different flight data recorder (FDR) collected 363 different
6 suppliers, due to the lack of conformity to tolerances and measurements before and after the battery fire incident of
7 understanding of design requirements [9]. which only two, the DC feed load current and the APU
8 battery DC bus voltage, were directly related to the faulty
9 B. Lack of Accurate and Timely Information Sharing batteries [12]. The FDRs were not designed to collect
10 individual cell data from the battery management system
11 Since the supply chain was spread across the globe, there (BMS), which could have given insight into the specific
12 were challenges in synchronizing changes to the design battery that caused the thermal runaway.
13 requirements down through the supply chain and production
14 information back up through the supply tiers [9]. Boeing D. Lack of Valid Testing on Innovative Technologies
15 tried addressing this challenge by implementing a web-based
tool called “Exostar”, which allowed the suppliers to enter The FAA review team observed that both existing and
16
their relevant information such as design and production new technologies incorporated in the 787 aircraft were not
17
requirements and production status of the components. tested for the specific 787 application [need ref]. The success
18
Contrary to the intended effect, this data sharing process did of these technologies, either in other applications or in
19 not improve the visibility across the supply chain due to the previous Boeing aircraft, was assumed to be carried over to
20 discrepancies in accuracy and delay and misinterpretation of the 787 as well [9]. LIBs, which have become one of the
21 data from the tool. The lack of familiarity of aerospace major concerns for 787 reliability, were adopted from
22 manufacturing standards and cultural differences in terms of another industrial application and there were no failures
23 workmanship among suppliers from various locations reported in such application. Based on the data from this
24 contributed to this inefficiency in data sharing [7][9]. industrial application, GS Yuasa assumed a Poisson
25 For example, the FAA found discrepancies in the distribution for the LIB failure time and estimated a failure
26 dissemination of requirements for the primary electrical rate of less than 1 failure in 10 million flight hours [28].
27 power panel from Boeing to United Technologies Aerospace However, by the time the 787 was grounded in 2013, the
28 Systems (UTAS), then from UTAS to sub-tier supplier failure rate of the LIBs was 3 in 250,000 hours. The estimate
29 Equipment et Construction Electrique (ECE) and from ECE of less than 1 in 10 million hours was based on a 60%
30 to its printed circuit board component supplier. The FAA confidence interval, while a 90% confidence interval or
31 review team found deficiencies also in the process of passing higher is usually suggested for critical reliability applications
32 requirements down the levels of suppliers leading to 1) weak such as those of avionics [28].
33 design, which then manifested as part malfunctions once The level of DO-160 testing required was established at
34 they entered service, 2) variability in manufacturing, and 3) the time Boeing submitted the application to design, test, and
35 anomalous behavior of parts. build the 787 to the FAA, which would have been around
36 The bottom-up information flow was similarly hindered 2003 or 2004. Guidance on how to test LIBs was issued in
37 as seen in the instance where Vought, a tier-1 supplier, AC 20-184 in October 2015. This could have led to a
38 entered into a contract with Advanced Integration situation where technology outpaced the regulations.
Technology (AIT) as a tier-2 supplier to aid in integrating While the new technologies were given slack in testing,
39
systems. AIT was assigned the responsibility of quality of the processes that were considered “stable” were
40
communicating with other tier-2 and tier-3 suppliers on let to be inspected by the same technician who carried out by
41
behalf of Vought [7]. But due to cultural and geographical the process [29]. One of the former Department of
42 differences, the suppliers did not always communicate the Transportation inspectors stated that in many cases these
43 proper information. These differences led to delays in self-inspections were actually not conducted and were
44 supplying parts, which were not visible to Boeing and kept passed on by the workers who executed the process. This
45 Boeing from responding to delays in a timely manner, and in kind of flawed practices have led to many mistakes in the
46 understanding requirements changes. production line as per the Boeing workers [29].
47 Finally, the reliability assumptions for the electronics
48 C. Lack of Relevant Data and the testing of the electronics, including the battery, is of
49 grave concern, in part because Boeing has traditionally
50 Data collection for system health and usage monitoring assumed the constant failure rate and used the outdated
51 (HUMS), and prognostics and systems health management military handbook 217 for its reliability and safety
52 (PHM), provides the opportunity to assess the state of calculations. This handbook was last updated in 1997, and
53 operation of the airplane and its components, and predict the was considered inaccurate and unacceptable for use by the
54 reliability and safety [24]. However, this was not well military by a National Academy of Sciences study [31],[30]
55 executed in the Boeing 787 aircraft. For example, the FAA, and for aviation industry as well. [31]. The handbook based
56 Boeing, and Japan’s Transport Ministry conducted a method uses field failure data of un-related applications to
thorough analysis on the root cause of failure of the lithium- determine a point reliability value of aircraft without
57
ion batteries in the 787 [12]. However, they were unable to considering its specific complex use conditions.
58
59
60 For Review Only
 IEEE Access Page 6 of 28

1
changes to its engineering and business structure. However,
2 E. Difficulty in Fault Detection the problems that then occurred in the aircraft’s operation
3 have emphasized the need for strengthening the focus on
4 The 787 aircraft is a complex system with about 2.3 quality and for developing a reliability-centric approach to
5 million parts supplied and assembled from manufacturers supplier selection, training, and production management.
6 around the globe [25]. The CSRT noted [9] that when an In this regard, some practical guidelines follow.
7 issue was reported during the service of the aircraft, the Suppliers should consider IEEE 1332-2012, JA1000-
8 suppliers removed the parts they deemed to be defective, but 201205, and IEEE 1624 in the development stages. The
9 often found there was no fault. This could be due to the IEEE 1332-2012 document provides a standard set of
10 intermittent nature of electronics systems [32,33,34], In fact, reliability program objectives for use between customers and
11 it has been noted that cases of no-fault-found on airplanes producers, or within product development teams, to express
12 can be as high as 80% and Boeing often replaces electronics reliability program requirements early in the development of
13 line-replaceable units (LRUs) with LRUs that were flagged electronic products. SAE adopted the IEEE standard and
14 as failed but were no-fault-found once it was removed. This released it as JA1000, which is followed by various industry
15 practice is problematic considering the wearout and sectors. OEMs should take necessary steps to validate the
intermittent failure nature of electronics.
16 ability of the suppliers to meet the reliability requirements.
17 IEEE 1624-2008 Standard for Organizational Reliability
F. Lack of Balance Between Autonomy and Oversight Capability provides guidelines for assessing, in a systematic
18
19 manner, the effectiveness of an organization’s reliability
In Boeing’s 787 development model, the integration of
20 practices in ensuring or exceeding product reliability
sub-assemblies and final assembly was critical for the
21 requirements. Avoiding misinterpretations and having
hardware and software from different suppliers to fit together
detailed information on inputs and assumptions for
22 and operate properly. This required a balance of providing
predicting the reliability of hardware is essential in
23 autonomy to the suppliers to meet the design requirements
understanding the risks associated with using the prediction
24 and keeping a close oversight on the supplier processes.
results for future product integration and ensuring overall
25 However, Boeing did not opt for an on-site supplier support
system reliability.
26 which led to absence of a bi-directional technical
Manufacturers can ensure consistent prediction and
27 communication to keep the quality of the parts and sub-
reporting of reliability of hardware across product
28 assemblies in check [35]. For example, Mitsubishi Heavy
development teams by following established standard
29 Industries stated that Boeing did not adopt Mitsubishi’s early
procedures such as IEEE 1413-2010. IEEE 1413 aids in
30 testing and diagnosis principle [35], which in turn led to
providing sufficient information on inputs, assumptions, and
31 design flaws being carried over to next tiers of assemblies
uncertainties in the estimated reliability. Further, aerospace
32 and eventually to the aircraft operation on field.
standards such as AS9100, based on ISO 9001:2015, are
33 dedicated to ensuring product quality and process
VI. CONCLUSIONS
34 management for aircraft parts manufacturers.
35 Finally, while the grounding of the 787 in 2013 focused the
36 Evaluating the reliability of a complex system made of spotlight on Li-ion batteries, and on the complexity of the
37 multiple components, like an aircraft, is very difficult supply chain, there were also concerns pertaining to how the
38 especially during the development stages. As a matter of airplane could be certified within three months without
fact, many factors contribute to the difficulty of evaluating
39 knowing the root causes of failure. This is more so relevant
reliability during product development: tight scheduling for today, in light of the recent concerns with the 737 MAX, and
40
contracted deliveries, requirements on testing and validation, the role of Boeing and the FAA in understanding and
41
pressures for cost reduction, multiple tiers of suppliers of the evaluating reliability and safety issues.
42 many parts constituting the system, challenges with accurate
43 and timely data sharing, innovative technologies requiring VII. ACKNOWLEDGEMENT
44 specific testing procedures, and others.
45 In this paper, operational problems with the Boeing 787
46 We would like to thank Ms. Rhonda Walthall, Associate
aircraft have been analyzed to identify different
47 Director, Aftermarket Digital Strategies at UTC Aerospace
manufacturing and organizational factors that have impacted
48 Systems, for her invaluable comments that helped improve
the reliability performance of such a complex system in
49 the quality of paper.
operation. Reliability metrics, such as cumulative mean time
50 between failure (MTBF) and cumulative number of failure
51 events, have been estimated from publicly available data. A
VIII. REFERENCES
52 reliability growth analysis has been performed to study also
53 the impact of corrective actions carried out by Boeing on the
54 performance of the 787 aircraft. 1. Boeing, "Boeing in Brief," [Online]. Available:
55 Undoubtedly, there were enormous challenges inherent http://www.boeing.com/company/general-info/
56 in the development of a new product like the 787. And with 2. Boeing, "Boeing 787 Dreamliner - Design Highlights,"
57 the increase in reliability as one of the goals of the 787 Boeing, 2011. [Online]. Available:
58 project development, Boeing invested significantly in http://www.boeing.com/commercial/787/#/design-
59
60 For Review Only
Page 7 of 28 IEEE Access

1
highlights/visionary-design/composites/one-piece- royce-787/ana-grounds-dreamliners-citing-engine-
2
barrel/. [Accessed September 2015]. corrosion-risk-idUSLNE86M01M20120724
3
3. Boeing. “Boeing 787 From the Ground up”, [Online]. 16. N. Williard, W. He, C. Hendricks, and M. Pecht,
4
Available: “Lessons Learned From the 787 Dreamliner Issue on
5
https://www.boeing.com/commercial/aeromagazine/ar Lithium-Ion Battery Reliability”, Energies, Vol. 6,
6
ticles/qtr_4_06/article_04_2.html Issue 9, pp. 4682-4696, 2013.
7
4. Boeing, “Boeing 787 From Ground up”, [Online]. 17. G. Norris, “Boeing reveals 787 battery fix details”,
8
Available: Aviation week and Space Technology, 2013. [Online].
9
10 https://www.boeing.com/commercial/aeromagazine/ar Available: http://aviationweek.com/awin/boeing-
11 ticles/qtr_4_06/article_04_3.html reveals-787-battery-fix-details
12 5. Boeing, “787 No-bleed Systems: Saving Fuel and 18. “Boeing 787 Dreamliner: a timeline of problems”, The
13 Enhancing Operational Efficiencies”, [Online]. Telegraph. [Online]. Available:
14 Available: http://www.telegraph.co.uk/travel/comment/Boei
15 https://www.boeing.com/commercial/aeromagazine/ar ng-787-Dreamliner-a-timeline-of-problems/
16 ticles/qtr_4_07/article_02_1.html 19. “Aviation Incidents and Accidents, Airports and
17 6. Boeing, “787 Propulsion System”, [Online]. Available: more”, AeroInside. [Online]. Available:
18 https://www.boeing.com/commercial/aeromagazine/ar https://www.aeroinside.com/incidents/type/b788/boei
19 ticles/2012_q3/2/ ng-787-8-dreamliner
20 7. C. S. Tang, J. D. Zimmerman and J. I. Nelson, 20. C. Sloan, "787 Program Update: Delivery and Dispatch
21 "Managing New Product Development and Supply Reliability Finally Catching Up with Demand,"
22 Chain Risks: The Boeing 787 Case," Supply Chain Airways News, June 2015. [Online]. Available:
23 Forum, vol. 10, no. 2, 2009. http://airwaysnews.com/blog/2015/06/10/787-
24 8. K. Lu, “The Future of Metals”, American Association program-update-delivery-and-dispatch-reliability-
25 for the Advancement of Science, Vol. 328, 2010. finally-catching-up-with-demand/. [Accessed 25
26 9. Boeing 787-8 Critical Systems Review Team, "Boeing September 2015].
27 787-8 Design, Certification, and Manufacturing 21. G. Norris, "With Better Dispatch Reliability, Boeing
28 Systems Review," Federal Aviation Administration, 787 Deliveries Reach 350," Aviation Week, 24
29 Washington DC, 2014. Novemeber 2016. [Online]. Available:
30 10. D. Cole, “New York Times: Boeing’s South Carolina
http://aviationweek.com/commercial-aviation/better-
31 plant faces production issues that ‘have threatened to
dispatch-reliability-boeing-787-deliveries-reach-350.
32 compromise safety’”, CNN, April, 2019. [Online].
Available: [Accessed 29 February 2016].
33 22. Reliaiwki, “Crow-AMSAA (NHPP)”. [Online].
34 https://www.cnn.com/2019/04/20/politics/boeing-
south-carolina-plant/index.html Available: http://reliawiki.org/index.php/Crow-
35 AMSAA_(NHPP)
11. NTSB, “Auxiliary Power Unit Battery Fire Japan
36
Airlines Boeing 787-8, JA829J Boston, 23. Weibull.com, “Reliability Hot Wire”. [Online].
37
Massachusetts”, 2013. [Online]. Available: Available:
38
39 https://www.ntsb.gov/investigations/AccidentReports/ https://www.weibull.com/hotwire/issue9/relbasic
40 Reports/AIR1401.pdf s9.htm
41 12. Interim Factual Report: Boeing 787-8, JA829J Battery 24. M. Pecht, M. Kang, “Prognostics and Health
42 Fire; Case Number DCA13IA037; National Management of Electronics: Fundamentals, Machine
43 Transportation Safety Board, Office of Aviation Learning, and Internet of Things”, 2nd ed. Hoboken, NJ:
44 Safety: Washington, DC, USA, 2013. Available online: John Wiley & Sons; 2018.
45 http://www.ntsb.gov/investigations/2013/boeing_787/ 25. Boeing –World Class Supplier Quality. [Online].
46 DCA13IA037%20interim% 20factual%20report.pdf Available:
47 (accessed on 6 September 2013). http://787updates.newairplane.com/787-
48 13. NTSB Report, NTSB Identification: DCA13WA159. Suppliers/World-Class-Supplier-Quality
49 [Online]. Available: 26. FACTBOX, “Global supply chain for Boeing's 787”,
50 http://www.ntsb.gov/_layouts/ntsb.aviation/brief.aspx Reuters, 2008. http://www.reuters.com/article/us-
51 ?ev_id=20130924X54859&key=1
boeing-suppliers-idUSL715516020080908
52 14. Boeing, "Orders and Deliveries," Boeing, [Online].
27. R. Tinseth, “Our Supply Chain”, Randy’s Journal-
53 Available: http://www.boeing.com/company/about-
Boeing, 2013. [Online]. Available:
54 bca/#/orders-deliveries. [Accessed 16 March 2016].
http://www.boeingblogs.com/randy/archives/2013/02/
55 15. Y. Kubota, R. Jones, “ANA grounds Dreamliners, supply_chain.html
56 citing engine corrosion risk”, Reuters, 2012. [Online].
57 Available: https://www.reuters.com/article/uk-rolls-
58
59
60 For Review Only
 IEEE Access Page 8 of 28

1
28. All Nippon Airways Co. Ltd., “Aircraft Serious 40. G. Waldron, “ANA 787 suffers hydraulic leak before
2 Incident Investigation Report”, Japan Transport Safety takeoff”, Flight Global, September 2012. [Online].
3 Board, 2014. Available:
4 29. D. Wren and G. Smith, “Boeing SC lets mechanics
5
https://www.flightglobal.com/news/articles/ana-
inspect their own work, leading to repeated mistakes,
6 787-suffers-trent-1000-pump-failure-before-take-
workers say”, The Post and Courier, May 6, 2019.
7 [Online]. Available: off-376086/
8 https://www.postandcourier.com/business/boeing-sc- 41. AeroInside, “Aviation Incidents and Accidents,
9 lets-mechanics-inspect-their-own-work-leading- Airports and more”, 2014. [Online]. Available:
10 to/article_5ccc89ce-6cea-11e9-af3c- https://www.aeroinside.com/item/1613/united-b788-
11 bfe34127eb85.html near-new-orleans-on-dec-4th-2012-electrical-
12 30. National Research Council, “Reliability Growth: problems-causing-concerns-of-electrical-heat-on-
13 Enhancing Defense System Reliability”, The National board
14 Academies Press, 2014. 42. AeroInside, “Aviation Incidents and Accidents,
15 31. G. Pandian, D. Das, C. Li, E. Zio, M. Pecht, “Critique Airports and more”, 2014. [Online]. Available:
16 of reliability prediction techniques for avionics https://www.aeroinside.com/item/1667/qatar-b788-
17 applications”, Chinese Journal of Aeronautics,Vol. 31, near-doha-on-dec-8th-2012-generator-failure
18 issue 1, pp. 10-20, 2018. 43. Business, “Boeing Dreamliner: Two JAL flights
19 32. Qi. Haiyu., S. Ganesan and M. Pecht, “No-fault-found diverted after glitches”, BBC, 2013. [Online].
20 and Intermittent Failures in Electronic Products”, Available: http://www.bbc.com/news/business-
21 Microelectronics Reliability, Vol. 48, Issue 5, pp. 663- 24471093
22 674, May 2008. 44. AeroInside, “Aviation Incidents and Accidents,
23 33. Roozbeh Bakhshi, Surya Kunche and Michael Pecht, Airports and more”, 2014. [Online]. Available:
24 “Intermittent failures in hardware and https://www.aeroinside.com/item/3605/air-india-
25 software”, Journal of Electronic Packaging, Vol. 136, b788-near-berlin-on-jan-19th-2014-complete-loss-of-
26 Issue 1, pp 011014-1, DOI:10.1115/1.4026639. March transponders
27 2014. 45. AeroInside, “Aviation Incidents and Accidents,
28 34. G. Zhang, C. Kwan, R. Xu, N. Vichare, and M. Airports and more”, 2014. [Online]. Available:
29 Pecht, “An Enhanced Prognostic Model for https://www.aeroinside.com/item/3665/air-india-
30 Intermittent Failures in Digital Electronics”, IEEE b788-near-kuala-lumpur-on-feb-5th-2014-all-flight-
31 Aerospce Conference, Big Sky, MT, March 2007. management-computers-failed
32 35. S. V. Gudmundsson, “Global Partnering: The Boeing 46. AeroInside, “Aviation Incidents and Accidents,
33 787 Dreamliner and Beyond”, 2015. [Online]. Airports and more”, 2014. [Online]. Available:
34 Available:
35 https://www.aeroinside.com/item/4847/air-india-
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2
36 b788-near-bari-on-oct-17th-2014-loss-of-
654993
37 36. C. Harress, “Boeing 787: A Complete Timeline of The
communication
38 47. AeroInside, “Aviation Incidents and Accidents,
Dreamliner’s Legacy of Failure, After Cracks
39 Airports and more”, 2014. [Online]. Available:
Discovered in Wings”, International Business Times,
40 2014. [Online]. Available: https://www.aeroinside.com/item/5234/india-
41 http://www.ibtimes.com/boeing-787-complete- b788-near-budapest-on-jan-22nd-2015-engine-
42 shut-down-in-flight
timeline-dreamliners-legacy-failure-after-cracks-
43 48. AeroInside, “Aviation Incidents and Accidents,
discovered-wings-1560491
44 Airports and more”, 2014. [Online]. Available:
37. J. Ostrower, “Delamination prompts Boeing to inspect
45
787 fleet”, Flight Global. [Online]. Available: https://www.aeroinside.com/item/5369/aeromexi
46
47
https://www.flightglobal.com/news/articles/delaminati co-b788-near-paris-on-feb-23rd-2015-low-crew-
on-prompts-boeing-to-inspect-787-fleet-367793/ oxygen-pressure
48
38. Y. Kubota, R. Jones, “ANA grounds Dreamliners, 49. AeroInside, “Aviation Incidents and Accidents,
49
50 citing engine corrosion”, Reuters, July 2012. [Online]. Airports and more”, 2014. [Online]. Available:
51 Available: http://www.reuters.com/article/uk- https://www.aeroinside.com/item/5468/aeromexi
52 rolls-royce-idUSLNE86M01M20120724 co-b788-near-santa-maria-on-mar-19th-2015-
53 39. National Transportation Safety Board, "Aviation electrical-failure
54 Accident Database & Synopses," FAA-Federal 50. S. Gibbs, “US aviation authority: Boeing 787 bug
55 Aviation Administration, [Online]. Available: could cause 'loss of control', Theguardian, 2015.
56 http://www.ntsb.gov/_layouts/ntsb.aviation/Results.as [Online]. Available:
57 px?queryId=0939a615-9e15-4271-ae13- https://www.theguardian.com/business/2015/may
58 62bb10c4d8f1. [Accessed 20 February 2016].
59
60 For Review Only
Page 9 of 28 IEEE Access

1
/01/us-aviation-authority-boeing-787-dreamliner-
2
3 bug-could-cause-loss-of-control
4 51. AeroInside, “Aviation Incidents and Accidents,
5 Airports and more”, 2014. [Online]. Available:
6 https://www.aeroinside.com/item/5917/air-india-
7 b788-at-melbourne-on-jun-25th-2015-flight-
8 control-technical-issue
9 52. AeroInside, “Aviation Incidents and Accidents,
10 Airports and more”, 2014. [Online]. Available:
11 https://www.aeroinside.com/item/6040/air-india-
12 b788-at-amritsar-on-jul-12th-2015-could-not-retract-
13 left-main-gear
14 53. AeroInside, “Aviation Incidents and Accidents,
15 Airports and more”, 2014. [Online]. Available:
16 https://www.aeroinside.com/item/6961/jal-b788-near-
17 tokyo-on-jan-29th-2016-engine-shut-down-in-flight
18 54. AeroInside, “Aviation Incidents and Accidents,
19 Airports and more”, 2014. [Online]. Available:
20 https://www.aeroinside.com/incidents/type/b788/boei
21 ng-787-8-dreamliner
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60 For Review Only
 IEEE Access Page 10 of 28

1
2
3
4 IX. APPENDIX
5
6 Table A.1 Events associated with the Boeing 787 Dreamliner, November 2011–January 2013
7
8 Date System Component Failure Event Description Ref Airline
9 Mode
10
11
12 November Landing Hydraulic Failure to The ANA-operated flight had [36] All Nippon
13 2011 gear valve open/clos to make a second attempt at Airways
14 e landing using alternate Registration
15 extension backup, after a (ANA):
16 faulty hydraulic valve could JA801A
17 not deploy the landing wheel.
18
February Fuselage Stiffening De- Stiffening rods/shear ties [37] All
19 2012 rods laminatio used to connect the fuselage Dreamliner
20 n skeleton with the skins had Aircraft
21 delaminated from the skins.
22
23 July 2012 Engine Gearbox Corrosio ANA grounded five of its 11 [38] ANA
24 ancillary n Dreamliners, due to corrosion
25 system of parts of the gearbox used
26 to drive ancillary systems in
27 the Rolls Royce Trent 1000
28 engine.
29
30 July 2012 GEnx Fan shaft Fracture One of the shafts that connect [39] Pre-delivery
31 engine the fan and the booster to the
32 low-pressure turbine of the
dual-shaft GEnx engine
33
fractured at the rear end of
34
the threads where the
35
retaining nut is assembled.
36 This occurred during a pre-
37 delivery taxi test.
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 11 of 28 IEEE Access

1
2
3 Date System Component Failure Event Description Ref Airline
4 Mode
5
6
7 September Engine Hydraulic Leak ANA aborted a flight from [40] ANA:
8 2012 ancillary system Okayama Airport after JA801A
9 system detecting smoke-like
10 emission due to misting of oil
11 dripping from hydraulic
12 pump on the hot engine
13 December Control Electrical Alarm Aircraft headed from [41] United
14 2012 system panel Houston to Newark, Airlines -
15 emergency-landed in New Registration
16 Orleans after a false alarm N26902;
17 indicated generator failure. Qatar –
18 Short circuiting on an Registration
19 electrical panel was found to A7-BCA
be the cause of the false
20
alarm. The same problems
21
were reported in one of the
22
Qatar Airways aircraft and
23 another United Airlines
24 aircraft in the same month.
25 December Engine Fuel system Leak FAA ordered an inspection [42] All Boeing
26 2012 ancillary into improperly installed fuel Dreamliners
27 system line connectors after finding
28 that it could result in leaks
29 leading to fuel exhaustion,
30 thermal runaway, engine
31 power loss or shutdown
32
33 January Li-ion Battery Smoke Heavy smoke was found [39] Japan
34 2013 battery emitting from the electronic Airlines
35 system equipment bay in the aft Registration:
36 cabin of a parked aircraft at JA829J
37 Logan International Airport,
38 Boston. The smoke was
39 attributed to the thermal
40 runaway caused by internal
41
42
43
44
45 For Review Only
46
47
 IEEE Access Page 12 of 28

1
2
3 Date System Component Failure Event Description Ref Airline
4 Mode
5
6
7 short circuiting of one of the
8 APU Li-ion battery cells
9
10
11
12
13
14
15
January Li-ion Battery Smoke The flight on its way to [39] ANA:
16
2013 battery Tokyo from Yamaguchi Ube JA804A
17 system Airport received an Engine
18 Indicating and Crew Alerting
19 System (EICAS) message
20 reporting battery failure
21 accompanied by an unusual
22 smell in the cockpit. Battery
23 heating and thermal runaway
24 were reported to be the
25 probable causes for smoke.
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 13 of 28 IEEE Access

1
2
3 Table A.2 Events associated with the Boeing 787 Dreamliner, April 2013–August 2015
4
5 Date System Component Failure mode Event description Ref Airline
6
7
8
9 September 18 Engine Fuel system Alarm The flight powered by two Rolls- [13] LOT Polish
10 2013 ancillary Royce Trent 1000-67B turbofan Airlines
11 system engines experienced a Registration:
12 maintenance status message “ENG SP-LRB
13 FUEL FILTER R” on its way
14 from Beijing to Warsaw Chopin
15 Airport. Later on, a maintenance
16 investigation found that the engine
17 fuel filter in the right engine had
18 not been installed. Further
19 examination revealed that the fuel
20 filter was not installed in the left
engine as well.
21
22
23
24
25 September 19 Engine Fuel system Alarm Following the SP-LRB incident, [13] LOT Polish
26 2013 ancillary this flight was checked the next Airlines
27 system day to reveal that there were no Registration:
28 fuel filters installed in this aircraft SP-LRC
29 too.
30
31 October 10 Electrical Unknown Toilets fail to Aircraft headed from Moscow to [43] JAL: JA832J
32 2013 flush Tokyo had to land midway due to
33 electrical problems. It was
34 reported that toilets did not flush
35 due to an electrical failure.
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
 IEEE Access Page 14 of 28

1
2
3 Date System Component Failure mode Event description Ref Airline
4
5
6
7 January 20 Communicatio Transponder Failure to The aircraft headed from London [44] Air India
8 2014 ns function to Delhi vanished completely from Registration:
9 the secondary ATC radar due to VT-ANE
10 transponder failure. After
11 negotiating with the air traffic
12 authorities, the crew was asked to
13 return to London and fix the issue
14 before flying
15 February 5th Control Flight Failure to The aircraft headed from Australia [45] Air India
16 2014 System management function to India landed midway in Registration:
17 Malaysia due to the failure of all VT-ANJ
18 three flight management control
19 systems.
20 October 17 Communicatio Unknown Failure to The aircraft headed from Delhi to [46] Air India
21 2014 ns function Rome had to be intercepted by the Registration:
22 Italian Air Force due to a lack of VT-ANQ
23 communication. It was later
24 reported that the aircraft had lost
25 its communication capability due
to a technical malfunction.
26
January 22 GEnx Engine Unknown Unknown The flight, which was bound to [47] Air India
27
2015 Mumbai from London, had to Registration:
28
divert to Budapest, Hungary, due VT-ANL
29
to failure of the right engine,
30 General Electric GEnx-1B.
31 February 23rd Air supply Unknown Drop in cabin The aircraft was stopped after [48] Aeromexico:
32 2015 pressure takeoff at FL240, citing a drop in N961AM
33 cabin pressure.
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 15 of 28 IEEE Access

1
2
3 Date System Component Failure mode Event description Ref Airline
4
5
6
7 March 19th Communicatio Transponder Failure to The aircraft that was scheduled [49] Aeromexico
8 2015 ns function from Madrid to Mexico City made Registration:
9 an emergency landing in the N964AM
10 Azores when the crew complained
11 of an electrical failure. The radar
12 data had suggested that the aircraft
13 was no longer able to provide
14 position and Mode-S data with its
15 transponder.
16
17
18
19 April 30, Control Software Failure to In Boeing’s own laboratory [50] All 787s
20 2015 System function testing, conducted after the planes
21 were sold to the customer, it was
22 found that if the generator was
23 operated for around 8 months
24 continuously, the control unit
25 software could overflow and go
26 into a fail-safe mode. This means
27 that the generator could stop
28 working and power to the aircraft
could be cut off, even if the plane
29
were in flight.
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
 IEEE Access Page 16 of 28

1
2
3 Date System Component Failure mode Event description Ref Airline
4
5
6
7 June 25, 2015 Electrical Flight controls [Unknown] The aircraft flying from [51] Air India
8 Melbourne to New Delhi had to Registration:
9 land 70 minutes after takeoff due VT-ANR
10 to a technical issue. It was
11 reported that there was a minor
12 issue in flight controls that did not
13 allow the airline to continue the
14 flight.
15 July 17th 2015 Landing Main Gear Failure to The aircraft landed 45 minutes [52] Air India
16 system function after takeoff due to malfunction of Registration:
17 the left-hand main gear. It was VT-ANV
18 reported that the drag brace
19 actuator had become defective and
20 had to be replaced.
21 January 29th Engine Anti-ice Imbalance on A few of the blades in one of the [53] Japan Airlines
22 2016 ancillary turbine GEnx-1B engines used in a JAL Registration:
23 system aircraft from Vancouver to Tokyo JL17
24 experienced ice formation. This
25 led to partial ice shedding and, in
26 turn, imbalance in the turbine
27 which caused rubbing of the
28 blades onto the fan case surface
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 17 of 28 IEEE Access

1
2
3 Table A.3 Failure Times Calculation (Based on problem reports from [54])
4
5
End of
6 Fiscal End of year
7 year year reported Est. no. Hrs.
8 quarter reported Cumulative cumulative new in flown in Cum. hrs. Events Cum.
9 Year (qtr.) Period Deliveries deliveries deliveries deliveries service qtr. flown in qtr. events MTBF
10 Jul-Sept
11 2011 3 2011 1 1
12 Oct-Dec
13 4 2011 2 3 3 3 1 1095 1095
14 Jan-Mar
15 2012 1 2012 5 8 3 3285 4380 0 0 0
16 Apr-Jun
17 2 2012 6 14 8 8760 13140 0 0 0
18 Jul-Sept
19 3 2012 12 26 14 15330 28470 3 3 9490
20 Oct-Dec
21 4 2012 23 46 49 49 26 28470 56940 5 8 7118
22 Jan-Mar
23 2013 1 2013 1 50 49 53655 110595 8 16 6912
24 Apr-Jun
25 2 2013 16 66 50 54750 165345 6 22 7516
26 Jul-Sept
27 3 2013 23 89 66 72270 237615 9 31 7665
28 Oct-Dec
29 4 2013 25 65 114 114 89 97455 335070 3 34 9855
30 Jan-Mar
31 2014 1 2014 18 132 114 124830 459900 4 38 12103
32 Apr-Jun
33 2 2014 30 162 132 144540 604440 5 43 14057
34 Jul-Sept
35 3 2014 31 193 162 177390 781830 7 50 15637
36 Oct-Dec
37 4 2014 35 114 228 228 193 211335 993165 4 54 18392
38 Jan-Mar
39 2015 1 2015 30 258 228 249660 1242825 6 60 20714
40 Apr-Jun
41 2 2015 34 292 258 282510 1525335 6 66 23111
42
43
44
45 For Review Only
46
47
 IEEE Access Page 18 of 28

1
2
3
4 End of
5 Fiscal End of year
year year reported Est. no. Hrs.
6
quarter reported Cumulative cumulative new in flown in Cum. hrs. Events Cum.
7
Year (qtr.) Period Deliveries deliveries deliveries deliveries service qtr. flown in qtr. events MTBF
8
Jul-Sept
9
3 2015 37 329 292 319740 1845075 2 68 27133
10
Oct-Dec
11 4 2015 34 135 363 363 329 360255 2205330 6 74 29802
12 Jan-Mar
13 2016 1 2016 30 393 363 397485 2602815 7 81 32134
14 Apr-Jun
15 2 2016 38 431 393 430335 3033150 4 85 35684
16 Jul-Sept
17 3 2016 36 467 431 471945 3505095 8 93 37689
18 Oct-Dec
19 4 2016 0 104 467 467 511365 4016460 93 43188
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 19 of 28 IEEE Access

1
2
3 currently a Chair Professor in mechanical engineering and a Professor in
4 applied mathematics with the University of Maryland. In 2015, he received
5 the IEEE Components, Packaging, and Manufacturing Award for visionary
6 leadership in the development of physics-of-failure-based and prognostics-
7 based approaches to electronic packaging reliability. He received the
8 Distinguished Chinese Academy of Sciences President’s International
9 Fellowship. In 2013, he received the University of Wisconsin-Madison’s
10 Guru Prasad Pandian is a Master of Science student at
College of Engineering Distinguished Achievement Award. In 2011, he
11 received the University of Maryland’s Innovation Award for his new
Mechanical Engineering department at University of Maryland, College
12 park. He has been working as a research assistant with Center for Advanced
concepts in risk management. In 2010, he received the IEEE Exceptional
13 Life Cycle Engineering at University of Maryland since Fall of 2015. His
Technical Achievement Award for his innovations in the area of prognostics
14 and systems health management. In 2008, he was awarded the highest
research interests are evaluating reliability of electronic packages and
15 reliability honor, the IEEE Reliability Society’s Lifetime Achievement
specifically reliability of lead-free solders.
16 Award.
17
18
Enrico Zio (M’06) received an M.Sc. degree in nuclear
19
engineering from the Politecnico di Milano, Milano,
20
Italy, in 1991, another M.Sc. degree in mechanical
21
engineering from the University of California, Los
22 Angeles, CA, USA, in 1995, a Ph.D. degree in nuclear
23 engineering from the Politecnico di Milano, in 1995 and
24 another Ph.D. degree in probabilistic risk assessment
25 from the Massachusetts Institute of Technology,
26 Cambridge, USA. He is the Director of the Chair in Complex Systems and
27 Michael G. Pecht (M’83–SM’90–F’92) received the B.S. the Energetic Challenge of the Foundation of Electricite’ de France (EDF) at
28 degree in physics, the M.S. degree in electrical engineering, and the M.S. and Ecole CentraleSupelec (France), and full professor, President and Rector's
29 Ph.D. degree in engineering mechanics from the University of Wisconsin at delegate of the Alumni Association and past-Director of the Graduate School
30 Madison. He is a World-Renowned Expert in strategic planning, design, test, at Politecnico di Milano (Italy), adjoint and guest honor professor at
31 and risk assessment of information systems. He is currently a Professional University of Stavanger (Norway), Beihang University, Tsinghua University,
32 Engineer, an ASME Fellow, an SAE Fellow, and an IMAPS Fellow. He is Wuhan University (China), City University of Hong Kong. His research
33 the Editor-in-Chief of the IEEE Access. He served as a Chief Editor of the topics are: analysis of the reliability, safety and security, vulnerability and
34 IEEE Transactions on Reliability for nine years, and as a Chief Editor of resilience of complex systems under stationary and dynamic conditions,
35 the Microelectronics Reliability for 16 years. He has also served on three particularly by Monte Carlo simulation methods; development of soft
36 NAS studies, two U.S. Congressional investigations in automotive safety, computing techniques (neural networks, support vector machines, fuzzy and
37 and as an Expert for the FDA. He is the Founder and the Director of the neuro-fuzzy logic systems, genetic algorithms, differential evolution) for
38 Center for Advanced Life Cycle Engineering (CALCE) with the University safety, reliability and maintenance applications, system monitoring, fault
39 of Maryland, which is funded by over 150 of the world’s leading electronics diagnosis and prognosis, and optimal design and maintenance.
40 companies at more than US 6M/year. He received the NSF Innovation Award
41 in 2009 and the National Defense Industries Association Award. He is
42
43
44
45 For Review Only
46
47
 IEEE Access Page 20 of 28

1
2
3 He has authored or co-authored seven international books and over 300
4 papers in international journals. His research interests include analysis of the
5 reliability, safety, and security of complex systems under stationary and
6 dynamic conditions, particularly by Monte Carlo simulation methods, and
7 development of soft computing techniques for safety, reliability and
8 maintenance applications, system monitoring, and fault diagnosis and
9 prognosis.
10
11 Melinda R. Hodkiewicz received the B.A. degree
12 (Hons.) in metallurgy and science of materials from
13 Oxford University in 1985, and the Ph.D. degree in
14 mechanical engineering from The University of Western
15 Australia in 2004. She is currently the BHP Billiton
16 Fellow for Engineering for Remote Operations with The
17 University of Western Australia (UWA). Previously she
18 was with industry in operations and maintenance roles.
She currently leads the System Health Laboratory at
19
UWA, where she is involved in the areas of asset health, maintenance and
20
safety. From 2012 to 2015, she was a member of the PC251 ISO55000 Asset
21
Management Standard Committee. In 2015 she was awarded the MESA
22 Medal, a lifetime achievement award for services to the Asset Management
23 community in Australia. In 2018 she was a visiting fellow at the Alan Turing
24 Institute, the UK’s National Centre for Data Science and AI. She is also a
25 Chartered Engineer (U.K.), a member of the Institute of Materials, Minerals
26 and Mining (IOM3), the IEEE and the Asset Management Council.
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 For Review Only
46
47
Page 21 of 28 IEEE Access

1
2
3
4
Data-driven Reliability Analysis of Boeing 787 Dreamliner
5
6 Author Bios
7
8
9
10
11
12 Guru Prasad Pandian is a Master of Science student at Mechanical Engineering department at
13 University of Maryland, College park. He has been working as a research assistant with Center for Advanced Life
14 Cycle Engineering at University of Maryland since Fall of 2015. His research interests are evaluating reliability of
15 electronic packages and specifically reliability of lead-free solders.
16
17
18
19
20
21
22
23
24
25
26
27 Michael G. Pecht (M’83–SM’90–F’92) received the B.S. degree in physics, the M.S. degree in
28 electrical engineering, and the M.S. and Ph.D. degree in engineering mechanics from the University of Wisconsin at
29 Madison. He is a World-Renowned Expert in strategic planning, design, test, and risk assessment of information
30 systems. He is currently a Professional Engineer, an ASME Fellow, an SAE Fellow, and an IMAPS Fellow. He is the
31 Editor-in-Chief of the IEEE Access. He served as a Chief Editor of the IEEE Transactions on Reliability for nine
32 years, and as a Chief Editor of the Microelectronics Reliability for 16 years. He has also served on three NAS studies,
33 two U.S. Congressional investigations in automotive safety, and as an Expert for the FDA. He is the Founder and the
34 Director of the Center for Advanced Life Cycle Engineering (CALCE) with the University of Maryland, which is
35 funded by over 150 of the world’s leading electronics companies at more than US 6M/year. He received the NSF
36 Innovation Award in 2009 and the National Defense Industries Association Award. He is currently a Chair Professor
37 in mechanical engineering and a Professor in applied mathematics with the University of Maryland. In 2015, he
38 received the IEEE Components, Packaging, and Manufacturing Award for visionary leadership in the development of
39 physics-of-failure-based and prognostics-based approaches to electronic packaging reliability. He received the
40 Distinguished Chinese Academy of Sciences President’s International Fellowship. In 2013, he received the University
41 of Wisconsin-Madison’s College of Engineering Distinguished Achievement Award. In 2011, he received the
42 University of Maryland’s Innovation Award for his new concepts in risk management. In 2010, he received the IEEE
43 Exceptional Technical Achievement Award for his innovations in the area of prognostics and systems health
management. In 2008, he was awarded the highest reliability honor, the IEEE Reliability Society’s Lifetime
44
Achievement Award.
45
46
47 Enrico Zio (M’06) received an M.Sc. degree in nuclear engineering from the Politecnico di
48 Milano, Milano, Italy, in 1991, another M.Sc. degree in mechanical engineering from the
49 University of California, Los Angeles, CA, USA, in 1995, a Ph.D. degree in nuclear engineering
50 from the Politecnico di Milano, in 1995 and another Ph.D. degree in probabilistic risk assessment
51 from the Massachusetts Institute of Technology, Cambridge, USA. He is the Director of the Chair
52 in Complex Systems and the Energetic Challenge of the Foundation of Electricite’ de France
53 (EDF) at Ecole CentraleSupelec (France), and full professor, President and Rector's delegate of
54 the Alumni Association and past-Director of the Graduate School at Politecnico di Milano (Italy),
55 adjoint and guest honor professor at University of Stavanger (Norway), Beihang University, Tsinghua University,
56 Wuhan University (China), City University of Hong Kong. His research topics are: analysis of the reliability, safety
57
58
59
60 For Review Only
 IEEE Access Page 22 of 28

1
2
3 and security, vulnerability and resilience of complex systems under stationary and dynamic conditions, particularly
4 by Monte Carlo simulation methods; development of soft computing techniques (neural networks, support vector
5 machines, fuzzy and neuro-fuzzy logic systems, genetic algorithms, differential evolution) for safety, reliability and
6 maintenance applications, system monitoring, fault diagnosis and prognosis, and optimal design and maintenance.
7
8 He has authored or co-authored seven international books and over 300 papers in international journals. His research
9 interests include analysis of the reliability, safety, and security of complex systems under stationary and dynamic
10 conditions, particularly by Monte Carlo simulation methods, and development of soft computing techniques for safety,
11 reliability and maintenance applications, system monitoring, and fault diagnosis and prognosis.
12
13 Melinda R. Hodkiewicz received the B.A. degree (Hons.) in metallurgy and science of materials
14 from Oxford University in 1985, and the Ph.D. degree in mechanical engineering from The
15 University of Western Australia in 2004. She is currently the BHP Billiton Fellow for Engineering
16 for Remote Operations with The University of Western Australia (UWA). Previously she was with
17 industry in operations and maintenance roles. She currently leads the System Health Laboratory
18 at UWA, where she is involved in the areas of asset health, maintenance and safety. From 2012 to
2015, she was a member of the PC251 ISO55000 Asset Management Standard Committee. In
19
2015 she was awarded the MESA Medal, a lifetime achievement award for services to the Asset
20
Management community in Australia. In 2018 she was a visiting fellow at the Alan Turing
21
Institute, the UK’s National Centre for Data Science and AI. She is also a Chartered Engineer (U.K.), a member of
22 the Institute of Materials, Minerals and Mining (IOM3), the IEEE and the Asset Management Council.
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60 For Review Only
Page 23 of 28 IEEE Access

1
2
3
4
Data-driven Reliability Analysis of Boeing 787 Dreamliner
5
6 Guru Prasad Pandian, Enrico Zio, Melinda Hodkiewicz, and Michael Pecht
7
8 Response to feedback received for manuscript # Access-2017-07930
9
10
11 Reviewer’s comments Author’s Response Changes in Manuscript
12
13 Reviewer 1: Your paper is well Thank you No changes
14 organized
15
16
17 Reviewer 1: It is not really like a We performed significant analysis Complete Section IV: “Li-ion
18 paper with high academic quality. on the reliability of 787 aircraft batteries (LIBs) were used to power
19 Please add some insights and proving why it was indeed in the auxiliary power units and other
20 resubmit again. trouble during its launch. The data selected electrical/electronic
21 is also shown to corroborate the equipment during ground and flight
22 need of the re-design work to get operations to a … The FAA issued
23 the aircraft back into operation. We a new airworthiness directive to
24 added further insights pertaining to install the redesigned batteries on
25 the lack of regulation on battery all 787 airplanes to be returned to
26 qualifications and its potential service.”
27 reliability implications. We also
28 added content on the effectiveness Section V, paragraphs 1, 2, and 3:
29 of Boeing’s reliability program and “From the various documents and
30 supply chain management. trends, it can be argued Boeing did
31 not adopt an effective Reliability
32 Program Plan (RPP), where best
33 practice tasks are implemented to
34 produce reliable products..
35 Section V, Sub-Section F: “In
36 Boeing’s 787 development model,
the integration of sub-assemblies
37
and final assembly was critical for
38
the hardware and software... which
39
in turn led to design flaws being
40 carried over to next tiers of
41 assemblies and eventually to the
42 aircraft operation on field.”
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60 For Review Only
 IEEE Access Page 24 of 28

1
2
3 Reviewer’s comments Author’s Response Changes in Manuscript
4
5
6 Reviewer 1: It's more like a review We agree in part with the reviewer, Table 1;
7 and summation of existing papers. that a large part of this paper
8 And the number of paper is not a pertains to the aggregation of data. Section II, 2nd and 3rd paragraph:
9 lot. But this was needed because such “In July 2012, the Japanese
10 data, besides being very difficult to airline ANA grounded five of its
11 obtain, is not all in one place. 787s due to the potential risk of
12 Furthermore, other than new reports corrosion in some of the engine parts
13 and government documents, there [Error! Reference source not
are no papers on this subject. Our found.]. This event was followed by
14
goal was to collate as much data as even bigger problems revealing
15
we could collect and then use this themselves in the form of fires in
16
data to assess the reliability growth 787 aircrafts… Nevertheless,
17 process and to assess the key Boeing made changes, including a
18 challenges to reliability. revision of the internal battery
19 components to minimize the
20 chances of initiating a short circuit,
21 as well as better insulation of the
22 cells and the addition of a new
23 containment and venting system
24 [15]. On March 12, after less than
25 one month of testing, the FAA
26 accepted Boeing’s redesign.
27 Error! Reference source not
28 found. in the Appendix presents a
29 list of the 787’s technical issues
30 reported to the authorities …This
31 information suggests there were a
32 range of different failure modes
33 responsible for the failure events.”
34
35 Section III:
36 4th paragraph:
“The Maximum likelihood
37
estimation (MLE) technique, a
38
classic way to estimate parameters
39
[Error! Reference source not
40 found.], has also been used . By
41 grouping the data, the number of
42 failures in each quarter has been
43 used to conduct the analysis.”
44
45 Complete Section IV:
46 “Li-ion batteries (LIBs) were used
47 to power the auxiliary power units
48 and other selected
49 electrical/electronic equipment
50 during ground and flight operations
51 to a … The FAA issued a new
52 airworthiness directive to install the
53 redesigned batteries on all 787
54 airplanes to be returned to service.”
55
56
57
58
59
60 For Review Only
Page 25 of 28 IEEE Access

1
2
3 Reviewer’s comments Author’s Response Changes in Manuscript
4
5
6 Reviewer 1: This paper is well Thank you. We have added Conclusions section
7 organized and easily thoughts and content to address all
8 comprehensive. So I suggest editor the concerns
9 give the authors another chance to
10 add in some key points.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60 For Review Only
 IEEE Access Page 26 of 28

1
2
3 Reviewer 1: It is not technically We have now added key points to Section II, 2nd and 3rd paragraph:
4 sound, basically there is no key the paper; and made what we “In July 2012, the Japanese
5 point in this draft. already had stronger. A new section airline ANA grounded five of its
6 has been added on “Reliability 787s due to the potential risk of
7 program plan”: RPP objectives and corrosion in some of the engine parts
8 how Boeing reacted to the [Error! Reference source not
9 objectives. In particular, we related found.]. This event was followed by
10 the reliability drop in initial years of two airplane fires in associated with
11 787 operation to their reliability Li-ion batteries… Nevertheless,
12 program. Boeing made changes, including a
13 revision of the internal battery
14 components to minimize the
15 chances of initiating a short circuit,
16 as well as better insulation of the
17 cells and the addition of a new
18 containment and venting system
19 [15]. On March 12, after less than
one month of testing, the FAA
20
accepted Boeing’s redesign.
21
Error! Reference source not
22
found. in the Appendix presents a
23 list of the 787’s technical issues
24 reported to the authorities …This
25 information suggests there were a
26 range of different failure modes
27 responsible for the failure events.”
28
29 Section III:
30 4th paragraph:
31 “The maximum likelihood
32 estimation (MLE) technique, a
33 classic way to estimate parameters
34 [Error! Reference source not
35 found.], has also been used. By
36 grouping the data, the number of
37 failures in each quarter has been
38 used to conduct the analysis.”
39
40 Complete Section IV: “Li-ion
41 batteries (LIBs) were used to power
42 the auxiliary power units and other
selected electrical/electronic
43
equipment during ground and flight
44
operations to a … The FAA issued
45
a new airworthiness directive to
46 install the redesigned batteries on
47 all 787 airplanes to be returned to
48 service.”
49
50 Section V, paragraph 1, 2, and 3: “It
51 is clear from FAA and NTSB report
52 and journals published on
53 Dreamliner failures that Boeing did
54 not adopt an effective Reliability
55 Program Plan (RPP)…
56
57
58
59
60 For Review Only
Page 27 of 28 IEEE Access

1
2
3 Reviewer’s comments Author’s Response Changes in Manuscript
4
5
6 Section V, Sub-Section F: “In
7 Boeing’s 787 development model,
8 the integration of sub-assemblies
9 and final assembly is critical for the
10 parts from different suppliers to fit
11 together. This required an optimal
12 balance of providing autonomy to
13 the suppliers to meet the design…
…..flaws being carried over to next
14
tiers of assemblies and eventually
15
to the aircraft operation on field. If
16
Boeing had employed on-site
17 technical support, some of the
18 technical issues could have been
19 caught early in the product
20 development stage which would
21 have prevented the delays as well as
22 operational issues.”
23
24
25 Reviewer 1: The references are Thank you No Changes
26 good and useful.
27
28 Reviewer 2: It would be interesting Section V, 2nd paragraph:
29 to gather developmental data to We have now addressed this issue “From the various documents and
30 determine how these issues got past in sub-section B of section V. In trends, it can be argued Boeing did
31 DT/OT. Brief comments allude to addition, in the new draft we have not adopt an effective Reliability
32 possible issues linked it?? to objectives of an Program Plan (RPP), where best
33 effective Reliability Program Plan practice tasks are implemented to
34 and how Boeing missed addressing produce reliable products [Error!
35 it?? (see the updates in Section V Reference source not found.].
36 paragraph 2 and 3). Boeing opted to widen its supplier
37 base and reduce costs by including
38 We have also added a table that manufacturers who were new to the
39 shows the number of Boeing 787s aircraft development industry. The
40 delivered over a period of 5 years to events that led to delays during
help with the analysis. manufacturing and failures during
41
operation are a testament to
42
Boeing’s flawed practice.
43
The following sub-sections
44 describe Boeing’s practices in
45 planning and managing the
46 development cycle and supply
47 chain, the challenges with
48 information sharing with a tiered,
49 globally-dispersed supplier base,
50 with developing a proper
51 diagnostics and prognostics
52 approach, with testing of new
53 technologies, and with oversight of
54 a complex product development
55 process.”
56
57
58
59
60 For Review Only
 IEEE Access Page 28 of 28

1
2
3 Reviewer’s comments Author’s Response Changes in Manuscript
4
5
6 Reviewer 2: Excellent analysis with Thank you No Changes
7 limited data.
8
9 Reviewer 2: assumptions seemed Thank you No changes
10 sound
11 Reviewer 2: The subject matter is Thank you No changes
12 presented in a comprehensive
13 manner
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60 For Review Only