IIE Transactions (1997) 29, 91±101

Comparison of regression and neural network models

for prediction of inspection pro®les for aging aircraft

JAMES T. LUXHéJ1, TREFOR P. WILLIAMS2 and HUAN-JYH SHYUR1

1
Department of Industrial Engineering, Rutgers University, P.O. Box 909, Piscataway, NJ 08855-0909, USA
2
Department of Civil Engineering, Rutgers University, P.O. Box 909, Piscataway, NJ 08855-0909, USA

Received May 1994 and accepted May 1996

Currently under phase 2 development by the Federal Aviation Administration (FAA), the Safety Performance Analysis System
(SPAS) contains `alert' indicators of aircraft safety performance that can signal potential problem areas for inspectors. The Service
Diculty Reporting (SDR) system is one component of SPAS and contains data related to the identi®cation of abnormal,
potentially unsafe conditions in aircraft and/or aircraft components/equipment.
SPAS contains performance indicators to assist safety inspectors in diagnosing an airline's safety `pro®le' compared with others
in the same peer class. This paper details the development of SDR prediction models for the DC-9 aircraft by analyzing sample
data from the SDR database that have been merged with aircraft utilization data. Both multiple regression and neural networks are
used to create prediction models for the overall number of SDRs and for SDR cracking and corrosion cases. These prediction
models establish a range for the number of SDRs outside which safety advisory warnings would be issued. It appears that a data
`grouping' strategy to create aircraft `pro®les' is very e€ective at enhancing the predictive accuracy of the models. The results from
each competing modeling approach are compared and managerial implications to improve the SDR performance indicator in
SPAS are provided.

1. Introduction
A repairable or `maintained' system is `a system which,
after failing to perform one or more of its functions sa-
tisfactorily, can be restored to fully satisfactory perfor-
mance by any method, other than replacement of the
entire system' [1]. E€ective and ecient maintenance
management is essential not only for production systems
but also for large scale service systems, such as air and
surface transport systems. These repairable systems are
subject to aging mechanisms such as wear, fatigue, creep,
and stress corrosion. Inspection and diagnostic activities
are integral components of an e€ective maintenance
strategy in an attempt to ensure system safety, reliability,
and availability.
The Federal Aviation Administration (FAA) has es-
tablished a Center for Computational Modeling of Air-
craft Structures (CMAS) at Rutgers University. One
CMAS research project has focused on analyzing the
contribution of the FAA's Service Diculty Reporting
(SDR) database to aircraft safety. The SDR system
contains data related to the identi®cation of abnormal,
potentially unsafe conditions in aircraft or aircraft com-
ponents and equipment.
Estimation of the total number of SDRs in a given
time interval that a particular airline would be expected
to have, adjusting for age of the aircraft, ¯ight time, and
landings, could help to identify situations in need of
heightened level of surveillance by the FAA's safety in-
spectors, for example if the airline's number of SDRs is
far above or below what should be expected. An excessive
number of SDRs in a given time period could suggest
mechanical, operating, or design problems with certain
aircraft. Although too few SDRs reported in a given time
may not necessarily be problematic, an expert panel of
safety inspectors noted that a very low number of SDRs
for an airline in a given time period could possibly suggest
organizational or management problems, lack of reg-
ulatory compliance, airline maintenance cutbacks, or ®-
nancial or labor problems. Both situations would merit
closer scrutiny by FAA safety inspectors.
The CMAS research is only one initiative of a larger
FAA research program, termed the Safety Performance
Analysis System (SPAS), that is an analytical tool in-
tended to support FAA inspection activities [2, 3] and
that contains numerous indicators of safety performance
for signaling potential problem areas for inspector con-
sideration [4±6].
The numerous performance indicators that are cur-
rently de®ned in SPAS assist in diagnosing an airline's
`pro®le' compared with others in the same peer class. The
currently planned SDR performance indicator is the

0740-817X Ó 1997 ``IIE''

92 Luxhùj et al.

number of SDR records for the airline for the de®ned
period [4]. This preliminary SDR performance indicator
in SPAS, by not allowing inspectors to di€erentiate be-
tween di€erent types of problem, is too simplistic to be of
practical value. The Rutgers University CMAS research
examined this planned indicator, and the result of our
research e€ort was the construction of more re®ned,
speci®c SDR performance indicators. The tracking of
performance indicators also facilitates the identi®cation
of unfavorable trends. The eventual goal of this research
is to develop an intelligent decision system that will be a
hybrid of expert system and neural network technologies
supported by aviation databases to facilitate organiza-
tional coordination and ecient workload scheduling for
aircraft safety inspectors under budgetary and stang
constraints.
2. Research methodology
The currently planned SDR performance indicator is S,
which is simply the number of SDR records for the airline
for the de®ned period. The count of records is not nor-
malized. If S > 0, the indicator status is set as `expected';
if S = 0, the indicator status is set as `advisory' [4]. It is
expected that over a six-month period, normal operations
by an airline will lead to ®nding a non-zero number of
SDRs due to routine and non-routine maintenance. This
`alert' indicator is too general to be of practical value to
safety inspectors; for instance, it fails to di€erentiate by
age of the aircraft, ¯ight time, and number of landings.
Although many prediction methods exist in the lit-
erature, this research focuses on only two modeling ap-
proaches to develop more re®ned SDR predictors:
multiple regression and neural networks. Multiple re-
gression represents a `classical' approach to multivariate
data analysis whereas the emerging ®eld of neural net-
works represents a `new' approach to nonlinear data
analysis.
The regression and neural network models presented in
this paper may be used to predict the average aggregate
number of SDRs in a given time interval for the DC-9
aircraft. In addition to ¯ight hours and number of land-
ings, aging mechanisms such as wear, fatigue, creep, and
stress corrosion contribute to reported incidents of
cracking and corrosion in an aircraft's fuselage and other
major structural components, and models are also de-
veloped to predict the average number of SDRs for
cracking and corrosion cases. The identi®cation of un-
favorable trends will enable the FAA to specify that the
airlines take preemptive maintenance measures.
3. Data description
The CMAS research team was provided with a subset of
the SDR database that had been merged with the Aircraft
Utilization (ARS) database for the same set of planes.
This merged database was created by Battelle [7] and
consisted of 1308 observations for the DC-9 aircraft for
the period April 1974 to March 1990. Table 1 displays
sample data. Only the following quantitative data for
each plane were available in the merged database:
· age,
· estimated ¯ight hours, and
· estimated number of landings.
Because actual data on ¯ight hours and landings were not
reported directly in the SDR database, the estimated
¯ight hours and estimated landings are derived from the
original delivery date of the plane to the ®rst airline, the
date of the ARS data reference, and the SDR date. The
equations developed by Battelle for these derived values
are reported in Rice [7] and are presented below:
Estimated flight hours ˆ ‰…SDR date ÿ service date†=
…ARS date ÿ service date†Š
 FHSCUM;
Estimated number ˆ ‰…SDR date ÿ service date†=
of landings …ARS date ÿ service date†Š
 LDGSCUM;
where

Table 1. Sample of SDR and ARS `merged' data [7]

Aircraft Serial SDR date Part name Part location Part condition Estimated age Estimated Estimated
model numbera ¯ight hours landings
DC9 333 84±03±22 Skin E + E Compt Cracked 17.74 32 619.03 53 999.20
DC9 333 84±03±22 Skin Aft bag bin Cracked 17.74 32 619.03 53 999.20
DC9 333 86±07±07 Skin Fuselage Cracked 20.03 36 836.23 60 980.56
DC9 444 80±06±20 Skin Galley door Cracked 13.24 34 396.44 33 888.77
DC9 444 81±12±01 Skin FS625 Corroded 14.69 38 160.55 37 597.32
DC9 444 87±05±11 Skin Rt wheel well Cracked 20.14 52 299.10 51 527.19
DC9 444 87±05±11 Skin STA 580-590 Cracked 20.14 52 299.10 51 527.19
a
Fictitious serial numbers are used owing to con®dentiality of data.
Prediction of inspection pro®les for aging aircraft
SDR date ˆ date of the SDR report …SDR database†;
service date ˆ original delivery date of the plane to
the first airline …ARS database†;
ARS date ˆ date of the ARS report …ARS database†;
FHSCUM ˆ cumulative fuselage flight hours
…ARS database†; and
LDGSCUM ˆ cumulative fuselage landings
…ARS database†:
Because the ARS date time lagged the SDR date, Rice
extrapolated the quantitative ARS data on ¯ight hours
and landings to the SDR date. He developed a multiplier
by calculating the ratio of [(SDR date ) service date)/
(ARS date ) service date)] and then extrapolated the
¯ight hours and landings at the ARS date to the date of
the SDR.
4. Multiple regression models
Initially, regression models were created from the 1308
DC-9 observations in their original format, referred to as
the `ungrouped' data. For the ungrouped data, the
number of SDRs for each airplane is based on the cu-
mulative number of data records (each record represents
only one SDR). When cases with missing data were
eliminated, there were a total of 1229 usable data cases.
The coecients of multiple determination, or R2 values,
for these models were very low, with the `best' model
having an R2 of 0.2448 and a coecient of variation
(C.V.) of 69.85. The C.V. reported here is the ratio of the
root-mean-square error of the model to the sample mean
of the dependent variable multiplied by 100 and indicates
how well the model ®ts the data. If the model does not ®t
the data well, then the C.V. becomes large. It appeared
that there was much noise in the data because a plot of
the ungrouped data revealed extensive ¯uctuations.
5. Data grouping strategies
In an attempt to create robust SDR prediction models
that will provide SDR pro®les for a representative DC-9,
di€erent data grouping strategies are used. Such an ap-
proach was used in Brammer [8], Fabrycky [9], Fabrycky
et al. [10], Frisch [11], Luxhùj and Jones [12], Luxhùj and
Rizzo [13], and Luxhùj [14±16], to create large-scale lo-
gistics models for the U.S. Navy. These `population'
models were developed to determine both maintenance
and system repair/replacement strategies for large gro-
upings of similar equipment on the basis of operating
hours, operating environment, failure mode, etc.
By using multiple regression models, data grouping
strategies for age, estimated ¯ight hours, and estimated
93
landings are developed on the basis of a smaller set of
averaged data to predict the total expected number of
SDRs per year, the number of SDRs per year for cracked
cases, and the number of SDRs per year for corrosion
cases for the DC-9 aircraft. The `best' grouping strategy
for each model case is then selected based on the highest
R2 value.
To provide a means for checking the SDR predictions
against existing data, the data were partitioned into two
di€erent sets on the basis of aircraft serial numbers. The
®rst set was used to build the prediction model and the
second was used to evaluate the prediction model's per-
formance on un®t data. Such an approach is useful for
testing prediction model generality [17]. This approach is
also used in neural network modeling and is analogous to
creating a `training' set of data to build the model and a
`production' set of data to evaluate model performance
on new data. These terms are used in this paper to dis-
tinguish between the two data sets. The original data were
partitioned into mutually exclusive training and produc-
tion sets by using serial numbers for the di€erent aircraft.
Two-thirds of the data were placed into the training set,
and one-third into the production set.
After the data have been partitioned into training and
production sets, a grouping strategy is similarly applied
to each data set. For example, an `age'grouping strategy
is outlined below:
1. Group the data to create age `cohorts' (i.e., groups of
1, 2, 3, ... -year old planes).
2. Calculate the `average' ¯ight hours and number of
landings for each `age cohort'.
3. Calculate the average number of SDRs per number
of aircraft in each `age cohort'.
Forward stepwise regression is used where variables are
added one at a time. Partial correlation coecients are
examined to identify an additional predictor variable that
explains both a signi®cant portion and the largest portion
of the error remaining from the ®rst regression equation.
The forward stepwise procedure selects the `best' regres-
sion model based on the highest R2 from the following list
of possible explanatory variables: age, ¯ight hours,
number of landings, (age)2, (¯ight hours)2, (number of
landings)2, age ´ ¯ight hours, age ´ number of landings,
¯ight hours ´ number of landings, ¯ight hours/age, and
number of landings/age. The default stopping criterion
for the F test to determine which variable enters the
model uses a signi®cance level of 0.15. In the second stage
of our analysis, the best prediction model was chosen on
the basis of lowest Mean Square Error (MSE) on the
training and production data, because MSE is a better
indicator of predictive accuracy. The quadratic terms
were considered in an inherently linear model to evaluate
any nonlinear relationships and the impact of interaction
terms was evaluated. The forward stepwise procedure was
used to ®nd a prediction equation with an R2 close to 1
94
Table 2. Results of Data Grouping Procedure
No. of data records
Model Training
Overall no. of SDRs 805
No. of SDRs (cracking) 572
No. of SDRs (corrosion) 242
and to provide an equation that was economical: one that
used only a few independent variables.
As a result of the grouping strategy, all interpretations
are now with respect to the average number of SDRs per
year. In the example above, the dependent variable be-
comes the average number of SDRs for a representative
DC-9 with a `pro®le' of estimated ¯ight hours and esti-
mated landings as de®ned by its associated age cohort.
For the grouped data, we now have the number of SDRs
for each airplane with respect to an interval (i.e., age,
¯ight hours, or landings). The di€erent structure of the
data between grouped and ungrouped records led to
structural di€erences between the regression models and
to the use of di€erent explanatory variables.
The grouping procedure gave the results shown in
Table 2.
A prediction model for the overall number of SDRs
per year for a representative DC-9 that uses the `age' data
grouping strategy is as follows:
Overall no of SDRs ˆ …0:00256264  agesq†
ÿ …4:038133  10ÿ9  fhrsq†
‡ …0:002347  fhr=age†
ÿ 4:173934:
Note that this prediction model makes use of only three
independent variables: the age squared (agesq), the ¯ight
hours squared (fhrsq), and ¯ight hours/age (fhr/age). The
R2 is 0.9297, which indicates that this model can explain
92.97% of variability of the expected number of overall
SDRs per year about its mean. This model was developed
from 16 grouped data records that corresponded to air-
craft ranging from approximately 8 to 24 years old.
An important point to remember when using this
model is that one must have a suciently large data
sample of DC-9 aircraft in order to compute `averages' of
estimated landings and ¯ight hours for a speci®ed aircraft
age. The more data that one has, the better one can model
a representative aircraft with the data grouping strategy
as previously discussed.
6. Regression-modeling adequacy issues
The regression models were examined for multico-
llinearity, because a high degree of multicollinearity makes
Luxhùj et al.
`Grouped' no. of data records
Production Training Production
424 16 14
306 16 16
127 10 9
the results not generalizable as the parameter estimates in
the model may not be stable owing to the high variance of
the estimated coecients. Because ¯ying hours, number of
landings, and the age of an aircraft are interrelated, mul-
ticollinearity is inherent in the independent variables.
Two statistical measures of multicollinearity are the
tolerance (TOL) value and the variance in¯ation factor
(VIF) [18]. The tolerance value is equal to one minus the
proportion of a variable's variance that is explained by
the other predictors. A low tolerance value indicates a
high degree of collinearity. The variance in¯ation factor is
the reciprocal of the tolerance value, so a high variance
in¯ation factor suggests a high degree of collinearity
present in the model. The VIF and TOL measures assume
normality and are typically relative measures. A high
(above 0.10) and a low VIF (below 10) usually suggest a
relatively small degree of multicollinearity [18].
While parsimonious regression models were developed
by observing the VIF and TOL measures during model
building and selection, an attempt was also made to re-
move multicollinearity by removing the linear trend from
the observed variables. Both the dependent and in-
dependent variables were transformed by replacing their
observed values with their natural logarithms. Although
this approach was successful in reducing multico-
llinearity, the resulting regression models all had higher
coecients of variation and lower R2 values than models
without such variable transformations.
There are times in regression modeling when the as-
sumption of constant error variance (i.e., homosce-
dasticity) may be unreasonable and heteroscedastic error
disturbances will occur. When heteroscedasticity is pre-
sent, ordinary least-squares estimation places more
weight on the observations with large error variances
than on those with small error variances. The White Test
is used in this study to test for heteroscedasticity [17]. In
the White Test, the null hypothesis of homoscedasticity is
evaluated and the test does not depend critically on
normality. We report the results of the White Test on our
data later in this section.
Alternative grouping strategies to `age' were also ex-
amined. Graphical analysis was used to examine the tra-
deo€ of the number of observations versus adjusted R2
values to determine interval grouping sizes for estimated
landings and estimated ¯ight hours. When using the data
grouping strategy of estimated landings, the suggested
Prediction of inspection pro®les for aging aircraft 95

interval size is 4000 landings for the SDR cracking and
corrosion cases and 5500 landings for the total number of
SDRs. When using the data grouping strategy of esti-
mated ¯ight hours, the suggested interval grouping size is
4000 hours. When analyzing the graphs, the goal is to ®nd
an interval grouping size that maximizes the adjusted R2
yet results in the use of a reasonable number of observa-
tions (i.e., n ³ 16, which corresponds to aircraft ranging
from 8 to 24 years old) to facilitate model development.
Also, there were upper limits to the interval sizes for
landings and ¯ight hours beyond which too few groups
resulted. The adjusted R2 is used because the number of
predictors is changing for each alternative interval size.
As discussed earlier, prediction models were developed
from training data and evaluated on production data.
Because the goal was to maximize the accuracy of the
SDR predictions, the MSE was used for comparative
purposes. Although the MSE has some bias, it is an es-
timator with very low variance.
Table 3 presents the `best' SDR regression models for
the DC-9 aircraft comparing across grouping strategies,
predictor variables, and outcome variable. The table also
displays the squared partial correlation coecients that
can be used to assess the relative importance of the dif-
ferent independent variables used in the regression mod-
els. The VIFs for the overall SDR and corrosion models
are acceptable and suggest a relatively small degree of
multicollinearity. However, the VIF for the cracking
model suggests a moderate degree of collinearity, and this
model should be used with caution as the parameter es-
timates might not be stable. The application of the White
Test resulted in the acceptance of the null hypothesis of
homoscedasticity at the 5% signi®cance level for all three
models and suggests that the assumption of constant er-
ror variances is reasonable. For the overall SDR predic-
tion model, the White Test resulted in the following
regression equation for the regression residuals:
e2 ˆ …ÿ0:0007  agesq† ‡ …1:89304  10ÿ10  fhrsq†
‡ …0:00001052  fhr/age† ‡ 0:228357;
with an R2 of 0.1038. The test statistic NR2 equalled (16)
(0.1038) = 1.6608, which follows a v2 distribution with 3
degrees of freedom. The critical value of the v2 with 3
degrees of freedom at the 5% signi®cance level is 7.81.
Because 1.6608 < 7.81, we accept the null hypothesis of
homoscedasticity. The White Test was similarly applied
to the SDR cracking and corrosion cases.
On the basis of an analysis of the 1229 data ob-
servations for merged SDR and ARS data, it appears
that the data grouping strategy results in SDR prediction
models that can be used to predict expected reporting
pro®les for a representative DC-9. Con®dence intervals
can be calculated for the expected number of SDRs per
year so that a range of values can be reported along with
a point estimate. For example, Fig. 1 displays the re-
siduals and con®dence limits for the interval (CLI) that
includes the variation for both the mean and the error
term. In essence, this ®gure graphically displays the
prediction interval for the overall number of SDRs
across all airlines for 95% con®dence. Such an approach
establishes control limits or threshold levels outside
which national SDR advisory warnings would be posted.
To construct con®dence intervals for a particular age
group for a given airline, it is necessary to consider the
number of aircraft in that age group owned by that
airline. SDR prediction models for each airline could be
developed by following the same grouping methodology
as outlined above, but with the data partitioned by age
and airline. Such models were not developed in this
study, as only 2 of 22 airlines had a sucient number of
data observations by airline.
It appears that ungrouped SDR and ARS data are not
useful for prediction purposes. Grouped data strategies
show promise in predicting SDR pro®les based on the

Table 3. SDR multiple regression models

Dependent variable Grouped no of R2 C.V. MSE Independent Squared VIF

observations variables partial
correlation
Training Production Training Production coecient
data data data data
Overall no. of SDRs Agesq 0.9137 11.39
Grouping strategy: age 16 14 0.9297 19.40 0.1953 0.9219 Fhrsq 0.5202 11.19
Increment: 1 year Fhr / age 0.6070 3.52

No. of SDRs (cracking) Fhr / age 0.7593 124.81

Grouping strategy: 16 15 0.7899 6.69 0.0061 0.0265 Age ´ ldg 0.3764 124.81
¯ight hours
Increment: 4000 hours

No. of SDRs (corrosion) Agesq 0.9661 4.76

Grouping strategy: age 10 9 0.9780 12.29 0.0321 1.6918 Fhrsq 0.9333 9.87
Increment: 1 year Fhr / age 0.6962 4.32
96
Fig. 1. Residual analysis and 95% con®dence limits for overall
SDR prediction model. Data sorted and grouped by age (in-
crement = 1 year).
DC-9 analysis. These data grouping strategies generally
result in robust models that are useful in developing air-
craft population pro®les. A plausible reason for the ap-
parent success of the grouping strategy is that computing
the average number of SDRs for an interval (i.e., age,
¯ight hours, number of landings) results in the dependent
variable becoming approximately normal owing to the
Central Limit Theorem.
Of the three prediction models, the model to predict
the overall expected number of SDRs appears the `best'.
It has the second highest R2 value (0.9297), a low degree
of multicollinearity, and low MSEs on both the training
(0.1953) and production (0.9219)
p data. For the training
data, the magnitude of the MSE is low relative to the
sample mean of 2.63 SDRs (i.e., ratiop = 0.168). For the
production data, the ratio of the MSE to the sample
mean
p of 2.92 SDRs is higher (i.e., ratio = 0.329). If the
MSE > 0.33 ´ sample mean, then normality is not a
reasonable assumption and additional distributional in-
formation is needed to construct a useful con®dence in-
terval. When compared with the overall SDR prediction
model, the model for corrosion cases has a larger R2 , but
there is a degradation of performance on p the production
data based on MSE. The ratios of the MSE values to
the sample means for the training and production data
are 0.095 and 0.567, respectively. The model for corrosion
was built on the smallest number of averaged data, and
this could account for its degraded performance on the
production data. The SDR prediction model for cracking
cases has the smallest R2, however, it has the best per-
formance
p on the production data. The ratios of the
MSE values to the sample means are 0.064 and 0.092;
thus, dispersion of the residuals around the mean is small.
When compared with the overall SDR prediction model,
the smaller number of averaged observations used in
building the model for cracking could account for the
smaller R2. Stem-and-leaf displays [19] for all models
indicate that the shapes of the distributions for the re-
Luxhùj et al.
siduals are unimodal and bell-shaped. Box plot diagrams
of the residuals for all models imply symmetrical data sets
and that the medians are nearly zero.
A limitation on these prediction models is that the
results presented in this paper are based on a relatively
small sample of merged DC-9 SDR and ARS data (i.e.,
1229 observations) for the period 1974±90. Generalizing
the results to other aircraft types should be done with
caution. The value or contribution of this study's ®ndings
exists in the methods and techniques used to identify the
factors in¯uencing the expected number of SDRs.
7. Neural network models
A parallel CMAS research e€ort focused on the devel-
opment of neural networks to determine patterns in SDR
reporting. Emanating from research in Arti®cial In-
telligence (AI) [20±22], neural networks attempt to si-
mulate the functioning of human biological neurons.
Neural networks have been particularly useful in pattern
recognition problems that involve capturing and learning
complex underlying (but consistent) trends in data.
Neural networks are highly nonlinear and, in some cases,
are capable of producing better approximations than
multiple regression, which produces a linear approxima-
tion [23, 24]. Neural network learning supports incre-
mental updating and is easier to embed in an intelligent
decision system because batch processing is not required.
Although neural networks o€er an alternative to regres-
sion that will determine functional relationships between
variables to predict an outcome measure, neural network
outcomes lack a simple interpretation of results. For in-
stance, the modeling technique does not provide objective
criteria to decide what set of predictors is more important
for the prediction. Neural networks can also su€er from
over®tting of the data and lack of prediction generality.
The limitations of neural networks with respect to out-
liers, multicollinearity, and other problems inherent in
real world data have received scant attention.
Backpropagation neural networks are the most com-
monly used neural network architectures. These neural
networks are especially good for pattern recognition. The
initial program employs an analog, three-layer, back-
propagation network. To develop a backpropagation
model, a training set of data patterns that consist of both
inputs and the actual outputs observed must be devel-
oped. During training the neural network processes pat-
terns in a two-step procedure. In the ®rst or forward
phase of backpropagation learning, an input pattern is
applied to the network, and the resulting activity is al-
lowed to spread through the network to the output layer.
The program compares the actual output pattern gener-
ated for the given input with the corresponding training
set output. This comparison results in an error for each
neurode in the output layer. In the second, or backward,
Prediction of inspection pro®les for aging aircraft
phase, the error from the output layer is propagated back
through the network to adjust the interconnection
weights between layers. This learning process is repeated
until the error between the actual and desired output
converges to a prede®ned threshold [25].
In neural network modeling, R2 compares the accuracy
of the model with the accuracy of a trivial benchmark
model where the prediction is simply the mean of all the
sample patterns. A perfect ®t would result in an R2 of 1, a
very good ®t near 1, and a poor ®t near 0. If the neural
network model predictions are worse than one could
predict by just using the mean of the sample case outputs,
R2 will be 0. Although not precisely interpreted in the
same manner as the R2 in regression modeling, never-
theless, the R2 from neural network modeling can be used
as an approximation when comparing model adequacy
with a multiple regression model.
General Regression Neural Networks (GRNN) are
known for their ability to train on sparse data sets. It has
been found that GRNN gives better performance than
backpropagation for some problems [26]. It is particu-
larly useful for problems involving continuous function
approximation [26]. A GRNN network is a three-layer
network that contains one hidden neuron for each
training pattern. GRNN training di€ers from back-
propagation networks because training occurs in only one
pass. A GRNN is capable of functioning after only a few
training patterns have been entered.
8. Initial results with neural networks
The neural network models were developed with the
`NeuroShell 2' [27] computer program. The program
implements several di€erent types of neural network
models. Initial model development focused on the use of
backpropagation and general regression neural networks
with the ungrouped data.
In the initial stage of neural network development it
was concluded that neural networks created with un-
grouped data do not provide acceptable results. R2 values
across seven alternative network architectures ranged
from 0.13 to 0.45 with MSE values ranging from 3.92 to
Table 4. SDR neural network models
Output No. of patterns Backpropagation (BP) modela
Training Production R2
data data
Overall no. of SDRs 16 14 0.9452
No. of SDRs (cracking) 16 15 0.6899
No. of SDRs 10 9 0.9411
(corrosion)
a
For all BP models, inputs are Age, Fhr, and Ldg.
b
For all hybrid models, inputs are Age, Fhr, Ldg, Class 1,¼,Class n, where n is the number of class intervals.
97
6.2. It became necessary to transform the input data to
obtain neural nets that compare favorably with the re-
gression models. To summarize the early attempts with
neural network modeling for SDR prediction, the results
with ungrouped data are not good across a variety of
architectures and di€erent learning parameters. This led
to a search for more re®ned modeling strategies.
9. Creation of neural network models with
`grouped' data
Neural network models for SDR prediction were also
created with alternative data grouping strategies as pre-
viously outlined and the same training and production
data sets as those used in the regression analysis. The
SDR neural network models are presented in Table 4.
Training times for the backpropagation models were in-
signi®cant. Since model `®t' and prediction accuracy were
deemed to be most important, R2 and MSE were used to
select the `best' neural network con®guration. The best
data grouping strategies as determined from the regres-
sion analysis were similarly applied in neural network
modeling. These neural network models can be used to
predict the average number of SDRs by using a data
grouping strategy of one-year time increments for the
overall number of SDRs and for the number of corrosion
cases. To predict the average number of SDRs for
cracking cases, the data grouping strategy was based on
increments of 4000 ¯ight hours. In all cases, the MSE was
lower on the training data than on the production data.
Especially note that although the neural network for the
corrosion case performed well on the training data
(R2 = 0.9411, MSE = 0.086), the MSE on the production
data increased signi®cantly (MSE = 3.125). It should also
be observed that the model for corrosion cases had the
smallest number of training and production patterns de-
rived from data groupings with the smallest number of
observations of the three models constructed. Thus this
model should be used with caution on un®t data as it does
not appear to generalize well.
As in regression modeling, 90% or 95% `con®dence
intervals' could be developed for the overall number of
Hybrid modelb
MSE MSE R2 MSE MSE n
(training) (production) (training) (production)
0.152 0.541 0.9603 0.110 2.626 4
0.009 0.409 0.8404 0.005 0.019 2
0.086 3.125 0.9727 0.040 3.502 3
98
SDRs and the number of SDRs for cracking and corro-
sion cases. These con®dence intervals could be displayed
in a fashion analogous to quality control charts serving as
more re®ned `alert' indicators for inspectors that specify
upper and lower safety control limits by aircraft type.
Burke et al. [28] report on a two-stage neural network
that models the relationship between the frequency of
vibration of a beam and the correct control action for
minimizing that vibration. Initially, backpropagation was
tried as a modeling technique; however, the modeling
results were poor. It has been documented [29] that neural
networks often try to `over®t' the training data. Burke
et al. discovered that the opposite sometimes occurs when
occasionally a way is needed to `wrinkle' the data surface
to facilitate a mapping from the independent variable(s)
to the dependent variable.
In the beam de¯ection problem, Burke et al. develop a
`granular decomposition' of the problem by transforming
it into a coarse or granular problem for the ®rst-stage
neural network and a re®ned problem for the second-
stage network. To coarsen the problem, it is transformed
by having a ®rst-stage neural network classify input fre-
quency to the beam so that it corresponds to one of seven
output classes of voltage. A backpropagation neural
network is then used to relate the vector of frequency and
classes to the optimal voltage. Using the granular de-
composition approach, Burke et al. report that an R2
value of 0.99 is obtained for both the 160 pattern training
set or full 200 pattern training set relating frequency and
class to the optimal voltage.
The concept of a two-stage hybrid neural network is
tested in this research to develop SDR prediction models
to determine whether any incremental improvements
could be obtained in prediction accuracy. Table 4 also
summarizes the results from these hybrid neural net-
works. The ®rst stage uses a Probabilistic Neural Net-
work (PNN) to classify the age of a DC-9 aircraft into its
corresponding `class' for the expected number of SDRs.
A PNN is a supervised neural network that is used to
train quickly on sparse data sets [30]. This neural network
separates input patterns into some de®ned output cate-
gories. In the process of training, the PNN clusters pat-
terns by producing activations in the output layer. The
value of the activations correspond to the probability
mass function estimate for that category. It was thought
that the use of a PNN is this study could be helpful in
`wrinkling' the SDR data and facilitate the classi®cation
of SDRs based upon an input pro®le of aircraft data.
For the overall SDR prediction model, the PNN is
used in this study to classify the number of SDRs into one
of four classes: class 1 for 0 £ S £ 2, class 2 for
2 < S £ 4, class 3 for 4 < S £ 6, and class 4 for
6 < S £ 8, where S represents the number of SDRs. The
PNN is used in the ®rst stage to classify the age of a DC-9
aircraft into its corresponding class for expected number
of SDRs. This vector of age and class then is fed into a
Luxhùj et al.
backpropagation neural network to predict the number
of SDRs. The second stage then feeds the classi®ed out-
put along with the above quantitative data to a back-
propagation neural network to predict the number of
SDRs. As with multiple regression, models were devel-
oped to predict the overall number of SDRs and the
number of SDRs for cracking and corrosion cases. For
the SDR cracking and corrosion cases, only two and
three `classes' were required, respectively, given the range
for the number of SDRs in each case.
In all SDR cases, prediction results with the hybrid
models were better on the training data than from solely
using a three-layer backpropagation architecture. How-
ever, the MSEs from the production data improved only
in the cracking case. Further investigations are required
with larger data sets to determine the extent of the ben-
e®ts of a two-stage approach, as the training time sig-
ni®cantly increases with the hybrid model.
10. Conclusions and recommendations
As a result of this research, new SDR alert indicators that
can have con®dence intervals have been developed to
predict the number of SDRs in the overall, cracking, and
corrosion cases for the DC-9 aircraft. These re®ned in-
dicators o€er signi®cant improvements over the planned
SDR indicator, which simply represents a count of the
total number of SDRs for any given period. For the
multiple regression analysis, the modeling strategy of
grouping the data on the basis of age, ¯ight hours, or
number of landings to predict the average number of
SDRs yielded better results than using the ungrouped
data. Initial results from ungrouped data with back-
propagation neural network models across a wide range
of hidden nodes, learning rates, momentum parameters,
and training strategy choices were poor. However, neural
network models with the data groupings from the re-
gression analysis achieved a marked improvement in
SDR prediction results.
When a hybrid neural network model was created that
used two stages, the SDR prediction accuracy on the
training data improved slightly for the models to predict
the overall number of SDRs and for cracking cases when
compared with a three-layer backpropagation archi-
tecture. However, results with the production data were
mixed. Three-layer backpropagation models were also
developed with the independent variables determined
from the regression analysis for each SDR case. However,
the prediction accuracy improved on the corrosion model
only for the training data, and improved with the pro-
duction data for the cracking model only when compared
with the backpropagation models using the original in-
puts of age, ¯ight hours, and number of landings.
The `best' models compared across modeling methods
are identi®ed in Table 5. The `best' models for each case
Prediction of inspection pro®les for aging aircraft 99
Table 5. Summary of `best' SDR prediction models across methods

SDRs overall SDRs cracking SDRs corrosion

2 2 2
Modeling method R MSE R MSE R MSE
Training Training Production Training Training Production Training Training Production
data data data data data data data data data
Regression 0.9297 0.1953 0.9219b 0.7899 0.0061 0.0265b 0.9780 0.0321 1.6918a
Backpropagation 0.9452 0.1520 0.5410a 0.6899 0.0090 0.4090 0.9411 0.0860 3.1250b
neural network (BPNN)
BPNN (with predictor 0.9318 0.1890 1.0170 0.6918 0.0090 0.0290 0.9806 0.0290 4.7030
variables from
regression)
Hybrid neural network 0.9603 0.1100 2.6260 0.8404 0.0050 0.0190a 0.9727 0.0400 3.5020
a
First choice.
b
Second choice.

were selected on the basis of prediction accuracy with the
production data. In the case to predict the overall number
of SDRs, the three-layer backpropagation model performs
the best. To predict the number of SDR cracking cases, the
two-stage hybrid neural network is selected, and a regres-
sion model is selected as the `best' method to predict the
number of SDR corrosion cases. However, an analysis of
Table 5 reveals that the regression models are strong sec-
ond choices with respect to prediction accuracy. Moreover,
the regression models typically take less time to develop
than neural network models and there is a rich theory for
testing regression model adequacy. The modest improve-
ments in predictive accuracy from using a neural network
in this SDR study do not seem to support the extra `costs' of
computational time and modeling e€ort required to ®nd a
neural network that can outperform a regression model.
In this study, the information gained from regression
analysis regarding the `best' data grouping strategies
helped to improve the performance of a neural network
model. However, the use of regression analysis to identify
the `best' set of explanatory variables to use as inputs to a
neural network needs further investigation.
In this study to develop national SDR prediction
models, the original ungrouped data set appeared to be
noisy. A `population concept' proved to be a very e€ec-
tive modeling technique both for regression analysis and
in the construction of neural networks for determining
strategic safety inspection indicators. An important
technical issue in using population modeling techniques
has to do with failure prediction for parts that have been
repaired or replaced. The failure rates of a new part in an
old aircraft and new versus repaired parts will a€ect the
inherent characteristics of the aircraft population. One
modeling approach is to assume that a repair returns the
part to the condition that it was in just before failure, so
that the part can remain in the same original population.
However, this assumption might not be realistic, and re-
paired parts might need to be modeled with a separate
grouping with new population characteristics. For a de-
tailed discussion of preventive maintenance in population
models, see Agee and Gallion [31].
While the population concept is constructive for de-
veloping models to predict national norms for SDR re-
porting, there is a loss of information in grouping the
data. It is recognized that SDR reporting pro®les will
vary by di€erences in ¯ying patterns, airlines, location,
¯eet size, etc. Research is continuing with alternative
FAA data sources to identify the variability inherent in
these underlying factors and its contribution to under-
standing national SDR reporting pro®les. The current
methodology could be extended, for example, to develop
SDR prediction models for the DC-9 aircraft for peer
classes of airlines based on ¯eet size. The CMAS research
team is gaining access to larger FAA data sources that
will enable the construction of SDR prediction models by
aircraft type and by airline. The de®nition of the classes
could be re®ned to depend on the size of the population
of aircraft of a given age owned by an individual airline.
This CMAS research has already been expanded to
include the use of neural networks to predict maintenance
requirements for individual components of aging aircraft
[32] and to develop reliability curve ®tting techniques for
constant and monotonically increasing hazard rates [33].
The use of an expert system that uses Bayesian prob-
ability theory to handle uncertainty in safety diagnostic
procedures is also being investigated. The research ®nd-
ings for the SDR prediction models coupled with the
design of an expert system for inspection diagnostics lead
to the development of an integrated decision support
system for FAA safety inspectors. Such a decision sup-
port system for aircraft safety could be used for inspec-
tion workload planning and scheduling, for monitoring
speci®c part locations and for forecasting part removal
dates for repair, overhaul, etc., based upon an aircraft's
`inspection pro®le' as characterized by its associated po-
pulation characteristics.
100
Acknowledgements
We acknowledge the support of the Federal Aviation
Administration's Safety Performance Analysis System
(SPAS) program and Mr John Lapointe and Mr Michael
Vu. In addition, we are grateful for the comments and
suggestions from Professor Candace Yano.
This article is based on research performed at Rutgers
University. The contents of this paper re¯ect the view of
the authors who are solely responsible for the accuracy of
the facts, analyses, conclusions, and recommendations
presented herein, and do not necessarily re¯ect the ocial
view or policy of the Federal Aviation Administration.
References
[1] Ascher, H.E. and Feingold, H. (1984) Repairable Systems Relia-
bility (Lecture Notes in Statistics, vol. 7), Marcel Dekker, New
York.
[2] Safety Performance Analysis Subsystem (1992) Functional De-
scription Document, U.S. Department of Transportation, Volpe
National Transportation Systems Center, Cambridge, MA,
March.
[3] Safety Performance Analysis Subsystem (1992) Prototype Con-
cept Document, U.S. Department of Transportation, Volpe Na-
tional Transportation Systems Center, Cambridge, MA, April.
[4] Safety Performance Analysis Subsystem (1992) Indicators Sug-
gested for SPAS Prototype (DRAFT), U.S. Department of
Transportation, Volpe National Transportation Systems Center,
Cambridge, MA, August.
[5] Safety Performance Analysis Subsystem (1992) Continuing Ana-
lysis: Indicator Graphs and Tables, U.S. Department of Trans-
portation, Volpe National Transportation Systems Center,
Cambridge, MA, October.
[6] Safety Performance Analysis Subsystem (1992) Continuing Ana-
lysis: Additional Indicator De®nitions (DRAFT), U.S. Depart-
ment of Transportation, Volpe National Transportation Systems
Center, Cambridge, MA, October.
[7] Rice, R.C. (1991) Repair database assessment. Battelle Summary
Report (Contract no. DTRS-57-89-C-00006).
[8] Brammer, K.W. (1985) A transient state maintenance require-
ments planning model. M.S. thesis, Virginia Polytechnic Institute
and State University.
[9] Fabrycky, W.J. (1981) Logistics systems design using ®nite
queueing analysis, in Proceedings of the International Logistics
Congress, pp. II-65±II-71.
[10] Fabrycky, W.J., Malmborg, C.J., Moore, T.P. and Brammer,
K.W. (1984) Repairable equipment population systems (REPS)
demonstrator user's guide (IBM-PC). Virginia Polytechnic In-
stitute and State University.
[11] Frisch, F. (1993) Mortality and spareparts: a conceptual analysis,
in Proceedings of the 1983 Federal Acquisition Research Sympo-
sium, pp. 467±480.
[12] Luxhùj, J.T. and Jones, M.S. (1988) A computerized population
model for system repair/replacement. Computers and Industrial
Engineering, 14 (3), 345±359.
[13] Luxhùj, J.T. and Rizzo, T.P. (1988) Probabilistic spares provi-
sioning for population models. Journal of Business Logistics, 9,
(1), 95±117.
[14] Luxhùj, J.T. (1991) Sensitivity analysis of maintained systems
using a population model: a case study. International Journal of
Quality and Reliability Management, 8 (1), 56±70.
Luxhùj et al.
[15] Luxhùj, J.T. (1991) Importance measures for system components
in population models. International Journal of Quality and Relia-
bility Management, 8 (2), 58±69.
[16] Luxhùj, J.T. (1992) Replacement analysis for components of large
scale production systems. International Journal of Production
Economics, 27, 97±110.
[17] Pindyck, R.S. and Rubinfeld, D.L. (1991) Econometric Models
and Economic Forecasts, McGraw-Hill, New York.
[18] Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (1992)
Multivariate Data Analysis, 3rd edn, Macmillan, New York.
[19] Hoaglin, D.C., Mosteller, F. and Tukey, J.W. (1983) Under-
standing Robust and Exploratory Data Analysis, John Wiley and
Sons, New York.
[20] McCulloch, W.S. and Pitts, W. (1943) A logical calculus of ideas
immanent in nervous activity. Bulletin of Mathematical Bio-
physics, 5, 115±133.
[21] Hop®eld, J.J. (1982) Neural networks and physical systems with
emergent collective abilities. Proceedings of the National Academy
of Science, 79, 2554±2558.
[22] Hop®eld, J.J. (1984) Neurons with graded response have collective
computational properties like those of two-state neurons. Pro-
ceedings of the National Academy of Science, 81, 3088±3092.
[23] Simpson, P. (1990) Arti®cial Neural Systems, Pergamon Press,
New York.
[24] Wasserman, P. (1989) Neural Computing: Theory and Practice,
Van Nostrand Reinhold, New York.
[25] Caudil, M. (1991), Neural network training tips and techniques,
AI Expert, 6 (1), 56±61.
[26] Specht, D. (1991) A general regression neural network. IEEE
Transactions on Neural Networks, 2 (6), 568±576.
[27] `NeuroShell 2' (1993) Ward Systems Group, Frederick, MD.
[28] Burke, L.I., Vaithyanathan, S. and Flanders, S.W. (1993) A hy-
brid neural network approach to beam vibration minimization.
Technical Paper, Department of Industrial Engineering, Lehigh
University.
[29] Weigend, A., Huberman, B.A. and Rumelhart, D.E. (1990) Pre-
dicting the future: a connectionist approach. International Journal
of Neural Systems, 1 (3), 193±210.
[30] Specht, D. (1990) Probabilistic neural networks. Neural Networks,
3, 109±118.
[31] Agee, M.H. and Gallion, M.S. (1986) Simulation of population
maintenance requirements. Volume II of a Final Report, Research
Contract No. N00039-84-C-0346 entitled Algorithmic Develop-
ment and Testing of Spare Parts Mortality/Support Systems
Software, Naval Electronics System Command.
[32] Shyur, H.-J., Luxhùj, J. T. and Williams, T. P. (1996) Using
neural networks to predict component inspection requirements for
aging aircraft. Computers and Industrial Engineering, 30(2), 257±
267.
[33] Luxhùj, J.T. and Shyur, H.-J. (1995) Reliability curve ®tting for
aging helicopter components. Reliability Engineering and System
Safety, 46, 229±234.
Biographies
James T. Luxhùj is Associate Professor of Industrial Engineering at
Rutgers University. He completed his Ph.D. in Industrial Engineering
and Operations Research from Virginia Polytechnic Institute and State
University. Dr Luxhùj serves as a Department Editor for IIE Trans-
actions on Operations Engineering, as an IIE Faculty Advisor, and is a
senior member of IIE. He is a past Director of the Engineering Econ-
omy Division of IIE. He is a recipient of the SAE Teetor Award for
Engineering Education Excellence. His research interests include sys-
tems maintenance and reliability, production economics, and intelligent
decision systems. He is a member of Tau Beta Pi and Alpha Pi Mu.
Prediction of inspection pro®les for aging aircraft
Trefor P. Williams is Associate Professor of Civil Engineering at
Rutgers University. He received his Ph.D. in Civil Engineering from
Georgia Institute of Technology. Dr Williams's research interests in-
clude the application of arti®cial intelligence to transportation and
highway engineering problems, construction management, and trac
engineering. He is a member of the American Society of Civil Engineers
and is a registered professional engineer.
101
Huan-Jyh Shyur received his Ph.D. in Industrial Engineering from
Rutgers University. He has a B.S. degree in Industrial Engineering
from Tunghai University, Taiwan. He received his M.S. degree in In-
dustrial and Systems Engineering from National Chiao-Tung Uni-
versity, Taiwan. His research interests include accelerated life testing,
quality and reliability, nonparametric methods, neural network mod-
els, and aircraft safety. He is employed with Crown Communications,
an FAA contractor.