37-l
THE ROLE THAT COGNITIVE
SUMMARY
Crew Resource Management (CRM) is now an
important component of most airline training. It
is clear from the analysis of accidents that a
dysfunctional team in the cockpit Will cause
accidents.
Research in this area had focused on the degree
to which training cari facilitate team work in the
cockpit and the degree to which people may have
the personality structure, instrumentality and
expressiveness that would either promote or
interfere with their ability to work as a team
member. The research presented in this paper
examines four potential components of CRM.
These components are aviation knowledge,
general intelligence, personality, and cognitive
skills. In order to communicate effectively and
be a part of the team, the crew member must: 1)
know the subject matter that is the basis of work
that the team must execute; 2) have the
intelligence to understand dynamic situations,
formulate and communicate an idea; 3) be
willing to engage in an exchange of ideas; and 4)
have the processing capability to do multiple
tasks simultaneously and not be overwhelmed by
them.
Our paper Will present the results of a study of
115 first officers for a major airline. Our
dependent variables are scales derived from an
in-flight observation instrument design to
measure pilot performance in several areas -
Paper presented ut the RT0 HFh4
Systems”, held in Edinburgh,
ABILITY PLAYS IN CRM
Carl C. Hoffmann, Ph.D.
Kathleen P. Hoffmann
Hoffmann Research Associates
111 Providence Road
Chapel Hill, NC, 275 14, USA
Gary G. Kay, Ph.D.
Georgetown Medical Center
Washington, DC, 20007, USA
CRM among them. Our independent variables
are an extensive job knowledge test developed
by the airline, several standard intelligence tests,
a personality inventory and CogScreen. Our
findings indicate that cognitive ability is the
major component of predicting good
performance on our CRM scale.
INTRODUCTION
For 20 years, airlines have recognized that
communication and coordination of activities in
the cockpit is critical to crew performance. A
study by Cooper, White & Lauber (1979) (p.
207) (using (Ruffell Smith (1979)-simulator B-
747 study) set up a series of problems for a crew
on a simulated flight. They found that most crew
performance problems were related to
breakdowns in crew coordination, not to a lack
of technical knowledge and skill. “High error”
crews experienced diffculties in the areas of
communication, crew interaction and integration.
Other performance deficiencies were associated
with poor leadership and the failure of the flight
crew to exchange information in a timely
fashion.
Foushee and Manor (198 1) (p. 209) analyzed the
cockpit voice recordings from the Ruffell Smith
(1979) simulation study. Overall, there was a
tendency for crews who communicated less to
perform less well, but the type or quality of
communication played an even more pivotal
role. There was a negative relationship between
Symposium on “Collaborative Crew Perjormance in Complex Operational
United Kingdom, 20-22 April 1998, and published in RT0 MP-4.
37-2
crew member observations about flight status
and en-or related to the operation of aircraft
systems. In short, when more information was
transferred about aspects of flight status, fewer
errors occurred relative to such problems as
mishandling of engines, hydraulic, and fuel
systems, the misreading and missetting of
instruments, the failure to use ice protections,
and SOforth.
Researchers have generally focused on
personality factors as the cause of the breakdown
in crew communication. As a result,
interventions have addressed personality factors
in attempts at remedying the problem. Robert
Helmreich has pioneered this effort, focusing on
exploring positive and negative aspects of two
personality dimensions, instrumentality and
expressiveness. (Helmreich and Spence, 1978;
Spence, Helmreich, & Holahan, 1979). Data on
these two dimensions is obtained using a self-
report instrument called the EPAQ (Extended
Personality Attributes Questionnaire):
1. Instrumental Traits relate to
achievement and goal seeking
(achievement motivation)
Instrumentality (I+): a cluster of positive
attributes reflecting goal-orientation and
independence (active, self-confident, cari stand
up to pressure)
Negative Instrumentality (I-): negative
characteristics reflecting arrogance, hostility, and
interpersonal invulnerability
(boastlûl, egotistical, dictatorial)
2. Expressive Traits relate to interpersonal
behaviors, sensitivity, and orientation.
Expressivity (E+): a cluster of positive
attributes reflecting inter-persona1 warmth
and sensitivity (gentle, kind, aware of
feelings of others)
Negative Communication (E-): self-
subordinating, subservient, or unassertive
characteristics (gullible, spineless,
subordinates self to others)
Verbal Aggressiveness (Eva-): verbal
passive-aggressive characteristics
(complaining, nagging, fussy)
Helmreich and his colleagues subsequently
added three measures of achievement
motivation. The entire inventory is known as the
Persona1 Characteristics Inventory (PCI).
In 1986, Chidester, Hehnreich and others
collected data as part of a CRM training program
from two samples of military pilots in order to
explore and validate subgroups that could be
distinguished from their configuration of
personality characteristics. Three clusters were
identified:
(1) Positive Instrumental/ Interpersonal
cluster, characterized by elevated levels
of both positive instrumental and
expressive traits, and below average
levels on Negative Instrumentality and
Verbal Aggressiveness.
(2) Negative Instrumental cluster,
characterized by elevated levels of
positive and negative instrumental traits,
high verbal aggressiveness, work master,
and competitiveness, and low levels of
positive expressivity. “In a sense, pilots
fitting this cluster cari be characterized as
instrumental, but not at a11expressive.
Individuals whose traits resemble this
pattem might be best described as rugged
individualists rather than team players.”
(Chidester, 1986).
(3) Low Motivation (sometimes labeled
“Negative Expressive”) cluster,
characterized by below-average scores on
positive instrumental and expressive

traits; they show some elevation in
Verbal Aggressiveness and Negative
Instrumentality. These pilots appear to
be neither instrumental nor expressive,
but show some elevation on Verbal
Aggressiveness and Negative
Instrumentality.
These clusters were replicated in a second
sample of pilots. Chidester, et al., considered
Cluster 1 optimal for the close interpersonal
coordination required of crew members in
multicrew aircraft (Chidester, et al., 1991). This
same study reports also evidence that personality
cluster membership may set some limits on the
effectiveness of CRM training. Individuals in
the Positive Instrumental cluster members appear
to gain the most from training and Low
Motivation cluster members gain the least.
In related research by Chidester et al. (1990),
these clusters were examined in relation to pilot
performance in full-mission simulations. Crews
led by a Positive Instrumental captain were rated
by observers as consistently effective and made
the fewest errors during abnormal simulation
conditions. Crews led by Low Motivation
captains made signifïcantly more errors and were
rated as less effective. Crews led by Negative
Instrumental captains performed more poorly
initially, but by the last abnormal segment, these
crews were performing as well as Positive
Instrumental crews.
The willingness to work as a member of a team,
the desire to listen and express oneself, the
ability to subordinate one’s ego while
maintaining independent judgment, and the
conscientiousness required to adhere to protocols
of communication are clearly important
determinants of cockpit performance. However,
personality does not tel1 the whole story of
effective CRM. Pilots must not only have the
desire to communicate, they also have to have
something to communicate, and be able to
communicate while doing many other tasks.
37-3
Therefore, intelligence, experience, job
knowledge, and processing capacity should a11be
related to CRM.
The relationship of general intelligence, “g”, to
pilot performance is as well documented as any
relationship in aviation psychology. In a study
of 1,400 undergraduate navigator trainees and
4,000 undergraduate pilot trainees, the best
predictor of overall performance, including tests
and performance in the aircraft, was g, followed
by knowledge of aviation information and
instrument comprehension (Olea and Ree, 1994,
p. 848). In fact, when measuring individual
performance, some prominent authors in aviation
argue that other measures show only small and
marginal incremental validity.
Although the names and
appearance of the tests used in
pilot selection vary, most are
measures of g. The
incremental validity of specifïc
knowledge (e.g., aviation
information, comprehension of
aircraft terms), psychomotor
abilities and personality scores
has been shown to be small but
signiflcant. (Ree and Carretta,
1995).
Experience also plays a role in CRM. In
examining decision making, authors have found
experience to be very predictive of performance
(O’Hare and Wiggins, 1993). While much of
this work has been done on individuals, other
authors have pointed to the group dynamic in
decision making in commercial aircraft. These
authors recognize the interaction of individual
abilities and styles and crew coordination.
Mosier-O’Neill and others have recognized the
interaction of individual abilities and styles and
crew coordination-individual within a group
(crew) within an organization (Mosier-O’Neill,
1989). There is a necessity among the crew for
shared assessments,shared mental models, and
37-4
coordination of actions. As a consequence, one
could expect experience in type of aircraft or
training to affect how pilots approach CRM.
Presumably, those pilots trained in equipment
requiring crew coordination, i.e., military airlift
command (MAC), may have more training and
knowledge of CRM than individual fighter
pilots.
Finally, effective performance as a pilot--and
presumably in CRM--requires the ability to
multitask, to do the job you are assigned while
coordinating with others.
A time-sharing ability refers to
the ability to perform multiple
tasks in combination.
Assessing a time-sharing
ability is important for
selection and training in any
given complex task situation.
(Ackerman, Schneider, &
Wickens, 1984, p. 71).
This belief is shared by other authors (Damos,
1993; and Imhoff and Levine, 1981, p. 74) and is
supported by validation studies conducted with
CogScreen-Aeromedical Edition (reported in
Kay, 1995).
CogScreen is a computerized cognitive screening
test originally designed for the U.S. Federal
Aviation Administration as an instrument for
evaluating pilots’ neuro-cognitive fitness-to-fly.
The selection of tests for CogScreen was based
on existing task analyses of the cognitive and
psychomotor demands of flying. Prior studies
have demonstrated that CogScreen is not only
sensitive to changes in brain functioning
resulting from trauma, substance abuse or illness,
but that it is also predictive of flight
performance. The cockpit-observed performance
deficits reported for pilots who had been referred
for clinical CogScreen examinations have been
shown to be reflected in their CogScreen
performance (Kay, ASMA92). Studies of the
relationship between CogScreen and flight
performance have also looked at operational
flight data. In one study, the frequency and
severity of flight performance violations was
measured by flight data recorders. The study
(Yakimovich, ASMA95) demonstrated that
variables from a small number of CogScreen
tests (i.e., Dual Task Test, Shifting Attention
Test, Divided Attention Test, Pathfïnder, and
Backward Digit Span) were able to account 30%
to 45% of the variante in the performance
violation index. Nearly a11of these tests measure
aspects of multitasking (i.e., divided attention,
mental flexibility, planning, and sequencing).
THE STUDY
We have examined the extent to which
persona@ dimensions, intelligence, experience,
aviation knowledge, and cognitive capacity cari
predict performance on crew resource
management tasks. Such a mode1 could help in
pilot selection, especially for commercial airlines
that rely greatly on hiring pilots already trained
in a number of different environments with
varying backgrounds in intelligence, job
knowledge, experience, training, and various
levels of ability to multitask. Along these lines,
we developed and conducted our study as part of
a pilot selection process for a major United
States airline.
In 1996 and 1997, this airline sought to validate
its newly designed selection procedure. The
major components of that selection procedure
were measures of general intelligence, job
knowledge, structured behavioral interviews,
personality measures, and tests of cognitive
ability. These measures were to be related to
performance in training and performance on the
job. The study included observations of
performance in the cockpit as part of a line audit
procedure, and a forma1 instrument was used as
part of the line audit. The instrument was based
upon a job analysis that established both the
behaviors on the job and the knowledge, skills

and abilities required for those behaviors.
Current pilots for the airline were to be tested on
the proposed selection battery, and information
was to be collected about their training and
performance on the job. The components of the
battery that were most predictive of the outcome
measures were then be used as part of the new
selection process. As part of the validation
process, a contrasting group of low-hour,
weekend general aviation pilots were used to
help add variability on the proposed selection
instruments.
POPULATION
The group that was studied and whose
information is reported here were a set of
randomly selected civilian airline pilots who had
been hired by the airline between 1988 and 1991
and who, at the time of the study, had reached
the position of First Officer. The vast majority
of the pilots in this group were trained in either
the Navy or the Air Force. They were fùrther
selected to be above the 50* percentile of
intelligence for Air Force Captains, and in fact
over 60% of this group was above the 80”
percentile. Further, this group had been screened
at the time of hire for both company and cockpit
“fit.” Pilots who were not interested in
continuing to fly and who were viewed as having
personalities that would conflict with the culture
of the company or with crew resource
management were selected out. Finally, these
individuals were put through a psychological
screen, and anyone displaying psychopathology
or a highly defensive personality was also
screened out. As a consequence, our study
involved a very homogenous group of aviators.
These people were a11well trained, highly
knowledgeable, very intelligent, almost a11male,
and almost a11Caucasian.
VARIABLES
Outcome Measures
37-5
Al1 training episodes from hire until June of
1997 were recorded for each of the 115 pilots in
our study. Any evidence of problems in training
was recorded. Difficulty in written or oral
exams, extra time in the simulator or in training
in general, and problems with proficiency checks
were a11noted. Our predictive variable was
dichotomous:
0 = no problems; 1 = evidence of any problems.
Fifieen percent of this group reported having
some problem in training during their career at
this airline.
This constrained variability led us to derive
additional outcome measures of performance to
validate the potential selection procedures.
Therefore, a new line evaluation instrument was
designed to satisfy the following criteria:
Be more sensitive to differences in
performance than current line check
ratings, thus providing needed variante
for validating selection devices;
Be valid, that is, reflect behaviors that are
part of a pilot’s job at the airline;
Be composed of items that are observable
and thus cari be reliably rated;
Have scales that are easy to interpret;
Be easy to use and capable of being Med
out in real-time, on the flight deck.
The new evaluation tool was pre-tested and
applied as part of a general line audit conducted
at the airline. It was used to evaluate
performance of First Officers comprising the
validation sample, as well as a sample of
Captains and First Offïcers covering the entire
theater of operations at the airline.
The instrument was developed based on a job
analysis performed for the selection process.
Line Check Airmen participated in focus groups
to revise task statements and provide ratings of
the relevance of each of the task statements. The
focus groups also worked to minimize the effect
of the type of aircraft, time in aircraft, familiarity
31-6

with airport, and weather conditions. We further Personality - (see Costa and McCrae, 1992)
controlled these factors throughout the project Neuroticism
through data collection and statistical controls, as Openness
well as rater training. Clearly defined standards Agreeableness
were developed to rate each task and to aid inter- Conscientiousness
rater reliability. The instrument was initially Extroversion
tested with an industrial psychologist, the Chief
of Flight Standards and the Chief of Line Check Knowledge-Based Tests
Airmen. Extensive training sessions were held
with Line Check Airmen that concentrated on Knowledge of Aerodynamics
applying the rating scales uniformly and Knowledge of Engineering
establishing procedures covering data collection Knowledge of Navigation
and completion of the rating forms. Each of our Knowledge of Meteorology
115 pilots was observed as pilot flying SOas to Knowledge of Aviation Physiology
compare performance of the same activities.
This information was collected and analyzed not Cognitive Processing - As measured by
only for our 115 pilots, but also for the 1500 CogScreen subtests:
Captains and First Offïcers observed in line audit
procedures. A factor analysis was done on the Math Accuracy
results and four factors emerged from the Visual Sequencing
analysis (see Table 1). The items associated with Symbol Digit Coding
CRM activities and their factor loadings are Matching to Sample
recorded in Appendix A. Predictor variables The Manikin Test
were derived from the following sources: Divided Attention
Auditory Sequence Comparison
Tests of Ability and Aptitude Pathjinder
Knowledge-Based Tests Shifting Attention Test
Personality Dual Tasking
Cognitive Processing
Prior Experience Prior Aviation Experience

Tests of Ability & Aptitude Fighter experience

Numerical Ability - As measured by the
Differential Aptitude Test
Verbal Ability - As measured by Differential
Aptitude Test
Mechanical Ability - As measured by the
Bennett Mechanical Aptitude Test
Spatial Ability - As measured by the
Minnesota Paper Form Board Test
Test of Reasoning - As measured by the
Watson-Glaser Test of Critical Thinking
Nonverbal Aptitude - As measured by
Raven’s Progressive Matrices
Large jet air transport
Turbo prop
PROCEDURES
Tests were administered over a two-day period,
with the knowledge-based test administered first.
The other tests were given in six different
sequences in order to prevent fatigue from
systematically affecting performance. Al1 tests
were administered and scored by computer. The
information was compiled and added to the line
observation data and training information, as
well as the pilot’s prior experience before being
hired by the airline. Only individuals with

complete information on a11measures were used
in the study, leaving 110 analyzable subjects.
RESULTS
The mean and standard deviations for the
variables used in this analysis are shown in Table
2. A histogram showing the distribution of CRM
scores is found in Figure 1. Clearly, this
outcome variable has more variante than those
involving simple passifail dichotomies.
In Table 3, we present the correlation
coefficients for each independent variable with
the CRM measures. Several findings are
notable. Using a 0.10 significance level as a first
indicator, we see that Divided Attention
Sequence Comparison Speed is negatively
related to CRM, while Divided Attention
Sequence Comparison Throughput is positively
correlated to CRM. The faster pilots make
comparisons, the higher their CRM rating.
Speed has the same relationship for the Manikin
test, and Shifting Attention Test Arrow Color
Condition. CRM is significantly correlated with
two accuracy measures: Matching to Sample
Accuracy and Shifting Attention Test Discovery
Accuracy. On process measures, significant
correlations were found for Shifting Attention
Test Discovery Condition Rule Shifts Completed
and Number of Failed Sets. The more successful
pilots are at systematically and flexibly applying
rules, the better they are at CRM. The Dual
Tracking Test (Boundary Hits) also produces
significant associations, but in unanticipated
ways: the correlation is significant - but non-
linear. The more tracking en-ors pilots made
(within limits), the better they are at CRM. No
intelligence tests or knowledge-based tests are
signifïcant. Furthermore, among the personality
measures, only Agreeableness is significantly
associated with CRM. Two experience variables
are associated with CRM: Prior Fighter
Experience is positive, while Jet Transport
Experience is negative.
37-7
A regression mode1 limited to CogScreen
variables (with r values at the p< 0.1 level); past
flight experience; Agreeableness; aviation
knowledge (of aerodynamics, engineering and
navigation); and the intelligence tests was used
to find a predictive equation composed of
variables that contributed significantly to CRM
performance at the .05 level of significance. The
final mode1 arrived at through stepwise
regression accounted for 35.3% of the variante
in CRM. These results are shown in Table 4.
DISCUSSION
From the parameters of the regression equation,
we see that there are many components that
predict CRM. Some of the fïndings make sense
intuitively, while others are surprising. As
expectedly, Agreeableness is positively
correlated with CRM. Agreeableness here is
defïned as trust, straightforwardness, altruism,
compliance, modesty and tender-mindedness.
Clearly, these personality dimensions are
indicative of someone who cari cooperate,
although not necessarily subserviently.
Knowledge of aerodynamics is positively
correlated with CRM, which suggests that the
more pilots know about the principles of flight,
the better they cari communicate. This
knowledge helps facilitate communication in tha
it provides the substance of the communication
and indicates a lesser degree of cognitive effort
and analysis required of pilots who cari quickly
access and recall their knowledge.
Of the three measures from CogScreen, two have
the expected relationship to CRM. Match to
Sample Accuracy is positively correlated, while
Divided Attention Sequence Comparison Speed
is negative, meaning the more accurate pilots are,
and the less time they spend reacting, the better
their scores on CRM. This finding indicates that
processing speed and accuracy facilitate CRM.
The one inconsistent relationship from
CogScreen is dual tasking: the more errors a
pilot makes on this measure, the higher the CRM
37-8
score. This relationship could be explained by a
particular test-taking strategy used by the better
pilots. They are apparently willing to let some
errors occur while they are optimizing both tasks
on the test; in other words, focusing on Perfect
accuracy here may result in lowered performance
in other areas.
We cari see several other surprising results.
Individuals with background in jet transport who
have been instructors scored lower than those
w-ithout these aspects of experience. In fact, this
finding is somewhat ironie, since these
individuals have been most likely exposed to
crew situation during training. Also ironically,
pilots with fighter experience and instructor
experience had a positive correlation to CRM
rating. (See Table 3.) This runs counter to the
popular belief that fïghter pilots are
individualistic, egocentric, non-team players. It
may be a result of the extensive screening for
cockpit fit that went into selecting these
individuals. Or, it could also be due to the
military’s practice of placing better students in
the fighter pilot career path. Finally, there is a
slight negative cor-relation of the Differential
Aptitude Test (DAT) measure of overall aptitude
and CRM. This relationship may be in part due
to the poor Upper-end differentiation of the DAT.
Overall, this mode1 explains 35% of the variante,
which is quite high compared to similar studies.
If the sum of squares is partitioned, the result is
that 33% of the variante is explained by
cognitive measures, 25% by experience, 16% by
agreeableness, 13% by knowledge and 9% by
intelligence.
From this study and this population, we see that
it is important to look beyond personality to
explain how pilots Will perform at CM.
Clearly, knowledge, cognitive ability, prior
experience and intelligence a11play a role in
CRM. The variables that play an important role
in this study are also significant in other studies
on how cognitive ability relates to performance,
as we discussed in the Introduction. In this
study, cognitive ability played the greatest role in
predicting performance on CRM. Our fïndings
indicate that not only must pilots have the kind
of personality that cari cooperate in the cockpit,
but they must also have the knowledge and
cognitive resources to perform well. Further, it
is also clear that prior experience working with
crews in large aircraft does not necessarily
indicate good CRh4 performance.
It should be kept in mind that this group of pilots
was a unique, relatively similar, group. Again,
they had almost a11been trained in the U.S.
military; they a11had to be at least at the median
level for Air Force Captains; and they a11had to
pass rigorous psychological screening and
persona1 interviews. But their homogeneity
makes it even more surprising that we cari see
and explain as much variante as we cari in such a
group. The screen on intelligence may explain
the negative relationship to the DAT found in the
regression equation.
Cognitive ability, along with job knowledge,
experience, personality, and general ability, play
a role in other measures of performance. In
Table 5, we present a grid that lists the
components of our measures that predict positive
performance in training and in two other
measures of cockpit performance: procedural
compliance and aircraft control activities. Here
we see that experience, personality, cognitive
ability and job knowledge, among the pilots we
tested, were important factors associated with
other performance measures. The factors
involving multi-tasking are the most important
factors in CogScreen. Notably absent from these
bivariate correlations are the measures of general
intelligence, but this is most likely due to the
narrow range of variante for the pilots that we
observed. Finally, pilots’ ability to perform on
knowledge-based tests is associated with their
ability to perform in training.
We believe that it is important to collect data on
populations that are far less homogeneous SOwe
 37-9

cari get a better indicator of what role cognitive

ability plays in CRM. Theoretically, it cari be
argued that because our sample was SOwell
trained, SOknowledgeable and SObright,
cognitive ability plays less of a role here than it
would among a broader population of pilots.
Because these pilots know their jobs well and
have been performing them for years, SOmuch of
what they do is now routine and “hard-wired”,
requiring less stress on cognitive capacity. On
the other hand, it cari be argued that cognitive
ability plays an important role here primarily
because it is the one dimension on which these
pilots were not originally screened. Only by
reproducing this study in a variety of populations
cari we build a mode1 of how a11of these factors
relate to performance in the cockpit.
37-10

BIBLIOGRAPHY

Ackerman, P.L., Schneider, W. & Wickens, C.D., “Deciding the Existence of a Time-Sharing
Ability: A Combined Methodological and Theoretical Approach.” Human Factors, 26, 1984,
pp. 71-82.

Chidester, T.R., Helmreich, R.L., Gregorich, S.E. & Geis, C.E., “Pilot Personality and Crew
Coordination: Implications for Training and Selection,” International Journal of Aviation
Psychology, 1(l), 1991, pp. 25-44.

Chidester, T.R., Kanki, B.G., Foushee, H.C., Dickinson, C.L. & Bowles, S.V., “Personality
Factors in Flight Operations: 1. Leader Characteristics and Crew Performance in Full-Mission
Air Transport Simulation”, (NASA Tech Memorandum No. 102259), Moffett Field, CA, NASA-
Ames Research Center, 1990.

Cooper, G.E., White, M.D., and Lauber, J.K. (Eds.), “Resource Management on the Flight De&”
(NASA Conference Publication No 2 120; NTIS No. N80-22283), Moffett Field, CA, NASA-
Ames Research Center, 1979.

Costa, Paul T., Jr., and McCrae, Robert R., Revised NE0 Personality Inventory (NE0 PI-R) and
NE0 Five-Factor Inventory (NEO-FFI), Psychological Assessment Resources, Inc., 1992.

Damos, Diane L., “Using Meta Analysis to Compare the Predictive Validity of Single- and
Multiple-Task Measures to Flight Performance”, Human Factors, 35, 1993, pp. 615-628.

Foushee, H.C. & Helmreich, R.L., “Group Interaction and Flight Crew performance.” In E.D.
Wiener & D.C. Negel (Eds.), Human Factors in Aviation, San Diego, CA, Academic, 1988, pp.
189-227.

Helmreich, R. & Spence, J., “The Work and Family Orientation Questionnaire: An Objective
Instrument to Assess Components of Achievement Motivation and Attitudes toward Family and
Career”, JSAS Catalog of Selected Documents in Psychology, 8, 1978, p. 35.

Imhoff, D.L., & Levine, J.M., “Perceptual-motor and Cognitive Performance Task Battery for
Pilot Selection”, (AFHRL Tech. Rep. No. 87-20, AD-A094 3 17), Brooks Air Force Base, TX,
Manpower & Personnel Division, Air Force Human Resources Laboratory, 1981.

Kay, Gary G., CogScreen: Professional Manual, Psychological Assessment Resources, Inc.,
Odessa, Florida, 1995.

Kay, G.G., Hordinsky, J.R., 8z Pakul, B., “Neuropsychological Assessment of Aviators: A

Comparison of Traditional and Computer-based Approaches”, Annual Meeting of the Aerospace
Medical Association, 1992.

Mosier-O’Neill, K.L., “A Contextual Analysis of Pilot Decision Making.” In R.S. Jensen (Ed.),
Proceedings of the Fijlh International Symposium on Aviation PsychoZogv, Columbus, OH, Ohio
State University, 1989.
 37-11

O’Hare, David & Wiggins, Mark, “Expertise in Aeronautical Decision Making: A Cognitive
Ski11Analysis.” In R.S. Jensen & D. Neumeister (Eds.), Proceedings ofthe Seventh
International Symposium on Aviation Psycholoay, Columbus, OH, Ohio State University, 1993.

Olea, M.M. & Ree, M.J., “Predicting Pilot and Navigator Criteria: Not Much More Than g”,
Journal of Applied Psychology, 79, 1994, pp. 845-95 1.

Pred, R.S., Spence, J.T. & Hehnreich, R.L., “The Development of New Scales for the Jenkins
Activity Survey Measure of the Type A Construct”, Social and Behavioral Science Documents
(MSNo. 2769), 16, 1986, pp. 51-52.

Ree, M.J., and Carretta, T.R., “The Central Role of g in Military Pilot Selection”, Armstrong
Laboratory, Human Resources Directorate, Manpower and Personnel Research Division, 1995.

Spence, J., Hehnreich, R.,& Holahan, C., “Negative and Positive Components of Psychological
Masculinity and Femininity and their Relationships to Self reports of Neurotic and Acting-act
Behaviors”, Journal of Personality and Social Psychology, 37, (1979), pp. 1673-1682. [This
may be the original validating study on the Personality Attributes Questionnaire.]

Yakimovich, N.V., Strongin, G.L., Govorushenko, V.V., Schroeder, D., & Kay, G.G., “Flight
Performance and CogScreen Test Battery Performance in Russian Pilots”, Annual Meeting of the
Aerospace Medical Association, 1995.
37-12

Table 1

Factors Cronbach’s Alpha

Procedures Compliance and Checklists .92

Crew Resource Management .96

Aircraft Control Activities .90

Planning and Preparation .83

 37-13

Figure 1

-1.4 -1.1 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.7 0.8 0.9 1

Factor Scores
37-14

Table 2
Means and Standard Deviations of Measures with Signifïcant Correlations with CRM

Measure n Mean Standard

Deviation
Backward Digit Span Accrcy 110 87.17 15.79
Math Accrcy 108 70.37 14.07
Math Speed 108 27.30 8.40
Math Thrpt 108 1.72 0.71
Visual Seq Comprsn Accrcy 110 98.45 2.69
Visual Seq Comprsn Speed 110 2.17 0.48
Visual Seq Comprsn Thrpt 110 28.52 5.96
Symbol Digit Coding Accrcy 110 99.36 1.28
Symbol Digit Coding Thrpt 110 34.67 5.74
Immediate Recall Accrcy 110 96.23 10.18
Delayed Recall Accrcy 110 92.3 1 18.80
Matching to sample Accrcy 110 95.73 4.68
Matching to sample Speed 110 1.36 0.24
Matching to sample Thrpt 110 43.56 7.67
Ma&in test Accrcy 110 95.64 6.21
Manikin test Speed 110 1.47 0.32
Manikin test Thrpt 110 40.74 8.78
Div attn test Indctor alone speed 110 0.34 0.08
Div attn test Indctr alone prem resp 110 3.43 2.26
Div attn test Indctr dual speed 110 0.61 0.20
Div attn test Indctr dual prem resp 110 1.67 1.93
Div attn test Seq comprsn accrcy 110 95.11 7.27
Div attn test Seq comprsn speed 110 2.19 0.71
Div attn test Seq comprsn thrpt 110 28.37 8.04
Auditory seq comprsn Accrcy 109 97.06 5.49
Auditory seq comprsn Speed 109 0.61 0.14
Auditory Seq comprsn Thrpt 109 100.28 23.34
Pathfinder Number accrcy 109 99.53 1.54
Pathfinder Number speed 109 0.79 0.16
Pathfinder Number thrpt 109 78.77 15.29
Pathtïnder Number coordination 109 1.15 0.54
Pathfinder Letter accrcy 110 99.35 2.02
Pathfinder Letter speed 110 0.77 0.39
Pathfinder Letter thrpt 110 82.75 15.55
Pathfinder Letter coordination 110 1.25 0.47
Pathfinder Combined accrcy 110 98.49 2.94
Pathtïnder Combined speed 110 1.10 0.47
Pathfinder Combined thrpt 110 58.74 14.55
Pathfinder Combined coordination 110 1.17 0.42
Shftng attntn test Arrow dirctn accrcy 110 99.19 2.70
Shfing attntn test Arrow dirctn speed 110 0.53 0.09
 37-15

Shftng attntn test Arrow dirctn thrpt 110 114.27 18.24

Shfing attntn test Arrow color accrcy 110 99.32 3.04
Shfing attntn test Arrow color speed 110 0.61 0.09
Shftng attntn test Arrow color thrpt 110 99.39 15.07
Shftng attntn test Instrctn accrcy 110 98.45 2.39
Shfing attntn test Instrctn speed 110 0.72 0.13
Shftng attntn test Instrctn thrpt 110 84.26 14.85
Shftng attntn test Discvry accrcy 110 65.12 14.12
Shftng attntn test Discvry speed 110 0.90 0.26
Shftng attntn test Discvry thrpt 110 47.01 15.39
Shftng attntn test Discvry rule shft cmp 110 6.77 2.71
Shftng attntn test Discvry failed set 110 1.82 1.60
Shftng attntn test Discvry persev err(s) 110 2.35 3.08
Shftng attntn test Discvry noncncpt resp 110 2.32 3.45
Dual task test Tracking alone error 108 16.67 14.38
Dual task test Tracking boundary hits 108 1.88 3.17
Dual task test Tracking dual error 109 53.49 25.07
Dual task test Tracking dual hits 109 2.48 2.91
Dual task test Prev num alone accrcy 109 92.81 6.59
Dual task test Prev num alone speed 109 0.50 0.26
Dual task test Prev num alone thrpt 109 138.97 63.31
Dual task test Prev num dual accrcy 108 87.44 11.53
Dual task test Prev num dual speed 108 0.57 0.26
Dual task test Prev num dual thrpt 108 118.75 74.27
Bennets number correct 109 58.32 5.00
DAT num number correct 110 20.88 2.77
DAT vrb number correct 110 28.10 1.98
DAT tot number correct 110 48.98 3.76
JKT tot number correct # Crrct-rescore 109 69.92 9.13
JKT Aerodynamics % Crrct-rescore 109 75.73 12.86
JKT Air Navigation % Crrct-rescore 109 70.02 12.69
JKT Engineering % Crrct-rescore 109 85.12 9.42
JKT Aviation Physiology % Crrct-rescore 109 67.58 27.39
JKT Meteorology % Crrct-rescore 109 64.80 27.33
JKT Airmans Info Manual % Crrct-rescore 109 28.90 33.54
MIN tot number correct 110 48.27 6.34
Ravens tot number correct 109 23.98 4.07
Watson tot number correct 109 32.74 4.17
Total interview score 110 30.15 3.86
Neuroticism 106 53.77 14.79
Extraversion 106 122.49 12.87
Openness 106 109.19 15.15
Agreeableness 106 126.75 15.39
Conscientiousness 106 138.33 13.77
Fighter, capt instr, trner instr, ME jet 110 0.45 0.50
Transport, capt instr, trner instr, ME jet 110 0.29 0.46
37-16

Table 3
Correlation of Predictor Measures with CRM

Measure n Correlation Significance

Coefficient Level
Backward Digit Span Accrcy 110 -.ll .25
Math Accrcy 108 .08 .44
Math Speed 108 -.09 .36
Math Thrpt 108 .07 .50
Visual Seq Comprsn Accrcy 110 .lO .31
Visual Seq Comprsn Speed 110 -.03 .72
Visual Seq Comprsn Thrpt 110 .07 .50
Symbol Digit Coding Accrcy 110 -.08 .42
Symbol Digit Coding Thrpt 110 .06 .54
Immediate Recall Accrcy 110 .ll .25
Delayed Recall Accrcy 110 -.12 .21
Matching to sample Accrcy 110 .20 .03
Matching to sample Speed 110 .03 .72
Matching to sample Thrpt 110 .02 .84
Manikin test Accrcy 110 -.05 .57
Manikin test Speed 110 -0.17 .08
Manikin test Thrpt 110 .12 .20
Div attn test Indctor alone speed 110 .007 .94
Div attn test Indctr alone prem resp 110 -.05 .58
Div attn test Indctr dual speed 110 .04 .70
Div attn test Indctr dual prem resp 110 .07 .48
Div attn test Seq comprsn accrcy 110 .08 .40
Div attn test Seq comprsn speed 110 -0.16 .lO
Div attn test Seq comprsn thrpt 110 .18 .06
Auditory seq comprsn Accrcy 109 -.05 .62
Auditory seq comprsn Speed 109 .04 .67
Auditory Seq comprsn Thrpt 109 -.04 .70
Pathfïnder Number accrcy 109 .03 .79
Pathfïnder Number speed 109 -.ll .28
Pathfïnder Number thrpt 109 .09 .38
Pathfinder Number coordination 109 -.04 .65
Pathfinder Letter accrcy 110 .02 .82
Pathfïnder Letter speed 110 -.15 .13
Pathfinder Letter thrpt 110 .02 .80
Pathfïnder Letter coordination 110 .08 .39
Pathfinder Combined accrcy 110 -.lO .28
Pathfïnder Combined speed 110 -.lO -30
Pathfinder Combined thrpt 110 -.02 .84
Pathfinder Combined coordination 110 .08 .38
Shftng attntn test Arrow dirctn accrcy 110 -.06 .56
Shftng attntn test Arrow dirctn speed 110 -.09 .34
Shftng attntn test Arrow dirctn thrpt 110 .07 .50
 37-17

Shftng attntn test Arrow color accrcy 110 .lO .29

Shftng attntn test Arrow color speed 110 -.17 .08
Shftng attntn test Arrow color thrpt 110 .16 .09
Shftng attntn test Instrctn accrcy 110 .ll .24
Shftng attntn test Instrctn speed 110 -.13 .19
Shftng attntn test Instrctn thrpt 110 .12 .20
Shftng attntn test Discvry accrcy 110 .16 .lO
Shftng attntn test Discvry speed 110 -.002 .98
Shftng attntn test Discvry thrpt 110 .12 .22
Shftng attntn test Discvry rule shft cmp 110 .19 .05
Shftng attntn test Discvry failed set 110 -.18 .05
Shftng attntn test Discvry persev err(s) 110 .07 .44
Shftng attntn test Discvry noncncpt resp 110 -.ll .26
Dual task test Tracking alone error 108 .003 .97
Dual task test Tracking boundary hits 108 -.07 .48
Dual task test Tracking dual error 109 .20 .03
Dual task test Tracking dual hits 109 .16 .09
Dual task test Prev num alone accrcy 109 -.03 .78
Dual task test Prev num alone speed 109 .lO .32
Dual task test Prev num alone thrpt 109 -.ll .24
Dual task test Prev num dual accrcy 108 .04 .66
Dual task test Prev num dual speed 108 -.09 .34
Dual task test Prev num dual thrpt 108 .12 .86
Bennets number correct 109 -.Ol .90
DAT num number correct 110 -.06 .52
DAT vrb number correct 110 .02 .87
DAT tot number correct 110 -.04 .70
JKT tot number correct # Crrct-rescore 109 .09 .33
JKT Aerodynamics % Crrct-rescore 109 .13 .19
JKT Air Navigation % Crrct-rescore 109 .06 .57
JKT Engineering % Crrct-rescore 109 .006 .94
JKT Aviation Physiology % Crrct-rescore 109 .03 .78
JKT Meteorology % Crrct-rescore 109 .Ol .88
JKT Airmans Info Manual % Crrct-rescore 109 .04 .66
MIN tot number correct 110 -.08 .42
Ravans tot number correct 109 .14 .13
Watson tot number correct 109 -.04 .70
Total interview score 110 .05 .58
Neuroticism 106 -.13 .17
Extraversion 106 .08 .40
Opemress 106 -.09 .35
Agreeableness 106 .25 .009
Fighter, capt instr, trner instr, ME jet 110 .21 .03
Transport, capt instr, trner instr, ME jet 110 -.31 .OOl
37-18

Table 4
Regression Analysis of CRM Mode1 R2 = .353

Variable Parameter Signifïcance

Estimate Level

Transprt, capt instr, trnr instr, ME jet -0.3609 0.0001

(A) Agreeableness 0.008 1 0.0020
JKT Aerodynamics % Crrct-rescore 0.0092 0.005 1
Matching to sample Accrcy 0.0257 0.0025
Div attn test Seq comprsn speed -0.1231 0.0278
Dual task test Tracking dual hits 0.0304 0.0239
DAT tot number correct -0.0273 0.0182

Intercept -2.4685 0.0115

 37-19

Table 5
Significance of Bivariate Correlations at .l Level

Tests of Ability and Aptitude Training Procedural Aircrafi CRM

Compliance Control
Numerical Ability
Verbal Ability
Mechanical Ability
Spatial Ability
Test of Reasoning
Nonverbal Aptitude

Personality Training Procedural Aircraft CRM

Compliance Control
Neuroticism
Openness
Agreeableness X X
Conscientiousness X
Extroversion

Dual Tasking
37-20

Prior Experience Training Procedural Aircrafi CRM

Compliance Control
Fighter experience X X X
Large jet air transport X X X
Turbo prop
 37-21

ACKNOWLEDGEMENTS

Our sincere thanks and appreciation to:

Sally H. Spetz, Ph.D.

April Sansom
37-22

APPENDIX A

CRM
Factor
Item Loading

16.4 Establishes and reinforces two-way communication 0.63

16.2 Provides feedback/accepts critique 0.61
21.7 Contributes proactively to the selection of a
course of action 0.60
21.8 Accepts and executes decisions when finalized 0.59
16.5 Asserts perspective safety and/or efficiency 0.56
17.4 Informs captain of task progress and status 0.56
17.3 Resolves disparities in interpretation, priority
and technique 0.55
21.4 Reviews assumptions and decisions before
selecting a course of action 0.53
21.1 Looks for multiple tues to identify the problem 0.52
17.1 Ba&s up other crew members 0.51
18.2 Adapts to changes 0.51
19.1 Prioritizes individual tasks 0.51
21.3 Contributes proactively to the research of options 0.50
16.1 Listens actively 0.49
20.2 Ensures that distractions do not detract from
overall crew situational awareness 0.49
18.1 Plans ahead 0.49
8.7 Maintains heading/navigation, altitude,
airspeed tolerances 0.45
18.3 Executes plans as briefed 0.45
16.6 Asks for clarification when necessary 0.44
20.3 Maintains automation mode awareness 0.44
17.2 Discloses mistakes and/or limitations promptly 0.43
20.1 Maintains situational awareness throughout flight 0.43
21.5 Considers operational priorities and risk
when selecting a course of action 0.42
19.2 Prepares for high workload during low workload 0.41
19.4 Uses the appropriate automation level for
reducing workload 0.41
21.2 States symptoms, not conclusions, when
initially identifying the problem 0.40
16.3 Uses standard terminology 0.38
14.2 Monitors autoflight systems for proper flight
path control and performance 0.35
21.6 Considers time restraints when selecting a
course of action 0.33