PERSONNELPSYCHOLOGY

1982, 35

AN OBJECTIVE GROUP QUESTIONNAIRE AS A

SUBSTITUTE FOR A PERSONAL INTERVIEW IN
THE PREDICTION OF SUCCESS IN MILITARY
TRAINING IN ISRAEL
JOSEF H. TUBIANA AND GERSHON BEN-SHAKHAR'
Hebrew University of JeNSakm, Israel

The need for reliable and valid measures of personality and

motivational factors in the prediction of success in military training
was discussed. The personnel classification system currently used
by the Israeli Army was briefly described. The personality factors
used in that system are measured by an interview, which is
individually administered to each enlisted man. The goal of the
present study was to replace this interview by an objective group
questionnaire, with the hope of saving time, manpower and effort
without any loss to predictive validity. The criterion for validation
of the system was the performance of the soldiers in elementary
training. This performance was assessed by commanding officers
and by peers. The results showed that the questionnaire is eventu-
ally equivalent to the interview as a predictor of performance in
military training. It was concluded that the questionnaire should be
preferred for economical reasons.

MANPOWERadministration of big institutions can benefit from

psychological testing in several ways. Development of valid mea-
sures for predicting performance in different levels and tasks in the
institution is crucial for an efficient classification of manpower to the
different branches of the institution. It is also very important for
selecting individuals to specific tasks. The present paper deals with
' We wish to express our gratitude to the Psychological Unit of the Israeli Army,
for its participation in all stages of this study, for the helpful advice, and for the
financial support. We also thank the various commanders of all the units which
participated in the study, for their cooperation.
The study was based on M.A. thesis conducted by Josef H. Tubiana under the
supervision of Gershon Ben-Shakhar.
Address requests for reprints to: Gershon Ben-Shakhar, Department of Psycholo-
gy, The Hebrew University of Jerusalem, Jerusalem, Israel.
Copyright 0 1982 Personnel Psychology, Inc.

349
350 PERSONNEL PSYCHOLOGY

the Israeli Defense Army-an organization that faces serious prob-

lems of manpower planning.
It is clear that any valid measure for predicting success in the
army must include personality and motivational factors. The mea-
surement of such factors is often problematic, and less reliable than
the measurement of cognitive factors.
At present the Israeli Army bases its classification decisions on a
combined measure that includes the following four factors: (1)
Knowledge of the Hebrew language. This factor is measured both by
an objective vocabulary test, and by assessment of the expression
ability through an interview. (2) Level of education as measured by
number of years in school. (3) Level of intelligence as measured by
two objective aptitude tests-a spatial test, and a verbal one. (4) A
motivational factor measured by a structured interview which
focuses on the motivation of the soldier to serve in a combat unit.
The four factors are linearly combined. The linear weights were
derived in a regression analysis based on a big sample (Reeb, 1968).
The interview is conducted by soldiers (most of them girls), aged
between 18 and 20 years. These interviewers are high school
graduates who were selected and trained to do this work. The
interview lasts for about 20 minutes and is administered to all males
half a year before their enlisting date. The outcome of this interview
is an evaluation of the enlisting candidate on six scales plus a general
score that predicts the candidates prospects to adjust to a combat
unit. All scores are given by the interviewer on the basis of specified
instructions.
The six traits assessed in the interview are activeness, sociability,
responsibility, independence, promptness and motivation to serve in
a combat unit. It should be noted that these traits are very similar to
the ones identified by Egbert, Meeland, Cline, Forgy, Spickler, and
Brown (1957) as important for success in combat. A general score of
the interview is derived by a linear combination of the individual
traits. The reliability of the interview was estimated by the correla-
tion between the overall ratings of two independent interviews
conducted by different persons with a few days interval. Reeb (1968)
found a Pearson correlation of 0.80 between the scores given by the
two interviewers to a sample of over 200 soldiers. In addition, Reeb
reports a correlation of 0.32 between the general score of the
interview and a criterion of rank while terminating the military
service.
The use of an interview to measure personality traits and motiva-
tion could be problematic. The interpersonal setting of the interview
is likely to produce biases due to different cultural backgrounds of
 TUBIANA AND BEN-SHAKHAR 35 1

the candidates and the interviewers (Beveridge, 1968; Rich, 1968;

Sidney and Brown, 1959). In spite of its being a structured interview
it is still not objective and is most likely affected by different styles
of interviewing. This factor was mentioned by Heller (1966). In
addition, it must be remembered that an individual interview admin-
istered to masses of persons is very expensive in terms of time,
manpower, and money. The training of the interviewers is costly,
and new persons have to be trained every few months.
The main target of the present study was to replace the interview
with an objective group questionnaire. The questionnaire developed
was based on the following principles: (1) It must cover the same
content covered by the interview. (2) It must be written in a simple
language such that every enlisted man can easily understand its
content. (3) It must be objective and easy to score. It was hoped to
achieve several advantages by the use of such a questionnaire: (1)
To save time and money. (2) To achieve a standardized and
objective assessment of all enlisted men, and thus avoid the possible
biases of the interview. (3) A group questionnaire is easy to
administer, thus it would be possible to give it to the enlisted men on
their enlisting day. The interview is presently administered about
half a year before enlisting, due to technical problems.
It was hoped that the replacement of the interview with the
questionnaire would not reduce the general validity of the classifica-
tion system. The criterion for validating the questionnaire, in this
study, was the performance of the soldiers in their elementary
training. The period of elementary training lasts between two and
eight months in the different units. The performance was assessed
by the soldiers direct officer and by his peers. Several studies have
demonstrated that sociometric evaluation by peers is a very good
indicator for ability in different tasks in the Army (Amir, Kovarsky
and Sharan, 1970; Downey and Yates, 1976; Kaufman and Johnson,
1971).

Method
Subjects. The questionnaire was administered to several thousand
enlisted males.2 About 25% of them were located at the end of the
elementary training for the follow-up. The reasons for this selection
were: (1) Soldiers who had too short a training period, or who
postponed their training period were excluded from the sample. (2)
Soldiers for whom we could not get the criterion information were

* For security reasons, we could not report the exact number.

352 PERSONNEL PSYCHOLOGY

excluded. This was caused either by technical problems or by a lack

of cooperation on the part of the commanding officers.
In order to check for a possible selection bias, comparisons were
made between the final sample and the group of soldiers that was
excluded from the study. The two groups were compared with
regard to the questionnaire and the interview total scores. In both
cases, the final sample showed a slightly better adjustment score.
The differences, however, were small (0.13 and 0.07 standard
deviation units for the questionnaire and the interview total scores
respectively). The standard deviation of the two groups were
essentially the same.
Procedure. In a pretest the final version of the questionnaire was
administered to a random sample of 200 soldiers on their enlisting
day. This version included 45 multiple choice, typical behaviour
items, 35 of them had 5 alternative answers which ranked from the
most to the least adaptive. The order of the ranks within each
question was randomly changed to avoid a set. The soldiers were
instructed to choose one alternative answer for each question-the
one which seemed to describe their behaviour best. The additional
10 items were related to social desirability (S.D.); they were
translated from Edwards S.D. questionnaire (Marlow and Crown,
1967). Each S.D. item had two alternative answers (right, not right).
The soldiers were instructed to choose one of these alternatives.
The S.D. items were randomly mixed with the other questions.
As a result of the pretest, six of the original 35 items were
excluded from the questionnaire. The criteria for elimination of
items were: (1) Correlation of greater than 0.4 between a given item
and the total S.D. score (the number of S.D. answers given to the 10
S.D. items), (2) A correlation of less than 0.3 between a given item
and the questionnaire total score (the sum of the 35 item scores).
The resulting questionnaire included 29 items. Their distribution
among the different dimensions was: activeness-4, responsibility-6,
independence-3, promptness-4, motivation-3, sociability-4. Five ad-
ditiona1 items were aimed at predicting the soldiers prospects of
adjusting to military life. The final version of the questionnaire was
administered to the soldiers on their enlisting day. It was adminis-
tered to them in groups of 30-50 each. Standardized oral instructions
were given to the soldiers as follows: The present questionnaire is
designed to help the army understand your habits and behaviour in a
variety of situations. The questionnaire is not a test, so there are no
right answers. It includes questions which refer to your everyday
life, and to situations that you may encounter within the army. Read
carefully each question and choose truthfully the one answer which
 TUBIANA AND BEN-SHAKHAR 353

would best describe your behaviour. These instructions are the

same as those given before the regular interview.
The internal consistency reliability of the final version of the
questionnaire was estimated by Cronbachs LY coefficient (Cronbach,
1951). It was found that LY = .88 for the questionnaire total score.
The LY coefficients for the separate scales were: 0.62, 0.66, 0.59,
0.65, 0.64 and 0.62, for activeness, sociability, responsibility, inde-
pendence, promptness and motivation to serve in a combat unit,
respectively.
The soldiers were located at the end of their elementary training in
their units. In each unit the Commanding Officers assessed the
performance of their soldiers. They ranked each soldier on the traits
that were previously measured through the questionnaire. They also
evaluated the soldiers prospects of becoming a good officer. In each
unit, a sociometric questionnaire was administered to the soldiers.
Each soldier chose among his platoon members those members who
had the potential to become commanding officers.

Results
Several measures were defined for analyses in this study:
XI-The interview general score that is presently used in the Army.
It is defined as a weighted average of the traits measured in the
interview. (Each trait was given a score between 1 and 5 , 5
representing the most adaptive level for that trait.)
Xz-The Hebrew Language score given by the interviewer. This
score ranged between 1 and 9.
X3-Level of education defined as number of years in school.
&-Level of intelligence defined as the average of two objective
aptitude tests (a verbal test and a spatial one).
XS-The questionnaire total score was defined as a sum of the
individual questions scores. In each question the scores ranged
from 1 to 5; the questions were rescored so that 5 would
represent the most adaptive level.
&-The criterion score was defined as the average of the three
following criteria: (1) The ranking of the soldiers performance
in training by his commanding officer. ( 2 ) The soldiers chances
of becoming a commander, as assessed by his commanding
officer. (3) The percent of the cases in which the soldier was
chosen by his platoon members in the sociometric question-
naire.
The Pearson correlation coefficient between the interview (XI)
354 PERSONNEL PSYCHOLOGY

and the questionnaire score (X5)was .476. Pearson correlation

coefficients of X I- X5with X6were calculated. These correlations
appear in Table 1.
Two regression analyses were conducted to compare the contri-
bution of the questionnaire score with that of the interview score to
the general prediction system used in the Army. The first regression
analysis included the four predictors presently used by the Army
(XI- &). The multiple correlation of those four variables with the
criterion was .436, and the marginal contribution of the interview
score to the variance explained by the other 3 predictors was 4.46%.
In the second regression analysis we replaced the interview score by
the questionnaire score. The multiple correlation this time was .458,
and the marginal contribution of the questionnaire score to the
explained variance was 6.42%.

Discussion
The results showed that the interview and the questionnaire are
essentially equivalent predictors of the present criterion. The inter-
view -XI, correlated .360 with the criterion while the questionnaire
X5correlated .336 with the same criterion. The contribution of those
two measures to a combined predictor of the criterion is also very
similar. The set of predictors which includes the interview produces
multiple correlations of 0.436, with the present criterion. When the
questionnaire replaced the interview the multiple correlation was
raised to a value of 0.458.
The regression analyses also show that the marginal contribution

TABLE 1
Pearson Correlation Coeficients of the Predictors (XIto X,) with the Criterion
(Xd. Means and Standard Deviations of Each Predictor ( N = 459)
The predicting Pearson Standard
variables correlation Mean Deviation
XI-The interview
general score .360 24.03 4.67
X,The Hebrew
language score .235 8.42 .81
X3-Level of
education .212 11.59 1.30
&-Level of
intelligence .327 61.56 14.37
X-The questionnaire
total score .336 24.17 3.57
Note.--Au correlations arc significant at the ,001 level.
 TUBIANA AND BEN-SHAKHAR 355

of the questionnaire in terms of percentage of explained variance of

the criterion is somewhat bigger than the marginal contribution of
the interview. The explanation of that result may be derived from
Table 2. As can be seen from this table, the interview correlates
higher with the three other predictors used (X2, X3,and X4)than the
questionnaire. These differences in inter-correlations can be ac-
counted for by the subjective nature of the interview. It is obvious
that the interview situation cannot be limited to personality and
motivational variables, and such factors as intelligence, level of
education and knowledge of the language must reveal themselves. It
is, therefore, not surprising that the interview could have some
advantage over the objective measure of the questionnaire which is
more limited to personality and motivation. It is not clear, however,
what weights are given by the interviewers to the different factors
that determine the final score of the interview. According to Daws
(1979) it can be hypothesized that these weights are far from
optimal. It is thus plausible that by combining the personality
variables with other predictors, such as level of education and
intelligence, the objective tool could have an advantage. This
conclusion is in accordance with both the analyses of Meehl (1957
and Sawyer (1966). The present results demonstrate the possible
advantage of an objective questionnaire over the clinical interview
even when the latter is mechanically combined with other variables
as was recommended by Sawyer (1966) and by Shinedling, Howell
and Carlson (1975).
The relatively low correlation found between the interview and
the questionnaire scores is surprising. One possible explanation
could be that the interview is likely to be influenced by external
variables such as level of education and intelligence, and possibly
cultural background and appearance. These variables are not mea-
sured by the questionnaire.

TABLE 2
Pearson Correlation Coeficients of Variables X,,X3,X, With the Interview and
the Questionnaire Vaiables (XI,
X,)
( N = 459)
Hebrew language Level of Level of
score education intelligence
xz x3 x4
Interview score
XI .204 .416 .334
Questionnaire Score
XC .187 ,183 .I50
356 PERSONNEL PSYCHOLOGY

Two conclusions can be drawn from the present results: (1) An

objective questionnaire could replace the clinical interview without
any loss to predictive validity (possibly a gain in validity of a
combined predictor), and with a great profit in terms of time,
manpower and money. This is a technical conclusion that should be
of interest to the institution involved. (2) A more general conclusion
is that even the use of a structured interview with a reasonable
reliability could be problematic. It is true that such an interview
could be a valid predictor of performance, but this validity is
influenced by variables not intended to be measured by the inter-
view (level of education, intelligence, etc.). These external variables
could enhance the validity of the interview, but could reduce the
general predictive validity of a combination of variables. It seems to
us that any interviewer acts not only as a generator of information,
but at least to some extent, combines and summarizes the data. This
latter attribute of the interview makes it, in our view, inferior to a
mechanistic method of generating information. In addition, a ques-
tion could be raised as to the social fairness of the interview,
since it is more likely to be influenced by irrelevant variables such as
ethnic background, sex and general appearance.

REFERENCES
Amir, Y. Kovarsky, J. and Sharan, S. Peer nomination as a prediction of multistage
promotions in a remified organization. Journal of Applied Psychology, 1970, 54,
462-469.
Beveridge, W. E. Problem solving interviews. London: George Allen and Unwin
Ltd., 1%8.
Cronbach, L. J. Coefficient alpha and internal structure of tests. Psychometrika,
1951, 16, 297-334.
Dawes, R. M. The robust beauty of improper linear models in decision making.
American Psychologist, 1979, 34, 571-582.
Downey, R. G., Meland, F. F. and Yates, L. G. Evaluation of a peer rating system of
predicting subsequent promotion of senior military officers. Journal of Applied
Psychology, 1976, 61, 205-209.
Egbert, R. L., Meeland, T., Cline, V. B., Forgy, E. W., Spickler, M. W. and
Brown, C. Fighter 1: An analysis of combat fighters and nonfighters. Hum. RRO
Tech. Rep., 1957, 44, Vi - 69 p.
Heller, K., Davis, J. D., and Myers, R. A. The effects of interviewer style in
interview. Journal of Consulting Psychology, 1966, 30, 507-508.
Kaufman, G. G. and Jonson, J. C. Scaling peer ratings. An examination of the
differential validity of positive and negative nomination. Journal of Applied
Psychology, 1971, 59, 302-305.
Marlowe, D. and Crowne, D. P. The approval motive. Studies in evaluative
dependence. (Secondary printing, January, 1967). New York, London, Sydney:
John Wdey and Sons, Inc, 1967.
Meehl, P. E. When shall we use our heads instead of the formula? Journal of
Counseling Psychology, 1957, 4, 268-273.
 TUBIANA AND BEN-SHAKHAR 357

Rich J. Interviewing children and adolescents. London: Macmillan and Co., 1968.
Reeb, M. Construction of a questionnaire to replace a valid structured interview in
Israel Defense Forces. Megamot, Behavioural Sciences Quarterly, 1968, XVI, 69-
14.
Sawyer, J. Measurement of prediction, clinical and statistical. Psychological Bulle-
tin, 1966,66, 178-200.
Shinedling, M., Howell, J. Z., and Carlson, G . Another perspective on clinical
judgement. Psychological Reports, 1975, 36, 383-389.
Sidney, E. and Brown, M. M. The skills of interviewing. London: Traistock
Publication, 1959.

All in-text references underlined in blue are linked to publications on ResearchGate, letting you access and re