A Contribution to the Study of Actuarial and Individual Methods of Prediction

Author(s): Theodore R. Sarbin

Source: American Journal of Sociology, Vol. 48, No. 5 (Mar., 1943), pp. 593-602
Published by: The University of Chicago Press
Stable URL: http://www.jstor.org/stable/2769183 .
Accessed: 15/01/2015 19:21

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

The University of Chicago Press is collaborating with JSTOR to digitize, preserve and extend access to
American Journal of Sociology.

http://www.jstor.org

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 A CONTRIBUTION TO THE STUDY OF ACTUARIAL AND
INDIVIDUAL METHODS OF PREDICTION'
THEODORE R. SARBIN

ABSTRACT
Arepredictionsofconductmoreaccuratewhenmade by case-studymethodsthan by actuarial methods?
This studyis an experimentalratherthan a polemicalattemptto determinetherelativeaccuracyofthe two
modes of prediction.A restrictedformof behavior-academic success-was predictedfromclinical (case-
study)materialand froma previouslyderivedregressionequation. The correlationcoefficients demonstrate
that the case-studymethodwhichpresumablyaccounts foran innumerableassortmentof variablesis no
moreaccurate than a simplestatisticalmethodwhichaccountsforonly two variables. Analysisof the pre-
dictionssuggeststhat the case-studymethodtakes behaviorsegmentswithknownpredictiveweightsand
applies otherweightswhichare less efficient.Case-studypredictions-at least on thegroundsofefficiency-
shouldnot be substitutedforactuarialpredictions.As a complementto the actuarialpredictions,theclini-
cal predictionsadd nothing.The possibilityof generalizingto the predictionof otherkinds of criteriais
discussed.

INTRODUCTION That this hypothesisis too generalto

The object of thispaper is to illuminate submit to experimentalanalysis becomes
some of the darkcornersof the conceptof immediately evident. First,the term"be-
predictionin the social sciences. In recent havior"heremeanstoo much. Second,no
yearsa numberof writers2 have expressed standardsexistto tellus whena case study
themselves to the effectthatthe clinicalor is complete. Third,statisticalindicesand
individualmethodofpredicting behavioris regressionequationsare not available for
superiorto theactuarialor statisticalmeth- the predictionof manyformsof behavior.
od. In orderto determine the soundnessof In orderto make the hypothesistestable,
thisassertion,one wouldhave to testa hy- we must narrowit down to coincidewith
pothesiswhichmightbe formulated some- theserequirements:(a) the criterion to be
whatas follows:A completecase studywill predictedmustbe subjectto definition and
increasethe accuracyof predictionof be- measurement;(b) experiencetables or re-
haviorover that obtainedfromthe use of gressionequationsmustbe availablebefore-
statisticaltables based on experiencewith hand in orderto make statisticalpredic-
relativelyfewvariables. tions;(c) theindividualswhosebehavioris
to be predictedmusthave had at least one
I This articleis a condensationoftwopapers,one
clinicalinterview;and (d), in additionto
of whichwas read at the I94I meetingsof the Mid-
determined
thestatistically variables,other
westernPsychologicalAssociation,the other at a
seminaron "Problems and Methods of Prediction" data whichare presumably associatedwith
conductedby ProfessorE. W. Burgess duringthe thecriterionmustbe made availableto the
author's tenureat the Universityof Chicago as a clinician.
Fellow ofthe Social Science Research Council. I am In performingtheirdailyactivities,clini-
indebtedto Dr. E. S. Bordinformanyvaluable sug-
gestionsand to Dr. J. G. Darley forpermissionto cal counselors3make predictionsof aca-
use data fromthe clinical filesof the Universityof demicachievement.They providea situa-
MinnesotaTestingBureau. tionwhereby thegeneralhypothesis maybe
2
Among them are: G. W. Allport,"The Psy- testedand wherethefourrequirements just
chologist'sFrame of Reference,"PsychologicalBul- mentionedare satisfied.The criterionof
letin,XXXVII (I940), I-28; E. W. Burgess and
L. S. Cottrell,Jr.,PredictingSuccess or Failure in 3Clinical counselors are clinical psychologists
Marriage (New York: Prentice-Hall,I939); M. S. with experienceand trainingin workingwith col-
Viteles,"The ClinicalViewpointin Vocational Psy-lege students. Four of the five cliniciansin this
chology,"Journal ofAppliedPsychology,IX (I925), studypossessed the Ph.D. degreeor its equivalent.
I3I-38; HowTo CounselStudents All fivehad had considerableexperiencein clinical
E. G. Williamson,
(NewYork: McGraw-Hill BookCo., I939). counselingworkwithuniversitystudents.
593

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
594 THE AMERICAN JOURNAL OF SOCIOLOGY

successis honor-point ratio,admittedly not gressionequationand thensolvingforthe

a highlyreliablemeasurement, butnonethe- most probable honor-pointratio. These
less one thatis in use in mostinstitutiQns of predictionswere likewisecorrelatedwith
higherlearning.It is definedas theratioof honor-point ratios. Table i gives the re-
creditstogradeswhichhavebeenconverted sults.
into honorpoints. Previouslyderivedre- By comparingthe correlations obtained
gressionequationswereavailable in which by the clinicalmethodwiththoseobtained
academic achievementis predictedby a fromtheactuarialmethod,we see thatthe
combination oftwovariables: rankin high- correlations are not significantly different.
schoolgraduating classand collegeaptitude The apparentdifferences whichseem to fa-
test. All subjectshad one interviewwith vor the statisticalmode of predictionare
the clinicianprior to exposureto college shownto be notsignificant4 by theapplica-
classes. Data-availableto thepredictors in tionofFisher'sz test.5 If thetrendwereto
additionto themeasurement variableswere remainthe same with largernumbersof
in theformof additionaltestsof aptitude,
achievement, vocationalinterest,and per- TABLE 1
sonality;an eight-pageindividualrecord COEFFICIENTS OF CORRELATION BETWEEN AC-
TUAL HONOR-POINT RATIOS AND CLINICAL
form;a preliminary interviewer's formand
AND STATISTICAL PREDICTIONS
impressions;and, finally,the counselor's
ownobservations.Now thehypothesis may
be statedmorespecifically:By virtueofthe Type ofPrediction Men Women
case-studymethodemployed, clinical coun- Clinical............... . 35 .69
selors'predictionsofacademicsuccesswillbe Statistical............. .45 .70
moreaccuratethanthosedetermined fromre-
gressionequations.In short,the testingof
cases,we couldsafelyconcludethatstatisti-
thehypothesis involvescomparingthepre-
equationson cal predictionsare moreaccurate. At any
dictionsmade fromregression
rate,we can draw the conclusionthat the
two variablesto the predictionsmade by
in thisstudydid not\pre-
cliniciansfroman innumerable assortment clinicalcounselors
dict college achievementmore accurately
ofvariables.
thandid thestatisticalmethod.
DATA AND RESULTS
The plausibilityof the hypothesisthat
case-studypredictionsare superioris not
This bringsus to a consideration of the withoutfoundation.Startingfroma posi-
data. Predictionsof academicachievement tionwhichis logicallyacceptable-namely,
weremade forI62 Freshmen-73men and that itemsin the historyof an individual
89 women-whomatriculated in the fallof are associatedwithsubsequentbehavior-
I939 in theartscollegeoftheUniversity of cliniciansseekto discoverand weightitems
Minnesota.Thesepredictions weremadeby in theclient'spreviousand currenthistory.
fiveclinicalcounselorson thebasis of tests, Information gatheredfroma surveyof the
information obtainedfromthe individual individual'spast, then,should have value
recordforms,preliminary interviewer's re- forpredicting eventsin thefuture.That the
ports,and whateverdata mightbe gathered two itemsin the regressionequation-the
and summarizedin the interview.These high-school rankand collegeaptitudetest-
weremadeon an eight-point scale and were
ra- 4 The usual confidencelimits are used in this
correlatedwiththe actual honor-point
study. If a difference is significantbeyondthe i per
tios at the end of the quarter. Statistical cent point, we considerit a real difference.A dif-
predictionsweremade by the simpleproc- ferencethat is significantto the 5 per cent point is
ess of substituting the values of the two suggestiveand calls forfurtherexperimentation.
variables-high-school rankand collegeap- s R. A. Fisher, StatisticalMethodsfor Research
titudetest-in the previouslyderivedre- Workers(London: Oliver & Boyd, I938), p. 2I5.

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 ACTUARIAL AND INDIVIDUAL METHODS OF PREDICTION 595

could summarizeall the pertinentand sig- in whichtestsand estimatesof traitsother

nificantdata thatcan be useddoesnotseem than those determinedby aptitude tests
plausible. So manyothervariablesseemto werecorrelatedwithachievement.9 His re-
be untouched.What of strivings, habitsof port shows essentiallynegative results.
workand play,specialaptitudes,emotional Whateverthe value of thesedata forpro-
patterns,systematicdistractions, and the dictingotherformsof behavior,theyseem
hundredsofotherconditions whichseemto to be valuelessforpredictingachievement
be relatedto this complexformof social in college.
psychological behaviorknownas academic The data just mentioned were,ofcourse,
achievement?Have theseformsofbehavior onlya fewofthemanykindsthatare avail-
no influence on thecriterionotherthanthat able to the clinician.The questionstillre-
accountedforby the two variables? The mains: Of the wealthof data available to
answeris usuallygivenin theaffirmative- the interviewer, how muchis used system-
thatthesetraitsareassociatedwiththecri- aticallyin makingpredictions of academic
terionof academic achievementand, fur- success?The knowledgethat all thiscase-
ther,thatthesesubtlefactorscan be identi- studyinformation is at hand in contrastto
fiedand weightedby skilfulinterviewers.the usual fewvariablesin regression equa-
Someoftheseso-called"nonintellective" tions has led many interviewers to work
factorswere systematicallyobserved on froma hypothesissuch as the following:
each of thecases in thisstudyby meansof approximately correctaverageweightsare
the StrongVocationalInterestBlank6and appliedto testand otherdata by clinicians
theDarleyPersonalInventory.7 Fourmeas- -even to the measurements forwhichre-
uresderivedfromtheStrongblankwerecor- gressionweightshave been determined. If
relatedwith(a) actualachievement and (b) this hypothesisis true,then the multiple
clinicalpredictions.None of the correla- correlation betweenthecriterion
coefficients
tions obtained was significantly differentand themeasuresin theregression equation
fromzero. These measureswere: (i) pres- willnotdiffer significantlyfromthemultiple
enceor absenceofprimary patternofinter- correlationcoefficients
oftheclinicalpredic-
est;8 (2) agreementof expressedinterest tionswiththesamemeasures.
with measuredinterest;(3) masculinity- Two ways of testingthishypothesis are
femininity; and (4) occupationallevel(this available. The twomeasuresin theregres-
variableappliesonlyto themenand is con- sionequationcan be correlatedwithactual
sideredby some to be an estimateof the gradeaveragesand also withclinicalpredic-
levelofvocationalaspiration).On theDar- tions.By usingR2,thecoefficient ofmulti-
ley inventoryno significantassociations ple determination, we can makeroughcom-
could be foundwithfourof the traitscon- parisonsto see theproportion ofthevariance
sidered. These traitshad been tentatively accountedforby the two measures. An-
named "personalmorale," "social adjust- othermethod,indicatedby Bordin,IO is to
ment," "familyadjustment,"and "emo- determinethe varianceratio, that is, the
tionality." square of the standarderrorof estimateof
That these variablesare not associated one multipleR divided by the square of'
withcollegeachievement is no newfinding. the standarderrorof estimateof the other
Harrishas reviewedseveralhundredstudies multipleR. Thisgivesus theactualvariance
6 StanfordUniversity:StanfordUniversity 9Daniel Harris, "Factors AffectingCollege
Press,
I938.
Grades: A ReviewoftheLiterature,I930-37," Psy-
chologicalBulletin,XXXVII (I940), I25-66.
7 NewYork: Psychological
Corporation,
I94I.
IO E. S. Bordin,"Tests of Significance
Appropri-
8J. G. Darley, "A PreliminaryStudy of Rela- ate to PredictionStudies" (in preparation). This
tionsbetweenAttitude,Adjustment,and Vocational test is not entirelyappropriate,forwe have to as-
InterestTests," Journalof EducationalPsycIology, sume the equivalence of the means and sigmas for
XXIX (I938), 467-73. both the honor-pointratios and the predictions.

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
596 THE AMERICAN JOURNALOF SOCIOLOGY

accountedforin the predictedvariableby the F values in Table 3 are significant to

the predictorvariables. First we examine the i percentpoint. Thesedata maybe in-
Table 2 to see if the multipleR values ap- terpreted in thisfashion:Moreof thevari-
pear different. anceintheclinicalpredictionscan be attrib-
The coefficients of correlation(R) are uted to the use of the twomeasuresunder
easilyconvertedintocoefficients ofmultiple discussionthancan be attributedby these
determination (R2).II For men the R2 val- same measuresto the actual achievement.
ues are .3I and .49,respectively.Interpret- These measures-the collegeaptitudetest
ing,we wouldsay thatthehigh-school rank and rankin high-school graduatingclass-
and collegeaptitudetestaccountfor3I per havemeaningsfor theinterviewers di.fferent
centofthevarianceinhonor-point ratioand fromwhattheyhavein termsofachievement.
for49 per cent in the clinicalpredictions. Furthermore, the cliniciansdependalmost
For womenthe figuresare 46 and 66 per entirely on thesetwovariablesas seenfrom
cent,respectively.It would appear, then, the highmultipleR's whenclinicalpredic-
thatat least i8 or 20 percentofthevariance tionsare correlatedwithhigh-schoolrank
in theclinicalpredictions was due to "other and collegeaptitudetest.
TABLE 2 TABLE 3*
MULTIPLE CORRELATION COEFFICIENTS OF SQUARED STANDARD ERRORS OF ESTIMATE
CLINICAL PREDICTIONS AND OF HONOR- FOR MULTIPLE CORRELATION
POINT RATIOS WITH HIGH-SCHOOL RANK COEFFICIENTS IN TABLE 2
AND COLLEGE APTITUDE TEST

Men Women
Men Women
0-1 *23 . ............. -54 .40
Honor-point ratios with high- 0-4 * 232 . ............. .13 .1 2
school rank and aptitude test. .56 .68 F .4.27 3.22
Clinical predictions with high-
school rank and aptitude test.. .70 .8i rank; 3 =
*Code: i = Honor-pointratio; 2 = High-school
Collegeaptitudetest;4 Clinicalpredictions.

factors"ifthetwovariableshad beengiven In thelightofthesedata it wouldappear

efficientweightsbytheclinicians.However, that the case-studymethodtakesbehavior
the previouslysubmittedevidenceon the segmentswithknownweightsand applies
nonintellective factorssystematically stud- otherweightswhichare less efficient.
ied showsthatthose"otherfactors"at least The data, to this point, demonstrate
have no consistent relationshipwiththecri- that, because they are no more accurate,
terion. At this point we may safelysay clinicalpredictionscannotreplacestatisti-
that the cliniciansoverestimate the contri- cal predictions.This leads to the question:
butionof the two measurement variables. If thecliniciancannotreplacethestatistical
In orderto show that the varianceac- prediction, can he serveas a complement to
countedforby thesemultipleR's cannotbe it? This question can be answeredquite
consideredhomogeneous, we use Snedecor's easily by pooling the clinicalpredictions
varianceratio. We findF by dividingthe with the high-school rank and the college
squareof the standarderrorof estimateof aptitudetest. By comparingthe multiple
the multipleR for clinicalpredictionsby correlation withand withoutthe
coefficients
thesquareofthestandarderrorofestimate clinicalpredictions, we can answertheques-
ofthemultipleR fortheactualhonor-point tion(see Table 4).
ratios(see Table 3). Neitherofthedifferences shownin Table
For the appropriatedegreesof freedom, 4 is statisticallysignificant.These results
Methods (New are to be expectedin viewofthehighinter-
I, J. P. Guilford,Psychometric

York: McGraw-HillBook Co., 1936), p. 386. correlationsbetween clinical predictions

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 ACTUARIALAND INDIVIDUAL METHODS OF PREDICTION 597

and high-school rank and collegeaptitude ferentfromthe statisticalpredictions.For

test. As reportedbefore,thepredictions of womenthechi-squarevalue showsa doubt-
the counselorsare based forthe mostpart ful difference-favoring the actuarialpre-
on the same data as go into the regression dictionsin regardto accuracy. (When the
equation. We can draw the conclusion, predictions forthe menand thewomenare
then,thatclinicalpredictionsdo notadd to combined,thechi-squarevalue is belowthe
thevalidityofstatisticalpredictions
ofaca- 2 percentpointofsignificance-again favor-
demicachievement. ingtheactuarialmethod.)The obviouscon-
The clinicalpredictions
weremade on an clusionis that the interviewers in thisex-
eight-stepscale. Some clinical workers perimentcould predictacademic achieve-
maintainthatpredicting involvestoo much ment no better than a simple regression
guesswork whendealingwithso finea scale. TABLE 5
Afterall, theysay,in practicewe makepre- ANALYSIS OF CLINICAL PREDICTIONS AND
dictionsin termsof successor failure,not ACTUARIAL PREDICTIONS IN TERMS
OF SUCCESS AND FAILURE
TABLE 4
MULTIPLE CORRELATION COEFFICIENTS OF HON-
HONOR-POINTRATIO
OR-POINT RATIOS WITH HIGH-SCHOOL RANK
AND APTITUDE TEST AND WITH HIGH-
SCHOOL RANK, APTITUDE TEST, AND PREDIC- PREDICTION Men Women
TIONS

Success Failure Success Failure

Men Women
Clinicalpredic-
Honor-point ratios with high- tions:
school rank and aptitude test. . 56 .68 Success ....... 25 I5 49 20

Honor-point ratios with high- Failure. II 2I 2 i8

school rank, aptitude test, and
clinicalpredictions...... ...... *57 .73 Statisticalpre-
dictions:
Success... ... 27 I2 48 I4
Failure. IO 24 3 24
usuallyin termsofdegreeof successor fail-
ure. We can do better,theycontinue,ifwe P>.20; P<
x2= I.25; X2=4.3I; .05.
can make predictionson a two-pointscale
wheresuccessis consideredto be "C" aver- equation,whetherthe criterionwas scaled
age or betterand failureless than"C" av- in eightstepsor in two.
erage. Successwouldthenbe defined as i.OI Up to nowwe have consideredthevalid-
honor-point ratioor higher;failure,as any as comparedwith
ityof clinicalpredictions
value below. actuarial predictions. Nothing has been
The predictionsof the counselorsand said withregardto the reliabilityor con-
those of the regressionequation were as- sistencyof thetwomodesofprediction.In
sembled in fourfoldtables, as shown in the plan of thisexperiment it was possible
Table 5. The chi-squaremethodwas used to determinethe consistency of thepredic-
to testthehypothesis.'2The actuarialpre- tions by two methods: (a) a case-reader
dictionsare taken for the hypothetical or read each case and made predictionsinde-
standarddistribution;the clinicalpredic- pendentlyof the counselorsand (b) six
tions,as theexperimental distribution. monthslater the same case-readerre-read
This analysisreveals that formen the all buthisowncases and againmadepredic-
are not significantly
clinicalpredictions dif- tions. In this way we had determinations
of reliabilityanalogous to the "alternate
I2 Because of the small numberof cases in some
form"and "test-retest"methodsused in
of the cells,Yates's correctionhas been applied (see
G. W. Snedecor,StatisticalMethods[Ames,Iowa: computingtest reliability.Table 6 shows
CollegiatePress, I937], pp. I60-62). thereliabilities.

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
598 THE AMERICAN JOURNALOF SOCIOLOGY

We must not confusethe reliabilityof the way to betterpredictions,we should

predictions
statistical as representedbysuch know, for example,whetherinterviewers
a statisticas thestandarderrorof estimate overestimate or underestimate in theirpre-
withtheconsistency ofestablishinga predic- dictionsand whethertheirpredictionsfol-
tionfroma regression equation. Giventhe low thesame dispersionas thecriterion.
equation and the values of the predictor The questionsto be answeredare two:
variables,the consistencyof such predic- "Are the means of the clinicalpredictions
tionswouldbe represented by a coefficientunderestimations or overestimations of the
ofunity.That is to say,theregression equa- actual honor-point ratios?" and "Are the
tionwillalwaysyieldthe same resultfora meansof the actuarialpredictions underes-
givensetofscores.This is notso in thecase timationsor overestimations?" (see Tables
of clinicalpredictionswherethecorrelation 7 and 8).
of one case-readerwithhimselfis only.78, TABLE 7
and ofcounselorswithcase-readers, .68. MEANS OF CLINICAL PREDICTIONS AND HONOR-
POINT RATIOS AND CRITICAL RATIOS FOR
TABLE6 DIFFERENCES BETWEEN MEANS
RELIABILITY COEFFICIENTS OF COUNSELORS'
PREDICTIONS WITH CASE-READER'S PREDIC-
Men Women
TIONS AND CASE-READER'S PREDICTIONS
WITH SECOND PREDICTIONS Clinical predictions .... I .24 I.50
Reliability Honor-pointratios. ....... .92 I . I9

Coefficient
Differences ............. .32 .3I
Criticalratios......... ... 3.I5 4.57
withcase-reader.68
All counselors
CounselorA ............. .64
CounselorB ....... .. .82 TABLE 8
CounselorC .............. 88
MEANS OF STATISTICALPREDICTIONS AND HON-
CounselorD ............. . 72
OR-POINTRATIOS AND CRITICAL RATIOS OF
CounselorE .. . . 68
DIFFERENCES BETWEEN MEANS
Case-readerwithself......... 78
Fromthepointofviewofefficiency in the Men Women
practicalsituationwherepredictionsserve
Statisticalpredictions... . I .04 I.I7
as the basis foremployment or collegead- Honor-pointratios........ .92 I9 I.

missions,the regressionequation is to be Differences ............ .. . I 2 .02

preferred.The marginoferroris knownfor Critical ratios ......... ... I .35 .9I

theregression equation(S.E.est.); forclinical

predictionsthe marginof errorvariesfrom As seenin Table 7, theinterviewers over-
one interviewer to another,and thiserroris estimate the
college achievementof the
added to thatwhichis inherent in thedata
group studied.The criticalratios, deter-
used. In termsofconsistency ofprediction,
minedbyLindquist'sformula,'3 whichtakes
then,theactuarialis to be preferred to the
intoaccounttherelationship betweenvari-
clinicalmethod. ables,showthedifferences to be significant.
None of the previousevidenceprovides The means of the statisticalpredictions as
detailedinformation withregardto thena- shownin Table 8
revealthat theyare not
tureof errorscommittedby the clinicians significantly different fromthe criterion.14
in makingpredictions.To determine in de-
tail thecauses forclinicalpredictions being R-
I3
CR. MIM-M2
lessaccurateis beyondthescopeofthepres- JD2 + S _
entpaper. The experimental designdoesal- 2r(SD.,)(SD..)

low,however, forcomparisons to be made of I4 For some suggestionsas to the reason forthe

two important statistics, namely, the overestimationson the part of the interviewers, the
mean and the variance.In orderto point readeris referredto thewriter'sPh.D. thesis,on file

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 ACTUARIAL AND INDIVIDUAL METHODS OF PREDICTION 599

We may conclude,then,that,on the aver- to themean,thesafertheprediction.As a

age, clinicalinterviewers overestimatethe matterof fact, the interviewers used the
gradestheirclientswillreceive. uppermosttwo intervalsonlya fewtimes;
One morequestion: "Arethedispersions thelowesttwo,hardlyat all. Hanks found
of theclinicalpredictions and of thestatis- similarresultsin studying prediction ofper-
ticalpredictions less thanthedispersions of sonalitytestscoresfromcase material.'7He
the honor-point ratios?" This questionis interpreted the decreasedvarianceto be "a
stimulatedby theclaimof some advocates feelingof caution" on the part of the
of thecase-study methodofprediction that judges.
the regressionequation demonstratesthe
TABLE 9
well-known phenomenonof statisticalre-
gression-predictions are therefore bunched STANDARD DEVIATIONS OF CLINICAL PREDIC-
toocloseto themean. The case-study meth- TIONS, STATISTICAL PREDICTIONS, AND HON-
od can allow forthisand makepredictions OR-POINT RATIOS, WITH VARIANCE RATIOS
whichfollowthe distribution curveof the
criterion.If we can showthatthevariance Men Women
oftheclinicalpredictions is closeto thevari- Clinical predictions....... . 50 . 6o
ance of the criterion,then the case-study Honor-pointratios........ .89 .86
methodhas a remarkably uniqueproperty. Variance ratio (F) ......
Statisticalpredictions .. .
. 3.I3 2 .05
.42 .50
We knowbeforehand that the distribution Honor-pointratios........ .89 .86
of predictions fromthe regression equation Variance ratio (F) ...... . 4.49 2.96
willnot be the same as the distribution of
thecriterion, althoughits formwillbe sim-
ilar (see Table 9). SUMMARYOF RESULTS
Fordegrees offreedom (N - I), 72, 72, i. Many well-knownsocial scientists
and 88, 88, respectively, formenand wom- maintain,expresslyor by implication, that
en, theF valuesare significant to the i per the case-studymethodof predictionis su-
centpoint. The distributions ofboththesta- periorto theactuarialmethod.Evidenceis
tisticaland theclinicalpredictions aresignifi- submittedin this studywhichshowsthat
cantlydifferent, therefore, fromthe distri- clinical predictionsof academic achieve-
butionof thehonor-point ratios.'5 ment,made on an eight-step scale, are not
That decreasedvarianceis the chiefdis- moreaccuratethan statisticalpredictions.
advantageof the regressionequation was 2. Factorssuchas interest, inferred level
pointedoutlongago by Hull,,6and therea- ofaspiration, andpersonality traitsas meas-
sons forthisphenomenonare well known. uredin thisexperiment appearnotto be re-
The reasonsforthe decreasedvariancein lated to achievementin college. Further-
the clinical predictionsare not so well more,clinicalinterviewers do not use these
known. One suggestionis that the inter- measuressystematically, so that actually
viewerswantedto "play safe." The closer theygivelittleorno weightto theminmak-
ingpredictions.
in the libraryof Ohio State University,"The Rela- 3. In formulating predictions, counselors
tive Accuracyof Clinicaland StatisticalPredictions relyforthemostparton rankin high-school
of Academic Success" (I941), pp. IOO-IOI.
graduatingclass and college aptitudetest
ISVariance ratios were computedbetweenclini- results-thesamevariablesin theregression
cal predictionsand statisticalpredictions. For men equation-althoughmanydifferent kindsof
the F value did not fallwithinthe 5 per centpoint;
for women the clinical predictionshad a slightly data are available.
highervariance-the F value fallingjust withinthe
17 L. M. Hanks, Jr., "PredictionfromCase Ma-
5 per cent point.
terial to PersonalityTest Data: A Methodological
I6 C. L. Hull, AptitudeTesting(New York: World Study of Types," ArchivesofPsychology,I936, No.
Book Co., I928), pp. 470-7I. 207, p. i8.

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 6oo THE AMERICAN JOURNALOF SOCIOLOGY

4. Case-studypredictions-at least on fromregression equationsat what level of

thegroundsofefficiency-should notbe sub- abilityto accept students. Some have ar-
stitutedforactuarialpredictions.As a com- guedthattheclinicalinterview mustsupple-
plementto the actuarialpredictions,the mentthe testproceduresso that so-called
clinicalpredictions add nothing. "intangible"factorswhichgo intoachieve-
5. Whencast in theformofsuccess-fail- mentmaybe appropriately weighted.From
ureinsteadoftheeight-interval scale,clini- theevidenceofthisstudy,thisclinicalstep
cal predictionsare shown to be no more is unnecessary.Predictions ofcollegegrades
valid thanthemoreeasilyobtainedstatisti- can be made withas muchaccuracyby the
cal predictions. simpledeviceof placinga straight-edge on
6. The reliabilityof clinicalpredictions an alignmentchart. In short,a competent
variesfrom.64 to .88 (correlation between statisticalclerkcan makepredictions as well
interviewer and case-reader). Predictions as a highlytrainedclinicalworker.
made by the same case-readerfromtwo Can we generalizeand say thatthisset
readingsofthesamecases correlated to .78. ofprocedures can be usedinindustrial selec-
7. Clinicalpredictionsoverestimatethe tionas well? Not onlyon thebasis of data
criterion by about a thirdofa letter-grade; presentedherebut also becauseofthelogic
thedifferences arestatistically
reliable.Sta- ofprediction'8 is the answerin theaffirma-
tisticalpredictions do not overestimate the tive. Some yearsago Vitelessubmittedar-
criterion by any significant amount. gumentsagainst the actuarialapproach,19
8. As expected,the varianceof thepre- which were logically controvertedby
dictionsmade by statisticalmethodsis sig- Freyd.20 Notwithstanding, the so-called
nificantly less thanthe varianceof the cri- "clinical"interpretations ofobjectivemeas-
terion(a resultof the phenomenonof sta- uremetit have continuedunchecked.If the
tisticalregression).The varianceofthepre- trainedpsychologist, sociologist,vocational
dictionsmade by the case-studymethodis counselor, orcase-worker can tellanybetter
likewisesignificantly less thanthevariance thana testwhether a manis fitfora particu-
of thecriterion. lar job, he shouldmake explicitthe factors
Anyjurysittingin judgmenton thecase upon which he disregardsthe objective
of theclinicalversustheactuarialmethods measurements and thensubmithis predic-
muston thebasisofefficiency and economy tions to experimentalvalidation. Unless
declareoverwhelmingly in favorof thesta- checked by statisticalstudies, the case-
tistical method for predictingacademic studymethodin thesocialscienceswillbe-
achievement.Even thoughthesmalldiffer- comeintellectually bankrupt.
ences whichuniformly favorthe actuarial May we go further and declarethatsta-
methodarenotstatistically reliable,thefac- tisticalproceduresshall be substitutedfor
toroftimeand efficiency willdecideinfavor clinical proceduresin the diagnosisand
of the regressionequationwithits known treatmentof all behaviordisorders?If we
marginoferror. interpret thestatisticalmethodmorebroad-
ly thanthe administration of mentaltests,
DISCUSSION AND IMPLICATIONS thenwe can againanswerin theaffirmative.
Whatare theimplications ofthefindings As a matteroffact,Wittmanhas presented
as theyrelateto the morespecificproblem evidencewhichfullysupportsthisproposi-
of predictionin academic achievement?
A8 Theodore R. Sarbin, "The Logic of Prediction
Sinceprediction servesas an aid to thecon-
trolofevents,theresultsofprediction stud- in Psychology"(in preparation).
ies can be utilizedin theselectionand dis- 19 op. cit.
tributionprocedureswithininstitutions of M. Freyd,"The StatisticalViewpointin Voca-
20
higherlearning.Admissions officerscan de- tionalSelection,"JournalofAppliedPsychology, IX
cideon thebasisofprediction tablesderived (I925), 349-56.

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
 ACTUARIALAND INDIVIDUAL METHODS OF PREDICTION 6oi

tion.21She comparedthe prognosesof pa- the hypothesis are shownto correspondto

tientsin a mentalhospitalmadeby attend- events,thenthe traitsidentified can be in-
ingpsychiatrists withprognosesmadefrom cludedwithothermeasuresin a regression
a well-constructed ratingscale. The results equation.This is consideredby the author
showedthe statisticalpredictionsto be su- to be one of thechieffunctions of theclini-
periorto the "intuitive"judgmentsof the cian: the formulating of hypothesesto be
psychiatrists. testedby himor by othersin thefieldor in
Realistic clinicians and case-workers thelaboratory.Anotherimportant function
knowthattheirpredictions aremadeon the is thatofservingas an agentof treatment.
basis of an informalstatisticalmethod. By thesame statisticalmethodwe can pre-
Thosewhoholdthatthecase-studymethod dict whichtreatmentmethodis most ap-
can do more than the statisticalmethod, propriatefora given symptomcomplex.22
even in the predictionof humanbehavior Wherethetreatment involvesa secondper-
problems,mustsubmitevidence.The onus son,suchas play therapy, relationship ther-
probandis seemsto fallupon thosewhoad- apy,grouptherapy, etc.,theinterviewer has
vocate the individualmode of prediction. a definiterole. In diagnosis,however,his
A wordis in orderhere relevantto the roleis a secondary one. Exceptas hegathers
clinicalmethodas used in thisstudy.Pre- data to includein theformalor informal re-
dictionsweremadeon thebasis of thetests gressionequation,hisdiagnosescanbe made
and otherwrittendata together withinfor- fromexperiencetables. If such an itemas
mationgatheredin the face-to-face inter- "Does yourfatherbeat yourmother?"were
view. The author would agree with any foundto have predictivevalue, then the
critic who says that more interviewingclinicianwould serve to ask the question
wouldprobablyproducebetterpredictions. and to recordthe answer. The weightit
The betterpredictions, however,wouldbe wouldhave in theregression equation,how-
madeon thebasisofthefrequency interpre- ever, would be determinedby considera-
tationof probability-thesame basis upon tions otherthan what the clinicianintui-
whichthe regressionequationis built. As tivelythoughtthe weightshouldbe. This
the numberof clinicalperiodsis increased, conclusionfollowsfromevidencepresented
the numberof observationsis likewisein- elsewhere23 whichdemonstrates thata diag-
creased.Similarobservations are orderedto nosiswithouta futurereferent has no utility
a class, and the interviewer thenmakes a so faras treatment or controlis concerned.
statisticalpredictionon the basis of many As soon as a diagnosisis made meaningful,
instances.It is conceivablethat the clini- it takestheform-implicitly or explicitly"-
cian mightdiscovercertaintraitswhichare ofa prediction.As shownby Reichenbach,
not apparentfromone interview.But he predictionsare expressedas probability
can onlytesttheadequacyofhisdeductions statements.24 Onlyafterstatisticalmanipu-
by submitting his generalizationsto test- lationshave takenplace, eitherinformally,
bypredicting futurebehaviorand by com- as in the case of generalizing fromclinical
paringhis predictionswitheventsas they experience, or formally, as in thecase ofre-
occur. gressionequations,can we derivesuch a
If advocatesofthecase studycan identify probability statement.
traits,dispositions,attitudes,or motiva- The implications of theseparagraphsfor
tions,thenlet thesebe put in theformofa
hypothesisand tested.If predictionsfrom Theodore R. Sarbin, "Clinical Psychology-
22

VI
Art or Science," Psychometrica, (I94I), 39I-400.

Mary PhyllisWittman,"A Scale forMeasuring

21
23 Sarbin, "The Logic of Predictionin Psychol-
Prognosisin SchizophrenicPatients,"Elgin Papers, ogy."
IV (I94I), 20-33; in the
"EvaluationofPrognosis
Bulletin,XXX-
FunctionalPsychoses,"Psychological 24 Hans Reichenbach,Experienceand Prediction

VIII (I94I), 535-36. (Chicago: Universityof Chicago Press, I938).

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions
602 THE AMERICAN JOURNAL OF SOCIOLOGY

psychologists and othersocialscientistsmay tion. If theproposition is acceptedthatthe

be summarized: diagnostic function belongsappropriately to
beingmoreeasilyde- thestatistician(sincemeaningful
predictions,
i. Statistical
diagnoses
termined andat leastas accurate, areto be arepredictions offuturebehavior),thenthe
preferredtopredictions madebytheclinical clinician-in-training or apprentice case-
method. workerwillspendmoreofhis trainingtime
2. If clinicalpredictionscan be shownto be in twoactivities:(a) learningfromempiri-
morevalid,thedata whichareusedby the cal studieswhichtreatment devicesare ap-
clinicianshouldbe made explicit,so that propriateto certainsymptomcomplexes
theymaybe quantified andincluded inpre- and (b) learninghow to administer thevar-
dictiontables. ious typesoftherapy.In addition,he must
3. One ofthechieftasksoftheclinician is to
formulate hypothesestobe testedbyexperi- learnhowas an interviewer he can be a diag-
mentalmethods. nostic tool. But the diagnosis itselfshould
4. Another taskis thatofserving as an agent be made fromexperiencetables,regression
of treatment. equations,or similarmathematical aids.25
5. Diagnosisis a secondary function in that UNIVERSITY OF MINNESOTA
predictivediagnosesaremorereadilymade
bytheactuarialmethod;theclinician, how- Paul Wallin discusses the generalproblemof
25

ever,maycollectdata calledforby hisfor- thispaper and providesmanyvaluable researchsug-

malorcrudeequations. gestionsin his chapteron "Case Study Methods,"
in Paul Horst et al., The PredictionofPersonalAd-
Forthetraining and clini- justment(New York:
ofcase-workers Social ScienceResearchCoun-
contribu- cil, 194I).
cians,thisstudyhas an important

This content downloaded from 130.207.93.124 on Thu, 15 Jan 2015 19:21:48 PM

All use subject to JSTOR Terms and Conditions