Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Guenole Englert Taylor 2003 Article

Guenole Englert Taylor 2003 Article

Ratings: (0)|Views: 38 |Likes:
Published by m.rose
Ethnic Group Differences in Cognitive Ability Test Scores within a New Zealand Applicant Sample
Ethnic Group Differences in Cognitive Ability Test Scores within a New Zealand Applicant Sample

More info:

Published by: m.rose on Feb 27, 2009
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





  New Zealand Journal Of Psychology Vol.
 No.1, June 2003
Ethnic Group Differences in CognitiveAbility Test Scores within a NewZealand Applicant Sample
Nigel Guenole
Selector Group 
Paul Englert
OPRA Consulting Group 
Paul J. Taylor
Chinese University of Hong Kong 
Given the widespread use of cognitive ability tests foremployment selection in New Zealand, and overseasevidence of substantial ethnic group differences in cognitiveability test scores, a study was conducted to examine theextent to which cognitive test score distributions differ as afunction of ethnicity within a New Zealand sample. Anexamination of 157 Maori and 82 European verbal andnumeric ability test scores from within a New Zealandgovernment organization revealed sizeable and statisticallysignificant mean differences between the two ethnic groupson two of three cognitive tests evaluated. Specifically, Maoriscored, on average, 0.55 standard deviations lower thanEuropean applicants on a measure of verbal reasoning, and1.79 standard deviations lower on a measure of numericalbusiness analysis. No mean difference was observedbetween ethnic groups for a test of General numericreasoning. In light of these substantial differences on two ofthe three tests, we discuss strategies that organizationsusing cognitive tests can employ to minimize adverseimpact on Maori applicants, as well as further research thatis needed.Many organizations in New Zealand strive to achievemultiple objectives with their personnel selectionprocedures, including maximizing both predictive validityand selection utility (i.e., cost effectiveness), as well asachieving and maintaining an ethnically diverse workforce.These goals, however, can conflict, such that selectionmethods that achieve one goal (e.g., predictive validity)work against another goal (e.g., diversity). Overseasresearch suggests that a cognitive ability test s such anexample of a selection method that supports one goal at theexpense of another (Huffcutt & Roth, 1998; Roth, Bevier,Bobko, Switzer & Tyler, 2001). Cognitive ability tests havebeen found to be one of the most valid forms of predictingfuture job performance for a wide range of jobs (Schmidt &Hunter, 1998). For this reason, some authors havesuggested that abandoning their use in employmentdecisions would result in a substantial sacrifice in workforceproductivity (Gottfredson, 1994; Hunter & Hunter, 1984).Indeed, Schmidt and Hunter (1998) have argued that, sincecognitive ability tests have such high predictive validities,other selection methods should simply be considered asadding incremental validity to the selection decision oncethe cognitive ability of the candidate has been assessed,implying that cognitive ability testing should be a majorcomponent of a thorough selection practice for many jobs.While more recent reviews of the validity of alternativeselection methods suggest that, when estimates of rangerestriction and criterion reliability are standardized acrossstudies, structured employment interviews are at least asvalid (Robertson & Smith, 2001), if not slightly more valid(Hermelin & Robertson, 2001) than cognitive ability tests,cognitive ability tests clearly remain one of the more validpredictors of job performance.Cognitive ability tests play a prominent role in the personnelselection systems of many organizations both overseas andin New Zealand. In a recent survey of selection practiceswithin 100 randomly selected New Zealand organisationsand 30 recruitment firms, Taylor, Keelry and McDonnell(2002) found that almost one-half of the organisationssampled use cognitive ability tests for selecting managerialpersonnel - over twice the proportion used a decade ago(Taylor, Mills & O'Driscoll, 1993) - and that almost two-thirdsof recruitment firms use cognitive tests in selection. In fact,
  New Zealand Journal Of Psychology Vol.
 No.1, June 2003
 2Maori European
Test Male Female Gender Unknown Male Female Gender Unknown
49 6 27 3Verbal reasoning test 109 19.28 5.35 55 22.2 5.09 .55* .17 .22 - .89Numerical business analysis test 19 8.21 3.34 10 14.90 4.43 1.79* .46 .85 - 2.72General numerical reasoning test 29 14.72 5.76 17 14.76 5.55 .01 .31 - .61 - .62the use of cognitive tests in personnel selection is nowgreater in New Zealand than in many other countries,according to a recent cross-national survey of staff selectionpractices in 18 countries. This survey found that theprevalence of cognitive ability test use in New Zealand wasgreater than all but three other countries (Ryan, McFarland,Baron & Page, 1999).The value of cognitive ability testing for employee selectiondoes not, however, come without costs and somecontroversy. In the United States, for example, the use ofcognitive ability testing has been found to adversely impactAfrican American and Hispanic applicants as a result ofsubstantial differences in mean test scores (Sackett,Schmitt, Ellingson and Kabin, 2001). Large-scale meta-analyses have confirmed that African Americans scoreapproximately one standard deviation lower than Whites onmeasures of quantitative ability, verbal ability, andcomprehension and that similar, and slightly smallerdifferences (.7- .8 standard deviations) have been foundbetween Hispanic and White applicants (see, for example,Roth, Bevier, Boko, Switzer, & Tyler, 2001). Consequently,where staff selection is based largely on cognitive ability testscores, members of affected minority groups, such asAfrican Americans and Hispanics in the USA, receive feweremployment opportunities than Whites and some otherminority groups (Scientific Affairs Committee, 1994). Thissituation has led to a dilemma in the USA, in which relyingon cognitive ability tests has been seen by manyorganisations and researchers as sensible from theperspective of maximizing predictive validity, but doing sothreatens the achievement of social objectives, such asovercoming past social inequities, pluralism, and creating anethnically diverse workforce (Sackett & Wilk, 1994; Sackettet al., 2001).While much research has been conducted in the UnitedStates on ethnic differences on cognitive tests used inemployment selection, we know of no prior publishedresearch on the topic in New Zealand. Such research isimportant, given both the prevalence of cognitive testing foremployment in New Zealand, and government policyprograms aimed to decrease social and economic disparitybetween Maori and non-Maori. If differences in thedistributions of occupational cognitive ability test scoresbetween Maori and Europeans are near-zero (i.e., so smallas to have no practical significance), employers can usesuch tests with the confidence that doing so will result in noadverse impact on achieving an ethnically diverseworkforce. If, on the other hand, substantial differences incognitive test scores are found, as they have beenoverseas, then organizations wishing to employ anethnically diverse workforce must carefully consider whetherand how to use such tests. The purpose of the presentstudy was to investigate whether ethnic group differencesexist among a sample of New Zealand job applicants, usinghistorical recruitment data from an organisation that hadadministered cognitive tests as part of their staff selectionprocedure.
Archival test score data were available from a large NewZealand government organisation on applicants who hadcompleted one or more of three cognitive ability tests whileapplying for professional level positions, such as analysts,senior analysts, or finance positions.Participants in this research were applicants for analyst,senior analyst, and finance positions. The data used werehistorical, and were collected as part of the recruitmentprocess. The testing for the candidates in this analysisoccurred over the period 1997-2001, including test scoresfor 239 candidates. These included 76 Maori male testscores, 74 Maori female test scores, 40 European male testscores, 39 European female test scores, and 10 scores (7Maori and 3 European) of unknown gender. The breakdownfor each test is presented in Table 1.For the purposes of this research, job candidates wereclassified as Maori if they had identified themselves whenapplying as either Maori or Maori and any other origin,including European. Only candidates who indicated solelyNew Zealand European heritage were classified as
  New Zealand Journal Of Psychology Vol.
 No.1, June 2003
 3European for the purposes of this research. Excluded fromthis analysis were all test scores for candidates whoindicated an ethnic origin other than NZ European, or Maoriand any other ethnic origin (that is, any ethnic origin otherthan that identified for inclusion in this analysis), as samplesizes were inadequate for meaningful analysis.Given that the focus of this paper is on mean differences intest scores, we have not examined employment offer data.Consequently, inferences about whether adverse impactresulted in this particular organisation are impossible toidentify. Any adverse impact is likely to have beenminimized by the existing recruitment approach thatprovided equal weighting to structured employmentinterviews, cognitive testing, and work sample tests(provided ethnic differences were not as prominent on theseother recruitment methods).As the data were from a government organisation, allpositions were advertised in external newspapers, jobswebsites, or both. However, information with regard towhether candidates were internal or external wasunavailable. Ideally. In future research, this variable shouldbe controlled. This would minimise any chance that resultsobserved are due to different ethnic composition of theinternal and external samples, if, for example, internalcandidates have an advantage over external candidates.Findings from overseas research suggests that ethnicdifferences in cognitive ability test scores are associatedwith similar (though less pronounced) ethnic differences inother selection methods that have a large cognitivecomponent, such as in-basket exercises within assessmentcentres (Goldstein, Yusko & Nicolopoulos, 2001).Why a substantial mean difference was found for onenumerical test (the numerical business analysis test) but notthe other (the general numeric reasoning test) is not entirelyclear. The difference in findings for these two tests may bedue to differences in the amount of business knowledge andexperience required by the tests. The principle differencebetween the two tests is that the test on which no differencewas observed does hot assume any business knowledge,while the test on which differences were observed assumesthe person has had exposure to business terminology (e.g.net operating profit and gross profit). If Maori in the samplehad less exposure to business terminology included in thetest on which differences were observed this lack ofexperience may have accounted for the effect that wasobserved.While the present study identified ethnic differences in testscores, these results do not necessarily constitute test bias.Test bias is observed when systematic differences arefound between ethnic groups not only in mean test scores,but also in how tests predict job performance ratings. Forexample, the verbal analysis test would only be consideredbiased if it was found to differentially predict jobperformance for Maori and European applicants (e.g. onaverage, Maori scored lower on the test but performed the job just as effectively as Europeans).In order for evidence of test bias to be established,additional information is needed on applicants' actual jobperformance once employed, so that differences inregression slopes and intercepts can be assessed when jobperformance ratings are regressed on test scores withinethnic groups. Job performance ratings were not availablefor the present study, and so we were unable to exploreevidence of test bias in the present study. While overseasresearch both in the area of scholastic achievement and jobperformance indicates that tests of cognitive ability predictequally well across ethnic groups -despite the fact thatsizeable mean group differences exist (Roth et al., 200 I),we know of no data published to date on test bias in NewZealand. This is an important area in need of futureresearch.
Practical Significance of Mean Test Score
Regardless of whether mean test score differences found inthe present study reflect test bias, such differences wouldstill lead to adverse impact for Maori as long as personnelselection decisions are based, even in part, on scores onsuch tests. If the present findings generalize to otherorganisations that place considerable weight on cognitiveability test scores in their staff selection processes, Maoriapplicants, as a group, are less likely than Europeans to beselected.The practical implications of these findings can besubstantial, even if cognitive ability tests are simply used asa screening device, where those applicants who fail toachieve a particular cut-off score are removed from theselection process. For example, consider the case in whichthe standardized mean difference between Maori andEuropean cognitive test scores is d = .5, similar to the ethnicgroup difference we found for the verbal reasoning test inthe present study (d = .55), which of the three cognitivetests assessed, resulted in a mean score difference in

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->