Reporting Results ational, ang ational placement in the diagnos: difficulties: in the des . 1 desi tional eacaeacs 'N Occupational Placement for = and effective deployment of Fesources; in clinical dia see development cal assessment; and in research, POSS and neuropsycholog. forms of guid. S Of areas of spe. Of individualiseq — general public — may interpret and use the information in different ways Some of these uses may be neither intended nor legitimate — thus parents and pupil, to whom the resuits of assessments made by teachers and school Psychologists primarily for educational placement are relayed, may interpret and use the scores very differently from school personnel. Candidates whe, are rejected after applying for a job may interpret and use any information they are given in ways which were not expected by the organisations concemed. Patients may place very different constructions on scores from those intended by doctors or clinical psychologists. By omitting to collect and report relevant Contextual information when evaluating educational programmes, and falling to Set what they do report in an appropriate interpretational context, researchers ‘may inadvertently give misleading impressions as the results are set in the Context of alternative assumptions made by policy makers, and thus used to generate very different policies to those indicated by the results. Given this diversity of audiences, multiple audiences for the same scores, and the myriad of direct and indirect uses to which test scores are put. its not i Iso follows that the attempt to do so Possible to guard against all misuse. Yet it al ——— by restricting testing to psychologists cannot work Muttiple a as results, and many of them, like parents, teachers and managers 18 do their own jobs effectively. What is needed is general pul eee A primary responsibility of those reporting test a direct and indirect ical profession in general - must therefore be to educat ofthe General Section Users of test scores. The 1991 and subsequent editions were prepared to assist in this process. APM77outa oo this point, refer back to that Section, 4 PS 7 ded, here. anti to identity the psychological nature ofthe aed and distinguish them from related Notions. The tie ave been asse' bility. Togethe; ceductive and reproductive al iy. Tog "rpm vt of the agreement between the two component ic and prescriptive implications. Scores Nevertheless, despite their undoubted value, these constructs do Pot ‘the ways of thought and selection practices associateg sO an oncept of gis often equated with, and felt to legitimise the them. Thus, i ae, {As shown in the General Section, this wider notig, one associated witht) snot supported by scientific evidence and use not infrequently leads to the adoption of selection, Placement, and ment practices in both schools and workplaces which are damaging to the ind. viduals concerned, the organisations in which they work (or will work), andthe societies in which they live. The second requirement is to ensure that users understand the limitations of current measures of eductive ability. One of these is that they do not give adequate credit to those who have used their abilities to evolve new sets of non-verbal strategies. Such strategies, which may be those required to play chess, develop mathematical theory, or manage complex organisations, alow People to solve otherwise resistant problems. This limitation is of particular 'mportance among older people and in occupational settings. Jaques'5, Kohn and Schooler, and Lempert et al” have shown that, contrary to the normal interpretation of data showing that older people get lower scores than younger reooe ae reat of the increase in scores with date of birth), older ieee opportunities to deal with complex and demanding infor ‘OP cognitive abilities that are less likely to be possessed bY younger people. Further, these abilities may be precisely those required to understand and influence the Operati. Preasely se a should be clear about this isnot thera on of complex organisations. We i information about a Older people have more stored, reprodu he developed the abilities ate tee nea nats being argued i that n a The third require Quickly understand new developme’ ve . the APM and MHy he S10 insist on setting the information obtained w!” in 7 € Conte; K "3 ion, Pre-occUpations, xt of other information concerning the moti and t Broups Concemed, in, = Other aspects of competence of the cial Of “Patlonal setings, some of this information C2 0® The first requ and MHV measure index of The exter has important diagnosti ‘APM78obtained through carefully conducted Bay a Episode Interviewing™™, oF puided obee,, Ul Even It Inter servation — vi r framework presented in Competence in ne all inte reed are tt it may be possible to obtain it through puig Society. educa 8 the conducted by teachers who have ones a erieing and obse which they make a serious effort to identify ~ lSpmental environme, and who create individualised development ° practise and get recognition for their talents, rie develop a formal Schoo! Improvement Kit for ue gn 5 computerised tools to elicit people's values and re }OIs and to develop display when engaged in valued activities. Those — the competences they work should contact the author) Sted in participating inthis currenty in hand tg 'tis important to note that by no means all individuals will be positive, at least until set in the context of adeq ti human-resource management practices: recall the demonstration ati alls! that many “inteligent and socially skiled people use these abiities to ree detriment of their colleagues, organisations, and society Such wider information about Despite the availabilty of preliminary means of assessing the wider aspects of competence, the nature of these qualities means that it wll actualy be neces- sary to develop a new psychometric paradigm to assess them adequately. This new paradigm will have to acknowledge their value-laden nature and come to terms with the fact that people will only develop and display high-level compe- tencies (such as initiative, the ability to work with others, the ability to commu- nicate, and the ability to understand and influence their organisations) while engaged in activities they care about!®. tt follows that those responsible for reporting test scores need to encourage users to press for the necessary work ~ and, indeed, to encourage them to consider funding some of it themselves. n this context it should be noted that teachers, in particular, have failed to discharge their expert responsibilities by not insisting through ther — organisations, that the relevant research be carried out Without it they Perform their jobs effectively. The fourth requirement is to situate abies of individuals in the context of information On EA ay level competencies and an understanding ofthe way Wa uch of multiple competencies (including eductive ee appear 06 talents as people possess released. What people's oo on the extent to which the environment in ae to exercise, and thereby worked engaged with their values and permitted all information about the talents, and ‘the mutability of| high APM79on -ompetencies they will display in a, = the ¢ Xt ae to which the demands of the aSsessmg s the 7 ment setting on abet Finally, the talents they are said to Pos, age with _ vero are making the assessments note thei depend on ther th ns rather than weaknesses, and themselves fee] that regard them aS oe possess such motives and predispositions eople os eee sront those reporting RPM and MHV scores shoug én Ona nat arent discrepancies between the results and those hte ther tests. They should also specify the opportunities the person tai, with of Mt had to develop the abilities being assessed, any special Coaching he the test has ° ‘or she might have had, and any circumstances which might have led to Trisleag. scores (ep ifthe respondent's cultural background suggests that he or she " . F med with accuracy than others might be). ‘and discuss any 4PI wil be less concer Since it is much easier to obtain contextual information if tests have been self-administered in the presence of the person responsible for making the assessment, we recommend that, when the results will have serious implications for the lives of those being tested, testing should be individualised. ‘The fifth requirement is to ensure that users understand that what any one Person will do — and thus both what their talents appear to be and the effects of their actions — depends on what others do. There can therefore be no ques- ton of selecting “gited people" and expecting them to generate all the new Understanding needed to change their ‘Organisations or societies. Users need to Understand that itis essential for teachers and managers to focus on creating climates in which as many as possible of the talents of all pupils and subordi- ates are developed, utilised, and ir i ‘ Pen rewarded instead of focusing only on "people The sixth teachers and aes 'Sto draw attention to the crucial importance of their students nda the development of all ofthe talents of al of mates through placement and development instead Of calling for ‘Overall " champions" inteligence ang omen 4s shown in our discussion of the validity ofthe Eective parensi tegen 8 Cultural, distributed, characteristics. ache fective peers in that they ae and managersi«s are distinguished from their [Pend more time thinking about the motives and Maipient talents of their thi een ne Puplls, and subordinates. They create “deve compart St about ad ‘heir children, pupil, trainees, and staff a" reate ings oeeSS Practise and develop high-level chidvalsed developmental programmes forexstudent or subordinate, They m, ~ whether their pupils, children, ees creo he may en developed are recognised by al a ing that include high levels of ecutive ability eve, . also include the qualities required if a meng ad communicate effectively, those en, bing peo to tion to their organisations, and those resuied ys, enterprise, Effective teachers Create climates and managers also ui ‘The seventh requirement is to lead users to appreciate more fully both the limted discriminative power of psychological tests and the absence of justifica. tion for making fine distinctions. The frst point may be frcefly made by discussing the level of accuracy which can be expected from the APM - a test with relatively high discriminative power and reliability, The degree of indetermi- nance in test scores can be estimated bby calculating the Standard Error of Measurement (SEM). The SEM is based on a test's reliability and standard deva- tion, both of which vary with the population tested. I, for the sake of an exam- Ple, we use the standard deviation of APM Set Il obtained in the UK standards. ation (8.8) and take the commonly reported reliability of 9, it emerges that the SEM of APM Set Il is about 25. This means that about one in three of a random sample of people who were re-tested would get a score which differed from their first score by more than 2.5. Thus, a raw score of 30 should rate thought of as 30 plus or minus 25, ie 27.5 to 325. Reference to i a shows that, among 20 year olds, the fist of these scores falls at the a Percentle and the second atthe 95th percent. Athough ts angst tainty is clearly greater than most people would like, a = have much educational tests (which are widely used in occupational isabity, and IOwer Poorer discrimination, lower retest reliability, poorer gener# validity than the RPM's, APMBIue RESULTS || more important: even if such iscriinagy astructs being measured WOUId stil be yp Uy abife performance of people with marinas ® acond point i stil de, the €O! ise the ree peed be indtinguishable. There would also be crome, ple with the same scores, ve The st different scores WO ation between the Pe! rage undue fa formance of peor ith in the precision of mental Measurement tho eee res are urged to do so in terms of broad reporting RPM and MHV scor percentile bands only. Research Supplement No.3 presents in ore detail the arguments reporting results as IQs. These are summarised in ie General Section, Here itis sufficient to say that the notion of an IQ tends to reinforce misplaced faith in the unity and explanatory power of Intelligence as a construct, and in the stab, ity and immutability of test scores. It creates a false impression of, accuracy, and ‘encourages users to make unjustifiably fine discriminations. It also reinforces a range of undesirable educational practices. In addition to helping users to understand the issues relating to test validity and utility which have just been discussed, those responsible for the use of tests (as well as those advising on their use) should also promote a more critical use of the word ‘objectivity The quality of an evaluation or assessment is not determined by the accu- racy of the information it provides in relation to one or two variables, but by its comprehensiveness. In the assessment of an individual, this wil involve Betting at least a rough fix on all the important characteristics of the individual being assessed, Similar, a comprehensive evaluation of an educational sft development. or organistional-development programme wil cover os ieoate ssesrnnt of an individual cannot be described as “object! a © Person concemed is poor at the things that have bee becore goon “recording what he or she is good at or has the potenti tional and shll-dewtopren is no justification for evaluations of ens MPortant benefits and disbe eecaies which fall to record tele mo and diffcut to measure _ hi its ~ perhaps because these are rent ile saying that they have "no effect’ 0” Outcomes that are aiabien, “27° measure and for which reliable and valid t=pies sessment Over the past decades, users ofp themselves involved in highly pol ee tests bias and faimess. Anyone reportin wat Cor involved in this. Unfortunately, Ss * confused. Perhaps the most importne Temainpoltcise become embroiled in this controversy is that yo cee PO Oo ong As shown in Research Supplements 3 aaa was cated that many of the issues that are oe esearch % a es¢0rch to are nothing of the sort. Differences in pe Presented asi indi. scores 6 ce religious, gender, and ethnic and motivation and are associa a a ae huge differences in occup. differences arise and to spell out eres we ice ato and social policy. The explanations of many of re planes ous ones Thus gender ferences in selected popuaton of overt and covert selection were not anticipated by rent pho paerees findings suggest that th ear te Eee populon The igs sugge: 'e way in which problems associated with these difer. ences are to a resolved has more to do with developing appropriate educa- tional, occupational, and social policies than with modifying tests to eliminate “bias’. Nevertheless, to implement more appropriate policies in these areas, will be necessary for psychologists to broaden their framework for thinking about competence and its development and to develop the tools required to assess its components. This will rake it possible to introduce ways of identi ing, developing, utilising, and rewarding a wider range of talents in school, workplaces, and society more generally. Is isues of this sort that those involved in reporting test scores must be prepared to lead the users ofthe scores (including governments) to debate. In preparing themsees'o shader these responsibilities, readers should first reflect on just how Gaal tured out to be to construct a convincing picture of the vaicty ofthe ee ifcuties exist in a professional area which has been ee oa ne of than a century, how much more dificult "8° “ Tey HY psychological assessment procedures in 2 poised Pl : ; ements, to the authors also like to refer to the previously acre an Managing Exe books Competence in Madem Society, The sin of Nations FOF for fective Schooing' - and perhaps te veces pose ly refer to ens Best full more specific guidance they could use re specific gui rots EA Orr es that itriht ‘Appendix listing some ational perfor. ©=pin how the ‘onal, occupational, are not the obvi. ‘typically a resutt arise in ways that Testing and Saville and Hol Test Practice. In addition, a short me aereporting v4 TING RESUL' : ations in which ACCUSALIONS Of bias ja. = Sit to raise in si il be fo oy be appropriate de in occupational settings wil und at the end eas are likely to be, ma J Section to the Manu of encouraging users to unde He ag ts pratense ewalen log, tions of psychological ay tests outside the so-called "cognitive are, = few psychological or tte sat, or wider practical uy ofthe i re a ee introducing spprop bud Second fm these come not only from savings on recrutmene ee erode fomalied induction programmes, and los of personnel vo hag incurred expensive training and development but also from Avoidance of the havoc which can be created by an inappropriate appointee aod the loss of opportunities to do more important things both by the appointee himself or herself and by others whose time is drained away. The temptation to €conomse on the cost of selection i counter productive, Fs typically spend farmer assessing a piece of equipment which will perform limited functions and become obsolete in a few years than they do on staff: selection, placement, and develop. ment Third, the introduction of appropriate guidance, placement and develop. ment procedures prevents personal frustration and misery by reducing the ‘mount of time people spend floundering around in the job market While the need for those involved in assessment to Pay attention to the points ‘oned above may seem obvious, these are activities that have, in the past. been honoured more in the breach than in reality Unfortunately there are Raven's fora fujar a nit to allotents and eee we them SSiOn Of these Processes and the steps needed toReference Materia} Table APM12 Advanced Progressive Matrices Set | (Untied) smoothed 1992 Norms forthe United Kinguon, ——_______ ann WoW 8 ash ay 7 BO 4G) 14) ISG) ag, 2 —a ee : 7 tee 5 50 oe wD HoH t © 0% 0 0 ’ a a 5 Sos 8 6 rr pg gt Hae gy ra i er rr us s 224 ts ss eg PS Sta fe ; ee we ekors inns natant sansa soem ‘18 derived from the 1992 standardisation of the APM and SPM, “me aa ‘om age eee Table APM13 ‘Advanced Progressive Matrices Set Il (Untimed) ‘Smoothed 1992 Norms for the United “ms 1} 2D 13) 4a) a) 150) 18 Perantle 14a) vat 150) 1502 BB wo” nine pH vo we wom noe wow oe wo 0 0 0 7 7 7 7°72 455 5 9 9 9 87 wsUBres ue we oH ; uo So 196 189 191718 aa om a9 converted sng Table APM Data up toage of 16 derived from the 1979 standardisation ofthe SPM 18 ecved fom the 1982 standardisation ofthe SPM and AFM completed tee. See tert for sample APM8S,RE RRRARHRRESTLSBARARTIR RRA RERKARHRRSSSKKREIRARSWVRANS RERFRARSSSSLKLESRARLIR ARATE RERFARSSSKKRSIRARRSIRRARS TS RERRRESSRLKBBSARARVIIRARRS STS BRARESSRERBSRRRSIRRKE ESSA BRRRESRLKSBARRALSRRAAE ESS BRRBSRKESBRAARESRRARASESS OK BRRBSRBKLSSRARGRARKASES SPH RRRRRSSSELSSRALRARRATS 6 n 10 8 6 5 BRERRESSKKLYSRRGRARAAS 5 2 10 a ‘ 5 BRRBARKRFRERSSSRERSRRIRRNR BRRRERBSARSRABRLLTARIR RRR BRRRTREASRSSBRKKSRRTAKR RRRERRRARRHSGR ERE RRRRS Blanaannanenenannnrerressaasece ness APM GGTable APM15 advanced Progressive Matrices Set Il (Untimed) smoothed Norms for US Adolescents Age in Years (Months) 12 124 13 134, 4 rN 110) 120) 120) 3@) 136) fas ae pe ao Aas) to to to to to to to to to Percentile 122) 128) 13(2) 13(8)14(2)—*14(8) 15(2) 15(8) 162) (16) 95 7 «18 «190 «21 22 HSK HT 90 1 160«17:~—=«‘Hesia DT HK 75 120 130 «1415S isi B20 so 8 9 s on Ww 2 8 MS 2 5 6 6 7 7 8 8 9 9 «10 10 2 3 3 4 5 6 6 7 7 5 1 2 2 3 3 a 4 4 4 Data derived from the 1 1986 US standardisations using Table APM34.35 8 99 3 ° 2 9 eo 31 99 99 oe 2 " . 3 ae 98 ” » . 38 98 ”7 96 a °° 98 7 6 5 a °° 97 96 5 4 e * Fs ee 95 4 92 B * % 5 4 92 90 sa * a 92 90 88 a oo 3 90 88 85 2 4 3 a s no e 20 93 2 a 5 ” = 2 x0 5 20 15 70 Ee 0 9 s 5 70 “4 ” 88 86 2 n “ 2 16 85 82 7 OG 39 * 6 16 7s cd a i 3 “ 7 66 62 8 EJ “ B 61 57 56 50 46 a 2 56 51 50 45 BY ae " 50 45 4a 40 7 a 10 ory 39 36 33 32 25 9 oy 36 3 2 2 16 : 32 26 20 16 14 12 , " v 1B 2 " 2 5 10 ; ‘ ; i ‘ 5 8 7 6 6 7 6 6 5 is Data deri derived from the 1986 US standardisations using Table APM34.Table APM17 ‘Advanced Progressive Matrices Set | intimed) smoothed 1993 Norms for the USA “ Based on the 1993 standardisation of the SPM and APM in Des Moines, lowa. Table APM18 Advanced Progressive Matrices Set ll (Untimed) ‘Smoothed 1993 Norms for the USA Age in Yours ~~ os » 5s © & 2 S S Go o op mw Boe 8 f fre 6 6 © 6 eb FS 2 woe op on nw 7 @ 2 2 8 8 a » 9 2 ne 2 2 BY Ron 2 * en ne ee nr 5 pele eee so «2 75 Pa, 7 27 ” * s 50 » » 0» 8 8 BY oa Ry Fy 5s 5 oot 4 a 10 10 1010 " 7 7_§ —— 2 7 Based on the 1993 standardisation of the SPM and Tests completed at leisure.1983 Worm forthe USA in the Context of 1992 UX Data ‘Advanced Progresive Matices Set (Untied) Table ara 7 7 me uo» « «ou = eon SPM and APM in Des Moines low Sata base onthe 1983 standar dation of the SPM and APM in Dumfies, Scotend. (UC data based onthe 1952 nandardsation oftheTable APM20 Advanced Progressive Matrices Set I (Ui 1993 Norms for the USA in the Context of 1992 UK Data Age in Years “28 ee we ge yo so 1 2 2 s s x es « 3 & & H to to to &e ew 2» 2 8 8 we a 2 ® ” a a 2 ” e ¢ Percentile UK US UK US UK US UK US US WC Us UK US OS 5 B28 2BR BR NR RAAT HHH DDE D 0 31 30 31 30 31 30 31 30 30 30 30 30 2 20 wm we Dn 8 VUAANAANBKBBwwS BH BBBNHN DH w 50 ReRMRAH HAI HDI PBT HS wD 25 715 17 15 17 15 16 15 15 15 13 7 8 10 1210 12 10 12 1011 10 10 100 10 10 9 9 kT 5 9797978777675 645 443321 n 58 28 71 53 04 72 69 77 4121 67 OM BT Bw US data based on the 1993 standardisation of the SPM and APM in Des Moines, lowa. UK data based on the 1992 standardisation of the SPM and APM in Dumfries, Scotland. Tests completed at leisure in both cases.SSVUSHRKRS ABs [wuwanavaed BeRreses n fSaseus2es 36 5 20 7 4 2 sRxrtageryeses ” “ 2 10 95 azesss n 65 sassec esseR 9 5 Baserescges 31 25 "7 “4 R 10 BsesseR 5 SexeRee eS ssags R 10 7 5 93 90 86 81 5 n 65 Bas vss 36 3” 23 20 v7 4 2 7 Seger 75 Saseruse gr 36 30 25 20 ”7 4 R 10 RsRsVRSS n 65 Saxrsasgg SERSERSG REBAXBISSSSRSS BEERStable APM22 progressive Matrices Set It Table APM23 linute Time Lirnit) Advanced Progressive Matrices Set hed Norms for US Navy (20-Minute Time Limit) $00 ‘Smoothed Norms for Polan Age in Years: norms for Polen ns ns ‘Ag in Years ors) i 5 ey BS soe ee: eee eee ee eee ee —_ woe ee eye 7 » * = Percentle 96) 145) 956) 145) 119) * * * Bi 5 nips s & B® 2 = 3 * mn en wu mw x 2 3 . " % 7 8 9 wo on mn RB 3 5 " * ou os 6 7 7 ow 0 ® 0 3 ee s ’ 8 8 0 67 7 8 8s 9 ° 2857 on 195 eB ‘ i : s s 7 Extracted from data supplied by Alderton (for details see . —————————— Knapp & Court, 1992). {Based on the use of the APM with a random sample of 3,953, US male naval recruits at the Recruit Training Command in ‘San Diego between 1986 and 1990. The bottom 30% of the ‘general US population is under-represented. ‘Set (Untimed) Data from 1991 standardisation. {For details see Jaworowska & Szustrowa, 1991) ‘Advanced Progressive Matrices. 1997 Norms for Germany in the Context of 1992 UK Data (ont) ‘Agen Yours oo ss 8 SS & 8 we ese rr a re at o ee » © © © © © w e ee = 3 = 8% bop os 2 7 » 2 o © 8 Percent Ge Ge % > OM OR janie Geer a nonp * * 6 > 3» es MB BD M BR BH 2 5 uN . 8 oN oN nN NR wow DBM ba Oe 1s un vn 77 wo 6 BH M co * * fa ° sn nvnn 6» 9 B w ‘ a 3 uv vw ew 6 2 Ss 4 BS ns 10 ow 2 wv bo BW Bw 2 0 an 5 6 we 8 sw 7 sm 5s 8 seth 2 w_e_s_S ee te tA a 6 ow 5 tee qauad on data Colacied 0 1997 by Ralph Horn.Table APM26 Advanced Progressive Mavices Set (Untined) 1984 and 1992 Norms for the People's Republi In the Context of 1979 and 1992 UK Data i i ea vo prominWeoC mcdweme D s wow vom * oo “ s wer we Pa rn re = sas * rs : roe a _ 15) 6 16), . a “ “ 5 ” " M a 5 * oa oa 2 we om “ se sa) 10 “wo So me ma oe RE mG eG WERE AG RC MGC De Mw sR R RRM BB Bw ee ee 2 on we Booey eR BUR Bas eae ane ww BE Be s ooh 2M Pee ee we ne RR OM 2 "ee eee oe ee “nm wm ne 5 cia FR te ee ee ne RB OB *” sep Pie pre ree Te Te pe : ste pm a es “ew eee me we “= ™ aon me NENG sc Sra TEEaRETSE 7 2% =" a3 ae a ce1 Matrices Set l (Untimed Except for Us w; re 15-29 Year Olds NY and Poland 2 7 ; we pate jand, and the People’s Republic of China 0 ‘Age in Years (Months) _— ; ee 6 ws) 150) % ———$—_ «o © to 16 I [— ° 150 my © ' rat 1) 1645) ea . UR CRU UC mG ™ we ee eR oF wy . 5 pe Re Bney yg S OH aye ° ° 2 16 18 ww 9 ow n » 233 uy y 5 fo wm eM we we oy SMB RT sy ow en we we OER 8 5 7 6 7 B 7 6 9 B 10 A ; " wn» sree a s 5 6 9 4 6 98 5 ‘ aie es "I 5 2 ‘Age in Years (Months) vo my 8 ry 1 wm 5 5 “W]e x no a soe e im 7) mm] ww) um 3 veale QL PRR ve s 2 6 2» 8 mw © 2 DF = Bee Ce ee ee ee ee an Soy on ms mw wp n Bw B B Brae 2 uo om so oO ne noms nt 3 a cP] a 2 “4 3 B Ss “ : awe "7 8 ws s 9 wm Hw Soo S38 8s «w ow of « s 7 au vpio the age of 16 derived fom the UK 1579 standardisation ofthe SPM n= 3,50 converted sing Tab ADK A aon age 18 derived from the UK 1992 standarduation ofthe APM and SPM (= 5 SpE bed on 986 Aran sandarseaton fh SP we sme ot - Xe tand tn tanh nese oe peolas Republic of Cina = 18 ome ret" vs 1952 andardsation ofthe APM in Tian Jin, China (n= 3:26) — Lac gm data coed in 1878 and puiahed in Katze ard Hor OO. aya * nat ta exacted from data supplied by Alderton (see Knapp 8 Court. 1982) TH TA ny neu seit among random sample of 3.953 US male naval ects nda nae cepre stag B® between 1986 and 1990. The bottom 30% of the gener "deans "2 1991 standardisation using 30-minute time limit (= 1402) (1 ‘ow “stv rom the adut standardisation ofthe SPM and APM in Des Moinesset co-minute Time Lei) Table APMIS ates ne an tes and Australian ¥ aan for British, Australian, 2nd NEY? Zealand Undergraduat car 7 Students =e m a ‘Australi a te 505) (late 805) we se ns . ; 10 : 1s : \ 7 1" a ; 2 “ 0 ; ; ® n s 5 ; : " : : 2 ‘ 5 15 5 : ; E ; 19 " 7 » I x : : : 5 5 26 56 : C 2 5 32 6 : ® x 30 7 = 7% : = 5 4s 35 s = Py ~ “ “ 4s 5 bed ~ F 60 50 oO = : 2 * ° ‘ 2 7 70 : 5 z 6 80 = » 3 » 0 8s 3 5 2 5 5 90 F ” 5 » 7 E s 95 3 a bs 7 6 = 2 7 F ° ° > , B p 7 5 0 . ° % 383 745 381 639 2795 * British data collected by Blinkhorn ants toa — : relay dare cue by Gor pee este Uni by Sai oy he le & Holdsworth Ltd, and from appli Th tao ao ‘om the three samples were pooled, and percentiles ‘computed from the mean and $b of ecm The percentiles were computed from the and _ Physical education students at the Univesity Service tra ing .Table APM29 Advanced Progressive Matrices Seti Norms for University of California Pale ey) Stude Raw score Percentile From Paul (1985). Table APM30 Advanced Progressive Matrices Set Il Means and SDs for College Students, by Sex Time Limit. Mean SD on Australian university students (Yates & Forbes, 1967) 40 minutes Male Be 502 8 Female 2.75 463 (180 Romanian science students (Pitariu, 1986) 40 minutes ne Male ne 6st Female University of California (Berkeley) = 0 undergraduates (Paul, 1985) Untit nwo 48 \o Male 3623 SN ee‘Teacher Training 24.10 2454 2387 2215 2305 2488 2428 25.63 2245 22.63 2as2 20.03 1987 25.72 2400 2055 a9 22.24 261 tea 291 an an 412 416 4.00 6.33 3.86 448 433 394 548, 3.98 377 467 463 544 6.06 378 673 37 388 462 533 579 450 405 5.16 178 1316 Rn 281 8x8 132 v8 64 a no 108 100 287 164 New Zealand New Zealand Poland*Table APM32 Advanced Progressive Matrices Set ll ‘Means and SDs for a Range of Groups Mean = SD a Source Occupational Groups (40-minute time limit) UK police officers (assessed for possible 24 38 157 Feltham, 1988 accelerated promotion to senior rank) UK National Computing Centre Threshold Scheme 216 585361 PRU, 1985 for prospective data processing personnel I institute undergraduates 2448 5.09 «687_—~Penmey, 1991 Occupational Groups (30-minute time limit) UK National Computing Centre Threshold Scheme 2178 6.09153. PRU, 1985 for prospective data processing personnel College Groups (40-minute time limit) UK undergraduates 2680 4.40 64 PRU, 1985 UK psychology students 2488 (3.77 38 SHL, 1980 Applicants to UK psychology degree course 24280 467281 Gibson, 1975 Australian university students 2345 «494 745. Yates & Forbes, 1967 New Zealand university students 217 422 38T Yates & Forbes, 1967 Romanian science students 21.64 6.331316 Ptariu, 1986 New Zealand physiotherapy students 261 4.05 100 Yates & Forbes, 1967 New Zealand teacher training students 2159 4.76 «= 249——‘Yates & Forbes, 1967 New Zealand nursing students 1832 5.16 © -287,——Yates & Forbes, 1967 College Groups (no time limit) University of California, Berkeley, undergraduates 2798 4.40 48 Jensen, 1987 University of Alabama, Birmingham, undergraduates 20.14 5.92 131 McLaurin et af 1973 University of California, Berkeley, undergraduates 27.03 5.01 300 Paul, 1985 Miscellaneous Groups (no time limit) 19.04 © 6.56 1015 Kratzmeier & Horn, 1980 German 15-year-oldsTable APM33, ‘Advanced Progressive Matrices Seti Occupational Norms for a Range of Groups ve @ a we rm Ce = Soren “Tipu” atta MOM seal grad tee poke senna acomtany Onlrd aah ag oS et 28yrck appicanss manages ‘etlivng on “pend” aa” erg ponte ae, Sree trend __Urtimed orion sori “rns “40 rien”—_ econ) Somma——Dmimm—“aorang” Mane Serta 3 % 3 2 sue ™ % » ™ x > %0 ” 20 7 2 som 2 2 » om = no» 5 ” ” me eos x 2 2 3 a x 0 2 2» * os 2 2 a * ae) n un 2 ” 6 Bo om ow ow 3 8 nom ” no 0 2 ” 0 8 we 2 0 sR 2 wou 5 ° 7 o 6 won a ‘ ” a " «on . 71 3 15 eS 7 ° a a) “an UK general population data derived from the 1952 standardisation ofthe SPM and APM (Table APMIS). US general population data derived from the 1983 standardisation ofthe SPM and APM (Table APM). US Navy data extracted from deta supplied by Alderton (see Knapp & Court, 1992) (Table APMZ2. UK Police Ofiers’ data extracted from Feltham (1968) (Table APM33). ‘lian data from Giambelluca and ZizoH (1983). ‘Other data collected by Oxtord Prychologiss Pres.Table AP ert SPM and gable to convert APM Set t Raw scores eg aun SM | ar 1 17-21 19 2 2225 20 32 A 26-28 a i 4 at 2 3 5 32-33 a sa 6 34-35 a 55 7 36-37 2 ET 8 38-39 % 56 9 40-41 a 37 10 42 23 a " 43-44 2» a 2 45 30 3 3 46 31 “4 a7 32 5 15 43 33 16 “9 34 7 50 35 18 st 6 Note: It is not possible to estimate abilities corresponding to the minimum and maximum APM raw scores of 0 and 36 because the ability estimates are nity and infinity respectively, so equivalent raw scores on the SPM can- not be provided. Equivalences established by Andrich and Dawes (now Styles): See Research Supplement No. 4 to Manual for details. Table APM35S Ht Raw Scores Table to Convert APM Set 1 and APM Set ‘APM |percentile Points and PM Grades with 1s. Table APM36 Table for Conveniently Comparing Parsee a, Standard Deviations, Standard Scores, 5 PM Standard standard | 1 Percendle PM sore, Stanine Sana a ee grade score _Stanine deviation = line deviation 5 a 42.33 99472 me 48 5 0.07 wom FS 2a oe 48 Son moms te 8 | Lo te ' - 6 394A ana moss Sn eee ; FS Ae i 88 9 4345 ME fs 4 40 129 973 1 87 9 93 319k . aaa 128 96.9 ' 86 9 92 29.8 as 39 4 053 127-964 ' as 9 oy 27a te 4-060 90254 th 8G 4-987 126 95.8 1 84 8 125 95.2 1 83 8 9 233 as 4 on 124 945 ir 82 8 ee 2 v Sd 3 0.80 123° 93.7 te 8 8 a7 19.2 v 33 3 087 12 92900 oe 78 8 6 76° =20N 32 3-093 vw 9190078 8 a 159° 30 3 -1.00 120 90.8 Neo 8 a omu2 UW 28 30-107 190 998 76 8 eo ws 27 a 43 118 885 " 7s 7 2 ws ov 25) 3) a> 117° 87.1 " 73 7 woz N24 2 4a 116 858 " 72 7 gw 920 MB 20 133 15 841 " 70 7 7% 81 N22 20 41.40 14 824 0 69 7 7] 74 MA 20-147 113 808 " 67 7 77 63) Wo et-9 20 7153 2 788 " 66 7 ree Sse, 7 2-160 at 767 " 64 6 40.73 3% 48 Vv 16 2-167 10 749i 63 6 4067 m 42 ~V 14 20-3 109-726 20D 6 40.60 BR 36 OV 12 1-180 108 702 2 le «= 6.0 6 = +053 Rm 3A v WW + 71.87 107 68.1 Me 59 6 +047 n 27 v 1.0 1 193 106 655i ST 6 +040 7m 23 v 09 1-200 105 629 le 56 6 +033 0o 9 Vv os 1-207 104 606 We 55 6 40.27 6 Ww v 07 1 23 103 57.9 me 54 5 40.20 67 14 v 07 1 -2.20 102 552 ile 52 5 40.13 6 12 v 06 1 227 101 528 ile 51 5S 4007 6 10 v 06 1 2B 100 5005.0 5 ° Note: This table is provided to facilitate general comparisons only. RPM scores should not be converted to deviation IQs because the distribution of the scores is not Gaussian for most of the populations which have been studied:Appendix |: The 1993 the APM, SPM, ang M the USA V Ai rdisatio n of Mong Adults in The 1993 adut standardisation ofthe APM, spy a Mownes, lowa. Des Moines is one of four US ¢ tes, a = appronmate those ofthe US asa whole# Ths ere demograph Caren Om standardisation caried out there would approra lead one to ep imate Supplement No.3 to this Manual shows, the thre Des m, ofthe SPM among schoolchildren did indeed foines norms adult members of, many transient residents are missing ahousehal and ‘Armeans of overcoming ths dificuty was devised by Kh in 1949, and hs proces he been folowed ever since's©. Census data are used to identify a set of census blocks whch are between them, representative ofthe population to be stud in terms of ae, ebnct, rd soco- {economic status. The dwelngs in these blocks are then listed (usualy by sening researcher 0 he area forthe purpose), and a random sample of dwelingsin the blocs is then chosen. say taking a random start and then selecting every nth dweling on a repr bass Acomectons at .ced to cover dwelings which have been constructed or demolshed between the censs andthe time the study is conducted. Since itis, infact. a sample ofncviduals. not dweings wich segura, {tis then necessary for researchers to visit al the selected dwelings, obtan a complete ist of ther ‘residents, and then folow the sampling procedures developed by Kh to dent the nova wate the household who isto be asked to take part in the study. This procedure corecs re ett 4 sample of dwelings would otherwise yield a disproportionate numberof sgl peo of Inthe present study, jody Connar and Gary White of hearing secon of 6 TA Sexil Research ofthe University of Michigan used ther compute tases Sevens ST identify from the 1990 census 30 areas of Des Moines which were expects) © YET and addresses of 1,000 people who would be representative ofthe popsston ‘twas expected that this would yield about 650 usable sets of at Pri Poorman of the Paning Department in Ds Moines en ‘he dwelings inthe designated areas. From these, Kandy Lee, Our frtraced a random sample of dwelings. Ast tures ov. SOME TE ‘date and the samples for these areas had to be corrected ho visited the areas and listed the current dwellings 7 fomne ht “The researchers obtained the nares of he hea of SRC py amt Bret Registers. These heads of household were sent a bref ete ns aperwsens tne sy eens. APM103 =AMONG ADULTS INTHE USA pi: THE 1993, STANDAR! state University, outing the purpose ofthe study and lowa ig to se them. The advance wary her who that the researe by a covering I the Department of Psychology and the field workers rovided liaison between Gary Phye P the researcher fist isted separately all male and female residents (On visiting the dweling the Trervare then used to identify the person who was to be asked aged 17 and over. Kiss procedures to take part in the study. ‘Asan inducement to takepart, potential respondents were told that there would be a prize draw of a number of household goods [together the previously mentioned sampling procedure yielded 972 addresses. Ifa dwelng had been demolished, an adjacent dwelling was chosen. No substitutes were taken for those who refused to take part since such people are by that fact alone shown to be different from those who are willing to do so. It proved impossible to make contact with 72 households. The response rate from contacted households was 70%. 625 (compared with the initial target of 650) usable sets of data were obtained. Half were male and half female. 12% did not give sufficient information to enable their socio-economic status to be ascertained, Of the remainder, 23% had professional and managerial ‘occupations. This compares with 21% of Des Moines residents, and 20% of US residents, as. reported in the 1983 County and City Data Book. (These and other relevant data on Des Moines are available on page 91 of the 1990 edition of Research Supplement No.3 to this Manual). 4% did Not give sufcient information on their ethnicity to make classification possible. Of those who did, 8% indicated they were Black, 87% White, and most of the rest Hispanic or Asian, The 1990 Census figures for Des Moines are 7% Black and 89% White and, for the US as a whole, 12% Black ig om ee Overall these figures suggest that there is every reason to have confidence that or eas ts a ines representative of Des Moines and that its demographics approximate iple-Choice noThe isues associated with cultural, ethnic, ang thought through. Gender, religion, ethnicity, enormous power to predict behaviour es 24 ect tionis not whether psychological vecitona ne eather tests on ceupaonal ny 88nd cage Py tan can be made fom hormone oot it pe 28 say values, abiltes, motivational dispositions ang te aoe eee oa power of such social variables might alone society ee big ing ee the sein oation The way forwards to te ane ae bene monte ing clearer about the qualties which are dee th oe cates really Ned in seecion po sought, and the personal and social consequences Postions for when ent xe possess such qualities, Those involved in des, SINE Wa derting ene dures must not be content with a roe in which end 2254 Secon ny rt eH Pacement proce. tional positions: There are large dferences inthe average motivation valves and abi f oe ites Of the members of There are large differences in the educational, occupational, and sail performance of he members of the groups. Most importantly, access to (or exclusion from) the educational or occupational postons ‘concemed is associated with some major social benefit (or disadvantage). Under such circumstances there isa strong temptation to attack the tests used instead of seeking to understand and alter the social process in which they are embedded Infact mare ae {consideration ofthe issues pons to the condson ht the slon to mos abe pers Which are presented as problems of test bias or faimess is to be found by improve hensiveness ofthe assessments which are made boxing te rg panos SES and societal consequences whch ae expity discussed nd meron erp social arrangements available to recognise, develop, utilise, and canbe oped The naivety of many ofthe sumptions mae n disor ge te rethinking stimulated, by asking such questions as:"Are our METS ts) in America have, on average, lower bith weights thin WINE sedge biased because they have higher rates of infant mortality? din 19387 because Blacks in Ameria now score higher than the Bish 8 {7OUp would you expect the norms for the grou cB ow) norms for Whites in the Appalachian mountains the high om orp ‘wealthier counties in the United States of America. trees cecorii Before entering into ary serious dete aoa ‘ en ea Noto pos conearing test bas ad HE ea a the Manual and the pages of this Section which become quite clear in their own mind whatenone 1 FAIRNESS _ * indyadual and collective occupat et 8 te es Saba, «of the Manual to clanfy the the afferences that are likey to be vnc subgroups of the population. At the causes and consequences of discovered about ier scsi of relevant SSU6S which are to be in', and Raven listed in the booklet Equal ‘ul to consider the questions oe viowste prepared by Savile and Holdsworth Ltd in associa. ity and the Equal OPPOTUNTES Commission. However, vs raved here, users may also find it some of the issu how. These are intended to Provone 2 discussion which these differences. THEY found jensen” Sea ma for Best Test Jon for Racial Ev Beyond ths. users Opportunines. Gukance tron with the COmTIss nce ths bookdet does not CnAES wath sera to consider the questions Ite bel vee to a beter understanding ofthe SSUES ier the qualities that are really most 7 ducted the job analysis cons sree lamang ‘others? Would the assessment of some ‘based motivational dispositions which are likely t0 contribute to (and detract to exclude a stil higher question? (inthis context itis worth noting that Flynn ‘educational and occupational performance of people of Chinese and japanese orign in spe USA resuts from cutural and motvational rather than intellectual, factors. As a ‘ould well claim adverse impact if only intellectual tests were used are the clear (negative) implications for other in selection. More serious, however, be based on a wider range of relevant critena.) ethnic groups of caling for selection to . sre the qualties measured by the test real relevant to one or other of the diferent ‘ppes of extsanding performance that woud be useful inthe job function, to excivang ‘tose who are likely to behave in organisationally destructive ways, or to excluding those who are unlikely, after participating in perhaps costly staff-development programmes, to remain with the organisation? «Is the way the competencies are measured lkely to fail to engage with the motives anc sales lore fe pel bing sessed and ht ad he 0 9 A important and relevant competencies they possess? Is the organisational climate of the workplace likely to have a similar effect? ome + bsthe f sexsi te competencies are measured likely to discriminate against certain groups which are imelevant to job performance or retention? + te " rca a, ‘quick and inexpensive screening instrument to reduce the ity omen Ye deed gatbe ota 5 (( to identify those who are most who undoubtedly do posse: ies whilst inevitably excluding a number of people ‘Seproporuonate number of, those qualtes), does it have the effect of eliminating 2 € particular subgroup whose scores differ markedly on the {est question altho desired differ ee ‘they difler only marginal in their probabilty of possessing the + W small difere es n test hele of Serena rena Unlikely to have a major impact on performance © feelings of faimess? (Bear in mind that. n \iew ofthe increase t% in scores over ti m) ime, no one would dream of not using age-related APMI06+ Does the cause for concem reflect not so much a desire to ensure that those who are most likely to contribute outstanding performances are selected as a desire to ensure that a disproportionate number of certain groups are not, as a result of exclusion, subject to demeaning and degrading treatment by society? «+ Ig some of the knowledge, and are some of the skills, deemed important to job perfor- mance, and therefore tested in the selection process, either irrelevant to such perfor- mance or easily acquired in the course of work experience? Might it be more appropriate for the organisation to admit applicants more or less at random and pay more attention to developing its managers’ ability to think about develop, and utilise the diverse talents of their subordinates?