Psychological Assessment Copyright 1995 by the American Psychological Association, Inc.

1995. Vol.7. No. 2, 183-194 1040-3590/95/S3.00

Traditional Psychometric Tests and Proportionate Representation:

An Intervention and Program Evaluation Study
Dennis P. Saccuzzo and Nancy E. Johnson
San Diego State University

The Wechsler Intelligence Scale for Children—Revised (WISC-R) and Standard Raven Progressive
Matrices (SPM) Test were evaluated in the context of an intervention/program evaluation study
and in terms of a proportionate representation model of test bias. A total of 26,300 boys and girls
from 8 different ethnic backgrounds were evaluated over a 9-year period. An intervention based on
monitoring of and feedback to referral sources proved effective in improving proportionate repre-
sentation in the referral process. Moreover, the WISC-R and SPM showed approximately equal
predictive validity and no evidence of differential validity. Significant differences were found as a
function of ethnic background between those referred and those certified as gifted, as well as between
those referred and those who scored in the 98th percentile on either test. Implications for traditional
tests are discussed.

The controversy over the use of psychological tests for selec-
tion purposes and clinical decision making has been repeatedly
noted (Lewandowski & Saccuzzo, 1976) and rages on unabat-
edly (see Kaplan & Saccuzzo, 1993, Ch. 20 and 21). From a
legal standpoint, case law mandates the elimination of test bias
(Diana v. State Board of Education, 1970; Hobson v. Hansen,
1967; Guadalupe v. Tempe Elementary School District, 1978;
Larry P. v. Riles, 1979), whereas federal legislation requires
that fairness be maintained (Education of All Handicapped
Children Act of 1975; Title VII of the Civil Rights Act of 1964;
Rehabilitation Act of 1973; Americans With Disabilities Act of
1990).
A major problem is that there is no single clear definition of
test bias (Barrett & Depinet, 1991; Dunnette & Borman, 1979;
Reschly & Sabers, 1979; Thorndike, 1971). According to
Hunter and Schmidt (1976), three ethical positions underlie
the debate over test bias: unqualified individualism, the use of
quotas, and qualified individualism. The doctrine of unquali-
fied individualism simply states that tests should be used to se-
lect the most qualified person regardless of gender or ethnic con-
siderations. On the other extreme, the use of quotas would re-
quire a test to select a specific percentage of individuals as a
function of gender and ethnic background in order to be con-
Dennis P. Saccuzzo and Nancy E. Johnson, Department of Psychol-
ogy San Diego State University.
This research was funded by Grant R206A00569.
We express our appreciation to the San Diego Unified City Schools,
to the Gifted and Talented Education (GATE) Administrator David P.
Hermanson, and to the following school psychologists: Marcia Dijlosia,
Dimaris Michalek, Ben Sy, and Daniel Williams. We also wish to thank
Tracey L. Guertin for her role in the data collection and analysis.
Correspondence concerning this article should be addressed to Den-
nis P. Saccuzzo, San Diego State University, 6363 Alvarado Court, Suite
103, San Diego, California 92120-4913. Electronic mail may be sent via
the Internet to dsaccuzz@sunstroke.sdsu.edu.
sidered unbiased. Qualified individualism, the compromise be-
tween these extremes, states that although the best qualified per-
sons should be selected, information about gender and ethnic
background should be considered in the selection process.
These various positions are represented by at least four psy-
chological definitions or models of bias: regression, constant ra-
tio, regression plus point adjustment, and proportionate repre-
sentation. The regression model (Cleary, 1968) maintains that
separate regression lines should be used for different groups,
and those with the highest predicted criterion scores should be
selected. In the constant-ratio model (Thorndike, 1971), points
equal to roughly half of the average difference between the gen-
ders or various ethnic groups are added to the test scores of the
lower scoring group. A single regression line is then used for all
groups, and those with the highest predicted scores are selected.
Cole (1973) and Darlington (1971,1978) describe what can be
called a regression plus adjustment model. In this Cole/
Darlington model, separate regression equations are used for
each group, as in Cleary's (1968) regression model, but points
are added to the low-scoring groups as in Thorndike's constant-
ratio model. Finally, in the proportionate representation model
of test bias, a test is considered fair only if individuals from
different genders and ethnic backgrounds are selected in pro-
portion to the population of the community from which they
are selected.
Although psychologists may argue the relative merits of the
various models, recent case law and federal legislation are be-
ginning to impose legal definitions of bias. In Watson v. Fort
Worth Bank and Trust (1988), for example, the Supreme Court
held that statistical selection ratios are sufficient evidence of ad-
verse impact (a selection rate of any gender, race, or ethnic
group that is less than four-fifths that of the rate for the group
with the highest rate). Thus, the Court upheld a variation of the
proportionate representation model. Moreover, the 1991 Civil
Rights Act, Section 9, specifically prohibits any discriminatory
use of test scores.

183
184 DENNIS P. SACCUZZO AND NANCY E. JOHNSON
In concert with legal mandates, psychologists have an ethical
and professional responsibility to evaluate how tests are used in
the decision-making process. In this study, we examine one such
use: the use of intelligence test scores in the selection of individ-
uals for giftedness. The study spans a 9-year period in a large,
diverse school district in which two tests, the Wechsler Intelli-
gence Scale for Children—Revised (WISC-R; Wechsler, 1974)
and the Standard Raven Progressive Matrices (SPM; Raven,
1938) test, were administered on a wide-scale basis to thou-
sands of male and female children from eight major ethnic-
cultural backgrounds. It is important to note that the school
system itself did not use test scores as the sole determinant of
qualification for the gifted program. In the present study, we
provide information about an intervention and program evalu-
ation involving the large-scale use of tests. This intervention led
to a modification in the referral that resulted in significant in-
creases in the number of traditionally underrepresented chil-
dren referred for a gifted program. In addition, the dependent
measure was changed from the WISC-R to the SPM. Finally,
we examined each of the two tests in terms of a proportionate
representation model of test bias.
During the initial phase of the study, archival data taken be-
tween 1984 and 1990 were analyzed and presented to school
officials. These data, in which the WISC-R had been the pri-
mary tool to determine IQ and qualification for the gifted pro-
gram, revealed that children were neither referred nor selected
in proportion to their numbers in the district as a whole. Our
feedback of these results to the district led them to provide
training to teachers and other referral sources to help them
identify and refer potentially qualified students from underrep-
resented groups, such as Latino/Hispanics and African Ameri-
cans (Saccuzzo, Johnson, & Guertin, 1994). In addition, the
district shifted from the use of the WISC-R to the SPM in the
hope of finding a measure of IQ that would lead to proportion-
ate representation in the selection process.
The SPM consists of 60 matrix problems, which are sepa-
rated into five sets of 12 designs each. Within each set of 12, the
problems become increasingly difficult, and each of the five sets
is progressively more difficult. Each individual design has a
missing piece. The participant's task is to select the correct
piece to complete the design from among six to eight alterna-
tives. Correct responses are based on various organizing princi-
ples, such as increasing size, reduced or increased complexity,
and number of elements.
Because SPM stimuli (and other Raven Progressive Matrices
problems, such as the Advanced Raven Progressive Matrices)
are visually presented, it is easy to mistake the test as one of
visual perception or spatial reasoning. It is neither. As Cherkes-
Julkowski, Stolzenberg, and Segal (1990) have noted, "The Ra-
ven is as close to a study of pure thinking processes in the ab-
sence of the influence of specific content acquisition as is avail-
able" (p. 7). As Snow and colleagues have shown using radex
and hierarchical models, the SPM is among the best available
measures of general intelligence and complex reasoning
(Marshalek, Lohman, & Snow, 1983; Snow, Kyllonen, & Mar-
shalek, 1984). As a measure of general intelligence, the SPM
correlates highly with verbal measures of ability, even though
the stimuli themselves are completely nonverbal (Carpenter,
Just, & Shell, 1990). In fact, positron emission tomography
(PET) scans, which produce computer-generated images of the
brain, have shown that the entire brain is involved in solving
SPM problems, with the three most used areas being the right
cerebral hemisphere, the left temporal lobes, and the left frontal
lobes (Haier et al., 1988). The left temporal lobe involvement
is most likely due to the use of verbal codes in solving SPM
problems.
Because its stimuli are nonverbal, the SPM can be adminis-
tered fairly to individuals who speak a language other than En-
glish. Because stimuli are visually presented, rather than spo-
ken, they are not transitory. Thus, the stimulus remains in front
of the individual, which reduces the role of memory and even
attentional factors in performance (Cherkes-Julkowski et al.,
1990). Solving SPM problems does not depend heavily, as do all
language-based tests, on acquired knowledge, specific cultural
experiences, or reading ability. As Carpenter et al. (1990) have
noted, "The Raven measures the ability to reason and solve
problems involving new information, without relying exten-
sively on an explicit base of declarative knowledge derived either
from schooling or previous experience." In sum, the Raven Pro-
gressive Matrices Tests, such as the SPM, measure general intel-
ligence and correlate with measures of linguistic ability. These
tests use nonverbal stimuli and do not require a specific knowl-
edge base.
Previous investigations have found that the SPM has not only
been effective in identifying traditionally underrepresented chil-
dren for gifted programs but also correlates with their success in
school (Baska, 1986). In one study, Powers, Barkan, and Jones
(1986) found no significant differences between Hispanic and
Anglo-American children's mean scores, score variability, and
test reliability for the SPM. Other studies have supported the
validity of the SPM for Hispanic (Powers & Barkan, 1986) and
Navajo students (Sidles & MacAvoy, 1987). In these studies, the
SPM was found to be predictive of success in a program for
gifted students.
As Raven (1989) has noted, the Raven Progressive Matrices
Tests have been used in over 2,400 published psychological stud-
ies (Court, 1988; J. H. Court, personal communication, March
30, 1994; Court & Raven, 1977 & 1982), making the SPM and
related forms among the most researched psychological tests.
Until recently, however, the SPM has not been used widely in
applied clinical and educational settings. A major problem had
been the lack of adequate U.S. norms (Kaplan & Saccuzzo,
1989, 1993). An extensive and relatively current set of norms,
which include U.S. as well as worldwide norms, is now available
(Raven et al., 1986). More than 30,000 students ages 5 to 18
were chosen to be representative of school districts across the
United States in approximately 30 norming studies. Ethnicity
and socioeconomic factors made independent contributions to
the variance. Ethnic differences, which were attributed to
differences in birth weight, infant mortality, and the incidence
of serious childhood illness, showed a decline compared with
earlier reports (Burciaga, 1973;Hoffman, 1983;Jensen, 1980).
There were, however, no major Hispanic-White differences in
the Ontario-Montclair school district of California. Moreover,
the SPM was found to have equal predictive validity within each
group (Hoffman, 1983, 1986).
 PROPORTIONATE REPRESENTATION 185

so r

[J 1984 -1990
2
w • 1990 -1993
w

ETHNICITY
Figure 1. Average district enrollment ratio by ethnic background for the 1984-1990 and 1991-1993
periods.

It should be noted that although ethnic differences may be
declining, there remain differences in the mean scores as a func-
tion of ethnicity as well as socioeconomic background. Never-
theless, given reported equal predictive validities across ethnic
groups, the question remains as to whether the SPM can be used
to achieve proportionate representation for selection purposes.
Method
Participants
The participants were 16,985 children who were referred and evalu-
ated for the gifted program at San Diego City schools between the fall
of 1991 and the spring of 1993. Students were classified into one of eight
ethnic backgrounds as follows: 3,864 Latino-Hispanic, 6,286 White,
2,389 African-American, 483 Asian, 75 Native American, 104 Pacific
Islander, 1,419 Filipino, and 958 Indochinese. There were 1,407 classi-
fied as "other." Of the 16,985 participants, 51.5% (8,740) were girls,
48.5% (8,245) were boys. These children ranged in age from 5 years, 10
months to 18 years, 2 months, with a mean age of 9 years, 10 months
(SD = 2 years, 5 months). The majority were tested in the second
(7,067) and fifth (3,366) grades. Of the remaining, 22 were tested in the
first grade, 1,376 in the third, 774 in the fourth, 720 in the sixth, 1,866
in the seventh, 236 in the eighth, and 82 in the ninth, with grade data
missing on the remaining.
The data files of all children evaluated for giftedness in the same
school system between 1984 and 1990 also were examined. During this
time period, the school system had used the WISC-R as the primary
tool for identifying giftedness. A total of 9,315 students had been given
the WISC-R during this time period. Of these, 505 were Latino-His-
panic, 7,100 White, 601 African American, 283 Asian, 24 Native Amer-
ican, 13 Pacific Islander, 380 Filipino, 132 Indochinese, and the remain-
ing classified "other." Gender distribution was virtually identical, with
4,517 girls and 4,521 boys, and gender data missing for the remaining
participants. Participants ranged in age from 5 years, 0 months to 17
years, 2 months, with a mean of 8 years, 11 months (SD = 1 year, 10
months). In terms of grade, 181 were tested in the first grade, 3,354 in
the second, 2,031 in the third, 1,026 in the fourth, 761 in the fifth, 830
in the sixth, 363 in the seventh, 240 in the eighth, and 170 in the ninth,
with grade data missing for the remaining participants.
A major point of comparison was based on actual enrollment figures
by ethnic background during the course of the study. The average en-
rollment breakdown in the district as a whole by ethnic background
between 1984 and 1990 and between 1991 and 1993, respectively, was
as follows: 21.8 and 27.2% Latino-Hispanic, 44.4 and 37.2% White,
16.2 and 16.2% African American, 2.3 and 2.3% Asian, 0.4 and 0.6%
 186 DENNIS P. SACCUZZO AND NANCY E. JOHNSON
2.00 r
0.00
ETHNICITY
Figure 2. Proportion of children referred for giftedness testing compared to the average district population
for each of two time periods (1984-1990 and 1991 -1993) as a function of ethnic background.
Native American, 0.6 and 0.7% Pacific Islander, 7.8 and 8.0% Filipino,
and 7.4 and 7.7% Indochinese (see Figure 1). The proportion of stu-
dents in each ethnic background referred or certified was compared
with the actual proportion in the district as a whole. In addition, all
groups were evaluated in terms of the proportion who actually scored
two standard deviations above the mean (i.e., 98th percentile) on either
the WISC-R or the SPM as a function of gender and each of the eight
ethnic groups.
Procedure
Children were given either a WISC-R (entire 1984-1990 sample) or
an SPM Test (entire 1991-1993 sample) by a district school psycholo-
gist. The WISC-Rs were individually administered; the SPM was group
administered. As a part of the evaluation process, the school psycholo-
gists conducted a case study analysis of each child to evaluate for the
presence of risk factors and level of achievement. Achievement was eval-
uated in terms of standard scores on the California Test of ftasic Skills
(CTBS; McGraw-Hill, 1981) or the Abbreviated Stanford Achievement
Test (ASAT; Psychological Corporation, 1988). Six risk factors were
considered: cultural, language, economic, emotional, environmental,
and health. During the 1991-1993 period, cultural and language risk
were combined into a single category because of the high correlation
1984-1990
1991-1993
between these two variables. Risk was determined by a self-report ques-
tionnaire sent to parents and a risk questionnaire form completed by
teachers. These were then evaluated by a school psychologist as part of
a case analysis previously described by Saccuzzo, Johnson, and Russell
(1992) as follows:
Cultural risk included cultural values and beliefs that differ from
those of the dominant culture or limited experience in the domi-
nant culture. Economic risk included parental unemployment or
household income low enough that the child qualified for the free
lunch program. Emotional risk encompassed such factors as death
of a parent, child abuse, major psychiatric illness in the home, or
extended absence of a parent because of military service. Environ-
mental risk included transiency (three or more changes in schools)
and excessive absences from school because of home responsibili-
ties such as child care duties or working to help support the family.
Health factors included vision, speech, and hearing deficits that
required designated instructional service, motor problems that re-
quired adaptive physical education, or diseases that caused ab-
sences or hampered school progress, such as asthma. Language risk
included speaking English as a second language and lack of fluency
in English.
Regardless of gender, ethnic background, or risk, children were certi-
 PROPORTIONATE REPRESENTATION 187

fied as gifted if they obtained a Full Scale IQ of 130 on the WISC-R or Table 1
an IQ equivalent of 130 on the SPM (i.e., achieved a score in at least the Chi-Square Results on the SPM and WISC-R, Comparing
98th percentile) based on the Smoothed U.S. Norms reported by Raven Children Referred as the Observed With the District
et al. (1986). In addition, children who had two or more of the risk Population as the Expected
factors were certified as gifted if they had a Full Scale WISC-R IQ of 120
or SPM IQ equivalent of 120. Finally, children who obtained a WISC-R n n
or SPM score of three standard deviations above the mean were placed Ethnicity df (expected) (observed)
in a special "Seminar" program.
SPM
Results
Latino-Hispanic 4237.2 3864 45.16***
Figure 2 illustrates the referral patterns during each of the White 5795.0 6286 66.24***
two periods of the study. The figure shows the proportion of African American 2523.6 2389 8.70**
Asian 358.3 483 44.43***
children in each of the eight ethnic backgrounds who were re- Native American 93.5 75 3.67
ferred for giftedness testing (by teachers, parents, or through Pacific Islander 109.1 104 0.24
central nomination on the basis of high standardized achieve- Filipino 1246.2 1419 26.03***
ment test scores) divided by the average proportion for that eth- Indochinese 1199.5 959 52.25***
nic group in the district population for each of the two time
WISC-R
periods. For example for the Latino group, the 1984-1990 bar
graph is based on the average proportion of children referred Latino-Hispanic 1970.3 505 1393.50***
for giftedness testing who were Latino divided by the average White 4012.9 7100 4271.49***
African American 1464.2 601 607.22***
proportion of Latino children in the district during the 1984- Asian 198.8 283 36.43***
1990 period. The corresponding bar graph for the 1991-1993 Native American 36.2 24 4.10*
period is based on the average proportion of those referred for Pacific Islander 45.2 13 23.05***
giftedness testing who were Latinos divided by the average pro- Filipino 686.9 380 148.39***
portion of those in the district during the 1991-1993 period Indochinese 650.7 132 445.60***
who were Latino. A 1.00 proportion, of course, represents per- Note. SPM = Standard Raven Progressive Matrices Test; WISC-R =
fect proportionate representation; the closer to 1.00, the better Wechsler Intelligence Scale for Children—Revised.
in terms of proportionate representation. *p<.05. **p<.01. ***p<.001.
Inspection of Figure 2 reveals a marked bias in the referral
process during the 1984-1990 period. Only about 0.25 or less
of the Latinos, Pacific Islanders, and Indochinese were referred ethnic group in the district. This figure shows that there was a
compared with their numbers in the district during the 1984- dramatic increase toward proportionate representation of chil-
1990 period. By contrast, the proportion of Whites tested was dren certified as gifted during the 1991-1993 period. The pro-
1.75, or about 75% over the expected. portion of Latinos certified more than doubled, whereas that
Further inspection of Figure 2 shows the effectiveness of the for three groups, Native Americans, Pacific Islanders, and In-
intervention (i.e., monitoring, feedback, and training) that was dochinese, went from substantial underrepresentation com-
used to help teachers (the primary referral source) identify po- pared with their actual numbers in the district to a near-perfect
tentially gifted and underrepresented children. There was a proportionate representation. In fact, there was movement to-
marked increase in the proportion of Latinos, African Ameri- ward proportionate representation for all groups except the
cans, Pacific Islanders, and Indochinese represented in the re- Asians, who were even more overselected during the 1991 -1993
ferral process. period.
To evaluate the referral patterns, chi-square analyses were Results pertaining to certification for giftedness can be fur-
conducted comparing the number of children actually referred ther viewed from the context of referral patterns and biases. Fig-
in each ethnic background (observed frequencies) with the ure 4 shows the proportion of children certified as gifted in each
number expected on the basis of the proportion of children rep- ethnic group compared with the proportion actually referred
resented in each ethnic background during the respective time during each time period. Inspection of this figure reveals that
periods. Table 1 presents a summary of these analyses. Minor although groups such as Latinos, African-Americans, Native
discrepancies in Table 1 and in other tables in the number of Americans, Pacific Islanders, and Indochinese were grossly un-
cases are due to missing or incomplete data on a particular derrepresented in the referral process during the 1984-1990 pe-
participant. riod, of those who were referred, a proportionate number was
Inspection of Table 1 reveals that there were significant dis- found to be gifted. Thus, the referral sources were remarkably
crepancies from proportionate representation in the referral accurate in identifying children who would ultimately score in
process during both time periods. During the 1991-1993 pe- the upper ranges of IQ on the WISC-R. Further inspection of
riod, however, the discrepancies were far smaller. Figure 4, in comparison with Figure 3, shows that although in-
Figure 3 illustrates the proportion of children in each ethnic creasing the proportion of referred underrepresented children
group certified as gifted compared with the district proportion did lead to greater proportionate representation in the certifi-
during the respective time periods. For each ethnic background, cation process, fewer of the referred children from the under-
the proportion certified was divided by the proportion of that represented groups ultimately qualified as gifted.
 188 DENNIS P. SACCUZZO AND NANCY E. JOHNSON

2.5 r

fj 1984-1990

• 1991-1993

w
to

ETHNICITY
Figure 3. Proportion of children certified as gifted compared to the district proportion for two time peri-
ods (1984-1990 and 1991 -1993) as a function of ethnic background.

Table 2 provides statistically relevant support for the data
presented in Figures 3 and 4 and the statements made pertain-
ing to these figures. The table compares the number of children
actually certified as gifted (observed) with the corresponding
proportionate number in the district population and then to the
proportionate number in the referred population (to adjust for
differences in total «) during the time period under study. This
table shows that when more children were evaluated with the
SPM in the underrepresented groups, there was increased but
not perfect proportionate representation. Considering only the
referred population, however, there were few discrepancies from
proportionate representation during the 1984-1990 period.
Thus, there was a highly selective but relatively accurate referral
process during the 1984-1990 period, and of those children re-
ferred, all except African Americans were selected in propor-
tion to expectation. When more were referred (during the
1991-1993 period) more were found to be gifted, but propor-
tionally fewer were selected.
Table 3 provides further statistical support for the data pro-
vided in Figures 3 and 4, and Table 2, and the statements made
pertaining to them. Table 3 shows the means and standard de-
viations of the SPM and WISC-R IQ data for each of the re-
spective time periods. As Table 3 shows, there were clearcut
differences among the mean IQs for the ethnic backgrounds.
Analysis of variance (ANOVA) reveals that these differences
were statistically significant.
For the SPM, there was a significant effect when z scores were
evaluated for the eight levels of ethnic background, F(l, 5518)
= 60.58, p < .001. Scheffe post hoc multiple-comparison tests
revealed that the scores of the Whites and Asians were signifi-
cantly (p < .05) higher than those of all of the other groups
except Native Americans and that the scores of Filipinos were
significantly (p < .05) higher than those of Latinos. The
ANOVA for Full Scale IQ was also significant, F(7, 6686) =
42.38, p < .001. The Scheffe test revealed that Whites' and
Asians' scores were significantly (p < .05) higher than those of
African Americans, Indochinese, Filipinos, and Latinos and
that Latinos' scores were significantly (p < .05) higher than
those of the Indochinese. As Table 3 also shows, there were de-
clines in IQ scores across the two time periods for all ethnic
groups. This decline reflects the larger number and percentage
of students tested in the 1991-1993 period in the effort to
achieve proportionate representation in the referral process.
To examine possible ethnic differences on the SPM and
 PROPORTIONATE REPRESENTATION 189

WISC-R CERTIFIED (1984-1990)

SPM CERTIFIED (1991-1993)

HO

ETHNICITY
Figure 4. Proportion of children certified as gifted compared to the proportion actually referred for each
of two time periods (1984-1990 and 1991 -1993) as a function of ethnic background. WISC-R = Wechsler
Intelligence Scale for Children—Revised; SPM = Standard Raven Progressive Matrices.

WISC-R independently of risk factors, we used the following
system. First, we considered only those children who scored in
the 98th percentile or better on the SPM or WISC-R as the
observed frequencies. Next, we determined the number of chil-
dren in the school district and in the referred population in each
ethnic background that would represent 2% of that group for
each of the two time periods (i.e., the actual proportionate
number). For example, given the total number of Latino-His-
panics in the district, 902.5 would be expected to be in the top
2% in the district for the 1991-1993 time period. A total of
822.9 would be expected to be in the top 2% on the basis of the
total referred population. The actual proportionate number of
children in the top 2% for each ethnic background in the district
or referred population was used as the expected, whereas the
number of children in each background who actually scored in
the upper 2% (i.e., 98th percentile or better) was used as the
observed in a series of chi-square analyses. Table 4 summarizes
these analyses.
Most of the discrepancies were significant, especially for the
district population. Thus, for a completely purified sample, nei-
ther the SPM nor the WISC-R came close to proportionate rep-
resentation at the 98th percentile for the different ethnic
backgrounds.
To investigate the possibility of gender differences in SPM and
WISC-R performance, the number of boys and girls who actu-
ally scored in the 98th percentile and above on each test in each
ethnic group was compared with the expectation that there
would be equal numbers of boys and girls. As can be seen in
Table 5, there were no significant differences in performance for
boys and girls of any ethnic background on the SPM. For the
WISC-R, by contrast, significantly more White boys (2,630)
than girls (2,274) achieved an IQ score in the top 2% (Table 5).
Moreover, the trend was in favor of boys on the WISC-R for
every ethnic group except Pacific Islanders.
To examine performance of individuals at the highest level
of ability (i.e., 3 standard deviations above the mean), it was
necessary to generate local norms for the SPM, because the
norms provided in the manual only go up to the 99th percentile.
In constructing local norms for the 99.1 through 99.9 percen-
tile, we used the following procedure. On the basis of the table
of smoothed North American norms provided by Raven et al.
(1986), we selected all those children in our sample who had
190 DENNIS P. SACCUZZO AND NANCY E. JOHNSON

Table 2
Chi-Square Results With the Number Certified as the Observed and the Corresponding
Proportionate Number in the District or Referred Population as the Expected
District population Referred population
n
Ethnicity df (observed) (expected) x2 (expected)
SPM
Latino-Hispanic 1 938 1503.1 291.81*** 1370.5 181.46***
White 1 2771 2055.7 396.37*** 2232.5 217.94***
African American 1 463 895.2 249.01*** 845.5 204.28***
Asian 1 261 127.1 144.39*** 171.3 48.47***
Native American 1 32 33.2 0.04 27.6 0.70
Pacific Islander 1 37 38.7 0.07 38.7 0.07
Filipino 1 600 442.1 61.32*** 502.9 20.64***
Indochinese 1 424 425.5 0.01 342.6 20.61***

WISC-R
Latino-Hispanic 1 365 1459.3 1049.34*** 374.9 0.28
White 1 5314 2972.1 3318.79*** 5261.5 2.45
African American 1 371 1084.4 560.09** 441.8 12.15***
Asian 1 239 147.3 58.42*** 207.5 4.93*
Native American 1 18 26.8 2.89 20.1 0.22
Pacific Islander 1 10 33.5 16.54*** 6.7 1.63
Filipino 1 270 508.7 121.25*** 281.2 .46
Indochinese 1 107 482.0 314.36*** 100.4 .44

Note. SPM = Standard Raven Progressive Matrices Test; WISC-R = Wechsler Intelligence Scale for Chil-
dren—Revised.
*/?<.05. **p<.01. ***p<.001.

obtained a raw score in the 99th percentile for each age range in above the 99.87 percentile). A comparison of the proportion of
the table. We then examined the frequency distribution of raw students from the entire 1991-1993 sample, by ethnic back-
scores for each age range and attempted to break the scores ground, who obtained scores at the 99.9 percentile on the local
down into 10 groups occurring with equal frequency, represent- SPM norms versus the proportion, by ethnic background, who
ing the 99.0 through 99.9 percentile. obtained WISC-R Full Scale IQs of 145 or above during the
In the analyses that followed, the local norms (Guertin, John- 1984-1990 time period revealed significant diiferences in the
son, Saccuzzo, & Lopez, 1992) were used to identify students proportionate representation of children selected for the
who scored three standard deviations above the mean (i.e., schools' very gifted "Seminar" program for all ethnic back-
grounds except for the Whites, who represent about 35% of the
district. During the 1984-1990 period, Whites represented
Table 3 more than 80% of the children selected for the "Seminar" pro-
IQ Scores by Ethnic Background gram. During the 1991-1993 period, by contrast, Whites rep-
resented less than 60%.' It is important to emphasize that it is
WISC-R not possible to determine the extent to which this improved
(1984-1990) SPM"( 1991-1993) proportional representation is a function of any one component
SD
(i.e., the use of the SPM).
Ethnicity M SD M
As with the children who scored two standard deviations
Latino-Hispanic 129.23 11.84 109.30 15.00 above the mean, there was not proportionate representation for
White 132.65 10.93 120.10 13.35 children who scored three standard deviations above the mean
African American 125.83 11.60 107.65 15.15 for either test. A total of 508 children in our sample scored in
Asian 134.43 10.86 121.45 13.05 the 99.9th percentile on the SPM, as determined by local
Native American 131.17 8.35 118.60 13.80
Pacific Islander 128.08 12.06 115.15 13.20 norms. We compared the number who actually scored at that
Filipino 128.01 11.40 117.40 13.65 level with the number expected on the basis of district propor-
Indochinese 127.75 10.62 116.20 14.55 tion for each ethnic group. Results indicated that Latino-His-
panics and African Americans were significantly (p < .001) un-
Note. SPM = Standard Raven Progressive Matrices Test; WISC-R =
Wechsler Intelligence Scale for Children—Revised.
" SPM z scores were converted to deviation IQs (M = 100, SD = 15) for 1
ease of direct comparison across tests. A table of chi-squares is available on request from the authors.
 PROPORTIONATE REPRESENTATION 191

Table 4
Chi-Square Results With the Actual Number of Children in the Top 2% as the Observed
District population Referred population

Ethnicity df (observed) (expected) (expected)

SPM
Latino-Hispanic 1 379 902.5 417.11*** 822.9 318.39***
White 1 2009 1234.3 774.27*** 1340.5 559.42***
African American 1 209 537.5 239.60*** 507.7 207.44***
Asian 1 173 76.3 125.38*** 102.9 49.36***
Native American 1 21 19.9 0.06 16.6 1.18
Pacific Islander 1 12 23.2 5.46* 23.2 5.46*
Filipino 1 316 265.4 10.47** 301.9 0.72
Indochinese 1 199 255.5 13.53*** 205.7 0.23

WISC-R
Latino-Hispanic 265 1283.6 1033.62*** 329.7 13.46**
White 4904 2614.3 3606.97*** 4628.0 76.93***
African American 252 953.9 616.27*** 388.6 51.42***
Asian 202 129.5 41.45*** 182.5 2.14
Native American 13 23.6 4.75* 17.7 1.24
Pacific Islander 9 29.4 14.26*** 5.9 1.65
Filipino 182 447.5 170.47*** 247.3 18.00***
Indochinese 61 423.9 334.82*** 88.3 8.58**
Note. SPM = Standard Raven Progressive Matrices Test; WISC-R = Wechsler Intelligence Scale for Chil-
dren—Revised.
*p<.05. **p<.01. ***p<.001.

derrepresented, whereas Whites and Asians were significantly

overrepresented (p < .001; see Footnote 1).
Table 5 Figure 5 illustrates the gender composition for children who
Chi-Square Results Comparing the Number of Boys and Girls scored either at or above the 99.9 percentile on the SPM or had
of Each Ethnic Background Who Actually Scored in the Top a Full Scale WISC-R IQ of 145 or greater. As Figure 5 shows,
2% on the SPM or the WISC-R the substantial gender imbalance in favor of boys with the
WISC-R was essentially eliminated with the SPM.
Boys Predictive validity coefficients were determined for the SPM
Ethnicity c,if n (observed) x2
and WISC-R in a series of correlational analyses between the
SPM intelligence and achievement tests used by the school district.
Latino-Hispanic 374 181 0.19 Each child received a test of intelligence and an achievement
White 2002 1057 3.13 test during the same school year. Table 6 demonstrates corre-
African American 206 116 1.64 lations between performance on the SPM, as expressed in z
Asian 172 79 0.57 scores, and on the subtests of the CTBS for 1,707 children, as
Native American 21 12 0.21
Pacific Islander 12 6
well as correlations between CTBS and WISC-R for another
0
Filipino 315 170 0.99 1,925. As can be seen, the effect sizes were very similar for both
Indochinese 198 100 0.01 tests, especially when the SPM score was compared with the
WISC-R Full Scale IQ. Thus, both have approximately equal
WISC-R predictive validity for achievement test data. It should be noted
Latino-Hispanic 265 152 2.87 that given the restricted range of IQ scores, these correlations
White 4904 2630 13.21*** are somewhat lower than might be expected from a more gen-
African American 252 129 0.07 eral sample.
Asian 202 109 0.63
Native American 13 9 0.96 Correlations between the SPM and the ASAT for 10,023 chil-
Pacific Islander 9 3 0.50 dren of six ethnic backgrounds are summarized in Table 7. Cor-
Filipino 182 107 2.81 relation coefficients ranged from .22 to .31, median r = .25.
Indochinese 61 34 0.40 Inspection of these correlations reveals almost no differences in
Note. SPM = Standard Raven Progressive Matrices Test; WISC-R = the predictive validity across ethnic backgrounds (i.e., there was
Wechsler Intelligence Scale for Children—Revised. no differential validity as a function of ethnic background).
***p<.001. These findings are in accord with Sattler's (1988) review of 19
 192 DENNIS P. SACCUZZO AND NANCY E. JOHNSON

ential validity as a function of ethnic background. More impor-

tant, however, when test bias was examined in terms of a model
of proportionate representation, as may be mandated by case
law and federal legislation, neither test fared well. Use of the
SPM did lead to proportionate representation during the 1991 -
1993 period for Native Americans, Pacific Islanders, and In-
dochinese, all of whom had been substantially underrepre-
sented during the 1984-1990 period. It must be noted, however,
I that many more of these underrepresented children had been
referred for testing during the 1991 -1993 period when the SPM
(50%) was used. Compared with the referred sample, the WISC-R
proved quite accurate.
The success of the SPM with Indochinese during the 1991-
1993 time period is of interest in terms of evaluating non-En-
glish-speaking children. In the past the district had difficulties
evaluating giftedness in Indochinese children who spoke little
or no English. Because use of the SPM enabled evaluation of
ability independently of language, it was possible to assess these
children with the least bias heretofore achieved in terms of the
proportionate-representation model. With the SPM, Indo-
Chinese were selected almost exactly in proportion to their num-
WISC-R SPM bers in the district as a whole.
The SPM's ability to predict achievement, including lan-
Means of Certification guage achievement, probably is due to its high correlation with
Spearman's (1904, 1927a, 1927b)# factor (see Carpenter etal.,
Figure 5. Gender composition of the 99.9th percentile: WISC-R ver- 1990; Marshalek et al., 1983; Snow et al., 1984). Because tests
sus Standard Progressive Matrices. of language achievement are highly correlated with g, ^-satu-
rated tests such as the SPM share common variance with them.
Thus, the SPM can have clear advantages for measuring abilities
studies (Chapter 19, p. 567), which showed little difference in for individuals who speak a language other than English or are
the median validity coefficients between intelligence tests and
achievement for Anglo-American, African-American, and His-
panic-American children. Table 6
Correlation of SPM or WISC-R Performance With
Achievement Test (CTBS) Performance for
Discussion Children of Three Ethnic Backgrounds
The combined results showed that although it was possible to
CTBS subtest
approach proportionate representation in the referral process,
it was not possible to achieve such representation in the selec- Total Total Total
tion process for either the WISC-R (during the 1984-1990 time Ethnicity n language reading math
period) or the SPM (1991-1993 time period). Although there
was movement toward increased representation during the Whites
SPM 901 .19** .17** .22*
1991 -1993 period, it was not possible to tease out the extent to VIQ 1566 .16** .25** .18**
which the improved representation was a function of the SPM PIQ 1566 .11** .13** .17**
(compared with the WISC-R). The observed increases could FSIQ 1566 .16** .23** .21**
have been due to the change of tests but also may have been a African Americans
SPM 276 .24** .19** .13*
function of a changed referral process, a larger and more diverse VIQ 221 .07 .27** .14
sample, and possibly other factors that were not measured but PIQ 221 .02 .11 .18**
occurred over the course of time. Future research that directly FSIQ 221 .05 .24** .20**
compares these two tests on the same sample (in a counterbal- Latinos-Hispanics
SPM 530 .16** .08* .13**
anced design) can determine the specific effects of the test VIQ 138 .19* .14 .32**
change. Nevertheless, regardless of which test was used, signifi- PIQ 138 -.05 -.02 .17*
cant group mean differences were observed for the different eth- FSIQ 138 .08 .07 .27**
nic backgrounds.
The results did indicate that the SPM and the WISC-R have Note. SPM = Standard Raven Progressive Matrices Test; WISC-R =
Wechsler Intelligence Scale for Children—Revised; VIQ = Verbal IQ;
comparable validities for predicting academic achievement as PIQ = Performance IQ; FSIQ = Full Scale IQ; CTBS = Comprehensive
measured by standardized tests. Moreover, consistent with pre- Test of Basic Skills.
vious reports (Saltier, 1988), there was no evidence of differ- *p<.05. **p<.01.
 PROPORTIONATE REPRESENTATION 193

Table 7 Barrett, G. V., & Depinet, R. L. (1991). A reconsideration of testing for

Correlation ofSPM Performance With Achievement as competence rather than for intelligence. American Psychologist, 46,
Measured by the Abbreviated Stanford Achievement Test 1012-1024.
(ASAT)for Children of Six Ethnic Backgrounds Baska, L. (1986). The use of the Raven Advanced Progressive Matrices
for the selection of magnet junior high school students. Roeper Re-
ASAT subtest view, 8, 181-184.
Burciaga, L. E. (1973). A research study on the Raven Coloured Pro-
Total Total Total gressive Matrices among school children of the El Paso public schools.
Ethnicity n language reading math Doctoral dissertation, University of El Paso, Texas.
Carlson, J. S. (1989). Advances in research on intelligence: The dy-
Latino-Hispanic 2528 .25** .30** .31** namic assessment approach. The Mental Retardation and Learning
White 4020 .22** .23** .25**
African American 1581 .25** .26** .27** Disability Bulletin, 17, 1-20.
Asian 305 .24** .27** .26* Carlson, J. S., & Dillon, R. (1978). The effects of testing-the-limits pro-
Filipino 1002 .22** .24** .24** cedures on Raven matrices performance of deaf children. Volta Re-
Indochinese 587 .26** .25** .26* view, 4, 216-224.
Carlson, J. S., & Wiedl, K. H. (1978). Use of testing the limits proce-
Note. Standard Raven Progressive Matrices Test (SPM) performance dures in the assessment of intellectual capabilities in children with
was expressed in z scores. learning difficulties. American Journal of Mental Deficiency, 82, 599-
*p<.05. **p<.01. 564.
Carlson, J. S., & Wiedl, K. H. (1979). Towards a differential testing
from a different culture (Court, 1991). Moreover, because it approach: Testing-the-limits employing the Raven Progressive Matri-
does not depend on an explicit knowledge base, as does the ces. Intelligence, 3, 323-344.
WISC-R and other verbally weighted standardized tests, the Carpenter, P. A., Just, M. A., & Shell, P. (1990). What one intelligence
test measures: A theoretical account of the processing in the Raven
SPM appears to be better suited to traditionally underrepre- Progressive Matrices Test. Psychological Review, 97, 404-431.
sented children. Cherkes-Julkowski, M., Stolzenberg, J., & Segal, L. (1990). Prompted
Previous studies that have examined both the WISC-R and cognitive testing as a diagnostic compensation for attentional deficits:
the SPM (e.g., James, 1984; Kier, 1949; Meeker, 1973; Pearce, The Raven Standard Progressive Matrices and attention deficit disor-
1983; Tulkin & Newbrough, 1968) have been primarily con- der. Learning Disabilities, 2, 1-7.
cerned with the correlation between the two measures. Such Civil Rights Act of 1964,42 U.S.C. § 2000(e) (1988).
studies have reported correlations in the .70s and have been gen- Civil Rights Act of 1991, Pub. L. No. 102-166, 105 Stat. 1071(1991).
erally supportive of the SPM as a viable alternative to the Cleary, T. A. (1968). Test bias: Prediction of grades of Negro and white
WISC-R. Our findings, although not directly comparing the students in integrated colleges. Journal of Educational Measurement,
two tests, suggest that neither met the rigors of a proportionate- 5, 115-124.
representation model of test bias. Cole, N. S. (1973). Bias in selection. Journal of Educational Measure-
One of the reasons tests like the WISC-R and SPM continue ment, 10, 237-255.
to be used is that they are reliable and objective, and predict Court, J. H. (1988). A researcher's bibliography for Raven's Progressive
Matrices and Mill Hill Vocabulary Scales (7th ed.). Published by
meaningful criterion variables. To date, no other approach J. H. Court, 500 Goodwood Rd., Cumberland Park, South Australia
matches standardized tests in terms of reliability, objectivity, 5041.
and predictive validity. The question remains, however: Can Court, J. H. (1991). Asian applications of Raven's Progressive Matri-
traditional tests meet the rigors of a proportionate-representa- ces. Psychologia: An International Journal of Psychology in the Ori-
tion model of test bias while providing adequate predictive va- ent, 34 (2), 75-85.
lidity without differential validity? We believe that there remain Court, J. H., & Raven, J. (1977 & 1982). Summaries of reliability, va-
potentially promising options. One such option, suggested by lidity, and normative studies for Raven's Progressive Matrices and
Raven (1989), is the use of local ethnic norms for selection Vocabulary Scales. In A manualfor Raven's Progressive Matrices and
purposes. A second, suggested by Carlson (1989) and col- Mill Hill Vocabulary Scales (Research Suppl. No. 2). San Antonio,
leagues (Carlson & Dillon, 1978; Carlson & Wiedl, 1978), is TX: Psychological Corporation.
to use traditional tests in creative ways. For example, using a Darlington, R. B. (1971). Another look at "cultural fairness." Journal
of Educational Measurement, 8, 71-82.
dynamic testing approach with the SPM in which good test-
Darlington, R. B. (1978). Cultural test bias: Comment on Hunter and
taking strategies were incorporated into the test administration,
Schmidt. Psychological Bulletin, 85, 673-674.
Carlson and Wiedl (1979) were able to eliminate Hispanic- Diana v. State Board of Education, C-70 37 RFT (N. D. Cal. 1970).
White and Black-White differences in IQ. Before we abandon Dunnette, M. D., & Borman, W. C. (1979). Personnel selection and
what remains the most objective, reliable, and valid approach classification systems. Annual Review of Psychology, 30, 477-525.
to selection, it behooves us to determine whether innovative Education of All Handicapped Children's Act of 1975. Pub. L. No. 94-
uses of traditional psychometric tests can withstand the man- 142,20U.S.C.,§1412(5)(1990).
dates for proportionate representation without sacrificing Guadalupe v. Tempe Elementary School District, 587 F.2d 1022 (9th
validity. Cir. 1978).
Guertin, T, Johnson, N., Saccuzzo, D., & Lopez, C. (1992). Perfor-
References mance at the top: Raven norms for the very gifted. In D. P. Saccuzzo
Americans With Disabilities Act of 1990, 42 U.S.C. § 12101-12213 (Chair), Identifying underrepresented.disadvantaged gifted children:
(1992). Models and new approaches. Symposium conducted at the 72nd an-
194 DENNIS P. SACCUZZO AND NANCY E. JOHNSON
nual meeting of the Western Psychological Association, Portland,
OR.
Haier, R. }., Siegel, B. V., Nuechterlein, K. H., Hazlett, E., Wu, J. C,
Pack, J., Browning, H. L., & Buchsbaum, M. S. (1988). Cortical
glucose metabolic rate correlates of abstract reasoning and attention
studied with positron emission tomography. Intelligence, 12, 199-
217.
Hobson v. Hansen, 269 F. Supp. 401 (D.D.C. 1967).
Hoffman, H. V. (1983). Regression analysis of test bias in the Raven's
Progressive Matrices for Anglos and Mexican-Americans. Unpub-
lished doctoral dissertation, University of Arizona, Department of
Educational Psychology.
Hoffman, H. V. (1986). In J. Raven, J. H. Court, & J. Raven, Manual
for Raven's Progressive Matrices and Vocabulary tests: A compen-
dium of North American normative and validity studies, Research
Suppl. No. 3. London: H. K.. Lewis.
Hunter, J., & Schmidt, F. (1976). Critical analysis of the statistical and
ethical implications of various definitions of test bias. Psychological
Bulletin, 83, 1053-1071.
James, P. R. (1984). A correlational analysis between the Raven's Ma-
trices and WISC-R performance scales. The Volta Review, 86, 336-
341.
Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.
Kaplan, R., &Saccuzzo, D. P. (1989). Psychological testing: Principles,
applications, and issues (2nd ed.). Pacific Grove, CA: Brooks/Cole.
Kaplan, R., & Saccuzzo, D. P. (1993). Psychological testing: Principles,
applications, and issues (3rd ed.). Pacific Grove, CA: Brooks/Cole.
Kier, G. (1949). The Progressive Matrices as applied to school children.
British Journal of Statistical Psychology, 2, 140-150.
Larry P. v. Riles, 343 F. Supp. 1306 (N. D. Cal. 1972) (order granting
injunction), aff'd 502 F. 2nd 963 (9th Cir. 1974), 495 F. Suppl. 9
(N. D. Cal. 1979) (decision on the merits), aff'd (9th Cir. Jan 23,
1984).
Lewandowski, D. G., & Saccuzzo, D. P. (1976). The decline of psycho-
logical testing: Have traditional assessment procedures been fairly
evaluated? Professional Psychology, 8, 177-184.
Marshalek, B., Lohman, D. F, & Snow, R. E. (1983). The complexity
continuum in the radex and hierarchical models of intelligence. In-
telligence. 7, 107-127.
McGraw-Hill. (1981). Comprehensive Tests of Basic Skills, Forms U
and V. Monterey, CA: Author.
Meeker, M. (1973). Patterns ofgiftedness in Black, Anglo, and Chicano
Boys Ages 4-5 and 7-9. Paper presented to the First National Con-
ference for Disadvantaged Gifted, Ventura County, CA.
Pearce, N. (1983). A comparison of the WISC-R, Raven's Standard
Progressive Matrices, and Meeker's SOI-Screening Form for Gifted.
Gifted Child Quarterly, 27, 13-19.
Powers, S., & Barkan, J. H. (1986). Concurrent validity of the Standard
Progressive Matrices for hispanic and nonhispanic seventh-grade stu-
dents. Psychology in the Schools, 23, 333-336.
Powers, S., Barkan, J. H., & Jones, P. B. (1986). Reliability of the Stan-
dard Progressive Matrices Test for Hispanic and Anglo-American
children. Perceptual and Motor Skills, 62, 348-350.
Psychological Corporation. (1988). Stanford Achievement Test (8th
ed.). San Antonio, TX: Author.
Raven, J. (1989). The Raven Progressive Matrices: A review of national
norming studies and ethnic and socioeconomic variation within the
United States. Journal of Educational Measurement, 26, 1-16.
Raven, J. C. (1938). Progressive matrices. London: Lewis.
Raven, J., Summers, W. A., et al. (1986). A compendium of North
American normative and validity studies. In J. C. Raven, J. H. Court,
and J. Raven, A manual for Raven's Progressive Matrices and vocab-
ulary tests (Research Suppl. No. 3). San Antonio, TX: Psychological
Corporation.
Raven, J. C. (1958). Standard progressive matrices. San Antonio, TX:
Psychological Corporation.
Raven, J. C. (1960). Guide to the progressive matrices. London: H. K.
Lewis.
Rehabilitation Act of 1973, 29 U.S.C. § 794 (1973).
Reschly, D. J., & Sabers, D. L. (1979). Analysis of test bias in four
groups with the regression definition. Journal of Educational Mea-
surement, 16, 1-9.
Saccuzzo, D. P., Johnson, N. E., & Guertin, T. L. (1994). Identifying
underrepresented disadvantaged gifted and talented children: A mul-
tifaceted approach (Vols. 1 and 2). San Diego, CA: San Diego State
University. (ERIC Document Reproduction Service No. ED 368
095).
Saccuzzo, D. P., Johnson, N. E., & Russell, G. (1992). Verbal versus
performance IQs for gifted African-American, Caucasian, Filipino,
and Hispanic children. Psychological Assessment, 4, 239-244.
Saltier, J. M. (1988). Assessment of children (3rd ed.). San Diego, CA:
Author.
Sidles, C., & MacAvoy, J. (1987). Navajo adolescents' scores on a pri-
mary language questionnaire, the Raven Standard Progressive Matri-
ces (RSPM), and the Comprehensive Test of Basic Skills (CTBS): A
correlational study. Educational and Psychological Measurement, 47,
703-709.
Snow, R. E., Kyllonen, P. C., & Marshalek, B. (1984). The topography
of ability and learning correlations. In R. J. Sternberg (Ed.), Advances
in the psychology of human intelligence (Vol. 2, pp. 47-103). Hills-
dale, NJ: Erlbaum.
Spearman, C. (1904). "General intelligence" objectively determined
and measured. American Journal of Psychology, 15, 210-293.
Spearman, C. (1927a). The abilities of man. New York: MacMillan.
Spearman, C. (1927b). The nature of "intelligence" and the principles
of cognition (2nd ed.). London: MacMillan.
Thorndike, R. L. (1971). Concepts of culture-fairness. Journal of Edu-
cational Measurement, 8, 63-70.
Tulkin, S. R., & Newbrough, J. R. (1968). Social class, race, and sex
differences on the Raven (1956) Standard Progressive Matrices.
Journal of Consulting and Clinical Psychology, 15, 201-293.
Watson v. Fort Worth Bank and Trust, 487 U.S. 977, 47 FEP 102
(1988).
Wechsler, D. (1974). Manual for the Wechsler Intelligence Scale for
Children-Revised. New York: Psychological Corporation.
Received June 23, 1994
Revision received December 7, 1994
Accepted December 13, 1994 •