This article was downloaded by: [University of Arizona]

On: 23 August 2013, At: 08:47

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer
House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Clinical and Experimental

Neuropsychology
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/ncen20

The Boston Qualitative Scoring System as a Measure

of Executive Functioning in Rey-Osterrieth Complex
Figure Performance
Jessica Somerville , Geoffrey Tremont & Robert A. Stern
Published online: 09 Aug 2010.

To cite this article: Jessica Somerville , Geoffrey Tremont & Robert A. Stern (2000) The Boston Qualitative Scoring
System as a Measure of Executive Functioning in Rey-Osterrieth Complex Figure Performance, Journal of Clinical and
Experimental Neuropsychology, 22:5, 613-621

To link to this article: http://dx.doi.org/10.1076/1380-3395(200010)22:5;1-9;FT613

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall
not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other
liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Journal of Clinical and Experimental Neuropsychology 1380-3395/00/2205-613$15.00
2000, Vol. 22, No. 5, pp. 613-621 © Swets & Zeitlinger

The Boston Qualitative Scoring System as a Measure of

Executive Functioning in Rey-Osterrieth Complex Figure
Performance*
Jessica Somerville1, Geoffrey Tremont2, and Robert A. Stern3
1
University of Rhode Island, 2Department of Psychiatry and Human Behavior, Brown University School of
Medicine, 3Department of Psychiatry and Human Behavior and Department of Clinical Neurosciences, Brown
University School of Medicine
Downloaded by [University of Arizona] at 08:47 23 August 2013

ABSTRACT

The Boston Qualitative Scoring System (BQSS) for the Rey-Osterrieth Complex Figure (ROCF) includes
five scores (Planning, Fragmentation, Neatness, Perseveration, and Organization) developed to measure the
executive aspects of ROCF productions. To assess the convergent and discriminant validity of the BQSS,
these five scores were compared to scores on four traditional tests of executive functioning, as well as to
three non-executive measures, in 141 adult patients. BQSS executive variables significantly correlated with
the traditional executive measures and were less correlated with discriminant measures. The BQSS Organi-
zation summary score also significantly differentiated patients with either no, mild, or severe executive
dysfunction. These results support the construct validity of the BQSS and demonstrate that the ROCF may
be a useful measure of executive functioning.

The Rey-Osterrieth Complex Figure (ROCF)
(Osterrieth, 1944; Rey, 1941) has become one of
the most widely used neuropsychological tasks
in both clinical and research settings (Knight &
Kaplan, in press). The most commonly em-
ployed scoring methods for the ROCF, including
the original 36-point scoring system (Osterrieth,
1944) and the more recent system by Meyers
and Meyers (1995), provide a single, quantita-
tive summary score based on the presence and
accuracy of 18 units of the figure. The construc-
tion strategy, planning, and organizational ap-
proach used to draw the figure are not assessed
with this single-score method. However, this
more qualitative information may be valuable in
identifying executive dysfunction, as well as
understanding the reason for poor recall. For
example, Grossman and colleagues (1993) found
that patients with Parkinson’s disease used a
more disorganized approach when copying the
ROCF compared to controls. Similarly, poor
organizational strategies have been found in pa-
tients with obsessive-compulsive disorder (Sav-
age et al., 1999), and, in patients with frontal
lesions, ROCF recall was improved when orga-
nizational strategies were provided (Diamond,
DeLuca, & Kelley, 1997; Eslinger & Gratten,
1990).
Several qualitative approaches to scoring the
ROCF have been proposed in recent years (e.g.,
Bennett-Levy, 1984; Bernstein & Waber, 1996;
Binder, 1982; Chervinsky, Mitrushina, & Satz,
1992; Chiulli, Halland, LaRue, & Garry, 1995;
Hamby, Wilkins, & Barry, 1993; Loring, Lee, &
Meador, 1988; Rapport, Farchione, Dutra, Web-
ster, & Charter, 1996; Shorr, Delis, & Massman,

*
The authors would like to acknowledge the assistance of Sarah M. Halpert.
Address correspondence to: Dr. Robert Stern, Neuropsychology Program, Rhode Island Hospital, 110 Lockwood
Street, Suite 430, Providence, RI 02903. Tel.: ++1 401 444-4500. Fax: ++1 401 444-6643. E-mail: rstern@life-
span.org
Accepted for publication: April 7, 2000.
 614 JESSICA SOMERVILLE ET AL.
1992; Visser, 1973). However, these systems
focus on only one or two qualitative features of
performance, such as organization or symmetry,
or are not appropriate for use with adults. Many
of these qualitative scoring systems have dem-
onstrated the importance of executive function-
ing in ROCF performance (Troyer & Wishart,
1997), though they do not provide a comprehen-
sive system for the evaluation of executive skills
contributing to the ROCF.
To date, the only commercially available
qualitative scoring system available for adults is
the Boston Qualitative Scoring System (BQSS)
Downloaded by [University of Arizona] at 08:47 23 August 2013
(Stern et al., 1994; Stern et al., 1999). This sys-
tem provides a comprehensive set of qualitative
ratings, as well as salient quantitative summary
scores, with extensive normative data. In addi-
tion to measuring other qualitative features of
visuoconstructive skills (e.g., Accuracy, Place-
ment, Rotation, and Asymmetry), four of the 17
scores (Planning, Fragmentation, Neatness, and
Perseveration), as well as the Organization sum-
mary score, were developed to be sensitive to
executive dysfunction. These scores quantify the
approach used by the examinee when reproduc-
ing the figure. A poor organizational strategy
may result in a very fragmented approach, poor
placement of the figure on the page or elements
within the figure, compromised integrity of the
figure, and a haphazard order in which the ele-
ments are drawn. The ROCF can also include
repetitions of elements within the figure, reflect-
ing perseverative tendencies (Stern & Prohaska,
1996).
Research on the BQSS has provided evidence
for its reliability and validity (Stern et al., 1994;
Stern et al., 1999). Preliminary support for the
validity of the BQSS executive scores has also
been established in studies of patient popula-
tions with known executive dysfunction. For
example, Mahurin, Eckert, Velligan, Hazelton,
and Miller (1997) found that specific BQSS ex-
ecutive scores were significantly associated with
measures of functional abilities and cognitive
executive skills among patients with schizophre-
nia. A recent study by Schreiber, Javorsky, Rob-
inson, and Stern (2000) found that BQSS execu-
tive scores (i.e., Neatness, Planning, and Perse-
veration) successfully discriminated between
adult patients with Attention Deficit Hyperactiv-
ity Disorder (ADHD) and matched controls. Ad-
ditionally, the executive scores from the BQSS
have been found to be useful in discriminating
children with ADHD from controls (Cahn et al.,
1996). BQSS executive scores have also differ-
entiated controls from other groups with pre-
sumed executive dysfunction, such as patients
with traumatic brain injury (Javorsky, Rosen-
baum, & Stern, 1999) and recently detoxified
alcoholics (Dawson & Grant, 2000).
To examine convergent and discriminant va-
lidity, the present study investigated the hypoth-
esis that BQSS scores designed to measure exec-
utive functioning are more highly correlated
with traditional measures of executive function-
ing than with measures that are purportedly less
sensitive to executive functioning. In addition,
this study evaluated the usefulness of the BQSS
Organization summary score in differentiating
amongst groups of patients with differing levels
of executive dysfunction, as determined by stan-
dard measures of executive functioning.
METHOD
Participants
The sample consisted of 141 outpatients (66
women and 75 men) referred for neuropsychologi-
cal evaluation at the neuropsychology service of a
large urban university-based medical center. The
mean age of the sample was 44.1 years (SD = 14.2,
range = 18-81), the mean number of years of edu-
cation was 13.7 (SD = 3.0, range = 6-21), and 94%
of the sample was Caucasian. Patients represented
a variety of diagnostic categories, including: Trau-
matic brain injury (25%), psychiatric illness
(12%), stroke/cerebrovascular disease (10%), de-
mentia (9%), seizure disorder (6%), chronic fa-
tigue syndrome/fibromyalgia (5%), toxic/meta-
bolic disorder (5%), brain neoplasm (4%), devel-
opmental disorder (3%), and a variety of other
neurological/medical conditions (21%).
Procedures
Demographic and medical information was ob-
tained from medical records, and when possible,
directly confirmed by the patient and/or family
member(s). As part of a comprehensive neuropsy-
chological evaluation, patients were examined us-
ing the standardized procedures for the following
 BQSS AS A MEASURE OF EXECUTIVE FUNCTIONING 615

executive tasks: Trail Making Test (TMT; Reitan
& Wolfson, 1985), Wisconsin Card Sorting Test
(WCST; Heaton, 1981), Controlled Oral Word As-
sociation Test (COWA; Benton, 1968), and the
Similarities subtest from the WAIS-R (Wechsler,
1981). Executive functioning tasks were selected
based on their common use in neuropsychological
assessment (Lezak, 1995) and their sensitivity for
detecting executive impairment (Stern & Prohaska,
1996). One primary score was selected from each
task for use in this specific study. These included:
TMT Part B Total Time, WCST Perseverative Re-
sponses, COWA Total Words, and Similarities
Raw Score.
In order to examine the discriminant validity of
Downloaded by [University of Arizona] at 08:47 23 August 2013
Finally, the Digit Span Forward raw score from the
Digit Span Subtest of the WMS-R (Wechsler,
1987) was also used as a discriminant validity
measure.
Participants were administered the Rey-Oster-
rieth Complex Figure (ROCF) according to the
instructions in the BQSS test manual, using col-
ored markers to track the order in which the pro-
duction was drawn (Stern et al., 1999). Although
each participant received copy, immediate, and
delayed recall conditions, only the copy condition
was used for the purpose of this study because of
the focus on executive aspects of visuoconstruc-
tion, and not on memory. All productions were
scored by two trained research assistants using the

the BQSS executive scores, as well as to assure
that we were not merely examining generalized
severity of impairment, three additional variables
were selected. The first was a retention score from
the Logical Memory Subtest of the Wechsler Me-
mory Scale Revised (WMS-R; Wechsler, 1987).
Because organization can facilitate learning, the
percent of information retained from immediate to
delayed recall (retention score) was used instead of
the Logical Memory free recall scores. In addition,
the retention score (and not free recall score) from
the Visual Reproduction Subtest of the WMS-R
(Wechsler, 1987) was used for this same reason.
comprehensive scoring criteria provided in the
BQSS manual. The rater was blind to the scores on
the other neuropsychological measures. Interrater
reliability in our lab has been established and is
generally excellent; for the scores examined in this
study, the mean " was .81. The executive variables
of interest for this study were the Planning, Frag-
mentation, Neatness, and Perseveration qualitative
scores, and the Organization summary score (see
Table 1 for descriptions). The means, standard de-
viations, and ranges for the measures used in this
study are presented in Table 2.

Table 1. Description of the Boston Qualitative Scoring System (BQSS) Executive Functioning Scores.

Score Description

Fragmentation Measures integration of information (i.e., whether or not the individual ele-
ments are drawn as whole units). Range of scores: 0 (extreme fragmentation)
to 4 (no fragmentation).
Planning Measures overall planning ability based on the order in which elements are
drawn, placement on the page, placement within the figure, and overall integ-
rity of the production. Range of scores: 0 (extremely poor planning) to 4 (good
planning).
Neatness Rates how neatly the figure was drawn as evidenced by the number of wavy
lines, gaps and overshoots, cross-outs, rounded corners, etc. Range of scores:
0 (extremely messy) to 4 (very neat).
Perseveration Measures the extent of recognizably inappropriate repetition. May take one of
two forms: Repetition of components within a cluster or replication of an ele-
ment of the figure (Configural, Cluster, or Detail). Range of scores: 0 (extreme
perseveration) to 4 (no perseveration).
Organization Summary Score Arithmetic sum of the copy condition Fragmentation and Planning scores, pro-
viding a more omnibus measure of organizational skills. Range of scores: 0
(extremely poor organization) to 8 (good organization).

Note. Reproduced by special permission of the Publisher, Psychological Assessment Resources, Inc., 16204
North Florida Avenue, Lutz, Florida, 33549, from the Boston Qualitative Scoring System for the Rey-Osterrieth
Complex Figure Professional Manual, by Robert A. Stern, Ph.D., et al, Copyright 1994, 1996, 1998 by PAR, Inc.
Further reproduction is prohibited without permission of PAR, Inc.
 616 JESSICA SOMERVILLE ET AL.

Table 2. Descriptive Data for the BQSS Executive Scores and Other Neuropsychological Measures Used in the
Present Study.

N M (SD) Range
BQSS Scores
Fragmentation 141 2.75 (0.96) 1-4
Planning 141 2.26 (1.02) 0-4
Neatness 141 2.00 (0.91) 0-4
Perseveration 141 3.20 (1.21) 0-4
Organization 141 5.01 (1.70) 1-8
Other Measures
WCST-PR 141 21.12 (19.85) 3-114
TMT-B 141 101.91 (72.30) 34-360
COWA 141 35.00 (12.76) 5-83
Similarities 141 18.74 (4.88) 1-28
Downloaded by [University of Arizona] at 08:47 23 August 2013

Digits Forward 128 7.66 (2.19) 2-12

LM Retention 130 76.36 (24.63) 0-125
VR Retention 132 73.53 (28.66) 0-117

Note. BQSS = Boston Qualitative Scoring System. The BQSS Organization summary score ranges from 0 (very
poor) to 8 (very good). WCST-PR = Wisconsin Card Sorting Task-Perseverative Responses; TMT-B = Trail
Making Test-Part B; COWA = Controlled Oral Word Association-Total Words; Similarities = Similarities subtest
of the WAIS-R; Digits Forward = Digit Span subtest of the WMS-R, forward score; LM Retention = Logical
Memory Retention (Percent Retention from Immediate to Delayed Recall on the WMS-R Logical Memory
Subtest [Logical Memory II/Logical Memory I]); VR Retention = Visual Reproduction Retention (Percent Reten-
tion from Immediate to Delayed Recall on the WMS-R Visual Reproduction Memory Subtest [Logical Memory
II/Logical Memory I]).

RESULTS tween the BQSS variables and the executive

To investigate the convergent validity of the
BQSS scores, Pearson correlation coefficients
were derived between the five BQSS scores and
the four executive test scores. Discriminant va-
lidity was assessed by comparing the BQSS ex-
ecutive scores with the three variables presumed
not to be direct measures of executive function-
ing. In addition, the intracorrelations among the
BQSS scores and among the executive tests
were calculated.
As depicted in Table 3, all five BQSS mea-
sures were significantly correlated with WCST
Perseverative Responses and with COWA Total
Words. All BQSS scores, except the Fragmenta-
tion score, were also significantly correlated
with Trails B Time and with Similarities Raw
Score. In addition, the strengths of the intra-
correlations among the executive tests (mean
correlation = .37) were similar to the intra-
correlations among the BQSS variables (mean
correlation = .36), as well as those found be-
tests (mean correlation = .26) (note that the Or-
ganization score was excluded from the calcu-
lated mean correlations due to redundancy).
Table 4 presents correlations between the ex-
ecutive BQSS variables and the discriminant
validity measures. The executive BQSS vari-
ables were less correlated with the discriminant
validity measures (average correlation = .16)
than with the executive measures (average cor-
relation = .26). In fact, the executive BQSS
measures were less correlated with the discri-
minant measures than the other executive mea-
sures were with the same discriminant measures
(average correlation = .24).
To examine the clinical utility of the BQSS
for identifying patients with executive dysfunc-
tion, participants were placed into one of three
groups based on their performance on the execu-
tive measures used in the convergent validity
analyses. The three groups were determined as
follows: (1) No Executive Impairment (no im-
paired executive tests; n = 45), (2) Mild Execu-
 Downloaded by [University of Arizona] at 08:47 23 August 2013

Table 3. Correlations Between BQSS Scores and Executive Tests and Intracorrelations Among BQSS Variables and Executive Tests.

BQSS Executive Tests

Fragment Planning Neatness Persev Organiz1 WCST-PR TMT-B COWA Similar

BQSS Fragment –– .46*** .20* .34*** .85*** –.19* –.16 .31*** .16
Planning –– .24** .54*** .86*** –.30*** –.18* .44*** .25**
Neatness –– .38*** .23** –.26** –.29*** .26** .20*
Persev –– .52*** –.31*** –.21* .39*** .17*
Organiz –– –.28*** –.20* .44*** .24**
Executive Tests WCST-PR –– .19* –.46*** –.44***
TMT-B –– –.30*** –.37***
COWA –– –.44***
Similar ––

Note. BQSS = Boston Qualitative Scoring System. Fragment = Fragmentation; Persev = Perseveration; Organiz = Organization; WCST-PR = Wisconsin Card
Sorting Test-Perseverative Responses; TMT-B = Trail Making Test-Part B-Time; COWA = Controlled Oral Word Association-Total Words; Similar = Similarities
subtest of the WAIS-R.
1
The Organization score is the arithmetic sum of the Fragmentation score and the Planning score, thus accounting for the high correlations between Organization
and Fragmentation and Planning.
* p < .05; ** p < .01; *** p < .001.
BQSS AS A MEASURE OF EXECUTIVE FUNCTIONING
617
 618 JESSICA SOMERVILLE ET AL.

Table 4. Correlations Between Discriminant Variables and BQSS Executive Variables and Between Discriminant
Variables and Other Executive Functioning Variables.

Digit Span Forward LM Retention VR Retention

BQSS Fragmentation .10 .05 .09
Planning .27** .19 .26**
Neatness .18* .14 .17
Perseveration .16 .21* .15
Organization .22* .15 .21*
Executive Measures WCST-PR –.22* –.25** –.33***
TMT-B –.19* –.08 –.23**
COWA .42*** .22* .25**
Similarities .25** .20* .23**

Note. BQSS = Boston Qualitative Scoring System; LM Retention = Logical Memory Retention (Percent Reten-
Downloaded by [University of Arizona] at 08:47 23 August 2013

tion from Immediate to Delayed Recall on the WMS-R Logical Memory Subtest [Logical Memory II/Logical
Memory I]). VR Retention = Visual Reproduction Retention (Percent Retention from Immediate to Delayed
Recall on the WMS-R Visual Reproduction Subtest (Visual Reproduction II/Visual Reproduction I). WCST-PR
= Wisconsin Card Sorting Test-Perseverative Responses. TMT-B = Trail Making Test-Part B-Time. COWA =
Controlled Oral Word Association-Total Words. Similarities = Similarities subtest of the WAIS-R.
* p < .05; ** p < .01; *** p < .001.

tive Impairment (1-2 impaired executive tests; n
= 73), and (3) Significant Executive Impairment
(3-4 impaired executive tests; n = 23). A score
was considered impaired if it fell one standard
deviation or more below the published age
and/or education normative group. A one-way
Analysis of Variance (ANOVA) for the BQSS
Organization summary score, using the three
groups of executive impairment as the indepen-
dent variable, was statistically significant,
F(2,138) = 11.0, p < .001. Figure 1 graphically
presents the results for each group. Post-hoc
analyses, using the Tukey HSD, revealed that
patients in the No Executive Impairment group
performed significantly better on the Organiza-
tion score (M = 5.8, SD = 1.6) than patients in
the Mild Executive Impairment group (M = 4.8,
SD = 1.6), who in turn, performed significantly
better than patients in the Significant Executive
Impairment group (M = 3.9, SD = 1.6).
DISCUSSION
The present study found that BQSS scores de-
veloped to be sensitive to executive functioning
were significantly related to performance on
traditional neuropsychological tests measuring
various aspects of executive functioning, includ-
ing the WCST, Trail Making Test Part B, and
the WAIS-R Similarities subtest. The BQSS ex-
ecutive scores were generally unrelated to mea-
sures of verbal and visual memory retention and
basic attentional skills, as measured by Logical
Memory Retention, Visual Reproduction Reten-
tion, and the Digit Span Forward score, respec-
tively. These results provide preliminary support
for the convergent and discriminant validity of
the BQSS as a measure of the executive aspects
of ROCF performance.
The current results also demonstrate that the
BQSS Organization summary score may be a
clinically useful measure of executive dysfunc-
tion. Patients with no executive impairment had
significantly better organization (i.e., good plan-
ning and less fragmentation) on the ROCF than
patients with mild executive dysfunction, who in
turn, performed better than patients with severe
executive dysfunction, as measured by perfor-
mance on commonly used measures of executive
functioning. To examine whether this finding
was merely a function of overall cognitive im-
pairment (i.e., not specifically executive impair-
ment), we performed the same analysis using the
discriminant measures chosen for this study.
However, none of these measures (i.e., Digit
 BQSS AS A MEASURE OF EXECUTIVE FUNCTIONING 619

6.0

Mean BQSS Organization Score

5.0

4.0

3.0

2.0

1.0
Downloaded by [University of Arizona] at 08:47 23 August 2013

0 1-2 3-4

Number of Impaired Executive Tests

Fig. 1. Mean Boston Qualitative Scoring System Organization Score (+SE) for the No Executive Impairment
group (no impaired executive tests; n = 45), the Mild Executive Impairment group (1-2 impaired execu-
tive tests; n = 73), and the Significant Executive Impairment group (3-4 impaired executive tests; n =
23). All group comparisons are significant (p < .05).

Span Forward, Logical Memory Retention, Vi-
sual Reproduction Retention) resulted in signifi-
cant between-group differences, separating all
three groups (i.e., no, mild, and severe impair-
ment).
These results, along with other previous re-
search, empirically demonstrate that the ROCF
can yield information about multiple aspects of
individuals’ neuropsychological functioning.
Given that the administration of the ROCF (us-
ing BQSS scoring) can yield independent infor-
mation about visuoconstruction, visuospatial
memory, and executive functioning, it appears
to be a valuable component of any neuropsycho-
logical evaluation, particularly with the time
limitations imposed by the current healthcare
fiscal environment. These findings also high-
light the importance of examining qualitative
features of the ROCF and suggest that systems
using only one quantitative visuoconstruction
summary score may miss valuable information,
such as the organizational strategy employed by
the patient. The current findings are consistent
with previous research indicating that ROCF
organizational strategies could differentiate pa-
tients from healthy controls (Diamond et al.,
1997; Grossman et al., 1993; Savage et al.,
1999). The present results also confirm a previ-
ous examination of ROCF qualitative scoring
systems (Troyer & Wishart, 1997) which dem-
onstrated the importance of assessing executive
functioning with the ROCF.
Although the relationships between the BQSS
scores and the executive measures were signifi-
cantly related, the correlations were not as
strong as typically desired for validity coeffi-
cients and did not account for a large percentage
of shared variance. However, the correlations
among the traditional tests of executive func-
tioning were of similar magnitude, a finding also
seen in other correlational studies of executive
functioning (Boone, Ponton, Gorsuch, Gonzalez,
& Miller, 1998; Franzen & Petrick, 1995;
Humes, Welsh, Retzlaff, & Cookson, 1997; Per-
riene, 1993; Troyer & Wishart, 1997).
In conclusion, results of the present study
provide preliminary evidence for the construct
validity of the BQSS executive scores. Previous
 620 JESSICA SOMERVILLE ET AL.
research with the BQSS has also indicated the
utility of the BQSS as a measure of executive
functioning by its ability to predict group mem-
bership in patients with a variety of disorders
associated with executive impairment, including
ADHD (Cahn et al., 1996; Schreiber et al.,
2000), schizophrenia (Mahurin et al., 1997), al-
cohol abuse (Dawson & Grant, 2000), and trau-
matic brain injury (Javorsky et al., 1999). Addi-
tional research, however, is needed. For exam-
ple, it would be particularly interesting to exam-
ine patients with focal frontal lesions compared
to those with nonfrontal lesions. Additionally,
Downloaded by [University of Arizona] at 08:47 23 August 2013
performance patterns of the various BQSS exec-
utive variables could be examined with various
patient groups. That is, it is possible that differ-
ent patient populations may exhibit specific pat-
terns of executive dysfunction (e.g., high perse-
veration and poor organization, or extreme mes-
siness and fragmentation, etc.). Other future re-
search could include the relationship between
BQSS executive variables and other variables of
ROCF performance (i.e., visuoconstruction or
memory) and how they interact to result in the
final production, and in subsequent recall. Addi-
tionally, the validity of each specific executive
BQSS variable should be examined in future
research. Overall, based on the current findings,
the ROCF appears to be a useful clinical tool in
the assessment of executive dysfunction, though
further studies are recommended to enhance the
qualitative interpretation of specific executive
deficits.
REFERENCES
Bennett-Levy, J. (1984). Determinants of performance
on the Rey-Osterrieth Complex Figure Test: An
analysis, and a new technique for single-case as-
sessment. British Journal of Clinical Psychology,
23, 109-119.
Benton, A.L. (1968). Differential behavioral effects in
frontal lobe disease. Neuropsychologia, 6, 53-60.
Bernstein, J.H. & Waber, D.P. (1996). Developmental
Scoring System for the Rey-Osterrieth Complex
Figure. Odessa, FL: Psychological Assessment
Resources.
Binder, L.M. (1982). Constructional strategies on
Complex Figure drawings after unilateral brain
damage. Journal of Clinical Neuropsychology, 4,
51-58.
Boone, K.B., Ponton, M.O., Gorsuch, R.L., Gonzalez,
J.J., & Miller, B.L. (1998). Factor analysis of four
measures of prefrontal lobe functioning. Archives
of Clinical Neuropsychology, 13, 585-595.
Cahn, D.A., Marcotte, A.C., Stern, R.A., Arruda, J.A.,
Akshoomoff, N.A., & Leshko, I.C. (1996). The
Boston qualitative scoring system for the Rey-
Osterrieth Complex Figure: A study of children
with attention deficit hyperactivity disorder. The
Clinical Neuropsychologist, 10, 397-406.
Chervinsky, A.B., Mitrushina, M., & Satz, P. (1992).
Comparison of four methods of scoring the Rey-
Osterrieth Complex Figure Drawing Test on four
age groups of normal elderly. Brain Dysfunction,
5, 267-287.
Chiulli, S.J., Halland, K.Y., LaRue, A., & Garry, P.J.
(1995). Impact of age on drawing the Rey-Oster-
rieth figure. The Clinical Neuropsychologist, 9,
219-224.
Dawson, L.K., & Grant, I. (2000). Alcoholics’ initial
organizational and problem-solving skills predict
learning and memory performance on the Rey-
Osterrieth Complex Figure. Journal of the Interna-
tional Neuropsychological Society, 6, 12-19.
Diamond, B.J., DeLuca, J., & Kelley, S.M. (1997).
Memory and executive functions in amnesic and
non-amnesic patients with aneurysms of the ante-
rior communicating artery. Brain, 120, 1015-1025.
Eslinger, P.J., & Grattan, L.M. (1990). Influence of
organizational strategy on neuropsychological per-
formance in frontal lobe patients. Journal of Clini-
cal and Experimental Neuropsychology, 12, 54.
Franzen, M.D. & Petrick, J.D. (1995). Preliminary
norms for Design Fluency. Paper presented to the
103rd meeting of the American Psychological As-
sociation, NY.
Grossman, M., Carvell, S., Peltzer, L., Stern, M.B.,
Gollump, S., & Hurtig, H.I. (1993). Visual con-
struction of impairments in Parkinson’s disease.
Neuropsychology, 7, 536-547.
Hamby, S.L., Wilkins, J.W., & Barry, N.S. (1993).
Organizational quality on the Rey-Osterrieth and
Taylor Complex Figure tests: A new scoring sys-
tem. Psychological Assessment, 5, 27-33.
Heaton, R.K. (1981). Wisconsin Card Sorting Test
Manual. Odessa, FL: Psychological Assessment
Resources.
Humes, G.E., Welsh, M.C., Retzlaff, P. & Cookson,
N. (1997). Towers of Hanoi and London: Reliabil-
ity of two executive function tasks. Assessment, 4,
249-257.
 BQSS AS A MEASURE OF EXECUTIVE FUNCTIONING
Javorsky, D.J., Rosenbaum, J., & Stern, R.A. (1999).
Utility of the Boston Qualitative Scoring System
(BQSS) for the Rey-Osterrieth Complex Figure in
the evaluation of traumatic brain injury [abstract].
Archives of Clinical Neuropsychology, 14, 789-
790.
Knight, J. & Kaplan, E. (in press). Rey-Osterrieth
Complex Figure Handbook. Odessa, FL: Psycho-
logical Assessment Resources, Inc.
Lezak, M.D. (1995). Neuropsychological assessment
(3rd ed.). New York: Oxford University Press.
Loring, D.W., Lee, G.P., & Meador, K.J. (1988). Re-
vising the Rey-Osterrieth: Rating right hemisphere
recall. Archives of Clinical Neuropsychology, 3,
239-247.
Mahurin, R., Eckert, S., Velligan, D., Hazelton, B., &
Downloaded by [University of Arizona] at 08:47 23 August 2013
Miller. (1997). Rey figure as a predictor of every-
day functional ability in schizophrenia [Abstract].
Journal of Neuropsychiatry and Clinical Neuro-
sciences, 9, 36-37.
Meyers, J.E. & Meyers, K.R. (1995). Rey Complex
Figure Test and Recognition Trial. Odessa, FL:
Psychological Assessment Resources.
Osterrieth, P.A. (1944). Le test de copie d’une figure
complexe. Archives de Psychologie, 30, 206-356.
Perrine, K. (1993). Differential aspects of conceptual
processing in the Category Test and Wisconsin
Card Sorting Test. Journal of Clinical and Experi-
mental Neuropsychology, 15, 461-473.
Rapport, L.J., Farchione, T.J., Dutra, R.L., Webster,
J.S., & Charter, R. (1996). Measures of hemi-inat-
tention on the Rey Figure copy by the Lezak-
Osterrieth scoring method. The Clinical Neuropsy-
chologist, 10, 450-454.
Reitan, R.M., & Wolfson, D. (1985). The Halstead-
Reitan Neuropsychological Test Battery. Tucson,
AZ: Neuropsychological Press.
Rey, A. (1941). L’examin psychologique dans les cas
d’escephalopathie traumatique. Archives de
Psychologie, 28, 286-340.
Savage, C.R., Baer, L., Keuthen, N.J., Brown, H.D.,
Rauch, S.L., & Jenike, M.A. (1999). Organiza-
tional strategies mediate nonverbal memory im-
pairment in obsessive-compulsive disorder. Bio-
logical Psychiatry, 45, 905-916.
Schreiber, H.E., Javorsky, D.J., Robinson, J., & Stern,
R.A. (1999). Rey-Osterrieth Complex Figure per-
621
formance in adults with attention deficit hyperac-
tivity disorder: A Validation study of the Boston
Qualitative Scoring System. The Clinical Neuro-
psychologist, 13, 509-520.
Shorr, J.S., Delis, D.C., & Massman, P.J. (1992).
Memory for the Rey-Osterrieth Figure: Perceptual
clustering, encoding, and storage. Neuropsychol-
ogy, 6, 43-50.
Silva, S.G, Stern, R.A., Chaisson, N., Singer, E.A.,
Gaver, V., Watson, J.B., Golden, R.N., & Evan,
D.L. (1995). Evidence of mild visuoconstructive
impairments in HIV infection using the Boston
Qualitative Scoring System for the Rey-Osterrieth
Complex Figure [Abstract]. Journal of the Interna-
tional Neuropsychological Society, 1, 138.
Starkstein, S.E., & Robinson, R.G. (1997). Mecha-
nism of Disinhibition after brain lesions. The Jour-
nal of Nervous and Mental Disease, 185, 108-114.
Stern, R.A., Javorsky, D.J., Singer, E.A., Singer Har-
ris, N.G., Somerville, J.A., Duke, L.M., Thomp-
son, J., & Kaplan, E. (1999). The Boston Qualita-
tive Scoring System for the Rey-Osterrieth Com-
plex Figure. Odessa, FL: Psychological Assess-
ment Resources.
Stern, R.A. & Prohaska, M.L. (1996). Neuropsycho-
logical evaluation of executive functioning. In
Dickstein L.L., Riba, M.B., Oldham, J.M. (Eds.),
Review of Psychiatry, Vol.15 (pp. 243-266). Wash-
ington: American Psychiatric Press,.
Stern, R.A., Singer, E.A., Duke, L.M., Singer, N.G.,
Morey, C.E., Daughtrey, E.W., & Kaplan, E.
(1994). The Boston qualitative scoring system for
the Rey-Osterrieth Complex Figure: Description
and interrater reliability. The Clinical Neuropsy-
chologist, 8, 309-322.
Troyer, A.K., & Wishart, H.A. (1997). A comparison
of qualitative scoring systems for the Rey-
Osterrieth Complex Figure test. The Clinical Neu-
ropsychologist, 11, 381-390.
Visser, R.S.H. (1973). Manual of the Complex Figure
Test. Lisse, Netherlands: Swets & Zeitlinger.
Wechsler, D. (1981). Manual for the Wechsler Adult
Intelligence Scale – Revised. New York: Psycho-
logical Corporation.
Wechsler, D. (1987). Manual for the Wechsler Mem-
ory Scale – Revised. San Antonio, TX: Psychologi-
cal Corporation.