Brain & Development 31 (2009) 52–57

www.elsevier.com/locate/braindev

Original article

Predicting executive function task scores with the

Rey-Osterrieth Complex Figure
Tatsuya Ogino a,*, Kiyoko Watanabe b, Kousuke Nakano c, Yoko Kado d,
Teruko Morooka b, Akihito Takeuchi b, Makio Oka b, Satoshi Sanada e, Yoko Ohtuska b
a
Department of Children Studies, Faculty of Children Studies, Chugokugakuen University, Niwase 83, Okayama 701-0197, Japan
b
Department of Child Neurology, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama, Japan
c
Department of Pediatrics, Matsuyama Red Cross Hospital, Matsuyama, Japan
d
Department of Psychology, Faculty of Letters, Kansai University, Osaka, Japan
e
Division of Special Education, Faculty of Education, Okayama University, Okayama, Japan

Received 25 February 2008; received in revised form 15 July 2008; accepted 17 July 2008

Abstract

The Boston Qualitative Scoring System (BQSS) is one of the scoring methods of the Rey-Osterrieth Complex Figure Test
(ROCF). With BQSS, 17 qualitative ratings are generated, and subsequently 6 summary scores are calculated by combining several
qualitative scores. Previously we showed that 5 of 6 BQSS summary scores were correlated with the scores of the several executive
function tests in children. The objective of this study was to develop a new summary score which correlates with the executive func-
tion test scores more strongly than the existing summary scores. For this purpose, we conducted multiple regression analysis to pre-
dict PEN, CA, DSM of the Wisconsin Card Sorting Test Keio Version, and the Mazes scores from WISC-III, by the BQSS
qualitative scores derived from ROCF drawings in copy condition. The subjects were 78 children with various neuropsychological
disorders (5 years 5 months-14 years 11 months; mean: 9 years 2 months; F 22, M 56). Significant predictive models were generated
for PEN, CA, and Mazes scores. Among them the models for the Mazes scores were the most accurate. The second model for the
Mazes scores was most suitable for a new summary score. Its degree-of-freedom-adjusted coefficient of multiple determination and
multiple correlation coefficient reached 0.467 and 0.695, respectively. A new summary score should be applied in future studies to
evaluate its clinical usefulness.
Ó 2008 Elsevier B.V. All rights reserved.

Keywords: The Rey-Osterrieth Complex Figure Test; Boston Qualitative Scoring System; Executive function; Children; Pervasive developmental
disorders; Attention-deficit/hyperactivity disorder

1. Introduction and planning abilities in addition to visual cognitive

The Rey-Osterrieth Complex Figure Test (ROCF)
has been widely used for the assessment of visuocon-
structional ability and visual memory [1,2]. Because of
the complexity of the figure, the ROCF has been said
also to reflect executive functions such as organizational
* ROCF [6,7] (Psychological Assessment Resources, Inc.)
Corresponding author. Tel.: +81 86 293 6860; fax: +81 86 293
3993.
E-mail address: tatsu@cjc.ac.jp (T. Ogino).
functions [3,4]. Though there are a variety of scoring
methods for the ROCF, many of them are based on
Taylor’s 36-point system [5], and assess the presence
and accuracy of each drawn element of ROCF. Such
scoring methods, however, cannot evaluate qualitative
features such as drawing steps.
The Boston Qualitative Scoring System (BQSS) for the
published in 1999 is one of the most recent scoring methods
for ROCF which was developed to evaluate the qualitative

0387-7604/$ - see front matter Ó 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.braindev.2008.07.003
 T. Ogino et al. / Brain & Development 31 (2009) 52–57
features of ROCF as well as presence and accuracy. It pro-
vides, first, a comprehensive set of 17 qualitative ratings
based on the presence and accuracy of elements and the
process of drawing, and second, 6 quantitative summary
scores by combining the qualitative scores.
The BQSS was introduced predominantly for adults.
However, the BQSS should also be useful for children
because it enables us to evaluate ROCF multidimension-
ally, and its scoring criteria are very clear. In fact, Nak-
ano et al. [8] found that 5 of 6 summary scores showed
continuous development throughout childhood. In addi-
tion to this, some differences were noted in the age
ranges during which each score showed the most rapid
development. Furthermore, Watanabe et al. [9] showed
that, among children with neuropsychiatric impairment,
the BQSS summary scores significantly correlated with
the scores of various executive function tasks such as
the Wisconsin Card Sorting Test (WCST). Watanabe
et al. also stated that among those examinations, the
correlation between the summary scores and the scores
of the Mazes task was strong. However, there was no
summary score which showed specifically high correla-
tion with the scores of executive function tasks. Unex-
pectedly, the partial correlation coefficients between
the Organization score, which is considered to represent
organization abilities by the authors of BQSS, and the
scores of WCST or Mazes were smaller than those
between other summary scores and the executive func-
tion task scores. The salient features of BQSS are that
it provides abundant qualitative scales, and by using
the qualitative scale scores we can calculate quantitative
summary scores. Therefore, it might be possible to
develop a new summary score with new properties by
combining qualitative scores.
Here, we tried to predict the scores of executive func-
tion tests using BQSS qualitative scores with multiple
regression analysis in order to develop a new summary
score which correlates specifically with the scores of
executive function tests. As executive function tests, we
adopted WCST, which is considered to serve as a stan-
dard index of general executive functions [10], and the
Mazes task, which showed the strongest correlation with
BQSS summary scores in the study by Watanabe et al.
[9]. Our aim was to clarify the correlations that exist
innately between the test scores, not to uncover the char-
acteristics of specific disorders. Therefore, we chose
patients with various kinds and degrees of brain dys-
functions without mental retardation as the subjects in
order to maximize the variance of variables.
2. Subjects and methods
2.1. Subjects
The subjects consisted of 78 individuals (22 girls and
56 boys) aged 5 years 5 months to 14 years 11 months
53
(mean: 9 years 2 months), who were examined at the
Department of Child Neurology, Okayama University
Hospital between April 2002 and July 2005. All subjects
were diagnosed as having various neuropsychological
disorders: pervasive developmental disorders, 24 cases;
attention-deficit/hyperactivity disorder, 27; epilepsy, 25
(including 2 with pervasive developmental disorders, 5
with attention-deficit/hyperactivity disorder); and other
disorders, 9. Their full-scale IQs were 70 or higher
(71–127; mean: 93.0) as evaluated by the Wechsler Intel-
ligence Scale for Children Third Edition (WISC-III;
Nihon Bunka Kagakusha Co. Ltd, Tokyo). Though
all examinations were done for clinical purposes, we
received written consent from all parents of the partici-
pants following a full explanation of the assessments
and future publications. We presented small gifts to
the subjects after the tests. Fifty-six of 78 subjects were
the same as those of the previous study [9].
2.2. Procedure
Each subject underwent ROCF, several executive
function tasks including WCST, and WISC-III. The
details were described in the previous paper [9]. The
WISC-III was administered on a different day from
other tests.
The ROCF was administered strictly according to the
BQSS manual [6]. Three examination conditions, copy,
immediate recall and 20- to 30-min delayed recall were
performed. The ROCF was drawn with six colored felt
pens which were changed in a fixed order to enable the
examiner to track the drawing sequence. The point rat-
ing was strictly based on the criteria described in the
BQSS manual.
In the WCST, the subjects were instructed to classify
each card by one of three attributes: color, shape and
number. We used the Keio version (WCST-K) [11],
which includes two sessions, each of which is composed
of 48 trials. Mazes, one of the subtests of the WISC-III,
was used as a Mazes task.
2.3. Data analysis
The BQSS divides the ROCF into three sets of ele-
ments, namely, configural elements, clusters, and details.
These are hierarchically arranged in terms of structural
importance. For each of the three ROCF conditions,
17 kinds of qualitative scores were calculated by evalua-
tion of the presence, accuracy and placement of each ele-
ment, the accuracy of the size and orientation, the
neatness of drawing, and the drawing order and plan-
ning (Table 1). All these qualitative scores, except Asym-
metry, are based on an ordinal scale of 0 (extremely
poor) to 4 (excellent). We used 16 qualitative scores
for the copy condition, with the exception of Asymme-
try, as independent variables. The reason why we chose
54 T. Ogino et al. / Brain & Development 31 (2009) 52–57
Table 1
BQSS Qualitative Scores
(A) Evaluations of each element
Configural Presence
Configural Accuracy
(B) Assessments of the size and aspect of the whole drawing
Vertical expansion
Rotation
(C) Assessments of the drawing order and systemicity
Fragmentation
(D) Others
Perseveration
the qualitative scores only for the copy condition was
that we wanted to avoid problems of multi-collinearity
derived from excessive variables. Furthermore, it seemed
that if a measure reflecting executive function could be
calculated with only the product of copy condition, it
would be clinically useful.
As dependent variables predicted by BQSS qualita-
tive scores, we adopted Categories Achieved (CA), Per-
severative Errors of Nelson (PEN), and Difficulty
Maintaining Set (DMS) from WCST-K, each of which
was the sum of the scores of the first session and the sec-
ond session, and the raw score of the Mazes task from
WISC-III. Since some data were lost, there were 77
available subjects (F 22, M 55) for predicting the
WCST-K scores, and 72 (F 21, M 51) for the Mazes
score.
In order to predict the executive function scores by
BQSS qualitative scores for copy condition, step-wise
multivariate regression analyses were performed. As
the criteria to apply variables to or eliminate them from
predictive models, F = 2 was employed. When the stan-
dardized residual for each case exceeded 3, the case was
considered as an outlier. SPSS 12.0.2J for Windows was
used for statistical calculations.
3. Results
Using multiple regression analysis, significant predic-
tive models for CA and PEN of WCST-K and Mazes
scores were obtained (Table 2). There was no outlier
case in any of the analyses. No significant predictive
model was obtained for DMS of WCST-K.
Only one predictive model comprised of one variable,
Detail Presence, was obtained for CA. The degree-of-
freedom-adjusted coefficient of multiple determination
(adjusted R2) was so low that the model could explain
only 5.6% of the variance of CA. The multiple correla-
tion coefficient was 0.261.
Three significant predictive models for PEN were
obtained. The first model consisted of Neatness, the sec-
ond model consisted of Neatness and Reduction, and
the third model consisted of Neatness, Reduction, and
Cluster Presence Detail Presence
Cluster Accuracy
Cluster Placement Detail Placement
Horizontal Expansion Reduction
Planning Neatness
Confabulation Asymmetry
Cluster Presence as variables. The adjusted R2 for each
model were between 0.122 and 0.227, and the multiple
correlation coefficients were between 0.366 and
0.507.
Five significant predictive models were obtained for
the Mazes score. The adjusted R2 for the predictive
models for Mazes were higher than those of the models
for other dependent variables. Even the adjusted R2 of
the first model, which is composed of only one variable,
Configural Accuracy, was 0.429, which is higher than
that of the third model for PEN. The adjusted R2 of
the fifth model comprised of five variables, Configural
Accuracy, Planning, Reduction, Horizontal Expansion,
and Vertical Expansion, was as high as 0.569, and the
multiple correlation coefficient reached 0.774. However,
among the five variables of the fifth model, the partial
regression coefficients of Reduction and Horizontal
Expansion were negative values, in contrast to the other
variables. The maximum variance inflation factor (VIF)
of the partial regression coefficients of the five models
was only 1.667 (Vertical Expansion of the fifth model).
Although condition indices for the first and the second
model did not exceed 5.8, the index was 24.3 for the
third model, and more than 30 for the fourth and the
fifth model.
Partial correlation coefficients between the predicted
values and the actual values of the Mazes score with
age as the control variable were 0.433, 0,491, 0.535,
0.557, and 0.595 for the first model through the fifth
model, respectively. We previously reported the partial
correlation coefficients between the ordinary BQSS sum-
mary scores and the Mazes score [9]. In order to com-
pare the new summary score and the ordinary BQSS
summary scores, partial correlation coefficients between
the ordinary summary scores and the Mazes score were
calculated again with the new data. As a result, similar
values were obtained. Namely, the partial correlation
coefficients of the Mazes score with Copy Presence
and Accuracy, Immediate Presence and Accuracy,
Delayed Presence and Accuracy, Immediate Retention,
Delayed Retention, and Organization were 0.361,
0.371, 0.446, 0.224, 0.151, and 0.225, respectively.
Table 2
Summary of the models predicting WCST and Mazes scores
Y WCST (n = 77) Mazes (n = 72)
CA (n = 77) PEN (n = 77) DMS (n = 77) Raw score (n = 72)
Model 1 1 2 3 – 1 2 3 4 5

T. Ogino et al. / Brain & Development 31 (2009) 52–57

R 0.261 0.366 0.461 0.507 – 0.661 0.695 0.731 0.752 0.774
R*2 0.056 0.122 0.192 0.227 – 0.429 0.467 0.514 0.539 0.569
p 0.022 0.001 <0.001 <0.001 – <0.001 <0.001 <0.001 <0.001 <0.001
b1* (VIF) 0.261 (1.000) 0.366 (1.000) 0.363 (1.000) 0.280 (1.158) – 0.661 (1.000) 0.560 (1.226) 0.559 (1.226) 0.596 (1.272) 0.581 (1.278)
X1 Detail Neatness Neatness Neatness – Configural Configural Configural Configural Configural
Presence Accuracy Accuracy Accuracy Accuracy Accuracy
b2* (VIF) – – 0.281 (1.000) 0.255 (1.016) – – 0.236 (1.226) 0.263 (1.240) 0.263 (1.240) 0.194 (1.382)
X2 – – Reduction Reduction – – Planning Planning Planning Planning
b3* (VIF) – – – 0.228 (1.175) – – – 0.230 (1.014) 0.279 (1.093) 0.254 (1.111)
X3 – – – Cluster – – – Reduction Reduction Reduction
Presence
b4* (VIF) – – – – – – – – 0.185 (1.119) 0.302 (1.516)
X4 – – – – – – – – Horizontal Horizontal
Expansion Expansion
b5* (VIF) – – – – – – – – – 0.240 (1.667)
X5 – – – – – – – – – Vertical
Expansion
Maximum 6.069 4.592 22.132 27.490 3.770 5.826 24.331 30.422 33.984
condition index
R, multiple correlation coefficient; R*2, degree-of-freedom-adjusted coefficient of multiple determination; p, level of significance; bn*, standardized partial regression coefficient; VIF, variance
inflation factor; Xn, variable.

55
56 T. Ogino et al. / Brain & Development 31 (2009) 52–57
4. Discussion
Because of the complexity of the figure, strategies and
an organizational approach are needed to recognize the
presence and configuration of the elements of the
ROCF, and also to draw or recall. Reflecting this, signif-
icant correlations between BQSS scores and several
executive function task scores were reported in research
among adults [3]. As mentioned above, significant corre-
lations between BQSS summary scores and many kinds
of executive function task scores were reported also in
research among children with various neurological dis-
eases [9]. Unexpectedly, in their research, the BQSS
summary scores which showed significant correlations
with executive function task scores were Copy Presence
Accuracy, Immediate Presence Accuracy, and Delayed
Presence Accuracy, which were purported to be indices
of visuoperceptive and visual memory abilities. The par-
tial correlation coefficient between Delayed Presence
Accuracy and the Mazes score was the highest (0.446).
The purpose of our research was to develop a new
summary score which would show higher correlation
with executive function task scores. Using multiple
regression analysis we could obtain significant predictive
equations for CA and PEN of WCST-K and for the
Mazes score. The multiple correlation coefficient of the
third model of PEN with Neatness, Reduction, and
Cluster Presence as variables was moderate (0.507),
and that of the fifth model of the Mazes score with Con-
figural Accuracy, Planning, Reduction, Horizontal
Expansion, and Vertical Expansion as variables was
strong (0.774).
WCST has been considered to reflect planning, the
use of feedback, cognitive flexibility, and self-regulation
of impulsive reaction, so it is thought to serve as a stan-
dard index of general executive functions [10]. The
results of our study showed that the accuracies of the
predictive equations of scores from the same WCST-K
varied. The predictive equation of PEN was accompa-
nied by the highest multiple correlation coefficient, and
that of CA was the second highest. Contrary to this,
no significant predictive model of DSM was obtained.
This fact indicates that the psychological processes
mainly tapped by PEN, CA, and DSM are distinct. Sev-
eral studies on the factor analysis of WCST showed
three-factor solutions, namely, cognitive flexibility (or
perseveration), problem-solving (or nonperseverative
errors), and response maintenance (or inefficient sorting)
[12,13]. Perseverative errors and number of categories
completed resided in the cognitive flexibility factor,
and were purported to reflect, in particular, set-shifting
ability and mental flexibility among executive functions.
On the other hand, Failure-to Maintain-Set, which cor-
responds to DSM, was the only index residing in the
response maintenance factor. Our results indicate that
BQSS scores most strongly correlate with cognitive
flexibility among the processes reflected by WCST.
However, the reason why the accuracy of the predictive
equation for PEN and that for CA differ is unclear.
The highest multiple correlation coefficient and the
highest degree-of-freedom-adjusted coefficient of multi-
ple determination of the predictive models of the Mazes
score were very high, namely, 0.774 and 0.569, respec-
tively. The results indicated that the BQSS scores tap
mental processes reflected by the Mazes score more than
those reflected by PEN of WCST. The Mazes score was
shown to correlate with the score of the Tower of Lon-
don among executive function tests, so it was purported
to represent planning ability [14]. The frontal lobe, espe-
cially the prefrontal cortex, was found to be involved in
solving the Mazes task in studies with brain-damaged
patients or functional brain imaging [15,16]. The fact
that the Mazes score could be predicted excellently by
the BQSS scores indicates that frontal lobe functions,
especially planning ability, are strongly involved in the
execution of the ROCF task. However, there is a strong
possibility that in addition to planning ability, percep-
tual abilities to recognize complex figures and visuomo-
tor coordination abilities are also involved in
implementation of both the ROCF task and the Mazes
task. Further research is needed to clarify common men-
tal processes reflected by the ROCF task and the Mazes
task.
BQSS may have a broader range of applications
using the predictive equation of the Mazes score in this
study as a new summary score. The fifth model com-
prised of all five variables is not necessarily suitable
for a summary score. First, the partial regression coeffi-
cients of Reduction and Horizontal Expansion among
the qualitative scores put into the equation as variables
were negative values. This means that the higher the
Reduction and Horizontal Expansion scores, the lower
the Mazes score. Such a relationship is difficult to
explain logically. Second, although the VIF of each par-
tial regression coefficient was not so high, the condition
indices of the third, fourth, and fifth model were over 20,
indicating some possibilities of multi-collinearity. Con-
sidering that it is generally desirable to limit the number
of independent variables to two or three, the second pre-
dictive model seems to be the most suitable for the new
summary score.
In order to simplify the interpretation of the contri-
butions of the explanatory variables, we presented the
predictive equations generated by multiple regression
analysis with standardized partial regression coefficients
in Section 3. The second model for the Mazes score with
non-standardized partial regression coefficients was as
follows:
Mazes score ¼ 3:902 þ 2:378  configural accuracy
þ 1:198  Planning
 T. Ogino et al. / Brain & Development 31 (2009) 52–57
Considering usability in clinical practice, we recommend
the following simplified equation as a new summary
score based on the second model for the Mazes score:
New summary score ¼ 2  configural accuracy
þ Planning
The partial correlation coefficient between the predictive
values of the second model and the Mazes scores, using
age as the control variable, was 0.491. Among the ordin-
ary BQSS summary scores, the score which showed the
strongest correlation with the scores of Mazes was De-
layed Presence and Accuracy, and the partial correlation
coefficient was 0.446. Among the BQSS summary scores,
the Copy Presence and Accuracy score and the Organi-
zation score can be calculated only with the product of
the copy condition. The partial correlation coefficients
between those and the score of Mazes were 0.361 and
0.225, respectively. The second model of the Mazes
scores in this study showed a higher partial correlation
coefficient than those of existing summary scores. More-
over, the equation can be calculated using only the data
of the copy condition. This will make it easier to use the
new summary score in the clinical field. However, sev-
eral validity studies with patients suffering from various
neurological and psychological diseases will be neces-
sary in the future to elucidate the significance of the
new summary score.
References
[1] Rey A. L’examen psychologique dans les cas d’encephalopathie
traumatique. Arch Psychol 1941;28:286–340.
[2] Osterrieth PA. Le test de copie d’une figure complexe. Arch
Psychol 1944;30:206–356.
[3] Somerville J, Tremont G, Stern RA. The Boston Qualitative
Scoring System as a measure of executive functioning in Rey-
Osterrieth complex figure performance. J Clin Exp Neuropsychol
2000;22:613–21.
57
[4] Beebe DW, Ris MD, Brown TM, Dietrich KN. Executive
functioning and memory for the Rey-Osterreith Complex Figure
Task among community adolescents. Appl Neuropsychol
2004;11:91–8.
[5] Taylor EM. Psychological appraisal of children with cerebral
defects. Cambridge, MA: Harvard University Press; 1959.
[6] Stern RA, Javorsky DJ, Singer EA, Singer Harris NG, Somerville
JA, Duke LM, et al. The Boston Qualitative Scoring System for
the Rey-Osterrieth complex figure: professional manual. Odessa,
FL: Psychological Assessment Resources, Inc.; 1999.
[7] Stern RA, Singer EA, Duke LM, Singer NG, Morey CE,
Daughtrey EW, et al. The Boston Qualitative Scoring System
for the Rey-Osterrieth complex figure: description and interrater
reliability. Clin Neuropsychol 1994;8:309–22.
[8] Nakano K, Ogino T, Watanabe K, Hattori J, Ito M, Oka M, et al.
A developmental study of scores of the Boston Qualitative
Scoring System. Brain Dev 2006;28:641–8.
[9] Watanabe K, Ogino T, Nakano K, Hattori J, Kado Y, Sanada S,
et al. The Rey-Osterrieth complex figure as a measure of executive
function in childhood. Brain Dev 2005;27:564–9.
[10] Heaton RK, Chelune GJ, Talley JL, Kay GG, Curtis G.
Wisconsin card sorting test (WCST) manual revised and
expanded. Odessa, FL: Psychological Assessment Resources,
Inc.; 1993.
[11] Kashima H, Kato M. Tests for frontal function-pattern of frontal
dysfunction and its assessment. Shinkei Kenkyu No Shinpo
1993;37:93–110, [in Japanese].
[12] Greve KW, Love JM, Sherwin E, Mathias CW, Ramzinski P,
Levy J. Wisconsin card sorting test in chronic severe traumatic
brain injury: factor structure and performance subgroups. Brain
Inj 2002;16:29–40.
[13] Nagahama Y, Okina T, Suzuki N, Matsuzaki S, Yamauchi H,
Nabatame H, et al. Factor structure of a modified version of the
wisconsin card sorting test: an analysis of executive deficit in
Alzheimer’s disease and mild cognitive impairment. Dement
Geriatr Cogn Disord 2003;16:103–12.
[14] Krikorian R, Bartok J, Gay N. Tower of London procedure: a
standard method and developmental data. J Clin Exp Neuropsy-
chol 1994;16:840–50.
[15] Levin HS, Song J, Ewing-Cobbs L, Roberson G. Porteus Maze
performance following traumatic brain injury in children. Neu-
ropsychology 2001;15:557–67.
[16] Kirsch P, Lis S, Esslinger C, Gruppe H, Danos P, Broll J, et al.
Brain activation during mental maze solving. Neuropsychobiol-
ogy 2006;54:51–8.