See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/26509394

Development of a standardized scoring method for the Graphic Sequence Test

suitable for use in psychiatric populations

Article in Annals of General Psychiatry · April 2008

DOI: 10.1186/1744-859X-7-S1-S289 · Source: DOAJ

CITATIONS READS

3 588

13 authors, including:

Konstantinos Fountoulakis Panagiotis Panagiotidis

Aristotle University of Thessaloniki 424 General Military Training Hospital
538 PUBLICATIONS 12,977 CITATIONS 49 PUBLICATIONS 609 CITATIONS

SEE PROFILE SEE PROFILE

Melina Siamouli Natalia Papastergiou

Aristotle University of Thessaloniki Lausanne University Hospital
57 PUBLICATIONS 1,276 CITATIONS 11 PUBLICATIONS 67 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Editorial View project

All content following this page was uploaded by Esat Timuçin Oral on 16 May 2014.

The user has requested enhancement of the downloaded file.

 ORIGINAL STUDY
Development of a Standardized Scoring Method for
the Graphic Sequence Test Suitable for Use
in Psychiatric Populations
Konstantinos N. Fountoulakis, MD, PhD,* Panagiotis T. Panagiotidis, MD,*
Melina Siamouli, MD,* Stamatia Magiria, MD,* Stavroula Sokolaki, MD,*
Sotiris Kantartzis, MD,* Klairi Rova,* Natalia Papastergiou,* George Shoretstanitis,*
Timucin Oral, MD,w Theoharis Mavridis, MD, PhD,* Apostolos Iacovides, MD, PhD,*
and George Kaprinis, MD, PhD*
Abstract: Although the graphic version of the Alternating
Sequences Test which was introduced by Luria exists for years
little has been done to standardize it. The aim of the current
study was to develop a novel and detailed standardized method
of administration and scoring. The study sample included
93 normal control subjects (53 women and 40 men) aged
35.87 ± 12.62 and 127 patients suffering from schizophrenia
(54 women and 73 men) aged 34.07 ± 9.83. The psychometric
assessment included the Positive and Negative Symptoms Scale
the Young Mania Rating Scale, and the Montgomery-Asberg
Depression Rating Scale. A scoring method was developed and
was based on the frequencies of responses of healthy controls.
Cronbach a and test-retest and interrater reliability were very
good. Two indices and 6 subscales of the Standardized Graphic
Sequence Test were eventually developed. The Standardized
Graphic Sequence Test seems to be a reliable, valid, and
sensitive to change instrument based on Luria’s graphic
sequence test. The great advantage of this instrument is the fact
that it is paper and pencil, easily administered and little time
consuming. Further research is necessary to test its usefulness as
a neuropsychologic test.
Key Words: neuropsychology, schizophrenia, alternative sequence
test, standardization
(Cog Behav Neurol 2008;21:18–27)
T he Alternating Sequences Test was introduced by
Luria1–3 and assesses the ability of the frontal lobes to
inhibit inappropriate responses.4 There is a motor version
which demands the patient to make a sequence of hand
movements (eg, fist-palm-edge) and a graphic (drawing)
Received for publication July 18, 2007; accepted November 25, 2007.
From the *Third Department of Psychiatry, Aristotle University of
Thessaloniki, Greece; and wFifth Inpatient Department of Psychiatry
and Outpatient Unit of Mood Disorders, Bakirköy State Teaching
and Research Hospital for Neuropsychiatry, Istanbul, Turkey.
Reprints: Panagiotidis T. Panagiotis, MD, Third Department of
Psychiatry, Aristotle University of Thessaloniki, 10, Paisiou Street,
Neapoli Thessaloniki 567 28, Greece (e-mail: psypanpan@yahoo.gr).
Copyright r 2008 by Lippincott Williams & Wilkins
18
version which demands the patient to copy a simple
alternation of rectangles and peaks or letters. Impaired
patients tend to repeat the same movement or shape and
do not alternate with the other member of the pair.
Attention and concentration is essential in completing
this task; confusional or inattentive patients tend to
perform poorly.
Typically this group of tests is useful in the study of
frontal lobe patients5 including patients with schizophre-
nia, which are believed to manifest a type of hypofron-
tality. The term hypofrontality refers to a decreased
functioning of the frontal lobes. It is believed that
dopaminergic hypoactivity in the mesocortical dopami-
nergic system results in negative symptoms whereas
an increased dopaminergic activity in the mesolimbic
is believed to be the cause of positive symptoms of
schizophrenia. Thus, hypofrontality is supposed to relate
to negative symptoms and behavioral disinhibition in
these patients, and it is a widely accepted characteristic
of schizophrenia although it has never been sufficiently
proved.
However, although the graphic version of the test
exists for years, and is widely used, little has been done to
standardize it. Scoring is based on common errors
observed and in this way many details in the performance
of patients may be lost, especially when the test is used in
psychiatric populations.
Samples showing how patients with schizophrenia
perform in this task are shown in Figure 1.
The aim of the current study was to develop a novel
and detailed standardized method of administration and
scoring of the Graphic Sequence Test and to preliminary
test this method in schizophrenic patients. This new
scoring method aims to be reliable, valid, and sensitive to
change in response to treatment.
MATERIALS AND METHODS
Study Sample
The study sample included 93 normal control
subjects (53 women—56.98% and 40 men—43.02%)
aged 35.87 ± 12.62 (range: 18 to 68) and 127 patients
Cog Behav Neurol  Volume 21, Number 1, March 2008
Cog Behav Neurol  Volume 21, Number 1, March 2008
FIGURE 1. Copying of the template (A) of the Graphic Sequence Test by patients with schizophrenia (B to I).
suffering from schizophrenia according to DSM-IV-TR
(54 women—42.52% and 73 men—57.48%) aged
34.07 ± 9.83 (range: 18 to 66). The only inclusion or
exclusion criteria concerned the general somatic health (as
shown below). No other specific inclusion or exclusion
criteria were used and there is no systematic bias in the
whole sample. The patients’ sample cannot be considered
as being representative of the general schizophrenic
population; however, it reflects the properties and
qualities of patients with schizophrenia usually seen in
the outpatient and inpatient facilities of the psychiatric
department of a general hospital. Also the selection of
control sample obeyed to the same criteria. The only
additional criterion posed for controls was age and sex so
as the characteristics of the normal subjects to be similar
to those of patients with schizophrenia; in this frame, it
can be considered to reflect the general population with
similar age and sex.
All subjects were physically healthy with normal
clinical and laboratory findings. All control subjects and
patients gave informed consent and the protocol received
approval by the University’s Ethics Committee.
Clinical Diagnosis
The diagnosis was put according to DSM-IV-TR
criteria on the basis of a semistructured interview on
the basis of the Schedules for Clinical Assessment in
Neuropsychiatry version 2.0 (SCAN v 2.0).6 The normal
controls were assessed on the basis of a nonstructured free
clinical interview.
The Standardized Graphic Sequence
Test Procedure
The Standardized Graphic Sequence Test (SGST)
procedure demanded the subject to copy a shape of
r 2008 Lippincott Williams & Wilkins
Standardized Scoring for the Graphic Sequence Test
alternating squares and peaks. The shape includes 8
squares and 8 peaks and is shown in the Appendix.
Instead of the usual instructions given to the subjects,
who demanded the subject to continue drawing the
sequence until the end of the sheet, the SGST instructions
ask the subject to draw an identical shape on the same
piece of paper. The template shape was printed on the left
half of the sheet leaving space for the subject to reproduce
it on the right. No time limit was set and no time
recording was made.
The assessment included the Random Letter Test
for the assessment of attention and vigilance.5 It includes
the following 4 series of letters: LTPEAOAISTDALAA;
ANIABFSAMPZEOAD; PAKLATSXTOEABAA; and
ZYFMTSAHEOAAPAT. The first and third group
include 5 ‘‘A,’’ whereas the second and the fourth include
4 ‘‘A.’’ The testing demands the patient to hit the desk
when the examiner pronounces ‘‘A.’’ Errors of omission
and commission are recorded. It is expected (and verified
in the present study) that the mean number of errors
expected from normal controls in this test is around 0.2.
Both errors of omission and commission were registered
for this test.
The Psychometric Assessment
The psychometric assessment included the Positive
and Negative Symptoms Scale (PANSS),7 the Young
Mania Rating Scale (YMRS),8 and the Montgomery-
Asberg Depression Rating Scale (MADRS).9
Development of a Scoring Method
The scoring method was developed empirically on
the basis of qualitative and quantitative analysis of a large
number of tests from normal controls, schizophrenic,
bipolar, depressive, and anxiety patients. The scoring
19
Fountoulakis et al Cog Behav Neurol  Volume 21, Number 1, March 2008

method is shown with details in the Appendix. The

TABLE 1. Frequencies of Normal Controls’ Performance in
rational behind this effort was the impression the authors Each Item and Proposed Standardized Score
had that by using the traditional qualitative way of
Standard
assessing the graphic sequence test, a lot of information Raw Score No. Obs % of Obs Score
was lost, and the test was not sensitive enough for use in
No. ‘‘A’’ omissions
‘‘nonorganic’’ mental patients. The first step was to create 0 92 98.92 100
a list of errors, omissions, and other deviations from 1 1 1.08 0
normal, seen in the testing of the above-mentioned >1 0 0 0
subjects. The second phase led to the development of Total 93 100.00
13 items and their scoring instructions for field-testing. No. ‘‘A’’ intrusions
0 86 92.47 100
These 13 items eventually constituted the SGST. The aim 1 6 6.45 8
was to apply a scoring method solely on the basis of 2 1 1.08 1
percentiles derived from the normal controls group. The >2 0
authors did not proceed to try to calculate sensitivity and Total 93 100.00
Total no. missing elements (maximum 16)
specificity for one or more specific cut-off points, because 0 77 82.80 100
the overlapping between groups was significant and the 1 11 11.83 20
test seems to be useful to assess aspects of cognitive 2 5 5.38 5
function but not as a specific diagnostic test for a specific >2 0
illness. Total 93 100.00
No. fragmented connection points (maximum 39)
0 64 68.82 100
1 11 11.83 30
2 5 5.38 20
Statistical Analysis 3 3 3.23 10
The statistical analysis included the development of 4 3 3.23 10
5 2 2.15 8
frequency tables for scores of healthy controls so as to 6 2 2.15 5
arrive at percentile scores and develop a scoring method 7 2 2.15 3
for the scale. The Pearson R correlation coefficient, factor 7-Dek 1 1.08 1
analysis (varimax normalized rotation-correlations >12 0 0.00 0
Total 93 100.00
among items of the SGST were low enough to permit No. rectangles replaced by peaks (maximum 8)
an orthogonal rotation), and Item Analysis10 (calculation 0 89 95.70 100
of Cronbach a) were used to explore the internal structure 1 1 1.08 4
of the scale. Although a significant number of correla- 2 1 1.08 3
tions were calculated, no correction (eg, Bonferroni) was 3 1 1.08 2
>3 1 1.08 0
applied, simply because this would lead all correlations to Total 93 100.00
be considered nonsignificant. However, in this frame, No. peaks replaced by rectangles (maximum 8)
correlation analysis should be considered as exploratory. 0 93 100.00 100
Analysis of variance (ANOVA),11 was used to test the 1 0 0.00 0
Total 93 100.00
difference between groups, and was performed separately Total no. errors in rectangles (maximum 16)
for subjects below and above the age of 40. Discriminant 0 20 21.51 100
Function Analysis was also used to explore differences 1 10 10.75 70
between groups and the power of the scale in discriminat- 2 8 8.60 60
ing between them. The Pearson R correlation coefficient 3 6 6.45 50
4 10 10.75 40
was calculated to assess the test-retest reliability and the 5 13 13.98 30
interrater reliability. However, the calculation of correla- 6 5 5.38 20
tion coefficients is not a sufficient method to test reliability 7 6 6.45 15
and reproducibility of a method and its results, because it 8 3 3.23 10
9 4 4.30 8
is an index of correlation and not an index of 10 5 5.38 3
agreement.11–13 The calculation of means and SDs for 11 0 0.00 3
each SGST item and total score during the first (test) and 12 1 1.08 2
second (retest) applications may provide an impression of 13 0 0.00 2
the stability of results over time. 14 1 1.08 1
>14 1 1.08 0
Also, the means and the SDs of the differences Total 93 100.00
concerning each SGST item between test and retest were No. ‘‘dashes’’ or other intrusions between parts
calculated and the plots of the test versus retest and 0 80 86.02 100
difference versus average value for each variable were 1 12 12.90 14
>1 1 1.08 0
created. In fact, it is not possible to use statistics to define Total 93 100.00
acceptable agreement.11 However, these plots may assist No. Corrections
decision. This method was used in previous studies 0 72 77.42 100
concerning the validation of scientific methods.14,15 1 17 18.28 20

20 r 2008 Lippincott Williams & Wilkins

Cog Behav Neurol  Volume 21, Number 1, March 2008 Standardized Scoring for the Graphic Sequence Test

Psychopathology scales (PANNS), the YMRS, and the

TABLE 1. (continued)
MADRS are shown in Table 4.
Standard The correlations among the SGST items were low
Raw Score No. Obs % of Obs Score
enough to permit an orthogonal rotation in the factor
2 3 3.23 4 analysis. The results of the factor analysis (varimax
3 1 1.08 1
>3 0
normalized rotation) are shown in Table 5. The analysis
Total 93 100.00 produced 6 factors (by using the Keiser-Fleish criterion of
Give 1 point for every peak which is significantly (>50%) lower than eigenvalues larger than 1) explaining 62% of the total
expected variance. The scores in the subscales created on the basis
0 66 70.97 100 of these factors and the differences between groups in
1 17 18.28 30
2 5 5.38 10 these scales are also shown in Table 5. The 1-way
3 4 4.30 5 ANOVA revealed significant differences between the 2
4 0 0.00 5 diagnostic groups concerning the subscales (df = 6.120;
5 1 1.08 1 F = 10.605; P<0.001). The post hoc t test revealed that
>5 0
Total 93 100.00
the 2 groups differed in all subscales except Close-in (CI)
Give 1 point if the drawing does not lie on a straight line (Table 5).
0 71 76.34 100 The correlation coefficients among these subscales
1 22 23.66 20 are shown in Table 6. Some correlations among these
Total 93 100.00 scales are statistically significant but weak. A second
Give 1 point if the drawing is significantly (>30%) smaller than the
template factor analysis of these subscales produced 2 superfactors
0 88 94.62 100 explaining 24% of total variance each. The first one
1 5 5.38 5 included subscales 1, 4, and 5 and the second one included
Total 93 100.00 subscales 2, 3, and 6 (Table 7).
Give 1 point if the drawing is significantly (>30%) bigger than the
template
0 81 87.10 100
1 12 12.90 10 TABLE 2. Comparison of the Scores of Normal Controls and
Total 93 100.00 Schizophrenic Patients Above and Below 40 Years of Age With
Give 1 point if the drawing is closing in t Test as the Post Hoc Test
0 92 98.92 100
1 1 1.08 0 Controls Patients With Schizophrenia
Total 93 100.00 Mean SD Mean SD P
Give 1 point if there are extrarectangles or peaks
0 90 96.77 100 Below 40 y N = 60 N = 101
1 0 0.00 0 RLT-A 100.00 0.00 71.43 45.72 <0.001
2 3 3.23 0 RLT-B 84.14 21.31 65.00 40.05 <0.01
SGST-1 76.24 40.72 59.01 46.33 <0.001
Obs indicates observations. SGST-2 79.37 34.51 73.53 38.66 NS
SGST-3 95.81 19.89 90.44 29.00 NS
SGST-4 98.92 10.37 89.11 31.31 <0.05
SGST-5 48.51 31.04 34.31 27.38 <0.01
SGST-6 72.94 41.60 69.48 44.21 <0.05
RESULTS SGST-7 77.34 37.78 77.47 38.73 NS
SGST-8 74.37 37.95 65.18 43.00 <0.05
The frequency tables for scores of healthy controls SGST-9 80.22 34.70 57.23 40.10 <0.001
are shown in Table 1. In the same table the proposed SGST-10 97.96 13.86 92.48 25.78 NS
scoring for each item is also shown. This scoring method SGST-11 81.61 36.48 59.01 45.04 <0.001
is based solely on the frequencies of responses of healthy SGST-12 98.92 10.37 97.03 17.06 NS
controls (percentiles). SGST-13 98.92 10.37 92.08 27.14 NS
SGST 1081.12 134.85 956.34 143.20 <0.001
The 1-way ANOVA with diagnostic group as the Above 39 y N = 33 N = 26
independent variable and all the SGST items plus SGST RLT-A 96.77 17.96 84.62 37.55 NS
total score as dependent variables, performed twice for 2 RLT-B 87.13 15.98 62.46 43.00 <0.05
different age groups (below and over 40 y of age) revealed SGST-1 61.61 29.42 51.15 37.53 NS
SGST-2 50.06 29.51 45.65 29.95 NS
significant results both for subjects under the age of 40 SGST-3 93.00 16.70 78.08 37.46 NS
(df = 14, 152; F = 7.239; P<0.001), and for those above SGST-4 100.00 0.00 92.31 27.17 NS
this age (df = 14, 143; F = 6.802; P<0.001). The results SGST-5 41.61 28.33 39.42 29.62 NS
are shown in Table 2 along with post hoc tests. It seems SGST-6 75.84 24.20 69.81 31.89 NS
that in older subjects the differences tend to minimize SGST-7 64.00 27.71 60.19 30.60 NS
SGST-8 53.23 23.72 44.42 29.61 NS
because the performance of controls gets worse. SGST-9 59.03 25.38 43.27 27.71 <0.05
The Pearson R correlation coefficients, among SGST-10 86.29 27.05 91.54 17.65 NS
the SGST items in the total study sample are shown in SGST-11 66.77 36.91 50.00 40.79 NS
Table 3. SGST-12 100.00 0.00 92.31 27.17 NS
SGST-13 93.55 24.97 88.46 32.58 NS
The Pearson R correlation coefficients, among the SGST 945.00 76.94 846.62 145.32 <0.01
SGST items and the Positive, Negative, and General

r 2008 Lippincott Williams & Wilkins 21

Fountoulakis et al Cog Behav Neurol  Volume 21, Number 1, March 2008

TABLE 3. Pearson Correlation Coefficients (R) Among the SGST Items and Random Letter Test Scores in the Total Study Sample
RLT-A RLT-B SGST-1 SGST-2 SGST-3 SGST-4 SGST-5 SGST-6 SGST-7 SGST-8 SGST-9 SGST-10 SGST-11 SGST-12 SGST-13
RLT-A
RLT-B 0.48
SGST-1 0.20 0.26
SGST-2 0.06 0.05 0.06
SGST-3 0.12 0.13 0.16 0.02
SGST-4 0.14 0.13 0.23 0.07 0.13
SGST-5 0.12 0.17 0.02 0.17 0.01 0.17
SGST-6 0.18 0.12 0.13 0.04 0.03 0.13 0.10
SGST-7 0.02 0.05 0.14 0.12 0.09 0.05 0.02 0.19
SGST-8 0.10 0.08 0.01 0.10 0.01 0.04 0.18 0.03 0.03
SGST-9 0.11 0.17 0.04 0.06 0.11 0.02 0.23 0.08 0.01 0.15
SGST-10 0.04 0.02 0.05 0.06 0.01 0.00 0.05 0.03 0.12 0.06 0.05
SGST-11 0.12 0.09 0.22 0.05 0.09 0.03 0.01 0.04 0.16 0.11 0.09 0.15
SGST-12 0.03 0.15 0.00 0.07 0.01 0.01 0.07 0.04 0.02 0.04 0.00 0.04 0.04
SGST-13 0.00 0.02 0.18 0.04 0.10 0.00 0.10 0.09 0.05 0.08 0.07 0.12 0.04 0.04
SGST total 0.27 0.31 0.48 0.19 0.39 0.42 0.35 0.45 0.44 0.32 0.41 0.08 0.47 0.15 0.20

Values significant at P<0.05 are marked in bold characters.

Item Analysis (calculation of Cronbach a) Cron- When 8.222+0.0179(SGST-1)+0.0110(SGST-6)

bach a=0.41, with no item increasing dramatically the
a coefficient when omitted. This leads to the conclusion
that the various items assess different aspects of perfor-
mance and there is a factor structure underlying.
The Discriminant Function Analysis used the
diagnostic groups as the grouping variable and all SGST
items as the dependent variables. The results are shown in
Table 8. This analysis produced the following function:
+0.0192  (SGST-9)+0.0189  (SGST-11) >0 then the
subject is likely to be a normal control rather than a
schizophrenic patient. This function classified correctly
73.11% of controls and 79.52% of patients with schizo-
phrenia, which is a satisfactory performance.
The Pearson R correlation coefficient (R) for
interrater reliability is 0.80 for the total SGST scale and
ranges from 0.61 to 1.00 for individual items and subscales

TABLE 4. Pearson Correlation Coefficients (R) Among the SGST Items and the Psychometric Scales Scores in Schizophrenic
Patients
PANSS-General
PANSS-positive PANSS-negative Psychopathology YMRS MADRS
RLT-A 0.00 0.06 0.08 0.14 0.11
RLT-B 0.02 0.03 0.04 0.07 0.16
SGST-1 0.03 0.14 0.07 0.23 0.21
SGST-2 0.16 0.21 0.21 0.35 0.09
SGST-3 0.07 0.09 0.17 0.15 0.05
SGST-4 0.08 0.13 0.11 0.06 0.17
SGST-5 0.03 0.07 0.03 0.03 0.08
SGST-6 0.13 0.09 0.03 0.00 0.06
SGST-7 0.02 0.07 0.00 0.25 0.09
SGST-8 0.02 0.36 0.06 0.04 0.12
SGST-9 0.11 0.06 0.12 0.12 0.05
SGST-10 0.01 0.17 0.01 0.24 0.06
SGST-11 0.07 0.16 0.10 0.08 0.18
SGST-12 0.23 0.26 0.31 0.11 0.16
SGST-13 0.16 0.12 0.01 0.24 0.13
SGST 0.12 0.00 0.03 0.13 0.13
DcI 0.04 0.01 0.06 0.03 0.12
ME 0.02 0.09 0.01 0.08 0.20
P 0.02 0.07 0.15 0.04 0.05
C 0.13 0.09 0.03 0.00 0.06
DfI 0.18 0.08 0.16 0.36 0.01
E 0.07 0.06 0.12 0.19 0.19
S 0.05 0.03 0.08 0.22 0.19
CI 0.26 0.33 0.35 0.28 0.00
Values significant at P<0.05 are marked in bold characters.

22 r 2008 Lippincott Williams & Wilkins

Cog Behav Neurol  Volume 21, Number 1, March 2008 Standardized Scoring for the Graphic Sequence Test

TABLE 5. Factor Analysis of SGST Items (Varimax Normalized Rotation) of the Whole Sample and Comparison Between the 2
Diagnostic Groups (1-way ANOVA) Concerning SGST Subscales
Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6
SGST-1 0.75 0.01 0.23 0.06 0.05 0.11
SGST-2 0.02 0.69 0.12 0.13 0.09 0.41
SGST-3 0.44 0.20 0.01 0.52 0.03 0.28
SGST-4 0.75 0.11 0.23 0.00 0.07 0.14
SGST-5 0.11 0.68 0.14 0.19 0.13 0.17
SGST-6 0.18 0.13 0.27 0.14 0.73 0.00
SGST-7 0.06 0.07 0.34 0.07 0.78 0.05
SGST-8 0.06 0.63 0.37 0.07 0.02 0.17
SGST-9 0.06 0.11 0.09 0.79 0.05 0.16
SGST-10 0.07 0.05 0.73 0.05 0.01 0.06
SGST-11 0.34 0.08 0.56 0.20 0.09 0.25
SGST-12 0.01 0.05 0.02 0.02 0.06 0.84
SGST-13 0.32 0.11 0.08 0.60 0.21 0.13
% of total variance 12% 11% 10% 11% 9% 9%
Total variance explained 62%
Normal Controls Patients With Schizophrenia
Mean SD Mean SD P
DcI 858.70 77.44 761.89 103.09 <0.001
ME 281.23 40.67 230.11 68.71 <0.001
P 273.65 47.56 254.55 52.09 <0.05
C 303.83 30.39 277.23 45.99 <0.001
DfI 557.12 85.82 503.77 98.36 <0.001
E 200.47 55.25 178.56 62.19 <0.05
S 183.28 35.56 157.95 45.28 <0.001
CI 173.37 39.30 167.26 41.15 NS

(Table 9); concerning test-retest reliability, the same widely used, little data can be found in the literature.
coefficient was equal to 0.54 for the total SGST scale and Scoring is based on common errors observed. The
ranged from 0.03 to 0.80 for individual items and problem is that in this way many details in the
subscales (Table 9). Retest was done within 5 days of first performance of patients may be lost, and this is especially
testing. The calculation of means and SDs for each SGST true when the test is used in psychiatric populations.
item and total score during the first (test) and second Luria himself was rather closer to a ‘‘qualitative’’ and
(retest) applications as well as the plots of the test versus ‘‘clinical’’ interpretation rather than a ‘‘standardized’’
retest and difference versus average value for each variable and ‘‘quantified’’ one. Even the Luria-Nebraska battery
suggested that the SGST is reliable and replicable. uses a very simple way to score these tests.
The current study attempted to develop a standar-
DISCUSSION dized scoring method that would allow the examiner to
Although several decades have passed since the reliably quantify the subject’s performance in the Graphic
Alternating Sequences Tests were introduced by Luria,1–3 Sequence Test, which is one of the Alternating Sequences
little has been done to standardize them. This may be due Tests. This test demands the subject to copy a simple
to the complex pattern of these tests and a preference of
the examiners to score them on the basis of an ‘‘overall’’
impression or ‘‘qualitatively.’’ Specifically concerning the TABLE 7. Factor Analysis of the Subscales (Second Order
graphic version of the test, which exists for years, and is Factor Analysis)
Second-order Factor 1 Second-order Factor 2
Factor no. 1 0.10 0.73
TABLE 6. Correlation Coefficients Among the SGST Subscales Factor no. 2 0.71 0.06
DcI ME P C DfI E S Factor no. 3 0.49 0.25
Factor no. 4 0.12 0.69
DcI 1.00 Factor no. 5 0.35 0.54
ME 0.84 1.00 Factor no. 6 0.74 0.21
P 0.73 0.39 1.00 Explained variance 1.44 1.42
C 0.47 0.15 0.10 1.00 Proportion of 24% 24%
DfI 0.16 0.11 0.17 0.04 1.00 variance explained
E 0.14 0.06 0.18 0.06 0.77 1.00 Total variance 48%
S 0.17 0.15 0.12 0.05 0.59 0.08 1.00 explained
CI 0.00 0.01 0.02 0.05 0.64 0.39 0.06
The second order factor loadings which determine the grouping (above 0.4) are
Significant values at P<0.05 are marked in bold characters. marked in bold characters.

r 2008 Lippincott Williams & Wilkins 23

Fountoulakis et al Cog Behav Neurol  Volume 21, Number 1, March 2008

TABLE 8. Discriminant Function Analysis Results and Function Coefficients

Classified as
Diagnosis % Classified Correct Classified as Normal Controls Schizophrenic Patients Total
Normal controls 73.11 68 25 93
Schizophrenic patients 79.52 26 101 127
Total 73.47 94 126 230

Normal Controls Function Schizophrenic Patients

Coefficients Function Coefficients
Constant 66.6723 58.4503
SGST-1 0.0675 0.0495
SGST-6 0.0210 0.0100
SGST-9 0.0290 0.0098
SGST-11 0.0355 0.0167

drawing. Both the drawing template and the resulting the overlapping between groups was significant and the
Standardized Graphic Sequence Test along with the test seems to be useful to assess aspects of cognitive
scoring method developed by the current study are shown function but not as a specific diagnostic test for a specific
in the Appendix. The test and its scoring method proved illness.
to be highly reliable, stable, and sensitive to change after The correlation coefficients among individual SGST
treatment. Sensitivity to change after treatment is evident items although significant, are all below 0.30 suggesting
from examples of how performance on the SGST changes that each item assesses a distinct issue. This is also
after 2 months antipsychotic treatment, shown in Figure 2. reflected in factor analysis. The 6 factors that emerge
However, what remains is to apply the test to different explain roughly 10% of the total variance each. The
patient population, especially to patients suffering from SGST can be divided into subscales on the basis of the
‘‘organic’’ brain disease, before and after therapeutic factor analysis and its interpretation. In this way, 6
intervention. subscales can be created. The first factor includes items 1,
The scoring method is such that allows for 3, and 4 and largely reflects missing elements. Thus, it
maximum contrast and differentiation between normal may constitute the basis of a subscale named ‘‘Missing
subjects and patients and simultaneously leaves little Elements’’ (ME). The second one includes items 2, 5,
space for subjective assessment. The results of the and 8 and rather reflects errors in drawing leading to
Discriminant Function Analysis support this. However, distortions of the shape. Thus, it constitutes the basis of a
apart from Discriminant Function Analysis, the authors subscale under the title ‘‘Errors’’ (E). The third factor
did not proceed to try to calculate sensitivity and includes items 10 and 11 and reflects differences in size.
specificity for one or more specific cut-off points, because The resulting subscale is named ‘‘Size’’ (S). The fourth
factor includes items 3, 9, and 13 and is an index of
perseveration, and constitutes the basis of the ‘‘Persevera-
TABLE 9. Interrater and Test-retest Reliability Coefficients tion’’ (P) subscale. The fifth includes items 6 and 7 and
Interrater Reliability Test-retest Reliability represents corrections of the mistakes thus being the basis
Item (N = 35) (N = 35, after 24 h) of the ‘‘Corrections’’ (C) subscale. The sixth factor
SGST-1 0.95 0.43 includes items 2 (again) and 12 and similarly reflects
SGST-2 0.77 0.50
SGST-3 0.76 0.03
SGST-4 0.70 0.80
SGST-5 0.79 0.44
SGST-6 0.84 0.65
SGST-7 0.88 0.46
SGST-8 0.80 0.74
SGST-9 0.75 0.28
SGST-10 1.00 0.03
SGST-11 0.63 0.68
SGST-12 1.00 0.36
SGST-13 0.84 0.47
SGST 0.80 0.54
DcI 0.79 0.57
ME 0.68 0.71
P 0.84 0.31
C 0.70 0.65
DfI 0.61 0.54
E 0.75 0.61
FIGURE 2. Improvement in the performance in the Graphic
S 0.66 0.59
CI 0.65 0.54 Sequence Test of patients with schizophrenia after 2-months
treatment.

24 r 2008 Lippincott Williams & Wilkins

Cog Behav Neurol  Volume 21, Number 1, March 2008
closing-in, giving rise to the ‘‘Close-in’’ (CI) subscale.
Schizophrenic patients differ in all subscales from controls,
except the last one.
Correlations among these subscales are significant
but weak. The factor analysis of these subscales produced
2 superfactors, named ‘‘Indices.’’ The first (subscales 1, 4,
and 5) constitutes the ‘‘Deficit Index’’ (DcI), whereas the
second (subscales 2, 3, and 6) is the ‘‘Deformation Index’’
(DfI). It is important to note that all the items of the
SGST included in the DcI are easy for the normal subject,
whereas the more difficult ones (2, 5, and 8) are included
in the DfI. Patients differ from controls concerning both
indices (P<0.001). In the frame of the above, the SGST
is divided into the following 2 indices and 6 subscales:
a. DcI that includes the following 3 subscales:
1. Missing Elements (ME) subscale (items 1, 3, and 4).
2. Perseveration subscale (P) (items 3, 9, and 13).
3. Corrections (C) subscale (items 6 and 7).
b. DfI that includes the following 3 subscales:
1. Errors (E) subscale (items 2, 5, and 8).
2. Size (S) subscale ((items 10 and 11).
3. CI subscale (items 2 and 12).
The correlations among the psychometric scales (PANSS,
YMRS, and the MADRS) and individual items and
subscales of the SGST revealed some very interesting
points (Table 3). The PANSS-Positive subscale correlates
negatively only with closing-in. The PANSS-Negative
subscale correlates negatively with fragmentation and
closing-in and positively with lower size for peaks.
PANSS-General Psychopathology correlates negatively
with fragmentation and closing-in. The YMRS is rather
difficult to interpret in schizophrenic patients and in the
current study it was used to have a measure to compare
with bipolar patients in future studies. The MADRS
correlated positively with total missing elements, bigger
size of drawing, and negatively with errors. From the
above it is obvious that the relationship of schizophrenia
and its psychometric profile to the frontal lobe function as
assessed by the SGST is rather complex and nonlinear
and further research is necessary to uncover specific issues
and mechanisms.
The authors believe that future factor analysis with
the inclusion of different patient groups will help to
further elucidate the mechanism underlying the perfor-
mance in the SGST. A preliminary suggestion on the basis
of the data of the current study (Tables 1, 5) could be that
the fragmentation of the drawing is an early sign of
r 2008 Lippincott Williams & Wilkins
Standardized Scoring for the Graphic Sequence Test
disorder followed by closing-in, while intruding elements
appear latter in the procedure. However, this remains to
be tested and needs further and focused research.
Conclusively, the current study developed a reliable,
valid, and sensitive to change instrument for the testing of
frontal lobe function based on Luria’s graphic sequence
test. The great advantage of this instrument is the fact
that it is paper and pencil, easily administered and little
time consuming. Further research is necessary to test its
usefulness as a neuropsychologic test.
ACKNOWLEDGMENTS
The authors thank Dr Symeon Deres, director of the
Asklipeios Clinic, Veroia Greece, for his valuable help in the
recruitment of patients.
REFERENCES
1. Luria A. Higher Cortical Functions in Man. New York: Basic Books;
1966.
2. Luria A. Human Brain and Psychological Processes. New York:
Harper & Row; 1966.
3. Luria A. Traumatic Aphasia. The Hague: Moulton and Co; 1970.
4. Mesulam M. Principles of Behavioral Neurology. Philadelphia: FA
Davis Company; 1985.
5. Strub R, Black F. The Mental Status Examination in Neurology. 2nd
ed. Philadelphia: FA Davis Company; 1989.
6. Wing J, Babor T, Brugha T. SCAN: schedules for clinical
assessment in neuropsychiatry. Arch Gen Psychiatry. 1990;47:
589–593.
7. Kay SR, Opler LA, Lindenmayer JP. The positive and negative
syndrome scale (PANSS): rationale and standardisation. Br
J Psychiatry Suppl. 1989;(7):59–67.
8. Young RC, Biggs JT, Ziegler VE, et al. A rating scale for mania:
reliability, validity and sensitivity. Br J Psychiatry. 1978;133:
429–435.
9. Montgomery SA, Asberg M. A new depression scale designed to be
sensitive to change. Br J Psychiatry. 1979;134:382–389.
10. Anastasi A. Psychological Testing. 6th ed. New York: Macmillan
Publishing Company; 1988.
11. Altman D. Practical Statistics for Medical Research. London:
Chapman and Hall; 1991.
12. Bland J, Altman D. Statistical methods for assessing agreement
between two methods of clinical measurement. Lancet. 1986;1:
307–310.
13. Bartko J, Carpenter W. On the methods and theory of reliability.
J Nervous Mental Disord. 1976;163:307–317.
14. Fotiou F, Fountoulakis K, Goulas A, et al. Automated standardized
pupillometry with optical method for purposes of clinical practice
and research. Clin Physiol. 2000;20:336–347.
15. Fountoulakis KN, Iacovides A, Kleanthous S, et al. Reliability,
validity and psychometric properties of the Greek translation of the
major depression inventory. BMC Psychiatry. 2003;3:2.
25
Fountoulakis et al Cog Behav Neurol  Volume 21, Number 1, March 2008

APPENDIX
Standardized Graphic Sequence Test (SGST)
Fountoulakis et al 2007

Template:

Instruction:
Please copy the above drawing making a perfect identical one

1. Total number of missing elements (max 16)

Scoring Instructions: No. of Missing Score
Elements
The drawing includes 8 (eight) rectangles formations and 8 (eight) peaks which from now on will be called 0 100
‘elements’. Any other line the subject draws eg, a dash is considered an intrusion and is not included when 1 20
counting the elements 2 5
>2 0
2. Number of fragmented connection points (max 39)
Scoring Instructions: No. of Fragmented Score
Points
The drawing should be composed by a continuous line with transition between parts from rectangle to peak 0 100
and vice versa. In case the line is not continuous then there is a fragmentation at the point of contact 1 30
between parts 2 20
3 10
Correct 4 10

Fragmentation 5 8

If there are corrections of the drawing, then the corrected drawing is scored however the number of corrections 6 5
is also registered 7 3
7-12 1
3. Number of rectangles replaced by peaks (max 8) >12 0
Scoring Instructions: No. of Replaced Score
Rectangles
If there are corrections of the drawing then the corrected drawing is scored however the number of corrections 0 100
is also registered 1 4
2 3
3 2
>3 0
4. Number of peaks replaced by rectangles (max 8)
Scoring Instructions: No. of Replaced Score
Peaks
If there are corrections of the drawing then the corrected drawing is scored however the number of corrections 0 100
is also registered >0 0
5. Total number of errors in rectangles (max 16)
Scoring Instructions: No. of Score
Errors
If there are corrections of the drawing then the corrected drawing is scored however the number of corrections 0 100
is also registered. The errors in the rectangles are counted in the following way (max 16) 1 70
2 60
3 50
4 40
5 30
6 20
0
7 15
8 10
1 or
9 8
10 3
2 11 3
12 2
13 2
14 1
>14 0
6. Number of ‘dashes’ or other intrusions between parts
Scoring Instructions: No. of Score
Intrusions
If there are corrections of the drawing then the corrected drawing is scored however the number of corrections 0 100
is also registered 1 14
>1 0

26 r 2008 Lippincott Williams & Wilkins

Cog Behav Neurol  Volume 21, Number 1, March 2008 Standardized Scoring for the Graphic Sequence Test

(continued)
7. Number of corrections
Scoring Instructions: No. of Score
Corrections
If there are corrections of the drawing then the corrected drawing is scored however the number of corrections 0 100
is also registered 1 20

Original drawing 2 4

Corrected 3 1

Considered for scoring >3 0

8. Number of peaks significantly (>50%) lower than expected No. of Score

Peaks
0 100
1 30
2 10
3 5
4 5
5 1
>5 0
9. The drawing does not lie on a straight line Score
On straight 100
Not straight 20
10. The drawing is significantly (>30%) smaller than the template Score
Not smaller 100
Smaller 5
11. The drawing is significantly (>30%) bigger than the template Score
Not bigger 100
Bigger 10
12. The drawing is closing in
Scoring Instructions: Score
In case the drawing of the subjects comes too close or even in touch to the original template or even No close in 100
they overlap then there is a close-in Close in 0
13. There are extra elements No. of extra Score
elements
0 100
>0 0
Deficit Index (DcI)
Missing Elements (ME) subscale (items 1, 3 and 4)
Perseveration subscale (P) (items 3, 9 and 13)
Corrections (C) subscale (items 6 and 7)
Deformation Index (DfI)
Errors (E) subscale (items 2, 5 and 8)
Size (S) subscale (items 10 and 11)
Close-in (CI) subscale (items 2 and 12)

r 2008 Lippincott Williams & Wilkins 27

View publication stats