Original Paper
Eur Neurol 2010;63:364–369
DOI: 10.1159/000292498
Validation of the FOUR Score
(Spanish Version) in Acute Stroke:
An Interobserver Variability Study
Luis Idrovo a Blanca Fuentes a Josmarlin Medina a Laura Gabaldón a
Gerardo Ruiz-Ares a María José Abenza a María José Aguilar-Amat a
Patricia Martínez-Sánchez a Luis Rodríguez a Rubén Cazorla a Marta Martínez a
Alfonso Tafur b Eelco F.M. Wijdicks b Exuperio Diez-Tejedor a
a
Department of Neurology, Stroke Unit, La Paz University Hospital, Autonoma University of Madrid, IdiPAZ,
Madrid, Spain; b Department of Neurology, Division of Critical Care Neurology, Mayo Clinic College of Medicine,
Rochester, Minn., USA
Key Words
Glasgow Coma Scale ⴢ Full Outline of UnResponsiveness
score ⴢ Transient ischemic attack ⴢ Acute stroke
Abstract
Background: Methods to assess impaired consciousness in
acute stroke typically include the Glasgow Coma Scale (GCS),
but the verbal component has limitations in aphasic or intu-
bated patients. The FOUR (Full Outline of UnResponsiveness)
score, a new coma scale, evaluates 4 components: eye and
motor responses, brainstem reflexes and respiration. We
aimed to study the interobserver variability of the FOUR
score in acute stroke patients. Methods: We prospectively
enrolled consecutive patients with acute stroke admitted
from February to July 2008 to the stroke unit of our Neurol-
ogy Department. Patients were evaluated by neurology res-
idents and nurses using the FOUR score and the GCS. For
both scales, we obtained paired and total weighted kappa
values (Kw) and intraclass correlation coefficients (ICC). NIH
stroke scale was also recorded on admission. Results: We ob-
tained a total of 75 paired evaluations in 60 patients (41 ce-
rebral infarctions, 15 cerebral hemorrhages and 4 transient
© 2010 S. Karger AG, Basel
0014–3022/10/0636–0364$26.00/0
Fax +41 61 306 12 34
E-Mail karger@karger.ch Accessible online at:
www.karger.com www.karger.com/ene
Received: November 12, 2009
Accepted: February 22, 2010
Published online: June 16, 2010
ischemic attacks). Thirty-three (55%) patients were alert, 17
(28.3%) drowsy and 10 (16.7%) stuporous or comatose. The
overall rater agreement was excellent in the FOUR score (Kw
0.93; 95% CI 0.89–0.97) with an ICC of 0.94 (95% CI 0.91–0.96)
and in the GCS (Kw 0.96; 95% CI 0.94–0.98) with an ICC of 0.96
(95% CI 0.93–0.97). A good correlation was found between
the FOUR score and the GCS (␳ 0.83; p ! 0.01) and between
the FOUR score and the NIH stroke scale (␳ –0.78; p ! 0.001).
Conclusions: The FOUR score is a reliable scale for evaluating
the level of consciousness in acute stroke patients, showing
a good correlation with the GCS and the NIH stroke scale.
Copyright © 2010 S. Karger AG, Basel
Introduction
Acute stroke may possibly affect brain structures that
control consciousness. Initially, consciousness is rarely
impaired in acute stroke, unless it is strategically located
in the dorsal brainstem or when the brainstem shifts
from mass effect. Coma or any altered state of conscious-
ness predicts the outcome and thus is an important clin-
ical variable [1–3]. There are only limited methods for the
Exuperio Diez-Tejedor, MD, PhD
Department of Neurology, La Paz University Hospital
Paseo de la Castellana 261, Planta 11
ES–28046 Madrid (Spain)
Tel. +34 917 277 189, Fax +34 917 277 444, E-Mail hplapazneuro @ meditex.es
assessment of impaired consciousness in acute stroke,
and they typically include the NIH stroke scale and the
Glasgow Coma Scale (GCS). The NIH stroke scale broad-
ly defines consciousness as alert, not alert with or without
‘stimulation’, and coma, and subsequently evaluates com-
prehension and the response to 2 questions. The GCS is
the most widely used coma scale, but the verbal compo-
nent in particular becomes much less reliable in patients
with stroke due to dysphasia and dysarthria or is impos-
sible to assess in intubated patients [4–6]. Clearly, there is
a need for a more comprehensive coma scale that could
be useful in evaluating stroke patients, but prior attempts
to modify or replace the GCS have not been successful
[7–9]. The Full Outline of UnResponsiveness (FOUR)
score, a new coma scale developed in the Mayo Clinic,
evaluates 4 components: eye and motor responses, brain-
stem reflexes and respiration, and has been recently vali-
dated in acute neurological patients in an intensive care
unit setting [10, 11]. This new scale provides greater neu-
rological detail than the GCS and predicts the outcome
[10], but its reliability has not been specifically studied in
the stroke unit setting. Our aim was to study the interob-
server variability of a Spanish version of the FOUR score
in acute stroke patients.
Patients and Methods
We prospectively studied consecutive adult patients (118 years
old) with diagnosis of acute stroke (less than 48 h after the onset
of symptoms) admitted to the stroke unit of our Neurology De-
partment. This study was approved by the Clinical Research Eth-
ical Committee (HULP PI-644), and written informed consent
for the use of patient data was obtained from the patients or their
relatives. The validation process was divided into 2 phases.
Phase 1
This was a 4-week standardized process which included the
following steps: first, 2 translators (A.T. and L.I.) independently
translated the original FOUR instructions. Then, a third transla-
tor (P.M.-S.) created a draft of the Spanish translation of the
FOUR score that was subsequently presented to 3 Spanish-speak-
ing physicians for comments and suggestions (G.R.-A., M.J.A.
and B.F.)which led to a final Spanish version (fig.  1). The final
Spanish version was then translated back into English (J.M.).
Phase 2
We prospectively enrolled consecutive stroke patients between
February and July 2008. According to our stroke unit’s hospital-
ization clinical pathway, we admit acute stroke patients (isch-
emic/hemorrhagic stroke and transient ischemic attack; TIA)
with sudden neurological symptoms of less than 48 h after onset.
Patients with more than 48 h since the onset of stoke symptoms
were not included in this study analysis. On the day of admission,
the patients were evaluated with the GCS, the FOUR score and the
FOUR Score in Acute Stroke
NIH stroke scale (only by neurologists with NIH stroke scale cer-
tification), and 15 patients were reevaluated with the same scales
during hospitalization because of changes in their levels of con-
sciousness (LOC) (‘mental status’).
Patients were classified into 1 of 4 clinical categories (alert,
drowsy, stuporous and comatose) using definitions proposed by
Ropper [12] and were rated on both scales (FOUR score and GCS)
by 2 raters (2 neurology residents, R/R, or 2 nurses, N/N, or the
combination of a resident and a nurse, R/N). Each pair of raters
performed their examination (the order in which the raters exam-
ined the patients was random, with equal probability for either
observer to be the first or second) within 1 h of each other, without
knowledge of the other’s scores [10]. Six neurology residents (3rd
or 4th year) and 6 neuroscience nurses with 2–5 years of experi-
ence in stroke patient management participated in the data col-
lection. Raters watched a 15-min instructive presentation on both
scales that used videotaped patient examples and a detailed de-
scription of the scales’ components. Raters were provided with
pocket instruction cards for the FOUR score, scoring sheets with
detailed descriptions of both scales, and were asked to rate a few
patients before starting the validation process.
Statistical Analysis
For a power of 80% and a CI of 95% a total of 20 evaluations
for each rater combination were calculated. Cronbach’s ␣ was cal-
culated for each score to assess internal consistency, and Spear-
man’s correlation coefficients between the FOUR score and the
GCS were calculated to assess construct validity. For both scales,
we measured paired and total weighted kappa (Kw) values and
intraclass correlation coefficients (ICC) for overall scores with a
CI of 95%. Kappa does not take into account the degree of dis-
agreement between observers and all disagreement is treated
equally as total disagreement. We used Kw values to take into ac-
count the inherent incremental severity of the score despite not
being a continuum.
Kappa statistic values of 0.4 or less are considered poor, those
between 0.4 and 0.6 are considered fair to moderate, those be-
tween 0.6 and 0.8 suggest good interobserver agreement and those
greater than 0.8 suggest excellent agreement [13]. SPSS 16.0 for
Windows was used for the descriptive analysis, internal consis-
tency, construct validity and ICC calculations, and MedCalc
9.5.2.0 for Windows was used for Kappa statistics.
Results
We enrolled 60 patients with an average age of 69.5 8
14.4 years (median 75; range 31–88); 36 (60%) were men.
None of the patients received sedative drugs or neuro-
muscular blocking agents prior to neurological evalua-
tion. Neurological evaluation revealed that 33 (55%) pa-
tients were alert, 17 (28.3%) were drowsy and 10 (16.7%)
were stuporous or comatose. Patients scored an average
NIH stroke scale of 11.4 8 9.1 points (range 0–36). Forty-
one (68.3%) patients had a cerebral infarction, and the
distribution of ischemic stroke subtypes according to the
TOAST classification was: 16 with cardioembolism, 13
with small-vessel occlusion, 6 with large-artery athero-
Eur Neurol 2010;63:364–369 365
 Color version available online

Fig. 1. a Pocket instruction card of the

FOUR score (Spanish version).

366 Eur Neurol 2010;63:364–369 Idrovo et al.


ESCALA FOUR
Instrucciones para la valoración de las categorías individuales
Para la respuesta ocular (O), puntuar la mejor respuesta posi-
ble (de al menos tres intentos) intentando obtener el mejor nivel
de conciencia posible. Una puntuación O4 indica al menos tres
movimientos oculares voluntarios. Si se encontrara con los ojos
cerrados, el examinador deberá abrírselos y evaluar el seguimien-
to ocular ayudado con un objeto o con su dedo. El seguimiento
ocular con la apertura de un solo ojo será suficiente en casos de
edema palpebral o traumatismo facial. Si el seguimiento horizon-
tal está ausente, examine el seguimiento vertical. Alternativamen-
te, deberá demostrar parpadeo (en dos ocasiones) mediante una
orden verbal. Esto identificará un síndrome de cautiverio. Una
puntuación O3 indica la ausencia de seguimiento voluntario con
los ojos abiertos. Una puntuación O2 indica apertura ocular con
estímulo verbal. Una puntuación O1 indica apertura ocular con un
estímulo doloroso. O0 indica ausencia de apertura ocular con el
estímulo doloroso.
Para la respuesta motora (M), puntuar la mejor respuesta po-
sible en los brazos. Una puntuación M4 indica que el paciente rea-
liza por lo menos una de las tres posiciones (señales) de la mano
(pulgar arriba, puño, signo de la paz), pudiendo ser realizadas con
cualquier mano. Una puntuación M3 indica que el paciente ha to-
cado la mano del examinador luego que este haya realizado un
estímulo doloroso mediante la estimulación de la articulación
temporomandibular o del nervio supraorbitario. Una puntuación
M2 indica movimientos de flexión de las extremidades superiores.
Una puntuación M1 indica respuesta extensora al dolor. Si no hay
respuesta motora al dolor o el paciente presenta un estatus epilép-
tico mioclónico se puntuará como M0.
Para los reflejos del tronco cerebral (T), puntuar la mejor res-
puesta posible. Examine los reflejos pupilares y corneales. Prefe-
rentemente, los reflejos corneales se examinan mediante la aplica-
ción de dos o tres gotas de solución salina en la cornea desde una
distancia de 10–12 cm. La estimulación corneal con algodón puede
también ser utilizada para este propósito. El reflejo tusígeno pro-
vocado por succión traqueal se examina sólo si los reflejos pupila-
res y corneales están ausentes. Una puntuación T4 indica la presen-
cia de reflejos pupilares y corneales normales. Una puntuación T3
indica una pupila midriática y fija. Una puntuación T2 indica ausen-
cia del reflejo pupilar o corneal (cualquiera de los dos). Una pun-
tuación T1 indica ausencia de ambos reflejos. T0 indica ausencia de
reflejos pupilar corneal y tusígeno (utilizando succión traqueal).
Para la respiración (R), evaluar el patrón espontáneo de respi-
ración en un paciente no intubado y puntúe simplemente como
regular (R4), o irregular (R2), y respiración de Cheyne-Stokes (R3).
En pacientes ventilados mecánicamente, evalúe la onda de pre-
sión del patrón espontáneo de respiración o del activador del ven-
tilador mecánico (R1). El monitor del ventilador que muestre los
diferentes patrones respiratorios puede ser utilizado para identifi-
car las respiraciones generadas por el paciente con el ventilador
mecánico. Mientras el paciente está siendo evaluado, no se debe-
rán realizar ajustes de ventilación, y la puntuación se realizara pre-
ferentemente con una PaCO2 dentro de límites de normalidad. Un
examen estándar de apnea (difusión de oxígeno) puede ser nece-
sario cuando el paciente respire a la velocidad del ventilador me-
cánico (R0).
Fig. 1. b Spanish translation of the instructions for the evaluation
of the individual categories of the FOUR score.
FOUR Score in Acute Stroke
sclerosis, 4 of other determined etiology and 2 of unde-
termined etiology. Fifteen (25%) patients had a spontane-
ous cerebral hemorrhage (5 deep, 4 lobar, 2 massive, 2
cerebellum, 1 brainstem and 1 subarachnoid hemor-
rhage) and 4 (6.7%) had a TIA. Our 4 patients with TIA
included in the study were evaluated on admission after
regression of the major neurological symptoms. TIA
symptoms usually last less than 1 h; therefore, our TIA
patients were alert and asymptomatic at the moment of
evaluation. Two patients required endotracheal intuba-
tion, and 1 patient was diagnosed with locked-in syn-
drome. We obtained a total of 75 paired evaluations. Six-
ty evaluations were collected within 24 h of admission
and 15 were performed when changes in responsiveness
urged neurological examination.
The overall rater agreement was excellent for both the
FOUR score (Kw 0.93, 95% CI 0.89–0.97; ICC 0.94, 95%
CI 0.91–0.96) and the GCS (Kw 0.96, 95% CI 0.94–0.98;
ICC 0.96; 95% CI 0.93–0.97). Rater agreement was the
highest among neurology residents and lower Kw scores
were obtained by neuroscience nurses for both scales (ta-
ble 1). The brainstem component of the FOUR score and
the verbal component of the GCS had the lowest total
scores (Kw 0.66 and 0.86, respectively). Nevertheless,
good-to-excellent interobserver agreement was observed
on these scale components.
Rater agreement by LOC was comparable in stuporous
and comatose patients in both scales [FOUR score: 0.83
(95% CI 0.67–0.99); GCS: 0.83 (95% CI 0.61–0.99)]. In
alert and particularly in drowsy patients, we found less
interobserver agreement. Total Kw values of the FOUR
score and the GCS for the alert group were 0.70 (95% CI
0.46–0.99) and 0.84 (95% CI 0.72–0.96); for the drowsy
group the values were 0.43 (95% CI 0.11–0.75) and 0.76
(95% CI 0.56–0.96).
Interobsever agreement by type of stroke was similar
in both scales. The total Kw scores of the FOUR score for
the ischemic and hemorrhagic stroke groups were 0.89
(95% CI 0.72–0.99) and 0.96 (95% CI 0.92–0.99), respec-
tively; and for the GCS the scores were 0.97 (95% CI 0.95–
0.99) and 0.94 (95% CI 0.88–0.99), respectively.
A high degree of internal consistency was found by
measuring Cronbach’s ␣ for the FOUR score (0.84 for the
first rater; 0.85 for the second rater) and for the GCS (0.86
for the first rater; 0.87 for the second rater). Spearman’s
correlation coefficients between GCS and FOUR scores
were 0.83 for the first rater and 0.82 for the second rater
(p ! 0.01) (fig. 2a).
Spearman’s correlation coefficients between total NIH
stroke scale and FOUR scores showed an expected in-
Eur Neurol 2010;63:364–369 367
 16 16
15 15
14 14
13 13
12 12
11 11
10 10

FOUR score
FOUR score

9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0

3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7
a GCS b NIH stroke scale LOC

Fig. 2. a Correlation between the FOUR score and the GCS (␳ = 0.83, p ! 0.01). b Correlation between the FOUR
score and the NIH stroke scale’s LOC item (␳ –0.78, p ! 0.001).

Table 1. Interobserver variability of the FOUR score and the GCS estimated with Kw scores

Number FOUR score GCS

of evalua-
tions ocular motor brainstem respiratory total ocular motor verbal total

R/R 26 0.79 0.93 0.79 1 0.95 0.88 0.96 0.92 0.97

R/N 24 0.93 0.93 0.56 0.65 0.89 0.91 0.97 0.93 0.97
N/N 25 0.77 0.67 0.50 0.65 0.72 0.70 0.81 0.67 0.73
Total 75 0.85 0.92 0.66 0.91 0.93 0.90 0.96 0.86 0.96
95% CI 0.77–0.90 0.88–0.95 0.52–0.77 0.6–0.94 0.89–0.97 0.84–0.94 0.94–0.98 0.78–0.94 0.94–0.98

verse correlation (␳ –0.61; p ! 0.001). We also found an
inverse correlation (␳ –0.78; p ! 0.001) between the FOUR
score and the LOC item (1A + 1B + 1C; 0–7 points) of the
NIH stroke scale (fig. 2b).
Discussion
This is the first interobserver variability study of a
Spanish version of the FOUR score in an acute stroke unit
setting, and it shows not only a good correlation between
the GCS and the NIH stroke scale, but also a high degree
of interobserver agreement. Moreover, a Spanish version
of this new coma scale may facilitate its dissemination not
only in Spain but also in Spanish-speaking developing
countries and among Hispanic communities in North
America and Europe.
The FOUR score may have advantages over the GCS
and the LOC item of the NIH stroke scale. The FOUR
score can identify brainstem abnormalities in patients
with stroke of the vertebrobasilar territory and is able to
recognize locked-in syndrome [10, 11]. The FOUR score
is also able to identify signs of brain herniation (e.g.
brain edema in malignant middle cerebral artery in-
farction) that may require prompt neurosurgical proce-
dures [10, 11]. Furthermore, we noted in our patients
that language disturbances such as motor, transcortical
or global aphasia, predominantly due to cardioembolic
stroke, could be misclassified as altered consciousness
by raters using the GCS, but with the FOUR score these

368 Eur Neurol 2010;63:364–369 Idrovo et al.

dysphasic but alert patients were rated more according
to their LOC.
A prior study identified a negative correlation between
the GCS and the NIH stroke scale (␳ –0.57, p ! 0.001), but
no association has been previously established between
the NIH stroke scale and the FOUR score [14]. The FOUR
score and NIH stroke scale (total and LOC item scores)
showed an inverse correlation which was even stronger
than that between the GCS and NIH stroke scale (␳ –0.61
and –0.78 vs. –0.57; p ! 0.001).
We found a high degree of interobserver agreement in
the FOUR score with the best rater agreement among
neurology residents, followed by resident and nurse, and
nursing staff pairs. Individual components of the FOUR
score showed different Kw values (table 1). Brainstem re-
flexes had the lowest agreement among individual com-
ponents of the FOUR score; however, neurology residents
had better scores than neuroscience nurses. Rating of the
eye responses with the FOUR score was comparable be-
tween raters, and it was better than in the original valida-
tion study [10]. Perfect agreement in the respiration com-
ponent was found among neurology residents; however,
the combination of nurses and physicians had difficulty
identifying the irregular breathing and Cheyne-Stokes
respiration patterns seen in 4 patients. Increasing the in-
ternal validity of our study, all exams were completed
within 1 h of the original evaluation. This was the same
methodology as in the original validation of the scale [10].
Use of the FOUR score for evaluating acute stroke pa-
tients may have some limitations. Although this new scale
has been proven simple to apply, previous experience, more
training and perhaps more clinical skills (for brainstem re-
flexes and recognition of respiration patterns) could fur-
ther improve rater agreement [11]. The lowest agreement
was seen in the brainstem component and specifically in
the nurse-nurse evaluation. This further supports an im-
provement in reproducibility linked to level of training.
However, the Kw evaluation remained at least fair.
In our study, half of the patients were alert at the mo-
ment of neurological evaluation. This may have improved
rater agreement values, but in stuporous and comatose
patients we also found excellent agreement among raters.
Another caveat to be considered is that, as occurs with the
GCS, the clinical significance of the error between scores
is not a continuum. Therefore, therapeutic decisions shall
be individualized.
We conclude that the FOUR score is a reliable scale,
has good internal consistency, a good correlation with
the GCS and an inverse correlation with the NIH stroke
scale. We have found excellent interobserver agreement
in both the FOUR score and the GSC when neurology
residents evaluated the stroke patients. Nurses had lower
agreement in both scoring systems; nevertheless, the
agreement between nurses and residents was excellent.
The FOUR score is a reliable tool for evaluating LOC in
patients with acute stroke.
Acknowledgements
We appreciate the work of the staff nurses of the Department
of Neurology of La Paz University Hospital who collaborated in
this study.

References
1 Weir CJ, Bradford AP, Lees KR: The prog-
nostic value of the components of the
Glasgow Coma Scale following acute stroke.
QJM 2003;96:67–74.
2 Tsao JW, Hemphill JC 3rd, Johnston SC,
Smith WS, Bonovich DC: Initial Glasgow
Coma Scale score predicts outcome follow-
ing thrombolysis for posterior circulation
stroke. Arch Neurol 2005; 62:1126–1129.
3 Wijdicks EF, Rabinstein AA: Absolutely no
hope? Some ambiguity of futility of care in
devastating acute stroke. Crit Care Med
2004;32:2332–2342.
4 Teasdale G, Jennett B: Assessment of coma
and impaired consciousness: a practical
scale. Lancet 1974;ii:81–84.
5 Prasad K, Menon GR: Comparison of the
three strategies of verbal scoring of the
Glasgow Coma Scale in patients with stroke.
Cerebrovasc Dis 1998;8:79–85.
6 Levy DE, Bates D, Caronna JJ, Cartlidge NE,
Knill-Jones RP, Lapinski RH, Singer BH,
Shaw DA, Plum F: Prognosis in nontraumat-
ic coma. Ann Intern Med 1981;94:293–301.
7 Sternbach GL: The Glasgow coma scale. J
Emerg Med 2000;19:67–71.
8 Gill M, Martens K, Lynch EL, Salih A, Green
SM: Interrater reliability of 3 simplified neu-
rologic scales applied to adults presenting to
the emergency department with altered lev-
els of consciousness. Ann Emerg Med 2007;
49:403–407.
9 Healey C, Osler TM, Rogers FB, Healey MA,
Glance LG, Kilgo PD, Shackford SR, Mere-
dith JW: Improving the Glasgow Coma Scale
score: motor score alone is a better predictor.
J Trauma 2003;54:671–678.
10 Wijdicks EF, Bamlet WR, Maramattom BV,
Manno EM, McClelland RL: Validation of a
new coma scale: the FOUR score. Ann Neu-
rol 2005; 58:585–593.
11 Wolf CA, Wijdicks EF, Bamlet WR, McClel-
land RL: Further validation of the FOUR
score coma scale by intensive care nurses.
Mayo Clin Proc 2007;82: 435–438.
12 Ropper AH: Lateral displacement of the
brain and level of consciousness in patients
with an acute hemispheral mass. N Engl J
Med 1986;314:953–958.
13 Landis JR, Koch GG: The measurement of
observer agreement for categorical data. Bio-
metrics 1977;33:159–174.
14 Dominguez R, Vila JF, Augustovski F, Ira-
zola V, Castillo PR, Rotta Escalante R, Brott
TG, Meschia JF: Spanish cross-cultural ad-
aptation and validation of the National Insti-
tutes of Health stroke scale. Mayo Clin Proc
2006;81:476–480.

FOUR Score in Acute Stroke Eur Neurol 2010;63:364–369 369