897122

research-article2020
ASMXXX10.1177/1073191119897122AssessmentRivera et al.

Article
Assessment

Regression-Based Normative Data for

1­–13
© The Author(s) 2020
Article reuse guidelines:
Children From Latin America: Phonological sagepub.com/journals-permissions
DOI: 10.1177/1073191119897122
https://doi.org/10.1177/1073191119897122

Verbal Fluency Letters M, R, and P journals.sagepub.com/home/asm

Diego Rivera1 , Laiene Olabarrieta-Landa1, Wim Van der Elst2,

Isabel Gonzalez3, Rosario Ferrer-Cascales4, Ana Ivis Peñalver Guia5,
Alberto Rodriguez-Lorenzana6, Javier Galarza-del-Angel7 ,
María José Irías Escher8, and Juan Carlos Arango-Lasprilla9,10 ,11

Abstract
This study is part of a larger project to generate norms for letter verbal fluency test (VFT) in 3,284 children from
nine Latin American countries. The letter VFT (letters M, R, and P) was administered and multiple linear regressions,
including age, age2, MPE (mean parental education), MPE2, sex, and interactions were used as predictors. Results showed
significant differences across countries for all scores. Age affected scores linearly except for Ecuador (P-letter), in which
a quadratic effect was found. Scores increased linearly as a function of MPE, with the exception of Mexico (R-letter), in
which a quadratic effect was found. Age by MPE (M- and P-letters), and age by MPE2 (R-letter) interactions were found
in Mexico. Sex had an impact on letter R in Cuba, and letter P in Ecuador and Paraguay. Age2 by sex interaction was
found in Ecuador (P-letter). These norms will be useful for clinical neuropsychologists in these countries to evaluate
their patients’ verbal fluency.

Keywords
regression models, normative data, letter verbal fluency test, Spanish-speaking populations, children

Verbal fluency tasks (VFTs) are widely used in the evalua- Garcia et al., 2012). Verbal fluency, particularly letter flu-
tion of adults’ cognitive functioning and have gained ency tasks, has shown to be a sensitive assessment tool to
increased interest in the neuropsychological assessment of detect several neurocognitive conditions in children, includ-
the pediatric population. Letter VFT is one type of task that ing dyslexia (Malloy-Diniz et al., 2007), dysgraphia
requires the individual to produce, in 60 seconds, as many
words as possible that start with a given letter (A, F, M, P, 1
Departamento de Ciencias de la Salud, Universidad Pública de Navarra,
R, S; Al-Ghatani, Obonsawin, & Al Moutaery, 2009). Pamplona, España
2
This relatively simple, brief, and easy-to-administer task The Janssen Pharmaceutical Companies of Johnson & Johnson, Beerse,
Belgium
requires several higher order functions. These abilities are 3
Albizu University, Miami, FL, USA
related to the executive function, which is thought to be an 4
University of Alicante, Alicante, Spain
umbrella term for several cognitive skills including work- 5
National Institute of Neurology and Neurosurgery INN, Havana, La
ing memory (not repeating the already stated word), atten- Habana, Cuba
6
tion (simple and sustained), inhibition (blocking irrelevant Universidad de Las Américas, Quito, Pichincha, Ecuador
7
Universidad Autónoma de Baja California, Mexicali, Baja California, Mexico
information that comes to mind), and mental switching and 8
Universidad Nacional Autónoma de Honduras, Tegucigalpa, Francisco
flexibility (phonemic clusters; Delis, Kaplan, & Kramer, Morazán, Honduras
2001; Oliviera, Mograbi, Andrade, & Charchat-Fichman, 9
Biocruces Bizkaia Health Research Institute, Barakaldo, Spain
10
2016). When compared with semantic fluency, letter flu- IKERBASQUE, Basque Foundation for Science, Bilbao, Spain
11
ency has been shown to have an added executive demand. Department of Cell Biology and Histology, University of the Basque
Country (UPV/EHU), Leioa, Spain
As letter fluency is more strategic, it needs more cognitive
organization, requiring the individual to consider more than Corresponding Author:
one set of categories (Garcia et al., 2012; McCarthy & Juan Carlos Arango-Lasprilla, Grupo de Psicología y Salud, Biocruces
Bizkaia Health Research Institute, Hospital Universitario Cruces,
Warrington, 1990; Riva, Nichelli, & Devoti, 2000). IKERBASQUE, Basque Foundation for Science, Plaza de Cruces s/n.,
VFT performance has been shown to be a good indicator Barakaldo, Bizkaia 48903, Spain.
of the integrity of the frontal lobe (Al-Ghatani et al., 2009; Email: jcalasprilla@gmail.com
2 Assessment 00(0)
(Sandler et al., 1992), dyscalculia (Sundheim & Voeller,
2004), attention deficit disorder (Garcia et al., 2012;
Willcutt, Pennington, Olson, Chhabildas, & Hulslander,
2005), traumatic brain injury (Goldstrohm & Arffa, 2005),
stroke (Hirsch, 1993), Tourette’s syndrome (Eddy, Rickards,
& Cavanna, 2012), and autism (Begeer et al., 2013).
In children, the ability to organize and retrieve phonemic
information is believed to develop slowly, given the devel-
opment and late maturation of the frontal lobe (Riva et al.,
2000). Since this ability is still developing in children and
adolescents, several authors have highlighted the impor-
tance of considering age as a stratification variable in nor-
mative data (Lezak, Howieson, & Loring, 2004; Riva et al.,
2000; Strauss, Sherman, & Spreen, 2006). Research has
shown the amount of words that are produced in letter VFTs
progressively increase with age. It has been suggested that
the maximum development of those skills, and as a result of
peak performance, does not occur until adolescence or early
adulthood (Klenberg, Korkman, & Lahti-Nuuttila, 2001).
Another similarly imperative variable in the development
of normative data for children is educational level, which
has been shown to be strongly related to VFT performance
(Acevedo et al., 2000; Olabarrieta-Landa, Rivera, et al.,
2017). Educational level appears to be relevant because of
factors such as increased experience with pencil-and-paper
tasks (Ponton et al., 1996) and greater vocabulary knowl-
edge (Ruff, Light, Parker, & Levin, 1997). An interesting
finding that should be considered in the development of nor-
mative data for children is that not only the child’s education
but also higher parental educational level is related to the
child’s improved performance in fluency tasks (Ardila,
Rosselli, Matute, & Guajardo, 2005; Olabarrieta-Landa,
Rivera, et al., 2017; Van der Elst, Hurks, Wassenberg, Meijs,
& Jolles, 2011). Gender differences in VFT performance
have also been evaluated, but its impact is typically insig-
nificant (Malloy-Diniz et al., 2007; Nieto, Galtier, Barroso,
& Espinosa, 2008; Olabarrieta-Landa, Rivera, et al., 2017;
Ruffieux et al., 2009; Van der Elst et al., 2011).
Until now, most of the letter VFT normative data were
established in samples in the United States and Canada
(Gladsjo et al., 1999; Lucas et al., 1998; Mitrushina, 2005;
Strauss et al., 2006; Tombaugh, Kozak, & Rees, 1999). In
the English language, the most commonly used letters in
letter VFT in the United States and Canada are F, A, S
(Anderson, Northam, Hendy, & Wrenall, 2001; Delis et al.,
2001; Mitrushina, 2005; Tombaugh et al., 1999). Although
scarce, other countries have developed normative data for
letter fluency including Italy (Riva et al., 2000), Brazil
(Martins, Mograbi, Andrade Gabrig, & Charchat-Fichman,
2016), Sweden (Tallberg, Carlsson, & Lieberman, 2011),
Netherlands (Van der Elst et al., 2011), and Hong Kong
(Lee, Yuen, & Chan, 2002).
Important factors that have often been overlooked in ver-
bal fluency normative studies are culture and language.
Cognitive development and the approach to neuropsycho-
logical assessments seem to be different across cultures
(Olabarrieta-Landa, Rivera, et al., 2017; Oliviera et al.,
2016). Additionally, verbal fluency performance of lower
socioeconomic groups was found to be impacted by the
teacher’s qualification and the availability of libraries
(Ardila et al., 2005). Based on these findings, the develop-
ment of verbal fluency normative data should take into con-
sideration the performance of children with similar
sociodemographic factors (Olabarrieta-Landa, Rivera,
et al., 2017; Tallberg et al., 2011).
Unfortunately, only a handful of studies have estab-
lished normative data for Spanish-speaking children and
adolescents in Latin American countries, such as Colombia
(Beltran Dulcey & Solis-Uribe, 2012), Argentina (Butman,
Allegri, Harris, & Drake, 2000; Marino & Alderete, 2010),
and Mexico (Lozano & Ostrosky-Solis, 2006; Nieto et al.,
2008). Most of these studies have several limitations, in
particular small sample sizes. Recently Olabarrieta-Landa,
Rivera, et al. (2017) conducted a methodologically sound
normative study in nine Latin American countries. In the
latter study, normative data for letter VFTs using the let-
ters F, A, and S were derived based on a large sample size
(N = 4,373).
The development of appropriate norms for Spanish
speakers is imperative as professionals continue using
norms based on Western and European populations.
Moreover, the letters typically used with English speakers
(e.g., F, A, S) have been shown to be less than optimal for
Spanish speakers. Nonetheless, these continue to be used in
VFT assessments (Olabarrieta-Landa, Torre, López-
Mugartza, Bialystok, & Arango-Lasprilla, 2017). The
amount of words produced with letters F, A, S in Spanish
speakers has been shown to be less than in English, since
the Spanish language has less words that start with those
letters (Strauss et al., 2006). Additionally, using the letter S
increases the complexity of the task for children who are in
the process of developing their language abilities, as the let-
ter S, Z, and C have the same phoneme /S/ in some Spanish-
speaking countries (e.g., Mexico; Nieto et al., 2008). Based
on this, some authors have used the letter M instead of S
(Garcia et al., 2012; Nieto et al., 2008). Yet others have
decided to use a combination of the letters M, R, and P,
which seems to be more appropriate for Spanish speakers
(Artiola, Hermosillo, Heaton, & Pardee, 1999; Butman
et al., 2000; Garcia et al., 2012; Marino & Alderete, 2010;
Nieto et al., 2008).
Even though steps have been taken, the limited norma-
tive data currently available are outdated. To complicate
things further, most of the available normative data have a
small statistical power due to their small sample size, which
makes it harder to generalize the results. Therefore, the cur-
rent study was developed to provide innovative normative
data for Spanish-speaking children using the letters M, R, P,
Rivera et al. 3
and mean VFT total score for letter fluency, as these three
letters are believed to be the most appropriate ones for the
Spanish lexicon. This study is part of a larger project with a
sample of N = 3,284 to generate normative data for a series
of neuropsychological measures in healthy subjects aged 6
to 17 years from nine countries in Latin America (Rivera &
Arango-Lasprilla, 2017).
Method
Participants
The original sample consisted of N = 3,370 healthy chil-
dren and adolescents aged 6 to 17 years from nine Latin
American countries (Chile, Cuba, Ecuador, Guatemala,
Honduras, Mexico, Paraguay, Peru, and Puerto Rico).
Nonproportionate quota sampling was used, looking for a
symmetric distribution for strata of age, sex, and type of
school (public or private) for each country. For Cuba, type
of school was not considered in the sampling procedure as
all education is public (Gasperini, 2000). A total of n = 64
participants were excluded from the analyses because there
was no information regarding the educational level of their
parents, and n = 22 were excluded because of incomplete
information, yielding a final sample of 3,284 participants.
The final sample size per country ranged between n = 192
and n = 919. The demographic characteristics of the test
participants (age, sex, parent education, and type of school)
by country can be found in Table 1.
All participants met the following inclusion criteria: (a)
being between the ages of 6 and 17 years, (b) being born
and currently living in the country where the protocol was
administered, (c) Spanish as primary language, (d) having
an intelligence quotient (IQ) of ≥80 according to the test of
nonverbal intelligence (TONI-2; Brown, Sherbenou, &
Johnsen, 2009), (e) having a score of <19 on the Children’s
Depression Inventory (CDI; Kovacs, 1992), and (f) being
enrolled in a regular private or public school.
Participants were excluded based on the following crite-
ria: (a) having a history of a central nervous system disease
associated with neuropsychological problems (e.g., epi-
lepsy), (b) having a history of alcohol abuse and/or con-
sumption of psychotropic substances, (c) having some type
of active or uncontrolled systemic disease associated with
cognitive impairment (e.g., diabetes mellitus), (d) having a
history of psychiatric illness (e.g., major depression), (e)
having severe sensory deficits (e.g., loss of vision) that
affect the performance on the tests, (f) currently taking psy-
chiatric or other medications that could alter cognitive per-
formance, (g) having intellectual or learning disabilities or
other neurodevelopmental disorders, (h) having a history of
pre-, peri-, or postnatal problems (e.g., hypoxia), (i) having
a score of >5 on the Alcohol Use Disorders Identification
Test–Consumption (AUDIT-C; Bush, Kivlahan, McDonell,
Fihn, & Bradley, 1998) for participants aged 12 years and
older, and (j) using psychoactive substances such as heroin,
barbiturates, amphetamines, methamphetamines, or cocaine
in the last 6 months for participants aged 12 years and older.
Procedure
Prior to initiating the assessment, informed consent was
requested for all parents/guardians and children 12 years of
age and older. In the case of children younger than 12 years,
an assent signature was demanded. The neuropsychological
battery, which lasted approximately 120 minutes, was
administered individually in a single day to the participants
at schools and/or universities. All test participants were vol-
unteers who did not receive financial compensation for par-
ticipation. Data collection began in January 2016 and
finished in May 2017. More details can be found in Rivera
and Arango-Lasprilla (2017).
Measures
The participants were required to produce as many words as
possible in 1 minute beginning with the letters M, R, and P.
The VFTs were always administered in the same order, that
is, first letter M, subsequently letter R, and finally letter P.
For administration and scoring, the guidelines of
Olabarrieta-Landa, Rivera, et al. (2017) and Olabarrieta-
Landa, Torre, et al. (2017) were used.
Statistical Analyses
Exploratory Data Analysis. To determine significant differ-
ences between countries on the performance of each of the
VFT scores, three analyses of covariance (ANCOVA) mod-
els were fitted. Country was used as a fixed-effect factor in
the model, with age, mean parental education (MPE), and
sex as covariates. MPE was calculated using the mean year
of education for both parents/guardians. If a child had a
single parent/guardian, the MPE was calculated using her or
his years of education. Benjamini–Hochberg correction
(Benjamini & Hochberg, 1995) was used to conduct the
pairwise comparisons of the VFT test scores across coun-
tries. Furthermore, Pearson correlations between the VFT
scores (including M, R, P total scores, and mean VFT total
score) and the sociodemographic (age, MPE, and sex) vari-
ables were computed. Mean VFT total score was calculated
by summing M, R, and P total scores, and dividing it by 3:
[(M + R + P)/3].
The Effects of Demographic Variables and the Derivation of
Normative Data.  The letters M, R, P, and mean VFT total
scores were computed separately. The effects of demo-
graphic variables on VFT scores were evaluated by means
of multiple linear regression analyses. For each country,
4 Assessment 00(0)

Table 1.  Sample Distribution by Country, Age, MPE, Sex, and Type of School.

Age (Years) MPE Sex Type of school

Girl Boy Public Private

Age range
Country (Years) n M (SD) M (SD) n n n n
Chile 6 to 8 94 7.0 (.8) 13.1 (3.1) 47 47 46 48
  9 to 11 98 10.0 (.8) 12.5 (3.1) 50 48 49 49
  12 to 14 94 13.0 (.8) 12.0 (2.7) 46 48 47 47
  15 to 17 95 16.0 (.8) 11.8 (3.1) 48 47 49 46
  Total 381 11.5 (3.4) 12.4 (3.0) 191 (50.1%) 190 (49.9%) 191 (50.1%) 190 (49.9%)
Cuba 6 to 8 96 7.0 (.8) 15.9 (1.9) 48 48 96 0
  9 to 11 94 10.0 (.8) 15.9 (2.0) 46 48 94 0
  12 to 14 96 13.0 (.8) 16.5 (1.5) 48 48 96 0
  15 to 17 95 16.0 (.8) 16.4 (1.6) 48 47 95 0
  Total 381 11.5 (3.5) 16.2 (1.8) 190 (49.9%) 191 (50.1%) 381 (100%) 0 (0%)
Ecuador 6 to 8 79 7.0 (.8) 14.9 (3.4) 42 37 41 38
  9 to 11 73 9.9 (.8) 14.1 (3.7) 43 30 40 33
  12 to 14 75 13.0 (.8) 14.3 (3.2) 49 26 36 39
  15 to 17 70 16.0 (.8) 14.5 (4.1) 39 31 38 32
  Total 297 11.4 (3.4) 14.4 (3.6) 173 (58.2%) 124 (41.8%) 155 (52.2%) 142 (47.8%)
Guatemala 6 to 8 34 7.3 (1.0) 12.4 (4.1) 14 20 12 22
  9 to 11 94 9.9 (.8) 9.0 (3.4) 41 53 68 26
  12 to 14 52 12.4 (.7) 10.2 (4.1) 24 28 30 22
  15 to 17 22 15.6 (.7) 14.0 (3.9) 15 7 1 21
  Total 202 10.7 (2.5) 10.4 (4.1) 94 (46.5%) 108 (53.5%) 111 (55.0%) 91 (45.0%)
Honduras 6 to 8 70 7.0 (.8) 13.2 (3.6) 36 34 47 23
  9 to 11 96 10.0 (.8) 12.6 (3.6) 48 48 49 47
  12 to 14 70 13.0 (.9) 12.0 (3.9) 42 28 38 32
  15 to 17 56 15.9 (.9) 13.7 (3.3) 33 23 18 38
  Total 292 11.2 (3.2) 12.8 (3.7) 159 (54.5%) 133 (45.5%) 152 (52.1%) 140 (47.9%)
Mexico 6 to 8 247 6.9 (.8) 13.3 (4.1) 128 119 144 103
  9 to 11 229 10.0 (.8) 13.4 (4.0) 114 115 143 86
  12 to 14 218 13.0 (.8) 13.7 (3.9) 114 104 123 95
  15 to 17 225 16.0 (.8) 12.1 (3.5) 118 107 153 72
  Total 919 11.4 (3.5) 13.1 (3.9) 474 (51.6%) 445 (48.4%) 563 (61.3%) 356 (38.7%)
Paraguay 6 to 8 74 7.1 (.8) 13.7 (2.6) 38 36 35 39
  9 to 11 76 10.0 (.8) 13.9 (3.2) 50 26 42 34
  12 to 14 63 13.0 (.8) 14.8 (2.6) 31 32 25 38
  15 to 17 84 16.0 (.8) 13.9 (3.0) 41 43 36 48
  Total 297 11.6 (3.5) 14.1 (2.9) 160 (53.9%) 137 (46.1%) 138 (46.5%) 159 (53.5%)
Peru 6 to 8 66 7.1 (.8) 12.7 (2.3) 32 34 25 41
  9 to 11 76 9.9 (.7) 12.5 (2.4) 33 43 34 42
  12 to 14 96 13.1 (.8) 12.2 (2.5) 47 49 56 40
  15 to 17 85 16.0 (.7) 12.6 (2.5) 48 37 57 28
  Total 323 11.9 (3.3) 12.5 (2.4) 160 (49.5%) 163 (50.5%) 172 (53.3%) 151 (46.7%)
Puerto Rico 6 to 8 46 7.1 (.9) 14.6 (2.5) 29 17 32 14
  9 to 11 31 9.8 (.8) 14.7 (2.1) 18 13 16 15
  12 to 14 54 13.4 (.8) 14.7 (2.6) 28 26 31 23
  15 to 17 61 16.0 (.8) 14.5 (2.6) 32 29 39 22
  Total 192 12.1 (3.6) 14.6 (2.5) 107 (55.7%) 85 (44.3%) 118 (61.5%) 74 (38.5%)

Note. MPE = mean parental education; M = mean; SD = standard deviation.

Rivera et al. 5
separate regression models were fitted to establish country-
specific normative data. The full regression models included
as predictors: age, age2, MPE, MPE2, sex, and all two-way
interactions between these variables. Age and MPE were
centered (= calendar age in years—mean age in the sample
by country; MPE in years—mean education in the sample
by country) before computing the quadratic age and educa-
tion terms to avoid multicollinearity (Kutner, Nachtsheim,
Neter, & Li, 2005). Squared terms of age and years of edu-
cation were added in the full model to allow for quadratic
effects between these independent variables and the letter
scores (i.e., nonlinear relations between age/MPE and the
VLT test scores). Sex was dummy coded as boy = 1 and
girl = 0. The full regression model can be formally
described as 2 residuals was met in the normative sample), or via the
   
yi = B0 + B1 ⋅  Age − x Ageby  + B2 ⋅  Age − x Ageby 
 country i  country i
( ) ( )
2
+ B3 ⋅ MPE − xMPE by country + B4 ⋅ MPE − xMPE .by country
i i
+ B5 ⋅ Sexi + Bk ⋅ Interactionsi + εi , with the subscript i refer-
ring to the participants (i = {1, 2, . . ., ni }, and ni the number
of participants in a country), the subscript k referring to the
regression parameters (k = {1, 2, . . ., K}, and K = the num-
ber of fixed effects in the model including the intercept), and
the term Interactionsi being a shorthand notation to refer to
all two-way interactions between the fixed effects. The
model assumes that the residuals εi are normally distributed
(
with mean 0 and variance σε2 , that is, εi ∼ N 0, σε2 . ) coefficients ( ICCtest − restest ). The intraclass correlation coef-
Independent variables that were not statistically signifi-
cant (using an alpha level of 0.05/13 independent variables
≈ 0.003) in the full multiple regression model (i.e., the
model shown above that includes all independent variables
and their two-way interactions) were removed in a step-
wise hierarchical way. In the model-building procedure,
predictors were not removed as long as they were included
in a higher order term in the model (Aiken & West, 1991).
For all multiple linear regression models, the following
assumptions were evaluated: (a) multicollinearity (evalu-
ated by computing the variance inflation factor [VIF],
which should not be greater than 10, and by computing the
collinearity tolerance values which should not be greater
than 1); (b) homoscedasticity (evaluated by grouping the
participants into quartiles of the predicted test scores and
applying the Levene test on the residuals); (c) normality of
the standardized residuals (evaluated by conducting the
Kolmogorov–Smirnov test); and (d) the existence of influ-
ential values (evaluated by computing the maximum
Cook’s distance values and relating them to an F ( p, n − p )
distribution, for details see Kutner et al. (2005).
Using the final regression model obtained at the end of
the stepwise procedure, normative data were adjusted for
demographic variables. This was established by means of a
four-step procedure (Rivera et al., 2019; Van Breukelen &
Vlaeyen, 2005):
( )
1. The expected test score Yi computed based
on the fixed-effect parameter estimate of
the established final regression model:
Yi = B0 + B1 X 1i + B2 X 2i +  + Bk X ki .
2. To obtain the residual value ( ei ) , the model-pre-
dicted value (Yi ) is subtracted from the raw score of
the neuropsychological test (Yi ) , that is, ei = Yi − Yi .
3. Using the residual standard deviation ( SDe ) value
provided by the regression model, the residual is
standardized: zi = ei / SDe .
4. Finally, using the standard normal cumulative distri-
bution function, the exact percentile value corre-
sponding to the zi -score previously calculated is
obtained (if the model assumption of normality of the
empirical cumulative distribution function of the stan-
dardized residuals (if the standardized residuals were
not normally distributed in the normative sample).
This four-step process was applied to the letters M, R, P,
and mean VFT total scores separately for each country.
Preliminary Estimates of Reliability.  The reliability of each of
the VFT scores for the entire subsample was calculated
through the test–retest method, which evaluates the tem-
poral stability of the test scores. The reliability was calcu-
lated with two-way mixed intraclass correlation
ficient is frequently used to report test–retest reliability
(Weir, 2005). The subsample used to calculate the test–
retest reliability consisted of 166 participants from Chile
(n = 32; 19.9%), Cuba (n = 38; 23.6%), Ecuador (n = 16;
9.9%), and Mexico (n = 75; 46.6%). Of those, 64.0%
were boys and the average age was 10.9 (SD = 3.7) years.
The average time elapsed between the two test administra-
tions was 76.5 (SD = 10.4; range= 360, 126) days. All
analyses were performed using R 3.4.2 for Window (R
Development Core Team, 2017) and SPSS version 23
(IBM Corp., Armonk, NY).
Results
Exploratory Data Analysis
Positive correlations were found between age and all VFT
scores (rs ≥ .33; ps < .01). Moreover, correlations were
found between MPE and (a) all VFT scores in Cuba (rs ≥
.11; ps < .05) and Mexico (rs ≥ .09; ps < .01), and (b) let-
ters M and P (rs ≥ .16; ps < .05) in Guatemala. Sex was
correlated with letter R (Cuba; r = −.16; p < .01), and letter
P (Guatemala [r = −.14; p < .05], Mexico [r = −.07; p <
.05], and Paraguay [r = −.21; p < .01]).
The intercorrelation between letters M, R, and P scores
across countries were also significantly positive (rs ≥ .622
to 0.806; ps < .001). In view of the high correlations
6 Assessment 00(0)

Table 2.  Post Hoc Analysis of the Mean Differences Between Countries in Each VFT Score.

Mean difference (I − J)

(I) Country (J) Country Letter M Letter R Letter P Letter mean VFT
Chile Cuba −.46 −1.46* −.22 −.72*
  Ecuador −.17 −.59* −.10 −.29
  Guatemala −.81* −1.13* −.58 −.84*
  Honduras .16 .003 .23 .13
  Mexico −.46* −.63* .05 −.35*
  Paraguay −.79* −.44 −.45 −.56*
  Peru −1.33* −1.59* −1.28* −1.40*
  Puerto Rico .80* .70* 1.09* .86*
Cuba Ecuador .28 .88* .12 .43
  Guatemala −.36 .33 −.36 −.12
  Honduras .62* 1.46* .45 .85*
  Mexico −.01 .83* .27 .37*
  Paraguay −.34 1.03* −.23 .15
  Peru −.88* −.12 −1.06* −.68*
  Puerto Rico 1.26* 2.17* 1.31* 1.57*
Ecuador Guatemala −.64* −.54 −.48 −.55*
  Honduras .34 .59* .34 .42
  Mexico −.29 −.04 .16 −.06
  Paraguay −.62* .15 −.35 −.27
  Peru −1.16* −1.00* −1.17* −1.11*
  Puerto Rico .98* 1.29* 1.20* 1.14*
Guatemala Honduras .97* 1.13* .81* .97*
  Mexico .35 .50 .63* .49*
  Paraguay .02 .69* .13 .28
  Peru −.52 −.45 −.70* −.56*
  Puerto Rico 1.61* 1.83* 1.67* 1.70*
Honduras Mexico −.62* −.63* −.18 −.48*
  Paraguay −.96* −.43 −.69* −.69*
  Peru −1.50* −1.58* −1.51* −1.53*
  Puerto Rico .64* .70* .86* .72*
Mexico Paraguay −.33 .19 −.51* −.22
  Peru −.87* −.96* −1.33* −1.05*
  Puerto Rico 1.27* 1.33* 1.04* 1.20*
Paraguay Peru −.54* −1.15* −.82* −.84*
  Puerto Rico 1.60* 1.14* 1.55* 1.42*
Peru Puerto Rico 2.14* 2.29* 2.37* 2.25*

Note. Asterisk (*) indicates significant difference according to Benjamini–Hochberg correction. Each comparison is significant if the observed p is
smaller than a critical value defined as Pcrit = ( j / k )α .

between all VFT scores, a mean VFT total score ([M + R +
P]/3) was calculated and normative data were provided for
this overall test score summary metric.
ANCOVAs showed significant differences across the
nine countries for the letter M VFT score (F(8, 3272) = 10.793,
p < .001, partial η2 = .026), the letter R VFT score (F(8, 3265)
= 15.762, p < .001, partial η2 = .037), the letter P VFT
score (F(8, 3264) = 9.663, p < .001, partial η2 = .023), and
the mean VFT total score (F(8, 3264) = 9.663, p < .001, par-
tial η2 = .023). According to Benjamini–Hochberg correc-
tion, from the 36 total pair-comparisons, 22 significant
pair-comparisons were found in letter M, 25 in letter R, 19
in letter P, and 26 in mean VFT (see Table 2).
Model Assumptions
The assumptions of multiple linear regression analysis
were met for most final models. There was no multicol-
linearity, that is, the VIF values in all final models were at
most 3.264 (letter P in Ecuador), and thus, well below the
threshold value = 10 that is indicative for multicollinear-
ity. Collinearity tolerance values did not exceed the value
Rivera et al. 7
of 1. There were no influential cases in any of the models
considered (the maximum Cook’s distance value was
0.10; relating this value to an F(2, 290) distribution yields a
percentile value of 10, which is well below the threshold
percentile value = 50 that is indicative for the presence of
influential cases). The Levene test suggested no homosce-
dasticity in the majority of the final models, except for the
models of Chile (letter R), Cuba (letters M and R),
Ecuador (all scores), Guatemala (all scores), Honduras
(letter P), Paraguay (letter R), and Puerto Rico (letters M
and P). Standardized residuals for all but one of the mod-
els (letter P in Cuba) were normally distributed (all ps of
the Kolmogorov–Smirnov test > .01) so the standard nor-
mal cumulative distribution was used to convert z scores
to percentiles. For letter P in Cuba, the empirical cumula-
tive distribution function of the standardized residual was
used.
The Effects of Demographic Variables
Letter M. For all nine countries the final multiple linear
regression models for letter M scores were significant (see
Table 3). The letter M score increased linearly as a func-
tion of age in all countries. In Mexico, MPE influence on
letter M score, so that children and adolescents with higher
MPE obtained higher scores (see Figure 1A). Sex was not
related to letter M scores in any country. The amount of
2
variance (adjusted R ) explained by these predictors in the
letter M scores ranged from 24.9% (Guatemala) to 50.5%
(Honduras).
Letter R.  The final multiple linear regression models for let-
ter R VFT scores of all nine countries were significant
(Table 3). Letter R scores increased linearly as a function of
age in all countries. In Mexico, a significant interaction was
found between age and MPE, that is, the increase in letter R
scores as a function of age was more pronounced for chil-
dren with a higher MPE (i.e., larger slope for children with
a higher MPE; see Figure 1B). The participant’s sex affected
letter R scores for Cuba, such that girls achieved higher
scores than boys. The amount of variance explained by
2
these predictors (adjusted R ) ranged from 17.8% (Guate-
mala) to 41.4% (Peru).
Letter P.  In all nine countries, final multiple linear regres-
sion models for letter P scores were significant (Table 3). In
all countries letter P scores increased linearly as a function
of age. In Mexico, a significant interaction was found
between age and MPE, so that the MPE influence on chil-
dren and adolescents’ performance became more evident
while they got older (see Figure 1C). The participant’s sex
affected letter P scores for Paraguay, such that girls achieved
higher scores than boys (see Figure 1D). The amount of
variance explained by these predictors (adjusted R 2 ) in the
letter P scores ranged from 10.8% (Guatemala) to 43.4%
(Chile).
Mean VFT Total Score.  For all nine countries, the final multiple
linear regression models of the mean VFT total score were
significant (Table 3). In all countries, the mean VFT total
scores increased linearly as a function of age. In Mexico, a
significant interaction was found between age and MPE, so
that MPE influence on children and adolescents’ performance
became more evident while they got older, with huge dis-
crepancy between 5 and 20 MPE, starting from 12 years old.
The participant’s sex affected the mean VFT total scores for
Cuba and Paraguay, such that girls achieved higher scores
than boys. The amount of variance explained by these predic-
2
tors (adjusted R ) in the mean VFT total scores ranged from
23.0% (Guatemala) to 51.9% (Honduras).
Calculator of Normative Data.  The four-step normative pro-
cedure explained above allows to determine an exact per-
centile value for a VFT test scores by hand. However, this
method can be prone to human error, so to enhance user-
friendliness, the authors created a calculator in Microsoft
Excel that conducts all required computations. The clinician
simply fills-in the country, raw VFT scores and demo-
graphic characteristics of the tested child (i.e., age, MPE,
and sex), and the software automatically computes the stan-
dardized residuals and their corresponding exact percentile
values (see Figure 2). For example, if a 10-year-old Para-
guayan boy, whose parents have an MPE of 10 years of edu-
cation, scored 11 on the letter P, it belongs to the 90th
percentile (z score = 1.305). This tool is freely available for
all users and may be downloaded at https://neuropsycholo-
gylearning.com/datos-normativos-archivos-descargables/.
Preliminary Estimates of Reliability
The interclass coefficient of the VFT score for the subsam-
ple were 0.853 (95% confidence interval [CI] = [0.799,
0.892]; p < .001) for the letter M score, 0.849 (95% CI =
[0.794, 0.889]; p < .001) for the letter R score, 0.863 (95%
CI = [0.813, 0.899]; p < .001) for the letter P score, and
0.906 (95% CI = [0.872, 0.931]; p < .001) for the mean
VFT total score.
Discussion
The main aim of the present study was to establish the nor-
mal range of performance for letters M, R, P, and mean VFT
total score in a large sample of children aged 6 to 17 years
from nine countries in Latin America. As a first step in the
normative procedure, the influence of demographic vari-
ables on test performance was evaluated. Age was consis-
tently found to be the most important predictor of VFT
performance (Table 3), with older children outperforming
8
Table 3.  Final Multiple Linear Regression Models for VFT Scores.

Chile Cuba Ecuador Guatemala Honduras Mexico Paraguay Peru Puerto Rico

Letter B (β) B (β) B (β) B (β) B (β) B (β) B (β) B (β) B (β)
M (Intercept) 7.969** 8.842** 8.332** 7.978** 7.665** 8.446** 9.040** 9.603** 7.876**
  Age .741 (.648)** .730 (.643)** .694 (.637)** .670 (.503)** .839 (.712)** .715 (.637) ** .675 (.550)** .843 (.635)** .566 (.573)**
  MPE — — — — — .155 (.155)** — — —
  Adjusted R2 .419 .412 .403 .249 .505 .411 .300 .401 .325
R (Intercept) 7.15** 9.735** 7.935** 7.52** 7.02** 7.849** 7.86** 9.018** 7.129**
  Age .674 (.600)** .785 (.621)** .705 (.644)** .531 (.427)** .678 (.618)** .676 (.611)** .517 (.484)** .839 (.645)** .548 (.582)**
  MPE — — — — — .19 (.193)** — — —
  Sex −1.393 (−.159) — — — — — — —
  Age × MPE — — — — — .039 (.138)** — — —
  Adjusted R2 .359 .409 .413 .178 .380 .386 .231 .414 .335
P (Intercept) 9.132** 9.757** 9.438** 8.893** 8.754** 9.122** 1.872** 1.718** 8.717**
  Age .847 (.660)** .796 (.630)** .759 (.641)** .442 (.336)** .768 (.645)** .744 (.633)** .638 (.497)** .870 (.629)** .633 (.581)**
  MPE — — — — — .158 (.151)** — — —
  Sex — — — — — — −2.175 — —
(−.242)**
  Age × MPE — — — — — .024 (.081)* — — —
  Adjusted R2 .434 .395 .413 .108 .414 .400 .286 .394 .335
Mean VFT (Intercept) 8.084** 9.625** 8.568** 8.130** 7.813** 8.481** 9.516** 9.780** 7.933**
  Age .754 (.699)** .769 (.693)** .719 (.710) ** .548 (.484) ** .762 (.722)** .714 (.693)** .615 (.557)** .851 (.685)** .582 (.644)**
  MPE — — — — — .170 (.185)** — — —
  Sex — −.817 (−.107)** — — — — −1.285 — —
(−.167)**
  Age × MPE — — — — — .028 (.105)** — — —
  Adjusted R2 .488 .490 .502 .230 .519 .484 .323 .467 .412

Note. VFT = verbal fluency test; MPE = mean parental education; B = Beta unstandardized; β = Beta standardized. “—” indicates independent variable no significant in the regression model.
*p = .002. **p < .001.
Rivera et al. 9

Figure 1.  Predicted mean letter scores as a function of (A) mean parental education (MPE) for the letter M from the Mexican
sample (A); age and MPE interaction for the letter R (B) and the letter P (C) from the Mexican sample; and sex for the letter P from
Paraguayan sample (D).

their younger counterparts. Interestingly, the relation
between age and test performance was linear for all three
VFT scores in all nine countries. This means that the yearly
average change in the test scores remains constant over the
entire age range considered. Specifically, the cognitive abil-
ities assessed by verbal fluency tests seem to develop at a
constant rate during the entire age range considered in this
study (6-17 years). Sex and MPE also had a significant
effect on some VFT scores, but the effects of these indepen-
dent variables were not consistent across countries. Overall,
the correlations between MPE/sex and the three VFT scores
were low and never exceeded r = .22. Pearson univariate
correlations showed weak relations that disappeared at the
final multivariate regression models (i.e., Cuba with all let-
ters vs. MPE, and Guatemala with sex in letter P).
These results are in line with previous studies where age
was the most important and substantial influencer on letter
verbal fluency (e.g., Ruffieux et al., 2009; Tallberg et al.,
2011), while the impact of MPE and sex tends to be minor
(Olabarrieta-Landa, Rivera, et al., 2017; Van der Elst et al.,
2011). In pediatric populations, neurodevelopmental
aspects are essential to understand cognitive performance.
Verbal fluency, especially letter fluency task, requires the
development of executive functions and the frontal lobe,
both keeping their processes of maturation until young
adulthood (Klenberg et al., 2001; Korkman, Kemp, & Kirk,
2001; Malloy-Diniz et al., 2007). Thus, it is not surprising
that age has such an impact on this task. Moreover, letter
verbal fluency requires greater organizational and strategic
skills than semantic verbal fluency, and these abilities are
thought to develop slowly in children (Riva et al., 2000).
Literature suggests that the environment in which chil-
dren grow up may influence their future cognitive develop-
ment, especially MPE (Meador et al., 2011; Schady, 2011),
and parents’ occupation (Dubow, Boxer, & Huesmann,
2009). MPE is thought to help in the more advanced vocab-
ulary development of children through the mediation of
responsive and clear communication with the child
10 Assessment 00(0)
Figure 2.  Calculator in Microsoft Excel.
(Sohr-Preston et al., 2013). Finally, sex influence was very
small compared with age impact, and even though discrep-
ancies can be found in the literature regarding its influence,
sex differences in verbal abilities appear to be almost non-
existent (Hyde & Linn, 1988).
Importantly, even though different countries in Latin
America are comparable in terms of language spoken and
general cultural background, there were major differences
in VFT performance (see Table 2). For example, the age-,
sex-, and MPE-corrected letter M score of 8 equaled 30th
and 52th percentiles for Peru and Puerto Rico, respec-
tively. A typical inhabitant of Peru thus has 22 centiles of
difference compared with his or her counterpart from
Puerto Rico. This finding clearly stresses the importance
of having country-specific normative data for Latin
American countries. Several variables that may impact
letter verbal fluency, which were not assessed in this study,
could explain these differences in scores among countries.
For instance, language-related competences (e.g., vocabu-
lary level, reading, and writing) have been associated with
better performance in letter verbal fluency (Kochhann,
Holz, Beber, Chaves, & Fonseca, 2018; Olabarrieta-
Landa, Rivera, et al., 2017; Olabarrieta-Landa, Torre,
et al., 2017). In countries with bilingual population, such
as Puerto Rico, bilingualism might be an important vari-
able since it has been also related to letter verbal fluency,
with no clear results: While some studies have reported
that monolinguals outperform bilingual participants on
letter verbal fluency (Bialystok, Craik, & Luk, 2008;
Gollan, Montoya, & Werner, 2002), others have found that
bilingual participants outperform monolinguals when
vocabulary level is similar among both groups (Friesen,
Luo, Luk, & Bialystok, 2015; Luo, Luk, & Bialystok,
2010). Also, according to Manly, Touradji, Tang, and Stern
(2003), quality of education is a stronger predictor of cog-
nitive performance than years of education (variable that
has been considered in this study). In Latin America,
countries differ according to socioeconomic and education
systems (e.g., Puerto Rico vs. Cuba) that may impact qual-
ity of education, and thus, cognitive performance.
Finally, as Strauss et al. (2006) pointed out, the selection
of letters during this task will determine its difficulty.
Looking at final multiple linear regressions of letters M, R,
and P, their intercepts remain, in general, higher compared
with the constants of letters F, A, and S published by
Olabarrieta-Landa, Rivera, et al. (2017) with Spanish-
speaking children from the same Latin American countries.
For example, for Ecuador, the constant of letter S was 7.361
(Olabarrieta-Landa, Rivera, et al., 2017) while the constant
of letter M in the current study was 8.332. Hence, children
produce more words if a combination of the letters M, R,
and P is used. This supports the idea that those letters are
more appropriate to be used with Spanish speakers. In addi-
tion, MRP letters showed high temporal consistency (r ≥
.849), according to cutoff points proposed by Strauss et al.
(2006) for intraclass test–retest correlation coefficients.
Limitations
Several limitations of the current study can be given. First,
participants in this study were native Spanish speakers and
no data were collected on bilingualism. As stated above,
Rivera et al. 11
several studies have shown how verbal fluency test perfor-
mance may be influenced by this variable (Bialystok et al.,
2008; Friesen et al., 2015; Luo et al., 2010). Second, the
results of this study cannot be generalized to other Latin
American countries that were not included in this study
(such as Venezuela and Uruguay) or other countries where
people from these same countries are living (such as
United States of America). Third, even though in Mexico
the sample was collected from several geographic areas of
the country, for the rest of the countries of the study, only
people from a specific geographic area were recruited.
Last, normative data were generated using healthy chil-
dren and adolescents. Future studies should include clini-
cal samples (e.g., children with attention deficit/
hyperactivity disorder, brain injury, epilepsy) to obtain
proper cutoff points to calculate the sensitivity and speci-
ficity of VFT.
Conclusion
The current study offers verbal fluency test norms across
nine countries in Latin America with 3,284 healthy chil-
dren and adolescents aged 6 to 17 years. Most normative
data studies have focused on adult populations, so to
date, this represents the largest normative data study of
neuropsychological tests for children and adolescents in
the world.
In addition, for the creation of the normative data, more
up-to-date approaches have been used, such as multiple
regressions and residual values. Moreover, age and MPE
were introduced into the model as continuous variables.
Therefore, these norms are expected to be useful for those
who work in neuropsychological evaluation of children and
adolescents in these countries.
Author’s Note
Isabel Gonzalez is now affiliated with University of Colorado
School of Medicine, Aurora, Colorado.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect
to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
ORCID iDs
Diego Rivera https://orcid.org/0000-0001-7477-1893
Javier Galarza-del-Angel https://orcid.org/0000-0001-5865
-5198
Juan Carlos Arango Lasprilla https://orcid.org/0000-0002
-7184-8311
References
Acevedo, A., Loewenstein, D. A., Barker, W. W., Harwood, D.
G., Luis, C., Bravo, M., . . . Duara, R. (2000). Category flu-
ency test: Normative data for English- and Spanish-speaking
elderly. Journal of the International Neuropsychological
Society, 6, 760-769.
Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing
and interpreting interactions. Thousand Oaks, CA: Sage.
Al-Ghatani, A. M., Obonsawin, M. C., & Al Moutaery, K.
R. (2009). Normative data for the two equivalent forms
of the Arabic verbal fluency test. Pan Arab Journal of
Neuropsychology, 13(2), 57-65.
Anderson, V., Northam, E., Hendy, J., & Wrenall, J. (2001).
Developmental Neuropsychology: A clinical approach. New
York, NY: Psychology Press.
Ardila, A., Rosselli, M., Matute, E., & Guajardo, G. (2005). The
influence of parents’ educational level on the development of
executive functions. Developmental Neuropsychology, 28,
539-560.
Artiola, L., Hermosillo, D., Heaton, R., & Pardee, R. E. (1999).
Manual de normas y procedimientos para la bateria neu-
ropsicologica en espanol [Manual of norms and procedures
for neuropsychological battery in Spanish]. Tucson, AZ:
m Press.
Begeer, S., Wierda, M., Scheeren, A. M., Teunisse, J. P., Koot, H.
M., & Geurts, H. M. (2013). Verbal fluency in children with
autism spectrum disorders: Clustering and switching strate-
gies. Autism, 18, 1014-1018. doi:10.1177/1362361313500381
Beltran Dulcey, C., & Solis-Uribe, G. (2012). Evaluación
Neuropsicológica en adolescentes: Normas para población de
Bucaramanga [Neuropsychological Assessment in adolescents:
Norms for Bucaramanga population]. Revista Neuropsicología,
Neuropsiquiatría y Neurociencias, 12(2), 77-93.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false
discovery rate: A practical and powerful approach to mul-
tiple testing. Journal of the Royal Statistical Society: Series
B (Methodological), 57, 289-300. doi:10.1111/j.2517-6161
.1995.tb02031.x
Bialystok, E., Craik, F., & Luk, G. (2008). Cognitive control and
lexical access in younger and older bilinguals. Journal of
Experimental Psychology: Learning, Memory and Cognition,
34, 859-873.
Brown, L., Sherbenou, R. J., & Johnsen, S. K. (2009). Test of non-
verbal intelligence. Austin, TX: Pro-Ed.
Bush, K., Kivlahan, D. R., McDonell, M. B., Fihn, S. D., &
Bradley, K. A. (1998). The AUDIT alcohol consumption
questions (AUDIT-C): An effective brief screening test
for problem drinking. Archives of Internal Medicine, 158,
1789-1795.
Butman, J., Allegri, R. F., Harris, P., & Drake, M. (2000).
Fluencia verbal en español. Datos normativos en Argentina
[Verbal fluency in Spanish. Normative data from Argentina].
Medicina, 60, 561-564.
Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan
Executive Function System (DKEFS), Examiner’s manual.
San Antonio, TX: Psychological Corporation.
Dubow, E. F., Boxer, P., & Huesmann, L. R. (2009). Long-term
effects of parents’ education on children’s educational and
occupational success: Mediation by family interactions, child
12 Assessment 00(0)
aggression, and teenage aspirations. Merrill-Palmer Quarterly,
55, 224-249.
Eddy, C. M., Rickards, H. E., & Cavanna, A. E. (2012). Executive
functions in uncomplicated Tourette syndrome. Psychiatry
Research, 200, 46-48.
Friesen, D. C., Luo, L., Luk, G., & Bialystok, E. (2015). Proficiency
and control in verbal fluency performance across the lifespan
for monolinguals and bilinguals. Language, Cognition and
Neuroscience, 30, 238-250.
Garcia, E., Rodriguez, C., Martin, R., Jimenez, J. E., Hernandez,
S., & Dıaz, A. (2012). Test de fluidez verbal: Datos normati-
vos y desarrollo evolutivo en el alumnado de primaria [Verbal
Fluency Test: normative data and evolutionary development
in elementary students]. European Journal of Education and
Psychology, 5, 53-64.
Gasperini, L. (2000). The Cuban education system: Lessons and
dilemmas. Washington, DC: World Bank.
Gladsjo, J. A., Shuman, C. C., Evans, J. D., Peavy, G. M., Miller,
S. W., & Heaton, R. K. (1999). Norms for letter and category
fluency: Demographic corrections for age, education, and eth-
nicity. Assessment, 6, 147-178.
Goldstrohm, S. L., & Arffa, S. (2005). Preschool children with
mild to moderate traumatic brain injury: An explora-
tion of immediate and post-acute morbidity. Archives of
Clinical Neuropsychology, 20, 675-695. doi:10.1016/j.
acn.2005.02.005
Gollan, T. H., Montoya, R. I., & Werner, G. A. (2002).
Semantic and letter fluency in Spanish-English bilin-
guals. Neuropsychology, 16, 562–576. doi:10.1037/0894-
4105.16.4.562
Hirsch, K. W. (1993). Age of acquisition and naming. In F. J.
Stachowiak (Ed.), Developments in the assessment and reha-
bilitation of brain-damaged patients (pp. 225-232). Tübingen,
Germany: Gunter Narr Verlag.
Hyde, J. S., & Linn, M. C. (1988). Gender differences in verbal
ability: A meta-analysis. Psychological Bulletin, 104, 53-69.
Klenberg, L., Korkman, M., & Lahti-Nuuttila, P. (2001).
Differential development of attention and executive
functions in 3- to 12-year-old Finnish children. Developmen-
tal Neuropsychology, 20, 407-428. doi:10.1207/s15326942dn
2001_6
Kochhann, R., Holz, M. R., Beber, B. C., Chaves, M. L. F., &
Fonseca, R. P. (2018). Reading and writing habits as a predic-
tor of verbal fluency in elders. Psychology & Neuroscience,
11, 39-49.
Korkman, M., Kemp, S. L., & Kirk, U. (2001). Effects of age on
neurocognitive measures of children ages 5 to 12: A cross-
sectional study of 800 children from the United States.
Developmental Neuropsychology, 20, 331-354. doi:10.1207/
s15326942dn2001_2
Kovacs, M. (1992). The children’s depression inventory manual.
Toronto, Ontario, Canada: Multi-Health Systems.
Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005).
Applied linear statistical models (5th ed.). New York, NY:
McGraw-Hill.
Lee, T. M. C., Yuen, K. S. L., & Chan, C. C. H. (2002). Normative
data for neuropsychological measures of fluency, attention,
and memory measures for Hong Kong Chinese. Journal of
Clinical and Experimental Neuropsychology, 24, 615-632.
Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004).
Neuropsychological assessment (4th ed.). New York, NY:
Oxford University Press.
Lozano, A., & Ostrosky-Solis, F. (2006). Efecto de la edad y la
escolaridad en la fluidez verbal semántica: Datos normativos
en población hispanohablante [Effect of age and education
on semantic verbal fluency: Normative data for Spanish-
speaking population]. Revista Mexicana de Psicología, 23(1),
37-44.
Lucas, J. A., Ivnik, R. J., Smith, G. E., Bohac, D. L., Tangalos, E.
G., Graff-Radford, N. R., & Petersen, R. C. (1998). Mayo’s
older Americans normative studies: Category fluency norms.
Journal of Clinical and Experimental Neuropsychology, 20,
194-200.
Luo, L., Luk, G., & Bialystok, E. (2010). Effect of language pro-
ficiency and executive control on verbal fluency performance
in bilinguals. Cognition, 114, 29-41. doi:10.1016/j.cogni-
tion.2009.08.014
Malloy-Diniz, L. F., Bentes, R. C., Figueiredo, P. M., Brandãao-
Bretas, D., da Costa-Abrantes, S., Parizzi, A. M., .  .  . Salgado,
J. V. (2007). Normalización de una batería de tests para
evaluar las habilidades de comprensión del lenguaje, fluidez
verbal y denominación en niños brasileños de 7 a 10 años:
Resultados preliminares [Standardization of a battery of
tests to assess language comprehension skills, verbal fluency
and denomination in Brazilian children aged 7 to 10 years:
Preliminary results]. Revista de Neurología, 44, 275-280.
Manly, J. J., Touradji, P., Tang, M., & Stern, Y. (2003). Literacy
and memory decline among ethnically diverse elders. Journal
of Clinical and Experimental Neuropsychology, 25, 680-690.
doi:10.1076/jcen.25.5.680.14579
Marino, J., & Alderete, A. M. (2010). Valores normativos de prue-
bas de fluidez verbal, categoriales, fonológicas, gramatica-
les y combinadas y análisis comparativo de la capacidad de
iniciación [Normative values of verbal, categorical, phono-
logical, grammatical and combined fluency tests and compar-
ative analysis of initiation ability]. Revista Neuropsicología,
Neuropsiquiatría y Neurociencias, 10(1), 79-93.
Martins, R., Mograbi, D. C., Andrade Gabrig, I., & Charchat-
Fichman, H. (2016). Normative data and evidence of valid-
ity for the Rey auditory verbal learning test, verbal fluency
test, and Stroop test with Brazilian children. Psychology &
Neuroscience, 9, 54-67. doi:10.1037/pne0000041
McCarthy, R. A., & Warrington, E. K. (1990). Cognitive neuropsy-
chology: A clinical introduction. Orlando, FL: Academic Press.
Meador, K. J., Baker, G. A., Browning, N., Clayton-Smith, J.,
Cohen, M. J., Kalayjian, L. A., . . . Loring, D. W. (2011).
Relationship of child IQ to parental IQ and education in
children with fetal antiepileptic drug exposure. Epilepsy &
Behavior, 21, 147-152.
Mitrushina, M. (2005). Handbook of normative data for neuro-
psychological assessment. New York, NY: Oxford University
Press.
Nieto, A., Galtier, I., Barroso, J., & Espinosa, G. (2008). Fluencia
verbal en niños españoles en edad escolar: Estudio norma-
tivo piloto y análisis de las estrategias organizativas [Verbal
fluency in Spanish children of school age: Pilot normative
study and analysis of organizational strategies]. Revista de
Neurología, 46, 2-6.
Rivera et al. 13
Olabarrieta-Landa, L., Rivera, D., Ferrer-Cascales, R., Rodríguez
Lorenzana, A., Galarza-del-Angel, J., Peñalver Guia, A. I.,
. . .Arango-Lasprilla, J. C. (2017). Verbal fluency tests:
Normative data for Spanish Speaking pediatric population.
Neurorehabilitation, 41, 673-686.
Olabarrieta-Landa, L., Torre, E. L., López-Mugartza, J. C.,
Bialystok, E., & Arango-Lasprilla, J. C. (2017). Verbal fluency
tests: Developing a new model of administration and scoring
for Spanish language. NeuroRehabilitation, 41, 539-565.
Oliviera, R. M., Mograbi, D. C., Andrade, I., & Charchat-
Fichman, H. (2016). Normative data and evidence of validity
for the Rey Auditory Verbal Learning Test, Verbal Fluency
Test, and Stroop Test with Brazilian children. Psychology &
Neuroscience, 9, 54-67.
Ponton, M. O., Satz, P., Herrera, L., Ortiz, F., Urrutia, C., Young, R.,
D’Elia, L. F., . . . Namerow, N. (1996). Normative data strati-
fied by age and education for the Neuropsychological Screening
Battery for Hispanics (NeSBHIS): Initial report. Journal of the
International Neuropsychological Society, 2, 96-104.
R Development Core Team. (2017). R: A language and environ-
ment for statistical computing. Vienna, Austria: R Foundation
for Statistical Computing. Retrieved from http://www.R-proj-
ect.org/
Riva, D., Nichelli, F., & Devoti, M. (2000). Developmental aspects
of verbal fluency and confrontation naming in children. Brain
and Language, 71, 267-284.
Rivera, D., & Arango-Lasprilla, J. C. (2017). Methodology for the
development of normative data for Spanish-speaking pediat-
ric populations. NeuroRehabilitation, 41, 581-592.
Rivera, D., Olabarrieta-Landa, L., Van der Elst, W., Gonzalez,
I., Rodríguez-Agudelo, Y., Aguayo, A. A., . . . Arango-
Lasprilla, J. C. (2019). Normative data for verbal fluency
in healthy Latin American adults: Letter M, and fruits and
occupations categories. Neuropsychology, 33, 287-300.
doi:10.1037/neu0000518
Ruff, R. M., Light, R. H., Parker, S. B., & Levin, H. S. (1997).
The psychological construct of word fluency. Brain and
Language, 57, 394-405.
Ruffieux, N., Njamnshi, A. K., Mayer, E., Sztajzel, R., Eta, S. C.,
Doh, R. F., . . . Hauert, C. A. (2009). Neuropsychology in
Cameroon: First normative data for cognitive tests among
school-aged children. Child Neuropsychology, 16, 1-19.
Sandler, A. D., Watson, T. E., Footo, M., Levine, M. D., Coleman,
W. L., & Hooper, S. R. (1992). Neurodevelopmental study
of writing disorders in middle childhood. Developmental
Behavioral Pediatrics, 13, 17-23.
Schady, N. (2011). Parents’ education, mothers’ vocabulary, and
cognitive development in early childhood: Longitudinal evi-
dence from Ecuador. American Journal of Public Health,
101, 2299-2307.
Sohr-Preston, S. L., Scaramella, L. V., Martin, M. J., Neppl, T. K.,
Ontai, L., & Conger, R. (2013). Parental socioeconomic sta-
tus, communication, and children’s vocabulary development:
A third-generation test of the family investment model. Child
Development, 84, 1046-1062.
Strauss, E., Sherman, E., & Spreen, O. (2006). A compendium
of neuropsychological tests: Administration, norms, and
commentary (3rd ed.). New York, NY: Oxford University
Press.
Sundheim, S. T., & Voeller, K. K. (2004). Psychiatric impli-
cations of language disorders and learning disabilities:
Risks and management. Journal of Child Neurology, 19,
814-826.
Tallberg, I. M., Carlsson, S., & Lieberman, M. (2011). Children’s
word fluency strategies. Scandinavian Journal of Psychology,
52, 35-42.
Tombaugh, T. N., Kozak, J., & Rees, L. (1999). Normative data
stratified by age and education for two measures of ver-
bal fluency: FAS and animal naming. Archives of Clinical
Neuropsychology, 14, 167-177.
Van Breukelen, G. J., & Vlaeyen, J. W. (2005). Norming clinical
questionnaires with multiple regression: The Pain Cognition
List. Psychological Assessment, 17, 336-344.
Van der Elst, W., Hurks, P., Wassenberg, R., Meijs, C., & Jolles,
J. (2011). Animal verbal fluency and design fluency in school
aged children: Effects of age, sex, and mean level of parental
education, and regression-based normative data. Journal of
Clinical & Experimental Neuropsychology, 33, 1005-1015.
doi:10.1080/13803395.2011.589509
Weir, J. P. (2005). Quantifying test-retest reliability using the
intraclass correlation coefficient and the SEM. Journal of
Strength and Conditioning Research, 19, 231-240. doi:10.
1519/15184.1
Willcutt, E. G., Pennington, B. F., Olson, R. K., Chhabildas, N.,
& Hulslander, J. (2005). Neuropsychological analysis of
comorbidity between reading disability and attention defi-
cit hyperactivity disorder: In search of the common deficit.
Developmental Neuropsychology, 27, 35-78.