Archives of Clinical Neuropsychology 28 (2013) 262–271

Semantic Fluency and Phonemic Fluency: Regression-based Norms

for the Portuguese Population
Sara Cavaco1,2,3,*, Alexandra Gonçalves1,2, Cláudia Pinto 1,2, Eduarda Almeida 1,2, Filomena Gomes1,2,
Inês Moreira1,2, Joana Fernandes1,2, Armando Teixeira-Pinto 4,5

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

1
Laboratório de Neurobiologia do Comportamento Humano, Centro Hospitalar do Porto, Porto, Portugal
2
Unidade Multidisciplinar de Investigação Biomédica, Universidade do Porto, Porto, Portugal
3
Division of Behavioral Neurology and Cognitive Neuroscience, University of Iowa Carver College of Medicine
4
CIDES, CINTESIS, Faculdade de Medicina da Universidade do Porto, Porto, Portugal
5
Screening and Test Evaluation Program, Sydney School of Public Health, University of Sydney, Australia
*Corresponding author at: Centro Hospitalar do Porto, Serviço de Neurologia, Largo Prof. Abel Salazar, 4099-001 Porto, Portugal.
Tel.: + 351 222 077 500 (1260).
E-mail address: saramscavaco@gmail.com (S. Cavaco).
Accepted 31 December 2012

Abstract
The main goal of this study was to produce adjusted normative data for the Portuguese population on two verbal fluency measures: the
semantic fluency test (animals category) and the phonemic fluency test (letters M, R, and P). The study included 950 community-dwelling
individuals (624 women and 326 men) aged between 18 and 98 (mean ¼ 57.8, SD ¼ 19.0), who had educational backgrounds ranging from 0
to 20 years (mean ¼ 8.8, SD ¼ 5.2). The results showed that age and education were significantly associated with semantic fluency and phon-
emic fluency performance. These demographic characteristics accounted for 42% of the semantic fluency and between 23% and 31% of the
phonemic fluency performance variance. No significant sex effects were found. The normative data are presented as regression-based algo-
rithms to adjust test scores for age and education, with subsequent correspondence between adjusted scores and percentile distribution.

Keywords: Cognition; Neuropsychological tests; Standardization; Reliability; Verbal fluency disorders; Educational achievement

Introduction

The semantic and phonemic fluency tests are two of the most widely used instruments in clinical and experimental neuro-
psychology (Strauss, Sherman, & Spreen, 2006). Verbal fluency tests are brief assessment tools with relatively simple admin-
istration and scoring procedures. Semantic and phonemic fluency are measures of non-motor processing speed, language
production, and executive functions (Greenaway, Smith, Tangalos, Geda, & Ivnik, 2009).
Data from lesion patients (Baldo, Schwartz, Wilkins, & Dronkers, 2006; Gouveia, Brucki, Malheiros, & Bueno, 2007;
Szatkowska, Grabowska, & Szymanska, 2000) and functional neuroimaging of healthy individuals (Alvarez & Emory,
2006; Baciu, Juphard, Cousin, & Bas, 2005; Billingsley et al., 2004; Birn et al., 2010; Gauthier, Duyme, Zanca, & Capron,
2009; Kinkingnehun et al., 2007; Ravnkilde, Videbech, Rosenberg, Gjedde, & Gade, 2002; Tupak et al., 2012) provide
strong evidence for the involvement of the left frontal (particularly the dorsolateral prefrontal cortex) and temporal lobes
on both semantic fluency and phonemic fluency. However, some studies (e.g., Baldo et al., 2006; Billingsley et al., 2004;
Tupak et al., 2012) suggest that semantic fluency and phonemic fluency rely on partially different neural networks. Frontal
lobe lesions can disproportionately impair phonemic fluency, whereas temporal lobe damage impairs semantic fluency to a
greater extent (Baldo et al., 2006). There are indications of a greater left hemispheric involvement in phonemic fluency
than in semantic fluency (Billingsley et al., 2004). However, the inverse trend has also been reported (Tupak et al., 2012).

# The Author 2013. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
doi:10.1093/arclin/act001
 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271 263

Semantic fluency and phonemic fluency have been used to detect individuals at risk of developing dementia (Jacobs,
Marder, Sano, Stern, & Mayeux, 1995; Levy et al., 2002; Palmer, Backman, Winblad, & Fratiglioni, 2003; Santangelo
et al., 2007), to identify patients with mild cognitive impairment (Green et al., 2002) or dementia (Cerhan et al., 2002;
Crossley, D’arcy, & Rawson, 1997; Henry, Crawford, & Phillips, 2004), to differentiate different types of dementia (Libon
et al., 2009; Rascovsky et al., 2002; Rogers, Ivanoiu, Patterson, & Hodges, 2006), and to monitor the progression of
disease (Dujardin et al., 2004; Santangelo et al., 2007).
Even though Alzheimer’s disease (Clark et al., 2010; Murphy, Rich, & Troyer, 2006) and semantic dementia (Libon et al.,
2009; Rogers et al., 2006) affect both verbal fluency measures, the decline is more pronounced on semantic fluency than phon-
emic fluency. Neither semantic fluency nor phonemic fluency appears to be disproportionately affected in patients with pro-
gressive non-fluent aphasia, frontal variant frontotemporal dementia, or posterior cortical atrophy (Libon et al., 2009;
Rogers et al., 2006). Verbal fluency impairments are common manifestations of diseases with predominant frontostriatal de-
generation (Henry, Crawford, & Phillips, 2005; Williams-Gray et al., 2009). Deficits in phonemic fluency can even be found in
preclinical stages of Huntington’s disease (Larsson, Almkvist, Luszcz, & Wahlin, 2008). Poor performance on verbal fluency

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

tests, particularly semantic fluency, is one of the best predictors of dementia in Parkinson’s disease (Jacobs et al., 1995;
Williams-Gray et al., 2009).
Multiple studies from different countries have demonstrated that verbal fluency tests are highly influenced by demographic
characteristics, in particular age and educational level (e.g., Ivnik, Malec, Smith, & Tangalos, 1996; Kosmidis, Vlahou,
Panagiotaki, & Kiosseoglou, 2004; Lucas et al., 2005; Ratcliff et al., 1998; Ryu et al., 2012; Tombaugh, Kovak, & Rees,
1999). The presence of illiterates and the wide variety of educational backgrounds pose a major challenge to clinical neuro-
psychology in Portugal both researchers and clinicians. The need for adequate normative data is paramount.
The present study proposes a regression-based approach to establish Portuguese normative data for the semantic fluency test
and the phonemic fluency test. Some psychometric properties of the tests are also examined.

Methods

Subjects

Normative sample. Participants in this study included 950 community-dwelling Portuguese individuals (624 women and 326
men) between 18 and 98 years of age (mean ¼ 57.8, SD ¼ 19.0) and between 0 and 20 years of education (i.e., formal school-
ing completed with success; mean ¼ 8.8, SD ¼ 5.2). Equivalences from the “New Opportunities program” initiative (i.e., a
governmental program designed to enhance school certification and qualification levels of the Portuguese adult population)
were not credited. The semantic fluency test was administered to 949 subjects (624 women and 325 men; mean age ¼ 57.8,
SD ¼ 19.0; mean education ¼ 8.83, SD ¼ 5.2), whereas the phonemic fluency test was administered to 821 subjects (529
women and 292 men; mean age ¼ 55.8, SD ¼ 18.7; mean education ¼ 9.8, SD ¼ 4.8). The inter-rater reliability was explored
in a subgroup of 91 subjects of the normative sample (gender: 57 women and 34 men; age: 18– 67, mean ¼ 31.8, SD ¼ 13.6;
education: 4 – 18 years, mean ¼ 13.8, SD ¼ 3.6).
The inclusion criteria were ≥18 years of age, Portuguese as the first language, have lived in Portugal in the last 5 years, did
≥50% of formal schooling in Portugal or in a territory with Portuguese administration (for participants with ≥3 years of edu-
cation), and do not have significant auditory deficits after correction. Individuals with history of developmental disorders (e.g.,
learning disability), neurological disease (e.g., traumatic brain injury), or moderate to severe psychopathology (e.g., major
depression, psychosis, alcoholism) were not included. The phonemic fluency test was not applied to participants with less
than 4 years of education. Due to logistic issues, one participant did not perform the semantic fluency test and 18 participants
with ≥4 years of education did not complete the phonemic fluency test. All participants provided their written informed
consent in accordance with the Helsinki Declaration.

Procedures

Based on findings from previous studies (Petersson, Reis, Askelof, Castro-Caldas, & Ingvar, 2000; Reis & Castro-Caldas,
1997; Silva, Petersson, Faı́sca, Ingvar, & Reis, 2004), the application of the phonemic fluency test was restricted to individuals
with ≥4 years of education, because phonemic fluency requires specific knowledge (i.e., about the letters and their relationship
with word formation) that is traditionally acquired in formal education. Basic semantic knowledge about animals is usually
acquired in early childhood and outside the classroom. So, we considered it appropriate to apply the semantic fluency test
to illiterates, even though the effect of education on test performance is well recognized.
264 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271

Based on a preliminary study with 59 healthy subjects (mean age ¼ 32.7, SD ¼ 13.3; mean education ¼ 13.9, SD ¼ 3.4),
the chosen letters for standardization of the phonemic fluency test were M, R, and P, because these letters had different levels of
difficulty. These letters have been used in Portugal for clinical practice and neuropsychological research (Cavaco et al., 2012;
Reis & Castro-Caldas, 1997).

Verbal fluencytTests
Semantic fluency. The subjects were asked to generate the name of as many species of animals as possible within 1 min. The
instructions in Portuguese are presented in the Supplementary material. Regional designations of animals were accepted. Any
repetition of the same animal species (including name variations according to the animal gender or age) was not credited.
Credit was given to superordinate categories only if specific items within that category were not given during the trial. The
total test score corresponds to the number of animal species named within 60 s. Higher scores correspond to better
performance.

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

Phonemic fluency. The subjects were asked to produce orally as many words as possible beginning with a specific letter.
The test consists of three trials, of 1 min each. The instructions in Portuguese are presented in the Supplementary material.
Whenever the subject provided multiple responses with the same root (e.g., variations of gender, number) referring to the
same object, action, or concept, only the first response was credited. When the subject named numbers as responses, only
the first response was credited. The same word with two different meanings was admissible if the subject indicated the different
meanings. Slang terms and foreign words were admissible if they were of general use in Portugal. The total trial score corre-
sponds to the number of words correctly produced within 60 s. The total test score corresponds to the sum of the three trials.
Higher scores correspond to better performance.

Reliability studies. The internal reliability of the phonemic fluency test was analyzed using the total normative sample for the
test. To explore the inter-rater reliability, three raters examined the responses and quantified independently a series of semantic
fluency and phonemic fluency performances. These reliability studies focused on raw scores.

Statistical Analyses. Descriptive statistics (i.e., frequency, percentage, mean, and SD) were used for demographic character-
ization and for presentation of raw test scores. Pearson’s correlations (r) and shared variances (r 2) were used to explore the
effects of demographic characteristics (i.e., sex, age, and education) on test performances. Scatter plots were used to visualize
the associations between demographic variables and test results. The Mann – Whitney test was used to compare subgroups of
subjects (i.e., gender and education group) on test performance.
Multiple regression analyses, without variable selection, were conducted with the raw scores as dependent variables and age
and education (i.e., number of years or education group) as covariates. We considered the possibility of a quadratic effect for
age and education. The assumptions of homoscedasticity and normal distribution of the residuals were verified. The adjustment
of test scores for demographic characteristics was based on regression coefficients. The cumulative percent distribution of the
standardized regression residuals (standardized residuals ¼ residuals/SD) was used to identify the associated percentiles. The
residuals represent the difference between the score of an individual and the mean score of individuals with the same age and
education. Higher adjusted scores correspond to better performance.
Cronbach’s a was used to measure internal consistency of the phonemic fluency test. Two-way random single-measure
intraclass correlation coefficients (ICCs) were computed to assess the absolute agreement between phonemic fluency trials
and between different raters (inter-rater reliability) for both tests.

Results

Demographic Effects

Tables 1 and 2 present the normative raw scores grouped by sex, age, and education groups. The effects of the variables
gender, age, and education were investigated for each test. No statistically significant differences were found between
women and men on semantic fluency (mean ¼ 16.4, SD ¼ 5.2 vs. mean ¼ 16.6, SD ¼ 5.8; p ¼ .734) and phonemic fluency
(letter M: mean ¼ 10.2, SD ¼ 4.4 vs. mean ¼ 10.1, SD ¼ 4.3, p ¼ .770; letter R: mean ¼ 9.8, SD ¼ 4.1 vs. mean ¼ 10,
SD ¼ 4.3, p ¼ .693; letter P: mean ¼ 11, SD ¼ 4.5 vs. mean ¼ 10.7, SD ¼ 4.5, p ¼ .400; Total: mean ¼ 31, SD ¼ 11.6 vs.
mean ¼ 30.8, SD ¼ 11.9, p ¼ .732). Significant (p , .001) linear associations were found for age and number of years of edu-
cation (Table 3). Scatter plots revealed quadratic relations between these demographic variables and the semantic fluency and
phonemic fluency raw scores.
 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271 265

Table 1. Semantic fluency raw scores per sex, age, and education groups

Semantic fluency (n ¼ 949)

n Percent Animals

Sex
Women 624 65.8 16.4 (5.2)
Men 325 34.2 16.6 (5.8)
Age
18–29 110 11.6 21.1 (5.1)
30–39 76 8.0 18.3 (4.1)
40–49 98 10.3 17.6 (4.4)
50–59 161 17.0 17.5 (4.9)
60–69 205 21.6 16.9 (5.0)
70–79 179 18.9 13.8 (4.6)

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

80–89 103 10.9 12.1 (4.2)
≥90 17 1.8 8.8 (3.6)
Education
,3 53 5.6 10.9 (3.5)
3 58 6.1 11.4 (3.9)
4 199 21.0 13.0 (4.5)
5 –9 219 23.1 15.9 (4.1)
10–12 153 16.1 18.8 (4.6)
.12 267 28.1 20.2 (4.5)

Table 2. Phonemic fluency raw scores per sex, age, and education groups
Phonemic fluency (n ¼ 821)
n Percent M R P Total score

Sex
Women 529 64.4 10.2 (4.4) 9.8 (4.1) 11.0 (4.5) 31.0 (11.6)
Men 292 35.6 10.1 (4.3) 10.0 (4.3) 10.7 (4.5) 30.8 (11.9)
Age
18–29 104 12.7 12.2 (4.7) 11.4 (4.0) 12.8 (4.8) 36.4 (11.9)
30–39 75 9.1 11.0 (4.6) 10.4 (4.2) 10.3 (4.5) 31.6 (12.1)
40–49 93 11.3 10.9 (4.4) 10.4 (4.6) 10.5 (4.4) 31.8 (12.1)
50–59 152 18.5 10.1 (4.3) 9.7 (4.3) 10.9 (4.5) 30.7 (11.6)
60–69 191 23.3 10.5 (3.9) 10.1 (3.8) 11.5 (4.2) 32.1 (10.7)
70–79 126 15.3 8.6 (3.8) 9.2 (3.9) 10.5 (4.0) 28.3 (10.4)
80–89 69 8.4 7.7 (3.6) 7.8 (4.2) 9.1 (4.4) 24.6 (11.4)
≥90 11 1.3 6.6 (3.1) 6.5 (3.4) 7.9 (5.0) 21.0 (11.0)
Education
4 191 23.3 6.8 (3.2) 6.8 (3.1) 7.9 (3.54) 21.4 (8.7)
5 –9 218 26.6 9.5 (3.8) 9.3 (3.9) 9.9 (4.17) 28.7 (10.5)
10–12 149 18.1 10.4 (3.8) 10.5 (3.6) 12.3 (3.94) 33.2 (9.6)
.12 263 32.0 13.0 (3.9) 12.3 (3.8) 13.2 (4.04) 38.5 (10.2)

Literacy and Semantic Fluency

No significant differences were found between subjects with 0 and 1 year of education (n ¼ 35 vs. n ¼ 7; mean ¼ 10.8,
SD ¼ 4 vs. mean ¼ 10.7, SD ¼ 1.8; p ¼ .672), between 0 and 2 years of education (n ¼ 35 vs. n ¼ 11; mean ¼ 10.8, SD ¼
4 vs. mean ¼ 11.3, SD ¼ 2.7; p ¼ .477), and between 0 and 3 years of education (n ¼ 35 vs. n ¼ 58; mean ¼ 10.8, SD ¼ 4
vs. mean ¼ 11.4, SD ¼ 3.9; p ¼ .364). Subjects with 0 years of education had lower semantic fluency scores than subjects
with 4 years of education (n ¼ 35 vs. n ¼ 199; mean ¼ 10.8, SD ¼ 4 vs. mean ¼ 13, SD ¼ 4.5; p ¼ .006). This group differ-
ence (p ¼ .074) was no longer statistically significant when it was controlled for age.
The use of a broader criterion for illiteracy (i.e., 0– 2 years of education; Cavaco & Teixeira-Pinto, 2011) revealed that il-
literate subjects had poorer performance on the semantic fluency test than subjects with 4 years of education (n ¼ 53 vs. n ¼
199; mean ¼ 10.9, SD ¼ 3.5 vs. mean ¼ 13, SD ¼ 4.5; p ¼ .002). This group difference (p ¼ .025) remained statistically
significant when it was controlled for age.
266 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271

Table 3. Correlations and shared variances between test raw scores and demographic variables

Test Age Education

2
r r r r2

Semantic fluency
Animals 2.489* .240 .598* .358
Phonemic fluency
M 2.281* .079 .534* .285
R 2.208* .043 .492* .242
P 2.154 .024 .462* .214
Total score 2.237* .056 .550* .302
*p , .0001

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

Table 4. Partial b for the multiple regression analyses
Semantic fluency: Animals Phonemic fluency
M R P Total

Age 0.079* 0.086* 0.082* 0.041 0.208*

Age2 20.001*** 20.001** 20.001* ,0.001 20.002*
Education 0.801*** 0.749*** 0.756*** 1.041*** 2.546***
Education2 20.016** 20.013 20.015* 20.028*** 20.056**
*p , .05.
**p , .01.
***p , .001.

Normative Data

The multiple regression analyses were conducted with age, age2, number of years of education, and number of years of
education2 as the regression model. Each of these variables was found to be independently associated (p , .06) with perform-
ance on the semantic fluency test and the phonemic fluency test—letter M, R, and Total score. Both the linear and the quadratic
of effect of age were not significantly associated (p . .1) with performance on the P trial of the phonemic fluency test
(Table 4). Age, age2, number of years of education, and number of years of education2 partly explained (r 2) the variance of
both semantic fluency (42%) and phonemic fluency—letter M (30%), letter R (25%), letter P (23%), and Total score (31%).
Table 5 presents the regression-based algorithms to adjust test scores for age and number of years of education. In other
words, the algorithms convert raw scores of an individual into standardized Z scores. The percentiles associated with the
age and education adjusted scores are shown in Table 6. A user-friendly program is available online (http://neuropsi.up.pt/)
to adjust scores (Fig. 1). The clinician only needs to introduce the subject’s age, education, and number of words correctly
produced. For instance, if an individual with 45 years of age and 12 years of education generates 13 animal names and a
total of 19 words on the three phonemic fluency trials, the adjusted scores are 21.5 for the semantic fluency test and 21.7
for the phonemic fluency test. These adjusted scores fall within percentile range 3 – 5. Thus, 3 – 5% of the “normal” population
with 45 years of age and 12 years of education generates ≤13 words on the semantic fluency test and ≤19 words on the phon-
emic fluency test.

Internal Reliability

The absolute agreement among M, R, and P of the phonemic fluency test was ICC ¼ 0.71 (95% CI ¼ 0.67 – 0.74), whereas
the internal consistency was Cronbach’s a ¼ 0.89. The number of correct responses was significantly higher for letter M
than for letter R (t ¼ 2.52, 95% CI ¼ 0.06 – 0.50). Both letter M (t ¼ 6.50, 95% CI ¼ 0.53 – 0.99) and letter R (t ¼ 9.22,
95% CI ¼ 0.82 – 1.26) elicited fewer responses within 1 min than letter P.

Inter-rater Reliability

The inter-rater reliability for category animals was ICC ¼ 0.996 (95% CI: 0.995 – 0.998), for letter M was ICC ¼ 0.988
(95% CI: 0.982 – 0.992), for letter R was ICC ¼ 0.977 (95% CI: 0.960 –0.986), and for letter P was ICC ¼ 0.979 (95%
CI: 0.960 – 0.996).
 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271 267

Table 5. Algorithms to calculate adjusted scores

Semantic fluency (raw score 2 11.3984 2 (age * 0.0795) + (age2 * 0.0014) 2 (education * 0.8008) + (education2 * 0.0165))/4.1399
Phonemic fluency
M (raw score 2 2.5878 2 (age × 0.0858) + (age2 × 0.0009) 2 (education × 0.7495) + (education2 × 0.0133))/3.6498
R (raw score 2 2.0379 2 (age × 0.0817) + (age2 × 0.0007) 2 (education × 0.7555) + (education2 × 0.0150))/3.6089
P (raw score 2 2.2708 2 (age × 0.0409) + (age2 × 0.0002) 2 (education × 1.0408) + (education2 × 0.0275))/3.9190
Total score (raw score 2 6.8965 2 (age × 0.2084) + (age2 × 0.0019) 2 (education × 2.5458) + (education2 × 0.0558))/9.7035

Table 6. Empirical percentile ranks associated with age and education adjusted scores
Percentile ranks Scaled scores Adjusted scores
Semantic fluency: Animals Phonemic fluency

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

M R P Total score

1 3 21.9 22.2 22.2 22.2 22.2

2 4 21.8 22.0 21.9 22.0 21.9
3– 5 5 21.6 to 21.5 21.8 to 21.5 21.8 to 21.6 21.8 to 21.6 21.7 to 21.6
6– 10 6 21.4 to 21.3 21.5 to 21.2 21.6 to 21.3 21.5 to 21.3 21.5 to 21.2
11–18 7 21.2 to 20.9 21.2 to 20.9 21.2 to 21.0 21.2 to 21.0 21.2 to 20.9
19–28 8 20.9 to 20.6 20.9 to 20.6 20.9 to 20.6 21.0 to 20.6 20.9 to 20.6
29–40 9 20.6 to 20.3 20.6 to 20.3 20.6 to 20.3 20.6 to 20.3 20.6 to 20.3
41–59 10 20.2 to 0.2 20.3 to 0.1 20.3 to 0.2 20.3 to 0.2 20.3 to 0.2
60–71 11 0.2– 0.5 0.2– 0.5 0.2–0.5 0.2–0.5 20.2– 0.6
72–81 12 0.6– 0.8 0.5– 0.9 0.6–0.9 0.5–0.9 0.6– 0.9
82–89 13 0.9– 1.2 1.0– 1.3 0.9–1.2 0.9–1.3 0.9– 1.3
90–94 14 1.3– 1.7 1.3– 1.6 1.3–1.5 1.3–1.6 1.3– 1.6
95–97 15 1.8– 2.0 1.7– 2.0 1.6–1.9 1.7–1.9 1.7– 1.9
98 16 2.3 2.1 2.1 2.1 2.2
99 17 2.8 2.4 2.4 2.4 2.3
Minimum 22.3 22.9 22.7 22.7 229
Maximum 4.2 3.5 3.1 3.1 3.8

Discussion

In agreement with most normative studies, the performance on the semantic fluency test declined with age
(Benito-Cuadrado, Esteba-Castillo, Bohm, Cejudo-Bolivar, & Pena-Casanova, 2002; Crossley et al., 1997; Gladsjo et al.,
1999; Herrmann, Walter, Ehlis, & Fallgatter, 2006; Kosmidis et al., 2004; Tomer & Levin, 1993; Van der Elst, Boxtel,
Breukelen, & Jolles, 2006) and increased with education (Benito-Cuadrado et al., 2002; Kosmidis et al., 2004; Ratcliff
et al., 1998; Tombaugh et al., 1999). Similar to other reports (Steinberg, Bieliauskas, Smith, & Ivnik, 2005; Tallberg,
Ivachova, Tinchag, & Ostberg, 2008; Tombaugh et al., 1999), the effects of age on phonemic fluency were less pronounced
than the effects of education. In our study, no significant associations were found between verbal fluency tests and sex. Most
studies in the literature are consistent with these negative findings (Benito-Cuadrado et al., 2002; Gladsjo et al., 1999; Harrison,
Buxton, Husain, & Wise, 2000; Lucas et al., 1998; Tombaugh et al., 1999). Nonetheless, there are reports (Capitani, Laiacona, &
Barbarotto, 1999; Kosmidis et al., 2004; Van der Elst et al., 2006) of significant sex effects on semantic fluency when other
semantic categories (e.g., fruits, professions and tools) were used.
The Portuguese population provides a unique opportunity to explore the effects of schooling on verbal fluency tests.
Consistent with Manly and colleagues’ (1999) report, no significant differences on semantic fluency were found between sub-
jects with 0, 1, 2, and 3 years of education. The effects of education on semantic fluency were only significant after the third
grade. These results support the importance of literacy in semantic fluency (Reis & Castro-Caldas, 1997; Silva et al., 2004).
However, the reported pattern of association is specific to category “animals”. Other semantic categories (e.g., “supermarket”)
may be more or less affected by literacy, depending on their ecological or cultural relevance (Silva et al., 2004).
The wide range of educational backgrounds in the Portuguese population poses an important challenge for any normative or
clinical neuropsychology study. Illiteracy, elementary level education, high school level education, and college level education
coexist in most adult age groups. To collect enough data to split a normative or a clinical sample into subgroups of age and
education is problematic. The option in this study was to treat age and education as continuous variables and to use multiple
regression analysis to maximize the number of normative individuals contributing for each specific age and education. Similar
268 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

Fig. 1. Screenshot of the online tool to compute semantic and phonemic fluency adjusted scores (http://neuropsi.up.pt).

approach to establish normative data has been used in recent studies (Cavaco & Teixeira-Pinto, 2011; Van Breukelen &
Vlaeyen, 2005; Van der Elst et al., 2006; Van der Elst, Hurks, Wassenberg, Meijs, & Jolles, 2011). This regression-based nor-
mative method computes the expected test scores for each possible combination of age and education. This highly individua-
lized approach to normative test references facilitates the study of groups of patients with a wide range of demographic
characteristics.
The normative data are presented as algorithms to adjust test scores for age and education, with subsequent correspondence
between adjusted scores and percentile distribution. The adjusted scores can be interpreted as Z scores, because they have
normal distribution with mean 0 and standard deviation 1. The percentile ranks can be converted into scaled scores
(i.e., mean ¼ 10 and SD ¼ 3), as used by the Mayo’s Older Americans Normative Studies (Ivnik et al., 1996; Lucas et al.,
1998, 2005). These standardization procedures have significant advantages for both clinical and research practices. The adjust-
ment to the individual’s demographic characteristics and the use of common metrics facilitate the comparisons between tests
and between individuals.
In this normative study, education was operationalized as the number of years of formal regular schooling completed with
success. This approach is vulnerable to the numerous changes in the educational system that have occurred throughout the last
decades in Portugal. For instance, the curricula of regular school have suffered significant modifications. Since 2005, a large
portion of the Portuguese adult population with low education has enrolled in the “New Opportunities” initiative to enhance
their qualifications and to acquire primary (9th grade) or secondary (12th grade) level education certificates. However, the
actual equivalence between regular school and “New Opportunities” education programs is still unknown. So, for the
purpose of this study, only regular schooling was credited to the participants’ education background. Considering these con-
tinuing changes in education, future studies ought to assess the reliability of the normative algorithms and update the norms if
necessary.
The normative sample was almost exclusively white and was derived mostly from north and central regions of Portugal.
Other races and other regions were underrepresented in this study. Regional effects in verbal fluency performance (i.e.,
number of words generated), beyond the effects of age and education, are not to be expected in a small country with significant
internal migration. However, the generalizability of the norms to immigrants from other Portuguese speaking countries (i.e.,
 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271 269

immigrants from Brazil or the ex-colonies - Angola, Cape Verde, China—Macau, Guinea Bissau, India—Goa, Daman and Diu,
Mozambique, East-Timor) may be limited.
The study sample had an unbalanced ratio of women and men. This limitation of the study was minimized by the large
sample size and the absence of significant differences between women and men regarding age, education, and test results.
It is unlikely that the overrepresentation of women has produced a significant bias on the study results.
Letters M, R, and P are among the easiest for word production both in Portuguese (Senhorini, Amaro Junior, Mello Ayres,
Simone, & Busatto, 2006) and in English (Borkowski, Benton, & Spreen, 1967). This set of letters has been chosen for the
Spanish norms (Pena-Casanova et al., 2009). Other commonly used letters for phonemic fluency are: F, A, and S (Bolla,
Lindgren, Bonaccorsy, & Bleecker, 1990; Tallberg et al., 2008; Tombaugh et al., 1999; Tomer & Levin, 1993); C, F, and L
(Barry, Bates, & Labouvie, 2008; Benton & Hamsher, 1976; Ross, Furr, Carter, & Weinberg, 2006; Steinberg et al., 2005);
and P, R, and W (Benton and Hamsher, 1976; Ross et al., 2006). Consistent with a Brazilian study (Senhorini et al., 2006),
in our Portuguese normative sample, P was the letter that elicited more words within 1 min and letter M generated more
words than letter R. Regarding the internal reliability of the phonemic fluency test, using letters M, R, and P, the consistency

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

was high, but the absolute agreement between trials was modest. The internal consistency results are slightly higher than other
reports (Ruff, Light, Parker, & Levin, 1996; Tombaugh et al., 1999).
Both semantic fluency and phonemic fluency tests revealed excellent inter-rater reliability. Ross (2003) also reported high
inter-rater reliability in a phonemic fluency test. Future studies ought to explore other psychometric properties of the tests, such
as practice effects. Determining reliable change indices would be useful for the clinicians in retest situations. There are indications
in the literature that verbal fluency tests have good test–retest reliability (Harrison et al., 2000; Lemay, Bedard, Rouleau, &
Tremblay, 2004; Ross et al., 2007; Ruff et al., 1996). However, the repeated exposure is known to produce significant practice
effects (Cooper, Lacritz, Weiner, Rosenberg, & Cullum, 2004; Ross et al., 2007; Wilson, Watson, Baddeley, Emslie, & Evans,
2000), particularly in healthy individuals (Wilson et al., 2000).
Semantic fluency and phonemic fluency were standardized as part of a larger normalization project. Data from a series of
widely used neuropsychological instruments are being collected in the same (or substantially overlapping) study sample. This
conorming approach is believed to facilitate the comparison between results from different neuropsychological measures. The
collection of normative data from multiple tests is also beneficial because it provides a closer parallel to the clinical setting.

Supplementary Material

Supplementary material is available at Archives of Clinical Neuropsychology online.

Funding

This research received no specific grant from any funding agency, commercial or not-for-profit sectors. Armando
Teixeira-Pinto was supported by the Australian National Health and Medical Research Council Grant 402764 to the
Screening and Test Evaluation Program.

Conflit of interest

None declared.

Acknowledgments

Psychologists Alexandra Pereira, Alexandre Abreu, Ana Cristina Nunes, Ana Sampaio, Andreia Gonçalves Azevedo, Cátia
Alves, Cláudia Dias, Estelle Lima, Felı́cia Silva, Filipa Júlio, Liliana Noronha, Maria do Céu Seabra, Mariana Oliveira, Marta
Pinto, Patrı́cia Matos, Rui Quintas, Sandra Freitas, Sara Cabaço, Susana Pinto, Susana Vieira da Silva, and Vânia Santos
collaborated in the collection of data.

References

Alvarez, J. A., & Emory, E. (2006). Executive function and the frontal lobes: A meta-analytic review. Neuropsychology Review, 16, 17–42.
Baciu, M., Juphard, A., Cousin, E., & Bas, J. F. (2005). Evaluating fMRI methods for assessing hemispheric language dominance in healthy subjects. European
Journal of Radiology, 55, 209 –218.
270 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271

Baldo, J. V., Schwartz, S., Wilkins, D., & Dronkers, N. F. (2006). Role of frontal versus temporal cortex in verbal fluency as revealed by voxel-based lesion
symptom mapping. Journal of the International Neuropsychological Society, 12, 896– 900.
Barry, D., Bates, M. E., & Labouvie, E. (2008). FAS and CFL forms of verbal fluency differ in difficulty: A meta-analytic study. Applied Neuropsychology,
15, 97–106.
Benito-Cuadrado, M. M., Esteba-Castillo, S., Bohm, P., Cejudo-Bolivar, J., & Pena-Casanova, J. (2002). Semantic verbal fluency of animals: A normative and
predictive study in a Spanish population. Journal of Clinical and Experimental Neuropsychology, 24, 1117– 1122.
Benton, A. L., & Hamsher, K. (1976). Multilingual Aphasia Examination manual. Iowa City, IA: University of Iowa.
Billingsley, R. L., Simos, P. G., Castillo, E. M., Sarkari, S., Breier, J. I., Pataraia, E., et al. (2004). Spatio-temporal cortical dynamics of phonemic and semantic
fluency. Journal of Clinical & Experimental Neuropsychology, 26, 1031– 1043.
Birn, R. M., Kenworthy, L., Case, L., Caravella, R., Jones, T. B., Bandettini, P. A., et al. (2010). Neural systems supporting lexical search guided by letter and
semantic category cues: A self-paced overt response fMRI study of verbal fluency. Neuroimage, 49, 1099–1107.
Bolla, K. I., Lindgren, K. N., Bonaccorsy, C., & Bleecker, M. L. (1990). Predictors of verbal fluency (FAS) in the healthy elderly. Journal of Clinical
Psychology, 46, 623–628.
Borkowski, J. G., Benton, A. L., & Spreen, O. (1967). Word fluency and brain damage. Neuropsychologia, 5, 135 –140.
Capitani, E., Laiacona, M., & Barbarotto, R. (1999). Gender affects word retrieval of certain categories in semantic fluency tasks. Cortex, 35, 273– 278.

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

Cavaco, S., Martins da Silva, A., Coutinho, E., Marinho, A., Moreira, I., Gonçalves, A., et al. (2012). Are cognitive and olfactory dysfunctions in neuropsychi-
atric lupus erythematosus dependent on anxiety or depression? The Journal of Rheumatology, 39, 770– 776.
Cavaco, S., & Teixeira-Pinto, A. (2011). Dementia Rating Scale-2: Manual Técnico (Versão Portuguesa). Lisboa: Cegoc-Tea.
Cerhan, J. H., Ivnik, R. J., Smith, G. E., Tangalos, E. C., Petersen, R. C., & Boeve, B. F. (2002). Diagnostic utility of letter fluency, category fluency, and
fluency difference scores in Alzheimer’s disease. The Clinical Neuropsychologist, 16, 35–42.
Clark, L. J., Gatz, M., Zheng, L., Chen, Y. L., McCleary, C., & Mack, W. J. (2010). Longitudinal verbal fluency in normal aging, preclinical, and prevalent
Alzheimer’s disease. American Journal of Alzheimer’s Disease and Other Dementias, 24, 461– 8.
Cooper, D. B., Lacritz, L. H., Weiner, M. F., Rosenberg, R. N., & Cullum, C. M. (2004). Category fluency in Mild Cognitive Impairment: Reduced effect of
practice in test-retest conditions. Alzheimer Disease and Associated Disorders, 18, 120–2.
Crossley, M., D’arcy, C., & Rawson, N. S. B. (1997). Letter and category fluency in community-dwelling canadian seniors: A comparison of normal partici-
pants to those with dementia of the Alzheimer or vascular type. Journal of Clinical and Experimental Neuropsychology, 19, 52–62.
Dujardin, K., Defebvre, L., Duhamel, A., Lecouffe, P., Rogelet, P., Steinling, M., et al. (2004). Cognitive and SPECT characteristics predict progression of
Parkinson’s disease in newly diagnosed patients. Journal of Neurology, 251, 1383–1392.
Gauthier, C. T., Duyme, M., Zanca, M., & Capron, C. (2009). Sex and performance level effects on brain activation during a verbal fluency task: A functional
magnetic resonance imaging study. Cortex, 45, 164 –176.
Gladsjo, J. A., Schuman, C. C., Evans, J. D., Peavy, G. M., Miller, S. W., & Heaton, R. K. (1999). Norms for letter and category fluency: Demographic correc-
tions for age, education, and ethnicity. Assessment, 6, 147–178.
Gouveia, P. A., Brucki, S. M., Malheiros, S. M., & Bueno, O. F. (2007). Disorders in planning and strategy application in frontal lobe lesion patients. Brain and
Cognition, 63, 240–246.
Green, J., McDonald, W. M., Vitek, J. L., Evatt, M., Freeman, A., Haber, M., et al. (2002). Cognitive impairments in advanced PD without dementia.
Neurology, 59, 1320–1324.
Greenaway, M. C., Smith, G. E., Tangalos, E. G., Geda, Y. E., & Ivnik, R. J. (2009). Mayo older Americans normative studies: Factor analysis of an expanded
neuropsychological battery. The Clinical Neuropsychologist, 23, 7– 20.
Harrison, J. E., Buxton, P., Husain, M., & Wise, R. (2000). Short test of semantic and phonological fluency: Normal performance, validity and test-retest re-
liability. British Journal of Clinical Psychology, 39, 181–191.
Henry, J. D., Crawford, J. R., & Phillips, L. H. (2004). Verbal fluency performance in dementia of the Alzheimer’s type: A meta-analysis. Neuropsychologia,
42, 1212–1222.
Henry, J. D., Crawford, J. R., & Phillips, L. H. (2005). A meta-analytic review of verbal fluency deficits in Huntington’s disease. Neuropsychology, 19, 243–252.
Herrmann, M. J., Walter, A., Ehlis, A.-C., & Fallgatter, A. J. (2006). Cerebral oxygenation changes in the prefrontal cortex: Effects of age and gender.
Neurobiology of Aging, 27, 888–894.
Ivnik, R. J., Malec, J. F., Smith, G. E., & Tangalos, E. G. (1996). Neuropsychological tests’ norms above age 55: COWAT, BNT, MAE Token, WRAT-R
Reading, AMNART, STROOP, TMT, and JLO. The Clinical Neuropsychologist, 10, 262– 278.
Jacobs, D. M., Marder, K., Sano, M., Stern, Y., & Mayeux, R. (1995). Neuropsychological characteristics of preclinical dementia in Parkinson’s disease.
Neurology, 45, 1691–1696.
Kinkingnehun, S., Volle, E., Pelegrini-Issac, M., Golmard, J.-L., Lehericy, S., Boisgueheneuc, F., et al. (2007). A novel approach to clinical–radiological cor-
relations: Anatomo-Clinical Overlapping Maps (AnaCOM): Method and validation. Neuroimage, 37, 1237–1249.
Kosmidis, M. H., Vlahou, C. H., Panagiotaki, P., & Kiosseoglou, G. (2004). The verbal fluency task in the Greek population: Normative data, and clustering and
switching strategies. Journal of the International Neuropsychological Society, 10, 164– 172.
Larsson, M. U., Almkvist, O., Luszcz, M. A., & Wahlin, T. B. (2008). Phonemic fluency deficits in asymptomatic gene carriers for Huntington’s disease.
Neuropsychology, 22, 596–605.
Lemay, S., Bedard, M. A., Rouleau, I., & Tremblay, P. L. (2004). Practice effect and test-retest reliability of attentional and executive tests in middle-aged to
elderly subjects. The Clinical Neuropsychologist, 18, 284 –302.
Levy, G., Jacobs, D. M., Tang, M. X., Cote, L. J., Louis, E. D., Alfaro, B., et al. (2002). Memory and executive function impairment predict dementia in
Parkinson’s disease. Movement Disorders, 17, 1221– 1226.
Libon, D. J., McMillan, C., Gunawardena, D., Powers, C., Massimo, I., Khan, A., et al. (2009). Neurocognitive contributions to verbal fluency deficits in fron-
totemporal lobar degeneration. Neurology, 73, 535– 542.
Lucas, J. A., Ivnik, R. J., Smith, G. E., Bohac, D. L., Tangalos, E. G., Graff-Radford, N. R., et al. (1998). Mayo’s older Americans normative studies: Category
fluency norms. Journal of Clinical and Experimental Neuropsychology, 20, 194 –200.
 S. Cavaco et al. / Archives of Clinical Neuropsychology 28 (2013) 262–271 271

Lucas, J. A., Ivnik, R. J., Smith, G. E., Ferman, T. J., Willis, F. B., Petersen, R. C., et al. (2005). Mayo’s Oder African Americans Normative Studies: Norms for
Boston Naming Test, Controlled Oral Word Association, Category Fluency, Animal Naming, Token Test, Wrat-3 Reading, Trail Making Test, Stroop Test,
and Judgment of Line Orientation. The Clinical Neuropsychologist, 19, 243– 269.
Manly, J. J., Jacobs, D. M., Sano, M., Bell, K., Merchant, C. A., Small, S. A., et al. (1999). Effect of literacy on neuropsychological test performance in non-
demented, education-matched elders. Journal of the International Neuropsychological Society, 5, 191–202.
Murphy, K. J., Rich, J. B., & Troyer, A. K. (2006). Verbal fluency patterns in amnestic mild cognitive impairment are characteristic of Alzheimer’s type
dementia. Journal of the International Neuropsychological Society, 12, 570– 574.
Palmer, K., Backman, L., Winblad, B., & Fratiglioni, L. (2003). Detection of Alzheimer’s disease and dementia in the preclinical phase: Population based
cohort study. British Medical Journal, 326, 245– 249.
Pena-Casanova, J., Quinones-Ubeda, S., Gramunt-Fombuena, N., Quintana-Aparicio, M., Aguilar, M., Badenes, D., et al. (2009). Spanish Multicenter
Normative Studies (NEURONORMA Project): Norms for Verbal Fluency Tests. Archives of Clinical Neuropsychology, 24, 395–411.
Petersson, K. M., Reis, A., Askelof, S., Castro-Caldas, A., & Ingvar, M. (2000). Language processing modulated by literacy: A network analysis of verbal
repetition in literate and illiterate subjects. Journal of Cognitive Neuroscience, 12, 364–382.
Rascovsky, R., Salmon, D. P., Ho, G. J., Galasko, D., Peavy, G. M., Hansen, L. A., et al. (2002). Cognitive profiles differ in autopsy-confirmed frontotemporal
dementia and AD. Neurology, 58, 1801–1808.

Downloaded from https://academic.oup.com/acn/article/28/3/262/4747 by guest on 22 February 2021

Ratcliff, G., Ganguli, M., Chandra, V., Sharma, S., Belle, S., Seaberg, E., et al. (1998). Effects of literacy and education on measures of word fluency. Brain and
Language, 61, 115–122.
Ravnkilde, B., Videbech, P., Rosenberg, R., Gjedde, A., & Gade, A. (2002). Putative tests of frontal lobe function: A PET-study of brain activation during
Stroop’s test and verbal fluency. Journal of Clinical and Experimental Neuropsychology, 24, 534– 547.
Reis, A., & Castro-Caldas, A. (1997). Illiteracy: A cause for biased cognitive development. Journal of the International Neuropsychological Society, 3, 444–450.
Rogers, T. T., Ivanoiu, A., Patterson, K., & Hodges, J. R. (2006). Semantic memory in Alzheimer’s disease and the frontotemporal dementias: A longitudinal
study of 236 patients. Neuropsychology, 20, 319– 335.
Ross, T. P. (2003). The reliability of cluster and switch scores for the Controlled Oral Word Association Test. Archives of Clinical Neuropsychology, 18, 153–164.
Ross, T. P., Calhoun, E., Cox, T., Wenner, C., Kono, W., & Pleasant, M. (2007). The reliability and validity of qualitative scores for the Controlled Oral Word
Association Test. Archives of Clinical Neuropsychology, 22, 475–488.
Ross, T. P., Furr, A. E., Carter, S. E., & Weinberg, M. (2006). The psychometric equivalence of two alternate forms of the controlled oral word association test.
The Clinical Neuropsychologist, 20, 414– 431.
Ruff, R. M., Light, R. H., Parker, S. B., & Levin, H. S. (1996). Benton Controlled Oral Word Association Test: Reliability and updated norms. Archives of
Clinical Neuropsychology, 11, 329– 338.
Ryu, S. H., Kim, K. W., Kim, S., Park, J. H., Kim, T. H., Jeong, H. G., et al. (2012). Normative study of the category fluency test (CFT) from nationwide data on
community-dwelling elderly in Korea. Archives of Gerontology and Geriatrics, 54, 305 –309.
Santangelo, G., Trojano, L., Vitale, C., Ianniciello, M., Amboni, M., Grossi, D., et al. (2007). A neuropsychological longitudinal study in Parkinson’s patients
with and without hallucinations. Movement Disorders, 22, 2418–2425.
Senhorini, M. C. T., Amaro Junior, E., Mello Ayres, A., Simone, A., & Busatto, G. F. (2006). Phonemic fluency in Portuguese-speaking subjects in Brazil:
Ranking of letters. Journal of Clinical and Experimental Neuropsychology, 28, 1191– 1200.
Silva, C. G., Petersson, K. M., Faı́sca, L., Ingvar, M., & Reis, A. (2004). The effects of literacy and education on the quantitative and qualitative aspects of
semantic verbal fluency. Journal of Clinical and Experimental Neuropsychology, 26, 266–277.
Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., & Ivnik, R. J. (2005). Mayo’s older Americans normative studies: Age- and IQ-adjusted norms for the trail
making test, the stroop test, and MAE controlled oral word association test. The Clinical Neuropsychologist, 19, 329– 377.
Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms, and commentary (3rd ed.). New York:
Oxford University Press.
Szatkowska, I., Grabowska, A., & Szymanska, O. (2000). Phonological and semantic fluencies are mediated by different regions of the prefrontal cortex. Acta
Neurobiologiae Experimentalis, 60, 503–508.
Tallberg, I. M., Ivachova, E., Tinchag, K. J., & Ostberg, P. (2008). Swedish norms for word fluency tests: FAS, animals and verbs. Scandinavian Journal of
Psychology, 49, 479–485.
Tombaugh, T. N., Kovak, J., & Rees, L. (1999). Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming.
Archives of Clinical Neuropsychology, 14, 167–177.
Tomer, R., & Levin, B. E. (1993). Differential effects of aging on two verbal fluency tasks. Perceptual and Motor Skills, 76, 465–466.
Tupak, S. V., Badewien, M., Dresler, T., Hahn, T., Ernst, L. H., Herrmann, M. J., et al. (2012). Differential prefrontal and frontotemporal oxygenation patterns
during phonemic and semantic verbal fluency. Neuropsychologia, 50, 1565–1569.
Williams-Gray, C. H., Evans, J. R., Goris, A., Foltynie, T., Ban, M., Robbins, T. W., et al. (2009). The distinct cognitive syndromes of Parkinson’s disease: 5
year follow-up of the CamPalGN cohort. Brain, 132, 2958–2969.
Wilson, B. A., Watson, P. C., Baddeley, A. D., Emslie, H., & Evans, J. J. (2000). Improvement or simply practice? The effects of twenty repeated assessments
on people with and without brain injury. Journal of the International Neuropsychological Society, 6, 469–479.
Van Breukelen, G. J. P., & Vlaeyen, J. W. S. (2005). Norming clinical questionnaires with multiple regression: The pain cognition list. Psychological
Assessment, 17, 336 –344.
Van der Elst, W., Boxtel, M., Breukelen, G., & Jolles, J. (2006). Normative data for the animal, profession and letter M naming verbal fluency tests for Dutch
speaking participants and the effects of age, education, and sex. Journal of the International Neuropsychological Society, 12, 80– 89.
Van der Elst, W., Hurks, P., Wassenberg, R., Meijs, C., & Jolles, J. (2011). Animal verbal fluency and design fluency in school-aged children: Effects of age,
sex, and mean level of parental education, and regression-based normative data. Journal of Clinical and Experimental Neuropsychology, 33, 1005– 1015.