c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6

Available online at www.sciencedirect.com

ScienceDirect
Journal homepage: www.elsevier.com/locate/cortex

Exploratory Report

Complexity in neuropsychological assessments of

cognitive impairment: A network analysis approach

Giorgia Tosi a,b, Carolina Borsani b, Stefania Castiglioni b, Roberta Daini a,c,
Massimo Franceschi b and Daniele Romano a,c,*
a
University of Milano-Bicocca, Psychology Department, Milano MI, Italy
b
MultiMedica Hospital, Castellanza (VA), Department of Neurology, Castellanza VA, Italy
c
NeuroMi, Milan Centre for Neuroscience, University of Milano-Bicocca, Dept. of Neurology and Neuroscience, San
Gerardo Hospital, Monza MB, Italy

article info abstract

Article history: In a neuropsychological assessment, each test aims at measuring a single cognitive function.
Received 19 March 2019 However, test performance depends on an interconnected system of cognitive functions and
Reviewed 8 June 2019 individual characteristics. For a better understanding of cognitive deficits, it is fundamental
Revised 29 July 2019 to recognize this complexity and study the relationships between test performances.
Accepted 8 November 2019 This study aims to evaluate complexity in neuropsychological assessment through
Action editor Rob McIntosh network analysis (NA) in 165 healthy older adults, 191 patients with Alzheimer's disease
Published online 27 November 2019 (AD), and 129 patients with vascular encephalopathy (VaE).
NA is a flexible method to explore different domains where many variables are corre-
Keywords: lated with each other, and the relationships are key for understanding the domains. We
Dementia included general aspects of individual differences (i.e., age, sex, years of education) and the
Network analysis raw scores of a clinically used neuropsychological battery in the network.
Alzheimer Healthy subjects showed a segregated pattern, suggesting a good specificity of each test
Neuropsychological assessment in measuring a specific cognitive function. Moreover, the scores were related to age and
education. In the patient groups, the identified patterns changed in a consistent manner,
showing less specificity and new relationships between the various tests, thereby reducing
the impact of age and education on the performance. In particular, AD patients showed
worse performance on the tests but also a different balance between different tasks,
suggesting a reorganization of the cognitive system and not a mere decline.
These results provide a new perspective in looking at the complexity of cognitive
function assessment.
© 2019 Elsevier Ltd. All rights reserved.

* Corresponding author. University of Milano-Bicocca, Psychology Department, Piazza dell’Ateneo Nuovo 1, 20126, Milano MI, Italy.
E-mail addresses: g.tosi3@campus.unimib.it (G. Tosi), carolina.borsani@multimedica.it (C. Borsani), stefania.castiglioni@multi-
medica.it (S. Castiglioni), roberta.daini@unimib.it (R. Daini), massimo.franceschi1@gmail.com (M. Franceschi), daniele.romano@unimib.
it (D. Romano).
https://doi.org/10.1016/j.cortex.2019.11.004
0010-9452/© 2019 Elsevier Ltd. All rights reserved.
86 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
1. Introduction
The brain is a system of interacting complex functional net-
works (Buckner, Hrienen, & Yeo, 2013), and each neurological
deficit usually affects more than a single cognitive domain.
In the past few years, advances in functional neuroimaging
have provided new tools to examine the complexity of the
interactions between brain regions, allowing the identification
of connectivity patterns in the human brain (van den Heuvel &
Hulshoff Pol, 2010). Specifically, many studies have used graph
theory to analyse functional connectivity patterns in patients
with Alzheimer's disease (Huang, Lin, & Lin, 2016; Lehmann
et al., 2016) and other types of cognitive impairment
(Hafkemeijer et al., 2015) to associate dementia with complex
patterns of dysfunction. A similar approach to complexity is
lacking for the neuropsychological investigation of both
healthy and pathological ageing.
In clinical neuropsychology, it is necessary to consider the
entirety of patient performance on cognitive tests to distin-
guish healthy elderly individuals from those with cognitive
impairment and different forms of dementia. Single test scores
and performances on their own are poorly informative and
need to be integrated into a wider collection of information
(age, sex, anamnesis, other tests score, errors type and
execution modality, laboratory exams, neuroimaging) to
adequately assess the condition. Therefore, the quantitative
data given by the tests must be integrated with a qualitative
analysis of the performance (Mathias & Burke, 2009;
Mitrushina, Boone, Razani, & D'Elia, 2005). In this framework,
in analogy with the interacting networks of brain regions, a
model of the relations among neuropsychological test perfor-
mances could be useful for understanding the patient's
cognitive profile. Such an approach integrates a wider range of
information in a manner that may not immediately evident
and more systematically considers the underlying complexity.
Van Der Maas and collaborators (2006) have shown that
performances on cognitive tasks, measured with neuropsy-
chological tests, positively correlate with each other, sug-
gesting that people who score well on one cognitive test are
likely to score well on the others. In a neuropsychological
assessment, clinicians are well aware that each test requires
the contribution of many processes and that those processes
have to be spared to bring out the activity of the targeted
cognitive function. For example, the performances in tests
meant to assess verbal working memory are influenced by
language comprehension and vice versa (Baddeley, 2003).
For a better understanding of cognitive impairment, it is
fundamental to recognize these connections and to determine
how they could change in different conditions. Indeed, it is
likely that the cognitive impairment caused by Alzheimer's
disease (AD) not only reduces the proper functioning of one or
more specific cognitive functions (Weintraub, Wicklund, &
Salmon, 2012) but also might alter the balance among them,
as reflected by the different tests.
How can we quantify this balance of relations?
A flexible method that is potentially useful for exploring
relations among different domains is network analysis (NA).
NA is especially useful to explore domains where many
variables correlate, and their relationships are the core
information to be investigated. A network is a model
composed of a set of nodes, representing entities, and a set of
edges that connect the nodes, which represent their relations
(e.g., De Nooy, Mrvar, & Batagelj, 2011). NA provides a large
pull of information about the variables under investigation
and their relationships. For instance, the number of edges, the
number of negative and positive edges and their weights
(i.e., the strength of the connection). In clinical psychology, NA
has been recently proposed as a novel theoretical approach to
personality and psychopathology studies (Costantini et al.,
2015), and the intention of the current paper is to extend the
use of this technique to the neuropsychological domain
(Kellermann et al., 2016).
The employment of NA on the scores of various neuropsy-
chological tests can allow an evaluation of how they are related
to different conditions in healthy and pathological older adults.
NA results provide a novel perspective to approach neuropsy-
chological assessment in dementia in two ways. First, the score
of one test (or a set of tests) can be significantly lower in those
with dementia than in healthy elderly individuals. Second, we
suggest that there could be possible changes in the relations
between the different cognitive functions. These altered
relations would provide clues about the reorganization of the
cognitive system, providing evidence-based support for the
clinical detection of different patterns of cognitive functioning.
In this paper, NA was adopted to evaluate the balance
among the test performances in three groups of individuals:
healthy elderly individuals, patients affected by AD, and
patients with sub-cortical vascular encephalopathy (VaE). AD
and VaE patients were considered models of “cortical” and
“sub-cortical” cognitive impairment, respectively.
2. Materials and methods
2.1. Participants
A retrospective analysis of the medical records of the
Neurology Department of the MultiMedica Hospital of
Castellanza (Varese, Italy) from 2007 to 2017 was conducted.
We selected only patients who completed the extensive bat-
tery of tests for the clinical examination (see below). We
extracted two groups of patients from the database for this
study: patients with AD and patients with VaE. The AD group
included patients fulfilling diagnostic criteria for clinically
probable AD (McKhann et al., 2011). Exclusion criteria were (i)
age below 50 years old, (ii) presence of central nervous system
diseases (i.e., stroke, traumatic injury), and (iii) presence of
another psychological disorder (i.e., depression, anxiety) or a
history of intellectual disability.
The VaE group included patients with clinically and
neuroimaging-documented sub-cortical cerebrovascular
disease (Erkinjuntti et al., 2000a; b; Sachdev et al., 2014).
We based the selection of patients on brain imaging fea-
tures and included subjects with sub-cortical vascular im-
pairments clinically documented by neurologic diagnosis and
cerebral imaging according to the gold standard of clinical
neuropsychology literature (Erkinjuntti et al., 2000a, b). All
patients showed sub-cortical damage, including sub-cortical
complete infarctions, lacunar infarcts and small or micro-
 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
infarcts (Kalaria & Erkinjuntti, 2006). Moreover, the presence
of a cognitive disorder and the exclusion of other factors
accounting for the cognitive deficits (including AD pathology)
were encompassed by the neurologists' clinical judgement.
Additional exclusion criteria were the same as the AD
group. Only patients who completed the entire battery of tests
were included in the study.
We identified two segregated groups: 191 patients with AD
(mean age: 75.76 ± 6.71; mean years of education: 7.04 ± 3.58)
and 129 patients with VaE (mean age: 75.98 ± 7.22; mean years
of education: 6.53 ± 3.43).
All neurological diagnoses were double-checked (through
revision of clinical and instrumental recordings) by the clin-
ical neuropsychologists (the authors CB and SC) and then
further confirmed by an experienced neurologist specialized
in dementia assessment (the author MF).
Additionally, we included 165 healthy elderly individuals in
our study (mean age: 69.33 ± 5.94; mean years of education:
10.88 ± 3.90). These subjects completed a neuropsychological
evaluation for data collection as part of a local interventional
project at the University of Milano-Bicocca.
The exclusion criteria for this cohort were the same as for
the patient groups.
An experienced neuropsychologist (the author RD) super-
vised the selection of healthy participants.
Data and analysis code are available on the Open Science
Framework platform at the following link: https://osf.io/
8wn7e/?view_only¼2569a529e9424b0d86a31dcc05daa0fe.
No part of the study procedures or analysis was pre-
registered prior to the research being conducted.
2.2. Neuropsychological assessment
In the case of patients with multiple evaluations, we consid-
ered the first one that allowed the neurologist to make the
diagnosis.
The neuropsychological battery included the following tests:
Mini-Mental State Examination (MMSE) (Folstein, Folstein, &
McHugh, 1975); semantic fluency test (Spinnler & Tognoni,
1987); phonemic fluency test (Novelli, Capitani, Vallar, &
Cappa, 1986); token test (Spinnler & Tognoni, 1987); drawings
copy (Spinnler & Tognoni, 1987); ReyeOsterrieth complex fig-
uredcopy and delayed recall (Caffarra, Vezzadini, Dieci, Zonato,
& Venneri, 2002a); digit span forward (Orsini et al., 1987); prose
memory testdimmediate and delayed recall (Spinnler &
Tognoni, 1987); Corsi span (Orsini et al., 1987); semantically
related words list learningdimmediate recall, delayed recall
and detection through distractors (Mauri et al., 1997); Raven's
progressive matrices (Basso, Capitani, & Laiacona, 1987; Raven,
1947); attentive matrices (Spinnler & Tognoni, 1987); Trail Mak-
ing Testdpart A and part B, the latter was scored as over or
under the cut-off (Giovagnoli et al., 1996).
All the tests are included in the standard assessment used
at the neurology department of the MultiMedica Hospital of
Castellanza (VA, Italy) and built for the assessment of cogni-
tive impairments.
Healthy participants were administered the following
tests: MoCA (Conti, Bonazzi, Laiacona, Masina, & Vanelli,
2015); semantic fluency test (Novelli, et al., 1986); phonemic
fluency test (Novelli, et al., 1986); ReyeOsterrieth complex
87
figuredcopy and delayed recall (Caffarra et al., 2002a); digit
spandforward and backward (Monaco, Costa, Caltagirone, &
Carlesimo, 2013); prose memory test (Novelli, et al., 1986);
Corsi spandforward and backward (Monaco et al., 2013); Ra-
ven's progressive matrices (Basso et al., 1987; Raven, 1947);
Stroop test accuracy and time performance (Caffarra,
Vezzadini, Dieci, Zonato, & Venneri, 2002b); Trail Making
Testdpart A and part B, the latter was scored as over or under
the cut-off (Giovagnoli et al., 1996).
The retrospective nature of the study prevented matching
the tests included in the assessment of the healthy and the
patient groups. Although this is a limitation of the current
work, the cognitive domains investigated by the two sets of
tests were the same, so a qualitative comparison may be
considered informative. Future studies adopting a forward-
looking approach will be able to directly compare the healthy
and pathological conditions.
The following cognitive domains were investigated in both
the healthy and pathological participants: language (compre-
hension and lexical access), memory (visuospatial and verbal;
short- and long-term), visuoconstructive praxis, attention and
executive functions.
2.3. Statistical analysis
The analyses were performed in an R environment (R Core
Team, 2017) employing the specific package bootnet by two
authors (GT and DR) who were not involved in the case
selection or evaluation.
To describe the patterns of similarity and divergence be-
tween the groups, we examined age, education, sex and test
scores in the AD, VaE and healthy subjects.
2.3.1. Network analysis
A commonly used network model for estimating psychologi-
cal networks is a pairwise Markov random field (PMRF;
Costantini et al., 2015; Van Borkulo et al., 2014). A PMRF is a
network where nodes describe variables related by edges,
indicating conditional dependence between them. Variables
that are not connected after the weighting of the other vari-
ables are considered to be independent (Epskamp, Borsboom,
& Fried, 2018). In our work, we used a Gaussian graphical
model (GGM; Costantini et al., 2015; Lauritzen, 1996), in which
edges correspond to partial correlation coefficients, which
provides the remaining association between any two variables
after conditioning on all other variables.
This method employs the “least absolute shrinkage and
selection operator” (LASSO; Tibshirani, 1996), a form of regu-
larising penalty that treats positive and negative edge weights
equally, leading small connections to shrink to zero (Epskamp
et al., 2018; McNeish, 2015). The rationale behind the LASSO is
to return a conservative network model. If an edge is not set to
zero, it indicates that it is sufficiently robust to be included in
the model. This robust method may have a low sensitivity
(i.e., not all real edges are detected), but it has a high specificity
(i.e., few false positives) (Epskamp, Kruis, & Marsman, 2017),
suggesting a conservative and reliable approach.
The NA, together with the partial correlation coefficients of
the edges, provides other indices. A crucial factor is the
sparsity of the network, which is defined as the proportion of
88 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6

existing relationships compared to the number of total
possible connections between the variables (range 0e1).
Sparsity provides clues about the general connectivity of the
nodes; that is, a low sparsity value suggests a network with
few specific connections, while a high sparsity value means
that there are many connections in the network implying
their low specificity. To clarify the counterintuitive nature of
the term, it can be described as an edge density index. A
sparse network is a totally connected graph since it is char-
acterized by a high proportion of existing relationships rela-
tive to the number of total possible connections. Additionally,
local patterns could be investigated through centrality mea-
sures. Centrality indices can be conceived as operationalisa-
tions of node importance (Costantini et al., 2015). They are
determined by the relative importance of the focal node in the
specific network estimated. Among the numerous indices of
centrality that have been proposed, we focused on the
following measures:
- Betweenness is the number of shortest paths between any
two nodes that pass through the focal node; this measure
reveals whether the node has a mediation role and facili-
tates the connectivity among different nodes;
- Strength is the sum of the weights of the connections of the
focal node in absolute value; this measures the impact of
the focal node on its neighbours.
The nodes in our NA (18 in the healthy group and 23 in AD
and VaE groups) were the raw scores of the neuropsycholog-
ical tests described above, the sex, the age at the time of the
evaluation and the years of scholastic education.
3. Results and discussion
had more years of scholastic education (10.45; 11.31). More-
over, AD patients and healthy participants were characterized
by a higher percentage of females (AD: 68.59%; healthy:
73.33%) compared to the VaE group (45.74%).
Table 1 reports the mean and the confidence intervals for
each test score in the two groups of patients. Globally, the VaE
patients seemed to perform better than the AD patients at the
group level.
3.1. Network analysis
NA conveys very different information, and every single node
can be the target of specific interest and comments. Here, we
propose a possible path to read the NA results, which, in our
opinion, are worthy of discussion. We estimated the best
fitting network for each group. Then, we investigated the
shape of the network and its sparsity as an index of its devi-
ation from a totally connected (i.e., sparse) graph (Goswami,
Murthy, & Das, 2018). We then discussed the edges and cen-
trality indices of a few core nodes to verify how different
cognitive conditions impact the existing relationships be-
tween the neuropsychological performances. Namely, we
initially deepened the study of age and years of education,
followed by the study of the screening tests. Finally, we
focused on unexpected or missing connections. It is important
to note that, because of the exploratory nature of NA, the
examination of specific nodes and their edges can be per-
formed on any specific node and connection. In this work, the
focus was on screening tests for two reasons: first because
they are an example of the underlying logic that can be
adopted to interpret the results and second because of their
importance in the daily clinical use. Any reader, interested in
other nodes, can focus his/her attention on that node and
explore its specific connections.

Both groups of patients had similar ages [AD (lower bound of 3.1.1. Healthy participants
95% CI: 74.93; upper bound of 95% CI: 76.41); VaE (75.19; 76.77)] The 18 nodes in the network created 86 edges, with a sparsity
and years of education [AD (6.64; 7.43); VaE (6.15; 6.90)]. In index of .56 and the presence of few moderate to strong re-
contrast, the healthy subjects were younger (68.68; 69.98) and lations (Fig. 1B). A proportion of .56 existing connections

Table 1 e Average scores and 95% confidence intervals for neuropsychological tests, divided by the patient groups.

Test AD score lower CI upper CI VaE score lower CI upper CI

MMSE 20.87 20.52 21.23 24.26 23.92 24.61
Phonemic Fluency 16.59 15.60 17.58 18.78 17.92 19.65
Semantic Fluency 10.91 10.53 11.29 13.17 12.69 13.65
Token test 27.58 27.22 27.94 28.50 28.15 28.85
Digit span 4.71 4.61 4.82 4.62 4.52 4.72
Memory e Immediate Recall 1.62 1.42 1.81 3.28 3.04 3.53
Memory e Delayed Recall 1.17 .97 1.37 3.41 3.14 3.69
Learning e Immediate Recall 19.60 18.78 20.43 26.55 25.41 27.70
Learning e Immediate Recall (Intrusions) 3.19 2.79 3.59 3.08 2.68 3.47
Learning e Delayed Recall 1.28 1.04 1.52 4.23 3.82 4.65
Learning e Delayed Recall (Intrusions) .66 .51 .82 1.04 .87 1.21
Learning e Detection 21.24 20.79 21.69 24.33 23.78 24.88
Corsi span 3.74 3.65 3.83 3.88 3.79 3.96
Rey-OsterriethdCopy 18.71 17.80 19.61 19.95 19.10 20.80
Rey-OsterriethdDelayed Recall 2.43 2.11 2.74 6.39 5.80 6.98
Drawings Copy 9.80 9.57 10.03 9.52 9.31 9.73
Attentional Matrices 32.54 31.41 33.68 35.39 34.20 36.57
Raven Matrices 16.79 16.24 17.34 18.28 17.60 18.96
TMT Part A 131.99 123.54 140.44 110.05 103.86 116.25
 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
compared to the number of total possible connections (i.e.,
sparsity index) revealed relatively high specificity of the
network, with good independence of the nodes. On the basis
of the shape of the network, we did not identify any isolated
cluster of scores, suggesting that the different tests were
segregated and integrated at the same time with specific
connections among the various tests (Fig. 1A). This finding
suggested that the tests are quite selective (low sparsity and
relatively low edge coefficients) for measuring cognitive
functions in healthy people.
Education and age were both connected with many nodes
of test scores. Moreover, when expressed as centrality indices
(Fig. 2A), education and age were central to the network, as
indicated by high strength, but they did not have a role facil-
itating the connections (i.e., low betweenness). These results
suggested that age and educational level influenced cognitive
performance on different tests. On the other hand, the low
betweenness indicated that they cannot explain the majority
of correlation among the various tests.
The primary purpose of screening tests is to indicate the
probability of cognitive impairment (Cullen, O'Neill, Evans,
Coen, & Lawlor, 2007). Even if they are not intended to
replace a full neuropsychological assessment, they are thought
to explore more than one function to capture any symptoms
that may suggest cognitive deficits. Thus, we expected that
screening tests would correlate with different specific tests.
In our sample of healthy subjects, the MoCA test and the
Raven Matrices were connected with almost all the other
nodes and showed high centrality indices (betweenness and
strength). The strength indices suggested that the MoCA test
and the Raven Matrices were connected with the entire
network through numerous strong connections. Moreover,
they seemed to have a faciliatory role in mediating the asso-
ciations between the other tests, as suggested by the high
betweenness value (Fig. 1B). These results indicated that the
MoCA test and the Raven Matrices are non-specific and gen-
eral assessment tools, as is required for screening purposes.
3.1.2. VaE patients
The VaE patients included in the study showed a network of 23
nodes and 146 edges, with a sparsity value of .58. In the
healthy group, few robust connections showed the specificity
of the network because tests were integrated and segregated
at the same time where few but solid and selective relations
emerge. In the VaE sample, a few more connections appeared,
which were overall much weaker, suggesting the non-
specificity of the links and, in general, the low strength of
the relations (Fig. 1C, D).
Age and years of education seemed to have a marginal role
in VaE patient performances. Age and years of education
nodes still had many edges, but these were very weak and in a
context where every node was connected with everything.
Low centrality indices further suggest their marginal role in
the network (Fig. 2B).
The MMSE is primarily used as a screening test in cases of
suspected cognitive impairment. This test was well connected
with the other tests and reflected the global structure of the
network since they showed numerous but weak correlations.
Given the scarcity of robust relationships, the MMSE appeared
to have a central role in the pattern. It showed high
89
betweenness and strength that suggested its role in facili-
tating node connections, suggesting once again that it is a
valid test to integrate the information from the different
functions tested with the more specific tasks.
Importantly, the semantic fluency test arose as the most
central node, with a high number of relatively strong cor-
relations and many connections passing through it. Similar
to what we observed for the MMSE, semantic fluency
showed high betweenness and strength. It emerged as a
focal node with relatively strong connections that mediated
between other variables and connected most of its
neighbours. Semantic fluency appeared to be negatively
correlated with age and central to the other tests in both
healthy and VaE groups, confirming that performance on
this test generally decreases during the lifespan (Crossley,
D'Arcy, & Rawson, 1997; Moreno-Martı́nez, Laws, & Schulz,
2008).
An additional remark in the context of this network is in
regard to the token test that correlated with digit span and
Raven tests. Even if the token test is considered a verbal
comprehension test, our network analysis suggested a close
relationship with working memory and reasoning, as typically
observed in clinical practice.
3.1.3. AD patients
In this network, the 23 nodes generated 168 edges, with a
sparsity index of .66. The pattern appeared more connected
than the other groups, with a percentage of 66% edges (spar-
sity index .66) compared to the number of total possible con-
nections, and to have low specificity. From a speculative point
of view, the AD network pattern seemed to be located between
the control group and the VaE group, along a continuum that
goes from a high specific pattern, in which each function is
quite segregated and has a small number of robust relation-
ships, to a highly connected network characterized by specific
and weak connections. This network was marked by
numerous weak connections and few moderate-to-strong
connections that highlight a different pattern of balance
among the test performances compared to both the healthy
and VaE groups (Fig. 1F).
In the AD group, we identified two clusters: one centred on
memory functions and one incorporating the more “execu-
tive” function tests (Fig. 1E).
The memory cluster was composed of the prose memory
test (both immediate and delayed recall) and the semantically
related word-list test (immediate and delayed recall, detection
through distractors and intrusions). These tests and their sub-
scores were well connected among each other and segregated
from all the other tests.
On the other hand, we distinguished a frontal-executive
cluster that included tests assessing language, visuocon-
structive praxis, abstract reasoning, attention shifting ability,
and short-term memory. Notably, this pattern highlighted
the relationship between executive functions and lexicon
access tests that are typically considered to assess the
proper functionality of the frontal lobe in clinical practice. In
addition, short-term memory tests (digit and Corsi span
tests) fell into the executive cluster instead of the memory
cluster further indicating the attentional role of working
memory function.
90 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6

Fig. 1 e Networks and edge coefficient distributions. Panels A, C and E represent the estimated networks in the healthy
group, VaE patients and AD patients, respectively. Red edges indicate negative relations; green edges show positive ones.
The size and colour saturation of the edges represent the intensity of the relationships. Panels B, D and F report the edge
coefficient distribution in each group. Nodes included in the healthy group: Montreal Cognitive Assessment ¼ MoCA (Pirani,
Tulipani, & Neri, 2006); semantic fluency test ¼ SemFlu (Novelli et al., 1986); phonemic fluency test ¼ PhonFlu (Novelli et al.,
1986); ReyeOsterrieth complex figuredcopy and delayed recall ¼ ReyCopy and ReyDR (Caffarra et al., 2002a); digit
spandforward and backward ¼ DigitFw and DigitBw (Orsini et al., 1987); prose memory test ¼ Mem (Novelli et al., 1986);
Corsi spandforward and backward ¼ CorsiFw and CorsiBw (Orsini et al., 1987); Raven's progressive matrices ¼ Raven
(Basso et al., 1987; Raven, 1947); Stroop test accuracy and time performance ¼ StroopErr and StroopT (Caffarra et al., 2002b);
Trail Making Testdpart A and part B ¼ TMT_A and TMT_B (Giovagnoli, 1996); age at the assessment ¼ Age; sex of the
participant ¼ Sex; years of education ¼ Edu. Nodes included in the VaE and AD groups: Mini-Mental State
Examination ¼ MMSE (Folstein et al., 1975); semantic fluency test ¼ SemFlu (Spinnler & Tognoni, 1987); phonemic
fluency ¼ PhonFlu (Novelli et al., 1986); token test ¼ token (Spinnler & Tognoni, 1987); drawings copy ¼ DrawCopy (Spinnler
& Tognoni, 1987); ReyeOsterrieth complex figuredcopy and delayed recall ¼ ReyCopy and ReyDR (Caffarra et al., 2002a);
 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6 91

Age and years of education seemed to have a marginal role.
They were connected with few nodes and with weak con-
nections and reached low levels of centrality.
As hypothesized for the VaE group, it is possible that once
the symptoms of the cognitive impairment become
more prominent, the influences of other variables, such as age
and education, become decreasingly relevant to the test
performances.
The screening tests presented a high number of connec-
tions, as in the VaE group. The MMSE showed high strength
and influenced the network connecting the memory cluster
and the frontal-executive cluster, although the low between-
ness index suggested a marginal role in facilitating the con-
nections of the different nodes. In contrast, the Raven
matrices fulfilled an essential mediation role inside the
frontal-executive cluster, showing a high betweenness value
as well as high strength.
Finally, the semantic fluency test arose as a focal node with
high values of betweenness and strength. This variable
contributed to linking the two clusters that we identified, had
a facilitating role and influenced the whole network with a
high number of connections passing through it.
4. General discussion
The present work aimed to analyse a standard neuropsycho-
logical evaluation for cognitive impairment using NA in both
healthy and pathological conditions of cognitive ageing.
A network is a model composed of a set of nodes, repre-
senting entities, and a set of edges that connect the nodes,
representing their relations (e.g., De Nooy et al., 2011). NA is a
useful method to explore the interactions between many
variables that correlate with each other. It is a parsimonious
and understandable method to analyse complex systems.
This study showed the potential of NA in examining the
connections between cognitive tests in different mental state
conditions. A clear example is the condition of dementia,
which is characterized by impairment in multiple domains,
and a patient typically presents with many deficiencies in
performance at testing.
We used this novel perspective to approach neuropsy-
chological assessment in dementia and to study the reorga-
nization of the cognitive system using the changes in the
network of relations among the performances on different
neuropsychological tests.
The nodes considered in the network were age, sex, years
of education, and the raw scores obtained in the neuropsy-
chological tests. The domains assessed in the cognitive bat-
teries were memory, language, praxis, executive functions,
attention and abstract reasoning, together with the perfor-
mance in more general screening tests.
The group comparisons between the patients revealed that
VaE patients performed better than the AD patients, even
when they were matched for age, education, and sex. Unfor-
tunately, healthy participants were younger and had more
years of formal schooling; given the retrospective nature of
this work, we were not able to balance the groups in all the
essential characteristics. The group of healthy people was part
of a different data collection project than the groups of pa-
tients. This difference is an explicit limitation of the present
study, together with the various tests used for the healthy
group; nonetheless, the study highlights a great potentiality in
approaching cognitive impairments by considering global
performance and not a single task score.
The NA results were intended to measure, in addition to a
quantitative evaluation of the specific performances, the
balance among the different test scores, thus capturing the
reorganization of the cognitive mechanisms that is thought to
happen in dementia (Chen et al., 2013; Gits, 2016; Sohn, Yoo,
Na, & Jeong, 2014).
There are different types of information that can be extracted
from the NA. The aim of this paper was to show how this
method could be used to understand and increase the knowl-
edge about neuropsychological assessments in dementia.
We speculated that the three networks that emerged by NA
described a sort of a continuum from a high specific pattern, in
which each function is quite segregated and has a small
number of relatively strong relationships, to a highly con-
nected and diffuse network, characterized almost only by non-
specific and weak relations. On one side of this continuum, the
healthy group revealed high specificity and good indepen-
dence of the nodes. Looking at the general shape, we did not
find any specific cluster, confirming that the nodes were
segregated and, at the same time, well integrated into a
pattern characterized by relatively few robust correlations. We
suggest that this feature indicated that each test was specific
for one cognitive function, although not uniquely related to it.
On the opposite side of the continuum, the VaE patients
showed numerous weak connections, underlying a pattern
characterized by weak non-specific relations. This confirmed
a common observation in clinical practice. VaE patients may
have good performances on a specific test but fail to transfer
the skill in a more integrated and composite situation. The
disconnection of cognitive abilities might be difficult to iden-
tify in single test performances. In contrast, a comprehensive
and qualitative observation of the complete assessment can
highlight multiple mild impairments that affect different do-
mains. The multiple impairments are reflected by a pattern of
unspecific and fragile relations between the various tests in
the network structure.
The network of the AD patients was characterized by
numerous connections with weak strength and few connec-
tions with moderate-to-strong relations. We identified a

digit span forward ¼ Digit (Orsini et al., 1987); prose memory testdimmediate and delayed recall ¼ MemIR and MemDR
(Spinnler & Tognoni, 1987); Corsi span ¼ Corsi (Orsini et al., 1987); semantically related words list learningdimmediate
recall, delayed recall and detection ¼ ApprIR, AppDR, and AppD; detection through distractors ¼ ApprIR_I and AppDR_I
(Mauri et al., 1997); Raven's progressive matrices ¼ Raven (Basso et al., 1987; Raven, 1947); attentive matrices ¼ AttMat
(Spinnler & Tognoni, 1987); Trail Making Testdpart A and part B ¼ TMT_A and TMT_B (Giovagnoli, 1996); age at the
assessment ¼ Age; sex of the participant ¼ Sex; years of education ¼ Edu.
92 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6

Fig. 2 e Centrality indices and the correlations between them. Panels A, C and E report the centrality indices of the network
in the healthy group, VaE patients and AD patients, respectively. Strength centrality is higher if a node has a large number
of strong direct links with other nodes. Betweenness is high if the node is part of many shortest paths connecting two
different nodes. Centrality indices are standardized and expressed in z-scores. Panels B, D and F represent the correlation
plot between Strength and Betweenness in each group.

pattern composed of two clusters centred on memory and
frontal-executive functions. This pattern was a unique feature
among our three groups, suggesting a specific reorganization
of the cognitive abilities of AD patients, which has also been
confirmed by neuroimaging studies (Pariente et al., 2005;
Dickerson & Sperling, 2009). A similar NA study conducted in
patients with temporal lobe epilepsy suggested that these
reorganization mechanisms could be related to functional
compensation (Kellerman et al., 2016). Interestingly, the
scores on the memory tests seemed to segregate from the rest
of the network, somehow isolating that cognitive function
from the others. This result is in line with behavioural ob-
servations of patients with AD, who are strongly characterized
by memory impairments. The novelty is that there are not
 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
only memory performance decreases but also disconnections
of memory function from the functioning in the other cogni-
tive domains. Although some neuropsychological studies
have tried to distinguish between AD and VaE patients
(Mathias & Burke, 2009; Thompson, Stopford, Snowden, &
Neary, 2005), this remains a difficult differential diagnosis.
The present network analysis can provide a different point of
view in this debate, offering a new method for describing the
cognitive profile of these patients. Our results seemed to
suggest that a functional cognitive system, represented by the
network in the control group, needs fewer and stronger con-
nections than a damaged one.
The main purpose of screening tests is to obtain a reliable
and fast index of the performances in cognitive functioning
(Cullen et al., 2007). Therefore, these tests explore multiple
functions and offer a single score that accounts for the prob-
ability of having cognitive impairment.
Regarding the healthy group, centrality indices suggested
that the MoCA test and the Raven matrices influenced the
network with a high number of strong connections. Moreover,
they seemed to have a facilitating role between other func-
tions that had edges from different tests. A similar function
can be described in both VaE and AD patients for the MMSE.
Despite the controversial opinion about this test and its utility
(Dong et al., 2010; Hoops et al., 2009), the present study indi-
cated that the MMSE is correlated with most of the tests
included in a more comprehensive neuropsychological bat-
tery. However, the low strength of these connections confirms
that this test could not be used as a unique assessment tool.
Nevertheless, the MMSE could provide useful information
about the general level of the patient. In conclusion, the
MMSE, MoCA and Raven matrices emerged as adequate
screening tests since they can explore more than one func-
tion, and the general scores correlated with all the other tests
included in the usual neuropsychological batteries. They
appeared as non-specific and general assessment tools that
should be used for this scope, without assuming or over-
interpreting any specificity to the function impaired.
To better estimate the contribution of demographic infor-
mation such as age and the level of education, we included
them as nodes of the network and considered the raw scores
at each test. In the control group, age and education correlated
with the majority of the nodes. This result suggested that
performance on cognitive tests is affected by these variables,
stressing the importance of normative data for standardizing
the tests. The first clinical utility of normative data is to pro-
vide a set of norms that show normal variation in perfor-
mance depending on age and education (Tombaugh, 2004).
Thus, scores on different tests must be read while considering
the normal range of variation among healthy individuals
(Mitrushina, 2005).
Nonetheless, age and education lose much of their impact
on neuropsychological performance in cases of cognitive
impairment. Age and years of education seemed to have a
marginal role both in the VaE group and in the AD group. We
hypothesized that symptoms of cognitive impairment could
mask the influences of other variables, such as age and edu-
cation. The profile of patients with cognitive impairment
scarcely depended on personal variables, but the specific
features of the disease defined the impairments. Even if
93
cognitive impairment is generally related to increasing age
(Gao, Hendrie, Hall, & Hui, 1998; Wiederholt et al., 1993), our
network analysis suggested that personal differences are
poorly correlated with test performance.
Semantic fluency deserves an additional comment because
it emerged as a central node in all of the groups examined,
especially in the clinical groups. The semantic fluency node
showed a high number of strong correlations and many con-
nections passing through it in the three networks. Moreover,
this node contributed to linking the memory and frontal-
executive clusters in the AD group. In the healthy group, the
performances on this test showed a negative correlation with
age, suggesting a decreasing pattern during the lifespan and
confirming previous studies (Crossley et al., 1997; Moreno-
Martı́nez et al., 2008). The same pattern was identified in the
VaE patients but not in the AD subjects. Semantic fluency has
been reported to be strongly impaired in Alzheimer's disease
(Cerhan et al., 2002; Crossley et al., 1997), and particular in-
terest in the specific performances characterizing different
forms of dementia has been growing. Cerhan et al. (2002)
demonstrated that category fluency is sufficient to differen-
tiate AD patients from healthy control individuals. To explain
these findings, a temporal-lobe semantic-degradation theory
has been proposed (Butters, Granholm, Salmon, Grant, &
Wolfe, 1987; Salmon, Heindel, & Lange, 1999). The authors
suggested that access to temporal-lobe semantic stores is
necessary for category fluency, and this hypothesis is
coherent with the gradual deterioration of semantic memory
characteristic of patients with AD. Our findings confirm the
crucial importance of the semantic fluency test in assessing
the organization of cognitive networks.
The point of the present work is to illuminate NA potenti-
alities in examining a standard neuropsychological assess-
ment. NA has the potential to either provide information about
the relation of the tests or about the underlying cognitive
functions.
We showed that different information could be extracted
from the NA and outlined a method for using this new po-
tential tool in the interpretation of cognitive performances
and neuropsychological evaluations. Each test must be inter-
preted in conjunction with a wider system of information
about the patient and his/her condition, and the quantitative
data obtained during the assessment must be integrated with
a qualitative analysis of the whole performance. NA, showing
the relationship between variables and their respective
importance for each other, provides a methodological foun-
dation for this clinical process and offers a quantitative
method to characterize the complexity of the cognitive
assessment in a parsimonious and comprehensible manner.
The exemplification of typical patterns of cognitive func-
tioning among patients with different types of cognitive
impairment can inform both the interpretation of individual
performance and the description of cohorts of patients. The
usefulness for an interpretation of patient profiles lies in the
possibility to compare, at the moment from a qualitative point
of view, specific patterns of performances or the temporal
evolution of the disease with a reference network for the sus-
pected diagnosis. Future evolution of this approach could be
imagined in the context of a machine learning algorithm to
support the diagnosis of new patients. For the description of the
94 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
statistical pattern in large cohorts, NA provides a model of the
relations among neuropsychological test performances, inte-
grating several sources of information that considers the un-
derlying complexity. In this framework, NA can be part of
modern approaches to clinical neuropsychology in an attempt
to overcome limitations in clinical evaluations such as
the “impaired/spared” dichotomy when considering cognitive
performances.
5. Hypothesis generation
The exploratory nature of NA suggests that every edge can be
a relation that can be experimentally tested or on which a
researcher may focus his/her attention to calibrate new in-
terventions or theories. For instance, semantic fluency has
been reported to be strongly impaired in patients with AD
(Cerhan et al., 2002; Crossley et al., 1997), and particular in-
terest in the specific performances characterizing different
forms of dementia has been growing. Our data suggested that
semantic fluency is a central node. This result is consistent
with the theoretical framework of AD as temporal-lobe se-
mantic-degradation dementia (Butters et al., 1987; Salmon
et al., 1999). Based on our results and the semantic degrada-
tion theory, we expect to find a pattern where semantic
fluency is much less central in other types of dementia where
the access to temporal-lobe semantic stores is not a main
characteristic of the impairment (e.g., early stage of fronto-
temporal dementia).
The NA approach also provides potential ideas for where to
focus the cognitive stimulation of patients. Cognitive stimu-
lation may focus on tasks related to memory and attention to
impact two independent areas of the network. Alternatively,
cognitive stimulation may focus on semantic functions
because they seemed to be in between the memory and
attentional clusters, thus potentially impacting both areas.
We hypothesize that effective cognitive stimulation could
impact the focal node and its neighbours. For instance, it
might be useful with AD patients to work on semantic fluency
since it emerged as a central node with several connections.
Such stimulation could influence other cognitive functions
connected in the network. In contrast, stimulation focused on
the detection of previously acquired words through a dis-
tractor (AppD in Figs. 1E and 2E) would be less impactful
because of the low influence that this particular node has on
the general pattern, so that any eventual effect would be un-
likely to spread to other tasks.
Additionally, future directions may focus on different
clinical populations to define their specific patterns of cogni-
tive impairments. In addition to testing different populations,
another future direction may test the time-dependent evolu-
tion of the conditions. One of the anonymous reviewers sug-
gested that different performance levels on the MMSE among
our groups may have limited our results. Because NA is based
on correlations, the small MMSE differences unlikely
contributed to our results. Nonetheless, this observation
opens an interesting question. It is possible that a different
network characterizes the late stages of dementia than the
early stages. AD patients with a high MMSE score may have a
different balance among cognitive functions than patients
with a more compromised condition with much lower scores
on the MMSE. A possible prediction is that in the advanced
stages of dementia, the networks emerging from different
dementias would be undistinguishable because of a floor ef-
fect in the performances. It could be interesting to evaluate
the pattern in people at an early stage of AD with people at a
late stage of AD to see if the pattern remains constant
throughout the cognitive decline or if the pattern changes
with progression of the disease. This information would add
to the current study where different dementias were
compared, and this research would offer another potential
development in the study of the balance in cognitive function
in people with cognitive impairment.
CRediT authorship contribution statement
Giorgia Tosi: Conceptualization, Writing - original draft,
Writing - review & editing. Carolina Borsani: Writing - review
& editing. Stefania Castiglioni: Supervision, Writing - review &
editing. Roberta Daini: Supervision, Writing - review & editing.
Massimo Franceschi: Supervision, Writing - review & editing.
Daniele Romano: Supervision, Conceptualization, Writing -
original draft, Writing - review & editing, Visualization.
Open practices
The study in this article earned an Open Data badge for trans-
parent practices. Data for the study are available at https://osf.
io/8wn7e/?view_only¼2569a529e9424b0d86a31dcc05daa0fe.
Acknowledgement
We thank Miss Gaia Salvadore and Dr. Monika Zuzanna Wajs
for her careful proofreading of the manuscript.
We thank all the members of the Neurology Department of
the MultiMedica Hospital of Castellanza (Varese, Italy) for
their contribution to the general assessment and the caring of
the patients involved in the study.
references
Baddeley, A. (2003). Working memory and language: An overview.
Journal of Communication Disorders, 36(3), 189e208.
Basso, A., Capitani, E., & Laiacona, M. (1987). Raven's Coloured
Progressive Matrices: Normative values on 305 adult normal
controls. Functional Neurology, 2(2), 189e194.
Buckner, R. L., Hrienen, F. M., & Yeo, T. B. T. (2013). Opportunities
and limitations of intrinsic functional connectivity MRI. Nature
Reviews Neuroscience, 16, 832e837.
Butters, N., Granholm, E., Salmon, D. P., Grant, I., & Wolfe, J.
(1987). Episodic and semantic memory: A comparison of
amnesic and demented patients. Journal of Clinical and
Experimental Neuropsychology, 9(5), 479e497. https://doi.org/
10.1080/01688638708410764.
Caffarra, P., Vezzadini, G., Dieci, F., Zonato, F., & Venneri, A.
(2002a). Rey-osterrieth complex figure: Normative values in
 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
an Italian population sample. Neurological Sciences, 22(6),
443e447.
Caffarra, P., Vezzadini, G., Dieci, F., Zonato, F., & Venneri, A.
(2002b). Una versione abbreviata del test di Stroop: Dati
normativi nella popolazione italiana. Nuova Rivista Di
Neurologia, 12(4), 111e115.
Cerhan, J. H., Ivnik, R. J., Smith, G. E., Tangalos, E. C.,
Petersen, R. C., & Boeve, B. F. (2002). Diagnostic utility of letter
fluency, category fluency, and fluency difference scores in
Alzheimer's disease. The Clinical Neuropsychologist, 16(1), 35e42.
https://doi.org/10.1076/clin.16.1.35.8326.
Chen, G., Zhang, H.-Y., Xie, C., Chen, G., Zhang, Z.-J., Teng, G.-J.,
et al. (2013). Modular reorganization of brain resting state
networks and its independent validation in Alzheimer's
disease patients. Frontiers in Human Neuroscience, 7, 456.
https://doi.org/10.3389/fnhum.2013.00456.
Conti, S., Bonazzi, S., Laiacona, M., Masina, M., & Vanelli, M.
(2015). Montreal cognitive assessment (MoCA)-Italian version:
Regression based norms and equivalent scores. Neurological
Sciences, 36(2), 209e214. https://doi.org/10.1007/s10072-014-
1921-3.
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mo ~ ttus, R.,
Waldorp, L. J., et al. (2015). State of the aRt personality
research: A tutorial on network analysis of personality data in
R. Journal of Research in Personality, 54, 13e29. https://doi.org/
10.1016/j.jrp.2014.07.003.
Crossley, M., D'Arcy, C., & Rawson, N. S. B. (1997). Letter and
category fluency in community-dwelling Canadian seniors: A
comparison of normal participants to those with dementia of
the Alzheimer or vascular type. Journal of Clinical and
Experimental Neuropsychology, 19(1), 52e62. https://doi.org/
10.1080/01688639708403836.
Cullen, B., O'Neill, B., Evans, J. J., Coen, R. F., & Lawlor, B. A. (2007).
A review of screening tests for cognitive impairment. Journal of
Neurology, Neurosurgery and Psychiatry, 78(8), 790e799. https://
doi.org/10.1136/jnnp.2006.095414.
De Nooy, W., Mrvar, A., & Batagelj, V. (2011). Exploratory social
network analysis with Pajek (structural analysis in the social
sciences). Revised and Expanded.
Dickerson, B. C., & Sperling, R. A. (2009). Large-scale functional
brain network abnormalities in Alzheimer's disease: Insights
from functional neuroimaging. Behavioural Neurology, 21(1e2),
63e75.
Dong, Y., Sharma, V. K., Chan, B. P. L., Venketasubramanian, N.,
Teoh, H. L., Seet, R. C. S., & Chen, C. (2010). The Montreal
Cognitive Assessment (MoCA) is superior to the Mini-Mental
State Examination (MMSE) for the detection of vascular
cognitive impairment after acute stroke. Journal of the
Neurological Sciences, 299(1e2), 15e18.
Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating
psychological networks and their accuracy: A tutorial paper.
Behavior Research Methods, 50(1), 195e212. https://doi.org/
10.3758/s13428-017-0862-1.
Epskamp, S., Kruis, J., & Marsman, M. (2017). Estimating
psychopathological networks: Be careful what you wish for.
Plos One, 12(6), 1e13. https://doi.org/10.1371/
journal.pone.0179891.
Erkinjuntti, T., Inzitari, D., Pantoni, L., Wallin, A., Scheltens, P.,
Rockwood, K., & Desmond, D. W. (2000a). Research criteria for
subcortical vascular dementia in clinical trials. In Advances in
dementia research (pp. 23e30). Vienna: Springer.
Erkinjuntti, T., Inzitari, D., Pantoni, L., Wallin, A., Scheltens, P.,
Rockwood, K., et al. (2000b). Limitations of clincal criteria for
the diagnosis of vascular dementia in clinical trials: Is a focus
on subcortical vascular dementia a solution? Annals of the New
York Academy of Sciences, 903(1), 262e272.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental
state: A practical method for grading the cognitive state of
95
patients for the clinician. Journal of Psychiatric Research, 12(3),
189e198.
Gao, S., Hendrie, H. C., Hall, K. S., & Hui, S. (1998). The
relationships between age, sex, and the incidence of dementia
and Alzheimer disease: A meta-analysis. Archives of General
Psychiatry, 55(9), 809e815.
Giovagnoli, A. R., Del Pesce, M., Mascheroni, S., Simoncelli, M.,
Laiacona, M., & Capitani, E. (1996). Trail making test:
Normative values from 287 normal adult controls. Italian
Journal of Neurological Sciences, 17(4), 305e309. http://www.ncbi.
nlm.nih.gov/pubmed/8915764.
Gits, H. C. (2016). Relating connectivity and graph analysis to
cognitive function in Alzheimer's disease. Michigan Journal of
Medicine, 1(1), 45e65. https://doi.org/10.3998/
mjm.13761231.0001.111.
Goswami, S., Murthy, C. A., & Das, A. K. (2018). Sparsity measure
of a network graph: Gini index. Information Sciences, 462, 16e39.
https://doi.org/10.1016/j.ins.2018.05.044.
Hafkemeijer, A., Mo € ller, C., Dopper, E. G., Jiskoot, L. C.,
Schouten, T. M., van Swieten, J. C., & Scheltens, P. (2015).
Resting state functional connectivity differences between
behavioral variant frontotemporal dementia and Alzheimer's
disease. Frontiers in Human Neuroscience, 9, 474. https://doi.org/
10.3389/fnhum.2015.00474.
Hoops, S., Nazem, S., Siderowf, A. D., Duda, J. E., Xie, S. X.,
Stern, M. B., et al. (2009). Validity of the MoCA and MMSE in the
detection of MCI and dementia in Parkinson disease.
Neurology, 73(21), 1738e1745.
Huang, C. C., Lin, S. H., & Lin, C. P. (2016). The effect of network
template from normal subjects in the detection of network
impairment. Acta Neurobiologiae Experimentalis, 76, 294e303.
Kalaria, R. N., & Erkinjuntti, T. (2006). Small vessel disease and
subcortical vascular dementia. Journal of Clinical Neurology, 2(1),
1e11.
Kellermann, T. S., Bonilha, L., Eskandari, R., Garcia-Ramos, C.,
Lin, J. J., & Hermann, B. P. (2016). Mapping the
neuropsychological profile of temporal lobe epilepsy using
cognitive network topology and graph theory. Epilepsy &
Behavior, 63, 9e16. https://doi.org/10.1016/j.yebeh.2016.07.030.
Lauritzen, S. L. (1996). Graphical models (Vol. 17). Clarendon Press.
Lehmann, M., Madison, C., Ghosha, P. M., Millera, Z. A.,
Greiciusd, M. D., Kramera, J. H., et al. (2016). Loss of functional
connectivity is greater outside the default mode network in
non-familial early-onset Alzheimer's disease variants.
Neurobiology of Aging, 21(18), 4062e4072. https://doi.org/
10.1158/1078-0432.CCR-15-0428.Bioactivity.
Mathias, J. L., & Burke, J. (2009). Cognitive functioning in Alzheimer's
and vascular dementia: A meta-analysis. Neuropsychology, 23(4),
411e423. https://doi.org/10.1037/a0015384.
Mauri, M., Carlesino, G. A., Graceffa, A. M. S., Loasses, A.,
Lorusso, S., Sinforiani, E., & Caltagirone, C. (1997).
Standardizzazione di due nuovi test di memoria:
Apprendimento di liste di parole correlate e non correlate
semanticamente. Archivio Di Psicologia Neurologia e Psichiatria,
58, 621e645.
McKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T.,
Jack, C. R., Jr., Kawas, C. H., & Mohs, R. C. (2011). The diagnosis
of dementia due to Alzheimer's disease: Recommendations
from the National Institute on Aging-Alzheimer’s Association
workgroups on diagnostic guidelines for Alzheimer's disease.
Alzheimer's & dementia, 7(3), 263e269.
McNeish, D. M. (2015). Using lasso for predictor selection and
to assuage overfitting: A method long overlooked in
behavioral sciences. Multivariate Behavioral Research, 50(5),
471e484.
Mitrushina, M., Boone, K. B., Razani, J., & D'Elia, L. F. (2005).
Handbook of normative data for neuropsychological assessment.
Oxford University Press.
96 c o r t e x 1 2 4 ( 2 0 2 0 ) 8 5 e9 6
Monaco, M., Costa, A., Caltagirone, C., & Carlesimo, G. A. (2013).
Forward and backward span for verbal and visuo-spatial data:
Standardization and normative data from an Italian adult
population. Neurological Sciences, 34(5), 749e754. https://
doi.org/10.1007/s10072-012-1130-x.
Moreno-Martı́nez, F. J., Laws, K. R., & Schulz, J. (2008). The impact
of dementia, age and sex on category fluency: Greater deficits
in women with Alzheimer's disease. Cortex, 44(9), 1256e1264.
Novelli, P., Capitani, L., Vallar, C., & Cappa, S. (1986). Test di
fluenza verbale. Archivio di Psicologia. Neurologia e Psichiatria,
47(4).
Orsini, A., Grossi, D., Capitani, E., Laiacona, M., Papagno, C., &
Vallar, G. (1987). Verbal and spatial immediate memory span:
Normative data from 1355 adults and 1112 children. Italian
Journal of Neurological Sciences, 8(6), 539e548. http://www.ncbi.
nlm.nih.gov/pubmed/3429213.
Pariente, J., Cole, S., Henson, R., Clare, L., Kennedy, A., Rossor, M.,
& Frackowiak, R. S. (2005). Alzheimer's patients engage an
alternative network during a memory task. Annals of
Neurology: Official Journal of the American Neurological Association
and the Child Neurology Society, 58(6), 870e879.
Pirani, A., Tulipani, C., & Neri, M. (2006). Italian translation of MoCA
test and of its instructions. Retrieved from: http://www.
mocatest.org.
R Core Team. (2017). R: A language and environment for statistical
computing. Version 3.3. 3. Vienna, Austria: R Foundation for
Statistical Computing.
Raven, J. C. (1947). Progressive matrices: Sets A, AbB. Board and Book
Forms. https://ci.nii.ac.jp/naid/10015457223/.
Sachdev, P., Kalaria, R., O'Brien, J., Skoog, I., Alladi, S., &
Ganguli, M. (2014). “Diagnostic criteria for vascular cognitive
disorders: A VASCOG statement” Alzheimer. Drug and Alcohol
Dependence, 28(3), 206e218.
Salmon, D. P., Heindel, W. C., & Lange, K. L. (1999). Differential
decline in word generation from phonemic and semantic
categories during the course of Alzheimer's disease:
Implications for the integrity of semantic memory. Journal of
the International Neuropsychological Society, 5(7), 692e703.
https://doi.org/10.1017/S1355617799577126.
Sohn, W. S., Yoo, K., Na, D. L., & Jeong, Y. (2014). Progressive
changes in Hippocampal resting-state connectivity across
cognitive impairment. Alzheimer Disease and Associated
Disorders, 28(3), 239e246. https://doi.org/10.1097/
WAD.0000000000000027.
Spinnler, H., & Tognoni, G. (1987). Taratura e standardizzazione
italiana di test neuropsicologici. Italian Journal of Neurological
Sciences, 8(8), 8e120.
Thompson, J. C., Stopford, C. L., Snowden, J. S., & Neary, D. (2005).
Qualitative neuropsychological performance characteristics in
frontotemporal dementia and Alzheimer's disease. Journal of
Neurology, Neurosurgery & Psychiatry, 76(7), 920e927.
Tibshirani, R. (1996). Regression shrinkage and selection via the
lasso. Journal of the Royal Statistical Society. Series B
(Methodological), 267e288.
Tombaugh, T. N. (2004). Trail making test A and B: Normative data
stratified by age and education. Archives of Clinical
Neuropsychology, 19(2), 203e214.
Van Borkulo, C. D., Borsboom, D., Epskamp, S., Blanken, T. F.,
Boschloo, L., Schoevers, R. A., et al. (2014). A new method for
constructing networks from binary data. Scientific Reports, 4, 5918.
van den Heuvel, M. P., & Hulshoff Pol, H. E. (2010). Exploring the
brain network: A review on resting-state fMRI functional
connectivity. European Neuropsychopharmacology, 20(8),
519e534. https://doi.org/10.1016/j.euroneuro.2010.03.008.
Van Der Maas, H. L. J., Dolan, C. V., Grasman, R. P. P. P.,
Wicherts, J. M., Huizenga, H. M., & Raijmakers, M. E. J. (2006). A
dynamical model of general intelligence: The positive
manifold of intelligence by mutualism. Psychological Review,
113(4), 842e861. https://doi.org/10.1037/0033-295X.113.4.842.
Weintraub, S., Wicklund, A. H., & Salmon, D. P. (2012). The
neuropsychological profile of Alzheimer disease. Cold Spring
Harbor Perspectives in Medicine, 2(4), a006171. https://doi.org/
10.1101/cshperspect.a006171.
Wiederholt, W. C., Cahn, D., Butters, N. M., Salmon, D. P., Kritz-
Silverstein, D., & Barrett-Connor, E. (1993). Effects of age,
gender and education on selected neuropsychological tests in
an elderly community cohort. Journal of the American Geriatrics
Society, 41(6), 639e647.