PHD Thesis Adela Is Voran U

F rom syndromes to Symptoms:
Network Models of Psychosis and beyond
ADELA-MARIA ISVORANU
ISBN: 978-94-6421-539-7
© 2021 Adela-Maria Isvoranu

All rights reserved.
This research was supported by an NWO talent grant,

grant number 406.16.516
Cover design: @alphacreative.des

Layout: Yasmin Katlich, persoonlijkproefschrift.nl
Printed by: Ipskamp printing | proefschriften.net
PROMOTIECOMMISSIE
Promotiecommissie
Promotor: prof. dr. D. Borsboom Universiteit van Amsterdam
Promotor: Prof. dr. D. Borsboom

Copromotor: prof. dr. J.J. van Os Universiteit vanUtrecht
Universiteit Amsterdam
Copromotor:
Overige leden: Prof. dr. J. van
prof.Os
dr. C.L.H. Bockting Universiteit
Universiteit Utrecht
van Amsterdam
prof. dr. L. de Haan Universiteit van Amsterdam
prof. dr. H.L.J. van der Maas Universiteit van Amsterdam
Overige leden: Prof. dr. C.prof.
L. H. Bockting
dr. R.J. McNally Universiteit
Harvard University van
Amsterdam dr. L.L.N.J. Boyette Universiteit van Amsterdam
dr. M.K. Deserno Max Planck Institute for Human
Prof. dr. L. de Haan Universiteit van Amsterdam
Development
Prof. dr. H. L. J. van der Maas Universiteit van Amsterdam
Prof. dr. R. J. McNally Harvard University
Faculteit der Maatschappij- en Gedragswetenschappen
Dr. L. L. Boyette Universiteit van Amsterdam
Dr. M. K. Deserno Max Planck Institute for Human
Development
Faculteit: Faculteit der Maatschappij- en Gedragswetenschappen

Table of Contents
Chapter 1. General Introduction 9
Part I. Embracing Complexity in Psychopathology 15
Chapter 2. Introducing a Network Conceptualization to Psychosis 17
2.1 Introduction 19
2.2 Psychosis: disease model versus network model 20
2.3 Toward incorporating environmental and genetic risk into network models 26
2.4 Conceptualizing treatment from a network perspective 30
2.5 Concluding remarks 33
Chapter 3. Extended Network Analysis of Symptomatology 35
3.1 Introduction 37
3.2 Methods 38
3.3 Results 41
3.4 Discussion 47
Part II. On the Integration of Risk Factors into Network Models 53
Chapter 4. Environmental Impact in Psychotic Disorder 55
4.1 Introduction 57
4.2 Psychotic disorder – standard approaches and a network alternative 57
4.3 Network analysis of general population psychopathology data 58
Chapter 5. Pathways Between Childhood Trauma and Psychotic Symptoms 63
5.1 Introduction 65
5.2 Methods 67
5.3 Results 70
5.4 Discussion 77
Chapter 6. Short- and Long-Term Links between Cannabis (Ab)use and Psychotic 83
Symptomatology
6.1 Introduction 85
6.2 Methods 86
6.3 Results 89
6.4 Discussion 95
Chapter 7. From Speech Illusions to Onset of Psychotic Disorder 101
7.1 Introduction 103
7.2 Methods 105
7.3 Results 109
7.4 Discussion 115
Chapter 8. Toward Incorporating Genetic Risk Scores into Symptom Networks of 121
Psychosis
8.2 Methods 125
8.3 Results 127
8.4 Discussion 131
Part III. Psychotic Disorders and Comorbid Features 139
Chapter 9. Obsessive-Compulsive Symptoms in the At Risk Mental State for Psychosis 141
9.2 Methods 144
9.3 Results 147
9.4 Discussion 156
Chapter 10. Autistic Symptoms as Linking Pin to Social Functioning in Psychosis 163
10.2 Methods 167
10.3 Results 170
10.4 Discussion 177
Part IV. Methodological Notes and Novel Developments in Network Psychometrics 183
Chapter 11. Reporting Standards for Psychological Network Analyses in Cross-sectional Data 185
11.2 Reporting standards for the methods section 190
11.3 Reporting standards for the results section 195
11.4 Illustrative examples 200
Chapter 12. Network Models of Post-traumatic Stress Disorder: A Meta-analysis 205
12.2 Methods 211
12.3. Results 224
12.4. Discussion 235
12.5. Concluding remarks 241
Chapter 13. Which Estimation Method to Choose? Deriving Guidelines for Applied 245
Researchers
13.2 Methods 250
13.3. Results 262
13.4 Discussion 272
Chapter 14. Discussion 277
14.1 Summary 278
14.2 Clinical implications 278
14.3 Methodological considerations 281
14.4 Future research directions 284
Appendices 289
A. Supplement to Chapter 3 291
B. Supplement to Chapter 5 301
C. Supplement to Chapter 6 307
D. Supplement to Chapter 7 317
E. Supplement to Chapter 8 329
F. Supplement to Chapter 9 333
G. Supplement to Chapter 10 343
H. Supplement to Chapter 11 359
I. Supplement to Chapter 12 377
References 397
Summary 424
Nederlandse Samenvatting 428
Funding, Publications, and Short CV 432
Acknowledgements – Dankwoord 438
1
General Introduction
Chapter 1 | General Introduction
Unlike common mental disorders uncovered since the period of antiquity,

psychosis and the diagnostic concept of schizophrenia are relatively recent, first
being recognized in the 19th century, when Emil Kraepelin proposed the name
‘dementia praecox’ (Jablensky, 2010). Following his work, further subtypes and
dichotomies were suggested, with Eugen Bleuler coining the term ‘schizophrenia’
and arguing for a group of diseases rather than a single entity. Nonetheless, to date,
it is believed that Kraepelin’s categorical nosology is still very much at the center of
psychiatric discourse (Jablensky, 2010). Such categorical approaches, emerging from
the field of medicine and being successful through their practical utility, encourage
the notion of discrete entities with similar clinical presentation and prognosis.
While frequently the case with medical diseases, psychological and psychiatric
disorders rarely exist in isolation and often display much more complex etiology.
As such, categorical approaches often disregard evidence that mental disorders can
be multidimensional, with subjects meeting criteria for more than one disorder
simultaneously and not displaying clear boundaries between syndromes. More
than a century from the diagnostic concept of ‘dementia praecox’, schizophrenia–
and what is known now as the spectrum of psychotic disorders–remains puzzling,
with unclear incidence, symptom course and treatment across distinct groups of
individuals (Jablensky, 2010; van Os & Kapur, 2009).
Even though it cannot be refuted that categorical approaches, alongside the
development of diagnostic classifications such as the Diagnostic and Statistical
Manual of Mental Disorders (American Psychiatric Association, 2000), may have
promoted better diagnostic and communication, the failure of identifying clear
causes, genetic markers, and pathogenesis of psychosis raises uncertainty about
the validity of the construct. Recent years have therefore seen an increase in calls
toward discarding categorical approaches and replacing ‘schizophrenia’ with a
dimensional view or a continuum of ‘psychosis’, as well as more general calls for
advancing alternative approaches to the study of mental disorders (Bentall et al.,
1988; Borsboom & Cramer, 2013; Craddock & Owen, 2005; Jablensky, 2010;
Kendler et al., 2011; Tandon et al., 2008b).
In light of these calls, the current thesis focuses on an alternative
conceptualization of schizophrenia and psychotic disorders: the conceptualization
of mental disorders as networks of interacting symptoms (Borsboom, 2017;
Borsboom & Cramer, 2013). Rather than centering on psychosis as a syndrome,
the focus in this book shifts to concrete symptoms and their interrelations, as well as
their interactions with common risk factors: an approach that is often well aligned
Page | 10
with how clinicians and practitioners view and approach treatment. Specifically,
the aim of this book is fourfold: (1) to introduce network approaches in the context
of both broad symptomatology and psychotic symptomatology, (2) to expand on
these approaches by integrating risk factors outside symptomatology into network
models, (3) to move toward constructing network models of psychosis and common
comorbid features, and (4) to introduce guidelines for applied researchers, as well
as novel methodological developments in the field of network psychometrics. Each
of these goals is described in more detail below.
Part I of the book (Chapters 2-3) introduces the reader to novel methodological
conceptualizations, both in the context of mental health in general, as well as
particular to psychosis. To this end, Chapter 2 provides a detailed account of the
network approach to mental disorders in the context of psychotic symptomatology,
discussing differences between standard approaches to psychotic disorders and
network models. Further, the chapter provides a framework to study (environmental
and genetic) risk factors from a network perspective, as well as a brief discussion on
how network models can be integrated into treatment approaches. This chapter can
be read as a more extensive introduction to this book. Chapter 3 provides a more
general account on the use of network inference in psychopathology and psychiatry,
by focusing on a wide array of mental and physical conditions, as well as on the role of
functioning and quality of life in the larger picture of psychopathology. This chapter
briefly introduces how comorbidity can be understood from a network perspective.
Part II of the book (Chapters 4-8) expands on the first two chapters, by
focusing explicitly on the integration of risk factors into network models. While
there is widespread agreement that both genetic and environmental risk factors
can influence the onset and progression of psychosis, if all psychotic symptoms
were indeed caused by an underlying disease entity, one would expect to identify
that symptoms have the same (or at least similar) risk factors. That is because, as
elaborated in Chapter 2, risk factors are expected to influence the liability to develop
a disorder, while the disorder would, in turn, cause the symptoms. Current findings
in psychopathology suggest, however, that symptoms substantively differ in terms
of their risk factors (Fried & Nesse, 2014; van Os & Kapur, 2009), with genetic
vulnerability being partly shared with other disorders such as bipolar disorder
and autism (van Os & Kapur, 2009). These accounts support the importance of
investigating individual symptoms and their interactions not only with respect to
each other, but also to risk factors. This section therefore centers on (environmental
and genetic) risk factors in the context of psychotic symptomatology.
Page | 11
Chapter 1 | General Introduction
Part III of the book (Chapters 9-10) moves toward the construction of network
models that further include common comorbid features of psychosis, such as
obsessive-compulsive and autistic symptomatology–with which psychosis often
shares common etiology–as well as social functioning disfunctions. These chapters
highlight the importance of moving beyond a single syndrome and expand on how
comorbidity can be approached and understood from a network conceptualization
of psychopathology.
Part IV of the book (Chapters 11-13) aims to move beyond the field of psychosis
and shifts focus to methodological notes and developments in the more general
field of network psychometrics. This section could further be seen as a guide to
applied researchers–including clinicians and psychiatrists–aiding with reporting
standards for network models, as well as with choices in estimators suitable to
answer common research questions. Part IV includes an introduction to a novel
methodology allowing conducting meta-analysis of network models, alongside a
large network meta-analysis of post-traumatic stress disorder, the field in which to
date network approaches have become highly popular and widely used.
Finally, Chapter 14 concludes the thesis by integrating results both from a
clinical and a methodological point of view, as well as by discussing challenges of
current approaches and important future research directions.
Page | 12
Page | 13
Part I
EMBRACING COMPLEXITY
IN PSYCHOPATHOLOGY
Introducing a Network
2
Conceptualization to
Psychosis
This chapter is adapted from:

Isvoranu, A. M., Boyette, L. L., Guloksuz, S., Borsboom, D. Network Models of
Psychosis. In: Tamminga, C. A., Ivleva, E. I., Reininghas, U., van Os, J. (2020).
Psychotic Disorders: Comprehensive Conceptualization and Treatments. New York,
NY: Oxford University Press.
Chapter 2 | Psychosis: A Network Conceptualization
Abstract
Recent years have seen a rise in the modeling of psychotic disorders as networks of
interacting symptoms. The centerpiece of network modeling lies in the idea that
symptoms are active causal agents in producing disorder states, and that the study of
their causal interaction is central to progress in understanding and treating mental
disorders. The patters of interaction can be visualized in a network structure,
in which variables (e.g., symptoms, environmental factors, genetic factors) are
represented as nodes and the presence of an edge between any two nodes implies
the existence of a statistical association, which does not vanish upon controlling
for all of the other nodes in the network.
This first book chapter aims to give a general introduction to the network
approach to mental disorders in the context of psychotic symptomatology. We
discuss differences between standard approaches to psychotic disorders and network
models, as well as provide a framework to study (environmental and genetic) risk
factors from a network perspective. We complete the chapter with a discussion on
how network models can be integrated into treatment approaches.
Page | 18
2.1 Introduction
Disorders within the psychosis spectrum are highly heterogeneous and
multifactorial (Tandon et al., 2008a; Weinberger & Harrison, 2010). Despite
intensive research over the past century, the causes and pathogenesis of psychosis
are still unclear, and no genetic marker has been consistently linked to developing
a psychotic disorder (Borsboom & Cramer, 2013; Kendler et al., 2011; Tandon
et al., 2008b). In recent years, in an attempt to overcome this conundrum, the
conceptualization of mental disorders as networks of interacting symptoms has
gained considerable ground (Borsboom, 2017; Borsboom & Cramer, 2013).
This conceptualization aligns well with practitioners’ viewpoints as it focuses on
concrete symptoms and their interrelations, rather than on abstract latent disorders
or syndromes (Fried, 2015).
Even though direct influences of one symptom on another are routinely observed
in clinical practice (e.g., if a patient shows social withdrawal, this may soon lead to
the patient displaying paranoid ideation and vice versa), in classical (psychometric)
approaches to psychosis, which underlie most common psychometric practices in
research, such direct influences are not modeled. Instead, symptoms are treated as
passive psychometric indicators of a (set of) latent variable(s) – thus, it is assumed
that symptoms are simply a result of the underlying disorder, rather than influencing
each other (Borsboom, 2017; Borsboom & Cramer, 2013; Schmittmann et al.,
2013). As a result, correlations between symptoms are in a nontrivial sense spurious:
symptoms cluster together because of their common dependence on the disorder.
The assumption that correlations between symptoms arise from a common
cause, which has been deemed problematic by both psychometricians and
clinicians (Borsboom & Cramer, 2013; Fried, 2015), has spurred the development
of alternative psychometric approaches to mental disorders, in which symptoms are
viewed instead as networks of mutually interacting components. Collectively, these
lines of research have become known as the network approach to mental disorders.
The centerpiece of the network approach is the idea that symptoms are active causal
agents in producing disorder states, and that the study of their causal interaction is
central to progress in understanding and treating mental disorders.
This chapter aims to introduce the network approach to mental disorders in
the context of psychotic symptomatology. We first discuss standard approaches
to psychotic disorders, highlighting unresolved issues, and we then provide an
introduction to network models of psychosis, with a focus on the general theoretical
Page | 19
framework. We concentrate on how (environmental and genetic) risk factors can be

understood from a network perspective and how they can be included in network
models. We complete the chapter with a discussion on how network models can
be integrated into treatment approaches.
2.2 Psychosis: disease model versus network model

“For the past twenty years, we have been biding time and asserting that certain
etiological information is just around the corner (Gottesman et al., 1987); we still
wait (Sullivan, 2008). A reconsideration of our basic strategies and fundamental
assumptions may be in order.” (Tandon et al., 2008b)
The concept of schizophrenia, and generally the disorders within the psychosis
spectrum, as a disease entity emerged over a century ago (Tandon et al., 2008a)
and it soon became a central (theoretical and applied) framework in the fields
of psychiatry and clinical psychology. This framework is now known in mental
health research and psychiatry as the disease model (Borsboom & Cramer, 2013).
The scientific terminology used by clinicians and researchers alike quickly aligned
with this view, and to date idioms such as ‘suffering from schizophrenia’ are used
to indicate the presence of a mental disorder (Borsboom, 2017). In addition,
comprehensive research has argued in favor of using the term disease when
describing schizophrenia, as the term disease implies a discrete entity with a specific
etiology and therefore better aligns with the clarity and heuristic value of the disease
model (Tandon et al., 2008a). Today, the use of the term ‘schizophrenia’ almost
immediately implies disease entity and in spite of fuzzy and unreliable results,
biological approaches still focus on reverse-engineering this hypothesized disease
entity using case-controls paradigms (Guloksuz & van Os, 2018).
The disease model strongly relies on the assumption that the nature of symptoms
and the interactions between symptoms are not themselves relevant. It is argued that
their presence is simply a result of the underlying disorder and that both research
and treatment should focus on the disorder per se (see Figure 2.1). One example of
this line of thinking that was very influential in the past decades is the idea that
mental disorders are in fact diseases of the brain–literally ‘brain disorders’–, the
nature of which will be uncovered through neuroscientific investigation (Insel &
Cuthbert, 2015). Notably, it is not neuroscience per se that is the problem, but the
Page | 20
reductionist taxonomy in which current standard (psychometric) approaches are
grounded on.
Paranoia
Hallucinations
SZ
Hostility
Figure 2.1 (Simplified) visualization of the disease model for schizophrenia. According to this
model, schizophrenia (SZ) is the root cause of its symptoms (three symptom examples as extracted
from the Positive and Negative Syndrome Scale; Kay et al., 1987), which co-occur only because they
are all caused by the same disorder.
With this framework in mind, extensive research has focused on identifying

psychological and biological essences of mental disorders (Borsboom & Cramer,
2013) and on identifying common causes that give rise to symptomatology
(Borsboom, 2017). However, this quest has not been as successful as expected
and perhaps unsurprisingly the underlying assumption that symptoms are
interchangeable has proved problematic – to date few researchers and even fewer
clinicians defend it (Borsboom & Cramer, 2013; Fried, 2015; Kendler et al., 2011).
Unlike in a medical context, where symptoms point to a disorder, which can then be
confirmed with further testing (e.g., blood testing, x-ray), there is no diagnostic tool
for any mental disorder (Kapur et al., 2012). The diagnosis is entirely based on the
clinical manifestations, and the progress of a disorder is determined only by tracing
the progress of the symptoms and global functioning. Within a clinical context the
dynamics between the symptoms themselves therefore become of central interest
and a causal structure such as Paranoid Ideation  Anxiety  Social Isolation can
often be observed. Given that the disease model does not allow (neither theoretically
nor statistically) for such direct causal relations between symptoms, recent years
Page | 21
have seen an emergence of network models of mental disorders, in which disorders

are presumed to arise from this exact causal interplay between symptoms instead of
being the result of a common cause. This approach, which turns the disease model
on its head because it proposes that interactions between symptoms are themselves
the causes of disorder states, aligns well with clinical intuition and current findings
in the field; as a result, it currently spurs many novel research approaches to the old
questions of what mental disorders are and where they come from.
Following these developments, the hypothesis has been advanced that
common cause mechanisms for mental disorders cannot be found because no such
mechanisms exist (Borsboom, 2017). This hypothesis has spread quickly through
the research fields of psychology and psychiatry. In particular, it has been argued
that symptoms may cluster together not only because of a shared origin, but because
they have the power to directly trigger and influence each other (Borsboom &
Cramer, 2013; Fried, 2015). This novel theoretical framework became known
as the network approach to psychopathology (Borsboom & Cramer, 2013) and
the fundamental assumption underlying it is that mental disorders arise from a
complex interplay between symptoms and psychological, biological, and sociological
components (Epskamp, Waldorp, et al., 2018); this interplay can be captured in a
network model (see Figure 2.2). To date, network models have been applied to a
wide range of constructs, including depression (Boschloo, van Borkulo, et al., 2016;
Bringmann et al., 2015; Cramer, 2012; Fried, Bockting, et al., 2015; Fried, Epskamp,
et al., 2015; Fried & Nesse, 2014; van Borkulo et al., 2016; van de Leemput et al.,
2014), post-traumatic stress disorder (McNally et al., 2015), psychosis (Bak et al.,
2016; Fonseca-Pedrero et al., 2018; Galderisi et al., 2018; Isvoranu et al., 2016, 2017;
van Rooijen et al., 2017), autism (Deserno et al., 2016; Ruzzano et al., 2015), social
anxiety disorders (Beard et al., 2016; Heeren & McNally, 2016), substance abuse
(Rhemtulla et al., 2016), personality (Costantini et al., 2015; Cramer et al., 2012),
quality of life research (Kossakowski et al., 2015), and the more general structure
of psychopathology (Borsboom et al., 2011; Boschloo, Schoevers, et al., 2016; Tio
et al., 2016).
While networks models are based on innovative statistical methodology
and offer radically new opportunities for theory formation and mathematical
simulation of mental disorders, the notion that symptoms influence each other
has been well-known. In fact, such symptom-symptom interactions, transcending
traditional diagnostic categories, have long been observed and acknowledged by
clinical practitioners – network models just naturally accommodate it (Fried &
Page | 22
Cramer, 2017). Importantly, in contrast to the mysterious latent entities posited in
conventional statistical models of mental disorders, interactions between symptoms
are often mundane and well understood. For instance, if a person thinks they are
being followed by someone who would like to harm them (i.e., present signs of
paranoid ideation), this person may soon become too anxious to walk outside and
to freely carry out daily activities (i.e., present signs of social isolation). This behavior
could then in turn re-activate the initial paranoia, as the patient is not provided
with any counterevidence of the paranoid belief, allowing it to be sustained. In this
way, the symptoms enter a feedback loop in which they reinforce each other; such
feedback loops are often identified and are central in clinical practice. If symptom
interactions are sufficiently strong, or if the symptom network is sufficiently large,
the system can then enter into a stable state of sustained symptom activation.
This arises when the interactions between symptoms are such that the symptom
network is able to maintain its own activation – a state that we phenomenologically
recognize as a mental disorder.
To wit, within this framework, disorders are not ‘mere labels’ or social
constructions: a stable state of problem activation is as real as anything, and as such
in the network approach mental disorders are explained, but not explained away.
Importantly, however, in the network approach a disorder such as schizophrenia is
no longer understood as a disease entity that explains the symptoms. Instead, it is
an alternative stable state of a network that is constituted by the causal interactions
between symptoms. It can be illuminating to think of this in terms of an ecosystem
analogy. It is well known that ecosystems, such as lakes, have alternative stable states
(e.g., a lake can be clear of turbid; Scheffer et al., 2009). These stable states are a
function of a highly complex interplay between various components that together
determine their overall state (e.g., the number of fish in the lake, the state of the algae
population, the amount of nitrate in the soil, etc.). Importantly, the stable states
of a lake are real: the lake is either clear or turbid as a matter of objective empirical
fact. However, the stable state of the lake is not properly conceptualized as a latent
disorder that ‘underlies’ or ‘explains’ the state of relevant components in the system:
it refers to a global, emergent state of the lake that arises out of the interactions
between the components in the system, rather than a mysterious unknown latent
entity that is responsible for the aligned state of the system components as we
may encounter it (e.g., turbid water, an increased algae population, fewer fish).
In the same way, network approaches conceptualize disorders as emergent global
Page | 23
properties of the symptom interactions: they are real, but do not signify a central
underlying disorder in the form of a generally applicable pathogenic pathway.
Symptom interactions can be captured in a network structure (see Figure 2.2 for
a hypothetical example of a network structure). Within such a network structure,
symptoms are represented as nodes, while the relations between the symptoms are
represented as edges (Costantini et al., 2015; Epskamp et al., 2012). Notably, the
nodes within a network do not have to be not limited to symptoms, but can also
be, for instance, risk factors pertaining to the disorder (the next section details
on how the influence of genes and environment can be included in symptom
networks), as well as non-symptomatic variables or protective variables. Usually,
a blue edge indicates a positive relation between nodes, while a red edge indicates
a negative relation between nodes (Epskamp et al., 2012). For instance, if two
nodes representing the typical symptoms anxiety and paranoid ideation are linked
together by a blue edge, this link indicates that the two nodes may positively activate
each other (i.e., if a person is suffering from anxiety, the person is also most likely to
present symptoms of paranoid ideation and/ or vice versa). Likewise, if two nodes
representing the typical symptoms excitement and blunted affect are linked together
by a red edge, this link indicates that the two nodes may negatively activate each
other (i.e., if a person is suffering from blunted affect, the person is also less likely
to present symptoms of excitement and/ or vice versa). Factors that impinge on
the symptoms from outside together make up the external field of the symptoms.
Note that in this case ‘from outside’ means ‘from outside of the network’ rather
than ‘from outside of the person’, so in addition to precipitating life events, drug
use, or social circumstances, the external field can also include variable that relate
to brain function and other bodily processes (Borsboom, 2017).
Page | 24
Hal
Hos Del
Symptoms
Hal: Hallucinations
Del: Delusions
Exc: Excitement
Gra: Grandiosity
Par: Paranoia
Hos: Hostility
Par Exc
Gra
Figure 2.2 Visualization of a hypothetical network model of psychosis (six symptom examples as
extracted from the Positive and Negative Syndrome Scale; Kay et al., 1987), constructed using the
qgraph (Epskamp et al., 2012) package in the R Statistical Software (R Core Team, 2015). Blue edges
indicate positive associations, while red edges indicate negative associations between nodes.
In the absence of a strong theory on the structure of these interactions, networks are
commonly constructed from empirical data in an explorative fashion, by searching
for the network structure that best explains the associations present in a dataset
(Borsboom & Cramer, 2013; Costantini et al., 2015; Epskamp et al., 2012). Given
the rapidly increasing popularity of these data-driven network models, it is important
to note, however, that while there is a strong relation between network theory (which
deals with causal interactions between symptoms) and statistical network models
(which represent the statistical association between symptom measures and possibly
other variables), that relation is by no means one-to-one. That is, many different
statistical models could be used to inform a given network theory, and in some cases
network theories are in fact constructed without any explicit statistical model; for
example, by asking clinicians or respondents which symptoms activate each other
(Frewen et al., 2012; Kim, 2002; Ruzzano et al., 2015).
Page | 25
2.3 Toward incorporating environmental and

genetic risk into network models
1972: “We are optimistically hopeful that the current mass of research on families
of schizophrenics will discover an endophenotype, either biological or behavioral
(psychometric pattern), which will not only discriminate schizophrenics from
other psychotics, but will also be found in all the identical co-twins of schizophren-
ics whether concordant or discordant. All genetic theorizing will benefit from the
development of such an indicator.” (Gottesman & Shields, 1972)
Widespread research has focused on identifying biological and environmental

risk factors involved in the onset and progression of schizophrenia. Despite such
efforts, risk factors that cause psychotic manifestations remain elusive (Leask,
2004; van Os & Kapur, 2009). Meta-analytical work suggests that substance
abuse (especially cannabis), growing up in an urbanized area, and developmental
trauma are among the most common environmental influences that increase the
susceptibility to psychosis (van Os et al., 2010; van Os & Kapur, 2009). Other
factors include ethnicity, season of birth, prenatal stress and obstetric complications,
but to a much lesser extent and with more inconsistent results (Leask, 2004). In
terms of a biological component, the search for biomarkers for schizophrenia has
been extensive. However, to date, even though schizophrenia is known to be a
highly heritable mental condition, with a rough heritability estimate up to 80%,
the search for candidate genes for schizophrenia has showed no robust findings (van
Os & Kapur, 2009). The diagnosis of schizophrenia is therefore based entirely on
its clinical features and no other determinants have yet been incorporated into the
diagnostic scheme (Tsuang et al., 2001). Notably, if all psychotic symptoms were
indeed caused by an underlying disease entity – as schizophrenia is argued to be –
one would expect to identify that symptoms have the same (or at least similar) risk
factors (see Figure 2.3, panel A). That is because risk factors are expected to only
influence the liability to develop a disorder, while the disorder would, in turn, cause
the symptoms (Fried et al., 2014). Current findings in the field of psychosis and
depression suggest, however, that symptoms substantively differ in terms of their
risk factors (Fried & Nesse, 2014; Isvoranu et al., 2016, 2017), lending support to
the importance of investigating the individual symptoms of a disorder and their
causal associations – not only associations to each other, but also associations to
the external field of influence.
Page | 26
Paranoia
A.
Hallucinations
External Field SZ
Hostility
B.
External Field External Field
Hal Hal
Hos Del Hos Del

Symptoms
Hal: Hallucinations
Del: Delusions
Exc: Excitement
Gra: Grandiosity
Par: Paranoia
Hos: Hostility
Par Exc Par Exc
Gra Gra
Figure 2.3 A. Visualization showing how the external field (e.g., environmental risk factors, genetic
risk scores) would be expected to influence symptoms from a common cause perspective. The influence
takes place via the latent variable (schizophrenia), which then causes the symptoms.
B. Visualization showing how the external field (e.g., environmental risk factors, genetic risk scores)
would be expected to influence symptoms in network models. The influence takes place either as a
main effect, on the node itself or as an indirect secondary effect, on the symptom-symptom interactions.
In accordance, one can investigate the effects of the external field of influence (i.e.,
risk factors such as genes or environmental components) on a network structure
in two ways. First, we can examine whether such factors have a direct main effect
on the nodes (see Figure 2.3, panel B, left panel). As highlighted in the section
above, nodes within a network are not restricted to symptoms only, but can be
expanded to almost any variable – as long as the variable can both influence and be
influenced by other variables (Fried & Cramer, 2017). As such, a node can represent
any environmental, biological, or cognitive component that is deemed relevant
for the disorder. To assess the main effect of these components we can construct a
network that incorporates (1) environmental risk factors and symptoms, (2) genetic
Page | 27
risk factors–such as a polygenic risk score–and symptoms, (3) both environmental

risk factors and genetic risk scores and symptoms, depending on the availability of
the data. In this way, we do not only examine whether symptoms differ in terms
of their risk factors, but we can also identify associations between the risk factors
themselves, including potential gene by environment interrelations. As network
models are still novel, to date, we are aware of only one study that has incorporated
polygenic risk scores into symptom networks and found associations between
the polygenic risk score and mainly positive psychotic symptoms and depressive
symptoms (Isvoranu et al., 2019; see Chapter 8); we are however aware of work in
progress and expect extensive research to follow this avenue soon.
In terms of investigating the associations between environmental risk scores and
individual symptoms of mental disorders, several studies in the field of depression
(Fried et al., 2014; Fried, Bockting, et al., 2015; Fried & Nesse, 2014) and psychosis
(Isvoranu et al., 2016, 2017) have focused on assessing this interplay using network
analysis. Isvoranu and colleagues have identified that (1) when all symptoms of
the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987) are included
in a network alongside childhood trauma, general psychopathology symptoms
appear to mediate the associations between trauma and the positive and negative
symptoms (see Chapter 5); (2) when using general population data to examine
the interrelations between environmental exposure–as measured by urbanicity,
developmental trauma, and cannabis use–and psychopathology expression, the
three environmental factors distinctly affect the symptomatology, with the main
connective pathway being through cannabis use. In addition, strong associations
between cannabis use and urbanicity, as well as between developmental trauma and
cannabis use were identified (see Chapter 4). Even though this line of research is in
its early stages, it shows high promise in potentially unraveling means by which the
external field influences symptoms. In short, with a network framework in mind,
we convert from trying to understand and identify risk factors for schizophrenia to
trying to identify risk factors for symptoms of schizophrenia and for the interaction
between these symptoms.
A second means (see Figure 2.3, panel B, right panel) by which we can investigate
the influence of the external field on symptomatology is through examining whether
this has an indirect secondary effect on the network, by acting on the symptom-
symptom connections. Thus, a stressor in the external field (e.g., grief, traumatic
experiences, genetic components) cannot only be said to activate the network by
either activating one or more symptoms (as detailed above), but also by increasing
Page | 28
the strength of interactions between the symptoms themselves. Recent findings in large
general population cohorts support this hypothesis – environmental exposure to
risk factors such as childhood trauma, urbanicity, and cannabis use seem to alter
the connectivity between the symptoms in a dose-response fashion, progressively
triggering the transition toward a severe condition (Fusar-Poli et al., 2014; Guloksuz
et al., 2015, 2016; Smeets et al., 2015). Isvoranu and colleagues (2016; see Chapter
4) found that subjects using cannabis showed increased connectivity between
psychopathological dimensions such as depression and anxiety, as well as between
interpersonal sensitivity and phobic anxiety, compared to subjects not exposed to
any high-risk environmental factors. Correspondingly, Guloksuz and colleagues
(2016) showed using two general population cohorts that environmental exposure
increased the degree of symptom connectivity within all symptom dimensions and
related diagnostic categories, and further argued that the emergence of psychosis
may be related to severity of increasingly connected psychopathology rather than
to a specific illness category.
With these findings in mind, the influence of the external field as expressed
through adverse environmental determinants seems to lead to a more strongly
connected and thus less resilient symptom network, facilitating the transition to an
alternative stable state of mental disorder. In particular, it may further be possible
that part of the missing heritability issues faced at the moment are due to the fact
that genetic vulnerability may be expressed as increased connectivity of symptom
interactions rather than as a main effect on the network (or on the disorder). In
addition, the impact of the environment and/ or genetics does not appear to be
specific to particular diagnosis categories, but vulnerability factors seem to rather be
spread between the traditional diagnostic categories (e.g., childhood trauma, cannabis
use; Dannlowski et al., 2012; Heim et al., 2009; Read et al., 2005; Wiersma et al.,
2009, Arseneault et al., 2002; Bovasso, 2001; Patton et al., 2002). Finally, thinking of
this from a comorbidity perspective, if the external field has an influence on individual
symptoms and the symptom-symptom interactions–which we know are often shared
between disorders–the finding that vulnerability factors are similar and common
between different diagnostic categories is no longer surprising.
Page | 29
2.4 Conceptualizing treatment from a network

perspective
If a mental disorder indeed refers to a self-sustaining alternative stable state of
a symptom network, as network theory holds, then this has direct implications
for how to conceptualize treatment. To a considerable extent, existing treatments
can be re-interpreted from a network perspective, and we may perhaps better
understand the treatment mechanisms by looking at them in this way. Below,
we give an overview of some thoughts and observations that may fit within this
framework. It is also possible that, in the future, novel treatment protocols may
be partly based on network theory; although we are still far removed from having
operational network treatment, individualized network modeling has already
been implemented in proof-of-concept studies (Kroeze et al., 2017), as we will
also discuss.
As emphasized throughout this chapter, in a network model, there are
basically three factors that determine its dynamics: the nodes in the network, the
connections in the network, and the external field of factors outside of the network
that influence it (Borsboom, 2017). In accordance, we can consider three types
of interventions from a network perspective; namely, those that target nodes in
the network (henceforth: node-interventions), those that target connections in the
network (henceforth: edge-interventions), and those that target the external field
of factors outside of the network that influence it (henceforth field-interventions).
Assuming, for the moment, that we take the network to be the collection of
symptoms that a patient with a psychotic disorder endorses, a node-intervention
would be an intervention that is directly aimed at the reduction of a specific
symptom. For instance, the use of an antipsychotic drug targeting derogatory
auditory hallucinations may be considered an example of a node-intervention. In
a similar fashion, a wide array of cognitive behavioral techniques that are aimed
directly at challenging the content of a cognition could be considered node-
interventions. For instance, consider a patient who believes that the voice that he
or she hears originates from an outside source, but is willing to examine this belief.
A behavioral experiment, a common technique in cognitive behavioral therapy,
may be conducted, for instance asking the patient to tape the voice. Similarly, sleep
reduction for reducing insomnia, response-prevention for compulsions, use of a
drug with sedative qualities for agitation or interoceptive exposure for anxiety may
all be considered examples of node-interventions.
Page | 30
Edge-interventions–interventions targeting relations between symptoms–may lie in
various approaches in therapies that aim to teach a person how to improve cognitive,
behavioral and affective coping skills that may impact symptom-symptom relations.
For instance, in the case of derogatory auditory hallucinations, an edge-intervention
may involve being taught a new adequate coping strategy by a clinician or support
group, such as the use of distraction, humming, or setting limits and structure to the
contact, instead of an unhelpful coping strategy, such as loudly arguing with the voices
whenever they occur (for instance leading to agitation and/or social isolation due to
social rejection). Quite recently, several new therapies that are specifically designed to
increase general cognitive, behavioral and affective coping skills have gained ground
for psychosis, such as Metacognitive Training (MCT; Moritz & Woodward, 2007)
and Acceptance and Commitment Therapy (ACT; Hayes et al., 1999). Metacognitive
Training for psychosis aims to gain insight into unhelpful cognitive distortions
putatively thought to be related to delusional beliefs (e.g., overconfidence in decisions,
jumping to conclusions) and improve general metacognitive processes (‘thinking
about thinking’). Acceptance and Commitment Therapy aims, among others, to
increase psychological flexibility, such as more openness with unwanted thoughts and
feelings that are unlikely to be amenable to direct change strategies, and encourages
value-directed action. Instead of directly targeting symptoms when they occur, these
types of interventions may decrease the connections, and therefore the disruptive
impact of symptoms within a network.
Third, field-intervention targets factors that are not directly part of the symptom
network, but that do influence it from outside. For example, interventions include
approaches that aim to restore or improve a person’s community maintenance, such
as entering a work project, which can reduce multiple symptoms and provide a buffer
against different cascades of symptoms, and interventions that try to improve the
physical environment of the person (e.g., by providing housing). Also, interventions
aimed at increasing the social support network of a person, or fighting the stigma of
mental illness, may influence individuals’ symptom network from the outside.
We recognize that the distinction between types of interventions is not always
clear-cut and do not argue that a ‘direct’ intervention cannot have multiple and
indirect effects. The general aim of the above examples is to provide an idea of how
existing interventions could be interpreted in a network theoretical framework.
Currently, there is no way to utilize this information, but in the future one could
think of systematic intervention planning that uses network models for the
selection of (sequences of) interventions. In such cases, one would start with a
Page | 31
(partly) personalized network model capturing the person’s most important

symptoms and interactions between them. One then asks: assuming a given set
of node-, edge-, and field-interventions, what would be the most effective way of
changing and charging the symptom network to repel it from its disorder state
and create an enduring healthy attractor state? Network models could, in such a
methodology, play the role of navigation software, in that they can suggest a number
of routes that may result in effective treatment. Clearly, we have not even begun yet
with seriously engaging such approaches, but in time network-based intervention
planning may offer an attractive extension of the treatment professional’s toolkit.
One of the problems that will have to be solved in order for this type of approach
to work is that we need ways to assess a given person’s network structure. There
are currently two ways of approaching this problem, both of which are promising
but also need considerable work before they can be effectively put to use. First,
Frewen and colleagues (2012) have developed so-called Perceived Causal Relations
(PCR) scales. These scales first query which symptoms a person has, and then
query the extent to which combinations of symptoms influence each other. One
can administer these scales to clients themselves (resulting in a self-reported
symptom network) or to others (e.g., family, friends, healthcare professionals), to
get a qualitative idea of the most important connections between symptoms. The
advantage of this methodology is that it is relatively fast (one can administer a PCR
scale in less than an hour) and that it is straightforward to combine different sources
of evidence (e.g., one can easily combine networks resulting from self-assessment
and from assessment by others). The downside is that, if one wants to use the
methodology to get a view of the actual symptom network, one has to assume that
people can actually assess causal relations between symptoms. Although there is
some evidence that people naturally represent disorders in this way (Kim, 2002),
it is not clear that this representation is also accurate.
A second way of getting a glimpse of symptom-symptom interactions is by using
Ecological Momentary Assessment (EMA) protocols. In such approaches, which
have seen tremendous growth in recent years due to the availability of smartphones,
the person answers a short questionnaire multiple times a day. If sufficiently many
data-points are available, time-series analyses can be used to estimate the network
structure of the interactions between queried symptoms (Bringmann et al., 2013;
Epskamp, van Borkulo, et al., 2018). In a proof-of-concept study, Kroeze and
colleagues (2017) used this process to estimate a network structure for a number
of clients in therapy; the network was subsequently shown to the client, and
Page | 32
interventions were selected partly based on the network structure. Fisher and
Boswell (Fisher & Boswell, 2016) have pioneered similar treatment approaches
using time series analyses, although they did not explicitly use network structures
but instead based treatment on dynamic factor analysis models. The main advantage
of these approaches is that they can uncover symptom-symptom interactions that
people may not be able to identify themselves.
However, it should be noted that current approaches suffer from a number
of serious drawbacks that need to be addressed in future work: (a) data-analysis
methods are currently suboptimal as they do not allow to model mean trends but
only deviations from these trends, (b) there is no way to currently integrate processes
that operate at different time scales (moments, hours, days, weeks), (c) many symptoms
and external factors simply cannot be assessed using EMA, either for practical or for
principled reasons. Discussing such limitations of the EMA approaches – as well
as more general limitations of the network approach – is beyond the scope of this
chapter; for detailed theoretical and methodological considerations we direct the
reader to the cited review articles (Fried & Cramer, 2017; Guloksuz et al., 2017). A
promising future extension is nonetheless that it will become possible to combine
subjective assessments with more complex and objective behavioral and physical
measures (e.g., movement, location, physiological data, etc.).
2.5 Concluding Remarks

The field of mental health care is in need of conceptualizations of pathology that are
aligned with the complexity and multitude of symptoms and problems that most
patients present with. Tools that could advance our understanding of psychiatric
disorders, as well as provide substantiated intervention selection could be highly
beneficial to clinical practice. In time, the network approach may provide such
tools. Notably, it is unlikely that there is such a thing as a ‘one-size fits all treatment’,
especially given the inherent heterogeneity of psychosis. Thus, it is likely that
intervention planning and monitoring will require personalized network models,
which may vary from person to person. Endorsing integrative treatments approaches
that are tailored to the individual, as well as embracing new avenues that allow us to
combine subjective assessments with physical measures, will likely contribute to the
advancement of the field and clinical practice. These possibilities are within reach,
and in the next few decades we are bound to learn to what extent the promises of
network theory in coordinating research and treatment will be fulfilled.
Page | 33
Extended Network Analysis
3
of Symptomatology

Isvoranu, A. M., Abdin, E., Chong, S. A., Vaingankar, J., Borsboom, D., Subramaniam,
M. (2021). Extended Network Analysis of Symptomatology: From Psychopathology
to Chronic Illness. BMC Psychiatry, 21(1), 1-9.
Chapter 3 | Extended Network Analysis of Symptomatology
Abstract
Understanding complex associations between psychopathology and chronic
illness is instrumental in facilitating both research and treatment progress. The
current chapter is the first and only network-based study to provide such an
encompassing view of unique associations between a multitude of mental and
physical health-related domains. The current analyses were based on the Singapore
Mental Health Study, a cross-sectional study of adult Singapore residents. The
study sample consisted of 6616 respondents, of which 49.8% were male and 50.2%
female. A network structure was constructed to examine associations between
psychopathology, alcohol use, gambling, major chronic conditions, and functioning.
The network structure identified what we have labelled a Cartesian graph: a network
visibly split into a psychopathological domain and a physical health domain. The
borders between these domains were fuzzy and bridged by various cross-domain
associations, with functioning items playing an important role in bridging chronic
conditions to psychopathology. Current results deliver a comprehensive overview
of the complex relation between psychopathology, functioning, and chronic illness,
highlighting potential pathways to comorbidity.
Page | 36
3.1 Introduction
Mental illness is one of the most pressing contemporary problems, with impact on
health, social and economic issues. Despite significant research efforts, common
mental disorders within the general population remain a major concern, with
reports as high as 28.8% for anxiety disorders, 20.8% for mood disorders, and
14.6% for substance use disorders (Kessler & Zhao, 2010), as well as rates of up to
40% for subjects with a mental disorder to meet criteria for another class of lifetime
disorder (Merikangas et al., 2010).
In addition to high comorbidity between mental disorders, there is also vast
evidence that people with common mental health conditions are at higher risk of
developing physical illness, and conversely people with a diagnosis of physical illness
are at higher risk of developing mental health problems (Doherty & Gaughran,
2014). For instance, robust associations between immunological/ inflammatory
conditions and mood disorders (Merikangas et al., 2015) have been identified,
with depressed patients being 60% more likely to develop diabetes than their
non-depressed counterparts and prevalence rates of diabetes as high as three times
greater in subjects with bipolar disorder (Doherty & Gaughran, 2014). Further,
in patients with schizophrenia, cardiovascular disease is the most common cause
of death (Brown, 1997). Of note, while highly relevant, the comorbidity between
mental health and physical conditions is often neglected (Sartorius, 2018). Here we
argue better understanding this comorbidity may lead to improved prognosis and
outcomes. The aim of the current study was therefore to delve into the relationship
between mental and physical health conditions, as to highlight features important
in explaining the development of this comorbidity.
In recent years it has been suggested that some symptoms of particular diagnoses,
but not all, may account for the comorbidity patterns between diagnoses, indicating
that symptoms may have a unique role and may not be interchangeable (Boschloo et
al., 2015). This line of reasoning, now known as the network framework (Borsboom,
2017; Borsboom & Cramer, 2013; Epskamp et al., 2012) has been proposed as
an innovative tool in the study of psychopathology, and in the past decade it has
grown prominent in the fields of psychiatry and clinical psychology (Epskamp,
van Borkulo, et al., 2018; Fried et al., 2014; Isvoranu et al., 2016, 2019; McNally
et al., 2015; Rhemtulla et al., 2016). Within this framework, the focus shifts from
the diagnostic level to the symptom level, with the aim to highlight the unique
role of symptoms, and their potential causal associations. Network structures may
Page | 37
therefore be useful tools to study both within-diagnoses and between-diagnoses

symptom associations.
Further, the network approach suggests that the boundaries between mental
and physical disorders are porous (Borsboom et al., 2019), as physical symptoms can
cause psychopathological symptoms (e.g., pain  fatigue  depressed mood) and
vice versa (e.g., depressed mood  alcohol use  liver damage). If so, it is crucial
to chart the pathways by which these influence each other, as to ultimately reach
better treatment targets. The current research aims to highlight features that may
account for comorbidity between diagnoses and provide an encompassing view of
unique associations between psychopathological conditions and chronic illness
and functioning. To this end, we aimed to constructed a large network structure,
encompassing a multitude of symptoms and other health-related dimensions,
ranging from general psychopathology, to psychosis, alcohol use, chronic physical
conditions and functioning and health-related quality of life (HRQoL). To our
knowledge, this is the first and only network-based study encompassing such as
multitude of health-related domains, as well as the only existing network study
concerned with the comorbidity between mental and physical health conditions.
3.2 Methods
3.2.1 Sample
The sample analyzed (N=6616 respondents) was part of the Singapore Mental
Health Study (SMHS), a cross-sectional, population-based, epidemiological study
of adult Singapore residents aged 18 years and above. The study aimed to establish
lifetime and 12-month prevalence of mental disorders, as well as the current
use of mental health services, treatment gaps and loss of role functioning. The
subjects were randomly selected from a national registry that maintains the names,
sociodemographic details (e.g., age, gender and ethnicity), and household addresses
of all residents in Singapore. Inclusion criteria were being a Singapore citizen or
resident, 18 years or older, and able to speak and understand English, Chinese or
Malay. Exclusion criteria included being incapable of doing an interview due to
severe physical or mental health conditions, language barriers, living outside the
country, institutionalized or hospitalized throughout the duration of the survey
period, as well as incomplete or incorrect addresses. A disproportionate stratified
sampling was used where the 3 main ethnic groups (Chinese, Malays, and Indians)
were sampled in equivalent proportion of about 30% each. Further details of the
Page | 38
sample are available in the cited papers (Chong, Abdin, Vaingankar, et al., 2012;
Subramaniam et al., 2012).
3.2.2 Measures
All measures used in this study are reported in Table 3.1 and described in Appendix
A1. Due to the skip-structure of the interviews, we selected and included only
items that were answered by the full sample, focusing on sub-clinical levels of
psychopathology. Overall, we included items pertaining to the World Health
Organization-Composite International Diagnostic Interview (WMH-CIDI;
Kessler & Üstün, 2004), a modified CIDI checklist of chronic medical conditions,
the South Oaks Gambling Screen (SOGS; Lesieur & Blume, 1987), and the EQ5D
(Herdman et al., 2011).
3.2.3 Statistical analysis

Network construction
We constructed an undirected, weighted network model and included all measures
described in Table 3.1 as nodes, with each edge in the network reflecting the pairwise
conditional relation between two nodes, while controlling for all other nodes in
the network. We fitted an Ising Model to the data using the eLasso technique
implemented in the IsingFit R-package, version 0.3.1. (van Borkulo, Borsboom, et
al., 2014). The technique is based on the Ising Model as used in statistical physics,
and uses l1 regularized logistic regression (Tibshirani, 1994), commonly referred to
as the eLasso, conjointly with the extended Bayesian Information Criterion (EBIC;
Chen & Chen, 2008). The method has been shown successful in identifying the
most relevant features of a network constructed from binary data (van Borkulo,
Borsboom, et al., 2014).
We visualized the network using the qgraph R-package version 1.6.4. (Epskamp
et al., 2012). Blue (red) edges represent positive (negative) associations, and the
thicker the edge, the stronger the association between two nodes (Costantini et al.,
2015). The layout of the network is based on the Fruchterman-Reingold algorithm
(Fruchterman & Reingold, 1991), which places nodes with stronger and/ or more
connections closer to the center of the network and to each other.
Page | 39
Table 3.1 Study measures

Domain Instrument Measure Items
General WMH-CIDI Screening Section 26 items measuring: smoking,
Psychopathology mental and physical health, anxiety,
intermittent explosive disorder,
depression, generalized anxiety
attack, specific phobias, social
phobia, agoraphobia, attention deficit
hyperactivity disorder, oppositional
defiant disorder, separation anxiety.
Psychosis WMH-CIDI Psychosis Screen 1 item measuring psychosis
Obsessive- WMH-CIDI Obsessive- 1 item measuring compulsions, 1 item
Compulsive Compulsion measuring obsession
Disorder Disorder Section
Alcohol Use WMH-CIDI Alcohol Use 1 item measuring age of first alcoholic
beverage
Gambling SOGS Lifetime 1 item measuring lifetime gambling
Gambling
Major Chronic CIDI Major Chronic 6 items measuring presence of
Conditions Checklist Medical asthma, high blood sugar/ diabetes,
of Chronic Conditions hypertension, back problems,
Medical migraine headaches, and other
Conditions chronic conditions
Health-Related EQ5D Quality of Life 5 items measuring mobility, self-care,
Quality of Life and Functioning usual activities, pain or discomfort,
and anxiety or depression.
*Abbreviations: WMH-CIDI, World Health Organization-Composite International Diagnostic
Interview; SOGS, South Oaks Gambling Screen.
Centrality analysis
To investigate the centrality of each node in the network, we computed strength
(Opsahl et al., 2010) as a centrality measure. Node strength is a measure of the
number and strength of connections, quantifying how well a node is directly
connected to other nodes. Previous research showed strength to be the most robust
centrality measure (Epskamp, Borsboom, et al., 2017).
Network stability
To investigate the robustness and replicability of results we performed accuracy and
stability checks using the R package bootnet version 1.2.4. (Epskamp, Borsboom,
Page | 40
et al., 2017). We assessed the accuracy of the network connections, the stability
of strength centrality, and tested whether the network connections and centrality
estimates for different variables differ from each other.
3.3 Results
3.3.1 Patient characteristics
The study sample consisted of 6616 respondents, of which 49.8% were male and
50.2% female. The demographic profile distribution of subjects is reported in Table
3.2, and the item frequency and domain distribution are reported in Table A1 in
Supplement A. After treating ‘don’t know’ and ‘refused answers’ as missing data,
there were overall less than 0.5% missing data on the general psychopathology,
psychosis, OCD, gambling, and chronic conditions variables. In addition, there
were 1.13% missing data on the variable measuring age of first alcoholic drink,
and 15.45% missing data on the EQ5D, due to the instrument being administered
at a later time point than the rest of the measures. Given that the estimation
methodology employed requires full data, we imputed missing data1 using the
mice R-package version 3.6.0. prior to fitting the model (van Buuren & Groothuis-
Oudshoorn, 2011).
3.3.2 Network analysis

The resulting network structure is presented in Figure 3.1. The physical and mental
health self-report variables were not reverse-coded, a higher value thus indicating
better health reports. For all other variables, a higher value indicates more problems.
Overall, all nodes were associated with at least one other node in the network.
At a global level, the network was visually divided into two noticeable domains:
a mental health domain consisting mainly of psychopathological nodes (to the
right), and a physical health domain consisting mainly of nodes pertaining to
physical problems, such as chronic conditions and functioning and HRQoL (to
the left). We coin this conspicuously separated structure a Cartesian graph, after
the dualist philosopher Descartes. Noticeably, however, the borders between these
two domains are fuzzy and bridged by various cross-domain associations.
1 We chose imputation over listwise deletion, as most missing data were on the EQ5D variable and
were missing at random, as a result of later administration of the test. Of note, listwise deletion
would have resulted in a large loss of information of data available on the other 38 variables.
Page | 41
To summarize the results of our analysis, we will first highlight within-domain

associations, followed by between-domain associations. Of note, within each
domain (i.e., psychopathological and physical), there are multiple clusters
differentiated by color, pre-defined according to classic diagnostic categories (such
as the Diagnostic and Statistical Manual of Mental Disorders; American Psychiatric
Association, 2000). We will address these as clusters, to differentiate them from
what we refer to as domains (i.e., the Cartesian graph).
Table 3.2 Demographic profile distribution

Demographic Value Frequency %
Gender Male 3295 49.8%
Female 3321 50.2%
Age group 18-34 2292 34.6%
35-49 2359 35.7%
50-64 1551 23.4%
65+ 414 6.3%
Marital status Married 4293 64.9%
Separated 31 0.5%
Divorced 230 3.5%
Widowed 236 3.5%
Never Married 1826 27.6%
Language English 5262 79.6%
Chinese 540 8.2%
Malay 814 12.3%
Within-domain associations
Within-domain associations were common and stronger than between-domain
associations, with most items being associated with a multitude of other items
within the same domain.
Especially, the psychopathological domain displayed high connectivity, almost
all associations being positive (i.e., an increase in one item predicts an increase
in another item). The items belonging to attention deficit hyperactivity disorder
(ADHD), oppositional defiant disorder (ODD), and separation anxiety were
strongly interrelated. Smoking, gambling, and alcohol use were all linked with each
other. The anxiety items were associated with depression and intermittent explosive
Page | 42
disorder. Worrier was connected with all depression items and panic attack, which
was in turn associated with feeling sad, empty, depressed, and both anger attack
items. Obsessions were associated with agoraphobia, being really shy with people,
and with psychosis.
The physical health domain, while less well-connected, displayed strong
connectivity within and between functioning and HRQoL items and chronic
conditions. Notably, chronic conditions displayed a less clear clustering pattern and
did not group as well together, but were divided by the functioning and HRQoL
items. Mobility was associated with hypertension, high blood sugar/ diabetes, and
other chronic conditions. Pain/ discomfort was associated with back problems and
other chronic conditions. Self-report measures of physical and mental health clustered
within the functioning and HRQoL cluster, with higher reports of physical health
being negatively associated with mobility, high blood sugar/ diabetes, other chronic
conditions and pain/ discomfort. Higher reports of mental health were negatively
associated especially with anxiety/ depression.
Between-domain associations
While the two domains of the Cartesian graph are prominent and less connected,
the borders between these are fuzzy and bridged by various cross-domain
associations. The item measuring anxiety/ depression according to the functioning
and HRQoL measure was the main item connecting the two domains, located
in the center of the network. This was strongly associated especially to the
psychopathology items panic attack, loss of interest, worrier, obsessions, feeling upset
when separated from family members (adulthood) and to the physical health items
mobility, pain/discomfort, and mental health reports.
Other between-domain associations include hypertension, visibly and strongly
associated with gambling, asthma associated with the mood item restless, social
phobia, and with the ODD item breaking rules during childhood or teenage years.
The item migraine headaches was associated with psychosis, and very weakly with the
mood item restless, as well as with some types of phobia. High blood sugar/ diabetes
was mostly, albeit weakly, negatively associated with ODD and alcohol use.
Page | 43
Smoking
● 1: Smoker
Health
● 2: Physical Health
● 3: Mental Health
Anxiety
● 4: Panic Attack
16
Page | 44
Intermittent Explosive Disorder
17 ● 5: Anger Attack: Broke or Smashed Something
● 6: Anger Attack: Hit or Tried to Hit Someone
33 39 18
Depression
● 7: Sad, Empty, Depressed
● 8: Discouraged
●
14
9: Loss of Interest (e.g, work, hobbies)
●
34 10: Restless
● 11: Bad Mood
38 15
Generalized Anxiety Attack
● 12: Worrier
32
13 Specific Phobias
30 ● 13: Zoophobia
● 14: Aquaphobia
● 15: Latrophobia
●
42 20 16: Claustrophobia
● 17: Acrophobia
● 18: Aviophobia
2 35
Social Phobia
●
29 19: Really Shy with People
19 31
Agoraphobia
● 20: Agoraphobia
ADHD
●
40 3 28 21: Problems with Concentration (as a child)
●
36 22: Being Fidgety (as a child)
Oppositional Defiant Disorder

●
26 25 23: Losing Temper (childhood or adolescence)
1 ● 24: Breaking Rules (chilhood or teenage years)
Separation Anxiety
● 25: Feeling Upset when Separated from Mother
● 26: Feeling Upset when Separated from Family Members (adulthood)
41 12 4
Psychosis
24 ● 27: Psychosis
27 21
OCD
● 28: Obsessions
● 29: Compulsions
23 Gambling
●
8 30: Gambling
37 22
7 Alcohol Use
● 31: Alcohol Use
9 6 Functioning and Health−related Quality of Life

● 32: Mobility
● 33: Self−care
5 ● 34: Usual Activities
● 35: Pain/ Discomfort
● 36: Anxiety/ Depression
11 10
Chronic Conditions
● 37: Asthma
● 38: High Blood Sugar/ Diabetes
● 39: Hypertension
● 40: Back Problems
● 41: Migraine Headaches
● 42: Other
Figure 3.1 Network structure depicting the different domains of psychopathology, functioning, and chronic conditions, differentiated by colors. Blue edges
indicate positive associations, red edges indicate negative associations, and the thickness of an edge represents the strength of the association.
● 17: Acrophobia
● 18: Aviophobia
Social Phobia
●
29 19: Really Shy with People
19 31
Agoraphobia
● 20: Agoraphobia
Smoking ADHD
●
28 ● 21: Problems with Concentration (as a child)
1: Smoker
● 22: Being Fidgety (as a child)
Health
● 2: Physical Health Oppositional Defiant Disorder
● ●
26 25 23: Losing Temper (childhood or adolescence)
3: Mental Health
Anxiety
● 4: Panic Attack Separation Anxiety
● 25: Feeling Upset when Separated from Mother
● 26: Feeling Upset when Separated from Family Members (adulthood)
Intermittent Explosive Disorder
4 ● 5: Anger Attack: Broke or Smashed Something
● 6: Anger Attack: Hit or Tried to Hit Someone Psychosis
24 ● 27: Psychosis
Depression 21
● 7: Sad, Empty, Depressed OCD
● ● 28: Obsessions
8: Discouraged
● ● 29: Compulsions
9: Loss of Interest (e.g, work, hobbies)
● 10: Restless
● 11: Bad Mood 23 Gambling
● 30: Gambling
Generalized Anxiety Attack 22
● 12: Worrier Alcohol Use
● 31: Alcohol Use
Specific
6 Phobias
● 13: Zoophobia Functioning and Health−related Quality of Life
● ● 32: Mobility
14: Aquaphobia
● ● 33: Self−care
15: Latrophobia
● 5 ● 34: Usual Activities
16: Claustrophobia
● ● 35: Pain/ Discomfort
17: Acrophobia
● ● 36: Anxiety/ Depression
10 18: Aviophobia
Social Phobia Chronic Conditions

● ● 37: Asthma
19: Really Shy with People
● 38: High Blood Sugar/ Diabetes
● 39: Hypertension
Agoraphobia
● ● 40: Back Problems
20: Agoraphobia
● 41: Migraine Headaches
●
Page | 45
ADHD 42: Other
● 21: Problems with Concentration (as a child)
● 22: Being Fidgety (as a child)
Oppositional Defiant Disorder

● 23: Losing Temper (childhood or adolescence)
Strength
Usual Activities
34 ●
Discouraged8 ●
Mobility
32 ●
Page | 46
Pain/ Discomfort
35 ●
Panic Attack4 ●
Sad, Empty, Depressed7 ●
Loss of Interest (e.g., work, hobbies)9 ●
Losing Temper (childhood or adolescence)

23 ●
Obsessions
28 ●
Worrier
12 ●
Anxiety/ Depression
36 ●
Acrophobia
17 ●
Breaking Rules (childhood or teenage years)

24 ●
Physical Health2 ●
Really Shy with People

19 ●
Problems with Concentration as Child

21 ●
Aquaphobia
14 ●
Agoraphobia
20 ●
Bad Mood
11 ●
Restless
10 ●
Anger Attack: Hit or Tried to Hit Someone6 ●
Zoophobia
13 ●
Feeling Upset when Separated from Family Members (adulthood)

26 ●
Mental Health3 ●
Hypertension
39 ●
Anger Attack: Broke or Smashed Something5 ●
Claustrophobia
16 ●
Being Fidgety (as a child)

22 ●
Feeling Upset when Separated from Mother

25 ●
High Blood Sugar/ Diabetes

38 ●
Alcohol Use
31 ●
Latrophobia
15 ●
Self-care
33 ●
Compulsions
29 ●
Other
42 ●
Aviophobia
18 ●
Smoker1 ●
Migraine Headaches
41 ●
Back Problems
40 ●
Gambling
30 ●
Asthma
37 ●
Psychosis
27 ●
−2 −1 0 1 2
Figure 3.2 Centrality plot depicting the strength of each node in the network structure, ordered from the node with the highest strength to the node with the
lowest strength in the network. Node strength quantifies how strongly a node is directly connected to other nodes in the network (i.e., by summing all absolute
edge weights of edges connected to the given node). All values are standardized and higher values indicate greater centrality in the network.
3.3.3 Centrality analysis
The centrality plot is presented in Figure 3.2. The top 3 items with the highest
strength centrality were usual activities, discouraged, and mobility,2 while the 3 least
central items in terms of strength were psychosis, asthma, and gambling.3 Figure A4
in Supplement A provides an overview of all the significant and non-significant
differences between centrality items.
3.3.4 Network replicability and robustness

Appendix A3 and Figures A2-A5 detail on the results of the accuracy and stability
checks. Overall, our results suggest that the network model is very stable, many of
the identified edges and centrality measures are significantly different from each
other, and all findings are interpretable.
3.4 Discussion
The current study used a network approach in an aim to uncover associations,
at a subclinical level, between a wide array of psychopathological conditions,
functioning and HRQoL, and other chronic illness. To our knowledge, this is
the first study to focus on such a multitude of complex relations between different
physical and health-related domains. Overall, we identified what we have labeled
a Cartesian graph: a network graph split into two visible domains: a (mainly)
psychopathological domain (more generally referred to as the mental health
domain), and a (mainly) functioning and HRQoL and chronic conditions domain
(more generally referred to as the physical health domain). The borders between
these two domains are fuzzy and bridged by various cross-domain associations.
To date, there is wide evidence supporting the comorbidity between physical
conditions and mental disorders (Doherty & Gaughran, 2014; Merikangas et al.,
2015), with a majority of findings indicating mood and anxiety disorders as the main
comorbid feature (Buist-Bouwman et al., 2005; Katon & Ciechanowski, 2002;
Scott et al., 2007; Wells et al., 1989). Although the current study identified few links
between specific anxiety- and depression-related symptoms and chronic conditions,
2 Based on bootstrapping (Epskamp, Borsboom, et al., 2017), these items were significantly different
from many of the other items in the network, though not significantly different from each other
and from other high centrality items, including depressed, loss of interest, and pain/ discomfort.
3 Based on bootstrapping (Epskamp, Borsboom, et al., 2017), these items were significantly less central
from many of the other items in the network, but not significantly different from each other and
from other low centrality items, including back problems and migraine headaches.
Page | 47
most chronic conditions were associated with items related to functioning/

HRQoL, which were in turn associated to reports of anxiety/ depression– the
main bridging item between the domains. Notably, the anxiety/ depression item,
as well as the remaining functioning/ HRQoL items were designed to measure the
presence of current symptomatology, while the rest of psychopathological items
were designed to measure lifetime presence of symptomatology. Taken together and
in line with high rates of relapse for depression (Kupfer et al., 1992) and generalized
anxiety disorders (Ballenger, 2004; Yonkers et al., 2000), these findings suggest that
overall lifetime symptomatology may predict current symptomatology (i.e., subjects
with lifetime symptoms may report more current symptoms and vice versa), and
current symptomatology may in turn be linked to current levels of functioning
and HRQoL. Further, our results indicate that functioning and HRQoL play a
unique role and are a crucial bridging component in linking chronic conditions
to psychopathology. It may thus be that when chronic conditions are associated
with a decrease in functioning and low HRQoL reports, psychopathological
symptoms may be triggered. Similarly, chronic psychopathology affecting daily-
life functioning may lead to a rise in other physical chronic conditions. Previous
research indeed identified that better functional status and fewer depressive
symptoms were significantly associated with a higher quality of life in adults with
chronic conditions (Patrick et al., 2000). Centrality analyses further support these
findings, with functioning and depression items being most central in the current
network structure. In addition, in line with outcomes showing high comorbidity
between physical conditions and mental conditions (Doherty & Gaughran, 2014;
Merikangas et al., 2015), we found that self-reports of physical and mental well-
being were strongly linked together, indicating that subjects reporting poorer
mental health are more likely to also report poorer physical health and vice versa.
Other between-domain links included associations between asthma and
depression, social phobia, and ODD. Previous research identified that children
diagnosed with and taking medication for asthma were more likely to endorse
common behavioral problems (Saricoban et al., 2011), while lifetime and current
asthma diagnosis were associated with a range of mental disorders, including
social phobia and affective disorders (Goodwin et al., 2003). We further found
hypertension and gambling to be linked, even when controlling for alcohol use and
smoking, supporting findings on the detrimental effect of gambling on physical
health (Pietrzak et al., 2007). Further, within the psychopathology domain,
smoking, gambling, and alcohol were well-clustered items, the comorbidity
Page | 48
between the addictions being well-documented (Griffiths et al., 2010; McGrath
& Barrett, 2009; Petry et al., 2005). Smoking was further associated with psychosis,
in line with evidence that smoking is common in psychotic disorders (Quigley
& MacCabe, 2019). In addition, interestingly, the psychosis item was the only
psychopathological item that fell in between the two domains of the network,
being connected to psychopathology, but also to the chronic conditions through
its association with migraine headaches. Side-effects of antipsychotic medication
can include headaches (Stevens & Rodin, 2011; Stroup & Gray, 2018), but some
evidence suggests severe forms of migraine–such as migraine aura–can also be
associated with psychotic manifestation (Bourgeois & Mistry, 2010; Fuller et
al., 1993; van der Feltz-Cornelis et al., 2012). Psychosis and obsessions were also
interrelated, indicating this association may already present at subclinical levels of
psychopathology, and not only in patients (Swets et al., 2014), or in subjects at ultra-
high risk for psychosis (Zink et al., 2014). Finally, the obsessions item was one of the
more central items in the network, being extensively associated to psychopathology.
Of note, recent research showed OCD to have one of the largest treatment gaps
(89.8%) in Singapore (Chong, Abdin, Sherbourne, et al., 2012), highlighting the
importance of addressing symptomatology early and encouraging help-seeking
behavior.
Finally, within-domain and within-cluster associations were stronger and
predominantly positive, suggesting activation may spread faster within the same
domain. In addition, some psychopathology symptom clusters displayed lower
connectivity to others (e.g., specific phobias) than other symptom clusters (e.g.,
depression, anxiety, childhood disorders), indicating the latter may be more
comorbid. These results align with previous research (Boschloo et al., 2015)
investigating the network structure of the Diagnostic and Statistical Manual
for Mental Disorders (DSM; American Psychiatric Association, 2000). Of note,
previous research (Boschloo et al., 2015) relied on a skip structure, which is
problematic when constructing network structures (Borsboom et al., 2017). The
current study overcame this limitation4 and is thus the first to approximately assess
the structure of a wide variety of mental disorder symptoms.
The current research aimed to take a first step toward identifying important
features in the development of the comorbidity between mental and physical health,
4 While the design of the current study originally relied on a skip-structure (to unburden data collec-
tion), here we focused on complete cases only and did not include severity of items in the analyses
(i.e., focused on sub-clinical levels of psychopathology), thus overcoming this limitation.
Page | 49
by zooming into and bringing together a multitude of health-related domains.

While the research is exploratory in nature and preliminary, a key finding of our
research is the crucial role played by functioning in bridging chronic conditions and
psychopathology. This finding indicates that when chronic conditions are associated
with a decrease in functioning, psychopathological symptoms may be triggered
and vice-versa. Functioning may thus be a potential key target for treatment: by
tackling problems in functioning early on we may be able to circumvent problems
arising in other health-related domains. Further, functioning was especially related
to current complaints of anxiety and depression, which were in turn related to
long-term psychopathological complains, adding to the importance of addressing
functioning complaints in intervention strategies. In addition, we identified
gambling to be one of the addictions that paved ways to both physical and mental
health problems and psychosis to be the main psychopathological domain to fall
in between the physical and mental health domains. These results indicate that
approaching these conditions holistically by taking into account both physical
and mental health complaints is essential, as leaving out any one component may
lead to a faster activation of problems in that specific domain, ultimately leading
to feedback loops and complaints in both physical and mental health domains.
Alongside these main findings, we discussed within-domain and within-cluster
associations, pinpointing to depression, anxiety, and childhood disorders as being
more connected clusters and thus more likely to lead to activation of other disorders
and therefore comorbidity.
Of note, as highlighted above, our study is exploratory in nature and preliminary.
Future research is essential for expanding on our findings, by including more
diverse samples (e.g., focus on a world-wide population, clinical populations, and
so forth), as well as a wider array of variables concerned with chronic conditions.
Here, due to the nature of data collected, we were limited to investigating only
five types of common chronic conditions, as well as five functioning problems.
Network studies designed specifically to investigate this comorbidity could expand
on the inclusion and selection of variables, as to provide further information on
this comorbidity. Alongside the replication of our results, this will enable better
pinpointing of treatment targets, which may provide to be essential in reducing
the comorbidity between mental and physical health. Ultimately, experimental
designs built upon results from exploratory research can further lead to insights
into treatment development.
Page | 50
In sum, we highlighted complex associations between a multitude of health-related
domains. Our main findings include the identification of (1) a Cartesian graph
consisting of a mental and a physical health domain, (2) functioning and HRQoL
playing a crucial role in bridging chronic conditions and psychopathology, and
(3) several within- and between-domain associations informative for potential
pathways to comorbidity.
3.4.1 Limitations
Our results should be considered in light of several limitations. First, the current
study was based on cross-sectional data which precluded strong inferences on
causal direction, and therefore any conclusions regarding direction of causality
are tentative. Second, the WMH-CIDI (Kessler & Üstün, 2004) interview
encompasses self-report statements, and may be prone to bias due to social
desirability or under-reporting of symptomatology. Further, the current study
focused on complete data cases and did not include severity of items in the analyses.
Clinical samples may display different patterns of associations and current results
were discussed in light of sub-clinical level of psychopathology. Finally, the study
was carried out in a very specific population of residents of Singapore, and therefore
the extent to which findings generalize to other cultures is not yet known.
3.5 Concluding remarks

This chapter provides rich information on the complex associations between
psychopathological domains, functioning/ HRQoL, and chronic conditions.
Our results highlight the central role of functioning in bridging psychopathology
to chronic conditions, as well as a multitude of potential within- and between-
domain pathways to comorbidity, which can often be overlooked or simplified by
reductionist approaches to psychopathology. We assert investigating such unique
associations between different health domains may highlight potential pathways
to comorbidity, ultimately aiding research and treatment targets.
Page | 51
Part II
ON THE INTEGRATION
OF RISK FACTORS INTO
NETWORK MODELS
Environmental Impact in
4
Psychotic Disorder

Isvoranu, A. M., Borsboom, D., van Os, J. & Guloksuz, S. (2016). A Network
Approach to Environmental Impact in Psychotic Disorder: Brief Theoretical
Framework. Schizophrenia Bulletin, 42(4), 870-873.
Chapter 4 | Environmental Impact in Psychosis
Abstract
The spectrum of psychotic disorder represents a multifactorial and heterogeneous
condition, and is thought to result from a complex interplay between genetic and
environmental factors. In the current chapter, we briefly introduce a network-
analysis based framework to study environmental impact in psychotic disorders.
We further analyze this interplay using network analysis, which has been recently
proposed as a novel psychometric framework for the study of mental disorders.
Using general population data, we construct network models for the relation
between three environmental risk factors (cannabis use, developmental trauma,
and urban environment), dimensional measures of psychopathology (anxiety,
depression, interpersonal sensitivity, obsessive compulsive disorder, phobic anxiety,
somatizations, and hostility), and a composite measure of psychosis expression.
Results indicate the existence of specific paths between environmental factors and
symptoms, most often involving cannabis use. In addition, the analysis suggests
that symptom networks are more strongly connected for people exposed to
environmental risk factors, indicating that environmental exposure may lead to
less resilient symptom networks.
Page | 56
4.1 Introduction
Disorders in the psychosis spectrum represent one of the leading causes of long-
term disability worldwide (Howes & Murray, 2014). In spite of extensive efforts,
their etiology remains poorly understood, hampering progress in treatment and
prognosis (Miyamoto et al., 2012).
While collaborative efforts have confirmed the impact of genetic factors, an
important role in the onset and progression of psychotic disorders is thought to
be attributable to several environmental risk factors. Specifically, meta-analytical
reviews have shown that psychosis expression is associated with developmental
trauma, cannabis use, growing up in an urban environment, and minority group
position (van Os et al., 2010). These findings have resulted in extensive research–
using a broad range of methodological approaches–concerned with mapping and
understanding the relations between environmental factors and schizophrenia.
In the current chapter, we propose and describe a novel psychometric framework
for the study of disorders in the psychosis spectrum and show how it can be used
to augment these research efforts. The framework is based on the general idea
that mental disorders arise from the interaction between affective, cognitive,
and behavior components that make up its psychopathology, and is known as the
network approach (Borsboom & Cramer, 2013).
4.2 Psychotic disorder – standard approaches and a

network alternative
Standard approaches to psychosis spectrum diagnoses such as schizophrenia
conceptualize the construct as a latent condition that acts as a common cause of its
symptoms (Schmittmann et al., 2013). In the psychometric representation of this
conceptualization, the diagnosis of schizophrenia is represented as a latent variable
(or set of latent variables), and its symptoms–such as hallucinations and delusions–are
modeled as effects of this latent variable. As a result, symptoms of the disorder are
interpreted as passive psychometric indicators, and interactions between symptoms do
not form a central research interest. In accordance, this framework typically assumes
that environmental factors affect symptoms via the latent disorder (i.e., the disorder
mediates the relation between environment and symptoms). However, recent studies
show this assumption to be problematic, as individual symptoms of a disorder may be
Page | 57
influenced by different risk factors (e.g., see Chapter 5). This suggests that alternative
modeling options should be considered.
One such modeling option lies in the network approach: a novel psychometric
framework based on a dynamical systems perspective. In network models, mental
disorders such as schizophrenia are not conceptualized as common causes of
symptoms, but as conditions that arise from the interaction between symptoms.
Specifically, if symptoms engage in patterns of mutual reinforcement and feedback,
the system as a whole can get ‘locked’ in a state of extended (or even permanent)
symptom activation: a mental disorder. Individual differences in vulnerability are
naturally represented as differences in the connectivity of the network model: in
more strongly connected networks, symptoms feature a higher level of interaction,
which may mean that they will more easily activate each other (van Borkulo et al.,
2016) to render the system as a whole less resilient.
The relevant patterns of interaction can be visualized in a network structure
(Schmittmann et al., 2013), in which variables (here: risk factors and measures of
psychopathology) are represented as nodes (see Figure 4.1). The presence of an edge
between two nodes implies the existence of a statistical association, which does not
vanish upon controlling for all of the other nodes in the network (e.g., a partial
correlation). Thus, the presence of an edge is suggestive of the existence of a causal
relation, although it does not specify the nature or direction of such a relation. In
standard visualizations, blue (red) edges indicate positive (negative) connections.
4.3 Network analysis of general population

psychopathology data
In order to provide an example of how network models can be used to investigate
the association between schizophrenia and environmental exposure, we
constructed three networks of baseline data from the Early Developmental Stages
of Psychopathology (EDSP) study (Wittchen et al., 1998), a 10-year prospective
follow-up study investigating vulnerability and risk factors for onset and progression
of psychopathological syndromes (detailed information about the sample is available
elsewhere (Guloksuz et al., 2015; Wittchen et al., 1998).
First, we determine the network structure pertaining to three environmental
risk factors (cannabis use, developmental trauma, and urban environment), seven
dimensional measures of psychopathology (anxiety, depression, interpersonal
sensitivity, obsessive compulsive disorder, phobic anxiety, somatizations, and
Page | 58
hostility), and one composite dimensional measure of psychosis expression. We
estimate this network using the mgm R-package; details about the method and a
step-by-step tutorial on how to execute network analyses are available elsewhere
(Costantini et al., 2015; Haslbeck & Waldorp, 2020).
The resulting network (see Figure 4.1, panel a), shows a dense pattern of
connections between dimensions of psychopathology, and suggests that the
three environmental risk factors are differentially related to specific symptoms.
For example, developmental trauma is linked to psychosis expression and
somatization, while cannabis use is much more strongly related to other domains
of psychopathology such as depression, anxiety, obsessive-compulsive disorder, and
hostility. In addition, there is a strong positive link between trauma and cannabis
use. Urbanicity has the least strong direct impact on psychopathology symptoms,
featuring only one (weak) positive connection to somatization (i.e., people coming
from urban areas may be more prone to expressing symptoms of somatization). In
fact, the network suggests that the effect of urbanicity may be largely mediated by
cannabis use – people in urban areas may be more likely to use cannabis, which may
in turn lead to the development of, e.g., anxiety.
In recent research, environmental factors were shown to increase psychosis
expression via an increase in general psychopathology (Guloksuz et al., 2015). One
mechanism consistent with this finding is that environmental exposure results in
a more strongly connected and thus more vulnerable network. To evaluate this
hypothesis, we computed separate psychopathology networks for subjects who had
been exposed to cannabis use as an environmental risk factor (Figure 4.1, panel
b) and subjects who had not been exposed to any of the three aforementioned
environmental factors (Figure 4.1, panel c). Even though the two networks show
a high degree of similarity in terms of their structure, the network of exposed
individuals is more strongly connected, and a permutation test (van Borkulo et al.,
2016) shows the connectivity difference to be statistically significant (p=.04). The
difference appears primarily due to increased connectivity of the node hostility,
stronger connectivity between depression and anxiety, and connections between
interpersonal sensitivity and phobic anxiety.
Page | 59
(a) Interrelations Environmental Exposure & Symptomatology
Hos
OCD
Sen
Psychosis Expression
● Psy: Paranoid Ideation & Psychoticism
Dep
Pho
Psychopathology
● Dep: Depression
● Sen: Interpersonal Sensitivity
Psy ● OCD: Obssesive Compulsive Disorder
● Anx: Anxiety
● Pho: Phobic Anxiety
● Som: Somatization
● Hos: Hostility
Anx Environmental Exposure

Tra ● Urb: Urbanicity
● Tra: Trauma
● Can: Cannabis
Som
Can
Urb
(b) No Environmental Exposure (c) Environmental Exposure Cannabis
Hos Hos
Sen OCD Sen OCD

Dep Dep
Psy Psy
Pho Pho
Anx Anx
Som Som
Figure 4.1 (a) Network visualization of interrelations between environmental factors and symp-
tomatology. (b) (c) Network visualization of differences in psychopathology symptoms connectivity
between a group exposed to cannabis use and a non-environmentally exposed group.
Page | 60
Network models can serve to disentangle the mechanisms that underlie the relation
between environmental risk factors and disorders in the psychosis spectrum. The
present note has evaluated two network mechanisms by which this may occur. First,
environmental factors may exert main effects on specific (sets of) symptoms (e.g.,
cannabis-anxiety) that subsequently spread through the symptom network. Second,
environmental factors may increase the strength of interactions between symptoms,
leading to a more strongly connected and less resilient network structure. The
current examples are consistent with both of these processes.
Future research should replicate these results, and can be extended to unravel
the precise mechanisms that underlie these statistical associations. In addition,
such research should consider the role of genes in modifying the symptom network
structure. In particular, it seems likely that genetic vulnerability may be expressed
as increased connectivity of symptom interactions, which would serve to increase
the impact of environmental factors on the network structure and/or to amplify
the increase in connectivity that arises from environmental factors themselves.
Page | 61
Pathways Between Childhood
5
Trauma and Psychotic
Symptoms

Isvoranu, A. M., van Borkulo, C. D., Boyette, L. L., Wigman, J. T. W., Vinkers,
C. H., Borsboom, D., GROUP Investigators (2017). A Network Approach to
Psychosis: Pathways Between Childhood Trauma and Psychotic Symptoms.
Schizophrenia Bulletin, 43(1), 187-196.
Chapter 5 | Childhood Trauma and Psychotic Symptoms
Abstract
Childhood trauma has been identified as a potential risk factor for the onset of
psychotic disorders. However, to date, there is limited consensus with respect to
which symptoms may ensue after exposure to trauma in early life, and whether
specific pathways may account for these associations. The aim of the present
chapter was to use the novel network approach to investigate how different types of
traumatic childhood experiences relate to specific symptoms of psychotic disorders
and to identify pathways that may be involved in the relation between childhood
trauma and psychosis. We used data of patients diagnosed with a psychotic disorder
(N=552) from the longitudinal observational study Genetic Risk and Outcome of
Psychosis Project (GROUP) and included the five scales of the Childhood Trauma
Questionnaire-Short Form (CTQ-SF) and all original symptom dimensions of
the Positive and Negative Syndrome Scale (PANSS). The results show that all five
types of childhood trauma and positive and negative symptoms of psychosis are
connected through symptoms of general psychopathology. These findings are in line
with the theory of an affective pathway to psychosis after exposure to childhood
trauma, with anxiety as a main connective component, but they also point to
several additional connective paths between trauma and psychosis: e.g., through
poor impulse control (connecting abuse to grandiosity, excitement, and hostility)
and motor retardation (connecting neglect to most negative symptoms). The results
of the current chapter suggest that multiple paths may exist between trauma and
psychosis and may also be useful in mapping potential transdiagnostic processes.
Page | 64
5.1 Introduction
Childhood trauma (CT) has been extensively investigated as a potential risk factor
for the onset and course of psychosis, and was found to relate to some of the most
severe forms of symptomatology in adulthood, including hallucinations, delusions,
and paranoia (here used with the meaning ‘suspiciousness’; Larkin & Read, 2003;
Longden et al., 2016; John Read et al., 2003, 2005). However, in spite of intense
research into the topic, the nature of the relationship between CT and psychosis
is yet to be fully understood.
Current psychometric practices in psychopathology research conceptualize
psychotic disorders–and mental disorders in general–as common causes of symptoms
(Schmittmann et al., 2013). In other words, symptoms are taken to be indicators of an
underlying disease entity and correlations between symptoms can be fully explained
by the common influence of the latent variable. Despite decades of research, however,
finding such underlying causes for symptoms has been very rare (Fried, 2015); instead,
the causes appear to be multifactorial, thus challenging the likelihood of a common
cause explanation for associations between symptoms (Cramer, 2012; Fried &
Nesse, 2014; McNally et al., 2015; Wichers, 2014). As a result, in recent years, the
common cause approach to mental disorders has been called into question and the
dynamical systems conceptualization of psychopathology gained ground, leading to
the development of network models (Borsboom & Cramer, 2013). In network theory,
correlations between symptoms are no longer explained by the common latent factor
(i.e., mental disorder), but mental disorders are conceptualized as complex systems,
in which symptoms and psychological, biological, and sociological components have
autonomous causal power to influence each other (Borsboom & Cramer, 2013;
Cramer et al., 2010; Kendler et al., 2011). Within this framework, the associations
among symptoms are conceptualized to constitute the disorder: An external trigger
could produce a certain symptom (e.g., anxiety), which could in turn activate other
symptoms (e.g., paranoia). In addition, in contrast to the common cause model and in
line with clinicians’ viewpoints (Ma & Teasdale, 2004), network models allow for the
notion of mutually reinforcing symptom cycles (e.g., social withdrawal leading to poor
rapport and poor rapport leading in turn to social withdrawal). To date, the network
approach contributed to several advancements in psychopathology research (Fried,
Bockting, et al., 2015; McNally et al., 2015; Ruzzano et al., 2015) and personality
research (Cramer et al., 2012). For instance, it was found that losing a partner (i.e.,
bereavement) mainly impacted on the symptom ‘loneliness’, which in turn activated
Page | 65
other depression symptoms (Fried, Bockting, et al., 2015). In addition, non-DSM

(American Psychiatric Association, 2013) symptoms, such as sympathetic arousal,
were identified to be as central in a network (i.e., as important) as DSM depression
symptoms (Fried, Epskamp, et al., 2015). Finally, at least in depression, symptom
network structures appear to be associated with the prospective course of the disorder:
the symptom network is more strongly connected in the subgroup of individuals with
a worse prognosis (van Borkulo et al., 2016).
The current chapter used the network framework to address two issues in
psychosis research that have yet to be solved. First, there is limited consensus
whether CT only evokes specific psychotic symptoms or all symptoms to the same
degree. For instance, several studies have specifically linked CT to hallucinations
(Whitfield et al., 2005), to both hallucinations and delusions (Bentall et al.,
2012; Janssen et al., 2004; Larkin & Read, 2003; John Read & Argyle, 1999),
and found no significant link between CT and negative symptoms (John Read
et al., 2005). More recent research, however, highlighted the heterogeneity in the
relationship between CT and psychotic symptoms. In a large sample of patients
with psychosis, childhood abuse and neglect resulted in a vulnerability to develop
both (subthreshold) positive and negative symptoms (Vinkers et al., 2013); all
positive symptoms were more pronounced in the context of abuse (van Dam et
al., 2015). Furthermore, in a large recent epidemiological study, no evidence for
specificity of any of the childhood trauma variables (emotional neglect, physical
abuse, psychological abuse, or sexual abuse) to any of the psychotic experiences was
identified and CT was associated with a wide range of psychotic symptoms (van
Nierop et al., 2014).
Second, the exploration of potential mechanistic pathways that account for
the association between CT and psychosis is at very best in its infancy. It has been
hypothesized that traumatic experiences may result in structural and neurochemical
abnormalities in the brain and nervous system, thereby affecting the function of
the hypothalamus-pituitary-adrenal axis, which plays a role in stress response
(Carpenter et al., 2009; Dannlowski et al., 2012; John Read et al., 2001). It is
thus possible that CT may lead to psychosis due to increased emotional reactivity
to daily life stress, supporting the construct of an affective pathway to psychosis
(Myin-Germeys & van Os, 2007). Alternatively, cognitive models of psychosis
argue that trauma may lead to negative beliefs about the self, world, and others.
These beliefs may in turn lead to distressing interpretations of everyday events,
eventually resulting in psychotic experiences (Birchwood et al., 2000).
Page | 66
One limitation of many existing studies is that these investigated the relationship
between CT and positive psychotic symptoms (primarily hallucinations and
delusions), but less attention was given to other symptom scales in patient
populations. This may be an important omission, as the psychosis spectrum
is comprised of multiple symptom domains, covering in addition to positive
psychotic symptoms, also negative psychotic symptoms as well as affective and
cognitive symptoms (van Os & Kapur, 2009). In the present study, all these
symptom dimensions are considered. To this end, we used the most widely accepted
questionnaire to assess symptom severity in psychotic disorders: the Positive and
Negative Syndrome Scale (PANSS; Kay et al., 1987). The PANSS consists of three
subscales: the positive scale, the negative scale, and the general psychopathology
scale. The aim of the present chapter was to contribute to the efforts of revealing
the nature of the relationship between CT and psychosis, by analyzing the network
structure of all underlying symptoms of psychotic disorders and their relation to
CT. This may give insights into both the relationship between CT and psychotic
symptoms, and the potential pathways that may account for this association.
5.2 Methods
5.2.1 Participants
We analyzed data (database version 3.2) from the longitudinal observational cohort
study Genetic Risk and Outcome of Psychosis Project (GROUP; Korver et al., 2012),
which was designed to study vulnerability and protective factors for variation in
expression and course of non-affective psychotic disorders. The full GROUP sample
consists of patients with psychotic disorders (N=1120), their siblings (N=1057),
their parents (N=919), and a control group (N=590). In the present study, we
used data from the patient sample only. The patients were recruited from 36
mental health care institutions in The Netherlands and Belgium including four
academic medical centers (Amsterdam, Groningen, Maastricht, Utrecht). Inclusion
criteria for participating patients were age between 16 and 50 years, meeting
full DSM (American Psychiatric Association, 2013) criteria for a non-affective
psychotic disorder, maximum duration of illness of 10 years, and estimated level of
intelligence quotient above 70. More detailed information of sample characteristics
and recruitment methods has been previously published (Korver et al., 2012).
Page | 67
5.2.2 Symptomatology
We used the first wave of data from the Positive and Negative Syndrome Scale
(PANSS; Kay et al., 1987) as an interview-rated measure of symptom severity in the
patient population. The PANSS consists of 30 items divided in three subscales: the
positive scale (e.g., hallucinations, paranoia), the negative scale (e.g., social withdrawal,
blunted affect), and the general psychopathology scale (e.g., anxiety, depression, poor
impulse control, motor retardation). It is scored by a trained interviewer on a seven-
point Likert-type scale, ranging from 1 (absent) to 7 (very severe).
5.2.3 Childhood trauma

Childhood trauma was measured with the Dutch version of the Childhood Trauma
Questionnaire-Short Form (CTQ-SF; Bernstein et al., 2003). First wave data were
collected at the Maastricht site, while follow-up measurements were collected at
the other sites. The CTQ-SF is a self-report questionnaire, consisting of 24 items,
which were scored on a five-point Likert-type scale, ranging from 1 (never true) to
5 (always true). The CTQ measured five types of childhood maltreatment (before
the age of 17): physical neglect (failure of caretaker to provide basic necessities for
a child such as food, clothing, shelter); physical abuse (bodily assault on a child
posing a risk of or resulting in injury); emotional neglect (failure of caretaker
basic emotional and psychological needs for a child, such as love and nurturance);
emotional abuse (verbal assaults on a child, such as humiliation); and sexual
abuse (unwanted sexual contact or conduct between a child and an adult). Each
scale encompassed five items, with the exception of the sexual abuse scale, from
which the item ‘Molestation’ was removed in the current study due to improper
translation into Dutch (Thombs et al., 2009). A sum-score was calculated for each
scale and used when computing the networks – the sum of the sexual abuse scale
was multiplied by 5/4 due to the removed item (Bernstein & Fink, 1998).
5.2.4 Network construction

We constructed networks in which each of the PANSS symptoms and each of
the five scales of the CTQ-SF questionnaire are represented as nodes, and an edge
between two nodes indicates a partial correlation between the two variables, after
conditioning on all other variables in the dataset. Blue edges illustrate positive
partial correlations, red edges negative partial correlations, and the wider and more
saturated the edge, the stronger the association (Epskamp et al., 2012). State of
the art methodology from the field of statistical learning was utilized to select
Page | 68
which edges should be included in the network. These edges can be interpreted as
predictive effects. Thus, the networks lie in-between correlation networks (in which
connections represent zero-order correlations) and fully directed causal networks
(in which all connections are oriented): they represent the part of the pairwise
association structure that cannot be explained by other variables in the model.
More technically, we fitted a Gaussian Graphical Model (GGM; Lauritzen &
Wermuth, 1989) to the data. A GGM is an undirected network, in which a missing
edge indicates that two variables are independent after conditioning on the set
of remaining variables. To control for spurious connections that may result from
sampling error, as well as to estimate a more interpretable and sparse model, recent
literature (Costantini et al., 2015) suggests to employ L1 regularization (Tibshirani,
1994). L1 regularization corrects for Type 1 errors and reduces the overall strength
of parameter estimates, for model simplification (i.e., the small values will thus
become exactly zero, while the others values will be shrunken; Friedman et al.,
2008); in addition we used a minimum absolute value of .03 for visualization. L1
regularization utilizes a tuning parameter that controls the sparsity of the model,
which can be selected by minimizing the Extended Bayesian Information Criterion
(Chen & Chen, 2008). The EBIC itself utilizes a tuning hyperparameter γ, typically
set to .5, which was shown to yield accurate network estimations (Boccaletti et
al., 2006; Opsahl et al., 2010; van Borkulo, Borsboom, et al., 2014). This method
converges to the true network, assuming that a set of sparse pairwise interactions
indeed underlie the data (Barrat et al., 2004). For more details we refer the reader
to the cited tutorial paper (Costantini et al., 2015; Lauritzen & Wermuth, 1989).

The resulting network was further analyzed by investigating the importance
(centrality) of each node in the network. This can be captured in centrality
measures of which three were investigated here: node strength, betweenness,
and closeness (Barrat et al., 2004; Boccaletti et al., 2006; Opsahl et al., 2010).
In weighted networks, node strength is a measure of the number and strength of
connections. ‘Betweenness’ measures how often a node lies on the shortest path
between every combination of two other nodes, indicating to what extent the node
facilitates the flow of information through the network. The ‘closeness’ of a node
measures the average distance from that node to all other nodes in the network,
with high closeness indicating a short average distance between a given node and
the remaining nodes in the network.
Page | 69
Next, networks illustrating the shortest paths between each trauma scale and the
positive and negative symptoms of the PANSS were computed. In comparison to
the first network, these networks allow clear identification of possible pathways
and mediating items between trauma and psychotic symptoms. The shortest path
between two nodes represents the minimum number of steps needed to go from one
node to the other (Brandes, 2008), and it is computed using Dijkstra’s algorithm
(Dijkstra, 1959). This can be seen as a roadmap including all possible routes from
destination A to destination B, but only one of these routes being quicker – this
would then be the route highlighted in the shortest path networks. Our networks
illustrate what the shortest routes are from each CT scale to the different clusters
of positive and negative psychotic symptoms. Given that the partial correlations
between one node and all other nodes in the network are directly related to the
regression coefficients obtained in a multiple regression model, these can thus be
interpreted as predictive effects (Langley & Epskamp, 2015): Two connected nodes
predict each other, and any node that connects the two nodes (e.g., node B in
the pathway A - B - C) can be seen to mediate the predictive quality between
the two nodes. As such, a partial correlation network can be viewed as a skeleton
encompassing the existence of putative causal relations.
All analyses were performed using the R-statistical software (R Core Team,
2015). The networks were constructed and visualized using the R-package qgraph
(Epskamp et al., 2012). The layout used when computing the networks was
Fruchterman and Reingold’s (Fruchterman & Reingold, 1991) layout, which places
the nodes with stronger connections into the center of the network, and the nodes
with weaker connections closer to the periphery of the network.
5.3 Results
In total, after removing all missing data, 552 patients were included in the analyses.
The participating patients, of whom 75% were male, had a mean age of 30.8 years
(SD=7.27). Based on a cut-off score of low to moderate severity (Bernstein & Fink,
1998), approximately 25% of the patients reported being physically neglected, 79%
emotionally neglected, 25% sexually abused, 26% emotionally abused, and 16%
physically abused. A significant gender difference (p=.01) was identified on the
sexual abuse scale, with females reporting higher scores than males.
All missing data were removed using the listwise deletion method. Comparison
of participating patients with excluded patients revealed a significant difference
Page | 70
(p=.008) in gender, with the number of male patients being higher in the sample
excluded due to missing data. We have carried out an MCAR test using the R
package MissMech (Jamshidian et al., 2014), which confirmed that the PANSS and
CT data were missing at random. No significant differences were identified between
the omitted participants and participants who were included in the analytic sample
on age, diagnosis, scores on the PANSS interview, and childhood trauma reports
(all p>.05), with the exception of the score on the physical neglect scale of the
CTQ (p=.048). In addition, no significant differences were found (all comparison
p>.05) in the PANSS scores between participants who completed the PANSS and
the CTQ at baseline, and participants who completed the PANSS but were not
available for childhood trauma assessment.
Table 5.1 presents the demographic and clinical characteristics of the sample,
the mean sum-scores on the three PANSS dimensions, and the five CTQ-SF
dimensions of the CTQ-SF. Table 5.2 presents the item distribution and the item
labels for all following networks. The current data were not univariate normally
distributed, and as such, a nonparanormal transformation (Liu et al., 2009) to relax
the normality assumption was applied prior to constructing the networks.
The first network we constructed illustrates the relationship between CT
and all 3 scales of the PANSS (i.e., positive symptoms, negative symptoms, and
general psychopathology symptoms; see Figure 5.1). When evaluating the network,
strikingly, there is no connectivity between the CT scales and positive or negative
psychotic symptoms; CT only connects to the general psychopathology scale of
the PANSS. Node CT5 (physical abuse) is positively associated with GP1 (somatic
concern) and GP14 (poor impulse control) and negatively associated with GP5
(mannerism and posturing). Node CT4 (emotional abuse) is positively associated
with item GP2 (anxiety), node CT3 (sexual abuse) is positively associated with item
GP3 (guilt), and node CT1 (physical neglect) is positively associated with item GP7
(motor retardation). Centrality measures of the network (see Figure B1) show that
item GP9 (unusual thought content) has the highest betweenness, closeness, and
strength measures, suggesting this is as an influential node within the network. In
addition, all CT items are highly interconnected within their given scale, implying
that correlations among this subscale are much larger than correlations across scales.
Page | 71
Table 5.1 Demographic and clinical characteristics of participating patients: means (standard deviations)
Variable Men (N=418) Women (N=134) Total (N=552)
Age (years) 30.35 (6.59) 32.01 (8.99) 30.76 (7.27)
Diagnostic (%)
Schizophrenia 294 (53.3%) 86 (15.6%) 380 (68.9%)
Schizophreniform 12 (2.2%) 7 (1.2%) 19 (3.4%)
Schizo-affective 63 (11.4%) 28 (5.1%) 91 (16.5%)
Delusional 7 (1.2%) 0 7 (1.2%)
Psychotic NOS 42 (7.6%) 13 (2.4%) 55 (10%)
PANSS (sum)
Positive symptoms 12.93 (5.42) 11.16 (4.64) 12.50 (5.29)
Negative symptoms 13.65 (5.69) 12.04 (5.15) 13.26 (5.60)
General Psychopathology 27.67 (8.26) 25.18 (7.72) 27.06 (8.20)
CTQ-SF (sum)
Emotional Neglect 18.80 (4.25) 18.25 (4.75) 18.69 (4.38)
Physical Neglect 12.18 (1.82) 11.90 (1.83) 12.11 (1.83)
Emotional Abuse 8.83 (3.85) 9.96 (4.77) 9.11 (4.19)
Physical Abuse 6.25 (2.65) 6.28 (2.71) 6.26 (2.66)
Sexual Abuse 10.30 (3.04) 9.75 (2.86) 10.17 (3.00)
5.3.1 Shortest paths: childhood trauma - positive and negative psychotic symptoms
Following the symptoms network, we constructed five networks that depict
shortest paths between each of the five CT scales and all individual symptoms of
the PANSS. Due to high similarities between these networks and space constraints,
only two such networks will be described in this section and the other networks
are included in Supplement B. These two specific CT scales were chosen because
they highlight nearly all possible routes from other trauma scales to the psychotic
symptoms as well.
The shortest path networks illustrate pathways between the node CT3 (sexual
abuse) and the positive and negative symptoms (Figure 5.2, panel a), and the
node CT5 (physical abuse) and the positive and negative symptoms (Figure 5.2,
panel b). In other words, the networks display the shortest routes that connect the
nodes CT3 (sexual abuse) and CT5 (physical abuse) to each individual positive and
negative symptom of the PANSS.
Page | 72
Childhood Trauma
Childhood Trauma
Childhood Trauma
CT1: Physical neglect
CT4 CT2 CT1: Physical neglect
CT4 CT1: Physical neglect
CT4 CT2 CT2: Emotional neglect
CT2
GP1 CT2: Emotional neglect
CT2: Emotional neglect
CT3: Sexual abuse
GP1 CT5
GP1 CT3 CT3: Sexual abuse
P3 CT5 CT3: Sexual
CT4: Emotional
abuse abuse
CT5 CT1 CT3
P3 CT3 General Psychopahtology
P3 CT1 CT4: Emotional abuse
CT1 CT4: Emotional
CT5: Physicalabuse
absue General Psychopahtology
General Psychopahtology
CT5: Physical absue GP1: Somatic concern
CT5: Physical absue
GP1: Somatic concern
GP1:
GP2:Somatic concern
Anxiety
P1 Positive Symptoms GP2: Anxiety
GP9 GP2: Anxiety
GP3: Guilt
P1 P6 GP2 Positive Symptoms
GP9 P1 GP3 Positive Symptoms
P1: Delusions
GP2 GP3: Guilt
GP9 P6 GP3 GP3: Guilt
P6 GP2 P1: Delusions GP4: Tension
GP3 Delusions
GP6
P1:P2: Conceptual disorganization GP4: Tension
GP4:
GP5:Tension
Mannerism and posturing
GP6 P2: Conceptual disorganization
GP4 Conceptual disorganization
GP6 P2:P3: Hallucinations GP5: Mannerism and posturing
P5 GP4 GP5: Mannerism and posturing
GP4 P3: Hallucinations GP6: Depression
P5 GP15 Hallucinations
P5
P3:P4: Excitement GP6: Depression
GP15 GP6: Depression
GP15 P4: Excitement GP7: Motor retardation
GP16 Excitement
P4:P5: Grandiosity GP7: Motor retardation
GP16 GP7:
GP8:Motor retardation
Uncooperativeness
GP14 GP16 P5: Grandiosity
Grandiosity
P5:P6: Paranoia/ suspiciousness GP8: Uncooperativeness
GP14 GP8: Uncooperativeness
GP14 N7 N4 P6: Paranoia/ suspiciousness GP9: Unusual thought content
N4 Paranoia/
P6:P7: Hostilitysuspiciousness GP9: Unusual thought content
N7 GP11 N4
N7 P7: Hostility GP9: Unusual
GP10: thought content
Disorientation
GP11 GP5 P7: Hostility GP10: Disorientation
P4 GP11 GP10: Disorientation
GP5 N2 GP11: Poor attention
P4 GP5 Negative Symptoms
P7 P4 N2 GP11: Poor attention
P2 N2 GP10 Negative Symptoms GP11: Poor
GP12: Poorattention
judgement and insight
P7 Negative Symptoms
P7 P2 GP10 N1: Blunted affect GP12: Poor judgement and insight
P2 GP10 N1: Blunted affect GP12: Poor judgement
GP13: Disturbed and insight
willpower
GP13 Blunted
N1:N2: Emotional
affectwithdrawal GP13: Disturbed willpower
GP13 N2: Emotional withdrawal GP13: Disturbed willpower
GP14: Poor impulse control
GP13 Emotional withdrawal
GP12 N1 GP7 N2:N3: Poor rapport GP14: Poor impulse control
GP8 N1 N3: Poor rapport GP14:
GP15:Poor impulse control
Preoccupation
GP12 GP7
GP12 GP8 N1 GP7 Poor
N3:N4: rapport
Social withdrawal GP15: Preoccupation
GP8 N4: Social withdrawal GP15:
GP16:Preoccupation
Active social avoidance
N3 Social
N4:N5: withdrawal
Difficulty in abstract thinking GP16: Active social avoidance
N3 N5: Difficulty in abstract thinking GP16: Active social avoidance
N5 N3 N6 Difficulty abstract thinking
N5:N6: Lack of in
spontaneity
N5 N6
N5 N6 N6: Lack of spontaneity
Lack
N6:N7: of spontaneity
Stereotyped thinking
N7: Stereotyped thinking
N7: Stereotyped thinking
Figure 5.1 Network depicting the three dimensions (positive, negative, general psychopathology symptoms) of the Positive and Negative Syndrome Scale (PANSS)
and the five dimensions of the Childhood Trauma Questionnaire - Short Form (CTQ-SF). Symptom groups are differentiated by colors. Each edge within the net-
work corresponds to a partial correlation between two individual items. The thickness of an edge represents the absolute magnitude of the association (the thicker the
edge, the stronger the connection), while the color of the edge indicates the size of the correlation (blue for positive connections, red for negative connections).
Page | 73
Table 5.2 Items of the Positive and Negative Syndrome Scale (PANSS) and the Childhood Trauma
Questionnaire-Short Form (CTQ-SF) and their assigned colors and labels
Item label Domain color Item description

P1 Purple Delusions
P2 Purple Conceptual Disorganization
P3 Purple Hallucinations
P4 Purple Excitement
P5 Purple Grandiosity
P6 Purple Paranoia/ Suspiciousness
P7 Purple Hostility
N1 Blue Blunted Affect
N2 Blue Emotional Withdrawal
N3 Blue Poor Rapport
N4 Blue Social Withdrawal
N5 Blue Difficulty in Abstract Thinking
N6 Blue Lack of Spontaneity and Flow of Conversation
N7 Blue Stereotyped Thinking
GP1 Yellow Somatic Concern
GP2 Yellow Anxiety
GP3 Yellow Guilt
GP4 Yellow Tension
GP5 Yellow Mannerism and Posturing
GP6 Yellow Depression
GP7 Yellow Motor Retardation
GP8 Yellow Uncooperativeness
GP9 Yellow Unusual Thought Content
GP10 Yellow Disorientation
GP11 Yellow Poor Attention
GP12 Yellow Poor Judgment and Insight
GP13 Yellow Disturbed Willpower
GP14 Yellow Poor Impulse Control
GP15 Yellow Preoccupation
GP16 Yellow Active Social Avoidance
CT1 Maroon Physical Neglect
CT2 Maroon Emotional Neglect
CT3 Maroon Sexual Abuse
CT4 Maroon Emotional Abuse
CT5 Maroon Physical Abuse
Page | 74
In Figure 5.2, panel a, the shortest route to reach most negative psychotic symptoms
from node CT3 (sexual abuse) is via node CT1 (physical neglect) and GP7 (motor
retardation). The shortest route from CT3 (sexual abuse) to P1 (delusions),
P3 (hallucinations), and P6 (paranoia) is via CT4 (emotional abuse) and GP2
(anxiety), while the connectivity to the remaining positive nodes P2 (conceptual
disorganization), P4 (excitement), P5 (grandiosity), and P7 (hostility) runs through
CT5 (physical abuse) and GP14 (poor impulse control). Lastly, to reach node N7
(stereotyped thinking) from CT3 (sexual abuse), the shortest route is via node CT5
(physical abuse), GP14 (poor impulse control), and GP15 (preoccupation).
Similarly, in Figure 5.2, panel b, node CT1 (physical neglect) is connected to
node GP7 (motor retardation), which is in turn connected to the negative symptoms
N1 (blunted affect), N2 (emotional withdrawal), N3 (poor rapport), N4 (social
withdrawal) and N6 (lack of spontaneity and flow of conversation). Node CT5
(physical abuse) appears to be associated with positive symptoms through three
pathways: (1) CT5 connects to nodes GP1 (somatic concern) and GP9 (unusual
thought concern), which is in turn connected to nodes P1 (delusions) and P3
(hallucinations); (2) CT5 (physical abuse) connects to node GP14 (poor impulse
control), which is in turn connected to the nodes P2 (conceptual disorganization), P4
(excitement), P5 (grandiosity), and P7 (hostility); (3) CT5 (physical abuse) connects
to the trauma node CT4 (emotional abuse), which is in turn connected to item GP2
(anxiety), followed by item P6 (paranoia). The networks depicting the shortest paths
between the remaining three CT scales - CT1 (physical neglect), CT2 (emotional
neglect), and CT4 (emotional abuse) - and the positive, negative and general symptoms
are presented in Figures B2, B3, and B4 in Supplement B.
In addition, taking into account previous literature that focused mainly
on positive and negative psychotic symptoms, we constructed a network
that illustrates the relationship between CT and only these two scales of the
PANSS (see Figure B5 in Supplement B). This additional step highlights that
in the absence of the general psychopathology scale, there is direct connectivity
between CT and positive psychotic symptoms, via the nodes P6 (paranoia) and
P7 (hostility). When including the general psychopathology scale, however, this
connectivity is no longer present.
Page | 75
a. Shortest Paths: Sexual Abuse − Positive and Negative Symptoms b. Shortest Paths: Physical Abuse − Po
CT4
CT2
GP1 GP1
CT5 CT3 CT5
P3 P3
CT1
P1 P1
GP9 GP2 GP9
P6 P6
GP3
GP6
GP4
P5 P5
GP15 GP15
GP16
GP14 GP14
N7 N4 N7
GP11 GP11
GP5
P4 P4
N2
P7 P7
P2 GP10 P2
GP13 GP13
GP12 N1 GP7 GP12

GP8 GP8
N3
N5 N6 N5
ive and Negative Symptoms b. Shortest Paths: Physical Abuse − Positive and Negative Symptoms
CT4 CT4
CT2 CT2
GP1
CT3 CT5 CT3

P3
CT1 CT1
P1
GP2 GP9 GP2
P6
GP3 GP3
GP6 GP6
P4 GP4
P5
GP15
GP16 GP16
GP14
N4 N7 N4
GP11
GP5 GP5
P4
N2 N2
P7
GP10 P2 GP10
GP13
N1 GP7 GP12 N1 GP7

GP8
N3 N3
N6 N5 N6
Figure 5.2 a.Network depicting shortest paths between the Sexual Abuse scale (i.e., node CT3) of
the Childhood Trauma Questionnaire-Short Form (CTQ-SF) and the two main dimensions of the
Positive and Negative Syndrome Scale (PANSS): positive symptoms and negative symptoms.
b. Network depicting shortest paths between the Physical Abuse scale (i.e., node CT5) of the
CTQ-SF and the two main dimensions of the PANSS: positive symptoms and negative symptoms.
Dashed lines represent background connections existent within the network that are less relevant
when investigating shortest paths; thicker dashed lines represent stronger connections.
Page | 76
5.4 Discussion
The current chapter provides the first network-based analysis of the relationship
between CT and psychosis. In sum, we constructed a network that included the
three original symptom dimensions of the PANSS interview (i.e., positive symptoms,
negative symptoms, general psychopathology symptoms). Our results show that
the CT subscales are not directly associated with any of the positive or negative
symptoms – they are connected to the two scales only via general psychopathology
symptoms. In other words, general psychopathology symptoms appear to mediate
the relation between trauma and psychosis. By computing shortest path networks,
we illustrated which general psychopathology symptoms may be activating different
positive and negative symptoms (e.g., anxiety activating paranoia, delusions,
and hallucinations; poor impulse control activating grandiosity, hostility, and
excitement; motor retardation activating the cluster of negative symptoms). In
certain cases, a CT node is identified on the shortest pathway between another CT
node and psychotic symptoms (e.g., physical neglect, emotional abuse, and physical
abuse are identified on the shortest path from sexual abuse to psychotic symptoms).
In line with findings from previous research (Bentall et al., 2012; Shevlin et al.,
2008), this indicates that the effects of trauma to symptoms can propagate through
other types of trauma as well.
5.4.1 Research implications

Even though research has so far been successful in establishing a connection
between childhood trauma and psychotic symptoms, there is limited consensus
with regard to potential pathways that may account for this relation. Supporting
the idea of an affective pathway to psychosis, previous research found that subjects
with a history of CT reported increased negative affect, which moderated the
emotional reactivity to small daily stressors (Glaser et al., 2006). Likewise, higher
levels of daily life stress sensitivity were associated with higher levels of positive
symptoms (Lataster et al., 2010), and a history of life events was found to increase
the emotional reaction to daily life stressors. Additionally, the relationship between
sexual abuse and psychosis was found to be mediated by anxiety and depression
(Bebbington et al., 2011). Our results are in line with the existence of an affective
pathway to psychosis (Janssen et al., 2004) – trauma may lead to psychosis through
a pathway of heightened emotional distress (e.g., anxiety, tension, depression). In
the present chapter, anxiety was the main link between emotional abuse and the
Page | 77
positive symptoms cluster paranoia, delusions, and hallucinations. Recent studies

using the experience sampling method also found that an increase in anxiety
predicts the onset of paranoid episodes (Ben-Zeev et al., 2011; Thewissen et al.,
2011). Furthermore, unusual thought concern was the most central item in the
network, indicating that it is essential in facilitating the flow of information.
However, the current findings also suggest the existence of a connective role
of other symptoms of general psychopathology (e.g., poor impulse control, motor
retardation, and somatic concern). First, even though not often investigated in
relation to psychotic disorders, one previous study identified impulsivity as a
predictor of psychotic experiences (Chapman et al., 1984). Our results support
these findings and suggest that poor impulse control should receive more attention
as a potential risk factor in psychosis because it is the linking item between physical
abuse and the positive symptoms cluster grandiosity, hostility, and excitement.
Second, motor retardation (i.e., delayed or reduced movements, speech, and
decreased responsiveness to stimuli) has long been acknowledged as a central
component in major depression and bipolar disorder (Buyukdura et al., 2011).
In a recent network paper investigating the centrality of depression symptoms,
motor retardation displayed high centrality indices and was associated with energy
loss, concentration problems, sympathetic arousal, and suicidal ideation (Fried,
Epskamp, et al., 2015). It is thus not surprising that in the present analysis motor
retardation is strongly connected to blunted affect and lack of spontaneity and flow
of conversation, setting up the main pathway between physical neglect and other
negative symptoms.
Third, somatic concern and unusual thought concern were identified on the
pathway from physical abuse to delusions and hallucinations, suggesting that over-
attention toward somatic symptoms may also act as a trigger of psychotic symptoms.
Other potential pathways from CT to psychosis through symptoms of general
psychopathology may exist, given that our symptom network shows further links
(e.g., sexual abuse associated with guilt). Nonetheless, these are not apparent in
the shortest path networks and as such we argue that the three pathways described
above may be most influential in the association of CT to psychosis - clinical
practice and future interventions may benefit from taking these into consideration.
In addition, such findings re-emphasize the idea that CT is connected to a
wide array of symptoms that are present in several mental conditions (Hovens et
al., 2012; Nanni et al., 2012), and thus are not only specific to psychotic symptoms.
Indeed, childhood trauma has been found to increase the risk of anxiety disorders,
Page | 78
major depressive disorder, bipolar disorder, and personality disorders (Herman et
al., 1989; Leverich et al., 2002; Young et al., 1997). In our sample, comorbidity
rates with depression and bipolar disorder were low (Korver et al., 2012), but not
fulfilling the diagnostic criteria does not imply absence of subthreshold symptoms
(e.g., motor retardation is a central symptom in depression, but also a symptom
measured by the general psychopathology scale of the PANSS).
5.4.2 Limitations
Several limitations of the current study should be taken into consideration. First,
the majority of the sample studied in this report was male, and males often report
lower levels of early trauma compared to females (Tolin & Foa, 2006). In our
analysis, a higher number of male patients were excluded due to missing data. In
addition, males and females display different psychosis symptom profiles (John
Read & Beavan, 2013) and as such a different pattern of results may be observed
for a more gender-balanced sample. Unfortunately, because of unequal sample
sizes and power issues, separate analyses in the current dataset were not feasible.
Follow-up studies may investigate gender differences in relation to early trauma
and psychosis within a network framework. Second, the CTQ-SF is a self-report,
retrospective measure of CT and may thus be prone to bias (e.g., social desirability,
memory bias, demand characteristics). Third, CT data collection points were not
identical, thus making it possible that more severe symptoms could have influenced
the trauma reports. Nonetheless, recent research showed that reports on childhood
abuse in patients with psychotic disorders are reasonably stable over a 7-year period
and are not associated with current severity of psychotic symptoms (Fisher et al.,
2011). Fourth, the current analyses were based on cross-sectional data; the resulting
networks may be exemplary for individuals, but research on generalizability to
the individual level is warranted. Moreover, due to the between-subject design,
conclusions regarding direction of effects or causality should be drawn with caution.
Further research could extend on this analysis by using Bayesian techniques for
the application of causal modeling – such as those based on the theory of Directed
Acyclic Graphs – that may help elucidate further structure in the network
(Nagarajan et al., 2013; Scutari, 2010). Lastly, given the novelty of network models
in psychopathology, rigorous methods for assessing the reliability of the estimated
graph are required (e.g., a more clear identification of potential errors resulting
from sampling, validity of underlying assumptions; Kalisch et al., 2010) and still
Page | 79
in the process of being developed. We trust future studies will be able to overcome
these current shortcomings.

To our knowledge, the present chapter is the first study to investigate the relationship
between childhood trauma and psychotic experiences in patients diagnosed with a
psychotic disorder by using the network framework. This novel alternative approach
to psychopathology conceptualizes mental disorders as causal systems of interacting
symptoms. Our results suggest that several symptoms of general psychopathology
may mediate the relationship between trauma and psychosis, providing evidence
for multiple paths between trauma and psychosis, and re-emphasizing questions
regarding the specificity of trauma to psychosis.
Page | 80
Page | 81
Short- and Long-Term
6
Links between Cannabis
(Ab)use and Psychotic
Symptomatology

Isvoranu, A. M., Boyette, L. L., van Borkulo, C. D., de Haan, L., Borsboom,
D., GROUP Investigators (submitted). Short- and Long-Term Links between
Cannabis (Ab)use and Psychotic Symptomatology.
Chapter 6 |Cannabis (Ab)use and Psychotic Symptomatology
Abstract
Extensive research indicates a strong association between cannabis use and
psychotic symptomatology. While there is a general agreement that cannabis use
is associated with worsened outcome for schizophrenia, there is little consensus
as to the links between cannabis use and specific symptomatology. In the current
chapter we propose that the use of network models may further aid in disentangling
these associations. We analyzed data from N=1128 patients diagnosed with a
nonaffective psychotic disorders. We constructed three network models, each
consisting of the 30 symptoms as measured by the Positive and Negative Syndrome
Scale (PANSS), alongside three types of cannabis use measures (current cannabis
use, intensity of cannabis use in the past 12 months, and intensity of cannabis use
in the past 3 years). Further, to investigate potential causal directions between
common confounders, cannabis use, and other related factors, we estimated a
directed network structure. Our findings indicate that positive psychotic symptoms
and depression may be exacerbated by cannabis use, while cannabis use may aid in
reducing negative and general psychopathology symptoms. Further, we identified
longer intensity of cannabis use to potentially lead to smoking, suggesting nicotine
dependence may be an important health consequence of early frequent cannabis
use. Focus on negative and general psychopathological symptoms may be beneficial
in reducing substance use and its potential negative impact on symptomatology.
Overall, long-term retrospective measures of cannabis use may generally be more
informative of symptom course than current or short-term measures.
Page | 84
6.1 Introduction
Cannabis is one of the most widely used substances by individuals suffering from
a psychotic disorder, with prevalence rates estimated as high as 42.1% for lifetime
use (Green et al., 2005). To date, extensive research indicates a strong association
between cannabis use and psychotic symptomatology (for an overview see Dekker
et al., 2009) with several reports suggesting the presence of more positive symptoms
in patients (D’Souza et al., 2005; Hasan et al., 2020; Negrete et al., 1986) and
first-episode psychosis subjects (Archie & Gyomorey, 2009; Quattrone et al., 2020)
who use cannabis, as well as fewer negative symptoms (Archie & Gyomorey, 2009;
Bersani et al., 2002; D’Souza et al., 2005; Peralta & Cuesta, 1992). Other studies,
however, report worsened or unaffected negative symptoms (Bersani et al., 2002)
or even no associations between cannabis use and symptomatology (Kovasznay et
al., 1993). Reasons for these inconsistencies may lie in study design, heterogeneous
samples, confounder inclusion (e.g., concurrent tobacco smoking, gender, prior
history of cannabis use; Myles et al., 2012; van Gastel et al., 2013), or focus on one
particular domain of interest. In addition, current measures of cannabis use and
abuse vary vastly, ranging from measures of current use through urine testing, to
retrospective short or long-term reports (López-Pelayo et al., 2015). Nevertheless,
while findings in the field are not always consistent, there is ample evidence and
a general agreement that cannabis use is associated with a worsened outcome for
schizophrenia (Hasan et al., 2020; Seddon et al., 2016).
In the past decade, an increasing number of research studies in the fields of
psychology and psychiatry have argued in favor of studying mental disorders
as complex systems (Borsboom, 2017; Borsboom & Cramer, 2013; Isvoranu,
Boyette, et al., 2020) and novel methodological developments are fast advancing
(Epskamp, van Borkulo, et al., 2018; Epskamp, 2020a; Isvoranu, Epskamp, et al.,
2020). Network models have been at the forefront of such developments, and have
fast grown popular across a wide array of disciplines (Fried, van Borkulo, Cramer,
et al., 2016; Robinaugh, Hoekstra, et al., 2020). While initially used mainly in
the study of symptom interactions and development, more recently researchers
have argued that network models may be useful tools in the study of interactions
between genetic and environmental risk factors and symptomatology (Isvoranu et
al., 2017, 2019; see Chapter 5 and Chapter 8). In this way, it becomes possible to
highlight a wide array of unique associations between variables of interest, while
controlling for all other factors included in the network structure (Epskamp,
Page | 85
Waldorp, et al., 2018). Aligned with this reasoning, prior investigations using
general population data identified associations between cannabis use and overall
measures of psychopathology, including depression, anxiety, hostility and obsessive
compulsive disorder, as well as more connected network structures in subjects with
higher cannabis exposure (Isvoranu et al., 2016; see Chapter 4). Of note, these
investigations relied on reports from healthy subjects and the focus was not on
psychotic symptomatology, though a composite measure of psychosis was included
in the network structure.
Building on these developments, the current work proposes that the use of
network models may further aid in disentangling potential associations between
cannabis use and psychotic symptomatology. Our study aims to expand on previous
research findings, by focusing on a wide array of commonly measured symptoms
in patients with psychotic disorders. Overall, our aim is two-fold: 1) investigate
associations between short- and long-term use of cannabis and symptomatology,
in a sample of patients diagnosed with a non-affective psychotic disorder; and 2)
investigate potential causal links between common confounders, other related
factors, and cannabis use.
6.2 Methods
6.2.1 Participants
We analyzed data from the longitudinal observational cohort study Genetic Risk
and Outcome of Psychosis Project (GROUP), release database 7.0. The current study
included the patient sample only and the second wave of measurement, resulting
in a sample of N=1128. The patients were recruited from 36 mental health care
institutions across The Netherlands and Belgium. Inclusion criteria were age 16 to
50 years, DSM (American Psychiatric Association, 2000) criteria for a non-affective
psychotic disorder, and an estimated level of intelligence quotient above 70. For
further details on the GROUP sample please refer to Korver and colleagues (2012).
We used the second wave data from the Positive and Negative Syndrome Scale
(PANSS; Kay et al., 1987), an interview-rated measure of psychotic symptom
severity, consisting of 30 items that measure positive symptoms, negative symptoms,
and general psychopathology. The PANSS is scored on a 7-point Likert-type scale,
ranging from 1 (absent) to 7 (very severe).
Page | 86
6.2.3 Cannabis use
We used three different measures of cannabis use. First, we used a measure of
current cannabis use, as measured by a urine test. Urine was screened by means of
immunoassays for cannabis, with a cut-off of 50ng/ml. Given the relatively high
cut-off, a detection window of one month was used (Musshoff & Madea, 2006).
This resulted in a binary variable measured as ‘positive’ or ‘negative’. Further, the
special Substance Abused Module of the Composite International Diagnostic
Interview (CIDI; Kessler & Üstün, 2004) was used to cover tobacco and drug
abuse in detail, allowing for the assessment of cannabis use intensity over the past
12 months and over the past 3 years. These intensity measures, while originally
measured on a 4-point scale, were binarized due to requirements of the statistical
model used.

All analyses were performed using the R-statistical software, version 4.0.2. The
networks were constructed using the R package mgm version 1.2.10 (Haslbeck &
Waldorp, 2020) and the R package bnlearn version 4.5 (Scutari, 2010), and were
visualized using the R-package qgraph version 1.6.6 (Epskamp et al., 2012). We
followed a similar analysis pattern as Liu and colleagues (2020).
Markov Random Fields

We constructed three network models, each consisting of the 30 symptoms as
measured by the PANSS, alongside each one of the cannabis measures. In order
to account for the mixed type of data used in this study (i.e., the symptoms were
measured continuously, while the cannabis measures were binary) we fitted a
Mixed Graphical Model (Haslbeck & Waldorp, 2020) to our data. We used an
algorithm which includes an L1-penalty to obtain a sparse estimate and selected
the regularization parameter lambda using cross validation (CV) and employed
the OR-rule for selecting edges to be included (Haslbeck & Waldorp, 2020). As
the dataset featured missing datapoints, which the mgm package cannot handle,
we used the R package mice (van Buuren & Groothuis-Oudshoorn, 2011), version
3.9.0 to impute missing data using multiple imputation. In addition, because CV
can be sensitive to the pseudorandom number generator used prior to the sample
shuffle, we imputed the data 10 times, and only retained edges in our final network
that were included in the estimated networks based on at least 9 out of 10 imputed
datasets. Given that we investigate between-domain links, this estimation approach
Page | 87
is likely to ensure that the number of false-positives is reduced, while also retaining
a good edge discovery rate. To check for accuracy of estimations, we performed
1000 non-parametric bootstraps using the R-package bootnet version 1.4.3, as
recommended in the network psychometrics literature (Epskamp, Borsboom,
et al., 2017). We included the multiple imputation strategy in the bootstrapping
procedure. Specifically, to investigate the stability of the edge weights, participants
were randomly resampled 1000 times, and the bootstrapped confidence intervals
(CIs) of the edge weights were estimated.
All PANSS and cannabis use measure were represented as nodes in the network
structure. An edge between any two nodes indicates that the two variables were not
independent after conditioning on all other variables in the dataset. Blue edges in
the network indicate positive associations, and red edges negative associations. The
wider and more saturated the edge, the stronger the association. The layout of the
networks was based on a circular layout, with a manual placement of the cannabis-
related measure in the center of the network. All edges between cannabis measures
and symptomatology were manually unfaded, while the remaining edges are further
faded, for a better visualization of the central research question.
To exploratively investigate how much influence one can have on one node when
intervening on all its neighbors, we computed predictability measures. Predictability
can be defined as the shared variance of each node with all of its neighbors (Haslbeck
& Fried, 2017). Predictability was represented in every network by the outer circle
present in each node, with higher predictability resulting in a more filled up circle.
Directed Acyclic Graphs

To investigate potential causal directions between common confounders, other
related factors and symptomatology, we estimated a directed acyclic graph (DAG;
Kalisch & Bühlmann, 2007; Pearl, 2000), using the stable PC-algorithm (Colombo
& Maathuis, 2015; Kalisch et al., 2012) implemented in the bnlearn R package
(Scutari, 2010). The bnlearn package is capable of handling different types of
variables, including ordered categorical variables. We included the variables gender
(categorical), antipsychotic use (categorical), smoking (categorical), cannabis current
use (categorical), cannabis past 12 months intensity of use (ordinal), and cannabis
past 3 years intensity of use (ordinal). As the DAG estimation procedure cannot
handle missing data, the same approach as described above was used to handle the
missing data. Specifically, we imputed 10 datasets and we only retained the edges
that were included at least 9 times. The pc-algorithm returns both directed edges
Page | 88
and undirected edges (the latter when the algorithm cannot distinguish between
direction of effect). To check for stability in the estimation, we performed 1,000
non-parametric bootstraps, and investigated the proportion of times each directed
or undirected edge was included in the network.
Of note, DAG estimation is typically not recommended for network analysis
of symptoms (Epskamp, Waldorp, et al., 2018), as the assumption of acyclicity is
often not tenable in the context of symptomatology. As such, we did not include
the symptoms as part of the DAG. However, in the case of background variables
and variables with more specific time frames (such as the cannabis measures),
acyclicity is much more likely to hold. To aid the algorithm, we blacklisted several
edges from being included in the model: (1) all edges toward gender, as gender
can be regarded as exogenous variables that cannot be caused by other variables
in the analysis, (2) edges from current and recent cannabis smoking, cigarette
smoking, and drug use to intensity of cannabis use in the past, as smoking or
drug use now cannot cause cannabis use in the past, (3) edges from remission to
other variables in the network, as remission at a later time point cannot cause
anything in the past. No edges were whitelisted, meaning that all edges included
in the model were discovered by the algorithm.
6.3 Results
The study sample at baseline consisted of 1128 subjects. Table 6.1 includes
sociodemographic and clinical characteristics of the sample. Table C1 in
Supplement C includes the mean and standard deviations of all items included
in the network structure.
Table 6.1 Sociodemographic and clinical characteristics (N=1128)

Variable Mean (SD)
Gender (% male) 76.1
Age 30.6 (7.2)
Ethnicity (% Caucasian) 80.6
Estimated IQ 98.8 (16.6)
Age on onset first psychotic disorder 22.8 (7.04)
DSM-diagnosis of psychotic disorder (%)
Schizophrenia 63.9
Page | 89
Table 6.1Continued.
Variable Mean (SD)

Schizophreniform 3.0
Schizoaffective disorder 17.4
Delusional disorder 1.6
Psychotic disorder NOS 10
Other psychotic disorder 4.0
*Estimated with 4 subtests of the WAIS-III (van der Heijden et al., 2013)
6.3.1 Markov Random Fields

The estimated undirected network models are shown in Figure 6.1, with panels A,
B, and C highlighting associations between symptomatology and current cannabis
use, intensity of cannabis use in the past 12 months, and intensity of cannabis use
in the past 3 years, respectively.
The results of the three network structures generally align well, indicating a
dose-increase response pattern (i.e., many of the edges within the current cannabis
use network are a subset of the edges in the network highlighting intensity of
cannabis use in the past 12 months, which are in turn generally a subset of the
edges in the network highlighting intensity of cannabis use in the past 3 years).
Overall, while both positive and negative associations are identified in all network
structures, positive psychotic symptoms are mainly positively associated with
cannabis use, negative symptoms are negatively associated to cannabis use, while
general psychopathology symptoms show both positive and negative associations
to cannabis use.
Specifically, we identified positive associations between current cannabis use
and positive psychotic symptoms grandiosity (P5) and hostility (P7), as well as
general psychopathology symptom depression (GP6) and negative associations
between current cannabis use and negative symptoms difficulty in abstract thinking
(N5), stereotyped thinking (N7), as well as general psychopathology symptom
uncooperativeness (GP8). Further, intensity of cannabis use in the past 12 months was
positively associated with positive psychotic symptoms conceptual disorganization
(P2) and grandiosity (P5), as well as with general psychopathology symptoms
depression (GP6), poor judgement (GP12), and poor impulse control (GP14) and
negatively associated with positive psychotic symptom hallucinations, negative
psychotic symptom stereotyped thinking (N7), as well as general psychopathology
Page | 90
symptoms uncooperativeness (GP8) and poor attention (GP11). Finally, intensity
of cannabis use in the past 3 years was positively associated with positive psychotic
symptoms conceptual disorganization (P2), grandiosity (P5), and paranoia (P6), as
well as with general psychopathology symptoms depression (GP6), poor judgement
(GP12), and poor impulse control (GP14) and negatively associated with negative
symptom stereotyped thinking (N7), as well as general psychopathology symptoms
anxiety (GP2), mannerism and posturing (GP5), motor retardation (GP7), and
disturbed willpower (GP13).
The stability analyses (see Appendix C) of the MRFs suggest some caution should
be taken when interpreting the network structure, as the confidence intervals are
often wider than desired. In addition, while many of the edges were identified as
absent in the sample, these were more often present in the bootstrapped analyses
as either positive or negative edges. Generally, these effects were however weak (see
Figures C1, C2, and C3). In addition, Figure C4 highlights the edges that are present
in over 60% of the bootstraps for each network structure, which we regard as the
most stable edges. Thus, we focus our interpretation below mainly on these stable
edges. For the remainder of the edges, while we still briefly discuss the results, we
recommend caution in interpretation and replication of results.
6.3.2 Predictability
Predictability results identified predictability of all types of cannabis use to be the
lowest in all network structures. This indicates that the variance in cannabis use
was not well explained by the symptoms. Most symptoms showed much higher
predictability values, suggesting variance in the symptoms can often be explained
by other symptoms and cannabis use.
Page | 91
Page | 92
c) Intensity of Cannabis Use in the Past 3 Years Cannabis
a) Current Cannabis Use b) Intensity of Cannabis Use in the Past 12 Months
C: Cannabis Use
Positive Symptoms
GP16 P1 GP16 P1 P2 GP16 P1 P2
P2 P1: Delusions
GP15 P3 GP15 P3 GP15 P3 P2: Conceptual Diso
GP14 GP14
P3: Hallucinations
GP14 P4 P4 P4
P4: Hyperactivity
P5: Grandiosity
GP13 P5 GP13 P5 GP13 P5 P6: Paranoia
P7: Hostility
GP12 P6 GP12 P6 GP12 P6
Negative Symptom
N1: Blunted Affect
GP11 P7 GP11 P7 GP11 P7 N2: Emotional Withd
N3: Poor Rapport
N4: Social Withdraw
GP10 N1 GP10 N1 GP10 N1 N5: Difficulty in Abstr
N6: Lack of spontan
C C C N7: Stereotyped Thi
GP9 N2 GP9 N2 GP9 N2
General Psychopat
GP1: Somatic Conc
GP2: Anxiety
GP8 N3 GP8 N3 GP8 N3 GP3: Guilt
GP4: Tension
GP5: Mannerism an
GP6: Depression
GP7: Motor Retardat
GP6 N5 GP6 N5 GP6 N5 GP8: Uncooperative
GP9: Unusual Thou
GP5 N6 GP5 N6 GP5 N6 GP10: Disorientation
GP11: Poor Attentio
GP4 N7 GP4 N7 GP4 N7 GP12: Poor Judgem
GP3 GP1 GP3 GP2 GP1 GP3 GP2 GP1 GP13: Disturbed Will
GP2
GP14: Poor Impulse
GP15: Preoccupatio
GP16: Social Avoida
Figure 6.1 Network representation for a) current cannabis use and symptomatology; b) intensity of cannabis use in the past 12 months, and c) intensity of can-
nabis use in the past 3 years. Blue links indicate positive effects (partial correlations), and red links indicate negative effects. The layout is manually specified, with
the links between cannabis use and symptomatology highlighted.
Negative Symptoms
N1: Blunted Affect
P7 N2: Emotional Withdrawal
N3: Poor Rapport
N4: Social Withdrawal
N1 N5: Difficulty in Abstract Thinking
N6: Lack of spontaneity
C N7: Stereotyped Thinking
N2
Cannabis General Psychopathology
C: Cannabis Use GP1: Somatic Concern
GP2: Anxiety
N3
Positive Symptoms GP3: Guilt
P1: Delusions GP4: Tension
P2: Conceptual Disorganization GP5: Mannerism and Posturing
N4
P3: Hallucinations GP6: Depression
P4: Hyperactivity GP7: Motor Retardation
P5: Grandiosity
N5 GP8: Uncooperativeness
P5 P6: Paranoia GP9: Unusual Thought Content
P7:
N6 Hostility GP10: Disorientation
P6 GP11: Poor Attention
N7 Negative Symptoms GP12: Poor Judgement
GP3 GP2 GP1 N1: Blunted Affect GP13: Disturbed Willpower
P7 N2: Emotional Withdrawal GP14: Poor Impulse Control
N3: Poor Rapport GP15: Preoccupation
N4: Social Withdrawal GP16: Social Avoidance
N1 N5: Difficulty in Abstract Thinking
N6: Lack of spontaneity
N7: Stereotyped Thinking
N2
General Psychopathology
GP1: Somatic Concern
GP2: Anxiety
N3 GP3: Guilt
GP4: Tension
GP5: Mannerism and Posturing
N4
Page | 93
GP6: Depression
GP7: Motor Retardation
N5 GP8: Uncooperativeness
GP9: Unusual Thought Content
GP10: Disorientation
6.3.3 Directed Acyclic Graphs

The estimated DAG structure, in conjunction with inclusion proportions of each
edge is presented in Figure 6.2. First, the model predicted intensity of cannabis use
in the past 12 months may have led to increased intensity of cannabis use in the past 3
years, which in turn may have led to an increase in current cannabis use. In addition,
the model identified that both gender and antipsychotic use may have led to remission
in patients. Finally, the bootstrap analysis showed that intensity of cannabis use in
the past 3 years and gender were directly linked to current smoking in 64% and 50%
of the bootstraps respectively.
Anti−
psychotics 0.01
0.09
0 Smoking
0.88
0.07
Remission
0.06
0.06
0.54
0.01
0.68 0.01 0.02
0.06
0
0 0.26
0 00.02 0.3
Gender 0.09 0.64
0.09
0.07
Cannabis
(Urine)
0.01
0.2 1
0.06
Cannabis
(12 Months)
1
Cannabis
(3 Years)
Figure 6.2 Directed Acyclic Graph (DAG) estimation results, highlighting potential causal links be-
tween different types of cannabis use, common confounders, and other related factors. The network
shows the inclusion proportion of each edge based on 1000 bootstrap replications.
Page | 94
6.4 Discussion
The current study was, to our knowledge, the first study to 1) examine links
between short- and long-term use of cannabis and symptomatology and 2) potential
causal links between common confounders, related factors, and cannabis use,
in a large sample of patients diagnosed with a non-affective psychotic disorder.
Overall, we found both positive and negative associations between cannabis use
and symptomatology, as well as a dose-increase response pattern for cannabis
use intensity, with the number of associations increasing as the retrospective
measurement time frame of intensity increases. Predictability findings indicated
that symptoms explain a very low percentage of variance in cannabis use, but
considerable variance in the symptoms is explained in the network structure. This
could be explained both by other symptoms, as well as by the cannabis use variable
itself. In addition, we identified causal links between past intensity of cannabis use
and current use of cannabis and tobacco, as well as between gender, antipsychotic
medication, and remission.
6.4.1 Cannabis use and psychotic symptomatology

In line with previous research (D’Souza et al., 2005; Hasan et al., 2020; Negrete et
al., 1986), we found that both current use of cannabis, as well as higher intensity
of use in the past one and three years were positively linked to positive psychotic
symptoms. Although we did find one negative association between intensity of
cannabis use in the past one year and hallucinations, this negative link was not
identified above the threshold in our bootstrap analysis. Nonetheless, although not
a common finding, three studies investigating self-report reasons of patients using
cannabis listed ‘to decreases voices’ and ‘to decrease hallucinations’ as frequently
reported, ranging from 11% (Schofield et al., 2006) to 40% (Addington & Duchak,
1997; Goswami et al., 2003). Apart from this negative link, current cannabis use
was positively linked to hostility and grandiosity, and higher intensities of use in
the past years were further associated to conceptual disorganization and paranoia.
Our results support the notion that cannabis use is involved in the clinical course
of schizophrenia and may precipitate positive psychotic symptoms (Bersani et al.,
2002; Katz et al., 2010), as well as align with studies showing hostile behavior in
cannabis users (Ansell et al., 2015). Further, in self-report reasons for cannabis use,
a large proportion of patients report to be aware that cannabis negatively affected
positive symptoms (Dekker et al., 2009).
Page | 95
While the links between cannabis use and positive psychotic symptomatology were
mainly positive, cannabis use was negatively associated with negative symptoms,
especially with stereotyped thinking across all network structures. These findings
align with previous research identifying fewer negative symptoms in patients and
in subjects with first episode psychosis (Archie & Gyomorey, 2009; Bersani et
al., 2002; D’Souza et al., 2005; Peralta & Cuesta, 1992). Links between general
psychopathology and cannabis use showed both positive and negative associations
and a much clearer dose-increase response pattern. Of note, current cannabis
use was positively and steadily linked to depression across all intensity measures.
Recent meta-analyses on the topic found some evidence for an increased risk for
depression in cannabis users (Moore et al., 2007), and especially an increased risk
for developing depressive disorders for heavy cannabis users (Lev-Ran et al., 2014).
Further, a recent meta-analysis of subjects at enhanced clinical risk of psychosis
identified high prevalence of depressive disorder, with about 40% of at-risk subjects
having a comorbid diagnosis of depressive disorder (Fusar-Poli et al., 2014).
Comorbid diagnoses were in turn associated with higher disorganized behavior,
apathy, and impaired global functioning. Depressive symptoms are also common
in schizophrenia patients (Sim et al., 2004) and are associated with increased
symptomatology, as well as cognitive impairment (Dai et al., 2018). Our results
suggest that in patients suffering from a psychotic disorder, cannabis may further
exacerbate the presence of depressive symptoms and support previous findings
in the field highlighting the importance of affective symptoms in schizophrenia
(Boyette et al., 2020; Isvoranu et al., 2017; Myin-Germeys & van Os, 2007). A
loop in which symptoms lead to increased usage as a coping mechanism, but at
the same time cannabis use leads to increase in other symptoms may be plausible.
Other positive links were identified between higher intensity of use in the past 12
months and three years and poor judgement and poor impulse control, with the
latter however not identified in the bootstrap threshold. These links align with
findings showing an increased impulsivity and hostile behavior in cannabis users,
suggesting cannabis may link to more reckless behavior and impulsive behavior
(Ansell et al., 2015).
Finally and of note, intensity of cannabis use in the past three years was negatively
linked to anxiety, mannerism and posturing, motor retardation, and disturbed
willpower, with the first three present in a high proportion in the bootstrap analyses.
These findings align with the self-medication hypothesis (Khantzian, 1985), as well as
with self-reports from patients, for whom reason to use cannabis include ‘to relax’ and
Page | 96
to a lesser extent ‘to decrease slowed-down feelings caused by medication’ (Dekker et
al., 2009). Common side effects of medication include unrest and agitation, difficulty
sleeping, and muscle tension (Schofield et al., 2006). Fowler and colleagues (1998)
found that only cannabis abusers reported cannabis use for illness- and medication-
related reasons. Current results further suggest that longer intensity of use may be
linked to decreased side-effects of medication, as the above negative links emerged
only between increased intensity of cannabis use in the past three years, and not in
the remaining two network structures. Overall, longer term retrospective measures
of cannabis use may generally be more informative of symptom course than current
or short-term measures.
6.4.2 Cannabis use and confounding factors

Our secondary analysis identified, as expected, that retrospective measurement of
cannabis use intensity was linked to current cannabis use, suggesting that subjects
who used cannabis in the past are also more likely to be current cannabis users. Of
note, we recommend interpreting this as a part-whole relationship rather than a
causal one, as one cannot separately manipulate these factors (i.e., manipulating
past year use will, in part, also manipulate past three-year use, so these variables
are not logically independent). In addition, we found that intensity of cannabis
use in the past three years was directly linked to smoking, but not the other way
around, indicating that intensely smoking cannabis in the past 3 years led to
cigarette smoking. Aligned with current findings, Patton and colleagues (2005)
identified that weekly (or more frequent) cannabis use during the teens and young
adulthood was associated with an increased risk of tobacco use and progression
to nicotine dependence and argued that if this effect is identified to be causal,
nicotine dependence may be an important health consequence of early frequent
cannabis use.
Further, we identified directed links between gender and smoking, with males
being smokers more often than females. Gender was also linked to remission, with
females being more likely to remit (Ochoa et al., 2012; Thorup et al., 2014). In
addition, a weak link (only present in about 28% of the bootstraps) identified
current cannabis use to be linked to remission and further investigation indicated
that this association is negative, with cannabis smokers being less likely to enter
remission. This link is weak and not always present across bootstraps and as
such we recommend replication and caution in interpretation. Finally, we found
antipsychotic use to be linked to remission, but any clear statements regarding
Page | 97
this associations are not feasible. Only 35 patients did not receive antipsychotic
medication and further investigation showed that among patients who did not
receive antipsychotics, a much higher number were likely to be in remission. This
effect could however simply be a confound with a healthy state, indicating that
healthy patients who did not need treatment were also more likely to be the ones
in remission. We thus advise caution in interpreting this result.
6.4.3 Limitations
The current study should be considered in light of several limitations. First, the
multiple imputation strategy used for imputing was only used in one study before
(D. Liu et al., 2020) and is in need of further validation. However, exclusion of
missing data through listwise deletion would have resulted in loss of almost half of
the dataset and as such important and relevant information. In addition, based on
recent research (Madley-Dowd et al., 2019), unbiased results when data imputation
is used can be obtained even with large proportions of missing data (up to 90%
missing) and as such retaining all information and choosing data imputation may
be preferred over discarding missing data. Second, we used an estimation method
focused on the discovery of edges and as such lower specificity of this method may
be problematic (i.e., some edges included in the model may be more prone to being
false positive results). However, to account for this, we only retained edges in our
final network that were included in the estimated networks based on at least 9 out
of 10 imputed datasets. Third, the stability results indicated caution is needed in
the interpretation of the network structures and the bootstrap results indicated that
more edges could have been identified (other edges not identified in the sample were
sometimes–though below the 60% threshold–present in the bootstrap analyses),
and as such we recommend caution in interpreting missing edges in the network
structure. Of note, the current chapter doesn’t discuss differences in edge strength
between edges and we have focused our discussion on edges we identified in at
least 60% of the bootstraps and can be considered stable, to ensure accurate results.
Fourth, DAG methodology features strong assumptions unlikely to hold, such as
the acyclicity assumption, though we have focused our research on variables for
which this assumption was more likely to hold and did not include symptoms as
nodes in the DAG. Finally, all analyses were exploratory, and should be viewed as
hypothesis generating in need of further experimental testing.
Page | 98
To conclude, the current chapter aimed to identify links between short- and
long-term use of cannabis and symptomatology, as well as potential causal links
between common confounders, cannabis use, and other related factors. Our main
findings indicate that while cannabis use may aid in reducing negative and general
psychopathology symptoms–especially longer intensity of use may be linked to
decreased side-effects of medication–positive psychotic symptoms and depression
may be exacerbated by cannabis use. While the presence of negative links partly
supports the self-medication hypothesis, cannabis use may nonetheless worsen
the course of the disorder. Overall, longer term retrospective measures of cannabis
use may generally be more informative of symptom course than current or short-
term measures.
Finally, we identified longer intensity of cannabis use to be linked to smoking,
suggesting nicotine dependence may be an important health consequence of early
frequent cannabis use. In addition, we found a weak but, if established, potential
important link between current cannabis use and remission, with a higher
proportion of non-cannabis smokers remitting at a later time point. Treatment
courses focused on negative and general psychopathological symptoms may be
beneficial in reducing substance use and thus its potential negative impact on
positive and depressive symptomatology, though experimental studies are needed
before such clinical implementation.
Page | 99
From Speech Illusions to
7
Onset of Psychotic Disorder

Boyette, L. L.*, Isvoranu, A. M.*, Schirmbeck, F., Velthorst, E., Simons, C. J.,
Barrantes-Vidal, N., ... & EU-GEI High Risk Study (2020). From Speech Illusions
to Onset of Psychotic Disorder: Applying Network Analysis to an Experimental
Measure of Aberrant Experiences. Schizophrenia Bulletin Open.
*indicates shared first authorship

Chapter 7 | From Speech Illusions to Onset of Psychotic Disorder
Abstract
Aberrant perceptional experiences are a potential early marker of psychosis
development. Earlier studies have found experimentally assessed speech illusions
to be associated with positive symptoms in patients with psychotic disorders, but
findings for attenuated symptoms in individuals without psychotic disorders
have been inconsistent, with the role of affect also being unclear. The aim of
this chapter was to use the network approach to investigate how speech illusions
relate to individual symptoms and onset of a psychotic disorder. We estimated
a network model based on data from 289 Clinical High-Risk (CHR) subjects,
participating in the EU-GEI project. The network structure depicts statistical
associations between (affective and all) speech illusions, cross-sectional individual
attenuated positive and affective symptoms and transition to psychotic disorder,
after conditioning on all other variables in the network. Affective, not all, speech
illusions were found to be directly, albeit weakly, associated with hallucinatory
experiences. Hallucinatory experiences, in turn, were associated with delusional
ideation. Bizarre behavior was the only symptom in the network steadily predictive
of transition. Affective symptoms were highly interrelated, with depression showing
the highest overall strength of connections to and predictability by other symptoms.
Both speech illusions and transition showed low overall predictability by symptoms.
Our findings suggest that experimentally assessed speech illusions are not a mere
consequence of psychotic symptoms or disorder, but that their single assessment is
likely not useful for assessing transition risk.
Page | 102
7.1 Introduction
One of the oldest theories of delusion formation states that delusions often arise
in a secondary fashion, in an attempt to explain unusual experiences, which may
consist of an external event, such as a social encounter or an internal experience,
such as a sensory perception (James, 1890; Maher, 1988; Reed, 1972). According
to the aberrant salience theory (Kapur, 2003), these experiences are filled with an
augmented sense of significance–i.e., are perceived as unusual–due to a dysregulated
hyperdopaminergic state. Kapur (2003) conceptualizes hallucinations as a direct
experience of the aberrant salience of internal representations, and delusions
as a cognitive effort to make sense of one’s experiences. Findings from large
epidemiological studies in non-clinical samples support the view that hallucinatory
experiences (subthreshold hallucinations) often precede delusional ideation
(subthreshold delusions), and that their co-occurrence enhances the risk of transition
to psychotic disorder (Krabbendam et al., 2004; Smeets, Lataster, Dominguez, et
al., 2012; Smeets, Lataster, van Winkel, et al., 2012), possibly particularly in the
context of affective dysregulation (Hanssen et al., 2005; Krabbendam et al., 2005),
although findings for the latter are mixed (Addington et al., 2017; Fusar-Poli et al.,
2014; Lim et al., 2015). In an interview study of patients with psychotic disorders,
about 80% of the participants reported an onset of current delusions based on
aberrant perceptual experiences, including hallucinatory experiences (Freeman et
al., 2004).
Aberrant perceptual experiences can exist in all sensory modalities and in the
context of an external stimulus (referred to as an illusion) or without an external
stimulus (a hallucinatory experience, pseudo-hallucination or hallucination). A
potential cognitive mechanism of aberrant perceptual experiences is that they are
driven by a disproportionate influence of sensory expectations, i.e., ‘top-down
processing’, on sensory input, i.e., ‘bottom-up’ processing (Aleman et al., 2003;
Hugdahl, 2009). Empirical findings for this mechanism are inconsistent (Daalman
et al., 2012; Vercammen & Aleman, 2010). There is some empirical support that
experimentally assessed auditory illusions of speech (hereafter referred to as speech
illusions) are a marker of psychosis liability: Hoffman and colleagues (2007) found
that the degree of speech illusions, here operationalized as the number of words
falsely perceived in the external stimulus of incoherent multi-speaker ‘babble’,
predicted transition to psychotic disorder in individuals at high risk for psychosis.
As far as we are aware, there have been no studies aimed to replicate this finding.
Page | 103
Galdos et al. (2011), however, extended the aforementioned research and theories
in their development of the White Noise Task: an experimental task to assess
speech illusions, that intensifies the potential influence of top-down processing
by priming subjects that speech might be expected while presenting different
volume levels of speech on a white noise background. In order to study affectively
salient meaning, the task includes assessment of perceived affective content. Two
studies to date have demonstrated that speech illusions assessed with the White
Noise Task are associated with positive (and not negative) symptoms in patients
with psychotic disorders (Catalan et al., 2014; Galdos et al., 2011). Patients had
higher rates of speech illusions compared to their siblings and healthy controls,
particularly for speech illusions with affective content (hereafter referred to
as affective speech illusions; Galdos et al., 2011; Schepers et al., 2019). Galdos
and colleagues (2011) also found an association between speech illusions and
attenuated positive (and not negative) symptoms in a non-clinical sample. Other
studies in non-clinical samples, however, have not found an association between
attenuated positive symptoms and speech illusions (Catalan et al., 2014; Schepers,
Lousberg, et al., 2019; Schepers, van Os, et al., 2019), a negative association (Pries
et al., 2017), or only associations between speech illusions and some symptoms:
hallucinatory experiences and overall negative affect and not delusional ideation
(Rimvall et al., 2016). In a recent study, the data of Catalan and colleagues (2014)
was combined with more participants and re-analyzed. The authors found an
association between speech illusions and attenuated positive symptoms in healthy
controls, but this was not statistically significant after correcting for age, gender
and cognitive ability (de Artaza et al., 2018).
The inconsistent findings with regard to clinical and non-clinical samples and
for attenuated positive symptoms indicate that further research is required to
examine whether White Noise Task speech illusions are indicative of psychosis
proneness, as opposed to, for instance, resulting from consequences of psychotic
illness. For this purpose, it is desirable to use a similar design to Hoffman et al
(2007) and to include onset of a psychotic disorder as a main outcome of interest.
Since the vast majority of psychotic experiences in the general population are
transitory (Kaymaz et al., 2012), it is beneficial to study transition in individuals
at high risk for psychosis. The Ultra-/ Clinical High-Risk (UHR/ CHR) and At
Risk Mental State (ARMS) constructs have been developed for this purpose (Yung
et al., 1996).
Page | 104
Further, apart from Rimvall and colleagues (2016) who examined hallucinatory
experiences and delusional ideation separately, previous studies have all
used composite measures of symptoms. According to the network theory of
psychopathology, disorders may arise from individual symptom interactions,
which can be encoded in a network structure (Borsboom, 2017; Borsboom &
Cramer, 2013). Within the network framework, symptoms are no longer regarded
as merely indicators of a latent disorder, but rather interactions among symptoms
become of central interest. In addition, to allow for the visual representation of
these interactions, network models rely on advanced statistical techniques, such
as model selection routines (employed to find the best fitting model underlying
the data; Epskamp & Fried, 2018; Haslbeck & Waldorp, 2020; Williams et al.,
2019), and regularization routines (employed to estimate interpretable and sparse
models; Epskamp & Fried, 2018; van Borkulo et al., 2014). These routines allow
for highlighting many interactions simultaneously, while controlling for spurious
associations that may result from sampling error. For further details of the network
framework in the context of psychotic symptomatology, please refer to Chapter 2.
The network approach could thus provide new insights into how speech illusions
relate to individual symptoms and shed light on the complicated interplay between
individual symptoms preceding the onset of psychotic disorder.
Consequently, the current study aimed to use the network approach to examine
how experimentally assessed (affective and all) speech illusions relate to individual
attenuated positive and affective symptoms, their interrelations and onset of a
psychotic disorder in a Clinical High-Risk sample. The hypothesis that speech
illusions are indicative of psychosis proneness led us to expect a direct and strong
relation between speech illusions and hallucinatory experiences. We expected
delusional ideation to be intermediate in the relation between hallucinatory
experiences and transition to psychotic disorder. Relations with other attenuated
positive and affective symptoms were examined in an exploratory manner.
7.2 Methods
7.2.1 Participants and procedure
Data pertain to the Clinical High-Risk (CHR) sample participating in the
GxE Prodrome study of the European network of national networks studying
gene-environment interactions in schizophrenia (EU-GEI project; van Os et al.,
2014). The EU-GEI project aimed to identify the interactive genetic, clinical
Page | 105
and environmental determinants involved in the development, severity and

outcome of schizophrenia. EU-GEI Prodrome partners include the UK, the
Netherlands, Denmark, Austria, Switzerland, Germany, France and Spain, as
well as partnerships outside Europe with Australia and Brazil. All participating
sites obtained approval from their associated Medical Ethical Committees. The
EU-GEI project was conducted in accordance with the Declaration of Helsinki
for ethical conduct in research.
7.2.2 Instruments
Clinical symptomatology
Clinical High-Risk (CHR) status and transition to psychotic disorder were assessed
with the Comprehensive Assessment of At Risk Mental State (CAARMS; Yung et
al., 2005). The CAARMS is a valid and reliable instrument for assessing possible
psychotic prodromes and their course (Yung et al., 2005). CHR status is defined as
presence of attenuated positive symptoms or brief, self-limited psychotic symptoms
(BLIPS), genetic vulnerability for psychosis and persistent low or recently declined
functioning (Yung et al., 2005). Transition to clinical psychosis was prospectively
assessed as absent/present during 24-month follow-up. Transition was defined as
threshold-level positive symptoms (hallucinations, delusions or thought disorder)
occurring several times a week for a duration of at least one week (Yung et al., 2006).
The CAARMS symptom score levels are based on the level of severity (which may
include level of conviction, distress and impact on behavior), frequency, duration
and recency.
Baseline positive and affective symptoms were assessed with the extended version
of the Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962; Ventura et
al., 1995), a commonly used interview for assessment of psychotic symptom severity.
Positive and affective symptoms were selected for the current analysis based on the
five-factor framework solution reported in the meta-analysis conducted by Dazzi
and colleagues (2016). Positive symptoms within this framework are grandiosity,
suspiciousness, hallucinations, unusual thought content and bizarre behavior. Affective
symptoms are somatic concern, anxiety, depression, suicidality and guilt. The BPRS
items were scored on a 1 (absent) to 7 (extremely severe) scale by trained assessors.
Speech illusions
The White Noise Task (Galdos et al., 2011) is an experimental task for the illusion
of speech in white noise. Participants, who are positioned behind a laptop and wear
Page | 106
headphones, are presented with 75 sound fragments in random order consisting
of three types of stimuli composed of 25 sound fragments each: white noise only,
white noise and clearly audible neutral speech, white noise and barely audible
neutral speech. Subjects are asked to choose from five options: (1) hearing a positive
voice, (2) hearing a negative voice, (3) hearing a neutral voice, (4) no speech heard,
(5) heard speech but unsure whether the voice was positive, negative or neutral.
The rate of hearing a voice in the white noise only condition is the variable of
interest. Following the White Noise Task developers, Galdos and colleagues
(2011), dichotomous variables (present or absent) for speech illusions with all and
with affective content were created. Presence of ‘all’ speech illusions according to
Galdos et al (2011) indicated a positive response to option 1, 2 or 3. Catalan and
colleagues (2014) used an alternative operationalization for ‘all’ speech illusions,
indicating at least two positive responses to option 1, 2, 3, or 5. Presence of affective
speech illusions indicated a positive response to option 1 or 2 in both methods
(operationalizations confirmed through personal communication with one of the
Catalan et al. authors, JvO).

All analyses were performed using the R-statistical software, version 3.6.1. (R Core
Team, 2015). The network structure was constructed using the R-package mgm,
version 1.2.6. (Haslbeck & Waldorp, 2020) and visualized using the R-package
qgraph, version 1.6.4. (Epskamp et al., 2012).
We constructed two network models consisting of (1) all speech illusions (Galdos
method; Galdos et al., 2011) and (2) affective speech illusions, baseline attenuated
positive and affective symptoms as described above and prospective transition to
psychotic disorder. Given that variables in a network can be interpreted in a predictive
manner (Epskamp, Waldorp, et al., 2018), it is possible to see which baseline variable(s)
is/are likely to predict which subjects transition to clinical psychosis at a later stage
(i.e., inter-individual prediction). All items were represented as nodes. An edge
between any two nodes indicates that these two variables are statistically dependent
on each other after conditioning on all other variables in the network structure. The
wider and more saturated the edge, the stronger the association. Blue edges represent
positive associations, red edges represent negative associations.
In order to account for the mixed type of data used in this study (i.e., both
continuous and binary), we fitted a Mixed Graphical Model (Haslbeck & Waldorp,
2020) to our data. As the mgm software cannot currently handle missing data,
Page | 107
we only used cases with complete data. In addition, as the current data were not
univariate normally distributed, a nonparanormal transformation (H. Liu et al.,
2009) to relax the normality assumption was applied to the continuous variables
prior to constructing the networks. To obtain a sparse estimate, we used an
algorithm which includes an L1-penalty; we selected the regularization parameter
lambda using cross validation (CV). Because CV relies on the random seed and
may be less conservative than other estimation techniques for regularized networks,
we chose a more conservative estimation approach. Specifically, we estimated the
network structure 1000 times using different seeds for the lambda parameter and
only retained the edges that were non-zero in 95% of the cases. Given that we aimed
to investigate between-domain links, this estimation approach is likely to ensure
that the number of false-positives is reduced, hence ensuing more robust results.
While the employed methodology does not allow for the investigation of
the network stability using traditional techniques (Epskamp, Borsboom, et al.,
2017), the method in itself could be thought of being based on bootstrapping.
Nonetheless, to further ensure this, we carried out an additional stability analysis
using nonparametric bootstrap techniques, as described by Epskamp and colleagues
(2017), but designed for the method described above. Details on this analysis are
provided in the Supplement D, Appendix D1. In addition, we employed several
more robustness checks, including constructing (1) an extended network structure
including, in addition to the above symptoms, also cognitive symptoms, (2) a
network plot displaying edge variability across the estimation of the 1000 network
structures using different seeds for the lambda parameter, and (3) a network
structure of all speech illusions, positive and affective symptoms and transition to
clinical psychosis, according to the operationalization of Catalan and colleagues
(2014). Details on these additional checks are available in Supplement D.
The layout used when computing the network was the Fruchterman and
Reingold layout (Fruchterman & Reingold, 1991), which places nodes with stronger
connections in the center. The estimated network was further analyzed by exploring
the strength centrality measure of each node. Node strength is a measure of the
number and the strength of connections within a network structure (Opsahl et al.,
2010) and is generally identified as the most robust centrality measure (Epskamp,
Borsboom, et al., 2017). Finally, to explore how much influence one can have on one
node when intervening on all its neighbors, we computed predictability measures.
Predictability can be defined as the shared variance of each node with all of its
Page | 108
neighbors (Haslbeck & Fried, 2017) and it is represented by the outer circle in each
node: as predictability increases, the circle fills up with color.
7.3 Results
The overall sample consisted of N=345 individuals meeting CHR criteria, of which
N=289 (83.9%) had complete data on all variables and were included in the network
analysis. Table 7.1 shows the sociodemographic characteristics and Table 7.2 shows
the clinical characteristics of the overall and the network samples. Missing data were
found to be unevenly distributed among participants’ age, education levels and site
locations. No differences were found for gender, rates of speech illusions, symptom
levels and rates of transition to clinical psychotic disorder. 308 individuals had data
on both speech illusions and transition to psychotic disorder. Individuals with
all speech illusions according to Galdos et al. (2011) (N=64, 22.1%) transitioned
in 25% (N=16) of the cases and individuals without speech illusions (N=225)
in 16% (N=36) of the cases. Individuals with affective speech illusions (N=35,
12.1%) transitioned in 25.7% (N=9) of the cases and individuals without (N=254)
in 16.9% (N=43) of the cases. Individuals with all speech illusions according to
Catalan et al (2014) (N=137, 47.1%) transitioned in 18.2% (N=25) of the cases and
individuals without (N=152) in 17.8% (N=27) of the cases.
Table 7.1 Sociodemographic characteristics of Clinical High Risk (CHR) subjects

Overall sample Network sample
N=303-345 N=289
Gende % (N males) 53.6% (N=185) 52.9% (N=153)
Age Mean years (SD) 22.4 (4.9) 22.6 (4.9)
Site
Amsterdam, The Netherlands 4.3% (N=15) 4.8% (N=14)
Barcelona, Spain 6.7% (N=23) 8.0% (N=23)
Basel, Switzerland 7.0% (N=24) 7.3% (N=21)
Cologne, Germany 4.3% (N=15) 4.8% (N=14)
Copenhagen, Denmark 5.5% (N=19) 6.6% (N=19)
London, UK 28.4% (N=98) 27.7% (N=80)
Melbourne, Australia 10.4% (N=36) 4.8% (N=14)
Paris, France 5.8% (N=20) 6.9% (N=20)
Sao Paulo, Brazil 5.8% (N=20) 4.8% (N=14)
The Hague, The Netherlands 18.3% (N=63) 21.5% (N=62)
Vienna, Austria 3.5% (N=12) 2.8% (N=8)
Page | 109
Table 7.1Continued.
N=303-345 N=289
Highest level of education N=303 N=262
Compulsory education, no qualification 9.2% (N=28) 6.5% (N=17)
Compulsory education, with qualification 32.7% (N=99) 34.0% (N=89)
Tertiary, first level non-compulsory education 29.7% (N=90) 29.4% (N=77)
Vocational education, completed 14.2% (N=43) 14.9% (N=39)
Higher education, undergraduate 11.9% (N=36) 12.6% (N=33)
Higher education, postgraduate 2.3% (N=7) 2.7% (N=7)
Table 7.2 Clinical characteristics of Clinical High Risk (CHR) subjects

N=303-345 N=289
White Noise Task speech illusions N=308 N=289
All speech illusion (Galdos method)
% (N present) 21.4% (N=66) 22.1% (N=64)
Affective speech illusion
% (N present) 12.0% (N=37) 12.1% (N=35)
All speech illusion (Catalan method)
% (N present) 47.1% (N=145) 47.1% (N=137)
BPRS symptomso, Mean (SD) N=325 N=289
Somatic concern 2.00 (1.38) 1.98 (1.37)
Anxiety 3.45 (1.53) 3.42 (1.53)
Depression 3.64 (1.46) 3.62 (1.43)
Suicidality 2.20 (1.26) 2.18 (1.26)
Guilt 2.01 (1.29) 2.06 (1.29)
Grandiosity 1.38 (0.92) 1.38 (0.90)
Suspiciousness 2.53 (1.43) 2.48 (1.42)
Hallucinations 2.32 (1.39) 2.29 (1.38)
Unusual thought content 2.62 (1.48) 2.62 (1.47)
Bizarre behavior 1.39 (0.87) 1.39 (0.88)
Self-neglect 1.34 (0.74) 1.36 (0.76)
Disorientation 1.18 (0.59) 1.18 (0.61)
Conceptual Disorganization 1.29 (0.67) 1.28 (0.66)
Mannerisms and Posturing 1.05 (0.30) 1.06 (0.31)
Transition to clinical psychosis 18.8% (N=65) 18.0% (N=52)
% (N transition present)
o
Brief Psychiatric Rating Scale (BPRS) scores 1: absent, 2: very mild, 3: mild, 4: moderate, 5: moderate
severe, 6: severe, 7: extremely severe
Page | 110
7.3.1 Network Analysis
Figure 7.1 displays the network structure of all associations between Panel A.
affective speech illusions, Panel B. all speech illusions, attenuated positive and
affective symptoms and transition to clinical psychosis, as well as the associations
between the symptoms themselves, when conditioning on all other variables in the
network structure. Affective speech illusions were found to be directly associated with
hallucinations, but not all speech illusions. Hallucinations in turn were positively
associated with suspiciousness, unusual thought content, depression and suicidality
and negatively with grandiosity and guilt. The affective symptoms were highly
interrelated in both network structures and the overall connections were almost
identical, with the exception of the link to hallucinations. Prospective transition
to clinical psychosis was associated with bizarre behavior.
When exploring centrality measures for both networks (presented in Figure 7.2),
the items with the highest overall number and strength of connections to the other
variables were (in decreasing order) depression, hallucinations (2nd highest for the
affective speech illusions network, 3rd highest for the all speech illusions network),
unusual thought content (2nd highest for the all speech illusions network, 3rd highest
for the affective speech illusions network) and suicidality. Table D1 in the Appendix
includes the raw and standardized centrality values for both networks. Given that
closeness and betweenness are generally less stable centrality measures (Epskamp,
Borsboom, et al., 2017) and the current study is limited in power, we chose not to
interpret closeness and betweenness measures in the current chapter. Nonetheless,
Appendix D2, Figure D2 and Table D2 display the closeness and betweenness of
both network structures, alongside raw and standardized centrality values for
interested readers.
When exploring predictability measures, speech illusions and transition to clinical
psychosis displayed low predictability: none of the neighboring nodes explained a
high amount of shared variance. The items with highest predictability by their
neighboring nodes were (in decreasing order) depression (48%), suicidality (36%),
anxiety (32%), unusual thought content (31%) and hallucinations (26% for the all
speech illusions network, 27% for the affective speech illusions network).
Page | 111
Page | 112
Panel A. Network Affective Speech Illusions Panel B. Network All Speech Illusions
5 5
2 2 Affective Symptoms
● 1: Somatic Concern
3 ● 2: Anxiety
3
● 3: Depression
4 4 ● 4: Suicidality
1
●
12 12 1 5: Guilt
Positive Symptoms
● 6: Grandiosity
● 7: Suspiciousness
● 8: Hallucinations
● 9: Unusual Thought Content
● 10: Bizarre Behavior
8 8
Speech Illusions
10 10 ● 11: Speech Illusions
11 11
Transition
9 9 ● 12: Transition to Clinical Psychosis
6 6
7 7
Maximum: 0.45 Maximum: 0.45
Figure 7.1 A. Network of affective speech illusions, positive and affective symptoms and transition to psychotic disorder; B. Network of all speech illusions
(Galdos method), positive and affective symptoms and transition to psychotic disorder. Symptom groups are differentiated by color and the maximum value is set
to be the same in both networks for comparison purpose.
Strength
Depression ●●
Unusual Thought Content ●●
Hallucinations ● ●
Suicidality ●●
Anxiety ●●
Bizarre Behavior ●
● Type
● Affective Speech Illu
Guilt ●● ● All Speech Illusions
Suspiciousness ●●
●●
Somatic Concern ●●
●●
Grandiosity ●●
● ●
Transition to Clinical Psychosis ●●
Speech Illusions ● ●
−1 0 1 2
Type
● Affective Speech Illusions
● All Speech Illusions
Figure 7.2 Centrality Plot displaying the strength centrality for both the affective speech illusions and
all speech illusions networks. Centrality measures are shown as standardized z-scores and are ordered
by strength. Raw and standardized scores are included in Table D1 in Supplement D.
Page | 113
7.3.2 Supplementary content

In Appendix D1 we present the results of the stability analysis. These indicate
that some caution is necessary when interpreting current results, some of the
bootstrapped intervals being fairly wide. The edge between affective speech
illusions and hallucinations, which is of central interest here, is steadily identified,
though the bootstrapped interval is wide and the edge sometimes switched sign.
Nonetheless, the bootstrap mean is positive and very close to the value identified in
the chapter. In addition, it should be noted that the relation between a binary and
a continuous variable is on a different scale than the relation between continuous
variables themselves, and therefore the finding that these bootstrap intervals may
be wider is not surprising.
Further, in Appendix D3 we present an extended network structure which also
includes four additional cognitive symptoms, as described in the meta-analysis
by Dazzi and colleagues (2016). Within Appendix D4 we included two networks
displaying the edge variability across our chosen estimation procedure for the
affective speech illusions network structure and the affective speech illusions
extended network structure. Markedly, within the extended network structure,
the link between speech illusions and hallucinations did not pass the pre-defined
threshold (i.e., non-zero in 95% of the estimated networks), but it was nonetheless
consistently identified as a link (i.e., in 77.9% of the cases). Overall, the original and
extended networks were well aligned; the same edges were found, although not all
meeting the threshold. Of note, an additional link between affective speech illusions
and mannerism and posturing was identified in the extended network structure.
Adding more nodes seems to result in an increase in negative associations, reduced
power and a likely more unstable structure, due to the small sample size in relation
to the number of nodes. Therefore, apparent differences in the extended network
structure should be interpreted with caution.
Finally, in Appendix D5 we present the network structure of all speech illusions
and attenuated positive and affective symptoms and transition to clinical psychosis,
according to the operationalization of Catalan et al (2014), and discuss differences
between the network structures of all speech illusions constructed according to
the two different operationalizations (i.e., the Galdos operationalization and
the Catalan operationalization). While the results are generally well-aligned for
symptomatology, when using the operationalization of Catalan and colleagues
(2014), a negative association emerges between speech illusions and grandiosity,
while when using the Galdos operationalization (2011), no association between
Page | 114
speech illusions and other symptoms is identified. This indicates that the way the
speech illusions are operationalized is essential and can lead to different results and
network structures.
7.4 Discussion
The current study is the first to use a network approach to examine how
experimentally assessed speech illusions relate to individual attenuated positive
symptoms and transition to psychotic disorder. We found support for our first
hypothesis, a link between the experimentally assessed speech illusions and
attenuated positive symptoms that is specific to hallucinatory experiences, as would
be expected and in line with the findings of Rimvall et al. (2016). However, an
association was only found for speech illusions with affective content, and not all
content according to the Galdos or Catalan method (Catalan et al., 2014; Galdos et
al., 2011). The modest strength of the association, the specificity for hallucinatory
experiences, along with low rates of speech illusions and attenuated symptoms in
some non-clinical samples (discussed below) may explain why negative findings in
non-clinical samples have been observed (Catalan et al., 2014; Pries et al., 2017;
Schepers, Lousberg, et al., 2019; Schepers, van Os, et al., 2019). It might also be that
for some individuals hearing a voice in white noise is not a sign of hallucinatory
proneness, but indicative of a broader trait such as suggestibility or fantasy proneness
(Merckelbach & Van de Ven, 2001). However, the current combined findings of low
predictability of speech illusions by symptoms and transition to psychosis suggest
that speech illusions are not a mere artifact of positive or affective symptom severity
or presence of a psychotic disorder.
As expected, delusional ideation (specifically unusual thought content and
suspiciousness) was found to be intermediate between hallucinatory experiences
and transition. This is in line with longstanding clinical observations (James, 1890;
Maher, 1988; Reed, 1972) and empirical findings (Krabbendam et al., 2004; Smeets
et al., 2015; Smeets, Lataster, van Winkel, et al., 2012) and thus reflects imitable
symptomatic interplay between perceptual aberrations and transition to psychotic
disorder. However, we found the overall predictability of transition during two-year
follow-up by baseline assessment of speech illusions and all symptoms combined to
be strikingly low. A-posteriori analysis showed that none of the operationalizations
of speech illusions were associated with transition during two-year follow-up. These
findings suggest that a single assessment of speech illusions may not be useful as a
Page | 115
risk factor for psychosis. Of note, although the participants in the current study
were selected based on high risk criteria, the transition rate in our sample (around
18%) was found to be lower than the average reported for two year follow-up of
individuals meeting CHR status in meta-analytical data (29%; Fusar-Poli et al.,
2012). In their meta-analysis, Fusar-Poli et al. (2012) reported that transition risk
varied with the age of the subject, the provided treatment and the operationalization
of the high-risk syndrome and transition constructs. Our findings suggest that
there are important predictors not included in the current network, which likely
include risk and protective factors that may alter symptomatic course, such as
environmental exposure and treatment variables.
Galdos et al. (2011) reported app. 30% speech illusions in patients with
psychotic disorders, 14% in siblings of patients and 9% in healthy controls. The app.
22% prevalence rate of all speech illusions according to the Galdos method (2011)
in the present sample of individuals with CHR status seems in line with these
findings. Rates of affective speech illusions (app. 12% prevalence in the current
study) vary widely between studies, ranging from 9-18% in patients with psychotic
disorders (Catalan et al., 2014; Schepers, Lousberg, et al., 2019) and 0.4-15% in
healthy controls and individuals from the general population (Catalan et al., 2014;
Rimvall et al., 2016; Schepers, Lousberg, et al., 2019; Schepers, van Os, et al., 2019).
Also, prevalence rates of studies claiming to use the Catalan method (app. 47%
in the current study; Catalan et al., 2014) vary considerably: 33-47% in patients
with psychotic disorders (Catalan et al., 2014; Schepers, Lousberg, et al., 2019)
and 9-42% in healthy controls and individuals from the general (Catalan et al.,
2014; de Artaza et al., 2018; Rimvall et al., 2016; Schepers, Lousberg, et al., 2019;
Schepers, van Os, et al., 2019). Part of the variation may be explained by differences
in sample characteristics, such as age and cognitive ability. Schepers and colleagues
(2019) indeed suggested that speech illusions may be a trait-dependent risk marker
of psychosis and found support for this. To a lesser extent, part of the variation in
prevalence may be explained by the operationalization of speech illusions. Close
inspection of the operationalization of speech illusions revealed some differences
across studies (Rimvall et al., 2016; Schepers, Lousberg, et al., 2019), with not
all studies providing sufficient details for comparison. Research on the optimal
operationalization of speech illusions, including divergent and convergent validity
with other measures, is therefore warranted.
Given that there is some indication that cognitive ability may impact the relation
between speech illusions and positive symptoms (de Artaza et al., 2018; Schepers,
Page | 116
Lousberg, et al., 2019), within Supplement D we presented an extended network
structure also including cognitive symptoms. This analysis suggested a possible
additional relation between speech illusions and mannerisms and posturing.
Exploration of the network’s properties suggested that the variability between the
original and extended network structures is likely due to lower power when more
variables are included.
Further exploratory investigation suggested that bizarre behavior is a ‘gateway
symptom’ to psychotic disorder; it is the only individual symptom in our network
model steadily predictive (conditional on the other variables) of whether a subject
will transition to clinical psychosis or not. In the BPRS (Overall & Gorham, 1962;
Ventura et al., 1995), the current measure of baseline symptoms, this item reflects
behavior affected by hallucinations or delusions. Severely impacted behavior can
also be an indicator of threshold severity of delusions in the CAARMS, the current
measure of transition. Formal definition of a psychotic disorder, however, includes
a minimum duration of consistent positive symptoms over at least a week, which
could explain how one could have high levels of symptoms without meeting criteria
for transition. Of note, this would be transition to psychotic disorder according
to the CAARMS, which again varies from classification criteria for psychotic
disorders as defined in the DSM or ICD. One could argue that the distinction
between subthreshold psychosis and psychotic disorder is arbitrary, and that the
latter is in need of validation from biomarkers or other validators of course and
outcome, particularly given that these constructs do not adequately differentiate
levels of functioning or need for care (Yung et al., 2010). Other outcome measures
may also be considered. In the current study, for instance, depression and suicidality
are found to have relatively high levels of predictability and strength of connections
to other symptoms, including hallucinatory experiences.
Explorative investigation of the affective symptoms shows that they are
highly interconnected. Interestingly, speech illusions with affective content are
currently not found to be directly connected to affective symptoms, only through
attenuated positive symptoms. The current data on affective symptoms does not
allow for interpretation on what type of affect is represented; scores might for
instance represent a mood state, a comorbid affective disorder or the emotional
appraisal of psychotic experiences. Additionally, Fusar-Poli et al. (2014) point to
the problematic aspects of assuming a neat distinction between the constructs of
affect and psychosis, i.e., as readily distinguishable entities that might interact
or causally influence each other. Early observation-based theoretical models of
Page | 117
psychosis development describe a complex intertwinement between psychotic

experiences and affect in early symptom development. In Conrad’s classic 1958
stage model of psychosis development, the first stage, known as ‘delusional mood’
or ‘trema’, describes a build-up of a not yet specified, anticipatory sense, during
which first certain salient aspects and later the whole environment feel notably
changed and affectively charged. Conrad describes that in this phase patients can
experience a wide range of accompanying affect, such as excitement, fear, guilt
and depression, or any combination (Mishara, 2010). A large retrospective study of
Conrad’s stage model found support for this first ‘delusional mood’ stage (although
limited support for the latter stages; (Hambrecht et al., 1993).
7.4.1 Limitations
Several limitations of the current study should be considered. Firstly, the analyses
carried out here were mostly based on a single, cross-sectional assessment. Direction
of effects are implied, but cross-sectional networks may not necessarily reflect
how symptoms or states trigger each other over time (Bos et al., 2017). Second,
the estimation technique that we chose was designed to retain high specificity
and therefore weaker connections between variables may not have been detected.
Thus, absent connections should be viewed with some caution. Supplementary
exploration suggested possible additional relations between speech illusions and
mannerisms and posturing. Third, stability analyses show some caution is necessary
when interpreting the results and future research replicating these results in larger
samples is warranted. Fourth, mixed graphical models do not have good means of
handling missing data. As such, we included only the complete cases in our analysis.
Fifth, some negative associations emerged in the networks. These may be real effects,
but they can also be false positive effects or due to the presence of colliders in the
data (Pearl, 2000). Further, we identified different associations between all speech
illusions and symptomatology when using distinct operationalizations of speech
illusions (i.e., the Catalan operationalization versus the Galdos operationalization;
Catalan et al., 2014; Galdos et al., 2011), with the Galdos method identifying
no associations between all speech illusions and symptomatology and the Catalan
method identifying a negative association between all speech illusions and
grandiosity. This may result from low power and stability issues, but may also be
an indication that the way speech illusions are operationalized is substantial and
may affect results, including the network structure. Sixth, not all cases had full
two-year follow-up data. This means that we might have missed some cases who
Page | 118
did make transition to psychotic disorder. Finally, due to sample size and power,
we could not include all symptoms in the analysis. Also, other variables may be
of interest as well, such as measures of cognitive functioning and reasoning, and
potential modifiers of course, such as treatment variables. This, however, would
have required an even larger dataset or more assessment points.

Negative findings in non-clinical samples (Catalan et al., 2014; Pries et al., 2017;
Rimvall et al., 2016; Schepers, Lousberg, et al., 2019; Schepers, van Os, et al., 2019)
might lead to abandonment of new research on experimentally speech illusions as
a potential marker of psychosis proneness. Findings of the current study show that
although single assessment of speech illusions may not be useful as a risk factor for
psychosis, there is support for a specific and modest relation with hallucinatory
experiences. The current findings also show imitable symptomatic interplay between
aberrant perceptions and transition to psychotic disorder. While warranting
replication in larger samples, our findings argue against speech illusions being a
mere artifact of psychotic symptom severity or psychotic disorder. We therefore
believe that abandonment of speech illusions as a marker of psychosis liability
would be premature. Instead, we advocate for moving from studying associations
with composite symptom measures to individual symptom interactions. Future
studies may benefit from studying detailed trajectories of prospective symptom
development in individuals at risk for psychosis, with use of multiple, frequent
assessment periods, inclusion of potential modifiers of course and a variety of
clinical outcomes.
Page | 119
Toward Incorporating Genetic
8
Risk Scores into Symptom
Networks of Psychosis

Isvoranu, A. M., Guloksuz, S., Epskamp, S., van Os, J., Borsboom, D., GROUP
Investigators (2019). Toward Incorporating Genetic Risk Scores into Symptom
Networks of Psychosis. Psychological Medicine, 50(4), 636-643.
Chapter 8 | Genetic Risk in Symptom Networks of Psychosis
Abstract
Psychosis spectrum disorder is a heterogeneous, multifactorial clinical phenotype,
known to have a high heritability, only a minor portion of which can be explained by
molecular measures of genetic variation. This chapter proposes that the identification
of genetic variation underlying psychotic disorder may have suffered due to issues
in the psychometric conceptualization of the phenotype. Here we aim to open a
new line of research into the genetics of mental disorders by explicitly incorporating
genes into symptom networks. Specifically, we investigate whether links between a
polygenic risk score (PRS) for schizophrenia and measures of psychosis proneness
can be identified in a network model. We analyzed data from 2180 subjects (controls,
patients diagnosed with a non-affective psychotic disorder, and the first-degree
relatives of the patients). A network structure was computed to examine associations
between the 42 symptoms of the Community Assessment of Psychic Experiences
(CAPE) and the PRS for schizophrenia. The resulting network shows that the PRS
is directly connected to the spectrum of positive and depressive symptoms, with the
items conspiracy and no future being more often located on predictive pathways from
PRS to other symptoms. To our knowledge, the current exploratory study provides a
first application of the network framework to the field of behavior genetics research.
This allows for a novel outlook on the investigation of the relations between genome-
wide association study-based polygenic risk scores and symptoms of mental disorders,
by focusing on the dependencies among variables.
Page | 122
8.1 Introduction
Psychosis spectrum disorder is a potentially severe, heterogeneous, and multifactorial
disorder (Guloksuz & van Os, 2018; Keshavan et al., 2011). Although twin and
family studies indicate substantial heritability, few genetic variants are consistently
linked to psychotic disorder (Pardiñas et al., 2018). The difficulty of identifying
genes specific to psychosis has been explained in several ways (Sullivan, 2008). One
possibility that has received scant attention is that the conceptualization of the
phenotype may be suboptimal: typically, genetic studies use symptom counts (e.g.,
total scores defined on questionnaire data) or case-control designs that define cases
and controls as polythetic functions of symptom data (i.e., the definition of cases
corresponds to many distinct symptom profiles), thus defining the phenotype in a
highly simplified fashion. Essentially, these approaches assume that such compound
scores are estimates of a single underlying dimension (e.g., a liability spectrum)
that partly stands under genetic control (Franić et al., 2013). Additionally, genetic
research uses clear-cut diagnostic categories, even though dominant liability
spectrum theories are more consistent with a spectrum of related phenotypes
(van Os & Linscott, 2012), which feature no single set of necessary and sufficient
properties (e.g., a disease entity or common pathogenic pathway).
Several recent papers have argued that this conceptualization of mental disorders
may be too simplistic. Instead of reflecting a single disease entity, pathogenic pathway,
or unidimensional liability spectrum, disorders may result from causal interactions
between symptoms (e.g., delusions  paranoia  social isolation), which constitute a
network of symptoms and other components (Borsboom, 2017; Borsboom & Cramer,
2013; Cramer et al., 2010). Within this network conceptualization, the focus shifts
from investigating mental disorders–such as schizophrenia–as disease entities to
investigating interactions between symptoms (Borsboom, 2017; Borsboom & Cramer,
2013). Thus, mental disorders do not result from one central dysfunction (e.g., a brain
dysfunction) that causally produces symptoms, but from a complex interplay between
symptoms, psychological, biological and sociological components (Epskamp, 2017;
Isvoranu, Boyette, et al., 2020).
For psychosis spectrum disorders, which feature a clear genetic signal, this raises
the question of how genetic factors could relate to symptom-symptom interactions
in a network structure. In our view, genetic risk could plausibly influence symptom
networks in two ways. First, genetic risk could act as a direct main effect on symptoms.
In this case, genetic risk would be conductive to the liability to develop certain
Page | 123
symptoms of schizophrenia; this liability could be thought of in terms of genetic make-

up laying the foundations for symptoms to develop. For instance, one’s genetic make-
up could have an effect on the likelihood that the symptom ‘hallucinations’ occurs,
so that higher genetic risk influences the liability of developing the symptom, which
in turn would activate other neighboring symptoms in a network. Second, genetic
risk could function as a moderator of the network structure: genetic risk may involve
genetic factors that increase the strength of a causal connection between symptoms.
For instance, genetics may predispose a person to experience (more) anxiety in
response to hallucinations. In this scenario, genetic factors control part of the structure
of a network, such that a genetic risk would be expressed as the likelihood that one
symptom activates another. Analyses that would optimally represent the moderation
hypothesis are not yet fully developed, and therefore the current manuscript is focused
on the first approach.
To the extent that the above scenarios are plausible, current genetic studies
using total scores defined by these symptoms may rest on an inadequate definition
of the phenotype. In particular, genetic markers or polygenic risk scores may be
more profitably analyzed in relation to a network phenotype, instead of to a total
score or case-control status. The current chapter aims to introduce methodology
suited to incorporate genetic risk scores–in particular, genome-wide association
study (GWAS)-based polygenic risk scores (PRS) for schizophrenia–into symptom
networks, thus opening new lines of research into the genetics of mental disorders.
PRS is commonly used to summarize genetic effects and represents the sum of
trait-associated alleles across many genetic loci, weighted by effect sizes estimated
from GWAS (Euesden et al., 2015). PRS is generally used to identify genetic basis of
phenotypes and to construct risk prediction models (Dudbridge, 2013). Specifically,
here we investigate whether links between PRS for schizophrenia and measures of
psychosis proneness can be identified in a network model, by including the PRS as
a variable in the symptom network for psychosis spectrum disorder.
Page | 124
8.2 Methods
8.2.1 Participants
The sample analyzed was part of the longitudinal observational cohort study Genetic
Risk and Outcome of Psychosis Project (GROUP), release database 7.0 (Korver et al.,
2012). At baseline, the GROUP sample consisted of 1119 patients diagnosed with
a non-affective psychotic disorder, 1059 siblings of these patients, 920 parents, and
586 unrelated healthy controls. The patients were recruited from mental health
care institutions across The Netherlands and Belgium and the control subjects were
recruited through random mailing. Inclusion criteria for patients were age 16 to 50
years, DSM (American Psychiatric Association, 2000) criteria for a non-affective
psychotic disorder, maximum duration of illness of 10 years, and estimated level of
intelligence quotient above 70. For full details on the GROUP sample please refer
to the cited paper (Korver et al., 2012).
The present study used baseline data, restricted to the European white ethnic
group, as there is evidence for a differential impact of PRS in different ethnic
groups (Marden et al., 2014; van Os et al., 2017). Only subjects for which a PRS
was available were included in the analysis. Overall, a baseline sample of N=2180
individuals was analyzed here (patients, siblings, parents and controls).
We used baseline data from the Community Assessment of Psychic Experiences
(CAPE; Konings et al. 2006), a self-report measure of lifetime psychotic
experiences. All items, measuring frequency of positive (20 items), negative (14
items) and depressive symptoms (8 items) were included in the analyses and were
scored on a 4-point Likert type scale.
8.2.3 Network construction

We fitted a Gaussian Graphical Model (GGM; Lauritzen & Wermuth, 1989) to
the data (i.e., an undirected network). All items of the CAPE and the PRS were
represented as nodes. An edge between any two nodes indicates a partial correlation
between the 2 variables, after conditioning on all other variables in the dataset. To
account for the ordinal nature of the CAPE data, we used Spearman correlations
when estimating the network structure, as recommended by Epskamp and Fried
(2018). Due to the high sample size, we used unregularized model selection rather
Page | 125
than regularization techniques commonly used in estimating GGMs1 (Williams

& Rast, 2020). All analyses were performed using the R-statistical software, version
3.4.3 (R Core Team, 2015). The networks were constructed and visualized using
the R-package qgraph version 1.4.5 (Epskamp et al., 2012).
Blue (red) edges in the network indicate positive (negative) partial correlations,
and the wider and more saturated the edge, the stronger the partial correlation
(Epskamp et al., 2012). Because the present chapter specifically focuses on edges
between the PRS and symptoms, edges representing associations between PRS
and symptoms have been manually unfaded.2 For constructing the layout of the
network, we used a manually specified layout where the PRS is positioned in the
center of the network, while allowing the rest of the nodes in the network to cluster
based on their associations.
To display how the connectivity of edges is related across genetic and
symptomatic levels of analysis, we computed a predictive path diagram. A predictive
path diagram takes one node (in our case PRS) as a source node and then lists I)
the nodes that have an edge with PRS; II) the nodes to which the nodes that have
an edge with PRS are connected; III) the nodes to which the latter are connected,
and so forth. Within the so constructed diagram, the shortest predictive pathway
can be defined as the pathway that results from traveling the edges that, in each
step, maximize the quality of the prediction of the target node from the source
node, while controlling for all other variables in the network. To investigate which
nodes more often lie on the shortest predictive pathways from PRS to other nodes
we computed node-specific predictive betweenness as a centrality measure (see
Figure 8.3). Because betweenness is generally not a very stable centrality measure
(Epskamp, Borsboom, et al., 2017), we used both nonparametric and case-drop
bootstraps to investigate the extent of its variability (Epskamp et al., 2018).
1 The estimation procedure first runs the glasso algorithm (Friedman et al., 2008) for 100 different
tuning parameters to obtain 100 different network structures (ranging from sparse networks with
few edges to dense networks with many edges). Next, the algorithm refits all those networks with-
out regularization and selects the model that minimizes the Bayesian information criterion (BIC).
Subsequently, the algorithm adds and removes edges until BIC can no longer be improved. It has
been shown that BIC selection of GGMs selects the true model as N grows to infinity (Foygel &
Drton, 2011), subject to the sparsity assumption. In addition, the stepwise procedure ensures that
no single edge can be removed or added in the final model to improve fit.
2 This is because the software used for constructing the networks automatically fades edges with a
smaller effect: An effect between a variable from the external field of a symptom (such as an envi-
ronmental variable or a genetic risk score; Borsboom, 2017) will likely almost always be smaller than
the effect of the variables within the network (e.g., the relations between the symptoms themselves).
Page | 126
8.2.4 Network stability
As recommended in the literature (Borsboom et al., 2018), to investigate robustness
and replicability of results we performed accuracy and stability checks using the
R package bootnet (Epskamp, Borsboom, et al., 2017). In addition, we included a
stability analysis to investigate the average node-specific predictive betweenness
under case-dropping. For these results we refer the reader to Appendix E1.
8.3 Results
The study sample at baseline consisted of 2180 subjects (335 control subjects, 640
siblings, 630 parents and 575 patients), of which 47% female and 53% male. Within the
patient sample, the main diagnosis was schizophrenia, paranoid type (53%), followed
by psychotic disorder NOS (11%), schizoaffective disorder (10%) and schizophreniform
disorder (6%). The mean age of the subjects was 35.97 (14.61) years and the mean IQ
was 103.48 (16.17). In the current analyses a GWAS p value threshold of .05 was used.
In total, 194,665 SNPs were included in the polygenic risk score.

Figure 8.1 represents the network depicting positive, negative, and depressive
symptoms of the CAPE, as well as the PRS. Results at p<.05 show that the PRS
is directly connected to several individual symptoms of the CAPE, but especially
related to the spectrum of positive psychotic symptoms. Network analysis identified
relations between the PRS and the positive psychotic symptoms C10 (conspiracy),
C30 (thought echo), and C41 (Capgras). In addition, we identified positive relations
between the PRS and the depressive symptom C12 (no future).3 Stability analyses
show that the network and identified edges are generally stable (see Appendix E1).
3 Of note, when running the same analysis within the subsample of patients and relatives only, the
relations between the PRS and C10 (conspiracy), and C41 (capgras) are retained and no other edges
emerge. It is likely that edges between PRS and C12 (no future) and C30 (thought echo) do not
withhold due to a loss in power.
Page | 127
Page | 128
Figure 8.1 Network of the 42 CAPE (Konings et al., 2006) symptoms and the PRS for psychosis (N=2180). Blue (red) lines represent positive (negative) asso-
ciations between variables and the wider and more saturated the edge, the stronger the association (Epskamp et al., 2012). Please note that since the focus of the
chapter was to investigate the relations between the PRS and symptoms, the edges between the PRS and symptoms have been manually un-faded, while the edges
between the other nodes in the network retain transparency. Symptom groups are differentiated by color.
Figure 8.2 Predictive Path Diagram of the 42 CAPE (Konings et al., 2006) symptoms and the PRS for psychosis (N=2180). A predictive path diagram is based
on shortest pathway analysis (Brandes, 2008; Opsahl et al., 2010) and represents the first three levels of connectivity between variables. Specifically, here we
visualize 1) the immediate nodes to which the PRS is connected; 2) the immediate nodes to which the nodes that are connected to the PRS are connected; 3) the
nodes to which the latter are connected. Blue (red) lines represent positive (negative) associations between variables and the wider and more saturated the edge,
the stronger the association (Epskamp et al., 2012). Please note that since the focus of the chapter was to investigate the relations between the PRS and symp-
toms, the edges between the PRS and symptoms have been manually unfaded (i.e., within the first level of connectivity), while the edges between the other nodes
retained default plotting functions. Symptom groups are differentiated by color.
Page | 129
The predictive path diagram (see Figure 8.2) highlights secondary levels of
connectivity within the network – level two connections show strong relations
between symptoms C10 (conspiracy), C7 (persecution) and C41 (Capgras), as well
as between symptoms C12 (no future) and C14 (suicidal). Level three connections
display the strongest relations between symptoms C33 (hallucinations) and C34
(voices conversing), C32 (blunted emotions) and C27 (blunted affect), C20 (voodoo)
and C15 (telepathy), and C11 (important person) and C13 (special person). Within
this level, most symptoms were connected to most of the other symptoms. Notably,
both in level two and three, most items that had strong links with other items were
in fact linked to an item from the same dimension. Level three of connectivity
was formed mostly of the positive symptoms, while level four of connectivity was
formed mostly of the negative symptoms and the relations between these. Overall,
the predictive path diagram shows that it becomes possible to reach any other
symptom in the network from the PRS in no more than four steps. Nonetheless,
these results should be interpreted with caution considering the large variability
in edge weights and the presence of negative edges.
Bootstrapping showed that the estimation of node-specific predictive
betweenness (i.e., items that more often lie on the shortest pathways from PRS
to other nodes; see Figure 8.3) was considerably less precise than that of other
features of the network. In particular, Figure 8.3 showed that several nodes featured
high node-specific predictive betweenness in bootstrap samples but not in the
centrality of the estimated network structure based on the sample, indicating that
the shortest paths from PRS to other nodes can take various forms. A potentially
important result that was robustly present across case-drop bootstraps (see Figure
E3 in Supplement E) is that the most central symptoms in the current sample in
terms of node-specific predictive betweenness are symptoms C12 (no future) and
C10 (conspiracy). Other central symptoms–even though to a much lesser degree and
possibly as secondary items on the pathway between PRS and other nodes–were
items C25 (lack of activity), C30 (thought echo), and C39 ( failure). Additionally,
the nonparametric bootstrapped networks showed items C9 ( feeling pessimistic),
C21 (lack of energy) and C36 (unable to terminate) as further potentially important
nodes, even though these were not specifically captured as high node-specific
predictive betweenness items in the current network structure (see Figure 8.3,
black lines).
Page | 130
40
30
Value
20
●
10
●
●
● ●
● ● ● ● ● ●
0
● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20 C21 C22 C23 C24 C25 C26 C27 C28 C29 C30 C31 C32 C33 C34 C35 C36 C37 C38 C39 C40 C41 C42
Node
Figure 8.3 Node-specific predictive betwenneess (i.e., how often a node lies on the pathways between
two other nodes, of which one is always the PRS). The white dots represent the node-specific predic-
tive betweenness in the current sample, while the black lines represent the variability of node-specific
betweenness when using nonparametric bootstrapping over 1000 iterations.
8.4 Discussion
The current study provides, to our knowledge, a first application of the network
framework (Borsboom & Cramer, 2013) to the field of behavior genetics research.
This allows for a novel outlook on the investigation of the relations between
genome-wide association study-based polygenic risk scores and symptoms of mental
disorders, by focusing on the dependencies among variables.
Most links we identified in our data are links between the PRS and positive
psychotic symptoms, especially symptoms related to notions of conspiracy and
paranoia. This can be especially of interest as studies including risk factors in
network models of psychosis (e.g., cannabis use, childhood trauma, urbanization)
did not identify direct relations between any of these risk factors and positive
or negative psychotic symptoms (Isvoranu et al., 2016, 2017; see Chapter 4 and
Chapter 5). In addition, positive psychotic symptoms are generally found to have
few connections to other nodes, especially nodes related to real life functioning
(Galderisi et al., 2018). In light of this, a plausible hypothesis to put forward would
be that different types of risk factors may lead to different pathways to the onset
(and development) of psychotic disorders: genetic risk factors may have more
of a main effect on the liability to develop positive psychotic symptomatology,
while environmental risk factors may have more of a secondary effect on general
psychopathology symptoms. This may partly explain why environmental factors are
Page | 131
often shared between distinct mental disorder categories and is in line with previous
findings which argued that symptoms can have different risk factors (Epskamp,
Borsboom, et al., 2017; Fried, Bockting, et al., 2015; Isvoranu et al., 2016, 2017).
To further investigate symptom associations at different levels of connectivity,
the current study constructed for the first time a predictive path diagram. To a
certain degree, the predictive path diagram aims to point out toward potential
mediating items. Here, the second level of symptom-symptom interactions
consisted of mostly positive psychotic symptoms, while the third level of symptom-
symptom interactions had a predominance of negative psychotic symptoms. Most
of the symptoms connected to other symptoms from the same domain, suggesting
that node activation may first spread within the same domain. In addition, since
most links were identified between the PRS and positive psychotic symptoms, the
second level included most of the positive psychotic symptoms. The few negative
symptoms identified at the second level of symptom-symptom interactions are items
related to lack of enthusiasm, energy and (social) activities. The only two positive
psychotic symptoms identified at the third level of interaction are telepathy and
special person. Depressive symptoms are shared between the two levels and generally
have less strong relations within and between domains. It should be noted that
the symptoms with the strongest relations between different levels of connectivity
are highly similar in content to each other (e.g., strong relations between items
C10: Do you ever feel as if there is a conspiracy against you? and C7: Do you ever
feel as if you are being persecuted in some way? or between items C32: Do you ever
feel that your emotions are blunted? and C27: Do you ever feel that your feelings are
lacking in intensity?). This result may therefore be an artifact of the scale used to
measure symptomatology, in which many of the items are contextually similar
and therefore are likely highly correlated with each other. Further research using
different measurement scales is warranted before drawing strong conclusions based
on this methodology.
Finally, for the current study we developed a measure of node-specific predictive
betweenness, which allows for the investigation of items that more often lie on
the shortest pathway from PRS to other nodes in the network. We identified the
items conspiracy and no future as central items and hypothesize that these may be
important hubs in the network.
Page | 132
8.4.1 Methodological notes
Several qualifications of the network methodology are in order here. First, the
statistical approach of incorporating PRS in a network is suited to pick up direct
links between the PRS and psychotic symptoms. As such, this methodology is best
interpreted as incorporating the hypothesis that PRS contains genetic factors that
increase the liability to develop certain symptoms (i.e., acts through main effects
on the symptom thresholds in a network). This is because PRS is modeled as a
predictive node from the causal background.
Second, predictive nodes from the causal background are of course unlikely to
operate on the same timescale as symptoms in a symptom network. However, the
nature of statistical control in a partial correlation network which includes PRS as
a node is likely not that different from that of a network that exclusively contains
symptoms. If, for instance, PRS was a common cause for two symptoms within
a network structure (e.g., increased PRS leads to increased liability to develop
symptoms sadness and conspiracy) but not included in the network (and thus not
controlled for), a spurious edge between the two symptoms would be induced. If
PRS was indeed linked to symptoms, such spurious associations would then be
eliminated due to the inclusion of PRS as a node in the network. Of note, a node
only induces spurious connections if it is a common effect in a network (Epskamp,
Waldorp, et al., 2018), but PRS cannot be a common effect, because it is not caused
by symptoms. Therefore, to the extent that stable edges arise between PRS and
symptomatology, such edges are consistent with the hypothesis that PRS impinges
at the symptom level.
Third, it is widely known that genetic influences account for a small proportion
of the explained variance in a mental disorder and even though this proportion is
larger for psychosis, the effect is still modest (Greenwood et al., 2007). Naturally,
symptoms tend to cluster together and have a stronger influence on each other,
while variables from the external field of a disorder, such as environmental and
genetic factors also better cluster together within their domain (Santos et al.,
2017). Networks therefore often display weaker between-domain links (Isvoranu
et al., 2017; Santos et al., 2017), and may miss between-domain links due to
a lack of statistical power. In addition, recent studies showed that when using
estimation techniques relying on the lasso regularization (Tibshirani, 1994)–
which many prior network studies have been based on–specificity can be lower
than expected in dense networks with many small edges, leading to an increase
in false positives (Williams & Rast, 2020). To this end, we chose a different
Page | 133
and more conservative estimation technique here based on unregularized model

search, as recently recommended by Epskamp (2018). For low-dimensional
settings with high sample size, simulation studies using this technique have
shown good sensitivity as well as high specificity. We compared this method
with the classic lasso regularization method, as well as with a method based
on thresholded partial correlations. The unregularized model search performed
best in terms of goodness-of-fit measures. In addition, the edges from PRS to
conspiracy and no future were identified by two of the three methods, while the
edges from PRS to thought echo and Capgras were identified by all three methods
(see Figure E3 and Table E1). Overall, based on these results, the effects identified
in this study are unlikely to be simply a result of noise and may hold etiological
significance. In addition, bootstrap analyses (see Figure E1) show the edges from
PRS to C41 (Capgras) and C30 (thought echo) are the most robust (i.e., showing
up in 67% of the bootstraps and 75% of the bootstraps respectively). Notably, all
edges from the PRS are identified in at least 50% of the bootstraps.
8.4.2 Limitations
The findings of this study need to be considered in light of several limitations. First,
a PRS itself can be considered an important limitation. The signal between a PRS
and symptoms is generally weak (Greenwood et al., 2007), also resulting in weak
network connections, which are not overly consistent across different estimation
techniques. In addition, by adding up SNPs to construct the PRS we may obscure
details in the same way in which adding up items into sum-scores may obscure the
different symptoms – investigating individual SNPs in relation to symptoms may
therefore be a promising new avenue. It is further possible that the predictive power
of the PRS also becomes weaker in broader mixed samples across the spectrum.
The sample analyzed in this chapter is a heterogeneous sample, which includes
controls, siblings, parents and patients; in addition, as the CAPE is not designed
for measuring symptomatology in patients, ceiling effects may occur when applying
the CAPE to the patient population. However, given that the strong phenotypic
manifestation of the PRS can often be identified in healthy relatives of patients (van
Os et al., 2017), we chose to include the whole sample rather than focus on one
group only. Ideally it would be possible to carry out analyses and direct network
comparisons between different population groups; this was not possible in the
current study due to limited data.
Page | 134
Second, it is difficult to investigate to what degree any PRS effect would be
reducible to group membership (i.e., patients have much more psychopathology
and higher PRS scores). Including a group membership variable in a network would
be problematic (Fried & Cramer, 2017) due to the categorical nature of the variable
and therefore investigating to what extent there would be overlap between the two
variables was not possible. A future extension for network modeling of genetic data
may be modeling the multilevel structure of the data.
Third, the negative association between PRS and thought echo is a puzzling
finding, which could be related to some form of measurement bias or rarity,
but could also be a true effect given fixed level of other symptoms. Further
investigation showed that, even though the expectation would be that PRS would
positively correlate with all other variables, PRS and though echo were actually
also weakly marginally negatively correlated (r=-.004), thus ruling out that the
result is paradoxical (i.e., a marginal positive correlation was not made negative
upon conditioning). One possible alternative explanation to the stronger negative
association in the network is that it may be due to the presence of a collider in the
data (i.e., both PRS and thought echo cause a third variable in the dataset; Pearl,
2000). Another explanation may be that this is a spurious effect, resulting from the
violation of the normality assumption (i.e., the variable ‘thought echo’ is relatively
skewed to the left). Future research should focus on if the link between PRS and
thought echo is a genuine negative effect that can be replicated.
Finally, the current analyses were based on cross-sectional data and conclusions
regarding direction or causality should be drawn with caution. In addition, it may
be argued that results from cross-sectional networks may not be generalizable
to within-person dynamics (Bos et al., 2017). It should however be noted that
longitudinally investigating variables from the external field of a network, such
as environmental or genetic risk factors, is methodologically infeasible as these
background variables may not change themselves; in addition, as we argued in
this manuscript, if such variables act to facilitate the ease with which nodes in the
network are activated, the current analysis may in fact be more informative than a
within-subject analysis of dynamically changing factors would by itself be. Future
research could expand on our analyses by investigating whether PRS might further
have an effect on the interactions between symptoms, using network moderation
techniques (Haslbeck et al., 2019) or by relating PRS to parameters of networks
that govern dynamic changes in symptomatology.
Page | 135

Overall, the results of our study indicate that investigating pathways within a
network structure through which genetic components may affect the liability to
develop a mental disorder may be promising and may open new research avenues.
In this chapter we have taken a first step toward introducing this methodology.
Further investigating how (and whether) other polygenic risk scores are related to
(other) symptoms, as well as whether we can identify certain genetic segments for
specific symptoms could lead to biological informative structures. The current study
is exploratory in nature; in light of this, we discuss methodological considerations
and limitations of the approach and argue further research replicating these results
is essential. Ultimately, we aim to provide a first means toward bringing together
the field of behavior genetics and the network framework, and provide preliminary
results for symptom-specific biological pathways to psychosis.
Page | 136
Page | 137
Part III
PSYCHOTIC DISORDERS AND

COMORBID FEATURES
Obsessive-Compulsive
9
Symptoms and Other
Symptoms of the At Risk
Mental State for Psychosis

Ong, H. L.*, Isvoranu, A. M.*, Schirmbeck, F., McGuire, P., Valmaggia, L.,
Kempton, M. J., ... & EU-GEI High Risk Study (2021). Obsessive-Compulsive
Symptoms and Other Symptoms of the At Risk Mental State for Psychosis: A
Network Perspective. Schizophrenia Bulletin, 47(4), 1018-1028.

Chapter 9 | Obsessive-Compulsive Symptoms in UHR for Psychosis
Abstract
The high prevalence of obsessive-compulsive symptoms (OCS) among subjects
at Ultra-High Risk (UHR) for psychosis is well documented. However, the
network structure spanning the relations between OCS and symptoms of the at
risk mental state for psychosis as assessed with the Comprehensive Assessment of
At Risk Mental States (CAARMS) has not yet been investigated. This chapter
aimed to use a network approach to investigate the associations between OCS and
CAARMS symptoms in a large sample of individuals with different levels of risk for
psychosis. 341 UHR and 66 healthy participants were included, who participated in
the EU-GEI study. Data analysis consisted of constructing a network of CAARMS
symptoms, investigating central items in the network, and identifying the shortest
pathways between OCS and positive symptoms. Strong associations between OCS
and anxiety, social isolation and blunted affect were identified. Depression was the
most central symptom in terms of the number of connections, and anxiety was a key
item in bridging OCS to other symptoms. Shortest paths between OCS and positive
symptoms revealed that unusual thought content and perceptual abnormalities were
connected mainly via anxiety, while disorganized speech was connected via blunted
affect and cognitive change. Findings provide valuable insight into the central role
of depression and the potential connective component of anxiety between OCS
and other symptoms of the network. Interventions specifically aimed to reduce
affective symptoms might be crucial for the development and prospective course
of symptom co-occurrence.
Page | 142
9.1 Introduction
Obsessive-compulsive symptoms (OCS) have a prevalence rate of 30.7% and
comorbid obsessive-compulsive disorders (OCD) of 12.3% in patients with
a psychotic disorder (Swets et al., 2014). Accordingly, a review of the literature
reported increased prevalence rates of OCS (13.7%) and OCD (5.5%) in subjects
at Ultra-High Risk (UHR) for psychosis (Poyurovsky et al., 2005; Zink et al.,
2014) compared to the general population. Recent studies reported even higher
rates in individuals meeting UHR criteria (Soyata et al., 2018). Subjects at UHR
are defined by either experiencing brief intermittent psychotic symptoms (BLIPS),
or sub-threshold attenuated psychotic symptoms (APS), or having a genetic risk
for psychosis in combination with a significant decline in functioning (Fusar-Poli
et al., 2016). Of those individuals meeting UHR criteria 22% develop a psychotic
disorder within three years after presentation to clinical services (Fusar-Poli et al.,
2020). The high OCS prevalence rates in UHR subjects have resulted in increased
research interest in investigating effects of OCS on clinical outcomes, such as
psychosocial functioning and transition rates (Niendam et al., 2009; Soyata et al.,
2018; Zink et al., 2014). Results have been inconclusive. Some authors have found
higher impairment of functioning and positive symptoms in UHR subjects with
co-occurring OCS compared to those without (Niendam et al., 2009), whereas
others have reported no associations or even better functioning in the group with
OCS (Soyata et al., 2018; Zink et al., 2014).
To date, the association between OCS/ OCD and subthreshold symptoms of
psychosis has solely been reported as co-occurring prevalence rates or associations
on the syndrome level in the general population (van Dael et al., 2011), in healthy
relatives of patients (Schirmbeck et al., 2018, 2019) and in UHR subjects (Fontenelle
et al., 2012; Niendam et al., 2009; Zink et al., 2014). The investigation of the
effects of OCS/ OCD in UHR subjects assumes a clear distinction between these
constructs. However, phenomenologically distinguishing OCS from attenuated
psychotic symptoms e.g., intrusive thoughts from overvalued ideas is challenging
(de Haan & Zink, 2015). In accordance, the Comprehensive Assessment of At
Risk Mental State (CAARMS) for psychosis, the standard instrument for assessing
prodromal psychopathology and identification of the UHR status, assesses OCS
as part of psychopathology dimensions, thought to be associated with imminent
psychotic disorder (Yung et al., 2005). Hence, exploring associations between
Page | 143
OCS and symptoms of the at risk mental state for psychosis may uncover novel
perspectives on the patterns and interrelations between them.
The network approach to psychopathology (Borsboom, 2017; Borsboom &
Cramer, 2013) allows for the investigation of such direct relations and interactions
on the symptom level, without a priori assumptions regarding the adequacy of the
diagnostic categories constructed out of these symptoms. Classical psychometric
practices are typically based on latent variable theory, where psychopathological
symptoms are caused by a hypothetical underlying mental disorder (e.g., OCD),
implying that symptoms cannot be directly related to each other if the disorder
is absent (i.e., local independence; Borsboom, 2017; Borsboom & Cramer, 2013).
Within a network model, however, symptoms are viewed as active causal agents
that can influence one another. Hence, a disorder can be seen as emerging from a
system of interacting components. As a result, within a network structure, we can
investigate associations between individual symptoms, as well as potential bridging
elements (e.g., symptoms or other factors) connecting clusters of (other) interacting
symptoms (Borsboom, 2017).
To this end, the purpose of the current chapter was to explore the interplay
between OCS and other symptoms of the CAARMS using a network model, in a
sample with different levels of risk for psychosis, by including individuals meeting
UHR criteria as well as healthy controls. Including healthy control subjects helped
to broaden the focus to subclinical symptoms in a sample of non-help seeking
individuals and also increased power of the conducted analyses. We aimed to
identify aspects of symptom co-occurrence, such as symptoms that may play an
important role in facilitating the co-occurrence of OCS and psychotic symptoms.
9.2 Methods
9.2.1 Participants and procedure
Data were collected in the European network of national schizophrenia networks
studying Gene-Environment Interactions (EU-GEI study), which was conducted
between May 2010 and May 2015. The EU-GEI study was designed to identify
the interactive genetic, clinical, and environmental determinants involved in
the development, severity, and outcome of psychosis. Participants were part of
the High Risk Study of the EU-GEI cohort. A sample of UHR individuals were
recruited from 11 health care institutions (Amsterdam, Barcelona, Basel, Cologne,
Copenhagen, London, Melbourne, Paris, Sao Paulo, The Hague, and Vienna).
Page | 144
Controls were recruited from the same geographical catchment area as the UHR
group, but only from 4 centers (Kraan et al., 2018). The protocol of the EU-GEI
study was approved by the Medical Ethics Committees of all participating study
sites. In the current study, we included UHR individuals and healthy subjects for
whom CAARMS measures were available.
Inclusion criteria for UHR and control participants were being aged 14 to 35 at
the time of recruitment, ability to comprehend assessments and provide informed
consent. UHR subjects had to meet at least one of the UHR criteria as defined
by the following CAARMS criteria: (1) a schizotypal personality disorder or a
first degree relative with psychosis in combination with a significant decline in
functioning; (2) attenuated positive psychotic symptoms such as ideas of reference,
odd beliefs, magical thinking, or unusual perceptual experiences; and (3) a brief
psychotic episode of less than one-week duration that resolves without antipsychotic
medication. The exclusion criteria were: (1) having a psychotic episode for more
than one week, (2) symptoms relevant for inclusion are explained by a medical
disorder or drugs or alcohol dependency, (3) intelligence quotient less than 60.
Additional exclusion criteria for controls were the presence of an UHR status as
defined above or ever treated with any antipsychotic medication.
9.2.2 Materials
All participants completed a detailed sociodemographic questionnaire which
included questions on their age, gender, and ethnicity. Further, participants
completed the CAARMS, a 28-item semi-structured interview developed to assess
UHR criteria that is intensity, frequency, duration and recency of subthreshold
psychotic symptoms, as well as other dimensions of psychopathology thought
to be indicative of or associated with imminent psychotic disorder (Fusar-Poli,
Hobson, et al., 2012; Yung et al., 2005). The CAARMS can be clustered into seven
domains: positive symptoms, cognitive symptoms, emotional disturbance, negative
symptoms, behavioural change, motor changes and general psychopathology. As
part of general psychopathology, one CAARMS item assesses symptoms of OCD
by asking participants whether they repeatedly experience upsetting thoughts
which they cannot stop, whether they repeat actions they feel they should do,
whether they have to do things in a specific way to prevent feeling extremely
anxious, whether they need to check things repeatedly like light switches/ gas/
electrical appliances, or doors or whether they are doing something to prevent
unpleasant things from happening (rituals/ superstitions). Intensity of reported
Page | 145
OCS is measured on a 7-point Likert scale from 0: never present to 6: extreme,

with a higher score reflecting higher symptom severity. In addition, frequency of
symptoms is assessed also on a 7-point Likert scale from 0: absent to 6: continuously
present and a question assessing whether OCS manifested by substance abuse. In
the current study CAARMS intensity scores were included.
9.2.3 Data analysis

In the current analysis we estimated a network model of OCS and ARMS
symptoms, using the R statistical software version 3.6.3 (R Core Team, 2015), the
R package bootnet version 1.3 (Epskamp, Borsboom, et al., 2017), and we visualized
the resulting network structure using the R package qgraph 1.6.5 (Epskamp et al.,
2012). Due to the relatively low sample size in relation to the large number of nodes,
and in an attempt to ensure more stable network estimates, we chose to reduce
the number of nodes, prior to running the analyses. We did so by excluding low
variability items (i.e., items with more than 75% response of 0: never), which would
have had the potential to bias results (e.g., due to severe violations of the normality
assumption; Epskamp, Waldorp, et al., 2018). Low variability items were selected
based on the full sample, but these were identical when investigated in the UHR
only sample. Each remaining CAARMS item was represented as a node in the
network. The regularized partial correlation between two nodes, after controlling
for all other nodes in the network, was represented as an edge.
First, we estimated an undirected network structure (i.e., a Gaussian Graphical
Model; Lauritzen, 1996), which allows identifying the conditional associations
between variables (Epskamp, Waldorp, et al., 2018). To prevent spurious
connections and to estimate a sparse model, we adopted the least absolute shrinkage
and selection operator (LASSO) statistical regularization technique. We chose
the graphical LASSO, which uses the Extended Bayesian Information Criterion
(EBIC) model selection with a tuning hyperparameter set to .5, as commonly
employed in previous research (Galderisi et al., 2018; Isvoranu et al., 2017; van
Borkulo, Borsboom, et al., 2014). To account for the non-normal distribution
of some items and missing data, we based our analyses on Spearman correlations
using pairwise complete observations (Epskamp, Borsboom, et al., 2017; Isvoranu
& Epskamp, 2021).
Second, we assessed the importance of each node in the resulting network by
computing centrality indices of the network structure, using the R package qgraph
(Epskamp et al., 2012). Specifically, we computed strength centrality for each node
Page | 146
in the network (i.e., the sum of the weights of the connections, in absolute value),
as well as the node-specific predictive betweenness (i.e., how often a node lies on
the pathways between two other nodes, of which one is always the OCS node;
Barrat et al., 2004; Isvoranu et al., 2019; Newman, 2004). We chose to investigate
only these two centrality measures because (1) previous research has reported
that strength centrality is the most stable and interpretable centrality measure
(Epskamp, Borsboom, et al., 2017) and (2) our interest was in bridging nodes that
may play a role in linking OCS to other symptom clusters, and may thus pave the
way to co-occurrence.
Third, we used the pathways function from the R package qgraph (Epskamp
et al., 2012), as used in previous research (Isvoranu et al., 2017; see Chapter 5), to
examine the shortest pathways between the OCS node and each of the positive
psychotic symptoms. This examination allows us to identify the shortest path from
point A to point B for specific symptoms of interest (e.g., from OCS to each positive
symptom) and is computed using Dijkstra’s algorithm (Dijkstra, 1959).
Finally, we used the R package bootnet version 1.3 to conduct robustness analyses
to ensure stability and accuracy of results (Epskamp, Borsboom, et al., 2017).
Appendix F1 details on these analyses.
9.3 Results
9.3.1 Sample characteristics
Of the 344 UHR subjects and 67 healthy controls originally included in the
EU-GEI study, 341 UHR and 66 healthy participants provided comprehensive
data on the CAARMS and were included in the current study. Slightly more than
half of the sample (52.7%) were male, and the mean age was 22.5 years (SD=4.79).
29 (8.2%) participants fulfilled the diagnostic criteria for an obsessive-compulsive
disorder (OCD), 53 (21.0%) reported moderate to severe obsessions and 54
(14.4%) reported moderate to severe compulsions, respectively. Table 9.1 shows
sociodemographic and clinical characteristics for the overall sample and the UHR
and healthy controls subgroup separately. In addition, Table F1 in Supplement F
presents the means and standard deviations of individual CAARMS items.
Page | 147
Table 9.1 Sociodemographic and clinical characteristics for the overall sample and subjects at ultra-high
risk (UHR) and healthy controls (HC) apart.
Overall sample UHR HC
N=407 N=341 N=66
Age (mean, SD) 22.52 (4.79) 22.42 (4.91) 22.91 (4.09)
Gender (male %) 53.3 53.4 53.0
Ethnicity (Caucasian %) 69.7 71.2 62.1
Years of education (mean, SD) 14.68 (3.08) 14.37 (3.07) 16.15 (2.72)
Current employment (yes %) 65.6 60.5 90.7
Current living (alone %) 13.3 14.8 6.1
DSM comorbid diagnosis (yes %)
Depression 27.8 33.0 0
OCD 8.2 9.7 0
Anxiety disorder 38.8 45.1 6.3
CAARMS (mean, SD)
Positive 8.63 (5.01) 10.09 (3.96) 1.12 (2.54)
Negative 6.02 (3.87) 6.94 (3.41) 1.35 (2.47)
Cognitive 2.70 (1.83) 3.14 (1.65) 0.48 (0.91)
Emotional 2.71 (2.45) 3.16 (2.40) 0.39 (0.95)
Social 6.92 (4.51) 8.02 (3.96) 1.39 (2.70)
Motor 1.90 (2.46) 2.24 (2.55) 0.25 (0.89)
General 12.83 (7.59) 14.96 (6.33) 2.52 (3.71)
Medication use
Antidepressant (yes %) 29.8
Antipsychotics (yes %) 9.6
Anxiolytics (yes %) 9.2
Table 9.2 Node labels and corresponding symptom names of the 28 CAARMS symptoms
Item Label Item Description Domains
P1 Unusual thought Positive Symptoms
P2 Non-bizarre ideas Positive Symptoms
P3 Perceptual abnormalities Positive Symptoms
P4 Disorganized speech Positive Symptoms
N1 Anhedonia Negative Symptoms
N2 Avolition Negative Symptoms
Page | 148
Table 9.2 Continued.
Item Label Item Description Domains

N3 Alogia Negative Symptoms
C1 Subjective cognitive change Cognitive Symptoms
C2 Observed cognitive change Cognitive Symptoms
E1 Emotional disturbance Emotional Symptoms
E2 Observed blunted affect Emotional Symptoms
E3* Observed inappropriate affect Emotional Symptoms
B1 Aggression Behavioral Symptoms
B2 Social isolation Behavioral Symptoms
B3 Impaired role function Behavioral Symptoms
B4* Disorganizing, odd, stigmatizing behavior Behavioral Symptoms
M1* Subjective motor change Motor Symptoms
M2* Observed motor change Motor Symptoms
M3* Impaired bodily sensation Motor Symptoms
M4* Impaired autonomic functioning Motor Symptoms
GP1 Depression General Psychopathology Symptoms
GP2 Suicidality General Psychopathology Symptoms
GP3* Mania General Psychopathology Symptoms
GP4 Mood swings General Psychopathology Symptoms
GP5 Anxiety General Psychopathology Symptoms
GP6 OCD symptoms General Psychopathology Symptoms
GP7 Dissociative symptoms General Psychopathology Symptoms
GP8 Impaired tolerance to normal stress General Psychopathology Symptoms
Note: *Seven symptoms were excluded from the network analysis due to low variability.
9.3.2 General network structure

We identified and excluded the following seven symptoms with low variability
(i.e., items with more than 75% response of 0: never): GP3 (mania), E3 (observed
inappropriate affect), B4 (disorganizing, odd, stigmatizing behavior), M1 (subjective
motor change), M2 (observed motor change), M3 (impaired bodily sensation),
and M4 (impaired autonomic functioning) from the network structure. Figure
9.1 presents the network constellation of the 21 remaining symptoms from the
CAARMS. All symptoms were connected, either directly or indirectly, with other
symptoms. Overall, there were stronger connections with items within the same
domain than across domains, as defined in Table 9.2. While robustness analyses
Page | 149
indicate some caution is needed when interpreting the strength and presence of
weaker edges, our results here focus on highlighting the stronger and thus likely
robust edges in the network structure. For instance, strong connections were found
between GP1 and GP2 (depression and suicidality), P1 and P2 (unusual thought
and non-bizarre ideas), N1 and N2 (anhedonia and avolition), and B2 and B3 (social
isolation and impaired role function). Across domains, a strong connection was
found between GP1 and N1 (depression and anhedonia). Interestingly, GP6 (OCS)
was indirectly connected to other CAARMS items mainly via GP5 (anxiety), B2
(social isolation), and E2 (observed blunted affect).
9.3.3 Strength and node-specific predictive betweenness measures

Figure 9.2 (left) illustrates the magnitude of the node-strength for each symptom.
In the entire network, GP1 (depression) was the most central symptom, significantly
stronger than most other nodes in the network. GP6 (OCS) was the least central
symptom, indicating that, as compared to other CAARMS symptoms, GP6 (OCS)
was not a well-connected node in the network. Figure 9.3 illustrates the node-specific
predictive betweenness values for each node in the network – i.e., how often a node
lies on the pathways between two other nodes, of which one is always the OCS.
The white dots represent the node-specific predictive betweenness in the current
sample, while the black lines represent the variability of the measure across 1000
nonparametric bootstrap iterations. Item GP5 (anxiety) had the highest node-
specific predictive betweenness score, followed by item E2 (observed blunted affect).
This finding suggests that anxiety may be a main connector between OCS and
other ARMS symptoms.
9.3.4 Robustness analysis

Robustness analyses (see Appendix F1) revealed some relatively wide bootstrapped
confidence intervals (CIs). We therefore recommend caution when interpreting the
network structure, especially when interpreting the strength and presence of weaker
edges. Nonetheless, the bootstrap mean was generally close to the sample mean,
and the bootstraps indicate interpretable results (see Figure F1 in Supplement F).
The strength centrality stability plot is shown in Figure 9.2 (right), while Figure
9.3 includes the variability of the node-specific predictive betweenness across 1000
nonparametric bootstrap iterations (i.e., black lines). The CS coefficient obtained for
strength was CS=.59. This is above the preferred .5 cut-off, indicating stable results.
For significance difference testing, the node with the highest strength centrality,
Page | 150
GP1 (depression), showed significantly higher node strength than most of the
other nodes (refer to Figure F3 in Supplement F for detail). Of note, correcting for
multiple testing when carrying out significance difference testing is not feasible, as
elaborately explained in the cited paper (Epskamp, Borsboom, et al., 2017). While
the functionality for calculating the CS coefficient and running significance
difference testing for node-specific predictive betweenness is not yet implemented
in the bootnet R package (Epskamp, Borsboom, et al., 2017), the bootstrap results
show node GP5 (anxiety) was consistently identified as the most central bridging
node, followed by node E2 (observed blunted affect). Further, in the additional
case-drop bootstrap analysis carried out (see Figure F4 in Supplement F), node GP5
remains visibly the main and most stable bridging node in the network. Markedly,
in general and aligned with previous research (Epskamp, Borsboom, et al., 2017;
Isvoranu et al., 2019), the node-specific predictive betweenness is unstable and
shows high variability; in spite of this, node GP5 (anxiety) stands out as a stable
bridging node both in the nonparametric and case-drop bootstraps.
Page | 151
Page | 152
Figure 9.1 Network structure of 21 CAARMS symptoms. Item groups are differentiated by color.The color of the edge indicates the size of the association (blue
for positive associations; red for negative associations).
Strength
Depression ●
Anhedonia ●
● strength
Avolition ●
●
Subjective cognitive change 1.0 ● ● ● ● ● ●
●
Social isolation ● ●
Anxiety ● ●
Emotional disturbance ●
0.5
Impaired role function ●
Disorganised speech ●
Aggression ●
Suicidality ● 0.0
Alogia ●
Non−bizarre ideas ●
Unusual thought ●
Average correlation with original sample

−0.5
Impaired tolerance to normal stress ●
Perceptual abnormalities ●
Mood swings ●
Observed cognitive change ● −1.0
Observed blunted affect ● 90% 80% 70% 60% 50% 40% 30%
Sampled cases
Dissociative symptoms ●
OCS symptoms ●
−2 −1 0 1 2
Figure 9.2 (left) Node-strength centrality for each CAARMS symptom and (right) stability of strength centrality. Centrality measures are shown as standard-
ized z-scores. The right panel indicates the average correlation with the original sample after reducing the sample size through case-dropping bootstrapping.
Page | 153
9.3.5 Shortest pathways: OCS to positive symptoms

Overall, all positive symptoms of psychosis (P1, P2, P3, P4) were indirectly
connected to GP6 (OCS) via other symptoms (Figure 9.4, left). Results of shortest
pathways analyses showed that the main connective pathways were through node
GP5 (anxiety) and through nodes E2 (observed blunted affect) and C2 (observed
cognitive change), directly linked to P4 (disorganized speech). Our robustness
analysis (Figure 9.4, right) showed that these were the most stable pathways. The
pathways from GP5 (anxiety) to the remaining positive symptoms P1, P2, and P3
were less stable and should thus be interpreted with caution, with multiple potential
connective components, most often including GP8 (impaired tolerance to normal
stress) and B1 (aggression) to P3 (perceptual abnormalities), and direct pathways
from GP5 (anxiety) to P1 (unusual thought). Other potential pathways could also
be identified, but since these were very small and, in most cases, present in less than
20% of the bootstraps, we do not highlight these.
15
●
10
Value
5
●
● ● ● ●
● ● ●
0 ● ● ● ● ● ● ● ● ● ● ● ●
GP1 GP2 GP4 GP5 GP6 GP7 GP8 P1 P2 P3 P4 N1 N2 N3 C1 C2 E1 E2 B1 B2 B3
Node
Figure 9.3 Node-specific predictive betweenness. The white dots represent the node-specific predic-
tive betweenness in the current sample, while the black lines represent the variability of node-specific
betweenness across 1000 nonparametric bootstrap iterations.
Page | 154
GP2 GP2
GP4 GP4
0.03
0.03
0
GP1 GP1 0.03
P3 0.01 P3
0 0
N1 N1 0.44 0.02
B1 B1
0.26
0.19 0.01
N2 N2
0 0.04
P1 0 0 P1
0.14 0.01
0
0 0 0.04 0.02
0.27 0.18
GP8 GP8 0.61
B3 B3
0
GP7 0.01 0.28
0.01 GP7
P2 0.06 0.01 P2
0.02 0.21
0 0.05 0 0
C1 0.01C10
0.01 0.18
B2 B2 0.17
GP5 GP5 0.01
0.01 0 0.01
0 0.15
0 0.01 0.2
0.01 0.06
E1 0.03 0.01
E1 0.14
0
0.04
0 0.13
P4 0.85 0.02 P4
0.01
N3 0 N3
00
GP6 GP6
0 0.92
0.01
0.71 0.24
E2 C2 E2 0.68 C2
Figure 9.4 (left) Network showing the shortest paths between the OCS and the positive symptoms of the CAARMS scale (P1, P2, P3 & P4) and stability
analyses (right). The right panel displays the extent to which the pathways were identified across 1000 nonparametric bootstraps (e.g., the 0.85 edge between GP6
and GP5 indicated the edges was identified in 85% of the bootstraps.
Page | 155
9.4 Discussion
The current study was the first to examine symptom-level associations of OCS
and other symptoms of the ARMS in individuals with different levels of risk
for psychosis, and the first to evaluate the centrality measures and bridging
characteristics of symptoms. Overall, depression was found to be the most central
symptom in terms of direct connectivity, while anxiety was the main node linking
OCS to other symptom clusters. While OCS did not occupy a central position in
the network, they were directly connected to social isolation, anxiety, and blunted
affect, and via anxiety and blunted affect to positive CAARMS symptoms. In
general, connections between symptoms within each domain were stronger than
connections between domains.
In the current study, comorbid depression (27.8%) and anxiety disorders
(38.8%) frequently occurred. The high centrality of depression and bridging
characteristic of anxiety suggests that these variables might be essential in their
influence on other symptoms of the network. The role of affective symptoms in
UHR subjects has been of increased research interest. Meta-analyses reported that
over 50% of UHR individuals fulfilled the criteria for a co-occurring depression
or anxiety disorder (Albert et al., 2018; Fusar-Poli et al., 2017). Both depression
and anxiety have been found associated with higher symptom severity of UHR
symptoms and poorer global functioning (Fusar-Poli et al., 2014; Rutigliano et
al., 2016). Fusar-Poli and colleagues (2012) concluded that affective disorders
are likely to impact on the overall longitudinal outcome of UHR subjects albeit
no associations have been found with transition to psychosis (Albert et al., 2018;
Fusar-Poli et al., 2014). Co-occurring OCD in 9.7% of the current UHR subjects
exceed previously reported mean estimates of 5.5% in subjects at increased risk
for psychosis (Zink et al., 2014), but high variability between studies has been
reported (Albert et al., 2018). Some previous studies reported associations
between OCS and severity of psychotic symptoms on the syndrome level in UHR
subjects (Niendam et al., 2009) and psychotic experiences in individuals from
the general population (van Dael et al., 2011).
However, the majority of studies did not find associations with positive
symptom severity (Fontenelle et al., 2011; Soyata et al., 2018; Zink et al., 2014)
and a review of the literature does not show an increased or decreased risk for
transition to psychosis in UHR subjects with OCS/ OCD (Albert et al., 2018).
In the current CAARMS network OCS were not identified as central symptoms,
Page | 156
nor directly linked to positive symptoms. Low centrality might be explained by the
content of the item, which is a more standalone item, compared to most other items
that are part of clearly structured subdomains and show stronger within-domain
associations. These associations might thus drive other items to be more central,
including other general psychopathology items (e.g., depression and suicidality are
very strongly related to each other). In addition, while other items are associated
with a wide array of items from multiple subdomains, the OCS node has less and
more well-defined associations, resulting in lower centrality, but suggesting more
clear pathways. Direct associations with social isolation can be explained by often
observed avoidance and social withdrawal in individuals with OCD, while social
isolation has also been identified as a predictor for later development of OCD
(Grisham et al., 2011). Shortest paths analyses furthe showed that OCS were rather
indirectly related to symptoms of the positive cluster, via anxiety and blunted affect
coupled with observed cognitive change.
One possible interpretation of these interrelations suggests that neurobiological
alterations–such as aberrant activation of the dopamine system and associated
aberrant salience assignment to irrelevant stimuli–may cause uncertainty and
consequently anxiety and emotional distress (van Os & Kapur, 2009). In an attempt
to make sense of these experiences and in combination with cognitive alterations
such as a tendency to jump to conclusion, psychotic symptoms might develop.
Similarly, Szechtman and Woody (2006) emphasized feelings of uncertainty
accompanied with feelings of insecurity (disturbances in the so-called security
motivation system) as important underlying mechanisms of OCS and OCD as
an attempt to gain control and reduce distress. Another aspect of the aberrant
salience model, namely failing to appropriately respond to meaningful reward
cues and diminished reward processing, has been directly linked to anhedonia
and blunted affect (Whitton et al., 2015). These negative symptoms are closely
associated with cognitive deficits (Harvey et al., 2005; Mcintyre et al., 2016), which
again are conceptually related to disorganization in psychotic disorders (Feigenson
et al., 2014). More recently studies suggest that reward dysfunctions and associated
blunted affect might also constitute a neuropathological mechanism in OCD
(Pushkarskaya et al., 2019). These findings might explain the connecting role of
blunted affect in the network and account for the frequent co-occurrence of OCS
and psychotic symptoms (Alves-Pinto et al., 2019). In addition to shared underlying
neurobiological alterations, causal interactions on the symptom level might also be
assumed. Psychotic-like experiences could lead to increased anxiety, rumination,
Page | 157
worries and doubt, which are strongly related to obsessive thoughts and might
be associated with a subsequent attempt to reduce the resulting anxiety through
compulsive behavior. In this line, early psychopathological concepts speculated
that sometimes OCS might be seen as an attempt to control for the feeling of self-
disintegration (Bleuler, 1911).
In support of this hypothesis, one study found inverse associations between
OCS and disorganization (Guillem et al., 2009). However, more recent research
associated co-occurring OCS with dysfunctional coping tendencies (Renkema
et al., 2020), increased general psychopathology and does not support direct
associations with increased or decreased severity of positive symptoms, including
disorganization (Zink et al., 2014). The link between increased anxiety, worry and
fear and the co-occurrence of OCS and psychosis has also been reported on the
trait level. Schreuder and colleagues (2017) found increased neuroticism in patients
with psychosis who reported comorbid OCS compared to those without OCS.
Exploratory analyses suggested a mediating effect of neuroticism on the course of
OCS severity over time. To disentangle hypothesized causal pathways, analyses of
prospective data are necessary.
Taken together, current findings indicate the importance of affective
symptoms and their direct relations and interactions with other symptoms of
the CAARMS. Noteworthy, in a previous network study anxiety was found to
be the main connective component between experienced traumatic events and
psychotic symptoms (Isvoranu et al., 2017; see Chapter 5). Based on these findings
it seems advisable to not only focus treatment on psychotic experiences in UHR
subjects, but extend the focus to emotional disturbances. Interventions aimed
to reduce experienced anxiety and distress could be important treatment targets
as they seem to play a central role in the network and might be crucial for the
development of symptoms.
9.4.1 Limitations
The current findings should be interpreted considering some limitations. First,
while our robustness analysis indicates that the resulting network and centrality
indices are interpretable, this should be done with some caution, especially for
weaker links in the network and centrality measures.
Second, the current study is exploratory, and interpretation of results are
hypothetical in need for further investigations. Moreover, the current analysis was
based on cross-sectional data; thus, it is not possible to infer causality, such as the
Page | 158
direction of edges. Future research should consider longitudinal data to specify
symptoms that actively trigger other symptoms as indicators of the prospective
development of the at risk mental state for psychosis. Unfortunately, due to missing
data and the relatively small sample size, we were not able to explore prospective
symptom interaction. Along this line, we also chose to only include CAARMS
intensity scores in the current study. Future studies in larger sample should integrate
information on frequencies of occurring symptoms.
Third, we included both healthy and UHR subjects in our analysis to evaluate
subclinical symptom networks across the risk spectrum for psychosis. This approach
led to increased power and accounted for a broad distribution of symptomatic
expression at the subclinical level. Nonetheless, we included within Supplement F a
network structure constructed in the UHR sample only, which shows comparable
results, with the note that many of the connections were weaker or missing,
indicating a significant loss in power. The edges in the UHR sample only network
were mainly–and as expected–a subset of the edges in the overall network structure.
Sample sizes were too small to incorporate network comparison tests, investigating
whether network structure or centrality measures would be different between
groups. Analyses with large enough samples of non-help seeking controls, UHR
subjects, as well as first episode patients would be desirable.
Forth, findings are limited by OCS assessment with one item of the CAARMS.
Future studies should include state of the art instrument to assess OCS such as the
Yale Brown Obsessive and Compulsive Scale (Goodman et al., 1989). In addition,
the majority of individuals included in the current study is Caucasian/ White.
Replication in a more racially diverse sample is necessary to investigate to which
extent findings generalize to non-Caucasian populations.
Finally, in an attempt to ensure more stable network estimates and avoid
potential biased results (e.g., due to severe violations of normality assumption), we
reduced the number of nodes in the network structure by excluding low variability
items from the network structures. This led to the exclusion of clinically interesting
dimensions such as mania or motor symptoms, which could have contributed
to knowledge on the interdependence of mixed affective symptoms and motor
symptoms with symptoms of obsession, compulsion, anxiety, and psychosis. For the
interested reader we included an extended network structure with all 28 CAARMS
items in Appendix F3 in Supplement F. Of note, the network structure is for
reference only and should be interpreted with the utmost caution and replication
in larger samples is needed. All the main links identified and discussed in the
Page | 159
current manuscript are retained in the extended network structure, including the
associations between OCS and other symptoms.

The current chapter aimed to expand on our understanding of the symptom-level
association between OCS and other symptoms of the ARMS, in a large sample
of individuals with different levels of risk for psychosis, using novel network
models. Our results did not reveal a central role of OCS in the network, nor direct
associations with positive symptoms, but indirect associations mainly via anxiety.
In addition, results suggested a central role of depressed mood in the symptom
network. In sum, these findings point to the critical role of affective symptoms in
the network of the ARMS in individuals with different levels of risk for psychosis.
Page | 160
Page | 161
Autistic Symptoms as
10
Linking Pin to Social
Functioning in Psychosis

Isvoranu, A. M.*, Ziermans, T.*, Schirmbeck, F., Borsboom, D., Geurts, H., de
Haan, L., GROUP Investigators (2021). Autistic Symptoms as Linking Pin to
Social Functioning in Psychosis: A Network Approach. Schizophrenia Bulletin.

Chapter 10 | Autistic Symptoms and Social Functioning in Psychosis
Abstract
Psychotic and autistic symptoms are related to social functioning in individuals
with psychotic disorders (PD). The present chapter used a network approach to
a) evaluate the relationship between social functioning and autistic or psychotic
symptoms and b) investigate whether relations are similar in individuals with and
without PD. We estimated an undirected network model in a sample of 504 PD,
572 familial risk for psychosis (FR), and 337 typical comparisons (TC), with a mean
age of 34.9 years. Symptoms were assessed with the Autism Spectrum Quotient
(AQ; 5 nodes) and the Community Assessment of Psychic Experiences (CAPE; 9
nodes). Social functioning was measured with the Social Functioning Scale (SFS;
7 nodes). We identified statistically significant differences between the FR and
PD samples in global strength (p<.001) and in network structure (p<.001). Our
results show that autistic symptoms are more negatively and more closely related
to social functioning than psychotic symptoms, particularly with regard to social
interactions. More and stronger connections between nodes were observed for the
PD network than for the FR and TC networks, while the latter two were similar
both in density (p=.11) and network structure (p=.19). The most central items in
terms of strength for PD were bizarre experiences, social skills, and paranoia. Presence
of autistic symptoms is negatively associated with social functioning across different
psychosis vulnerability, but in the PD network symptoms may reinforce each other
more easily. These findings emphasize the need for increased clinical awareness of
comorbid autistic symptoms in individuals with a psychotic disorder.
Page | 164
10.1 Introduction
Psychotic disorders (PD) are typically accompanied by impairments in social
functioning, which are often persistent and resilient to therapeutic interventions
(Oorschot et al., 2012; Robinson et al., 2004; Wykes et al., 2008). For many
people with a PD diagnosis and their relatives, improvement in the social domain
is one of the most preferred outcomes of treatment (de Crescenzo et al., 2019),
therefore we need a better understanding of the nature of social dysfunction in PD.
In light of this, there has been renewed interest in symptomatic overlap between
PD and autism spectrum disorders (ASD), which has unearthed more robust data
illustrating the increased co-occurrence of diagnoses and symptoms within both
spectra (Barlati et al., 2019; Chandrasekhar et al., 2020; Chisholm et al., 2015; de
Crescenzo et al., 2019; Deste et al., 2018; Kincaid et al., 2017; Lugnegard et al.,
2015; Sullivan et al., 2013). Furthermore, for individuals with PD, it has become
abundantly clear that the presence of comorbid autistic symptoms can have a
substantial impact on social cognition and social functioning in individuals with
PD (Bechi et al., 2020; Borsboom & Cramer, 2013; Deste, Vita, Penn, et al., 2020;
Vaskinn & Abu-Akel, 2019; Ziermans et al., 2020). Here we argue that a better
understanding of relations between specific trait clusters is essential to determine
appropriate intervention targets.
Network models are relatively novel statistical tools in psychiatric research
that grew popular in the past decade, as they allow zooming into interactions
at the symptom level.1 The appeal of the network approach is that it can
overcome limitations of more traditional research strategies by conceptualizing
psychopathology as mutually interacting elements of a complex network with
many features (e.g., symptoms, biological components, environmental risk factors
(Borsboom, 2017; Borsboom et al., 2019; Borsboom & Cramer, 2013; Fonseca-
Pedrero et al., 2018; Isvoranu et al., 2019; Isvoranu, Boyette, et al., 2020). Network
analysis can provide different types of information on symptom relations, such
as visual representation of the overall network structure, clustering, as well as
the relative centrality of specific symptom clusters or ‘nodes’ (i.e., which nodes
are connected to most other nodes, as well as which nodes connect specific
clusters). In addition, network models have been argued to be especially useful
when investigating links between different domains of interest (e.g., comorbid
1 For clarity, the term ‘symptom(s)’ is used here regardless of whether the underlying assessments are
perceived as more indicative of traits or state-like symptoms.
Page | 165
features, risk factors and symptomatology; Boschloo et al., 2015; Cramer et al.,
2010; Isvoranu, Boyette, et al., 2020), as they may allow pinpointing specific
bridging components (e.g., nodes, links), which may provide useful information
in identifying suitable treatment targets.
Although recent studies have looked into symptom network structures in PD
populations (van Rooijen et al., 2017), as well as the impact of potential etiological
(Isvoranu et al., 2017, 2019) and comorbid features (van Rooijen et al., 2018),
none have included the relative impact of comorbid autistic features. Zhou and
colleagues (2019) carried out a meta-analysis and network analysis (N=2469 college
students) of relations between schizotypal and autistic symptoms. The outcome
showed that negative symptoms were strongly correlated with autistic social/
communicative symptoms, whereas positive symptoms were inversely correlated
with autistic symptoms. However, it is unclear to what extent these findings in
student populations translate to more heterogeneous non-clinical or clinical
populations and whether the co-occurrence of symptoms can have an incremental
impact on outcome.
The main aim of the present chapter was therefore to use a network approach
to identify specific nodes within the autism-psychosis spectrum that may play a
pivotal role in determining the quality of social functioning in PD. In addition,
following the idea that both psychosis and autism may represent broad spectrum
phenotypes that can also affect individuals at a sub-clinical level (Wüsten et al.,
2018), we investigated to what extent such network characteristics are similar in
individuals with different levels of risk for psychosis: individuals with a familial
risk for psychosis (FR), who share a genetic and environmental liability to psychosis
due to their relatedness and physical proximity to an individual with a psychotic
disorder, and typical comparisons (TC) drawn from the general population with
a default risk for psychosis. In line with the previous study by Ziermans and
colleagues (2020), who established in the same sample that autistic symptoms in
general can explain more variance in social functioning for PD than psychotic
symptoms, we expected that autistic symptom clusters would be more closely related
to social functioning clusters than psychotic symptom clusters. Furthermore,
aligned with previous research showing more densely connected networks for
clinical populations (van Borkulo et al., 2016; van Rooijen et al., 2018; Wigman
et al., 2015) we expected that relations within the PD network would be stronger
than for FR and TC networks.
Page | 166
10.2 Methods
10.2.1 Sample characteristics
We used data from the Genetic Risk and Outcome of Psychosis (GROUP) study,
data release 6.0. GROUP is a multi-site longitudinal observational study carried
out between April 2004 and December 2013, investigating vulnerability and
protective factors that may influence the onset and course of psychotic disorders.
Data was collected from a consortium of four university psychiatric centers:
AMC Amsterdam, UMC Groningen, UMC Utrecht and Maastricht University.
At baseline, full data included 1120 PD, 1057 siblings, 919 parents and 590 TC.
For further information about the sample please refer to the paper by Korver and
colleagues (2012).
Current analyses included a subset of the full data, from the third data collection
time point (i.e., six-year follow-up) and from individuals who completed all
measures described below: 504 PD, 572 FR (siblings only) and 337 TC participants,
of which 53.9% were male, with an age range between 21 and 63 years, and a mean
age of 34.9 years (SD=8.65), and a mean IQ of 108 (abbreviated IQ based on
Arithmetic, Block Design, Digit Symbol-Coding, and Information of the Dutch
version of the Wechsler Adult Intelligence Scale, Third Edition; van der Heijden
et al., 2013). The majority of participants were Caucasian (88.04%), followed by
Mixed ethnicity (6.65%), Surinamese (1.49%), Moroccan (1.06%), Turkish (0.71%),
Other (0.64%), Asian (0.14%), Antillean (0.07%), while for 0.35% of the sample
it was unknown. Of note, the sample is the same sample previously analyzed by
Ziermans and colleagues (2020).
10.2.2 Materials
All subjects completed three measures: 1) the Autism Spectrum Quotient, 2) the
Community Assessment of Psychic Experiences, and 3) the Social Functioning
Scale, as well as demographic attributes. For brevity only a short description is
provided here, and we refer to the Ziermans et al.’s study (2020) and their validation
papers (Baron-Cohen et al., 2001; Birchwood et al., 1990; Konings et al., 2006)
for a more detailed description. The use of these questionnaires resulted in a total
of 23 variables included in the network model.
Page | 167
Autistic symptoms
The Autism Spectrum Quotient (AQ; Baron-Cohen et al., 2001) is a measure
of autistic features in individuals. It is a brief, self-administered questionnaire
consisting of 50 items. The items are rated on a four-point scale (1= definitely agree
to 4= definitely disagree) and divided into five 10-item subcategories: social skills,
attention switching, attention to detail, communication, and imagination. Items
were scored in the traditional manner, i.e., recoded into dichotomous scores with
a range of 0-50. A higher score indicates a higher level of autistic features in the
specified area.
Psychotic symptoms
The Community Assessment of Psychic Experiences (CAPE; Konings et al.,
2006) is a measure aimed at detecting psychotic-like experiences. It consists of
42 items, rated on a four-point frequency scale (1= never to 4= nearly always) and
a subsequent distress scale (not included in this study). As this assessment was
conducted for a follow-up assessment, the reference period for reporting experiences
was the past 3 years. The CAPE items correspond to three different symptom
dimensions: 1) positive symptoms, 2) negative symptoms, and 3) depression.
Symptom dimensions can be further subdivided into nine clusters of psychotic-
like experiences based on evidence from factor-analytic studies (Wüsten et al.,
2018). These are positive symptoms: bizarre experiences (7 items), hallucinations (4
items), paranoia (5 items), grandiosity (2 items), magical thinking (2 items); negative
symptoms: social withdrawal (4 items), affective flattening (3 items), avolition (7
items); and a single cluster for depression (8 items). A higher score on any dimension
indicates a higher level of symptomatology.
Social functioning
The Social Functioning Scale (SFS); (Birchwood et al., 1990) is an assessment
of social functioning within the past three months, specifically designed to the
needs and impairments of individuals with schizophrenia. The SFS consists of
seven different dimensions: withdrawal (5 items), interaction (4 items), prosocial
(22 items), independence performance (13 items), independence competence (13
items), recreation (15 items) and occupation (max. 8 (conditional) items). A score
key is used to calculate raw scores, and a higher score indicates a higher level of
social functioning on all subscales.
Page | 168
10.2.3 Network analyses
We carried out all analyses in the R statistical software (R Core Team, 2015), version
4.0.2, using the R packages bootnet (Epskamp, Borsboom, et al., 2017) version 1.4.3,
qgraph (Epskamp et al., 2012) version 1.6.5, and networktools (Jones, 2017) version
1.2.3. Due to the low number of missing data for individual items (under 1% for
the three scales), missing data were imputed using the R package mice (van Buuren
& Groothuis-Oudshoorn, 2011) version 3.11.0, to allow retaining all available
data in the calculation of the scales. The overall dimensions of the three scales, as
described above, and age and estimated IQ as covariates, were used as nodes in the
network. An edge between two nodes was used to represent a partial correlation
between the nodes, while controlling for all other nodes in the network (Epskamp,
Waldorp, et al., 2018). Blue edges were used to illustrate positive associations, red
edges negative associations, and the wider and more saturated the edge, the stronger
the association.
Network estimation
To estimate the network models, we fitted Gaussian graphical models (GGMs;
i.e., undirected network structures) in each group separately. To ensure sparse and
interpretable models, we used the least absolute shrinkage and selection operator
(LASSO; Tibshirani, 1994) statistical regularization technique. Specifically, we
employed the graphical LASSO algorithm (Friedman et al., 2008), which further
uses the Extended Bayesian Information Criterion (EBIC) model selection, with
a tuning hyperparameter set to .5, which was shown to return accurate network
estimations (van Borkulo, Borsboom, et al., 2014) and has now been widely used
when estimating network structures across various research domains (Abacioglu
et al., 2019; Birkeland et al., 2020; Isvoranu et al., 2017). Missing data for the
individual items included in the network were handled using pairwise estimation
and Spearman correlations were used when estimating the network structure, as
recommended for nonnormality distributed data (Isvoranu & Epskamp, 2021; see
Chapter 13). The network layout was matched using the average of the Fruchterman
and Reingold layout algorithm (Fruchterman & Reingold, 1991; i.e., each network
was first plotted individually and then the layout was averaged), which places the
nodes with stronger connections closer to the center of the network.
Page | 169
Network comparison
Network structures were constructed for control, family, and patient data. To
assess the differences between samples we used the Network Comparison Test
(NCT; van Borkulo et al., 2017). The NCT is a permutation test that allows
investigating differences in the global strength of the networks, their structure, as
well as differences in individual edges and centrality measures. Of note, we only
investigated differences in edges when either the global structure or global strength
was identified as significantly different. Due to the conservative nature of the NCT,
we report results both with and without a Bonferroni correction.
Centrality estimation
Centrality indices (Opsahl et al., 2010) were computed to investigate which nodes
display the highest connectivity and play a central role in bridging the different
dimensions. Specifically, we computed strength (i.e., a measure of how well
connected a node is) and bridge strength (i.e., a measure of how well a node connects
to other clusters) as centrality measures. These measures were chosen as they were
well-aligned with our research aims, while still expected to be robust according to
previous research on network robustness (Epskamp, Borsboom, et al., 2017).
Robustness analysis
Finally, we conducted a robustness analysis, to assess the stability and accuracy of
our results, as recommended by Epskamp and colleagues (2017). More information
about this analysis is available in Appendix G in the Supplement G.
10.3 Results
10.3.1 Network analysis & comparison
Table G1 in the Supplement G presents the means and standard deviations of all
individual items. No significant differences were identified between the FR sample
and the TC sample in terms of global strength (p=.11) and network structure
(p=.19), and therefore we focus our results on the comparison between the FR
sample and the PD sample, of which sample sizes are higher and more comparable.
Figure 10.1 below presents the network structures a) for the FR sample, b) for the
PD sample, c) highlighting edge and centrality differences between the two networks
Page | 170
as identified by the NCT.2 Overall, within both networks, negative symptoms and
depression were more strongly linked to positive psychotic symptoms, while autistic
symptoms were more closely related to social functioning than psychotic symptoms.
The items social skills (A1), attention switching (A2), and communication skills (A4)
were negatively linked to withdrawal (S1) and interpersonal behavior (S2), the latter
being coded in the opposite direction of traditional symptoms (i.e., a higher score
indicating better functioning). For both networks, the item attention to detail (A3)
did not cluster well with the remaining autism nodes, and instead was more closely
related to the CAPE positive psychotic symptoms and depression. Within both
networks, communication skills (A4) were linked to paranoia (P3), while attention
switching (A2) was linked to bizarre experiences (P1) and depression (D).
Of note, we identified statistically significant differences between the FR and
PD samples both in global strength (p<.001) and in network structure (p<.001).
Furthermore, we identified several statistically significant differences between edges
(see Table 10.1) and centrality measures of the two network structures (see Table
10.2). Especially within the same domain, relations were more connected for the
PD network, as more and stronger connections between nodes were observed than
for the FR network. All links between autism spectrum items were stronger for
the PD networks, as well as within the social functioning scale and the psychotic
experiences scales. Within the PD network, the item paranoia (P3) was positively
linked to recreational activities (S6), while the item grandiosity (P4) was negatively
linked to attention switching (A2) and imagination (A5).
2 Figure 10.1c includes two network structures, as to allow pinpointing where specifically the differ-
ences lie (e.g., in the presence/absence of edges, as would be the case with the edge between IQ and
C9, which is present in the PD sample only, or in the strength of the edges, as would be the case with
for instance the edge between C87 and C8, which is present in both networks).
Page | 171
= 572 b) PD sample, n = 504
S7 S7
A3 A3
S5 S5
Page | 172
N3 S4 N3 Covariantes
S1
a) FR sample, n = 572 b) PD sample, nS1= 504
Age: Age
D S7 S7 D IQ: IQ
N1 A3 A3 N1
N2 N2
S2 S6 S2
S5 S3 S5 Autism Spectrum Quotient
S4 N3 S4 N3 Covariantes
A1: Social skills
S1 S1
Age: Age
A1 A1 A2: Attention switching
P3 D P3 D IQ: IQ
N1 N1
A2 N2
A2 N2 A3: Attention to detail
S6 S2 S6 S2
S3 S3
Autism
A4: Spectrum Quotient
Communication skills
P4 P4 A1: Social skills
A4 A4 A5: Imagination
A1
Age A1 A2: Attention switching
P3 P3
P1 A2 A2 P1 A3: Attention to detail
A5 A5
CAPE Positive Symptoms
A4: Communication skills
P4 P4
A4 A4 Imagination
A5: Bizarre
P1: experiences
Age Age
P5 P5
A5 P1 A5 P1 P2: Hallucinations
IQ IQ CAPE Positive Symptoms
P2 P2
P3: Bizarre experiences
P1: Paranoia
Maximum: 0.41 P5 Maximum: 0.41
P5
IQ IQ
P4: Hallucinations
P2: Grandiosity
P2 P2
P3: Magical
P5: Paranoia thinking
P4: Grandiosity
c) Edge Differences P5: Magical thinking

CAPE Negative Symptoms
c) Edge Differences
S7 S7 N1: Social withdrawal

S7 S7 N1: Social withdrawal
A3 A3 N2: Affective flattening
A3 A3 N2: Affective flattening
N3: Avolition
N3: Avolition
S5 S5 S5 S5
N3 N3 CAPE Depression
S4 S4 CAPE Depression
N3 S1 S4 S1
N3
D: Depression
S1 S1 D: Depression
D D
N1 N1
D N2 D N2 Social Functioning
S6 S2
N1 S6 S2 N1
S3 S3 S1: Withdrawal
N2 N2 Social Functioning
S2 S6 S2
S2: Interpersonal behavior
S3 S1: Withdrawal
S3: Prosocial activities
A1 P3 A1 P3
A2 A2
S2:
Independence performance
S5: Prosocial
S3: competence
Independenceactivities
A1 P3 A1 P3
P4 P4 S6: Recreational activities
A2 A4 A2 A4 S4: Independence performance
Age Age
S7: Occupation/ employment
A5 P1 A5 P1 S5: Independence competence
S3 S3
Autism Spectrum Quotient
A1: Social skills
P4 P4
A4 A1 A4 A1
A5: Imagination
A2: Attention switching
P3 Age P3
A2 A2 A3: Attention to detail
A5 P1 A5 P1
P4 P4
CAPE Positive Symptoms
A4 A4 A5: Imagination
Age Age P1: Bizarre experiences
A5 P5 P1 A5 P1P5
CAPE
P2: Positive Symptoms
Hallucinations
IQ IQ
P2 P2 P1: Bizarre experiences
P3: Paranoia
P5 P5
Maximum: 0.41 Maximum: 0.41 P2: Hallucinations
IQ IQ P4: Grandiosity
P2 P2
P3: Paranoia
P5: Magical thinking
P4: Grandiosity
P5: Magical thinking
c) Edge Differences
c) Edge Differences CAPE Negative Symptoms
S7 S7
S7 S7
N1: Social withdrawal
N1: Social withdrawal
A3 A3 A3 A3 N2: Affective
N2: Affective flattening
flattening
N3: Avolition
N3: Avolition
S5 S5
S5 S5
N3 N3 CAPE Depression
S4 S4
N3 S1 S1 N3 CAPE Depression
D: Depression
S4
S1 D S1 D D: Depression
N1 N1
S6 D S2 S6 S2 D
S3 N1 S3 N1 S1: Withdrawal
S2 S6 S2
3 S3 S3: Prosocial
S1: Withdrawalactivities
A1 P3 A1 P3
A2 A2 S4: Independence
S2: performance
Interpersonal behavior
S5: Independence competence
P4 P4
S3: Prosocial activities
A1 P3 A1 P3 S6: Recreational activities
A4 A4
Age Age S4: Independence performance
A2 A2 S7: Occupation/ employment
A5 P1 A5 P1
S5: Independence competence
P4 P4 S6: Recreational activities
A4 P5 A4 P5
IQ Age IQ
P2 P2 S7: Occupation/ employment
A5 P1 Maximum: 0.41 A5 P1 Maximum: 0.41
P5 P5
IQ
Figure 10.1P2Network structure a) for the FR sample (left panel); b) for theIQPD sample (right panel);
P2 c) highlighting significant edge and centrality differences
between the FR sampleMaximum:
and PD0.41sample (bottom panel). A grey and wider border around a node (bottomMaximum: 0.41
panel) indicates a significant difference between the two
groups in strength centrality for that node (i.e., a higher value in the group where the border is present). Item groups are differentiated by color. The dashed edges
in the background (bottom panel) indicate the presence of other edges in the network, which were not identified as significantly different between the networks.
The color of the edge indicates the size of the association (blue for positive associations; red for negative associations).
Page | 173
Table 10.1 Significant edge differences between FR and PD samples

Variable 1 Variable 2 p value
Social Skills Attention Switching .007
IQ Imagination .044
Social Skills Imagination .047
Bizarre Experiences Hallucinations <.001*
Attention Switching Grandiosity .011
Imagination Grandiosity .008
Paranoia Grandiosity .006
Bizarre Experiences Magical Thinking .001
Grandiosity Magical Thinking .001
Bizarre Experiences Social Withdrawal .026
Communication Skills Affective Flattening .039
Affective Flattening Avolition .011
IQ Depression .008
Attention to Detail Depression .048
Avolition Prosocial Activities .007
Withdrawal Prosocial Activities .013
Imagination Independence Performance .036
IQ Independence Competence .049
Independence Performance Independence Competence <.001*
Paranoia Recreational Activities .003
Prosocial Activities Recreational Activities .008
Imagination Occupation Employment .002*
Withdrawal Occupation Employment .002
Prosocial Activities Occupation Employment .024
Independence Competence Occupation Employment <.001*
*Remained significant after Bonferroni correction
Page | 174
Table 10.2 Significant strength centrality differences between FR and PD samples
Variable p value
IQ <.001*
Social Skills .002*
Attention Detail <.001*
Imagination <.001*
Bizarre Experiences <.001*
Hallucinations <.001*
Paranoia <.001*
Grandiosity <.001*
Magical Thinking <.001*
Depression .033
Interpersonal Behavior .008
Prosocial Activities .028
Independence Performance <.001*
Independence Competence <.001*
Recreational Activities .049
Occupation Employment <.001*
*Remained significant after Bonferroni correction
10.3.2 Centrality analysis

Figure 10.2 illustrates the node strength for each node included in the network,
both for the FR and the PD samples. The three most central items in terms of
strength for the FR sample were avolition (N3), social withdrawal (N1), and
depression (D), while the three most central items in terms of strength for the PD
sample were bizarre experiences (P1), social skills (A1), and paranoia (P3).
10.3.3 Robustness analysis

The results of the robustness analyses are presented and described in Supplement
G, Appendix G1. Generally, the results show high stability both for the FR and the
PD network structures.
Page | 175
Strength: FR Strength: PD
1.25
1.00
0.75
0.50
0.25
0.00
Age IQ* A1* A2 A3* A4 A5* P1* P2* P3* P4* P5* N1 N2 N3 D* S1 S2* S3* S4* S5* S6* S7*
Figure 10.2 Strength centrality for each node included in the network. The orange line indicates
strength centrality for the FR sample, while the purple line indicates strength centrality for the PD
sample.
Bridge Strength: FR Bridge Strength: PD
0.6
0.4
0.2
0.0
Age IQ A1 A2 A3 A4 A5 P1 P2 P3 P4 P5 N1 N2 N3 D S1 S2 S3 S4 S5 S6 S7
Figure 10.3 Bridge strength centrality for each node included in the network. The orange line
indicates bridge strength centrality for the FR sample, while the purple line indicates bridge strength
centrality for the PD sample.
Figure 10.3 illustrates the bridge strength for each node included in the network,
both for the FR and the PD samples. The three most central items in terms of bridge
strength for the FR sample were withdrawal (S1), social skills (A1), and attention
switching (A2), while the three most central items in terms of bridge strength for
the PD sample were IQ, imagination (A5), and social skills (A1).
Page | 176
10.4 Discussion
The current study presents a bird’s-eye view of relations between autistic and
psychotic symptom clusters, and social functioning in individuals with PD, FR, as
well as TC. The main aims were to identify specific nodes of particular importance
to social functioning in PD and to compare symptom networks across groups.
Our analyses revealed several important features. First, across all group networks,
all autistic nodes, except attention to detail, were more closely related to nodes
reflecting frequency of social interactions than psychotic symptom nodes; second,
the FR an TC networks were highly similar in terms of global strength and network
structure, prompting further focus on PD and FR network comparisons only; third,
the PD network was generally characterized by stronger and more connections
than the FR network, but only few individual edges differed significantly after
Bonferroni correction; and fourth, centrality measures indicated that the relative
importance of single nodes may differ between the PD and FR networks. The
implications of these findings are discussed further in the paragraphs below.
In a previous theory-driven study of this sample, it was established that autistic
symptoms in general can explain more variance in social functioning for PD than
psychotic symptoms (Ziermans et al., 2020). The current network analyses add
a purely data-driven approach to the same data, which takes into account many
more relations between symptoms to create a more realistic image of how specific
symptom clusters may interact with each other. By doing so, we were able to show
that, as expected, for both the FR and PD networks, autistic symptom clusters
were more directly related to social functioning nodes than psychotic symptom
clusters, in particular for ‘Withdrawal’ and ‘Interpersonal behavior’. These SFS
subscales are heavily based on frequencies of social contact with people outside of
the home situation and, to a lesser extent, the perceived quality of these contacts.
In addition, Schneider and colleagues (Schneider et al., 2017) have demonstrated
with experience sampling method (ESM) data that these two scales are reflective
of social isolation and avoidance in daily life of PD individuals, though not of a
lack of interest in socializing. Together these findings suggest that standardized
assessment of autistic symptoms could be a helpful indicator of social vulnerabilities
and resilience in PD, in line with previous findings (Deste, Vita, Nibbio, et al.,
2020; Harvey et al., 2019; Vaskinn & Abu-Akel, 2019). Furthermore, it is also
striking that most positive psychotic symptom clusters were not strongly related
to social functioning, with the exception of ‘Paranoia’, which was more strongly
Page | 177
and positively associated with ‘Recreational activities’ in PD. The positive nature
of this association may strike one as counterintuitive, but ‘Recreational activities’
also includes many items that one can initiate in solitude, such as reading, watching
tv, or listening to the radio. Speculatively, these may represent activities that an
individual may be more prone to initiate in a paranoid state of mind. However,
there is also evidence that people with a prior diagnosis of psychosis spend equal
or more time socializing in online chat rooms and online gaming than controls
(Highton-Williamson et al., 2015; Jakubowska et al., 2019), which may also be
reflected by this association.
One of the general objectives of the GROUP project (Korver et al., 2012) is
to investigate vulnerability factors contributing to the expression of psychosis. As
such, the FR group of unaffected siblings in this study represents a high-risk group,
with shared genetic makeup and family environment, that can provide information
on (relations between) symptoms associated with imminent psychosis. However,
the current FR network structure was highly similar to the TC network. This
could suggest that interrelations between autistic, psychotic and social functioning
features are either a poor indicator of latent manifestations of psychotic disorders, or
they may only become clinically relevant for psychosis onset in high-risk individuals
if other (mediating) factors are entered into the equation. There is some evidence
in favor of the latter, as a recent network study based on data collected with ESM
showed that inclusion of a ‘stress’ node resulted in a higher number of network
connections, with stress having a central position in the network structure across
TC, FR, and PD groups, and also showing direct connections with subsequent
psychotic experiences (Klippel et al., 2018). Alternatively, the current similarity
between the two network structures could result from the relatively small sample
size of TC compared to the FR group, leading to insufficient power to detect
effects. Future studies including high-risk samples should aim to replicate current
findings and are encouraged to incorporate known risk factors, such as distress and
childhood trauma (Isvoranu et al., 2016, 2017) into network structures.
Given its global similarity with the TC network and the larger and more
comparable sample size, the FR network was subsequently used for network
comparisons with PD. In line with our expectation, the PD network was denser
than the FR network, with more connections identified especially between nodes
of the same domain. Denser network structures have been previously reported for
PD (Klippel et al., 2018), and more so for non-remission compared to remission (van
Rooijen et al., 2018), but also for other clinical patient populations (van Borkulo
Page | 178
et al., 2016; Wigman et al., 2015). Presumably, and in line with network theory
(Borsboom, 2017), this could reflect the capacity of symptoms to reinforce each
other much more steadily, resulting in a higher level of vulnerability (van Borkulo
et al., 2016), e.g., the occurrence of high frequent hallucinations may more easily
trigger an increase of other positive symptoms such as thought interference (as
indicated by the ‘bizarre experiences’ node, in some studies referred to as ‘delusions’;
Therman & Ziermans, 2015). Alternatively, highly clustered nodes could also be
indicative of the presence of latent variables (Epskamp, Kruis, et al., 2017). Of
note, research in other domains–especially depression–also identified less dense
network structures in clinical populations (Beard et al., 2016; Fried, van Borkulo,
Epskamp, et al., 2016). Such conflicting findings could indicate that symptom
reactivity may be different across various mental disorders, and raises the question
of whether network topology differs across distinct mental health conditions.
In line with this, it was found that treatment-responsive individuals diagnosed
with psychotic disorders had more densely connected symptom networks after
antipsychotic treatment than did treatment-responsive individuals at baseline,
which could indicate that in the presence of a more densely connected network
structure (i.e., even under greater severity), treatment may be more efficient.
Identifying how these specific mechanisms act may be an important next step
in the field of network modeling. Simultaneously, strong associations could also
indicate that symptom remission spreads more fluently within subdomains of the
PD network, and thus promoting and focusing on improving social functioning
would be highly beneficial, especially for the PD patients. Although there were
many individual edges that differed significantly between the PD and FR network
(see Table 10.1 and Figure 10.1c), only a handful of (stronger) connections in PD
survived Bonferroni correction (Table 10.1), including the connection between
hallucinations and bizarre experiences. This could indicate the strong presence of
feedback loops between the psychotic symptoms or could indicate that these items
may partly ask for the same information (e.g., one may count hallucinations among
bizarre experiences). In the latter case it would nonetheless be expected that this
overlap would be seen in the FR network as well.
Centrality analyses generated a similar pattern for FR an PD in terms of
strength centrality (see Figure 10.2), although many significant differences were
identified between the FR and PD sample (see Table 10.2), with most surviving the
Bonferroni correction as well. The strongest individual nodes for FR were negative
and depressive symptoms and for PD positive and autistic symptoms. Especially in
Page | 179
the PD network, these were most strongly connected to their neighboring nodes,
which may explain to a large extent their high strength value. Although this may
come as no surprise, it does emphasize that an increase in positive and autistic
symptoms may quickly impact neighboring nodes and thus lead the network
structure faster into a disorder state, though at this stage this remains hypothesis
generating. In terms of bridge centrality (Figure 10.3), i.e., how well individual nodes
connect to other node clusters, the overall profile was rather different for FR and
PD. For brevity, we highlight only two individual nodes here, fully acknowledging
that other findings might merit further discussion as well. First, IQ was entered
in the networks because it differentiated substantially between groups. Although
virtually irrelevant within the FR network, and showing marginal strength in the
PD network, it does show widespread connections to individual nodes in the PD
network structure. This illustrates that lower IQ is as much a part of the clinical
phenotype of PD as any other node in the network. Second, ‘Attention to detail’,
unlike any other node, does not cluster within its expected domain (i.e., AQ node
cluster) and is relatively unrelated to other nodes. This is strikingly similar to the
network results in a non-clinical student population (Zhou et al., 2019), as well as
with previous research that identified, in two clinical samples, attention to detail
to be the least central node and only weakly connected to other nodes (van Heijst
et al., 2020). This finding further fits with different factor models of the AQ, in
which attention to detail comprises its own separate factor. We therefore suggest
attention to detail is subject to further scrutiny with regard to its construct validity.
10.4.1 Limitations
The current study should be considered in light of several limitations. First, our
analyses are exploratory in nature and our findings are hypothesis-generating.
Second, current results are based on cross-sectional data and thus causality cannot
be inferred. Third, our study did not identify significant differences between the
family sample and the control sample in terms of global strength and network
structure. This may be a real effect, or may result from the limited power of the
NCT to identify effects (i.e., the control sample is considered a small sample for
network analysis). Future research with larger sample sizes can address this issue
in more detail. Given these findings, as well as our aim to retain comparable
sample sizes while comparing the networks using the NCT (i.e., as to avoid effects
due to sample size), we chose not to collapse TC and FR samples, but keep these
independent. Of note, while means (and thus severity) should not play a role in
Page | 180
the network structure and the NCT, the variance could affect the results if floor
or ceiling effect are present (Wigman et al., 2015). Here we did not identify such
pronounced effects. In addition, it was not possible to investigate significant bridge
strength centrality differences between the two samples as done with strength
centrality, due to limited implementation of these new measures within the NCT.
Fourth, denser network structures could also be an indication of the presence of
a common cause (e.g., a central dysfunction in the brain), but this seems unlikely
as even constructs within the same domain (e.g., positive vs. negative psychotic
symptoms) are generally considered to derive from different etiological pathways.
Finally, the CS coefficient for bridge strength for the PD network was below the
preferred .5 cut-off score, though above the recommended .25 cut-off score, and
thus we advise caution when interpreting this measure.

To conclude, the present study was the first to use a data-driven network approach
in an aim to identify features within the autism and psychosis spectra that may
play a pivotal role in the social functioning of PD patients. We established that the
presence of autistic symptoms has a general negative effect on social functioning, but
in the more intrinsic PD network autistic and psychotic symptoms may reinforce
each other more strongly than for unaffected individuals. On the one hand this
suggests an increased vulnerability for bi-spectral symptomatology, on the other
hand a greater potential for more (transfer) effect of treatments. Consequently, these
findings emphasize the need for increased clinical awareness and default assessment
of comorbid autistic symptoms in individuals with a PD to help enrich their daily
social environments.
Page | 181
Part IV
Methodological Notes in
Network Psychometrics and
Novel Developments
Reporting Standards for
11
Psychological Network Analyses
in Cross-sectional Data

Burger, J.*, Isvoranu, A. M.*, Lunansky, G., Haslbeck, J. M. B., Epskamp, S.,
Hoekstra, R. H. A., Fried, E. I., Borsboom, D., Blanken, T. F. (under review). A
Tutorial on Reporting Psychological Network Analyses in Cross-sectional Data.
Psychological Methods.

Chapter 11 | Reporting Standards for Psychological Network Analyses
Abstract
Statistical network models describing multivariate dependency structures in
psychological data have gained increasing popularity. Such comparably novel
statistical techniques require specific guidelines to make them accessible to the
research community. In this literature, researchers have previously provided
tutorials guiding the estimation of networks and their accuracy. However, there
is currently little guidance in determining what parts of the analyses and results
should be documented in a scientific report. A lack of such reporting standards
may foster researcher degrees of freedom and could provide fertile ground for
questionable reporting practices. In this chapter we introduce reporting standards
for network analyses in cross-sectional data, along with a tutorial and two examples.
The presented guidelines are aimed at researchers, as well as the broader scientific
community, such as reviewers and journal editors evaluating scientific work. We
conclude this chapter by discussing how the network literature specifically can
benefit from such guidelines for reporting and transparency.
Page | 186
11.1 Introduction
Over the past decade, there has been a rapid increase in empirical contributions
applying network analytic methods across many psychological disciplines. This
increasing interest in psychological network analysis furthermore resulted in a large
number of papers and special issues in journals such as Psychometrika, The European
Journal of Personality, The European Journal of Psychological Assessment, BMC
Medicine, and The Journal of Traumatic Stress. However, there is a lack of clear
guidelines on how to report psychological network analyses. The present chapter
introduces such guidelines, aiming to enable researchers to identify all elements of
their analyses that should be included in a scientific report. We argue that reporting
guidelines can facilitate the evaluation of network contributions by the broader
scientific community, including reviewers, editors, journalists, and science writers.
11.1.1 Questionable reporting practices and the benefit of reporting standards

While there are several tutorials on estimating networks from psychological data
(Costantini et al., 2015; Epskamp, Borsboom, et al., 2017; Epskamp & Fried, 2018;
Haslbeck, Bringmann, et al., 2020; Jones et al., 2018; Williams & Mulder, 2020),
as of yet, there is no guidance as for how researchers should report the results of
network analyses in a scientific paper. There are general reporting standards for
statistical analyses, such as the Journal Article Reporting Standards for Quantitative
Research in Psychology published by the APA Publications and Communications
Board Task Force (Appelbaum et al., 2018). However, specific types of multivariate
analyses contain explicit elements that go beyond the scope of generic reporting
standards (Hoyle & Isherwood, 2013). For this reason, more tailored reporting
standards do exist for other types of multivariate analyses, such as structural
equation modeling (Schreiber et al., 2006). At present, however, there are no
explicated standards on how to report the results of network analyses.
A lack of clear reporting standards, in turn, may hinder rigorous scientific
communication: Wigboldus and Dotsch (2016) highlight that a large part of
the degrees of freedom in empirical research resulting in questionable research
practices are in fact gray areas that pertain to questionable reporting practices.
To this end, objective reporting standards for network analysis would be an
important contribution toward making empirical network studies more rigorous.
Since such norms are not yet established in the network literature, the goal of
the present chapter is to explicate what we refer to as ‘minimal shared norms’ in
Page | 187
reporting psychological network analyses. By making these shared norms explicit,

they can be extended and debated, and they will increase the replicability and
reproducibility of network analysis, both of which will move the field of network
psychometrics forward.
11.1.2 A brief introduction to psychological network analysis

While a detailed introduction to psychological network analysis is beyond the scope
of this chapter, in this section we briefly introduce this methodology as to keep the
chapter self-contained. A network is any system in which certain constructs can be
represented with nodes (circles), which are connected by edges (lines) denoting the
strength of connections between the nodes. In psychological networks, nodes are
typically used to represent observed variables of a dataset, and edges the strength of
associations between two variables, typically after controlling for all other variables
in the dataset. This type of model is termed a Pairwise Markov Random Field,
which includes commonly used network models depending on the data used:
Gaussian graphical models (GGM)–also termed partial correlation networks–for
continuous data (Epskamp, Waldorp, et al., 2018; Lauritzen, 1996), Ising models
for binary data (Epskamp, Maris, et al., 2018; Ising, 1925; Marsman et al., 2018),
and mixed graphical models (MGM) for mixed data (Haslbeck & Waldorp, 2020).
Psychological networks can be estimated with (penalized) maximum likelihood
estimation, Bayesian estimation, or by performing (logistic) regressions per variable.
As is the case for statistical models in general, a crucial aspect of psychological
network analysis is that estimated models are subject to sampling variation. As a
result, edges may falsely be included while not being present in the true model,
and differences in edge weights may be strong merely due to chance. To address
such chance fluctuations, psychological network analyses should always include
both model selection methods and checks for stability and accuracy. Model
selection algorithms are diverse, but generally fall under one of three categories:
a) pruning/ thresholding methods, which merely remove or hide edges that do
not meet some criterium, as defined by a classical statistical significance level
or a lower Bayes factor; b) model search strategies, which use extensive model
search methods to iteratively arrive at an optimal network structure; and c)
regularization methods, which use penalized maximum likelihood estimation
to shrink parameters to zero, potentially removing them from the network. Each
of these strategies has their pros and cons. For example, regularization techniques
may work well in retrieving an interpretable structure at low sample sizes, but may
Page | 188
also feature a lower specificity rate than desired (Williams, Rhemtulla, Wysocki,
& Rast, 2019). Checks for stability and accuracy usually involve the use of data-
driven resampling methods such as bootstrapping (Epskamp et al., 2018), or
Bayesian sampling methods (Williams & Mulder, 2020) to assess and visualize
uncertainty around parameter estimates.
11.1.3 Scope of models and software

In this chapter, whenever we refer to ‘network models’, we intend to designate
statistical models that are designed to capture pairwise interactions between
variables and that are estimated on cross-sectional data. Our focus lies on cross-
sectional networks, because network analyses for this type of data account for the
largest part of empirical network contributions over the past ten years: 83% of the
identified empirical papers between 2008 and 2018 as reported by Robinaugh,
Hoekstra, et al. (2020). Of note, there are many other types of psychological
network analyses than the ones we discuss here, such as models estimated in panel
data and time-series data (Epskamp, 2020a; Gates & Molenaar, 2012; Haslbeck,
Bringmann, et al., 2020). These are beyond the scope of the present chapter as they
require different reporting standards due to differences in data structure, estimation
methods, and model assumptions.
Within the domain of cross-sectional network analysis, there is a wealth of
software options. Depending on the choice of software, different reporting elements,
such as specific test statistics, might be required to ensure interpretability of the
results. Here, we focus on software implemented in the open-source environment R
(R Core Team, 2015), specifically on packages that have been most frequently used
in the past decade in empirical, psychological network contributions (Robinaugh,
Hoekstra, et al., 2020). An overview of the software packages that we cover in this
chapter can be found in Table 11.1 at the end of this chapter. While we focus on a
specific set of R-packages, most of the discussed reporting standards represent core
elements of cross-sectional network analysis in psychological data. We therefore
expect that the introduced reporting standards will also be applicable to other
software, albeit not in regard to the specific test statistics included in this chapter.
For instance, reporting parameter uncertainty is not a unique standard of the
packages discussed in this chapter, but should be included for any contribution that
estimates partial correlation networks. Consequently, the listed packages should
be seen as examples of how the core reporting standards introduced here should be
Page | 189
applied to software that is frequently used in the literature, rather than restricting
the domain of reporting standards to this type of software alone.
11.1.4 Organization of the proposed reporting standards

This chapter adopts the typical structure of a psychological report according to APA
standards (American Psychological Association, 2020). We provide a reporting
routine for both the Methods and the Results section, using the following structure:
i. General analysis routine: These sections contain reporting standards that
are applicable to all analyses as defined above, independent of specific research
questions. We recommend to always report these elements.
ii. Analysis-specific routine: These sections contain reporting standards that
will only apply to specific research questions and analyses within the network
analytic framework. Not all of these will be of interest for every empirical
network contribution and are therefore only applicable if they align with
their specific research question.
iii. What to watch out for: The main focus of this chapter lies on providing
reporting standards and not interpretation guidelines. However, some
reporting standards are closely related to interpretation. Therefore, in the
What to watch out for boxes, we discuss some of the considerations that are
important when applying network analyses to psychological data.
Finally, to illustrate these norms and reporting standards, we include two examples
of network analyses on openly available data, with two distinct research goals,
available in Supplement H. Further, we include an overview of most network
estimation packages and functions referred to in this chapter, along with
information on important arguments, current estimation defaults, applicable input
data, and parameter interpretation (see Table 11.1).
11.2 Reporting standards for the methods section

Many considerations may not be specific to network models but apply to a wide
range of quantitative research in psychology. Yet, these considerations can have
important implications for the estimation and interpretation of psychological
networks in particular, as will be highlighted in the What to watch out for boxes.
Page | 190
General analysis routine
We distinguish reporting standards pertaining to features of the data (section
11.2.1), and features of the statistical method (section 11.2.2 – 11.2.4).
11.2.1 Sample, instrument and processing choices

Sample collection. We recommend to specifically consider and report how the
participants were recruited and whether a subpopulation was included in the
analyses (e.g., depressed patients; see Box A). Subsample selection can occur because
of recruitment strategies (e.g., collecting data in clinical practice) or by selection
after data collection (e.g., only include participants that scored higher than a certain
cut-off). Make sure to report on subsamples in either case. Report the number of
participants for whom data was collected and the number of participants that were
included in the network analyses.
Variables. As with any other study, it is important to precisely report what

instruments were used to collect the data, what versions of these instruments, if
applicable (Flake & Fried, 2019), and the overall number of variables. When the data
are pre-processed before being included in the analyses (e.g., variable selection or
transformation), carefully report on these processing steps and indicate the number
of variables included in the network analyses. Pre-processing choices concern, but
are not limited to, collapsing variables (e.g., aggregating variables such as loss or
increase of appetite), collapsing categories (e.g., binarization of Likert-scale data),
data transformations (e.g., in case of violating assumptions; see Box A), imputation
or removal of missing data (e.g., listwise deletion of cases). An exhaustive list of
choices that warrant justification is listed elsewhere (Flake & Fried, 2019). For
the variables that are included in the network, we recommend comparing the
distribution of the variables with the assumptions of the estimation method and
checking any violations (e.g., skewness of the data; see Box A).
Deterministic relations between variables and skip-structures. The manuscript

should specifically report if the scale used to construct the network contains a so-
called skip-structure, i.e., some questions in the questionnaire are skipped based on
responses to previous questions. This can occur when variables are deterministically
related and participants should only answer one question or the other (e.g., report
either on weight loss or weight gain) or when certain follow-up items are only
administered to a subset of participants (e.g., only assessing nuanced depressive
Page | 191
symptomatology if one of the core depression symptoms is present). This creates a

missingness problem for the data that should be addressed. The manuscript should
indicate how this dependency was handled. Some methods, such as imputing zeroes
for skipped items, will induce dependency relationships in the data that bias the
network structure and can lead to faulty inference (see Borsboom et al., 2017). The
latter problem will hold for any deterministic relationship included in the network
(e.g., including a sum-score variable together with the components that make up
the sum-score) and should be avoided.
11.2.2 Estimation method

The estimation method used when performing a network analysis should always be
reported (see Table 11.1 for a description of commonly used estimation techniques).
In addition to the estimation method, mention any additional specifications. For
example, when the networks are thresholded, report the chosen thresholds; when
regularization is used, report the parameter settings. Of note, even if researchers
stick to default arguments (i.e., the standard settings that are used in the estimation
procedure), these should be reported since defaults in software packages can change,
which in turn would make reproducing analyses difficult.1 Finally, we advise
considering the assumptions of each estimation method (see Box A).
11.2.3 Accuracy and stability of edge-estimates

As with any procedure that involves parameter estimation, it is important to assess
how accurate our estimates are. In the context of the most common estimation
techniques in network analysis, this assessment has been implemented in a bootstrap
procedure. In this procedure, the model is estimated repeatedly under sampled or
simulated data and statistics of interest (e.g., edge weights) are computed repeatedly
(Efron, 1979). As such, bootstrapping allows to approximate the sampling
distribution of the parameters. The sampling distribution can then be inspected
visually (for details see e.g., Epskamp, et al., 2018). Specifically, in the methodology
section of the manuscript, we advise reporting the number of bootstrap samples, as
well as the type of bootstrap method employed (in the above case ‘nonparametric’).
For methods that make use of Bayesian inference, such as BGGM (Williams &
Mulder, 2020), there are equivalent measures to assess accuracy and stability, such
as credibility intervals for estimates and convergence diagnostics.
1 Within the R statistical software (R Core Team, 2015), the defaults of each package can be checked
using the “?” + name of the function within a statistical package (e.g., ?estimateNetwork).
Page | 192
11.2.4 Statistical packages
Finally, we recommend reporting the statistical software and packages that are used,
including their versions. Full reproducibility is guaranteed only if this information is
shared along with code and data, because statistical packages can change estimation
defaults when they are updated. With this information, the reader can mimic the
analyses under identical estimation settings and reproduce all results, for example
using the checkpoint package in R (Ooi et al., 2020). We further recommend including
any seed-settings in the code that have been used in conducting analyses (e.g., if
estimation techniques based on cross-validation, or the Network Comparison Test
were used; Haslbeck & Waldorp, 2020; van Borkulo et al., 2017).
Analysis-specific routine
Following the general estimation routine, depending on the specific research
question of interest, different reporting routines are recommended.
11.2.5 Group comparisons

If groups are compared, we recommend reporting which methods have been
employed to compare groups: e.g., correlating adjacency matrices; comparing
networks using the Network Comparison Test (van Borkulo et al., 2017);
comparisons based on the posterior predictive distribution or model selection in
Bayesian GGMs (Williams et al., 2020); estimating moderated network models
in mgm (Haslbeck, 2020; Haslbeck & Waldorp, 2020).
11.2.6 Centrality indices

Centrality metrics can be computed to quantify the role of nodes in a network
(Costantini et al., 2015; Opsahl et al., 2010). If such inferences are of interest, we
recommend reporting which centrality metrics were computed (e.g., ‘strength’),
alongside accuracy of parameter estimates (e.g., case-drop bootstrap as described
in Epskamp et al., 2018; see Box A).
11.2.7 Differences between edges

If comparing edges across groups is of interest, we recommend reporting how the
edge weights were compared (e.g., using the bootstrapped difference-test in the
R package bootnet; Epskamp et al., 2018). Further, if hypotheses are tested in a
Bayesian context (Williams & Mulder, 2020), these should be stated explicitly
(e.g., A – B > C – D).
Page | 193
11.2.8 Clustering
If clustering of nodes is of interest (Hennig, 2015), we recommend reporting which
clustering method was employed when running the analyses (e.g., Exploratory
Graph Analysis; Golino & Epskamp, 2017), as well as if and how the stability of
the identified clusters was checked.
What to watch out for, Box A:

Dataset Instrument design. It is important to consider how the instrument used to gather
the data was constructed. For instance, variables included in a network may come
from a single questionnaire that was constructed to measure a latent variable, and is
therefore intended to measure a single underlying construct. If a set of items does in
fact depend on the same latent variable, but the items are interpreted as measuring
distinct factors, possible distortions in e.g., centrality estimates should be taken
into account (Fried & Cramer, 2017; Hallquist et al., 2019).
Variable Assumptions of estimation methods. For each estimation method, model
distribution assumptions should be considered and violations of these assumptions should
be addressed. Main assumptions include 1) independent cases; 2) the presence
of (log) linear relationships and pairwise interactions only; 3) missing data are
Missing (Completely) at Random (Rubin, 1976; only for estimation methods that
can handle missing data); 4) relevant distributional assumptions of the variables
included in the network.
Variance. Certain restrictions to variance, such as floor/ceiling effects or
restrictions in range, can affect statistical relationships. This should be considered
when interpreting edges and the importance of variables: e.g., suicidal ideation is
typically restricted in variance but clinically relevant; see also ‘centrality’ below and
Fried et al. (2018). Note that these artifacts not only pertain to networks estimated
from continuous data, but also if estimated from binary data, for example, if
symptoms are coded as present versus absent, and most participants in the sample
are healthy individuals without symptoms.
Subsample Biases due to subsample selection (e.g., Berkson’s bias). Sample selection is
selection important because it can lead to unexpected patterns in the data. For instance, if a
subpopulation (e.g., depressed patients) is recruited based on a cut-off on the total
score of symptoms included in the network structure, one may find that, in that
subpopulation, many edges between symptoms are negative. The reason for this
result is that the total score is composed of the individual item scores. As a simple
example that illustrates the effect, suppose one throws coins A and B repeatedly and
only selects cases in which only one of them falls heads (i.e., total score=1). Within
this set of throws (i.e., conditioning on the total score), the correlation between the
outcomes of the tosses for the two coins will be negative, because if coin A falls
heads then, given a total score of 1, coin B must have fallen tails. This effect has
been referred to as Berkson’s bias (de Ron et al., 2019). However, it has been noted
that Berkson’s bias is but one of various effects of conditioning, and that these need
not constitute bias in the statistical sense (Haslbeck, Ryan, & Dablander, 2020).
Page | 194
What to watch out for, Box A:
Nevertheless, it is important for researchers to realize that creating subsamples
based on functions of the variables in the network will often have strong effects on
the network structures found in these subsamples.
Variable Variable selection. The structure of network estimation results typically depends
inclusion on which variables were included in the analysis. This is due to the fact that
conditional dependencies are used in network estimation: conditioning on different
sets of variables can therefore lead to different network structures. This implies
that the network structure may change if variables are included in or excluded
from the model.
Centrality Local network properties. Centrality is not a characteristic of a variable, but it is
determined within the estimated network (see also ‘variable distribution’ and ‘variable
inclusion’; Bringmann et al., 2019; Fried et al., 2018. Thus, a variable that is peripheral
in one network may be central in another. For instance, the symptom of insomnia may
be on the periphery of a depression network and of a generalized anxiety network.
At the same time, it may connect the depression network to the generalized anxiety
network and thus may be highly central in the combined network.
11.3 Reporting standards for the results section

General analysis routine
11.3.1 Final sample size
As with general statistical guidelines (e.g., Appelbaum et al., 2018), all information
regarding sample size should be reported, including removal of outliers, removal
of missing data, data imputation, data transformations, and so forth). For further
details please refer to Table 11.1 and Box B.
11.3.2 Results of the accuracy and stability checks

Results on how accurate parameters are estimated (e.g., Epskamp et al., 2018) should
be reported. Usually, reports include plots giving information on bootstrapped
confidence intervals (CIs), inclusion probabilities, or case-drop bootstraps, but
which specific method to use is based on the choice of software. Of note, which
stability analysis to use is conditional on the research questions to be addressed
(e.g., if centrality is not analyzed, reporting stability results for centrality may not
be relevant). For most existing analyses and research questions, stability analyses
are available.
Page | 195
Analysis-specific routine
Depending on the specific research question(s) of the researchers, different
reporting routines are recommended. Below we discuss aspects important to report
for different choices of analysis routine.
11.3.3 Network visualization

When a network plot is included in the manuscript, we recommend using a
colorblind-friendly theme, as well as reporting:
• What the edges represent (for example, partial correlations in the normal
distribution; see Table 11.1);
• Information about the plot, such as the size of the smallest and largest edges
in the network and whether any specific visualization tools were used (e.g.,
in qgraph; Epskamp et al., 2012; whether a minimum, maximum or cut
value were used when plotting the network);
• How the layout of the network was set (e.g., manually or using a pre-
defined algorithm).
11.3.4 Global network properties

If the overall network structure is of interest, we recommend reporting the network
density and average absolute edge weights.
Network density and average absolute edge weights. The network density refers
to the number of estimated edges relative to the total number of possible edges and
is used to give an indication of the sparsity of the network. If visualized with qgraph
(Epskamp et al., 2012), parameters adjust the color saturation and width of an edge
to the absolute weight and scale relative to the strongest weight of the graph, thus
one cannot get a clear notion on the average edge weight from visualization alone
(Epskamp et al., 2012), and reporting this is essential.
11.3.5 Local network properties

In addition to investigating the global structure of the network, one can examine
certain local properties of the network. Below we list properties that are often of
interest, alongside recommendations for what should be reported for these properties.
Centrality indices. If centrality is of interest (Costantini et al., 2015; Opsahl et

al., 2010), we recommend including a supplementary table or appendix reporting
Page | 196
the raw centrality scores, as well as raw centrality scores in the centrality plot itself,2
as exact parameter values can often not be inferred from centrality plots with
high precision. To assess the degree to which centrality estimates are subject to
sampling error, we recommend reporting results of centrality stability (i.e., a case-
drop bootstrap plot for the reported centrality indices), as well as the correlation
stability coefficient (CS coefficient; Epskamp et al., 2018).
Predictability. If predictability of nodes is of interest (Haslbeck & Fried, 2017;

Haslbeck & Waldorp, 2018, 2020), we recommend to specify which predictability
measure was chosen for which type of variable (e.g., R 2 for continuous variables),
and to include the predictability measures in the network plot. In addition, we
recommend including a supplementary table or appendix reporting the raw
predictability scores, as exact parameter values can often not be inferred from the
network plot alone with high precision.
Specific features of the network. If more specific features of the network are of
interest, such as a specific edge A– B, we recommend reporting the stability of that
particular edge. Likewise, if specific nodes are of interest, say node A, it is important
to report the stability of the edges between node A and its connecting nodes, as
well as the stability of the centrality for that particular node. Finally, if comparing
the strength of two edges, or the centrality of two nodes, we recommend reporting
the results of the bootstrapped difference test. These may also be informative in
other settings, e.g., when interested in the overall stability of the network structure.
Finally, if clustering of nodes is of interest, we recommend reporting the number
of resulting clusters, as well as the stability of the clusters.
11.3.6 Group comparisons

When interested in comparing the network structure between different groups,
we recommend reporting:
• the sample size per group after data preprocessing choices (e.g., removal of
outliers, removal of missing data, data imputation, data transformations);
• if a particular statistical test was used to compare the groups: the resulting p
values or Bayes Factors, and whether these were adjusted for multiple testing;
2 The current default in qgraph provides z-scores up to version 1.6.9. This, however, may inflate
dissimilarity between centrality indices, and we therefore recommend to use raw scores instead.
Page | 197
• if the chosen comparison method allows, the stability of each network

structure should be reported alongside the network comparisons.
When comparing networks visually, arguments used for visualization become
crucial (e.g., minimum, maximum, and cut values; whether the same layout was
used, etc.), as well as the correlation between the adjacency matrices of the two (or
more) network structures. We thus recommend:
• using the same layout when comparing network structures. Note that merely
comparing networks visually may be misleading and is not recommended in
isolation (e.g., without also carrying out a statistical test), even if the layout
is fixed across networks (e.g., equal layouts might suggest that network
structures are more similar than they actually are).
• setting the same value as the strongest edge in both networks (e.g., in qgraph
by setting the same maximum value) in both network structures.
What to watch out for, Box B:

Features of the Sparsity. A central assumption of most of the models highlighted in the
network structure current manuscript is the assumption of sparsity. If this assumption is violated,
the performance of regularized estimation algorithms may be suboptimal
(Epskamp, Kruis, et al., 2017), because many edges that are small but nonzero
will be incorrectly set to 0. In this case, a nonregularized method can be used
as an alternative (Williams et al., 2019).
Collider structures. Collider structures occur when a variable is a common
effect of two or more variables. If a true causal collider structure (A  B 
C) underlies the data and the variables A and C are marginally uncorrelated
or weakly positively correlated, then the undirected network could feature
an edge between the causes (A – C), which is negative if both causal effects
are positive. As such, collider structures can produce strong and unexpected
negative edges in the network structure, which may hamper the interpretation
of results. While there is no principled way to detect collider structures, one
way to detect at least potential collider structures is by comparing the partial
correlations to marginal correlations. If a partial correlation is of a different sign
(e.g., negative) than a marginal correlation (e.g., positive), then this can signal
conditioning on a collider (in this case, also check whether the two variables
are both strongly connected to a third, which may be a common effect).
Network architecture. When interpreting a network structure, it is important
to keep an eye open for global features of the network. For instance, are there
hubs in the network? How do these hubs influence the network structure? Is
the network structure dense? Are there subnetworks?
Page | 198
Global network aspects can inform and drive the interpretation of the network.
Network architecture refers to the structure of the network as a whole; for
instance, well-known architectures include small world, scale-free, and random
graphs (Newman, 2010). Network architecture has been suggested to influence
the recovery of the network structure (van Borkulo, Borsboom, et al., 2014).
For example, if a network features locally dense structures in the form of strong
hubs (as in a scale free network), regularized estimation may have trouble
recovering this (as it promotes sparsity). In contrast, in a ring graph (as e.g.
used by Epskamp & Fried, 2018) each node has only two neighbors, which can
be easily recovered by a regularized estimation technique.
Network Plotting algorithms. Network plots are always dependent on plotting
visualization algorithms. Some of these algorithms (e.g., the Fruchterman-Reingold algorithm;
Fruchterman & Reingold, 1991) can be sensitive to small changes (e.g., small
differences in edge weights). Although network plots are informative visual
representations, the exact placement of nodes should not be interpreted as
standing in a one-to-one relation with features of the data. In order to arrive
at representations that optimally represent patterns in the data, one may utilize
multidimensional scaling-based algorithms (Jones et al., 2018).
Unstable network Accuracy and stability. Network stability is typically assessed by investigating
structures whether the same ordering of edge strengths or centrality estimates arises across
random subsamples of the data. Importantly, an unstable network structure
does not necessarily imply that the analysis failed and the network should be
discarded. This is because there are two reasons why orderings of edges may
be unstable under bootstrapping: a) there are estimation problems (e.g. N
is too small), and b) all edges are equally strong so that there is no ordering
in the first place (e.g., the network is a Curie-Weiss model; Marsman et al.,
2018). However, unstable network structures do limit the interpretation of
the network (e.g., if the centrality ordering is unstable for whatever reason,
centrality differences should not be interpreted). In general, instability should
be acknowledged, and findings from unstable network models should be
presented with caution. For regularized methods, we argue against merely
checking if bootstrapped CIs do (not) include 0, because the model selection
methods themselves are already designed to put edges to zero. Therefore,
doing additional checks on the CIs may lead to double thresholding. Rather,
the width of CIs reflects the accuracy of parameter estimates, irrespective of
whether they include 0 or not (Epskamp et al., 2018). Wide confidence intervals
imply caution in interpretation, especially when interpreting the strength of
edges, or the presence of weaker edges. While a clear definition of what wide
represents is not established, this resolution can be driven by the specificity of a
research question (e.g., if one particular edge is of central importance, then the
wider the CI is for that edge, the less confidence we can attach to the estimate).
In addition, with regularization techniques, most CIs should not cross zero.
Further, unstable centrality coefficients may be problematic and could indicate
low replicability and generalizability.
Page | 199

Missing data Missing values. It should be noted that not all estimators can handle missing
data (e.g., mgm or IsingFit will remove missing data listwise; see Table 11.1).
Besides the use of (multiple) imputation strategies, which have not yet
been studied in detail for network models, there are currently two ways for
handling missing data when estimating GGMs. First, some estimators, such
as EBICglasso and ggmModSelect, only require a correlation matrix as input,
which can be estimated using pairwise observations. The bootnet package
(Epskamp et al., 2018) does this by default for these estimators, and will use
the average of pairwise sample sizes as proxy for the sample size (e.g., for BIC
computation; Epskamp, 2020b). Specifically, the sample sizes used when
estimating each pairwise correlation separately are computed, and the average
of these is taken as the final sample size in the analyses. Second, the psychonetrics
package includes full information maximum likelihood estimation (Epskamp
et al., 2020), which will only use observed data to estimate the GGM structure.
Error rate Error rate. The error rate, as well as the circumstances under which the error
rate changes, should be considered. It is thus essential for researchers to consider
whether they are favoring the sensitivity or the specificity of a model. Some
estimation techniques (e.g., the EBICglasso algorithm; Epskamp & Fried, 2018)
have high sensitivity, but lower specificity. This means that weaker edges may be
more prone to Type I errors, i.e., to be false positives. Other estimation routines
may be more conservative, retaining high specificity, but lower sensitivity (i.e.,
some edges may be missing from the network). The main research question
should drive this choice, as well as whether the researcher is concerned with
erring on the side of discovery or on the side of caution.
11.4 Illustrative examples

To illustrate the highlighted norms and reporting standards, we provide two examples
of network analyses on openly available data, with two distinct research goals, in
Supplement H. Both examples contain the elements described under the general
analysis routine, as well as analysis specific elements matched with the indicated
research goal. First, using data from Burger and colleagues (2020), we aim to highlight
the analysis specific routine on group comparisons, network visualization, and global
network properties. Second, using open data (https://openpsychometrics.org/tests/
TMAS/) collected on the Taylor Manifest Anxiety Scale (Taylor, 1953), we aim to
highlight the analysis specific routine elements on centrality, differences between
edges, network visualization, and local network properties.
Page | 200
As clear norms have not yet been established in the network literature, the current
chapter explicates minimal shared norms in reporting psychological network
analyses. While network psychometrics is a relatively young field of research,
we recognize that many norms discussed here have important implications for
commonly used inferences. We therefore included two ‘What to watch out for’
boxes, where we discuss important considerations for network analysis, as well as
potential sources of misinterpretation of network structures.
Further, clear reporting standards for network psychometrics improve
transparency, which is necessary for reproducibility. Only if the scientific community
can follow exactly what analyses were conducted can we vet inferences drawn by
respective authors. This is especially relevant in a field that is still fairlynovel such
as network psychometrics, where we encounter new challenges regularly. Overall,
we trust the highlighted directions to aid researchers in identifying elements of
their analyses that are important to include in a scientific report, as well as to make
empirical network studies more rigorous.
Page | 201
Table 11.1 Overview and detailed information of commonly applied estimation routines in R
<package Bootnet
Model Parameter Input Bootnet Missing data
name>::<main Description Main defaults default Notes
Page | 202
(data) interpretation type* default set handling
function> differences
Ising Logisitic IsingFit::IsingFit Regularized Raw Gamma hyper- "IsingFit" Automatic None (rows Regularization
Model regressions/ estimation data (0/1 parameter is missing data with missing via glmnet
(binary) logliniear nodewise encoded) set to .25 and removal data need package
interactions logistic an AND-rule (listwise) and be removed
regressions and is used (both automatic before analysis)
EBIC model regression esti- median split
selection mates required if input data is
to be nonzero) not binary
IsingSampler::Esti- Unregularized Raw Pseudo- "IsingSam- Loglinear None (rows Saturated or
mateIsing estimation data (any likelihood pler" model used with missing pre-defined
using psuedo- binary estimation for up to 20 data need model only (boot-
likelihood, encoding) (method = "pl") nodes and be removed strap threshold
loglinear mod- univariate before analysis) with bootnet::-
eling or univar- logistic bootThreshold)
iate logistic regressions for
regressions >20 nodes
psychonetrics::Ising Maximum Raw psychonetrics N/A N/A Listwise Not possible with
likelihood data (any model** deletion many (over 20)
estimation binary (pairwise nodes
encoding) covariance
or matrix can
summary potentially be
statistics used as input)
(means +
covariance
matrix)
Table 11.1Continued.
<package Bootnet
Gaussian The parameters qgraph::EBICglasso Regularized Variance- Gamma "EBICg- Bootnet Pairwise dele- Poor performance
Graphical (i.e., edges) estimation covari- hyperparameter lasso" correlation tion (sample in large sample
Model represent using glasso ance/ is set to .5 defaults used size can be set sizes with
(normal the unique and EBIC correlation when raw to average of dense network
or ordinal association Model matrix data is used as sample sizes structures
data) among two selection input*** for each pair of
variables, after variables)
conditioning qgraph::qgraph(..., Unregularized Vari- Saturated "pcor" Bootnet Pairwise dele- Edges that are
on all other graph = "pcor") network ance-co- model (all correlation tion (sample not significant
variables in the (saturated) variance/ edges included) defaults used size can be set (based on p values
network**** correlation when raw to average of or bootstraps) can
matrix data is used as sample sizes be hidden, but no
input*** for each pair of model selection is
variables) performed
qgraph::ggmMod- Unregularized Vari- Gamma "ggmMod- Bootnet Pairwise dele- Slow with many
Select estimation ance-co- hyperparameter Select" correlation tion (sample nodes (>30) unless
using extensive variance/ is set to 0 (BIC) defaults used size can be set stepwise = FALSE
model search correlation when raw to average of is used
matrix data is used as sample sizes
input*** for each pair of
variables)
psychonetrics::ggm (Full Raw psychonetrics N/A N/A Missing data Ordinal data
information) data or model** handling supported with
maximum summary through full ordered = TRUE
likelihood statistics information (uses weighted
estimation (means + maximum least squares
covariance likelihood estimation)
matrix) is supported
with estima-
Page | 203
tor = "FIML"
<package Bootnet
Page | 204
Mixed (logistic / linear mgm::mgm(..., Regularized Raw data Gamma "mgm" Gamma hy- None (rows Default when
Graphical / multinomial) lambda- nodewise hyperparameter (criterion = perparameter with missing using bootnet but
Model regression Sel = "EBIC") regressions is set to .25 "EBIC") is set to 0.5, data need not when using
(normal/ weights based with EBIC type and level be removed mgm. Reduces
categori- on standardized model arguments are before analysis) to Ising model
cal/ count) data selection, automatically with only binary
potentially set, edges are variables. Edge
with automatically weights between
interaction signed if possi- continuous
effects (3- ble, listwise variables are not
way, 4-way, deletion partial correlation
etcetera) automatically coefficients (but
applied to have the same
data interpretation)
mgm::mgm(..., Regularized Raw data Number of "mgm" type and level None (rows Default when
lambdaSel = "CV") nodewise folds is set to 10 (criteri- arguments are with missing using mgm but
regressions on = "CV") automatically data need not when using
with k-fold set, edges are be removed bootnet. Reduces
cross-valida- automatically before analysis) to Ising model
tion model signed if with only binary
selection, possible, variables. Edge
potentially listwise weights between
with interac- deletion continuous

tion effects automatically variables are not
(3-way, 4-way, applied to partial correlation
etcetera) data coefficients (but
have the same
interpretation)
<package Bootnet
Cor- Bivariate qgraph::qgraph(..., Bivariate Variance- Saturated "cor" Bootnet Pairwise dele- Edges that are
relation marginal graph = "cor") estimation covari- model correlation tion (sample not significant
Network correlations ance/ (all edges defaults used size can be set (based on p values
(any) correlation included). when raw to the average or bootstraps) can
matrix data is used as of sample sizes be hidden, but no
input*** for each pair of model selection is
variables) performed
psychonetrics::corr Maximum Raw psychonetrics N/A N/A Missing data Ordinal data
likelihood data or model** handling supported with
estimation summary through full ordered = TRUE
statistics information (uses weighted
(means + maximum least squares
covariance likelihood estimation)
matrix) is supported
with estima-
tor = "FIML"
Relative Normalized lmg relaimpo::calc. Relative Raw data Saturated "relimp" No automated None (rows Returns directed
Impor- metric relative relimp importance model (all function with missing (not causal)
tance importance edges included) outside of data need network
Network measures bootnet be removed
(continu- wrapper before analysis)
ous)
*Input for bootnet ‘estimateNetwork’ function must always be raw data

**Models in psychonetrics are saturated by default. Functions such as ‘prune’, ‘stepup’, and ‘modelsearch’ can be used for exploratory model search
***Bootnet will automatically correlate the data if raw data is used (Pearson correlations from version 1.4 onwards). For ordinal data, polychoric correlations or Spearman
correlations can be used as input
Page | 205
****In cases in which a transformation is applied (e.g., huge; not standardized), the parameters should be interpreted as precision matrix elements
Network Models of Post-
12
traumatic Stress Disorder:
A Meta-analysis

Isvoranu, A. M., Epskamp, S., Cheung, M. W. L. (2021). Network Models of Post-
traumatic Stress Disorder: A Meta-Analysis. Journal of Abnormal Psychology.
Chapter 12 | Network Models of PTSD: A Meta-analysis
Abstract
Post-traumatic stress disorder (PTSD) researchers have increasingly used
psychological network models to investigate PTSD symptom interactions, as well
as to identify central driver symptoms. It is unclear, however, how generalizable such
results are. In the current chapter we have developed a meta-analytic framework for
aggregating network studies while taking between-study heterogeneity into account
and applied this framework in the first-ever meta-analytic study of PTSD symptom
networks. We analyzed the correlational structures of 52 different samples with
a total sample size of N=29,561 and estimated a single pooled network model
underlying the datasets, investigated the scope of between-study heterogeneity,
and assessed the performance of network models estimated from single studies.
Our main findings are that: (1) We identified large between-study heterogeneity,
indicating that it should be expected for networks of single studies to not perfectly
align with one-another, and meta-analytic approaches are vital for the study of
PTSD networks. (2) While several clear symptom-links, interpretable clusters,
and significant differences between strength of edges and centrality of nodes can
be identified in the network, no single or small set of nodes that clearly played a
more central role than other nodes could be pinpointed, except for the symptom
‘amnesia’ which was clearly the least central symptom. (3) Despite large between-
study heterogeneity, we found that network models estimated from single
samples can lead to similar network structures as the pooled network model. We
discuss the implications of these findings for both the PTSD literature as well as
methodological literature on network psychometrics.
Page | 208
12.1 Introduction
Over the past decade, a novel theoretical framework proposed in the study of
psychopathology grew popular and prominent, especially in clinical and psychiatric
research domains: the network approach (Borsboom et al., 2011; Borsboom, 2017;
Borsboom & Cramer, 2013; Cramer et al., 2010; Epskamp, van Borkulo, et al.,
2018; Fried, Bockting, et al., 2015; Fried, van Borkulo, Cramer, et al., 2016; Fried,
van Borkulo, Epskamp, et al., 2016; Isvoranu et al., 2016, 2017; Klaiber et al., 2015;
Rhemtulla et al., 2016; Robinaugh, Hoekstra, et al., 2020; van Rooijen et al., 2018).
The network approach to psychopathology proposes that mental disorders result
from dynamical interactions between symptoms (e.g., flashbacks of a traumatic
event could cause nightmares, which could in turn cause intrusive thoughts and
sleep disturbance), and that the symptoms themselves are what constitute a mental
disorder, rather than a non-observable common cause (Borsboom, 2008, 2017;
Borsboom & Cramer, 2013). From this perspective, symptoms become agents
in a causal system and are no longer regarded as merely passive indicators of a
latent unobserved entity (i.e., a mental disorder; Borsboom & Cramer, 2013;
Kendler, 2016). In response to the network perspective, which was first advanced
as theoretical groundwork, the exploratory methodological framework now known
as the field of network psychometrics (Epskamp, 2017; Epskamp, Maris, et al.,
2018) developed, with a large body of growing empirical research across diverse
disciplines currently and frequently employing this methodology (Abacioglu et
al., 2019; Blanken et al., 2019; Costantini & Perugini, 2016; Kossakowski et al.,
2015). The main type of network now commonly used is the Gaussian Graphical
Model (GGM): a network in which variables (e.g., symptoms) are represented as
nodes, which are connected by weighted edges that represent partial correlation
coefficients (Epskamp, Waldorp, et al., 2018; Lauritzen, 1996).
One of the fast-expanding research fields focused on identifying associations
between a wide array of symptoms and other factors, and commonly utilizing
network models, is the field of post-traumatic stress. Over the past half-decade,
numerous studies have been published that investigate associations between
symptomatology (McNally et al., 2015; van Loo et al., 2017), potential risk factors
(Armour et al., 2017; Choi et al., 2017; Mancini et al., 2019; Simons et al., 2019)
and pathways to comorbidity (Djelantik et al., 2020; Gilbar, 2020; Lazarov et
al., 2019; Malgaroli et al., 2018; Price et al., 2019; Vanzhula et al., 2019), yielding
novel results and hypotheses. In response to this rapid research expansion and in
Page | 209
an aim to synthesize current findings in the field, a thorough systematic review of

the network approach to posttraumatic stress disorder (PTSD) has been recently
carried out (Birkeland et al., 2020). While more general and overarching reviews
discussing theoretical, methodological, and empirical contributions of network
approaches to psychopathology (including PTSD as a sub-field) already exist
(Contreras et al., 2019; Robinaugh, Hoekstra, et al., 2020), to our knowledge, no
other review focused on network approaches to specific research areas has yet been
conducted. This further supports the rapid development of the network framework
in PTSD research. Of note, while aiming to synthesize current research findings
and summarize common features, existing reviews are mainly narrative, based on
for instance observation of strongest edges and centrality measures. This is due to
the novelty of network models, and the lack of methodological advances at the
time, which would have allowed conducting a comprehensive meta-analysis. While
certainly important contributions, such narrative work cannot handle cross-study
heterogeneity in a systematic way, and results could be impacted by investigating
several potentially underpowered (and unstable) results based on individual samples,
rather than investigating the results of a single well-powered meta-analytic analysis.
Consequently, and alongside the call for new developments in the field of
network psychometrics, we proposed the Meta-Analytic Gaussian Network
Aggregation (MAGNA; Epskamp et al., 2020), which is a novel methodology
that was derived from meta-analytic structural equation modeling (MASEM;
Cheung, 2015; Cheung & Chan, 2005) and can be used to perform a meta-analysis
of network models. MAGNA allows for estimating a single pooled GGM structure
from multiple studies, as well as estimate the size of heterogeneity in deviations
from this pooled GGM. In this way, it becomes possible to aggregate results across
a multitude of studies, providing a statistical and objective framework to summarize
these findings, while taking heterogeneity across different study domains (e.g.,
subjects from different populations that were exposed to different traumatic events)
into account.
The goal of the current research was therefore to expand on previous narrative
findings relating to network models of post-traumatic stress symptoms (PTSS).
We aimed to, for the first time ever, carry out a meta-analysis of existing research
focused on the network approach to PTSD. Specifically, we aimed to identify
common effects of current research by estimating a pooled network structure of
frequently assessed post-traumatic symptoms, to assess centrality of these symptoms
in the pooled network structure, and to investigate the size of between-study
Page | 210
heterogeneity by assessing deviations from the pooled network structure. Finally,
we investigated how similar results from individual network studies are compared
to this pooled cross-study network study. We discuss implications from these results
for PTSD research, but also for the replicability (Borsboom et al., 2017; Forbes et
al., 2017a), generalizability (Forbes et al., 2017b), and utility of centrality indices
(Bringmann et al., 2019).
12.2 Methods
12.2.1 Data sources and search strategies
Consistent with PRISMA (Moher et al., 2009) guidelines, we followed the search
strategies employed by Birkeland and colleagues (Birkeland et al., 2020), with an
extended timeframe and an additional search term. Specifically, we performed a
keyword search in PsychINFO, Medline, and Web of Science and limited our
search to studies published between January 2008 and January 2020. We used
a combination of a keyword pertaining to the network approach (i.e., ‘network
analysis’ OR ‘network approach’ OR ‘network model’ OR ‘network structure’
OR ‘network modelling’ OR ‘network psychometrics’) and a keyword denoting
a focus on PTSS (i.e., ‘posttraumatic stress disorder’ OR ‘posttraumatic disorder’
OR ‘PTSD’ OR ‘posttraumatic stress symptoms’ OR ‘PTS’ or ‘PTS symptom’ or
‘PTSD symptoms’). To further identify records, we examined papers that identified
themselves as reviews conducted on the network approach literature and selected
articles through cross-referencing.
12.2.2 Study selection

Studies were included if they: (1) were written in English, (2) were peer-reviewed,1
(3) were accepted for publication between the pre-defined timeframe, (4) estimated
a GGM of PTSS, (5) were based on cross-sectional or panel data, (6) made the
dataset or correlation matrices available online or via email when requested.
Of note, unlike classical meta-analytic techniques, but similar to MASEM,
the MAGNA framework requires correlation matrices of datasets as input. As
such, when the data or correlation matrices were not made available online, A.M.I
1 We did not search for unpublished manuscripts and as such our results might be influenced by
publication bias. However, we do not expect publication bias to play a role in network papers as
these are highly exploratory and typically not designed to test particular hypotheses. Further, the
number of unpublished manuscripts is likely very small due to the novelty of the field.
Page | 211
and S.E. contacted all corresponding authors to request either the dataset used
when estimating the network structure(s) or, when not possible to share this,
summary statistics of the data (i.e., Pearson correlations after listwise deletion,
Pearson correlations using pairwise estimation, Spearman correlations after
listwise deletion, Spearman correlations using pairwise estimation, the sample
size used, the name of the variables, the number of levels per variable in case
these were measured on an ordered scale, means and standard deviations, and a
description of the content of each variable). For the convenience of the authors,
an R (R Core Team, 2015) function was provided in the request email, which
solely required the dataset as input, and automatically compiled five files with
the requested information (see Table I1 in Supplement I for the R code; R Core
Team, 2015). Two reminder emails were sent to all authors before deciding to
exclude the respective study based on unresponsiveness.
12.2.3 Data extraction

From all eligible articles, we extracted the author’s last name, year of publication,
email address of the corresponding author(s), sample size (both pairwise average
and following listwise deletion when missing data were present, if this information
was available), number of variables assessing PTSS, population, measure used,
diagnostic system, and when available the data or correlation matrices (when not,
the automated procedure described in the Study selection above was employed).
Given the multitude of measures and PTSS symptoms commonly assessed,
for every study included in the meta-analysis, we examined all variables and
identified the symptoms that adhere to DSM-IV criteria for PTSD. One reviewer
conducted the abstraction of the data, while another reviewer verified its accuracy.
For studies in which different samples were available (e.g., different populations
or multiple time points), these were treated as multiple samples in the statistical
analyses (see Table 12.1).2
2 Figure I4 and Figure I5 in Supplement I show results based on collapsing multiple dependent samples
(repeated measures) into a single correlation matrix instead of treating these as separate samples.
Page | 212
Table 12.1 Study characteristics
Sample size Sample size DSM / Pearson Pearson Spearman Spearman
No Study Sample Measure
(pairwise) (listwise) ICD (pairwise) (listwise) (pairwise) (listwise)
Chinese adults who
survived the Wenchuan
McNally,
1 earthquake, and who had 359.4 344 PCL-C DSM-IV ✓ ✓ ✓ ✓
2015
lost at least one child in
the disaster
De Schryver, War-affected youth in
2 442 430 IES-R DSM-IV ✓ ✓ ✓ ✓
2015 northern Uganda
3 Armour, 2017 Veterans (US) 221 221 PCL-5 DSM-5 ✓ ✓ ✓ ✓
Female ministerial
Birkeland, employees present during
4 1088.8 1035 PCL-S DSM-IV ✓ ✓ ✓ ✓
2017 the 2011 Oslo bombing
attack (Norway)
Male ministerial
Birkeland, employees present during
804 770 PCL-S DSM-IV ✓ ✓ ✓ ✓
2017 the 2011 Oslo bombing
attack (Norway)
HIV-negative urban men
5 Choi, 2017 who have sex with men 286.3 281 DTS DSM-IV ✓ ✓ ✓ ✓
(US)
Adults who reported
McNally, having been sexually
6 177.1 165 PCL-C DSM-IV ✓ ✓ ✓ ✓
2017 abused during childhood
(US)
Mitchell,
7 Iraqi veterans 674.2 627 PCL-5 DSM-5 ✓ ✓ ✓ ✓
2017
Page | 213
Page | 214
Youth exposed to
UCLA
8 Russell, 2017 Hurricanes Katrina and 782 750 DSM-5 ✓ ✓ ✓ ✓
PTSD-RI
Gustav
Asylum seekers or
9 Spiller, 2017 refugees in treatment 146.4 136 PDS DSM-IV ✓ ✓ ✓ ✓
(Swiss)
Epskamp, Women with post-
10 358.9 358 PSS-SR DSM-IV ✓ ✓ ✓ ✓
2018 traumatic stress disorder
Treatment-seeking
11 Fried, 2018 517 517 HTQ DSM-IV ✓ ✓ X X
patients
Treatment-seeking
Fried, 2018 363 363 PSS-SR DSM-IV ✓ ✓ X X
patients
Treatment-seeking
Fried, 2018 923 923 PCL-C DSM-IV ✓ ✓ X X
soldiers
Treatment-seeking
Fried, 2018 928 928 HTQ DSM-IV ✓ ✓ X X
refugees
Malgaroli, Bereaved individuals who
12 263 263 Interview DSM-5 ✓ ✓ ✓ ✓
2018 had recently lost a spouse
13 Moshier, 2018 Veterans 378 378 PCL-5 DSM-5 ✓ ✓ X X
Moshier, 2018 Veterans 378 378 CAPS-5 DSM-5 ✓ ✓ X X
US military veterans
14 Phillips, 2018 912 912 DTS DSM-IV ✓ ✓ ✓ ✓
(clinical)
US military veterans
Phillips, 2018 138 138 DTS DSM-IV ✓ ✓ ✓ ✓
(subclinical)
von Stockert, Trauma-exposed U.S.
15 1268 1268 PCL-5 DSM-5 ✓ ✓ ✓ ✓
2018 military veterans
von Stockert, Trauma-exposed U.S.
611 611 PCL-5 DSM-5 ✓ ✓ ✓ ✓
2018 military veterans
Children and adolescents
exposed to at least one
16 Bartels, 2019 475 475 CATS DSM-5 ✓ ✓ ✓ ✓
potentially traumatic
event
Bereaved patients seeking
Djelantik,
17 treatment following 458 458 PCL-5 DSM-5 ✓ ✓ ✓ ✓
2019
psychological trauma
Youth survivors exposed
18 Ge, 2019 1073.12 1010 CRIES DSM-IV ✓ ✓ ✓ ✓
to Lushan earthquake
1073.85 1014 CRIES DSM-IV ✓ ✓ ✓ ✓
1088 1088 CRIES DSM-IV ✓ ✓ ✓ ✓
Males from the Jewish
population in Israel who
19 Gilbar, 2019 234 234 ITQ ICD-11 ✓ ✓ X X
received treatment for
domestic violence
Children and adolescents
20 de Haan, 2019 1611 1429 Multiple ICD-11 ✓ ✓ ✓ ✓
exposed to trauma
German general
21 Knefel, 2019a 275.4 258 ITQ ICD-11 ✓ ✓ ✓ ✓
population
Page | 215
Page | 216
Knefel, 2019a Israeli general population 336 336 ITQ ICD-11 ✓ ✓ ✓ ✓
Knefel, 2019a UK general population 447 447 ITQ ICD-11 ✓ ✓ ✓ ✓
Knefel, 2019a US general population 521.3 495 ITQ ICD-11 ✓ ✓ ✓ ✓
Scottish trauma center
22 Knefel, 2019b 192 183 ITQ ICD-11 ✓ ✓ ✓ ✓
patients
Lithuanian primary
Knefel, 2019b mental health 280 280 ITQ ICD-11 ✓ ✓ ✓ ✓
care patients
Welsh primary and
secondary
Knefel, 2019b 184.7 175 ITQ ICD-11 ✓ ✓ ✓ ✓
mental health service
users
Austrian survivors of
Knefel, 2019b 219 218 ITQ ICD-11 ✓ ✓ ✓ ✓
child maltreatment
Treatment-seeking
23 Lazarov, 2019 1489 1489 CAPS-IV DSM-IV ✓ ✓ ✓ ✓
veteran patients
Female students exposed
Mancini,
24 to the 2007 Virginia Tech 296 296 PSS-SR DSM-IV ✓ ✓ ✓ ✓
2019
campus tragedy
Female students exposed
to the 2007 Virginia Tech 258 257 PSS-SR DSM-IV ✓ ✓ ✓ ✓
campus tragedy
McElroy, Trauma-exposed Israeli
25 1003 1003 ITQ ICD-11 X X ✓ ✓
2019 adults
McElroy, Internally displaced
2203 1790 ITQ ICD-11 X X ✓ ✓
2019 persons Ukraine
Women with full or
26 Papini, 2019 subthreshold PTSD and 306 306 MPSS-SR DSM-IV ✓ ✓ ✓ ✓
substance use
Patients exposed to
various
27 Park, 2019 traumatic events and 249 249 CAPS DSM-IV ✓ ✓ X X
who were beginning
psychiatric treatment
Refugee resettled in a
28 Pfeiffer, 2019 419 419 CATS DSM-5 ✓ ✓ ✓ ✓
European country
Individuals who endorsed
a traumatic event that
29 Price, 2019 1184 1184 PCL-5 DSM-5 ✓ ✓ ✓ ✓
met Criterion A for a
diagnosis of PTSD
Israel Defense Force
30 Segal, 2019 infantry soldiers, pre- 902.6 873 PCL-S DSM-IV ✓ ✓ ✓ ✓
deployment
Israel Defense Force
Segal, 2019 infantry soldiers, post- 719.3 693 PCL-S DSM-IV ✓ ✓ ✓ ✓
combat
31 Simons, 2019 Iraqi veterans 273.53 269 PCL-M DSM-IV ✓ ✓ ✓ ✓
Page | 217
Page | 218
Clinical sample:
participants
Vanzhula,
32 met criteria for a 125.8 124 PCL-C DSM-IV ✓ ✓ ✓ ✓
2019
diagnosis of eating
disorder
Vanzhula, Nonclinical sample:
297.8 296 PCL-C DSM-IV ✓ ✓ ✓ ✓
2019 undergraduate students
33 Armour, 2020 US general population 417 417 PCL-5 DSM-5 ✓ ✓ ✓ ✓
*PCL-C: PTSD CheckList - Civilian Version; IES-R: Impact of Event Scale-Revised; PCL-5: PTSD Checklist for DSM-5; DTS: Davidson Trauma Scale; UCLA
PTSD-RI: University of California at Los Angeles Posttraumatic Stress Disorder Reaction Index; PDS: Posttraumatic Diagnostic Scale; PSS-SR: Post-traumatic
Stress Disorder Symptom Scale Self Report; HTQ: Harvard Trauma Questionnaire; PCBD: Persistent Complex Bereavement Disorder; CAPS-5: Clinician-
Administered PTSD Scale for DSM-5; CATS: Child and Adolescent Trauma Screen; CRIES: Child Revised Impact of Events Scale; ITQ: International Trauma
Questionnaire; CPTCI: Child Posttraumatic Cognitions Inventory; MPSS-SR: Modified PTSD Symptom Scale Self Report; DSM-IV: Diagnostic and Statistical
Manual of Mental Disorders, fourth edition; DSM-V: Diagnostic and Statistical Manual of Mental Disorders, fifth edition; ICD-11: International Classification
of Diseases, eleventh revision.
12.2.4 Statistical Analysis
All statistical analyses were conducted in the R statistical software version 4.1.0
(R Core Team, 2015), using the MAGNA framework (Epskamp et al, 2020)
implemented in the R package psychonetrics version 0.9 (Epskamp, 2020). We
employed a random-effects MAGNA model using an averaged individual estimate
of the sampling variation matrix. In random-effects MAGNA, we model the
marginal (not partial) pairwise correlation coefficient between two symptoms as
the composite of a correlation implied by a single pooled GGM structure, deviation
due to between-study random effects, and deviation due to sampling variation:
sample correlation = implied correlation by GGM + heterogeneity + sampling variation.
To do this, the MAGNA analysis first estimates the amount of sampling variation
on the reported correlations across studies. Next, the analysis takes each correlation
as a ‘variable’, for which it estimates a mean (fixed-effect) and variance-covariance
(random-effects) structure, while taking the previously estimated sampling variation
into account. In this analysis, the means of sample correlations are modeled using
a GGM, which we term the ‘pooled MAGNA network’. Therefore, random-
effects MAGNA is a multi-level model with a random effect on correlations (not
on the edge parameters themselves). Such a MAGNA analysis can involve many
parameters to be estimated: a 17-node network involves 136 network parameters,
136 variances and 9180 covariances to be estimated.
We investigated reported Pearson correlation matrices–for which MAGNA was
developed–in which listwise deletion was used. Sampling variation was handled by
first estimating a single averaged sampling variation matrix, which was constructed
by averaging estimated sample variation matrices per study (we refer to Epskamp
et al., 2020, for more details). We estimated model parameters through maximum
likelihood estimation, using the R package psychonetrics, which subsequently uses
the optimization algorithm implemented in the nlminb function for parameter
estimation. This routine returns several results of interest:
1. Parameter estimates for the pooled MAGNA network, which we used to
investigate edge weights and centrality estimates.
2. An estimated parameter variance-covariance matrix (Fisher information),
which we used to test the significance of edges as well as to assess differences
in centrality indices.
Page | 219
3. An estimated variance-covariance matrix of random effects on the implied

correlational structure, of which we obtained standard deviations of random
effects to assess heterogeneity across studies.
We used these to investigate (1) the edge-weights of the pooled MAGNA network,
(2) the centrality of nodes in the pooled MAGNA network, (3) the cross-study
heterogeneity. Simulations reported by Epskamp et al. (2020) show that the
MAGNA method estimates the pooled MAGNA network and cross-study
heterogeneity well given sufficient samples: at least 16 samples are needed for
acceptable levels of specificity, while more samples (32 & 64) lead to the best levels
of sensitivity and parameter accuracy. Furthermore, we investigated (4) how much
networks estimated from single studies reflect the meta-analytic results, and the
consistency in results across several methodological choices we could have made,
we also employed (5) a multiverse analysis. Below we discuss each of these analyses
in more detail.
Pooled MAGNA edge weights

The edge weights of the pooled MAGNA network represent the strength of
associations between two items in the network structure after conditioning on all
other items in the dataset. These are parameterized as partial correlation coefficients.
The analysis returns a single set of edge weights for a single pooled model containing
the expected edge weights across all studies. The parameter variance-covariance
matrix in addition returns the standard errors of these edge weights, which can
subsequently be used to obtain p values and confidence intervals.
Pooled MAGNA centrality indices

Centrality indices can be used to gauge the importance of nodes in any network
structure (Newman, 2010). We investigated three commonly assessed centrality
measures for weighted networks: strength, closeness, and betweenness (Opsahl et al.,
2010), in addition to two more recently proposed metrics that have grown popular
when analyzing psychological networks: expected influence (Robinaugh et al., 2016),
and the predictability3 of nodes (quantified as explained variance R 2; Haslbeck &
Waldorp, 2018). The metrics strength, expected influence and R 2 quantify direct
connectivity, being a function only of the nodes a node is connected to, and the
3 Predictability is technically not a centrality index, as unlike the other metrics the metric is specifi-
cally designed for the statistical model underlying the network representation rather than for the
network representation itself.
Page | 220
metrics closeness and betweenness also quantify indirect connectivity, being a
function of all other nodes in the network. Strength (also termed node strength or
weighted degree) is quantified by summing the absolute values of all edge weights
connected to a node. Expected influence is the same as strength but does not take
the absolute value of edge weights before summing them. R 2 or predictability is a
metric quantifying how much variance can be explained in one node by all other
nodes in the network, closeness is computed by taking the inverse of the sum
of lengths of edges on the shortest paths between one node and all other nodes
(with length defined as the inverse of the absolute edge weight), and betweenness
is computed by counting how often a node lies on the shortest paths between all
other nodes. For more information on how these measures are computed, we refer
to Opsahl et al. (2010) for strength, closeness and betweenness, to Robinaugh et al.
(2016) for expected influence, and to Haslbeck & Waldorp (2018) and Williams
(2018) for predictability.
In addition to reporting the obtained centrality indices, we employed a
parametric bootstrap routine to obtain centrality difference plots commonly
used in reported network analyses (Epskamp, Borsboom, et al., 2017). In this
method, sampling techniques are used to assess the significance of a difference
between centrality indices. This is typically done to gain insight in the stability of
centrality, as it is not possible to draw confidence regions on the commonly reported
centrality indices due to these relying on absolute values of edge weights and many
edge weights being estimated near the boundary of zero. While commonly a
nonparametric bootstrap is used, we used a parametric bootstrap routine that did
not involve estimating parameters, as the estimation routine is very slow.4 This
routine is as follows. First, we simulated 1,000,000 network models using the
estimated parameter variance-covariance matrix of the pooled MAGNA network
edge weights. Next, for each pair of variables and for each centrality index, we
computed the proportion of times the difference was below zero and the proportion
of times the difference was above zero. Finally, we took the lowest of these
proportions and multiplied the result by two to obtain a pvalue corresponding to
a two-sided difference test. The null-hypothesis for equality in centrality can then
4 Performing the MAGNA analysis reported here took over an hour of computation time on a rela-
tively powerful computer, and the multiverse analysis reported in the supplementary materials took
several days to run. In addition, these computations relied on GPU computing, and as such could
not be parallelized over CPU cores as typically done in nonparametric bootstraps of psychological
network models.
Page | 221
be rejected at different α levels using these p values. Of note, Epskamp, Borsboom,

and Fried (2018) report that the expected rejection rate given the null model of
equal centrality indices may actually be lower than a, indicating that this test can
be more conservative than expected.
Heterogeneity across studies

The MAGNA analysis returns an estimated variance-covariance matrix on the
variability around the correlational structure implied by the pooled GGM. To
ensure that this matrix is positive semi-definite, we estimated the Cholesky
decomposition of this variance-covariance matrix rather than estimating the
variances and covariances directly. This routine is further explained elsewhere
(Epskamp et al., 2020). To assess cross-study heterogeneity, we computed random
effect standard deviations by taking the square root of the estimated variances.
These random effect standard deviations give insight in how much the correlation
coefficient between two variables differs across studies after taking sampling
variation into account.
Comparison to single-study network models

We further investigated how individual network studies correspond to the obtained
pooled MAGNA network: if a network is estimated from a single sample, would
similar conclusions be drawn from the resulting network structure as would be
drawn from the meta-analytic results? To do this, for each correlation matrix
(Pearson correlations obtained through listwise deletion), we estimated network
models using four techniques that allow for correlation matrices to be used as
input: (1) the EBICglasso algorithm (Epskamp & Fried, 2018), which combines the
graphical LASSO (Friedman et al., 2008) regularization with model selection using
the extended Bayesian information criterion (EBIC; Foygel & Drton, 2010) and is
estimated through using the EBICglasso function in the qgraph package (Epskamp
et al., 2012), (2) the ggmModSelect algorithm using the qgraph package (Isvoranu
et al., 2019), which performs extensive stepwise unregularized model search, (3)
unthresholded partial correlation estimation using the qgraph package, and (4)
pruning at α=.05 using the psychonetrics package (Epskamp, 2020b; Epskamp et
al., 2020), which starts with estimating unthresholded partial correlations as in
(3), but subsequently removes non-significant edges and re-fits other edges using
maximum likelihood estimation while keeping weights of the removed edges fixed
Page | 222
to zero. We compared how well on average parameter estimates compared to the
pooled MAGNA network.
Consistency due to methodological choices

There were various methodological choices in this work leading to a plurality of
possible statistical results that could be obtained, which can be problematic especially
given the novelty of our methodology (Epskamp, 2019). Regarding the MAGNA
analysis, we chose to use a variant in which sampling variation is handled through
using the average of individually estimated sampling variation matrices. Instead of
using an average sampling variation matrix we could have also used full-information
maximum likelihood estimation and use a different sampling variation matrix per
study, and instead of using individually estimated sampling variation matrices we
could have also used a pooled sampling variation matrix. As such, there are four
variants of MAGNA analysis (Epskamp et al., 2020). Regarding the data, we used
Pearson correlation matrices obtained through listwise deletion (as these were most
in line with the assumptions underlying the MAGNA model and as we had the most
samples with these correlation matrices), but we could have also chosen to analyze
Spearman correlations or to use pairwise deletion. Some samples in our study included
observations of the same cases measured multiple times, leading to dependencies
between these samples that violate the assumption of independent samples. We
chose to include these samples in the analysis (to optimize the number of samples
and because mostly the time interval between samples was long), but we could have
also chosen to collapse correlation matrices of these samples into a single correlation
matrix per study. Finally, we chose to analyze the DSM-IV symptoms, but we could
have also chosen to analyze only symptoms shared between the DSM-IV and DSM-5
(analyzing all DSM-5 symptoms was computationally not feasible). To gain insight
in the impact of these methodological choices, we performed a multiverse analysis by
performing the analysis for each of the in total 64 potential analyses that could have
been performed (Steegen et al., 2016). Of these 64 analyses, we report the estimated
edge weights and centrality indices.
12.2.5 Bias assessment

As also highlighted in the systematic review carried out by Birkeland and colleagues
(2020), to date no established instruments yet exist to assess bias in network studies,
and regular meta-analytic instruments are not applicable for this current statistical
Page | 223
technique. Of note, the MAGNA framework accounts for the heterogeneity of the
data, as well as for the difference in sample size across the studies.
12.3 Results
In this section, we discuss the main results of our meta-analysis. More detailed
results, as well as several figures that were outside the scope of this chapter can be
found in the appendices of this chapter.
12.3.1 Systematic search

The study selection process is displayed in Figure 12.1; in total, the search returned
260 articles. After removing 140 duplicates, records were screened by title and
abstract for eligibility. For 70 articles where this was unclear, the full text was
further examined, and articles were excluded based on the criteria described above,
resulting in 33 eligible studies to be included in the meta-analysis, with a total of 52
samples that could be used in the analysis Table 12.1 presents study characteristics
for each included study. Figure I1 in Supplement I presents an overview of the
cumulative number of articles, as well as the number of articles published per year.
Sample sizes ranged from 124 to 1790, with a median sample size of 418 and a
mean sample size of 568.48 (SD=401.05). The total sample size was 29,561 cases
across all samples.
Based on our variable selection (i.e., symptoms that adhere to DSM-IV criteria for
PTSD), we identified 17 symptoms (further described in Table I1): intrusive thoughts,
nightmares, flashbacks, psychological reactivity, physiological reactivity, internal
avoidance, external avoidance, amnesia, loss of interest, feeling detached, emotional
numbing, irritability/ anger, hypervigilance, easily startled, difficulty concentrating,
sleep disturbance, and hopelessness.5 Thus, we employed MAGNA estimation on all
symptoms from the DSM-IV (17 symptoms). Not all studies included all symptoms,
but full information maximum likelihood estimation in MAGNA can handle
missing nodes. Figure 12.2 shows the number of studies that report each pair of
symptoms analyzed. A higher number of samples for each pair of variables is available
for the common DSM-IV and DSM-5 symptoms across different studies (i.e., the
5 Of note, analyses including all symptoms measured across all studies, as well as analyses including only
DSM-5 symptoms were not feasible due to a too high model complexity for the software to handle, as
well as a low number of studies including either DSM-5 symptoms or a wide array of other symptoms.
Page | 224
item ‘hopelessness’ is specific to the DSM-IV and not measured across all studies).
Some studies featured two variables designed to measure the same symptom, in which
case the average correlation with both variables was used as input to MAGNA.
Identification
Records identified through Additional records identified

database searching through other sources
(n = 258) (n = 2)
Records after duplicates removed

(n = 140)
Screening
Records screened Records excluded based on title and

(n = 140) abstract (n = 70)
Full-text articles assessed Full-text articles excluded, with

for eligibility reasons (n = 37)
Eligibility
(n = 70)
Not cross-sectional (n=2)
Binary datasets (n=2)
Not empirical (n=2)
Studies included in PTSD as one node only (n=1)
qualitative synthesis Records in another domain
(n = 33) of network science (n=19)
Data inadequacy (n=1)
No response to data request (n=8)
Included
Same data as another study (n = 2)

Studies included in
quantitative synthesis
(meta-analysis)
(n = 33)
Figure 12.1 PRSIMA flow diagram.
12.3.2 Pooled MAGNA edge weights

Figure 12.3 shows the estimated pooled MAGNA network for the DSM-IV PTSD
symptoms, and Table I2 presenta numeric results in a table. In Figure 12.3, all
edges not significant at α=.05 are hidden. Figure 12.4 and Figure I3 in Supplement
I furthermore show the estimated edge weights, the 95% confidence regions of
edges all edges, and the significance of these edges. The significance level of α=.0004
corresponds to an α level of .05 corrected for 136 tests and rounded to 4 digits.
Within the pooled MAGNA network structure across all DSM-IV symptoms,
notably all significant edges between all pairs of items were positive. Further, several
Page | 225
strong and stand-out links between symptoms emerged. The three strongest edges
were ‘easily startled’ – ‘hypervigilant’, ‘external avoidance’ – ‘internal avoidance’
and ‘emotional numbing’ – ‘feeling detached’. Also strong were the edges
‘physiological reactivity’ – ‘psychological reactivity’, ‘feeling detached’ – ‘loss of
interest’, and ‘flashbacks’ – ‘intrusive thoughts’, and three edges linked to the node
‘nightmares’: ‘sleep disturbance’, ‘intrusive thoughts’, and ‘flashbacks’. The network
model furthermore showed some clustering, most notably a cluster emerged with
the nodes ‘nightmares’, ‘flashbacks’, ‘sleep disturbance’ and ‘intrusive thoughts’, and
another cluster emerged with the nodes ‘emotional numbing’, ‘feeling detached’,
‘loss of interest’, and ‘hopelessness’.
intrusive thoughts
psychological reactivity
nightmares
physiological reactivity
flashbacks
intrusive thoughts 49
nightmares 45 46
internal avoidance
flashbacks 39 37 40
external avoidance
psychological reactivity 40 36 35 40
physiological reactivity 33 33 32 32 33
internal avoidance 47 45 40 39 32 48
loss of interest
amnesia
feeling detached
external avoidance 49 46 40 40 33 48 50
emotional numbing
amnesia 39 36 38 34 32 38 39 39
irritability anger
loss of interest 34 34 34 30 30 34 34 34 34
feeling detached 43 43 35 35 30 43 44 34 34 44
hypervigilant
difficulty concentrating
emotional numbing 43 44 36 35 31 44 44 35 34 43 44
easily startled
hopelessness (DSM IV only)
irritability anger 44 40 38 39 32 42 44 39 34 39 39 44
sleep disturbance
hypervigilant 48 46 40 39 33 48 49 39 34 43 44 43 49
easily startled 48 46 40 39 33 48 49 39 34 43 44 43 49 49
difficulty concentrating 39 35 38 35 31 38 39 38 34 35 35 39 38 38 39
sleep disturbance 38 35 38 34 31 38 38 38 34 34 35 38 38 38 38 38
hopelessness (DSM IV only) 23 23 23 19 19 23 23 23 23 23 23 23 23 23 23 23 23
Figure 12.2 Number of samples for each pair of variables for which Pearson correlations based on
listwise deletion were available. Sample sizes for other types of correlations can be seen in Figure I2 in
Supplement I.
Page | 226
psychological
reactivity
physiological
internal reactivity
avoidance
flashbacks
external
avoidance
intrusive
thoughts
easily
hypervigilant startled
nightmares
difficulty
concentrating
sleep
disturbance
amnesia irritability
anger
hopelessness
feeling loss of
detached interest
emotional
numbing
Figure 12.3 Estimated pooled MAGNA network for Pearson correlation matrices (listwise deletion)
including all DSM-IV PTSD symptoms. Nodes represent PTSD symptoms, and edges represent
partial correlation coefficients (all partial correlations in this plot are positive). Edges with weights
that were not significant at α=.05 are not shown.
Page | 227
Figure 12.4 Estimated edge weights of the pooled MAGNA and 95% confidence regions based on
the estimated standard errors. Only edges significant at α=.05 are shown. For estimates and confi-
dence regions of all edges, see Figure I3 in Supplement I. The α=.0004 level corresponds to a Bonfer-
roni corrected α level of .05.
Page | 228
12.3.3 Pooled MAGNA centrality indices
Figure 12.5 shows common centrality indices of nodes in the pooled MAGNA
network, and Figure 12.6 shows the results of our parametric bootstrapped difference
tests. The main finding was that there was no single or small set of nodes that clearly
played a more central role than other nodes. While several significant differences
could be detected between nodes on some of the metrics, these differences tended
to be small, with the exception of ‘amnesia’ clearly being the least central symptom,
both visually as well as significantly according to all metrics.
Overall, the metrics of direct connectivity (strength, expected influence, and
R ) strongly aligned with one-another. These results showed that ‘feeling detached’,
2
‘intrusive thoughts’ and ‘physiological reactivity’ were the most central nodes, being
significantly higher than all other nodes in terms of expected influence, than all
but one node in terms of R 2 , and significantly higher than several other nodes in
terms of strength. These nodes were followed by a large set of nodes of which with
similar levels of centrality: ‘easily startled’, ‘psychological reactivity’, ‘nightmares’,
‘difficulty concentrating’, ‘external avoidance’, ‘internal avoidance’, ‘loss of interest’,
‘emotional numbing’ and ‘flashbacks’.
The metrics of indirect connectivity, closeness and betweenness, diverged more
from one-another, with the exception that the symptoms ‘nightmares’ and ‘sleep
disturbance’ were the highest in both and also significantly higher than several
other nodes. The symptom ‘difficulty concentrating’ was also significantly higher
than several other symptoms in terms of closeness, but not in terms of betweenness.
The closeness metric revealed very little deviance between nodes, while betweenness
revealed more deviance. Betweenness, however, also featured fewer significant
differences, and more deviations across methodological choices in our multiverse
analysis (see Figure I5 in Supplement I). As such, betweenness centrality may not
be very stable even in our meta-analytic results, which is in line with commonly
reported instability of this metric (Epskamp, Borsboom, et al., 2017).
Page | 229
Strength Expected Influence R2 (Predictability) Closeness Betweenness
intrusive thoughts
nightmares
flashbacks
internal avoidance
external avoidance
amnesia
loss of interest
feeling detached
emotional numbing
irritability anger
hypervigilant
easily startled
sleep disturbance

0.0
0.3
0.6
0.9
0.0
0.3
0.6
0.9
0.0
0.1
0.2
0.3
0.4
0.5
0.000
0.001
0.002
0.003
0.004
10
20
30
Figure 12.5 Estimated centrality indices of the pooled MAGNA.
Page | 230
Node Strength R2 (Predictability)
feeling detached intrusive thoughts
intrusive thoughts feeling detached
physiological reactivity physiological reactivity
easily startled psychological reactivity
psychological reactivity nightmares
nightmares emotional numbing
difficulty concentrating easily startled
external avoidance internal avoidance
emotional numbing loss of interest
internal avoidance external avoidance
flashbacks flashbacks
loss of interest difficulty concentrating
sleep disturbance hypervigilant
hopelessness sleep disturbance
irritability anger hopelessness
hypervigilant irritability anger
amnesia amnesia
amnesia
hypervigilant
irritability anger
hopelessness
sleep disturbance
loss of interest
flashbacks
internal avoidance
emotional numbing
external avoidance
nightmares
easily startled
intrusive thoughts
feeling detached
amnesia
irritability anger
hopelessness
sleep disturbance
hypervigilant
flashbacks
external avoidance
loss of interest
internal avoidance
easily startled
emotional numbing
nightmares
feeling detached
intrusive thoughts
Expected Influence Closeness
feeling detached nightmares
physiological reactivity sleep disturbance
intrusive thoughts difficulty concentrating
psychological reactivity intrusive thoughts
difficulty concentrating psychological reactivity
easily startled physiological reactivity
internal avoidance external avoidance
emotional numbing loss of interest
loss of interest irritability anger
external avoidance internal avoidance
nightmares flashbacks
flashbacks easily startled
sleep disturbance feeling detached
hypervigilant hopelessness
hopelessness hypervigilant
irritability anger emotional numbing
amnesia amnesia
amnesia
irritability anger
hopelessness
hypervigilant
sleep disturbance
flashbacks
nightmares
external avoidance
loss of interest
emotional numbing
internal avoidance
easily startled
intrusive thoughts
feeling detached
amnesia
emotional numbing
hypervigilant
hopelessness
feeling detached
easily startled
flashbacks
internal avoidance
irritability anger
loss of interest
external avoidance
intrusive thoughts
sleep disturbance
nightmares
Betweenness
nightmares
sleep disturbance
loss of interest
intrusive thoughts p > .05
easily startled p < .05
external avoidance
physiological reactivity p < .01
internal avoidance
psychological reactivity p < .001
feeling detached
hopelessness
emotional numbing
p < .0004
flashbacks
irritability anger
hypervigilant
amnesia
amnesia
hypervigilant
irritability anger
flashbacks
emotional numbing
hopelessness
feeling detached
internal avoidance
external avoidance
easily startled
intrusive thoughts
loss of interest
sleep disturbance
nightmares
Figure 12.6 Centrality difference plots obtained through a parametric bootstrap. Each block indicates
the significance of the difference between centrality indices of two nodes. These were obtained by
sampling 1,000,000 network structures from the estimated asymptotic parameter variance-covariance
matrices. The α=.0004 level corresponds to a Bonferroni corrected α level of .05 rounded to 4 digits.
Page | 231
12.3.4 Heterogeneity across studies

Figure 12.7 shows the estimated random effect standard deviations on the
correlational structure implied by the pooled MAGNA network. As can be seen in
the figure, the standard deviations of the random effects are fairly large among all
possible correlations, ranging from 0.10 to 0.18. These values therefore are larger
than prior reported random effect sizes when only four of the samples were analyzed
using MAGNA estimation, and were also larger than the largest random effect
sizes used in simulation studies (Epskamp et al., 2020). While random effect sizes
were quite uniformly distributed over all possible correlations, it can be noted that
in general pairs of variables that featured a strong edge in the pooled MAGNA
network also featured higher random effect sizes on the correlations.
intrusive thoughts
nightmares
flashbacks
nightmares 0.13
flashbacks 0.13 0.14

internal avoidance
psychological reactivity 0.15 0.15 0.15

external avoidance
physiological reactivity 0.10 0.13 0.14 0.14
internal avoidance 0.12 0.12 0.12 0.15 0.12
external avoidance 0.12 0.13 0.14 0.15 0.13 0.18

loss of interest
amnesia
feeling detached
amnesia 0.13 0.14 0.13 0.13 0.13 0.14 0.13

emotional numbing
loss of interest 0.12 0.14 0.13 0.11 0.12 0.13 0.12 0.14
feeling detached 0.12 0.13 0.13 0.12 0.12 0.13 0.12 0.10 0.14
irritability anger
emotional numbing 0.12 0.12 0.11 0.12 0.13 0.12 0.14 0.12 0.12 0.12
hypervigilant
irritability anger 0.12 0.12 0.12 0.12 0.13 0.12 0.13 0.13 0.13 0.14 0.14
easily startled
hypervigilant 0.11 0.11 0.12 0.13 0.13 0.11 0.12 0.12 0.12 0.14 0.13 0.13
sleep disturbance
easily startled 0.11 0.14 0.13 0.12 0.13 0.12 0.13 0.11 0.13 0.13 0.12 0.14 0.16
difficulty concentrating 0.10 0.12 0.11 0.11 0.12 0.13 0.11 0.13 0.14 0.12 0.11 0.15 0.13 0.14
sleep disturbance 0.11 0.14 0.13 0.10 0.13 0.12 0.12 0.14 0.12 0.13 0.13 0.15 0.11 0.12 0.13
hopelessness (DSM IV only) 0.11 0.10 0.12 0.10 0.12 0.11 0.11 0.13 0.15 0.10 0.12 0.15 0.10 0.14 0.10 0.11
Figure 12.7 Estimated random effect standard deviations on the model-implied marginal correlation
structure among DSM-IV symptoms over studies. Higher values indicate larger differences between
studies in correlational structure.
Page | 232
12.3.5 Comparison to single-study network models
Figure 12.8 summarizes the single-study networks and how these relate to the
pooled MAGNA network structure. The top panels investigate if single-study
network models retrieve similar parameter weights (i.e., partial correlation estimates)
as obtained through MAGNA analysis when edges are included in a network, and
the bottom panels investigate if single-study network models identify the same edges
as MAGNA analysis with significance thresholding at α=.05. The top panels show
that when edges are set to non-zero, unthresholded partial correlations (pcor) and
EBICglasso estimation give, on average, parameter estimates that are closer to the
parameter values obtained through MAGNA analysis, compared to ggmModSelect
and psychonetrics (pruned at α=.05): the average deviation between the edge weight
in a single-study network compared to the pooled MAGNA network was centered
around zero with pcor and EBICglasso estimation, but generally larger than zero in
ggmModSelect and psychonetrics estimation. Investigating the bottom panels: on
average, single-study networks would include the strongest edges from the pooled
MAGNA network almost always (bottom right panel). The bottom left panel
shows that EBICglasso tended to include more edges that were not included in the
pooled MAGNA network than ggmModSelect and psychonetrics. However, the top
left panel shows that these edges were subsequently also estimated to be weaker
when included in the network, compared to ggmModSelect and psychonetrics. The
EBICglasso also included more often edges that were significant in the MAGNA
analysis than the ggmModSelect and psychonetrics. The pcor method did not perform
model selection, and as such always includes all edges.
12.3.6 Consistency due to methodological choices

Figure I4 and I5 in Supplement I show the results of the multiverse analysis. The
first figure shows that across all methodological choices very similar parameter
estimates would have been obtained, and the second figure shows that across
all methodological choices also very similar centrality indices would have been
obtained, especially with regard to strength and closeness. The multiverse analysis
showed the most differences on betweenness, and as such, betweenness centrality
may not be stable in these networks, which is in line with previous discussions on
this centrality index (Epskamp, Borsboom, et al., 2017).
Page | 233
untresholded partial correlations EBICglasso ggmModSelect psychonetrics (pruned at alpha = .05)
MAGNA non−significant edges (p > .05) MAGNA significant edges (p < .05)
Page | 234
0.2
0.1
0.0
when edge is non−zero

Average deviance from MAGNA
−0.1
1.00
0.75
0.50
0.25
Proportion included in network
0.00
0.00 0.01 0.02 0.03 0.1 0.2 0.3
Absolute MAGNA edge weight
Figure 12.8 Edge weight estimates based on separate analyses of the analyzed correlation matrices as single studies, compared to the results of the pooled
MAGNA network. Each correlation matrix was analyzed using four network estimation routines that take a correlation matrix as input (unthresholded/
saturated partial correlation networks, the EBICglasso algorithm, the ggmModSelect algorithm, and pruned partial correlation networks using the psychonetrics
package). Each symbol represents one of the 136 potential edges that can be included in a 17-node DSM-IV PTSD symptom network. The top panels show the
average estimated edge weight for edges when these were included in the model (estimated to be non-zero), and the bottom panels show the proportion of times
an edge was included in the model. The left panels show edges that were estimated to not differ significantly from zero at α=.05 in the pooled MAGNA network,
and the right panels show edges that were estimated to differ significantly from zero at α=.05 in the pooled MAGNA network. The x-axis shows the edge weight
in the pooled MAGNA network. Of note, unthresholded partial correlation estimation includes all edges.
12.4 Discussion
This article introduced the first ever meta-analysis of network models of PTSD.
Using the novel MAGNA (Epskamp et al., 2020) methodology, we identified
and discussed common effects of current research by estimating a singled pooled
network structure of frequently assessed post-traumatic symptoms, as well a
single pooled centrality plot of these symptoms. Further, we investigated the size
of between-study heterogeneity by assessing random-effect deviations from the
correlational structure that is implied by the pooled network structure. Finally, we
examined how well individual network studies perform in retrieving this pooled
cross-study network. Through a multiverse analysis reported in Supplement I, we
showed that these results were consistent through various methodological choices
that could have been made.
12.4.1 PTSD symptom associations

Overall, the identified common network structure was relatively dense, with high
intercorrelations between symptoms. Some especially stronger connections stood
out, most of these being within-cluster associations, as defined by the DSM-IV.
These included associations within a cluster of symptoms related to cognition and
mood: ‘emotional numbing’, ‘feeling detached’, ‘loss of interest’, ‘hopelessness’, a
cluster of symptoms related to re-experiencing: ‘nightmares’, ‘intrusive thoughts’,
‘flashbacks’, a cluster of the reactivity symptoms that is tightly linked to the re-
experiencing cluster, a cluster of the two symptoms on avoidance and a cluster of the
symptoms ‘hypervigilant’ and ‘easily startled’. Between these clusters, the symptom
‘sleep disturbance’ formed a major connection between the cognition and mood
and re-experiencing clusters, the symptom ‘difficulty concentrating’ formed bridges
between several clusters and the avoidance and hyperactivity/ startle clusters, and
there were strong links between the avoidance and the reactivity clusters. Finally,
the node ‘amnesia’ did not clearly cluster together with any of the other symptoms.
12.4.2 Centrality of PTSD symptoms

A main finding of the current meta-analysis is that there were no symptoms that
clearly played a most central role in the network. While the three symptoms ‘feeling
detached’, ‘intrusive thoughts’, and ‘physiological reactivity’ featured consistently
amongst the most central across metrics of direct connectivity, the symptoms
‘nightmares’ and ‘sleep disturbance’ featured higher on metrics of indirect
Page | 235
connectivity (because of their role in connecting the cognition and mood and re-
experiencing clusters). Further, several more symptoms were high across several
metrics (such as ‘psychological reactivity’ and ‘difficulty concentrating’), but few of
these most central nodes showed strong significant differences among each-other
across all metrics, nor did their raw centrality values differ much visually.
This is an important finding, especially in light of previous research that
identified high heterogeneity in terms of what individual studies determine
as central items (Birkeland et al., 2020). If we assume central symptoms to be
especially relevant for treatment, as suggested by previous research (Borsboom &
Cramer, 2013; Fried et al., 2018; Rodebaugh et al., 2018a; Schmittmann et al.,
2013) current results–at least within the PTSD framework–indicate symptoms
are mostly indistinguishable in centrality. Future research expanding the MAGNA
methodology to other mental disorders may identify whether this is a general result
extendable to most psychopathology, or whether it is a specific finding of the PTSD
network literature. In addition, while the current analyses identified little evidence
for a specific set of symptoms that are especially central across all subgroups, it may
be that in individual subgroups, more pronounced differences in centrality exist,
which could ultimately become important intervention targets.
While we did not identify symptoms that played a clear most central role, we did
identify a symptom that clearly was the least central: The symptom ‘amnesia’ was
significantly less central than most other symptoms on all five centrality indices.
This aligns well with previous studies in the field, including factor analytic studies
in which amnesia clearly stands out due to weak loadings (Armour et al., 2016;
Berntsen & Rubin, 2014; Birkeland et al., 2020; Rubin et al., 2008). In line with
such findings, the current meta-analysis further raises the question of whether
amnesia is indeed part of PTSD, whether it arises from external factors or other
comorbid disorders, or whether the item, dating back to the outset of PTSD
research, requires serious reconsideration in the light of this accumulating evidence.
12.4.3 Heterogeneity and generalizability of PTSD networks

In line with our previous analysis of only four samples (Epskamp et al., 2020) we
estimated large random effect sizes on the correlational structure, indicating large
differences between study domains. This has some important implications for the
literature on PTSD network models. First, we cannot expect a single sample to
recover a structure that is fully generalizable across all possible PTSD samples,
regardless of quality (e.g., sample size, reliability of measurement) of the study.
Page | 236
This is not surprising, as PTSD samples can consist of vastly different samples of
subjects from different cultures exposed to different types of trauma. Second, we
previously studied the performance of estimating a single pooled network without
taking study-heterogeneity into account (Epskamp et al., 2020) and found that this
performs poorly in the presence of strong random effect deviations. In particular,
specificity could severely drop in such an aggregated network model, indicating
that the model would include many spurious edges. To this end, PTSD network
literature that aims to aggregate over multiple studies should use methods that
take heterogeneity across study domains into account, such as the random-effects
MAGNA methodology used in this chapter. Finally, large cross-study heterogeneity
means that, even when comparing high quality samples such that expected
replicability given the estimation method is high (Williams, 2020), it cannot be
expected PTSD network models replicate perfectly across such diverse samples. This
marks an important finding that is relevant especially also to an ongoing discussion
on replicability of PTSD symptom networks. For example, in several studies Forbes
and colleagues (2017b, 2019, 2021) showcase differences in–mostly very weak–
edges in analyses based on multiple PTSD samples, and describe these differences
as “evidence for limited replicability” of network estimation tools. Such differences
however can readily be explained as a result of cross-study heterogeneity in addition
to sampling variation (Fried et al., 2020; Williams, 2020) and the performance of
network estimation tools (Isvoranu & Epskamp, 2021; see Chapter 13).
In the current chapter we identified very heterogeneous correlations between
both symptoms that featured strong edges in the network (such as ‘internal
avoidance’ and ‘external avoidance’) and symptoms with less prominent edges in
the GGM network structure, connecting different clusters (such as ‘nightmares’
and ‘easily startled’). It is important to note that the estimated random-effect size
is on the implied correlational structure, not on the GGM network itself. However,
we may expect that large random effects on marginal correlations between two
variables will translate to larger differences in direct edges between these variables
when a network model is estimated from a single sample. To this end, it could be
that large between-study differences are also to be expected in which edges will
be found to bridge different clusters in the network. As more and more studies on
PTSD are being conducted, more data will be able to address the heterogeneity
discussed above. Future meta-analytic studies carried out on specific sample
subpopulation and measures, when these become sufficient, may be able to identify
such differences.
Page | 237
12.4.4 How do single-study network analyses compare to meta-analytic network analyses?

In addition to estimating a pooled meta-analytic network structure, we studied
the correspondence between network models estimated from single datasets to the
network models estimated using MAGNA analysis on all datasets. We investigated
both regularized network estimation, as well as non-regularized model search. To
summarize, we found that all methods estimated similar structures (absence and
presence of edges), as well as parameter values (size of the partial correlation) as the
MAGNA network. Network models estimated using regularization techniques
led to more generalizable network parameters (partial correlation sizes), whereas
single-study network models estimated through unregularized model search led
to more generalizable network structures (absence and presence of edges). This is
in line with previous literature, as regularization techniques have, in part, been
developed with the specific aim to obtain parameter estimates that are less prone to
overfitting and work better in new samples (Hastie et al., 2009), but have also been
shown to perform poorer in retrieving network structures in simulation studies
(Williams & Rast, 2020).
12.4.5 Comparison to previous work

We compared our results with previous studies to check for consistency in our
findings. Mainly, we compared our results to those by Birkeland and colleagues
(2020b), who reviewed many of the same studies that were included in our meta-
analysis, and the results by Duek and colleagues (2020), who recently published a
PTSD symptom network analysis of 158,139 veterans. This study was not included
in our search, as it was published after the selected timeframe of publication. Figure
I6 in Supplement I shows a visual comparison of our meta-analytic results, the
estimated network by Duek and colleagues, and a visual representation of the most
common strongest edges reported by Birkeland and colleagues. Figure I6 reveals
overall a strong overlap between our results and those of Birkeland and colleagues
and Duek and colleagues.
Within both our pooled network structure and the review by Birkeland and
colleagues, the associations between ‘hypervigilant’ and ‘easily startled’, between
‘nightmares’ and ‘intrusive thoughts’, between ‘internal avoidance’ and ‘external
avoidance’, between ‘emotional numbing’ and ‘feeling detached’, and between
‘feeling detached’ and ‘loss of interest’ were visibly the strongest edges. Duek and
colleagues likewise also identified strong edges between all the above-mentioned
pairs of nodes. Both Duek and colleagues and our pooled network model also
Page | 238
featured a strong edge between ‘psychological reactivity’ and ‘physiological
reactivity’, which was not often among the strongest edges according to Birkeland
and colleagues. These findings indicate that in spite of high heterogeneity in terms
of type of sample (e.g., clinical, community, veteran, general population etc.), these
associations are likely to be common and emerge in most network structures.
Aligned with these findings, in the multisite study of PTSD symptoms carried
out by Fried and colleagues (Fried et al., 2018) in four trauma sample patients, these
associations were also consistently identified.
In terms of between-cluster associations, our results diverged more from
previous findings. Our analysis, like Duek and colleagues, showed that ‘nightmares’
and ‘sleep disturbance’ connected the main clusters of the network, but this was
less profound in the results by Birkeland and colleagues. The main differences can
be seen in the weaker edges connecting the various clusters. Our analysis showed
quite a large number of moderately strong positive edges connecting the clusters,
whereas Birkeland and colleagues identified far fewer connecting edges and Duek
and colleagues identified overall weaker edges including several negative edges. It
should be noted that Birkeland and colleagues only aimed to identify edges that are
often stronger than other edges, not at identifying all edges. As such, their summary
likely does not include smaller edges connecting clusters. The discrepancy between
the results by Duek and colleagues and our results could be due to differences in
estimation techniques, differences in samples, or possibly due to Berkson’s bias in
the analysis by Duek and colleagues inducing negative edges (de Ron et al., 2019).
In terms of centrality, Birkeland and colleagues only investigated strength
centrality as closeness and betweenness were not deemed stable in the reviewed
studies themselves,6 and Duek and colleagues only reported expected influence and
R 2 (predictability). Duek and colleagues made available their estimated network
structure through supplementary materials, however, which allowed us to also
investigate the other centrality indices (see Figure I7 in Supplement I). Birkeland
and colleagues identified ‘intrusive thoughts’ as the most often occurring most
central symptom and ‘amnesia’ as the most often occurring least central symptom,
which aligns with our results. Birkeland and colleagues furthermore identified the
symptoms ‘loss of interest’, ‘physiological reactivity’, ‘feeling detached’, ‘difficulty
concentrating’ and ‘hypervigilance’ as often occurring strong central symptoms,
6 Of note, while our analysis includes many of the same studies reviewed by Birkeland et al. (2020),
our analysis does not depend on the estimated network structures of these studies. As such, our
analysis is not impacted by potential instability in network models estimated from single studies.
Page | 239
most of which were also relatively central nodes according to our direct metrics of
connectivity, with the exception of ‘hypervigilance’. Duek and colleagues identified
‘feeling detached’, ‘intrusive thoughts’, ‘psychological reactivity’, ‘physiological
reactivity’ and, to a lesser extent, ‘loss of interest’ as central nodes according to
expected influence and R 2 , and ‘amnesia’ as the least central node. These results
align well with our findings. Figure I7 in Supplement I shows a strong overlap
between our centrality metrics and the ones based on the network reported by Duek
and colleagues, with the exception of ‘betweenness’. Of note, R 2 is on average a bit
higher in the network reported by Duek and colleagues, but this can be explained
also by our findings that edge weights from the ggmModSelect algorithm (the
algorithm used by Duek and colleagues) leads to slightly higher estimated edge
weights than the meta-analytic results.
In sum, previous results align well with our meta-analytic results. The review of
Birkeland and colleagues, the comparison with the results by Duek and colleagues,
and the results from our single study analysis show that conclusions drawn from
networks estimated from single studies can align with meta-analytic results,
regardless of whether the meta-analytic results are based on the same samples (as
is the case with our own single dataset study as well as the results of Birkeland
and colleagues) or not (as is the case with the results from Duek and colleagues).
Often, the same edges were identified as being the strongest and the same nodes
were identified as being among the most central.
12.4.6 Limitations
An important limitation of the current study consists of the restricted number of
symptoms included in the MAGNA analyses. While here we focused on DSM-IV
symptomatology (due to these being the symptoms assessed by most articles), many
studies included many more symptoms that we were unable to include in the network
structure. The multiverse results in Figure I4 and Figure I5 in Supplement I show
estimated edge weights based on using either DSM-IV symptoms or only the shared
symptoms between the DSM-IV and the DSM-5 (all DSM-IV symptoms except
‘hopelessness’). This figure shows a strong symmetry between these results. Due to a
lack of studies investigating only DSM-5 symptoms we could not perform the analysis
for DSM-5 symptoms, which would be an important avenue for future research.
A further limitation is the restricted availability of bias assessment methods for
network models. While the MAGNA methodology is able to address bias due to
heterogeneity and distinct sample sizes, other sources of bias were not accounted for
Page | 240
here, such as correlation bias7 due to the sample population (i.e., risk of Berkson’s
bias due to selection criteria; de Ron et al., 2019), or due the data type used as
input when calculating the correlations (e.g., risk of biased estimates by calculating
Pearson correlations based on ordinal data; Epskamp, 2017). We recommend
future research to focus on the investigation of potential sources of bias in network
studies, as well as identifying the size of such bias. Since network studies do not
use significance testing, publication bias is not an issue with the current analysis.
Final limitations involve the novel aspects of MAGNA estimation, which is
not yet capable of handling all characteristics found in real datasets. For example,
MAGNA treated data as continuous, while in fact many samples were ordered
categorical in nature and measured on a Likert scale with 3 to 5 response categories.
In Supplement I (see Figure I4 and Figure I5) we also investigated Spearman
correlations (of which we obtained fewer samples), which shows results to align
with the ones reported in this chapter. Nonetheless, a proper way of handling
ordered categorical data is still lacking. This is a general limitation that also
holds to MASEM modeling; while ordered categorical data can in principle be
handled in structural equation modeling and network psychometrics through
the use of polychoric correlations (Muthén, 1984), such polychoric correlations
cannot be used as input to MAGNA, as MAGNA uses the normal likelihood
to compute the sampling variation matrix. As such, a future direction may be to
better handle ordered categorical data in meta-analytic network modeling. Possible
future directions would be to extend MAGNA for handling potential violations
of independence. Alternatively, one could only include one sample or average
the samples. However, we opted to include all samples instead as the number of
studies with multiple samples of the same participants were low, usually included
large time-lags between samples, and we aimed to include all available data in the
analysis.

The field of network psychometrics has grown extremely popular within the past
decade, with numerous studies being published using this novel methodology.
Within this field, the PTSD research area has been especially fast expanding,
allowing for the first ever meta-analysis of PTSD network models.
7 The MAGNA framework relies on correlation matrices as input, and not on effect sizes. Therefore,
the risk of bias here is equivalent to the risk of bias for correlation structures.
Page | 241
The current study summarized and synthesized findings in the field of network
models of PTSD, with the aim of advancing current knowledge and bringing
together existing results. Our results highlighted common associations between
symptoms that are likely to emerge across distinct studies and populations, but
also very high heterogeneity between studies. Notably, there may not be such a
thing as one overall PTSD network structure and future research may benefit from
focusing on sub-populations (e.g., veterans) when aiming to construct a pooled
PTSD network structure. To date, the amount of data available is not sufficient
for such an analysis. In addition, and of note, over 16 different measures of PTSD
symptomatology were used across the pool of studies included in this meta-analysis,
further highlighting also high heterogeneity in measurement. Summarizing results
when the measures employed are themselves heterogeneous across studies is an
important challenge in the field of psychopathology. Finally, it may be that research
on PTSD symptom networks may not be fruitful in trying to present a generalizable
PTSD symptom network. An increase in the number of studies that are more
homogenous could result in specific meta-analytic research in subpopulations,
leading to estimated meta-analytic network models that are representative for those
select subpopulations.
Further, an important finding here is that most centrality estimates were
indistinguishable from each other, except for the symptom ‘amnesia’ which was
clearly the least central symptom. A wide array of research to date has argued that
centrality may be important for treatment interventions, though recent research
also argued that centrality measures for network models are not as straightforward
as previously thought (Bringmann et al., 2019; Rodebaugh et al., 2018b). Based
on our results, centrality estimates for PTSD symptoms, especially those taken
directly from graph theory (strength, closeness and betweenness) show few strong
differences, thus intervening on one symptom may not bring a substantial change
to the network structure. Of note here, it may be that in individual subgroups,
more pronounced differences in centrality exist, which could ultimately become
important intervention targets. Investigating this in more homogeneous samples
may be an important next step for research.
Finally, our results provide, for the first time, empirical insight in how well network
models estimated from single studies compare to generalizable network structures
across studies. Overall, most existing methods perform adequately in retrieving a
network structure which is close to the pooled network structure. As such, networks
estimated from a single PTSD sample can give results that may also generalize to
Page | 242
other PTSD samples, especially when investigating strongest edges. However, at the
same time we should not expect a single sample to result in a network model that will
hold true for all other potential samples, nor should we expect that a single network
model will hold true for all potential samples, as we identified large between-study
heterogeneity. Possibly, applying meta-analytic techniques to other fields of interest in
network psychometrics–such as depression, anxiety and psychosis–may lead to similar
conclusions. If this is true, then we cannot expect network studies to fully replicate
in new samples regardless of the quality of sample, simply due to the presence of
heterogeneity between study samples. To this end, the development of meta-analytic
techniques to aggregate samples of interest for network models while taking between-
study heterogeneity into account may be an important avenue for future research in
many fields of interest.
Page | 243
Which Estimation Method
13
to Choose?
Deriving Guidelines for
Applied Researchers

Isvoranu, A. M., Epskamp, S. (2021). Which Estimation Method to Choose
in Network Psychometrics? Deriving Guidelines for Applied Researchers.
Chapter 13 | Which Estimation Method to Choose?
Abstract
The Gaussian Graphical Model (GGM) has recently grown popular in psychological
research, with a large body of estimation methods being proposed and discussed
across various fields of study, and several algorithms being identified and recommend
as applicable to psychological datasets. Such high-dimensional model estimation,
however, is not trivial, and algorithms tend to perform differently in different
settings. In addition, psychological research poses unique challenges, including
placing a strong focus on weak edges (e.g., bridge edges), handling data measured
on ordered scales, and relatively limited sample sizes. As a result, there is currently
no consensus regarding which estimation procedure performs best in which setting.
In this large-scale simulation study, we aimed to overcome this gap in the literature
by comparing the performance of several estimation algorithms suitable for gaussian
and skewed ordered categorical data across a multitude of settings, as to arrive at
concrete guidelines from applied researchers. In total, we investigated 60 different
metrics across 564,000 simulated datasets. We summarized our findings through
a platform that allows for manually exploring simulation results. Overall, we found
that an exchange between discovery (e.g., sensitivity, edge weight correlation) and
caution (e.g., specificity, precision) should always be expected and achieving both–
which is a requirement for perfect replicability–is difficult. Further, we identified
that the estimation method is best chosen in light of each research question
and highlighted, alongside desirable asymptotic properties and low sample size
discovery, results according to most common research questions in the field.
Page | 246
13.1 Introduction
The Gaussian graphical model (GGM; Epskamp, Waldorp, Mõttus, & Borsboom,
2016; Lauritzen, 1996)–a network structure of variables represented by nodes
and linked by edges that are weighted by partial correlation coefficients–has
recently grown popular in psychological research, especially in the fields of
clinical psychology and psychiatry (Robinaugh, Hoekstra, et al., 2020). While
confirmatory fit of a given GGM is possible (Epskamp, Rhemtulla, et al., 2017;
Kan et al., 2019), in most cases a prior theoretical network structure is absent.
Many researchers therefore focus on exploratory estimation of GGMs: identifying
the structure (absence and presence of edges), as well as estimating the edge weights
(i.e., partial correlation coefficients). A large body of estimation methods have
been proposed and discussed across various fields of study (e.g., Drton & Perlman,
2004; Friedman, Hastie, & Tibshirani, 2008; Meinshausen, Meier, & Bühlmann,
2009), and several algorithms have been identified and recommend as applicable
to psychological datasets (e.g., Epskamp & Fried, 2018; Williams, Rhemtulla,
Wysocki, & Rast, 2019).
Of note, however, such high-dimensional model estimation is not trivial–
especially when the sample size is small relative to the number of potential
parameters–and algorithms tend to perform differently in different settings, such
as being more or less conservative with balancing the rate of discovering true edges
to the rate of discovering false edges. In addition, psychological research poses
unique questions and problems, including placing a strong focus on weak edges
(e.g., bridge edges), handling data measured on ordered scales, and relatively limited
sample sizes. As a result, there is currently no consensus regarding which estimation
procedure performs best in which setting. In this chapter, we aim to overcome this
gap in the literature by comparing the performance of several estimation algorithms
suitable for gaussian and skewed ordered categorical data across a multitude of
settings in a large-scale simulation study, as to arrive at concrete guidelines from
applied researchers.
13.1.1 GGM estimation from ordered categorical data

A common method for estimating GGM structures widely used in prior literature is
the EBICglasso algorithm (Epskamp & Fried, 2018), which estimates a regularized
GGM based on a correlation matrix as input, by combining the graphical LASSO
algorithm (glasso; Friedman et al., 2008) with the extended Bayesian information
Page | 247
criterion (EBIC; Chen & Chen, 2008) for tuning parameter selection. For ordered
categorical data, often a polychoric correlation matrix (Olsson, 1979) is used as
input. This routine is problematic for several reasons: First, recently Williams &
Rast (2020) highlighted that regularization is not required for models with large
sample sizes compared to the number of nodes, and may lead to poor estimation
in some cases. In particular, the tuning parameter selection performs poorly when
regularized GGM structures are used in EBIC computation, especially when
these networks are dense (i.e., contain many edges). Second, the use of polychoric
correlations as input for a likelihood-based estimation method is not the optimal
method of estimation in related modeling frameworks, such as structural equation
modeling. Within these frameworks, it is recommended to use weighted least
squares (WLS) estimation instead, to more properly handle sampling variation
in the data (Muthén, 1984). This is particularly important in smaller sample
sizes, as the polychoric correlations have been shown to cause large amounts of
sampling variation in network structures (Forbes et al., 2019b). Finally, it should
be noted that very little methodological research has studied the performance of
using polychoric correlations as input to the EBICglasso, and the studies that did
(Epskamp, 2017; Williams et al., 2019) only investigated variables that were not
skewed. This is in stark contrast with reality, where data used in GGM estimation
from psychological datasets are often skewed and measured on ordered categorical
scales (in psychological data often ranging between 3-point and 5-point scales).
13.1.2 Empirical questions in psychological research

Empirical questions for GGM estimation in psychological research are unique, and
warrant dedicated investigation by themselves. While network models were initially
often utilized in an aim to identify links between a wide array of symptoms pertaining
to a mental disorder (e.g., Fonseca-Pedrero et al., 2018; Fried & Nesse, 2015; McNally
et al., 2015), the field has fast advanced to the study of more complex processes, such
as investigating the comorbidity between disorders (e.g., Choi, Batchelder, Ehlinger,
Safren, & O’Cleirigh, 2017; Lazarov et al., 2019; Malgaroli, Maccallum, & Bonanno,
2018; Vanzhula, Calebs, Fewell, & Levinson, 2019), and aiming to identify bridging
links between environmental and genetic risk factors and symptomatology (Boyette
et al., 2020; Fried, Bockting, et al., 2015; Isvoranu et al., 2016, 2017, 2019). Often,
such links are weaker than symptom-symptom links and more difficult to stably and
soundly identify (Boyette et al., 2020; Isvoranu et al., 2019). However, these are often
Page | 248
essential both to the onset and progress of a mental disorder, and they may be key
aspects for successful intervention development.
Further, in recent years, replicability has been a central issue in the field of
psychology (Open Science Collaboration, 2015) and is currently of growing
interest in the field of network psychometrics (Borsboom et al., 2018; Epskamp,
Borsboom, et al., 2017; Forbes et al., 2017a; Fried et al., 2018). While recent
research showed heterogeneity is high even when focusing on one particular
disorder, and thus aiming to identify a one overall network structure replicable
across different populations is not feasible (Isvoranu, Epskamp, & Cheung, 2020;
see Chapter 12), choosing an inadequate estimation algorithm may also play an
important role in the replicability and generalizability of network structures.
Choosing an estimation technique that is best suited for the type of question
and data a researcher has is thus critical to estimate an interpretable and reliable
network structure. With the growing number of available GGM estimation
algorithms, this is however becoming more challenging and clear guidelines for
applied researchers are still lacking.
13.1.3 Aim of the chapter

The aim of the current chapter is therefore to investigate the performance of
existing GGM estimation algorithms across a multitude of settings commonly
encountered by applied researchers. Specifically, we focus on non-skewed and
skewed continuous and ordered categorical data, as psychological datasets are
often skewed and measured on continuous or ordered categorical scales. We study
13 different estimation algorithms, further described in Table 13.1. In particular,
we study the EBICglasso and ggmModSelect (two variants) algorithms that are
implemented in the qgraph R package; Epskamp et al., 2012), two variants of
full information maximum likelihood (FIML) and weight least squares (WLS)
estimation as implemented in the psychonetrics R package (Epskamp, 2020a,
2020b), two variants of mixed graphical model estimation as implemented in the
mgm R package (Haslbeck & Waldorp, 2020), two variants of Bayesian estimation
as implemented in the BGGM R package (Williams & Mulder, 2019), and two
more unregularized estimation procedures as implemented in the GGMnonreg R
package (Williams et al., 2019).
We assess the performance of these methods by simulating data from three
plausible network models (see Figure 13.1): (1) a network model derived from a
large empirical dataset of depression, anxiety and stress measures (Lovibond &
Page | 249
Lovibond, 1995), (2) a meta-analytic network model on post-traumatic stress

disorder (PTSD) symptoms, recently reported by Isvoranu and colleagues (2020;
see Chapter 12), and (3) a network model of personality traits. In the simulation
studies, we vary the number of nodes, sample size, and data type (continuous and
skewed ordered categorical), and investigate a large array of metrics designed to
assess the performance of retrieving edges, the accuracy of the edge weights, the
accuracy of centrality indices, the performance of retrieving bridging edges between
clusters of different disorders, and the replicability of these measures.
Finally, in order to summarize the results in a clear and intelligible way, as
to aid applied researchers in interpreting the results and guiding them toward
choosing suitable estimation algorithms, we created an interactive data visualization
tool using the Shiny framework (Winston et al., 2017). The interactive data
visualization tool will enable researchers to include information about their
study, such as expected sample size and network properties, as well as expected
performance attributes, and based on this information they will receive guidelines
on which estimation algorithm(s) may be best suited for their study. The current
manuscript includes a short example and demonstration of how the interactive data
visualization tool can be used, and describes results from the application to answer
several research questions.
In addition to the results reported here, all simulation results, as well as an
example of the simulation code, can be found in our online supplementary materials
at the Open Science Framework (OSF)1 and the source code of the interactive data
visualization tool can be found in our Github repository.2
13.2 Methods
13.2.1 Data transformation
To handle non-normal skewed data as well as ordered categorical data, we
varied between four different levels of transformations. In addition to studying
the performance of estimators without transforming the data, we investigated
rank transformations and nonparanormal transformations (H. Liu et al., 2009).
Transforming variables to rank orders is equivalent to using Spearman correlations
as input to estimators that use the correlation matrix as input (e.g., EBICglasso), but
also works for estimators that use raw data as input (e.g., mgm). The nonparanormal
1 https://osf.io/ycmvf/
2 https://github.com/AdelaIsvoranu/SEA/tree/thesis
Page | 250
transformation is a semi-parametric transformation designed to handle non-
normality (H. Liu et al., 2009). Finally, for several estimators we also investigated
dedicated options for studying ordered categorical variables. These we collectively
term ‘polychoric/ categorical’ transformation, and explain below for each estimator
separately. The polychoric/ categorical condition was only used when data were
ordered categorical.
13.2.2 Network estimation

In total, we investigated thirteen different network estimation methods, also
summarized in Table 13.1. For simplicity, throughout this chapter, we will assume
no missing data is present. Below we provide a short description of each estimation
algorithm investigated here.
qgraph. The qgraph package was used for two estimators: the EBICglasso, used
for regularized GGM estimation, and the ggmModSelect, used for unregularized
model search.
The EBICglasso algorithm (Epskamp & Fried, 2018) estimates a regularized
GGM based on a correlation matrix as input. The graphical LASSO (glasso;
Friedman et al., 2008) algorithm is used to estimate a range (by default 100)
of network models, and subsequently the EBIC is used to select which network
model fits best. A hypertuningparameter, γ, can be used to further penalize model
complexity, and is typically set to .5. We also set this hypertuning parameter to .5
in the simulation studies reported below. No thresholding is performed on the final
GGM by default, as the glasso already performs model selection by shrinking many
parameters to exactly equal zero. In the polychoric/ categorical condition, we used
polychoric correlations (termed cor_auto in line with the function used to obtain
these correlations) as input to the EBICglasso algorithm.
The ggmModSelect algorithm is a potentially fast non-regularized algorithm for
network estimation, based–like the EBICglasso–primarily on the glasso algorithm
in combination with BIC model selection. The ggmModSelect algorithm was
implemented in qgraph in response to the work by Williams & Rast (2020) and
has been used in prior literature (Isvoranu et al., 2019; Kan et al., 2019), but not
yet investigated in detail in published reports. Like the EBICglasso, the algorithm
generates a range of network structures (by default 100) by varying the glasso
tuning parameter. The key difference with the EBICglasso is that subsequently,
each network is re-estimated without regularization (also using glasso, but by
Page | 251
keeping the zeroes of each network fixed to zero and by setting the regularization
tuning parameter to zero). These non-regularized estimates are subsequently used
to optimize the EBIC (by default, γ=0 in which case the EBIC reduces to the
BIC), which should lead to better model selection than the EBICglasso algorithm,
as asymptotic convergence for the true model to be selected by EBIC is only valid
for non-regularized maximum likelihood estimates of the network parameters
(Foygel & Drton, 2010). After this process, the ggmModSelect algorithm continues
optimization by stepwise adding and removing edges until the EBIC criterion
is optimized (this can be slow in larger datasets, and can be disabled using the
stepwise=FALSE argument). In the simulations, we set γ=0, and use polychoric
correlations as input in the polychoric/ categorical condition.
psychonetrics. The psychonetrics package (Epskamp, 2020b, 2020a) includes
GGM estimation through the same estimators as typically used in structural
equation modeling (SEM) packages. In particular, the psychonetrics package mimics
the often used lavaan package (Rosseel, 2012) by including the full information
maximum likelihood (FIML) and weighted least squares (WLS) estimators. The
FIML estimator treats the data as continuous and assumes normality to estimate
the parameters, while the WLS estimator uses a weights matrix to minimize the
difference between observed and model implied correlational structures. For
ordered categorical data, the three-stage WLS estimator (Muthén, 1984) can be
used to model polychoric correlations: univariate and pairwise information is used
to estimate the thresholds (stage 1) and polychoric correlations (stage 2), and finally
a model is fitted to reproduce the thresholds/ polychoric correlations (stage 3).
This three-stage WLS estimator has not yet been used in GGM estimation, and is
evaluated in the polychoric/ categorical condition of our simulation studies. For
the FIML estimator, no polychoric/ categorical condition was available.
Both estimators can use pruning to estimate a network model. This process
starts with a GGM in which all edges are included, removes all edges that are
not significant at α=.01, and re-estimates the remaining edges while keeping the
removed edge weights fixed to zero. After pruning, the model can be refined with
further stepwise model estimation. In WLS estimator, a stepwise routine is used
that adds edges with the largest modification index until no modification index
significant at α=.01 can be found. For the ML estimator, the more sophisticated
modelsearch algorithm can be used (Epskamp, 2020a), which is similar to the
stepwise estimator in ggmModSelect, but utilizes modification indices and
significance of edges to speed up the model search.
Page | 252
Table 13.1 Estimation methods used in the simulation study
Method R package Functions Settings Polychoric/ Categorical
EBICglasso qgraph EBICglasso γ=.5 polychoric correlations as input
ggmModSelect qgraph ggmModSelect stepwise=TRUE polychoric correlations as input
ggmModSelect_stepwise qgraph ggmModSelect stepwise=FALSE polychoric correlations as input
FIML_prune psychonetrics ggm %>% prune α=.01 N/A
FIML_prune_ ggm %>% prune %>%
psychonetrics α=.01 N/A
modelsearch modelsearch
WLS_prune psychonetrics ggm %>% prune α=.01 three-stage WLS (Muthén, 1984)
WLS_prune_stepup psychonetrics ggm %>% prune %>% stepup α=.01 three-stage WLS (Muthén, 1984)
mgm_CV mgm mgm Selection via 10-fold cross-validation variables treated categorical
mgm_EBIC mgm mgm Selection via EBC (γ=.5) variables treated categorical
BGGM_explore BGGM explore %>% select BF cutoff=3 variables treated as ordinal
BGGM_estimate BGGM estimate %>% select 95% credibility interval variables treated as ordinal
GGM_bootstrap GMMnonreg GGM_bootstrap α=.01 N/A
GGM_regression GMMnonreg GGM_regression BIC optimization N/A
Page | 253
MGM. A modeling framework closely related to the GGM is mixed graphical

modeling (MGM), implemented in the mgm software packages (Haslbeck &
Waldorp, 2020). The mgm package makes use of node-wise penalized generalized
linear models for estimating edges connected to each node, and allows for continuous,
categorical and count variables to be used. When continuous variables are used, the
model is very similar to the GGM, differing only slightly in the parameterization
used (regression coefficients between standardized variables rather than partial
correlation coefficients). The mgm package allows for model selection in two ways:
choosing a model that optimizes the EBIC, and choosing a model that optimizes
cross-validation (CV) prediction accuracy. We assess both variants in our simulation
studies below. While mgm does not handle ordered categorical data directly, it does
allow for variables to be modeled as categorical variables. In our simulations, we assess
the performance of treating ordered categorical variables as categorical.
BGGM. All estimation methods discussed so far are frequentist, in that these return
point estimates obtained by maximizing the (pseudo) likelihood of the model
given the data. An alternative to frequentist estimation is Bayesian estimation, in
which Bayes’ rule is used to obtain a posterior distribution of the parameter given
the data. This distribution can subsequently be used for detailed inference on the
parameter values. The BGGM package contains several methods for estimating
GGM structures using Bayesian sampling methods (Williams & Mulder, 2020).
The first method we include utilizes the estimate function, which can be used to
form credibility intervals on the parameter values, allowing to test if a parameter
is non-zero at a given α level (here set to .05) by checking if zero is not in the
credibility interval (Williams, 2018). The second method we include utilizes the
explore function, which utilizes Bayes factors to determine evidence for the presence
(and absence) of edges. In our simulation studies, we use a default Bayes factor of 3
to determine the presence of an edge in the selected GGM structure. Both methods
allow for handing ordinal data through the use of a Gaussian copula.
GGMnonreg. The final package we used in the simulation studies is the

GGMnonreg package for non-regularized GGM estimation (Williams et al.,
2019). This package includes two different estimation methods. The first, GGM_
bootstrap, estimates a saturated partial correlation network, and subsequently uses
non-parametric bootstrapping to assess significance of edges in order to threshold
non-significant edges at some level of α. The second method, GGM_regression, uses
Page | 254
step-up model search for nodewise regression models to optimize some information
criterion (in our simulations: BIC).
13.2.3 Network model construction

We simulated data under three network models based on large sample sizes that can
be expected to represent target network structures in psychological research. The
network structures used in the simulation study are shown in Figure 13.1, with some
more details presented in Table 13.2. We made use of non-regularized estimation
procedures in each of these datasets as to obtain parameter values that are in line
with what can be expected in a psychological setting (regularized networks would
lead to biased parameter estimates that are shrunk to zero).
First, we analyzed the short version of the Depression Anxiety Stress Scales
(DASS21; Lovibond & Lovibond, 1995), which contains 21 items aimed at
measuring depression, anxiety and stress. We obtained the data from the Open-
Source Psychometrics Project (openpsychometrics.org), which contained N=39,775
full responses on 21 items measured on a 4-point scale. We analyzed this data
using the ggmModSelect algorithm with stepwise estimation and a rank-order
transformation (Spearman correlations). We termed this the DASS21 network.
Second, we used a meta-analytic network model on PTSD symptoms reported
by Isvoranu, Epskamp & Cheung (2020; see Chapter 12), estimated using meta-
analytic Gaussian network aggregation (MAGNA; Epskamp et al., 2020). In this
meta-analysis, 52 samples, with a total sample size of N=29,561, used in published
papers for estimating PTSD symptom networks were re-analyzed to obtain a single
aggregated GGM structure. This structure includes 17 nodes representing the
DSM-IV-TR PTSD symptoms. To obtain a network with edge weights of exactly
zero (i.e., absent edges), we removed all edges that were not significant at α=.05.
We termed this model the MAGNA network.
Finally, we analyzed the bfi dataset from the psychTools package (Revelle, 2015)
designed to measure the Big 5 personality traits (Benet-Martínez & John, 1998;
Digman, 1989; Goldberg, 1990, 1993; McCrae & Costa, 1997). This dataset
consists of 2800 observations of 25 personality inventory items. We estimated a
network model using the ggmModSelect algorithm with stepwise estimation. We
termed this model the BFI network.
Page | 255
Page | 256
DASS21 MAGNA (PTSD) BFI
Figure 13.1 Network structures under which data were generated.

Table 13.2 Characteristics of the three network models used in the simulation study
Sparsity (proportion Average absolute smallword Average
Graph # Nodes # Clusters
of zeroes) edge-weight index path length
DASS21 21 3 0.41 0.78 1.17 1.39
PTSD 17 1 0.45 0.10 1.12 0.16
BFI 25 5 0.64 0.13 1.10 1.63
13.2.4. Data generation

We generated four types of data: normally distributed continuous data, skewed
continuous data, uniformly distributed ordered-categorical data with four
levels, and skewed ordered-categorical data with four levels. Figure 13.2 shows
the intended target distributions for each of these types of data. Data were first
generated as normally continuous data, after which data were transformed for the
skewed and ordered-categorical conditions. In the skewed condition, we used the
exponential function to transform data, indicating that the data were log-normally
distributed. For the ordered-categorical conditions, we used thresholds models
(Epskamp & Fried, 2018), such that data with four levels were generated that were
either uniformly distributed or skewed distributed.
Normal Skewed
Density
Density
Uniform ordered Skewed ordered

Probability
Probability
Figure 13.2 Types of data generated in the simulation studies. All data were first generated as con-
tinuous normally distributed data, after which the data was transformed to skewed data through the
exponential function and to ordered-categorical data through threshold models.
Page | 257
13.2.5 Q uantifying network estimation accuracy

We investigated a total of 60 different measures to quantify the accuracy of
network estimation, which we describe further below, with the number in brackets
indicating the order in which the metrics appear in the web application we use to
study the results.
Sensitivity: ability to identify true edges. In line with previous studies (Epskamp
& Fried, 2018), we investigated the (1) sensitivity of the network structure, which
represents the proportion of edges in the true network that were also included in
the estimated network:
Number of true edges in the estimated network

𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 =
Total number of edges in the true network
Unlike prior research, we also included four additional variants of the sensitivity.
The (2) signed sensitivity is the sensitivity as defined above, except that the
numerator only counts edges that were estimated with the same sign as edges in
Number of true absent edges in the estimated network
the true𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆
network as=‘true edges.’ To investigate if an estimation method is capable
Total number of absent edges in the true network
of detecting the strongest edges, we also investigated the sensitivity of the top
(3) 50%, (4) 25%, and (5) 10% absolute edge weights from the true network. For
example, the top 25% sensitivity gives the proportion of the 25% strongest edges
in the true network that were also included in the estimated network. Sensitivity
should increase with sample size: When sensitivity is low, there is a risk that not
all edges are 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃
detected,= which can leadofto poorer replicability and inflated visual
Total number edges in the estimated network
heterogeneity across estimated networks (Hoekstra et al., 2020; Mansueto et al.,
2020), as well as a potentially false conclusion that the generating network model
was sparse (Epskamp, Kruis, et al., 2017).
Specificity & precision: ability to not include
Total number false
of edges in the edges. Similar to prior
true network
studies, we investigated the (6) specificity of the network structure, which represents
the proportion of missing edges in the true network that were correctly not included
in the estimated network:
Number of true absent edges in the estimated network

Page | 258

𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 =
Total number of edges in the estimated network
Total number of edges in the true network
The𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆
specificity =can Number
also beofseen
trueas
absent edges
1 minus theinfalse
the estimated network
positive rate (α; Williams &
Rast, 2020). Specificity should always be high, and any trend in specificity as a
function of sample size can indicate problems with the estimation method. In
addition, we also investigated the (7) precision, which is the proportion of included
edges in the estimated networks that were also included in the true network:

𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 =
Total number of edges in the estimated network
Like specificity, precision will be low if many edges are included in the network that
are actually not in the true network (false edges). Of note, specificity and precision
can differ. For instance, suppose that a true network contains 50 edges and 50
absent edges, and suppose that we estimate a network with only two edges: a true
and a false edge. Then, specificity is high (out of 50 potential false edges, we only
included one), but precision is low (out of 2 identified edges, 50% were false edges).
To investigate the prominence of falsely detected edges, we also computed the
precision in estimated edges with the top (8) 50%, (9) 25% and (10) 10% absolute
edge weight. This led to three more variants of precision. For example, the top 25%
precision gives the proportion of the 25% strongest edges in the estimated network
that were also true edges. A low top 25% precision would indicate that false edges
are very prominent in the network structure and visualization.
Edge weight accuracy: ability to estimate precise edge weights. All measures
above (sensitivity, specificity, and precision) only investigate if an edge is included
or not, not if the edge weight is estimated accurately (Epskamp, Borsboom, et al.,
2017). This is however important as the edge weight directly controls the network
visualization. A related topic is on the topic of how prominent false edges are, which
should be estimated to be zero but are included as non-zero edges in the network.
We used five measures to quantify these topics:
• (11) The Pearson correlation between the full vectorized edge weight
matrices, as previously described by Epskamp & Fried (2018).
• (12) The Pearson correlation between the absolute edge weights, as the sign
of edges may greatly inflate the correlation otherwise.
• (13) The average absolute deviation (bias) between the true edge weight and
the estimated edge weight of all edges (included and not included).
Page | 259
• (14) The average absolute deviation between the true edge weight and
the estimated edge weight for true edges that were included in the
estimated network.
• (15) The maximum fading of false edges. To compute this, we first compute
the opacity of the strongest false edge in a network: the absolute weight of that
edge divided by the absolute weight of the strongest edge in the network.
Next, we computed the maximum fading of false edges as one minus the
opacity of the strongest false edge. This measure is directly comparable to
the fading used of edges in the qgraph package (Epskamp et al., 2012), which
is often used to visualize psychological network structures, and quantifies
how prominently displayed a false edge can be in the estimated network.
For example, a maximum fading of 0.9 indicates that the strongest falsely
included edge was faded to white for 90%.
Centrality index accuracy: ability to identify important nodes. To investigate

the accuracy of centrality indices, we computed the (16, 21, 26) Pearson correlation
and the (17, 22, 27) Kendall correlation between the obtained centrality indices in
line with prior literature (Borsboom et al., 2017). In addition, we investigated the
ability of detecting nodes with the strongest centrality indices, by investigating how
many of the same nodes were correctly placed in the (18, 23, 28) top 1, (19, 24, 29)
top 3 and (20, 25, 30) top 5 of most central nodes in the network. We investigated
these five measures for node strength, closeness and betweenness.3
Bridge edges detection: ability to detect edges that connect clusters. To assess
the ability of methods to detect edges connecting clusters (which are often relatively
weak), we investigated a subset of the measures above exclusively for (potential)
edges connecting clusters in the DASS21 and BFI networks (the MAGNA
networks did not feature clusters). For these edges we computed the (31) sensitivity,
(32) signed sensitivity, (33) top 50% sensitivity, (34) top 25% sensitivity, (35) top
10% sensitivity, (36) specificity, (37) precision, (38) top 50% precision, (39) top
25% precision, and the (40) top 10% precision.
Network replicability: ability to replicate features in an independent dataset.

To assess replicability, we simulated a second dataset for each condition, and
3 We did not investigate expected influence (Robinaugh et al., 2016), as most edges in the network
were positive and we did not expect any difference with node strength)
Page | 260
estimated a second GGM model using the same procedure as the first GGM model.
The second network is thus intended as a replication of the first. We compared these
two networks on several metrics:
• (41) The correlation between the edge weights from the first network and
edge weights from the second network.
• (42) The proportion of edges from the first network that were also included
(replicated) in the second network. This was also estimated for bridge edges.
• The proportion of (43) top 50% / (44) 25% / (45) 10% edges (based on the
strength of their absolute weight) that were also included in the second
network (regardless of strength). This was also estimated for bridge edges.
• (46) The proportion of absent edges from the first network that were
also classified as absent in the second network. This was also estimated
for bridge edges.
• For the DASS21 and BFI networks, we also investigated the proportions
of (47) replicated bridge edges, (48) replicated top 50% bridge edges, (49)
replicated top 25% bridge edges, (50) replicated top 10% bridge edges, and
(51) replicated absent edges between clusters.
• The (52, 55, 58) Pearson and (53, 56, 59) Kendall correlations between node
strength, closeness and betweenness centrality indices from both networks.
• (54, 57, 60) Whether or not the most central edge according to node
strength, closeness and betweenness was the same.
13.2.6 Simulation setup summary

In sum, we simulated six different levels of sample size (150, 400, 600, 1000, 2500,
and 5000), four different kinds of data (normal continuous, skewed continuous,
uniform ordered categorical, and skewed ordered categorical), three different types
of true networks (MAGNA, DASS21, and BFI), four different transformations
(no transformation, rank/ Spearman, nonparanormal and polychoric/ categorical),
and 13 different estimators. The polychoric/ categorical transformation was only
used for nine out of 13 estimation methods, and only for data that was ordered
categorical. Furthermore, the stepwise estimators implemented in psychonetrics
(WLS stepup estimation and ML model search) were too computationally
challenging to assess for the two larger network models (DASS21 & BFI), and
therefore only included for the MAGNA network simulations. Every condition
was repeated 100 times, leading to a total of 282,000 simulation conditions. In
each condition, two datasets were generated: one for the main estimated network
Page | 261
for most metrics, and a second to assess replicability. As such, a total of 564,000
datasets were generated.
13.3 Results
Out of 282,000 simulation conditions, 1070 resulted in an error (e.g., especially
in low sample sizes some estimators ran into convergence issues), leading to a total
of 280,930 conditions and thus 561,860 simulated datasets to be included in the
results. As these were assessed on 45 (MAGNA) to 60 (DASS21 & BFI) metrics,
there were far too many results to report that could be fitted in the bounds of this
chapter. We therefore chose to summarize our findings through a platform that
allows for manually exploring simulation results, discussed in more detail below.
We use this app in deriving results for more concrete research questions below.
Primarily and as a general summary, an important finding of the current
simulation study is that an exchange between discovery (e.g., sensitivity, edge weight
correlation) and caution (e.g., specificity, precision) should always be expected and
achieving both–which is a requirement for perfect replicability (i.e., both very good
edge inclusion and no false positive edges)–is difficult. It is therefore important
to consider in light of each research question whether there is a favor for one or
the other: some methods perform better at retrieving a lot of true edges with an
appropriate edge weight, but may include also false edges. Other methods may be
very conservative, but fail to properly retrieve the global picture.
Further, findings in terms of transformations and data types: when data
are Gaussian, applying a nonparanormal on rank-transformation (Spearman
correlations as input) did not seem to impact performance of the estimators.
When data were skewed, a nonparanormal or rank-transformation improved
the performance in the majority of estimators across most datasets. For ordered
categorical data, there was hardly a difference between using a transformation or
not. Surprisingly, dedicated methods for handling ordered categorical (polychoric/
categorical ‘transformation’) data did not perform much better than other methods,
and actually performed worse in many cases. Notably, in ordered categorical data,
the mgm estimators that model responses as categorical had very poor performance
on replicated zeroes, likely related to the poorer specificity of these estimators
described below. For other estimators, there were no strong differences between
transformations and methods of modeling ordered categorical data. As rank-
transformations (Spearman correlations as input) worked well on all data types
Page | 262
and comparably to the more complicated nonparanormal transformation, we will
discuss results specific to rank-transformations below in more detail.
13.3.1 Simulation exploration app

We make results from our simulation study available through the simulation
explorer app (SEA), which is available on Github4. The app can be run locally by
using the following R commands:
library("shiny")
runGitHub("AdelaIsvoranu/SEA", ref = "thesis")
This code will install several required R packages when run for the first time.
Alternatively, the app is also hosted online and accessible via psychonetrics.org/
simulations. Figure 13.3 shows a screenshot of SEA. Included in the app are (1)
an overview of network models used in the simulation study, (2) an overview of
estimators used in the simulation study, (3) a recommender system, that can be
used to recommend estimators based on certain required conditions (e.g., find an
estimator that has an average specificity above .9 and an average sensitivity of 0.5 or
higher), (4) radar plots allowing to compare methods on several metrics for a given
setting (such as sample size), and (5) line plots for investigating performance as a
function of sample size. All results discussed below can be visualized using SEA. To
this end, we do not include any figures or tables in this chapter, as doing so would
mean selectively highlighting some results in favor of others.
13.3.2 Overall results

This section will highlight findings based on the overall results across the three
networks. We will start by investigating desirable asymptotic (i.e., high sample
size) properties, for (skewed) continuous and ordinal data, followed by desirable
low sample size properties.
Desirable asymptotic properties. First, we investigate asymptotic properties of

a network structure. We have defined asymptotic properties as low false inclusion
rate of edges, high rate of identifying (the strongest) edges in the network structure,
high correlations between the estimated network structure and the true network
Page | 263
structure, in the context of a very large sample size. That is, we were interested in
achieving best performance at a large sample size. We specifically operationalized
this question by looking at the following properties in SEA radar plots: N=5000,
specificity and precision, to investigate if the edges included in the network structures
were true edges, sensitivity, as well as sensitivity (top 50%), to investigate if the
(strongest) edges were detected, absolute correlation, and 1-bias to detect if similar
differences between strong and weak edges in the true network were estimated.
Overall, as expected, most estimators reached desirable asymptotic properties
with a high sample size. Exceptions were the EBICglasso algorithm, followed by
the mgm (CV; 10 folds) algorithm and, especially in the DASS21 network, the
ggmModSelect algorithm without stepwise estimation, both performing poorer in
terms of specificity and precision.
We conclude that at a high sample size (N=5000), most estimators work well,
but unregularized estimators (especially ggmModselect stepwise, FIML estimators,
and BGGM estimate) work best in retrieving a network structure with a low false
inclusion rate of edges, high rate of identifying (the strongest edges) in the network
structure, and high correlations between the estimated network structure and
the true network structure. In addition, as detailed above, in the case of skewed
normal data, rank-transformations (Spearman correlations as input) improve the
performance of estimators, while in the case of (skewed) ordered categorical data
transformations do not make a considerable difference.
Low sample size discovery. Further, we investigate low sample size discovery.
That is, we focus on desirable results obtained at low sample size, which is often
a common encounter in applied research (e.g., in mental health research where
patients are a central research concern). We were thus interested in whether we can
discover the most important edges and the overall network structure at low sample
size. In this setting we allow for a method to include false edges, as long as these are
hardly visible in default faded edge weight visualizations. In addition, we focused
on precision, rather than specificity, as we were interested in whether the edges
included in the network structures were true edges. We specifically operationalized
this question by looking at the following properties in SEA radar plots: N=300,
sensitivity (top 25%), to investigate if the strongest edges were included the network
structures, precision, to investigate if the edges included in the network structures
were true edges, and fade maximum false edge, to investigate whether the false
positives edges in the network structures were hardly visible.
Page | 264
Page | 265
Figure 13.3 Example of output provided by the simulation exploration app (SEA) available at psychonetrics.org/simulations.
While many estimators performed well in one or two of the three predefined
properties, the three regularized estimators, EBICglasso and the two mgm estimators
(EBIC and CV selection), worked best in terms of the fade maximum false edge
(faded edges were at most around 25% visible). Of these, EBICglasso and mgm (CV)
worked best in terms of sensitivity (top 25%), while the EBIC selection variant of
mgm performed poorer in sensitivity (top 25%) but better in precision. Overall,
these estimators performed well in discovering the strongest edges in the network
structures, while also ensuring that false edges were not prominently visible. Of
note, however, there was a clear inverse relationship between the sensitivity (top
25%) and precision, indicating that edges discovered by these methods are also
more likely to be false edges than edges discovered by other methods. Choosing
an unregularized estimator, on the other hand, will lead to a higher precision,
but at the cost of both losing power to detect strong edges, as well as the risk of
more prominently featured strong edges. Using a transformation improved the
performance of most estimators for continuous skewed data, but had little impact
in the case of skewed ordered categorical data, with the exception of the polychoric
correlations, which considerably improved the sensitivity (top 25%) in the case of
the EBICglasso algorithm in the MAGNA network.
Therefore, at these sample sizes we recommend even more so than in other cases
to let the choice of estimator be guided by the research question(s). If the goal of the
study is to investigate the presence of each individual discovered edge, and if these will
be displayed without fading, then regularized estimators should be avoided. However,
if the goal is to discover a structure that resembles a true network and to discover the
strongest edges, regularized estimators are preferred in combination with a fading
rule used to draw the edges, as it is default for example in the qgraph package.
13.3.3 Specific research questions

This section will highlight findings based on common research questions of applied
researchers. We will start by investigating visual network alignment, followed by
identification of central nodes (i.e., in terms of strength), identification of bridge
edges, and finally replicability.
Visual network alignment. Next, we investigate how well the visual network
picture aligns with the true network structure at a medium sample size (N=1000).
That is, we were interested in if the estimated network picture resembles the true
network picture (e.g., are the same edges strong and visually present in the estimated
Page | 266
and the true networks). In this setting we allow for a method to include false edges,
as long as they are hardly visible in default faded edge weight visualizations. We
specifically operationalized this question by looking at the following properties
in SEA radar plots: sensitivity (top 50%) to investigate if the strongest edges were
detected, correlation (absolute) and 1-bias (true edges) to detect in similar differences
between strong and weak edges in the true network were estimated, and fade
maximum false edge to detect how pronounced the strongest false edge was in the
estimated network.
Across all three empirical network structures used in simulations, the
EBICglasso algorithm performed particularly well on these benchmarks: the
algorithm consistently scored very high on all four measures, indicating that
the most important edges were routinely included and false edges were not
prominent in the network. The mgm methods also performed well in this setting,
with the EBIC variant performing better on fade maximum false edge criterium
(not showing false edges), while the cross-validation variant performed better
on sensitivity (top 50%). Unregularized estimators, such as the WLS estimators,
FIML, GGMnonreg and BGGM variants performed a bit worse, although not
very badly (only the DASS21 network showed top 50% sensitivity levels under
0.75 on average). These estimators were more conservative, leading to a lower
sensitivity top 50%, and tended to include false edges a bit more prominent in
the network compared to regularized estimators.
We conclude that at a relatively high, though attainable, sample size of N=1000,
regularized network estimators work best when the interest solely lies on the visual
representation of the network using standard fading of edges. Using regularized
estimators, strong edges are routinely included with an appropriate strength, and
false edges are not very prominent in the network.
Centrality. Further, we investigated how well the three most common centrality
indices (i.e., strength, closeness, and betweenness) of the estimated network
structures align with the true network structure, at a low, but common sample
size in social science (N=600). Specifically, we were interested in how well the
estimated centrality measures correlate with the true centrality measures, as well
as how well the centrality based on one dataset correlates with the centrality based
on another dataset. We investigated in particular the metrics in SEA radar plots:
node strength correlation, closeness correlation, and betweenness correlations, followed
Page | 267
by the metrics node strength correlation replication, closeness correlation replication,

and betweenness correlation replication.
Overall, the pattern of results indicates that strength as a centrality measure is
the most likely measure to show both a good correlation coefficient between the
true and simulated network structure, as well as good correlation replication. This
holds both for unregularized methods, specifically for ggmModselect stepwise, as
well as for regularized methods, specifically for the EBICglasso and mgm (EBIC;
gamma=.25). In the case of ordered categorical data in the BFI network, the
performance of all estimators dropped for the node strength correlation, suggesting
that for dense large network structures, ordered categorical data may pose more
problematic centrality results. Measures for closeness performed worse than measures
for node strength, and measures for betweenness performed even worse than those
for closeness, displaying very low correlation and correlation replications across all
estimators and all networks. The estimators that worked best were ggmModselect
stepwise and EBICglasso. In the case of skewed ordered categorical data, estimator
performance varied based on the chosen network.
We conclude that, if centrality is of interest with a relatively low sample size,
using the EBICglasso estimator or unregularized ggmModSelect stepwise will
likely give most confidence interpreting centrality indices. Strength as a centrality
measure showed highest correlation values between the true and estimated network
structures, as well as highest replication correlation, followed by closeness. In
the case of skewed ordered categorical data, the correlations however dropped,
indicating more caution is indeed when interpreting centrality measures in the
case of ordered categorical data. In addition, the performance of estimators varied
more across networks for the case of centrality, thus we recommend checking for the
network structure of interest. For the case of betweenness, our results showed that
global properties of betweenness are likely hard to estimate, and as such we would
not recommend anymore the use and interpretation of betweenness in network
studies. Nonetheless, if betweenness is central to the research question of the applied
researchers, taking an approach where investigating local properties of betweenness
(e.g., by bootstrapping betweenness to see which particular betweenness results are
more likely to come up often) should be common practice. Finally, and important,
while here we interpret results for centrality in the case of MAGNA, the PTSD
meta-analysis on which the MAGNA network is based showed little differences in
centrality between nodes, especially for strength and closeness. As such, focusing
on the latter two networks for the case of centrality may be warranted.
Page | 268
Bridge edges. Further, we focus on edges bridging different domains in a network
structure. That is, we focus on desirable results obtained at frequent, but slightly higher
sample sizes (N=1000). This is a common question in applied research, especially in
cases in which environmental (e.g., trauma) or genetic risk scores aim to be integrated
into network models. We were thus interested in whether we can steadily discover
bridging edges between different domains. In this setting we specifically focused
on the (top 25%) sensitivity, precision, and specificity of bridge edges. We specifically
operationalized this question by looking at the following properties in SEA radar
plots: N=1000, sensitivity bridge, sensitivity bridge (top 25%), precision bridge, and
specificity bridge, to get information on correct detection of (strong) edges, as well as
on whether the edges identified as bridging edges were true edges.
Of note, in the current case we only focused on two network structures, the
DASS21 and the BFI, as these were the only ones including different domains (i.e.,
in the MAGNA network does not include bridge edges). Here the performance
of estimators varied more across the two network structures, indicating that the
topology of the network may play an important role in how the estimators perform,
with slightly better performance for many of these in the BFI network. The main
results indicate a strong exchange between sensitivity and specificity/ precision
in the case of bridging edges. Most estimators that perform well on one side, will
perform less well on the other. The EBICglasso for instance shows good sensitivity
of bridge edges, as well as extremely good sensitivity of strongest bridge edges, but
lower precision and specificity, suggesting the presence of false positive edges in the
identified bridges. Mgm on the other hand, especially using EBIC (gamma=.25), as
well as ggmModSelect (stepwise=TRUE, gamma=0) show very good precision and
specificity, but lower sensitivity, indicating that some bridging edges may be missed.
In the case of the BFI network, the ggmModSelect (stepwise=TRUE, gamma=0)
algorithm perform very well on all counts, with very good precision and specificity
of bridge edges, as well as good sensitivity (~.75) and extremely good sensitivity
of strongest edges. While in the case of ordered categorical data the pattern of
results is similar, the performance of the estimators is poorer than in the case of
continuous data.
We conclude that, if bridging edges between different domains are of interest,
first and foremost the network structure and topology may play an important role
in the choice of estimators and the expectations on performance. In the case of
the DASS21 network, this is a large and very dense network structure, with a high
number of bridging edges between the different domains, which is an important
Page | 269
consideration. In the case of the BFI, while still dense, given that all measures are
related to personality traits, the bridging edges are stronger. The estimators perform
better in the latter case, while in the case of the DASS21 network the exchange
between specificity and sensitivity is more prominent. Overall, we identified that
the mgm (EBIC; gamma=.25) algorithm and the ggmModSelect (stepwise=TRUE,
gamma=0) algorithm performed consistently well on all conditions, while the
ggmModSelect (stepwise=TRUE, gamma=0) algorithm performed extremely
good in the case of the DASS21 network with continuous data. The ggmModSelect
(stepwise=TRUE, gamma=0) is focused on obtaining a local optimum, meaning
that no individual edges can be added or removed to improved fit. In this case,
every bridge edge is also individually evaluated, this being an extra property of the
estimator. As such, we recommend the use of the ggmModSelect (stepwise=TRUE,
gamma=0), or mgm (EBIC; gamma=.25) for denser network structures when the
expectation is that many and less strong bridging edges exist. Of note, if the interest
is only on the strongest bridging edges, or on either side of specificity or sensitivity,
most other estimators we evaluated performed very well.
Network replicability. Finally, we investigate how well network models estimated

from one dataset replicate in a second dataset drawn from exactly the same
distribution. To answer this, we simulated two datasets, and estimated a set of
replicability measures that summarize the overlap between networks estimated
from both datasets. We focus our discussion here on the sample sizes N=600
(relatively low, similar to the sample size studied by Forbes et al., 2019), and N=5000
(high, similar to the sample size studied by Forbes et al., 2017). We specifically
operationalized this question by looking at the following metrics in SEA radar
plots: replicated edges, summarizing the proportion of edges that replicated in the
second network, replicated zeroes, summarizing the proportion of absent edges that
were also absent in the second network, correlation replication, summarizing the
correlation between edge weights of both networks, and replicated edges (top 25%),
summarizing if the strongest 25% edges from the first network were also included
in the second network (regardless of strength).
Across all data types and transformations, most estimators performed
very similarly to one-another, with the exception of EBICglasso and mgm (CV;
10 folds). These algorithms tended to perform worse than other methods on
replicated zeroes and, to a lesser extent, replicated edges (especially in the BFI high
sample size condition). This performance is likely related to the poorer specificity
Page | 270
of these algorithms compared to other algorithms discussed earlier: due to a larger
number of falsely included edges less zeroes replicate, and the falsely included edges
themselves also do not replicate in other networks. Further, across all types and
transformations, the replicated edges (top 25%) metric was very high across the
board, indicating that the strongest edges estimated in one network were very likely
to also be included in a network estimated from a second dataset.
Across the conditions, there were some striking differences between the two
ggmModSelect methods. At high sample size, the variant with stepwise estimation
performed well (in combination with a transformation for skewed normal data)
on all measures (although the least in the DASS21 network, which featured the
most edges), while the variant without stepwise estimation performed worse on
replicated zeroes in the DASS21 and MAGNA networks and both replicated zeroes
and replicated edges in the BFI network. This performance may be related to the
difference in specificity discussed earlier. At the lower sample size of N=600, the
variant without stepwise estimation performed slightly better than the stepwise
variant on the replicated edges metric.
Using a nonparanormal or rank transformation, the mgm (EBIC; gamma=.25)
algorithm performed consistently well on all conditions at the lower sample size
of N=600, although other metrics (in particular the ggmModSelect and BGGM
variants) also performed well. Of note, performance on some of the measures was
not very high, and often ranged around .5-.75. It is arguable if, say, a rate of 60%
replicated edges is good or not. At larger sample sizes, unregularized methods all
performed quite well on all metrics.
We conclude that researchers should take expected replicability in the light
of sensitivity and specificity (Williams, 2020), and not expect near-perfect
replication of the network structure at low to medium sample sizes. Nonetheless,
with most methods most edges should be replicated in a second dataset, and the
overall structure should be similar. If the goal is to replicate individual edges,
the EBICglasso and mgm with cross-validation algorithms should be avoided.
At low sample sizes, the mgm with EBIC model selection performs well, as does
ggmModSelect. At high sample sizes, one of the unregularized methods with
desirable asymptotic properties should be used to optimize replicability.
Page | 271
13.4 Discussion
The current chapter is, to our knowledge, the only large-scale simulation study
designed to compare the performance of a wide array of estimation algorithms
suitable for Gaussian and skewed ordered categorical data across a multitude of
settings, as to arrive at concrete guidelines from applied researchers. All the results
from our simulation study are made available through the SEA app, accessible on
Github5. Overall, we advise that the estimation method is best chosen in light
of the research question(s) of the applied researcher, as some algorithms perform
better in retrieving a lot of true edges, but may include false positive edges, while
others may be more conservative and have a very low rate of false positive results,
but concurrently fail in retrieving all edges and the global picture. This trade-off
finding aligns well with previous findings from standard psychometrics analyses
(e.g., Kraemer, 1992). The current chapter highlighted results according to most
common research questions in the field, alongside desirable asymptotic properties
and low sample size discovery.
In particular, at high sample size (N=5000), we identified most estimators
to work well, but unregularized estimators to work best in retrieving a network
structure with a low false inclusion rate of edges, high rate of identifying (the
strongest) edges in the network structure, as well as high correlations between the
estimated network structure and the true network structure. At low sample size
(N=300), if the goal is to discover a network structure that resembles a true network
and to discover the strongest edges, regularized estimators should be preferred;
if the goal however is to focus on each individual discovered edge, regularized
estimators should be avoided.
In terms of specific research questions, we found that at an attainable sample
size of N=1000, if interested in the visual network alignment (i.e., investigating
overall structure and strong edges), regularized network estimators work best. If
interested in bridge edges between different domains, the topology of the network
was important, with estimators performing better when strong bridging edges were
present. When the expectation was that many and less strong bridging edges exist,
which is often the case of denser network structures, we found the ggmModSelect
(stepwise=TRUE, gamma=0) and mgm (EBIC; gamma=.25) estimators to
perform best. Of note, if the interest lies only on the strongest bridging edges,
Page | 272
or on either side of specificity or sensitivity, most other estimators we evaluated
performed very well. When investigating centrality, at a low but common sample
size in social science (N=600), using the EBICglasso estimator or the unregularized
ggmModSelect stepwise estimators gave most confidence interpreting centrality
indices. Strength as a centrality measure showed best results, followed by closeness.
Of note, in the case of skewed ordered categorical data, more caution is needed
when interpreting centrality measures. For the case of betweenness, our results
showed that global properties of betweenness are likely hard to estimate, and
as such we would not recommend the use and interpretation of betweenness in
network studies. Alternatively, investigating local properties of betweenness should
be common practice.
Finally, when data were Gaussian, applying a nonparanormal on rank-
transformation (Spearman correlations as input) did not impact performance of
the estimators. When data were skewed, a nonparanormal or rank-transformation
improved the performance in the majority of estimators across most datasets. Since
the latter worked well on all data types and comparably to the more complicated
nonparanormal transformation, we recommend Spearman correlations as
input in the case of skewed data. Surprisingly, for ordered categorical data, data
transformation did not make a substantial difference.
13.4.1 Limitations
The current study should be considered in light of several limitations. First, the
results of the simulations discussed in this chapter rely critically on the network
models used to generate data. We chose these three models to be representative
of network models we may expect to find in psychological research (all three are
based on a large sample of variables that would suit analysis using a psychological
network model). Nonetheless, other generating network structures with different
network characteristics may lead to different recommendations. To this end, we
make an example code of the simulation script used in this study available in the
online supplementary materials on the OSF. Second, we generated data using a
GGM with Gaussian variables, and subsequently transformed data to be ordered
categorical (using thresholds) or skewed (using an exponential function). Data can
also be ordered categorical or skewed in different ways. For example, the data could
follow different marginal distributions than the log-normal distributions used in
skewed continuous cases, and ordered categorical data could also follow a network
structure without relying on an underlying Gaussian distribution. As such, our
Page | 273
results do not generalize to all ways in which data can be skewed and/ or ordered
categorical. Finally, network psychometrics is a rapidly developing field, and new
estimation methods are proposed routinely. Future studies could include more novel
estimation methods (e.g., Lafit et al., 2019; Williams, 2021), which could perform
differently than the estimators currently assessed.

To conclude, regardless of the setting chosen, we found that an exchange between
discovery (e.g., sensitivity, edge weight correlation) and caution (e.g., specificity,
precision) should always be expected and achieving both–which is a requirement
for perfect replicability–is difficult, even more so when bridging edges are of
interest. Researchers should take expected replicability in the light of sensitivity
and specificity and not expect near-perfect replication of the network structure at
low to medium sample sizes. Nonetheless, with most methods most edges should be
replicated in a second dataset, and the overall structure should be similar. Finally,
the estimation method should be chosen in light of the research question(s) of the
applied researcher, as highlighted above. Further simulation conditions, which
can be used for investigating performance of estimators for additional research
questions, are available through the SEA.
Page | 274
Page | 275
14
Discussion
Chapter 14 | Discussion
14.1 Summary
Within the past decade, the fields of psychological science and psychiatry alike have
seen a relatively sudden shift from ingrained reductionistic approaches to mental
disorders, to alternative frameworks and complexity-based approaches. The latter
call for a reconceptualization of psychopathology, focusing on symptoms and
interactions between them rather than on syndromes, and align especially well
with how clinicians approach mental problems and their treatment. The current
thesis project centered, to a large extent, on building upon this framework of
thought, and aimed to take a first step toward the conceptualization of psychosis
as a complex system of interacting components. The first three parts of this book
therefore focused on (1) concrete symptoms and their interrelations, (2) links
between common symptomatology and well-established risk factors, and (3)
comorbidity between psychosis and other disorders with shared etiology. The
last part of the book (4) moved beyond the field of psychosis, by shifting focus to
methodological notes and developments in the field of network psychometrics.
These developments were advanced in an attempt to provide applied researchers
and practitioners with the necessary tools that would lead to better established and
replicable results, as well as overcome common challenges uncovered throughout
this PhD project. To summarize overall results, the discussion section below will
cover clinical implications, methodological considerations, and finally potential
future research directions.
14.2 Clinical implications

To date, network structures are commonly constructed from empirical data, as is
the case with most network structures in the current thesis. These are estimated
in an explorative fashion, by searching for best-fitting models to explain the links
present in a dataset (Borsboom & Cramer, 2013; Costantini et al., 2015; Epskamp
et al., 2012). To this end, it is essential to note that the current research, while
taking an important first step in challenging categorical approaches to psychosis,
is exploratory and hypothesis-generating. Within every applied chapter of the
book, we discussed potential implications for treatment and modest guidelines
for clinical implementation, with the acknowledgment that much work remains
to be done before network models can be directly implemented in clinical practice,
Page | 278
work including designs allowing hypothesis testing and experimental studies using
network structures.
Overall, several recurring findings throughout this book can he highlighted.
First, most of the symptoms investigated here were more strongly linked to other
symptoms within the same domain, suggesting that node activation may first
‘spread’ within the same domain before reaching other domains. This finding
is not surprising, as symptoms belonging to the same domain are often close in
content (e.g., delusions-unusual thought content) and in some instances may even
be overlapping in content (e.g., social withdrawal-social avoidance). Future studies
designed specifically for network modeling may address this problem by avoiding
the inclusion of items overlapping in content, allowing to better pinpoint within-
domain and between-domain symptom activation. The domain for which this
finding was less prominent is the domain of general psychopathology, which was
often found to be less clustered, with items strongly linked to other items from
varying domains, including positive and negative psychotic symptoms. In this
context, it can be that the presence of other pathology may facilitate the activation
of psychotic symptoms, leading to a quicker entry into a disorder state.
Second, building up on the findings above, a recurring aspect in this book was
the central role of affective symptoms in psychosis. Specifically, we identified that
anxiety was the main connecting component between childhood trauma and
paranoia, delusions, and hallucinations, as well as between obsessive-compulsive
symptoms (OCS) and other psychotic symptomatology. Further, depression was
often a highly central symptom both in at-risk populations, as well as in patient
populations, with findings indicating that depression may further be exacerbated
by use of cannabis–an often-abused substance in psychosis. Affective symptoms
were in general highly interrelated, while affective speech illusions (but not
all speech illusions) were found to be directly associated with hallucinatory
experiences. Previous research identified that heightened anxiety predicts the
onset of paranoid episodes (Ben-Zeev et al., 2011; Thewissen et al., 2011), while
the relationship between sexual abuse and psychosis was previously found to be
mediated by anxiety and depression (Bebbington et al., 2011), aligning well with the
findings of this book. Both depression and anxiety were also found to be associated
with higher symptom severity of ultra-high-risk symptoms of psychosis and
poorer global functioning (Fusar-Poli et al., 2014; Rutigliano et al., 2016). Taken
together and in line with these previous results, the findings in this thesis provide
strong support for the existence of an affective pathway to psychosis, potentially
Page | 279
through heightened emotional distress (e.g., anxiety, tension, depression). Fusar-

Poli and colleagues (2014) highlighted the problematic aspects of assuming a neat
distinction between the constructs of affect and psychosis, due to their evident
intertwinement, especially in initial symptom development. Early models of
psychosis development (e.g., Conrad’s classic 1958 stage model) describe these early
stages as being stages in which patients experience a wide range of affect, including
fear, guilt and depression (Mishara, 2010). The role of affective symptoms both in
at-risk and patient populations has been of increased research interest, though
affect is still not central in treatment approaches. Based on current findings, it
may be advisable to not only focus treatment on psychotic experiences, but extend
focus to emotional disturbances. Further, according to the patient’s history and risk
factors, interventions based on every individual’s vulnerabilities may be essential
(e.g., reducing experienced anxiety and distress could be important targets when
trauma or obsessive-compulsive thoughts are present, or e.g., focus on negative and
general psychopathology symptoms may be beneficial in subjects abusing cannabis).
Third, approaching psychopathology from a comorbidity perspective, if
variables from the external field (as defined in Chapter 2) are thought to have
an influence on individual symptoms, which we know are often shared between
disorders, the finding that vulnerability factors are similar and common between
different diagnostic categories is not surprising. Childhood trauma, for instance,
is often identified as a risk factor for anxiety disorders, major depressive disorder,
bipolar disorder, and personality disorders (Herman et al., 1989; Leverich et
al., 2002; Young et al., 1997), and is not specific to psychotic disorders. The
findings in the current thesis indicate that no trauma variable was directly linked
to psychotic experiences, but only to a wide array of symptoms present across
differing mental conditions (Hovens et al., 2012; Nanni et al., 2012). Similarly,
while cannabis use was linked to some positive symptoms, it was mainly linked to
negative and other general psychopathology symptoms. Finally, unlike other risk
factors, most links we identified between a polygenic risk score for psychosis and
symptomatology were links to positive psychotic symptoms, especially symptoms
related to notions of conspiracy and paranoia. In light of this, and in line with
previous findings which argued that symptoms can have different risk factors
(Fried, Bockting, et al., 2015; Isvoranu et al., 2016, 2017), the results of this thesis
indicate risk factors may lead to different pathways to the onset (and progress)
of psychotic disorders: genetic risk factors may have more of a main effect on
the liability to develop positive psychotic symptomatology, while environmental
Page | 280
risk factors may have more of a secondary effect on general psychopathology
symptoms. The latter may thus partly explain why environmental factors are
often shared between distinct mental disorder categories.
Overall, the current thesis aimed to provide a new means and framework
of thought toward approaching psychopathology, and especially risk factors in
psychopathological conditions. Focusing on the role of affect may be crucial for
the development and progression of symptoms and tailoring treatments according
to the vulnerabilities of every individual may be essential.
14.3 Methodological considerations

Several methodological notes and challenges of the network methodology are in
order. First and foremost, throughout this PhD project, a common challenge has
been identifying the most suitable methodological approach to study the links
between the external field (as defined in Chapter 2) and symptomatology. That
is because, naturally, symptoms tend to cluster together and have stronger links
with other symptoms, while variables pertaining to the external field (such as
genetic and environmental risk factors) cluster less with symptoms. Within most
of the network models in this thesis, therefore, we identified weaker between-
domain links - i.e., weaker bridge links. Recent research (e.g., Williams & Rast,
2020), as well as the large simulation study presented in Chapter 13 here showed
that estimation techniques relying on the lasso regularization (Tibshirani, 1994)
may be more prone to specificity issues, especially in dense network with many
small edges, thus leading to an increase in false positive edges. In addition, as
these links are often weak, sensitivity problems may be an additional issue, as we
may not always successfully identify all the bridging edges, especially when more
conservative estimation techniques are employed. This is especially concerning
in a field of research where the focus is turned toward identifying robust bridge
edges. To account for these issues, the studies in this thesis were accompanied by
multiple robustness checks, as well as by novel methodology and unregularized
estimation techniques. It is nonetheless noteworthy that there is a need for
further methodological developments, concerned primarily with identifying the
best and most robust ways to estimate such bridging components. As highlighted
in Chapter 13, with most existing methods, an exchange between discovery
(e.g., sensitivity, edge weight correlation) and caution (e.g., specificity, precision)
should always be expected and achieving both–which is a requirement for perfect
Page | 281
replicability–is difficult, even more so when bridging edges are of interest. Sample
size, number of variables, expected network topology all play an important role in
the performance of different estimators. The estimation method should thus be
chosen in light of the research question(s) of the applied researcher, while taking
all these aspects into consideration. Future methodological developments may
help overcome these limitations.
Second, at the core of the network methodology lies the idea that symptoms can
causally trigger and influence each other over time. While capturing conditional
independence relations, which can be indicative of the causal structure (Pearl,
2000), cross-sectional networks cannot capture clear causal links and are not
designed to uncover dynamics across time. Experience sampling methods and time-
series analysis are often the chosen alternative to uncover such dynamics. However,
it is unclear how variables belonging to the external field could be incorporated
into such models. Childhood trauma, for instance, or a polygenic risk do not vary
within an individual and thus cannot be measured over time. Asking a subject
multiple times per day/ week whether they were abused in childhood would not
yield any novel information, while at the same time would have a negative impact
on the well-being of the subject. Similarly, other common risk factors for psychosis
and other mental disorders (e.g., growing up in an urbanized area, ethnicity, season
of birth, prenatal stress and obstetric complications; van Os et al., 2010; van Os
& Kapur, 2009; Leask, 2004) are alike not varying over time. To this extent,
time-series analysis or panel data models would not be a suitable alternative for
uncovering causal links; experimental designs in longitudinal studies, while much
more difficult to implement, may be best alternatives in future research.
Third, an important note is that predictive nodes from the causal background
of the symptoms are unlikely to operate on the same timescale as symptoms in
a symptom network. However, under the assumption that these have a direct
relationship with current symptomatology regardless of the timescale, including
these in a network structure should not pose as an issue. The nature of statistical
control in a partial correlation network which includes variables outside of the
symptoms as nodes is likely not that different from that of a network that exclusively
contains symptoms. Taking for instance the example in Chapter 8, if a polygenic
risk score (PRS) was a common cause for two symptoms within a network structure
(e.g., increased PRS leads to increased liability to develop symptoms sadness and
conspiracy) but not included in the network (and thus not controlled for), a spurious
edge between the two symptoms would be induced. If PRS was indeed linked
Page | 282
to symptoms, such spurious associations would then be eliminated due to the
inclusion of PRS as a node in the network. Building on this note, an important
challenge in network models lies in missing nodes. By failing to include nodes
that that may co-vary with the variables in the network structure, such spurious
associations may be induced. While to date there is no clear-cut solution to this
issue, researchers are encouraged to think about the given construct they want
to study and aim to include all relevant measures that may impact that specific
construct. It is however unclear, especially within an extensive research field such
as the field of psychopathology, where such a multivariate system would start and
end. In addition, given that the field of network psychometrics is relatively young
and many current network-based studies are based on already existing data, not all
measures of interest may be available. Further, over-expanding network models may
lead to other limitations in interpretation, for instance due to underpowered studies
using too few data points for the number of parameters in the model. Upcoming
studies with designs and data collection specifically tailored to network models
may aid in overcoming these challenges.
Fourth, given the novelty of the network psychometrics field, no clear
minimal shared norms in reporting psychological network analyses have yet been
established in the literature. This often makes reproduction of results or clearly
following the procedure of other research difficult. While the community around
the field of network psychometrics has been very active in data and code sharing
online–an essential step in transparency and reproducibility–Chapter 11 further
aimed to introduce such first norms, as well as point to common inference issues.
Future research may expand or challenge these norms, but we trust this to be
an important avenue in aiding researchers to identify elements of their analyses
that are important to include in a scientific report, as well as to make empirical
network studies more rigorous.
In sum, there is much work yet to be done before many of the challenges
pertaining to the network framework are to be solved. At the same time, however,
there have been many advances made in the past years, with novel methodological
approaches and efforts to bring progress to a still maturing field of research.
Page | 283
14.4 Future research directions

In the coming years, many more promising research avenues remain to be
pursued. For instance, pre-registration and hypothesis testing of network models
may become much more common and even a standard. Most research to date
has relied on exploratory estimation of network models; however, as researchers
become more and more acquainted with the operating principles of such models,
setting up hypotheses about the presence, absence, or strength of associations in a
network structure will likely be more common and intuitive. In addition, ongoing
work concentrating on integrating prior knowledge on network connections (e.g.,
by taking into account clinician or patient knowledge) is being developed, and
including such sources of information will likely set up new research directions with
great potential for clinical implementation. Based on the current research in this
thesis, many new hypotheses have been put forward and investigating the extent
to which these can be supported is an essential next step in the field.
One potential mean to investigate these hypotheses may be through the use of
theoretical modeling of constructs and processes, which has gained much ground in
psychological research in the past year (Burger, van der Veen, et al., 2020; Robinaugh,
Haslbeck, et al., 2020). Such formal theories, reliant on mathematical modeling
of processes rather than on data, have been argued to be much needed tools for the
development of scientific theories. Though not without limitations (e.g., extensive
models incorporating different levels of explanation and a multitude of factors are
unlikely to be yet successful), the coming years are expected to see an important
expansion of these models, with new implementations in psychosis research and beyond.
Based on the exploratory findings in the current book, one could envision the
set-up of such a comprehensive theoretical model, thus allowing findings from
applied research to serve as the foundation blocks and basic architecture of one
or more mathematical models. Figure 14.1 below provides an illustration of a
multitude of potential contributing factors–as highlighted throughout this thesis–
to the mental health of a hypothetical person, George. George’s mental health is
entangled in many different layers of explanation, ranging from microscopic levels
of within-person thoughts and feelings, to macroscopic levels of societal norms
and change, and from slow processes such as genetic information, to fast processes
such as experiencing sudden side effects of medication. Based on the information
gathered about George, the set-up of a theoretical model could include general
structural features (for instance stronger within-domain associations, as is the
Page | 284
MACROSCOPIC

George often experiences

the stigma and discrimination that George avoids social
surround mental illness and thus situations and prefers to spend
hesitates to seek help. most of his time alone, at home.
George lives in a large city,

with easy access to alcohol and Alongside schizophrenia,
other drugs. e! George was also diagnosed with depression
org and obsessive-compulsive disorder.
Ge
I’m
SLOW FAST
George often experiences side

effects from his medication.
George had a very difficult

childhood. His father abused both him
and his mother.
George often reports paranoid
thoughts, hallucinations, feelings of sadness,
anxiety, and withdrawal.
George has
a genetic predisposition to
psychosis. His mother was diagnosed George often smokes
with schizophrenia when she was cannabis, as he believes it alleviates
young. some of his symptoms.
MICROSCOPIC

COMMON

RISK FACTORS

Ch. 4 Ch. 4
Genetic risk Urban living Cannabis use Childhood trauma
Ch. 8 Ch. 6 Ch. 6 Ch. 5

SYMPTOMATOLOGY
Ch. 3, 5, 7
Ch. 3, 4, 5, 7
Delusions Paranoia Anxiety Depression
Ch. 3, 5, 7
Ch. 5, 9, 10
Ch. 6
Hallucinations Negative symptoms
Ch. 9, 10
Ch. 5, 10
OTHER FACTORS
Ch. 9, 10
Medication side effects Comorbidities Social functioning
Fast Negative
Slow Positive
Figure 14.1 A glimpse into George’s life. Illustration of a multitude of potential contributing factors,
as highlighted throughout this thesis, to the mental health of a hypothetical person, George. Based on
the exploratory findings in the current book, we propose the setup of a theoretical model, including
the highlighted properties and processes.
Page | 285
is the example of delusions, paranoia, and hallucinations in Figure 14.1), but also
more particular features (for instance genetic risk increasing the liability of George
to develop positive symptoms, or depression and anxiety serving as mediators
between some risk factors and symptomatology). Other features could comprise
the inclusion of slow and fast time-scales (e.g., cannabis use on the short-term could
be modeled to lead to a decrease in the side effects of medication, while on a long-term
it could be modeled to lead to an increase in psychotic symptoms and depression),
trauma and urbanicity as risk factors could be modeled to increase the use of addictive
substances, and comorbidities could be modeled to a decrease in social functioning.
Figure 14.1 expands on this suggested architecture, highlighting the most common
features identified throughout this book, as well as in which chapter(s) evidence for
a particular link was found. Of note, many more factors are likely to play a role in
both the onset and progression of psychosis1 and the suggested layers are meant as
a starting point and as a display of how current exploratory findings could be used
to set up such a model, as to arrive to some basic architecture, which could then be
built upon and expanded. Once the setup is in place, one could simulate how an
intervention focused on one or more of the highlighted properties would change
the system as a whole. For instance, would reducing the substance use of George
have a positive impact on the positive symptomatology of George? Would this at
the same time increase the medication side effects, resulting in a loop where other
general psychopathology symptoms of George are once again triggered, leading back
to an increase in positive symptomatology, but through a different pathway? Would
focusing on other properties, such as the affective symptoms be then more beneficial?
Ultimately, theoretical models could be designed to capture both relations that are
generalizable across populations (e.g., in the case of more homogeneous properties), as
well as be developed with an idiographic framework in mind. The latter could focus on
one individual (e.g., George) and the modeling of properties and processes important
for that particular individual. Having the means to pinpoint where specifically an
intervention may be most favorable would bring a substantial contribution to applied
research and facilitate the development of innovative interventions.
1 Here we feature the role of three positive symptoms (i.e., delusions, paranoia, and hallucinations),
two general psychopathology symptoms (i.e., anxiety, depression), and more broadly negative symp-
toms. Of note, this is a simplification in itself, as the interactions between these three layers of
symptomatology may well themselves form an extensive theoretical model with differing properties
and time-scales. Likewise, childhood trauma could further be split into different types of trauma, as
highlighted in Chapter 5, with multiple potential routes to symptomatology, not all included here.
Page | 286
Page | 287
Appendices
A
Supplement to Chapter 3
Appendix A1. Measures

We included several measures assessing psychopathology, alcohol use, gambling,
major chronic conditions, and functioning. Below we include a detailed description
of each measure used.
Appendix A1.1. WMH-CIDI Interview

The World Health Organization-Composite International Diagnostic Interview
(WMH-CIDI; Kessler & Üstün, 2004) was used to assess overall psychopathology.
Due to the skip-structure of the WMH-CIDI, we selected and included only items
that were answered by the full sample and no follow-up or disorder-specific questions.
First, we included 26 items from the Screening Section of the WMH-CIDI,
focused on the presence of general psychopathology symptoms across lifetime.
These were as follows: 1) smoking, 2) physical health self-rating, 3) mental health
self-rating, 4) panic attacks, 5) anger attacks (i.e., breaking or smashing something),
6) anger attacks (i.e., hitting or attempting to hit someone), 7) depression, 8) feeling
discouraged, 9) loss of interest, 10) restlessness, 11) bad mood, 12) worrying, 13)
zoophobia, 14) aquaphobia, 15) latrophobia, 16) claustrophobia, 17) acrophobia,
18) aviophobia, 19) being really shy with people, 20) agoraphobia, 21) problems
with concentration as a child, 22) being fidgety as a child, 23) losing temper as a
child or teenager, 24) breaking rules as a child or teenager, 25) feeling upset when
separated from mother, 26) feeling upset when separated from family members in
adulthood. The smoking item, as well as the self-rating items related to physical
and mental health were binarized due to restrictions of the model used (i.e., being
a smoker was encoded as a 1, while ex-smokers or subjects who never smoked were
encoded as a 0; excellent and very good mental or physical health were encoded as
a 1, while good, fair, or poor were encoded as a 0.
Second, we included 1 item measuring psychosis, which was constructed based
on the items measured in the Psychosis Screen section of the WMH-CIDI. The
psychosis item was constructed as follows: If a subject endorsed any of the 6 screening
items measuring psychosis (i.e., seeing a vision; hearing voices; experiencing mind
control; feeling that their mind was being taken over by strange forces; experiencing
attempts at communication from strange forces; believing that there was a plot to
harm them), a 1 was encoded on the psychosis item included in the network. Of note,
if the subject answered on the follow-up question measuring whether any of the items
were endorsed when dreaming, half-asleep or under the influence of alcohol or drugs,
then the original psychosis measure was endorsed as not present.
Page | 292
Third, we included 2 items–obsessions and compulsions–measuring obsessive-
compulsive disorder, which were constructed based on the items measured in the
Obsessive-Compulsive Disorder Section of the WMH-CIDI. The obsession item
was constructed as follows: If a subject endorsed any of the 5 screening items
measuring obsessions (i.e., concerns about germs or contamination; concerns about
causing harm; concerns about symmetry and order; concerns about saving things;
and some other recurrent disturbing thoughts), a 1 was encoded on the obsession
item included in the network. The compulsions item was constructed in a similar
manner: If a subject endorsed any of the 5 screening items measuring compulsions
(i.e., repeatedly washing, cleaning, or decontaminating; checking things; ordering
or touching things; saving things; and some other behaviors they did over and over),
a 1 was encoded on the compulsions item included in the network.
Fourth, we included the first item of the Alcohol Use section of the WMH-CIDI,
measuring the age of drinking the first alcoholic beverage. If a subject reported that
they never drank alcohol, this was encoded as a 0 on the item measuring alcohol
use. All answers providing the age of the first alcoholic beverage, suggesting that
the subject drinks alcohol, were encoded as a 1.
Appendix A1.2. Other measures

We included several other measures, assessing gambling, major chronic conditions,
and functioning/health-related quality of life.
First, we included 1 item measuring gambling, which was constructed based on
the South Oaks Gambling Screen (SOGS; Lesieur & Blume, 1987). The gambling
item was constructed as follows: If a subject endorsed any of the 11 screening items
measuring gambling (i.e., playing cards for money; betting on horses, dogs, or other
animals; betting on sports; paying dice games, including craps, over and under or
other dice games; going to casinos; playing the numbers or betting on lotteries;
playing bingo; playing stock and/ or commodities market; playing slot machines,
poker machines, or other gambling machines; bowling, shooting pool, paying golf,
or some other game of skill for money; playing pull tabs or ‘paper’ games other
than lotteries), for more than once a week, a 1 was encoded on the gambling item
included in the network.
Second, we included 6 items measuring the presence of chronic conditions, based
on the modified version of the CIDI checklist of chronic medical conditions. We
included as standalone items the items with above 5% endorsement (i.e., asthma;
high blood sugar/ diabetes; hypertension; back problems; migraine headaches), to
Page | 293
allow for some variability in the data, as restricted by the model used. All items
under 5% endorsement were encoded in the ‘Other’ category item as follows: If a
subject endorsed any of the 10 items (i.e., arthritis or rheumatism; cancer diagnosed
within the last 3 years; a neurological condition, such as epilepsy, convulsions,
fainting spells, or Parkinson’s disease; stroke or major paralysis; a heart attack,
coronary heart disease, angina, congestive heart failure or other heart disease;
stomach ulcer; choric inflamed bowel, enteritis, or colitis; thyroid disease; kidney
failure; chronic lung disease such as bronchitis or emphysema), a 1 was encoded on
the other chronic conditions item included in the network.
Third, we included 5 items measuring quality-of life and functioning, based
on the EQ5D (Herdman et al., 2011). These were as follows: mobility, self-care,
usual activities, pain/ discomfort, and anxiety/ depression. Due to restrictions of
the model, the items were binarized, with no problems being encoded as a 0, and
some problems/ unable to do encoded as 1.
Appendix A2. Clinical characteristics
Table A1 Item frequency and domain distribution
Item Domain Frequency %

Smoking CIDI Screening Section 2062 31.2%
Physical Health* CIDI Screening Section 5329 80.5%
Mental Health* CIDI Screening Section 6059 91.6%
Panic Attack CIDI Screening Section 1995 30.2%
Anger CIDI Screening Section 1004 15.2%
Lost Control CIDI Screening Section 893 13.5%
Depressed CIDI Screening Section 1797 27.2%
Discouraged CIDI Screening Section 1669 25.2%
Loss Interest CIDI Screening Section 1286 19.4%
Restless CIDI Screening Section 425 6.4%
Bad Mood CIDI Screening Section 869 13.1%
Worrier CIDI Screening Section 1681 25.4%
Zoophobia CIDI Screening Section 1886 28.5%
Aquaphobia CIDI Screening Section 873 13.2%
Latrophobia CIDI Screening Section 1002 15.1%
Claustrophobia CIDI Screening Section 517 7.8%
Page | 294
Table A1Continued.
Item Domain Frequency %

Acrophobia CIDI Screening Section 1060 16.0%
Aerophobia CIDI Screening Section 317 4.8%
Very Shy CIDI Screening Section 993 15.0%
Agoraphobia CIDI Screening Section 501 7.6%
Concentration CIDI Screening Section 350 5.3%
Problems Child
Restless Child CIDI Screening Section 229 3.5%
Trouble Child CIDI Screening Section 532 8.0%
Break Rules Child CIDI Screening Section 1896 28.7%
Separation Child CIDI Screening Section 563 8.5%
Separation CIDI Screening Section 591 8.9%
Psychosis CIDI Psychosis Screen 306 4.6%
Obsessions CIDI Obsessive-Compulsive Disorder 581 8.8%
Section
Compulsions CIDI Obsessive-Compulsive Disorder 388 5.9%
Section
Gambling South Oaks Gambling Screen (SOGS) 810 12.2%
Alcohol Use CIDI Alcohol Use Section 3358 50.8%
Mobility EQ5D 225 3.4%
Self-Care EQ5D 34 0.5%
Usual Activities EQ5D 143 2.2%
Pain/ Discomfort EQ5D 840 12.7%
Anxiety/ Depression EQ5D 488 7.4%
Asthma CIDI Checklist of Chronic Medical 685 10.3%
Conditions
High Blood Sugar/ CIDI Checklist of Chronic Medical 653 9.9%
Diabetes Conditions
Hypertension CIDI Checklist of Chronic Medical 1095 16.5%
Conditions
Back Problems CIDI Checklist of Chronic Medical 436 6.6%
Conditions
Migraine Headaches CIDI Checklist of Chronic Medical 446 6.7%
Conditions
Other CIDI Checklist of Chronic Medical 1179 17.8%
Conditions
*Denotes reverse-coded items (i.e., a higher value indicates better health)
Page | 295
Appendix A3. Accuracy and stability check for the estimated network model
In order to check whether the estimated network connections and centrality
measures were accurate, we carried out bootstrap stability checks, using the
R-package bootnet, version 1.2.4 (Epskamp, Borsboom, et al., 2017). Figure A1
below displays bootstrap results for all edge weights, while Figure A3 displays
the average case-drop bootstraps for strength centrality. In addition, Figures A2
and A4 below display the difference test results for the network connections and
centrality estimates for different variables. An extensive interpretation of each result
is included in the respective figure caption.
To investigate the stability of edge weight parameters, we utilized nonparametric
bootstrap methods, and constructed a 95% bootstrapped confidence interval around
the regularized edge weight. When bootstrapping, a new dataset is computed
through re-sampling (with replacement) of the original data (here 1000 iterations),
thus giving insight into the edge weight variations across different bootstraps.
Figure A1 displays these results: the red dots represent the original sample values,
the black dots represent the bootstrap means, and the grey areas represent the 95%
bootstrapped confidence interval. A narrow interval indicates that the stability is
very good, while a wider interval indicates high variability and thus lower stability.
Here the bootstrapped confidence interval is very narrow, suggesting highly stable
and interpretable results.
To investigate whether the edges identified in the network are significantly
different than each other in terms of strength, we carried out a bootstrap stability
difference test (α=.05; see Figure A2). Each point on the x and y axes represents
a pair of edges identified in the network; of note, here the labels are omitted for
clarity. The gray boxes indicate that two edges do not significantly differ from
each other, while the black boxes indicate that two edges significantly differ from
each other. The diagonal represents the edge strength, and the edges are ordered
from weaker edges to stronger edges (i.e., from white to dark blue). Many edges
here, especially the stronger edges, are significantly different than other edges, thus
indicating that generally edge strength comparison is adequate.
Page | 296
● Bootstrap mean ● Sample
edge
−1 0 1 2 3 4
Figure A1 Bootstrapped 95% Confidence Intervals of edge weights, based on 1000 nonparametric
bootstrap samples.
Page | 297
edge
Figure A2 Edge difference test, based on 1000 nonparametric bootstrap samples.
To investigate the stability of the strength centrality indices, we used a case-

dropping bootstrap procedure (i.e., re-estimated the centrality indices with an
increasingly higher percentage of dropped-out cases from data). We then estimated
the correlation between the original centrality indices and those obtained from
the reduced subsamples. Figure A3 below presents these results. To quantify the
findings, we further computed the correlation stability coefficient (CS-coefficient),
representing the maximum proportion of cases which can be dropped from the
study population to maintain a correlation of at least .70 with the original centrality
indices. The recommendation of Epskamp and colleagues (Epskamp, Borsboom,
et al., 2017) is that the CS-coefficient should be minimum .25, but ideally higher
than .50. The CS-coefficient here was .75, thus indicating reliable results.
Page | 298
1.0 ● ● ● ● ● ●
●
●
●
●
0.5
type
0.0
● strength
−0.5
−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled people
Figure A3 Node strength stability, based on 1000 case-drop bootstraps.
To investigate whether the nodes in the network were significantly different than
each other in terms of strength centrality, we carried out a bootstrap stability
difference test (α=.05; see Figure A4). Each point on the x and y axes represents a
node in the network. The gray boxes indicate that two nodes do not significantly
differ from each other, while the black boxes indicate that two nodes significantly
differ from each other. The numbers in the diagonal represent the values of the
strength centrality measure of the specific node. The most and least central nodes
in the network are generally identified as significantly different than most of the
other nodes, though not significantly different from each other.
Page | 299

strength
strength
Usual
Usual Activities
Activities
Zoophobia 7.5
4.6
Discouraged
Discouraged
Worrier 7.4
6.1
Mobility
Mobility
Very Shy 7.2
5.2
Pain
Pain/ / Discomfort
Discomfort
Usual Activities 6.7
7.5
Panic
Panic Attack
Attack
Trouble Child 6.3
Depressed
Sad, Empty, Depressed
Smoker 6.3
2.8
Loss of Interest (e.g., work,Loss Interest
hobbies)
Separation Child 6.3
4.1
Trouble Child
Losing Temper (childhood or adolescence)
Separation 6.3
4.5
DataOCDShort$Obsessions
Obsessions
Self−Care 6.1
3.7
Worrier
Worrier
Restless Child 6.1
4.1
Anxiety
Anxiety/ / Depression
Depression
Restless 5.6
4.6
Acrophobia
Acrophobia
Physical Health 5.4
5.2
BreakPanic
Breaking Rules (childhood or teenageRules Child
years)
Attack 5.3
6.3
Physical
Physical
Pain Health
Health
/ Discomfort 5.2
6.7
Very Shy
Other
Really Shy with People 5.2
3.4
Concetration Problems
Problems with Concentration Child
Mobility
as Child 5.0
7.2
MigraineAquaphobia
Headaches
Aquaphobia 4.8
2.8
Agoraphobia
Mental Health
Agoraphobia 4.8
4.5
BadControl
Lost
Bad Mood
Mood 4.7
4.6
Restless
Loss Interest
Restless 4.6
6.3
Lost Control
Latrophobia
Anger Attack: Hit or Tried to Hit Someone 4.6
3.8
Zoophobia
Hypertension
Zoophobia 4.6
4.5
Feeling Upset when Separated from Family High Blood Sugar
Members Separation
/ Diabetes
(adulthood) 4.5
3.9
Mental
Mental Health
Health
Discouraged 4.5
7.4
Hypertension
Hypertension
Depressed 4.5
6.3
Anger
Anger Attack: BrokeDataPsychosisShort$Psychosis
or Smashed Something 4.5
1.3
Claustrophobia
Claustrophobia
DataOCDShort$Obsessions 4.4
6.1
Restless Child
DataOCDShort$Compulsions
Being Fidgety (as a child) 4.1
3.7
Separated Separation Child
Feeling Upset when DataGamblingShort$Gambling
from Mother 4.1
1.8
HighDataAlcohol$AlcoholUse
High BloodBlood Sugar
Sugar/ / Diabetes
Diabetes 3.9
3.8
DataAlcohol$AlcoholUse
ConcetrationAlcohol
ProblemsUse
Child 3.8
5.0
Latrophobia
Latrophobia
Claustrophobia 3.8
4.4
Self−Care
BreakSelf-care
Rules Child 3.7
5.3
Compulsions
Bad Mood 3.7
4.7
Other
Other
Back Problems 3.4
2.4
Aviophobia
Aviophobia
Aviophobia 3.1
Smoker
Smoker
Asthma 2.8
1.4
Migraine
Migraine Headaches
Headaches
Aquaphobia 2.8
4.8
BackBack
Anxiety Problems
Problems
/ Depression 2.4
5.6
DataGamblingShort$Gambling
Gambling
Anger 1.8
4.5

Asthma
Asthma
Agoraphobia 1.4
4.8
DataPsychosisShort$Psychosis
Acrophobia
Psychosis 1.3
5.4
Asthma
Back
Migraine
Smoker
Aviophobia
Other
Self−Care
Latrophobia
High
Separation
Restless
Claustrophobia
Anger
Hypertension
Mental
Separation
Zoophobia
Lost
Restless
Bad
Agoraphobia
Aquaphobia
Concetration
Very
Physical
Break
Acrophobia
Anxiety
Worrier
Trouble
Loss
Depressed
Panic
Pain
Mobility
Discouraged
Usual
Acrophobia
Agoraphobia
Anger
Anxiety
Aquaphobia
Asthma
Aviophobia
Back
Bad
Break
Claustrophobia
Concetration
Depressed
Discouraged
High
Hypertension
Latrophobia
Loss
Lost
Mental
Migraine
Mobility
Other
Pain
Panic
Physical
Restless
Restless
Self−Care
Separation
Separation
Smoker
Trouble
Usual
Very
Worrier
Zoophobia
Psychosis
Asthma
Gambling
Back
Migraine
Smoker
Aviophobia
Other
Compulsions
Self-care
Latrophobia
Alcohol Use
High
Feeling Upset
Being Child
Claustrophobia
Anger Attack: Broke or Smashed
Hypertension
Mental
Feeling
Zoophobia
Anger
Restless
Bad
Agoraphobia
Aquaphobia
Problems Problems
Really
Physical
Breaking
Acrophobia
Anxiety/
WorrierChild
Obsessions
Losing
Loss
Sad, Empty,
Panic
Pain/
Mobility
Discouraged
Usual
Mood
Control
/ Discomfort
Shy
Blood
Mood
Interest
Control
Problems
/ Discomfort
Shy
BloodUpset
Interest
Attack
Problems
Activities
Rules
AttackChild
Activities
Rules Child
Mood
Health
Child
Blood
/ Depression
Health
of Interest
Problems
Child
Discomfort
/ Depression
Health
Attack
Headaches
Activities
Fidgety (as a child)
Attack:
Health
Shy with People
Headaches
Temper
Health
Child when Separated
Sugar
Child
Sugar /when
Depression
Health
Headaches
Rules
Problems Child
with Concentration
Sugar/
Depressed
/ Diabetes
Hit or Tried to Hit Someone

Diabetes
(childhood or adolescence)
(childhood or teenage years)
(e.g., work, hobbies)

Child
Diabetes
Separated
as Child
from Mother
from Family Members (adulthood)

Something
Figure A4 Centrality difference test, based on 1000 nonparametric bootstrap samples.
Page | 300
B
Betweenness Closeness Strength

P7 ● ● ●
P6 ● ● ●
P5 ● ● ●
P4 ● ● ●
P3 ● ● ●
P2 ● ● ●
P1 ● ● ●
N7 ● ● ●
N6 ● ● ●
N5 ● ● ●
N4 ● ● ●
N3 ● ● ●
N2 ● ● ●
N1 ● ● ●
GP16 ● ● ●
GP15 ● ● ●
GP14 ● ● ●
GP13 ● ● ●
GP12 ● ● ●
GP11 ● ● ●
GP10 ● ● ●
GP9 ● ● ●
GP8 ● ● ●
GP7 ● ● ●
GP6 ● ● ●
GP5 ● ● ●
GP4 ● ● ●
GP3 ● ● ●
GP2 ● ● ●
GP1 ● ● ●
CT5 ● ● ●
CT4 ● ● ●
CT3 ● ● ●
CT2 ● ● ●
CT1 ● ● ●
0 1 2 3 −2 −1 0 1 −1 0 1
Figure B1 Standardized centrality.
Page | 302
Shortest Paths: Physical Neglect − Positive & Negative Symptoms
CT4
CT2
GP1
CT5 CT3
P3
CT1
P1
GP9 GP2
P6
GP3
GP6
GP4
P5 ● Positive Symptoms
GP15
● Negative Symptoms
GP16
● GPsychopathology
GP14
● Childhood Trauma
N7 N4
GP11
GP5
P4
N2
P7
P2 GP10
GP13
GP12 N1 GP7
GP8
N3
N5 N6
Figure B2 Shortest paths between physical neglect items and positive and negative symptomatology.
Page | 303
Shortest Paths: Emotional Neglect − Positive & Negative Symptoms
CT4
CT2
GP1
CT5 CT3
P3
CT1
P1
GP9 GP2
P6
GP3
GP6
GP4
GP15
GP16
GP14
N7 N4
GP11
GP5
P4
N2
P7
P2 GP10
GP13
GP12 N1 GP7
GP8
N3
N5 N6
Figure B3 Shortest paths between emotional neglect items and positive and negative symptomatology.
Page | 304
Shortest Paths: Emotional Abuse − Positive & Negative Symptoms
CT4 CT2
GP1
CT5 CT3
P3
CT1
P1
GP9 GP2
P6
GP3
GP6
GP4
GP15
GP16
GP14
N7 N4
GP11
GP5
P4
N2
P7
P2 GP10
GP13
GP12 N1 GP7
GP8
N3
N5 N6
Figure B4 Shortest paths between emotional abuse items and positive and negative symptomatology.
Page | 305
CT3
P7
CT5
P6
P5
CT4 P4
P1
CT1
● Positive Symptoms
P2
P3 ● Negative Symptoms
CT2
N5
N7
N3
N2
N6
N4
N1
Figure B5 Network illustration of the relationship between childhood trauma and the positive and
negative symptomatology scales only.
Page | 306
C
Table C1 Means and standard deviations cannabis use and PANSS

Variable Mean (SD)
Current cannabis use 0.13 (0.34)
Cannabis Use - past 12 months intensity of use 0.16 (0.37)
Cannabis Use - past 3 years intensity of use 0.21 (0.41)
Delusions 2.04 (1.37)
Conceptual disorganization 1.45 (0.91)
Hallucinations 1.81 (1.40)
Hyperactivity 1.24 (0.66)
Grandiosity 1.33 (0.87)
Paranoia 1.86 (1.23)
Hostility 1.18 (0.54)
Blunted affect 2.02 (1.21)
Emotional withdrawal 1.65 (0.96)
Poor rapport 1.42 (0.84)
Social withdrawal 1.90 (1.24)
Difficulty in abstract thinking 1.67 (1.09)
Lack of spontaneity 1.60 (1.02)
Stereotyped thinking 1.37 (0.80)
Somatic concern 1.48 (1.00)
Anxiety 2.00 (1.14)
Guilt 1.68 (1.09)
Tension 1.72 (0.96)
Mannerism and posturing 1.14 (0.49)
Depression 1.93 (1.19)
Motor retardation 1.53 (0.90)
Uncooperativeness 1.14 (0.55)
Unusual thought content 1.71 (1.15)
Disorientation 1.09 (0.36)
Poor attention 1.62 (0.98)
Poor judgement 1.59 (1.09)
Disturbed willpower 1.30 (0.72)
Poor impulse control 1.17 (0.56)
Preoccupation 1.26 (0.70)
Social avoidance 1.44 (0.90)
Page | 308
Appendix C. Accuracy and stability checks
We used the R-package bootnet version 1.4.3 (Epskamp, Borsboom, et al., 2017),
following the procedure described by Epskamp and colleagues. Specifically, we
investigated the accuracy of the edge weights using non-parametric bootstrapping (i.e.,
re-estimating the network after resampling). All results are based on 1000 iterations.
Bootstrap mean Sample
edge
P1−−GP9
N2−−N4
N3−−N6
P1−−P6
N4−−GP16
P7−−GP8
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N6−−GP7
P7−−GP14
N2−−N3
P2−−GP13
P4−−GP14
GP15−−GP16
N3−−N7
N1−−N2
P6−−GP16
GP3−−GP6
N1−−N6
N5−−N6
P5−−GP9
P5−−GP12
GP2−−GP3
P2−−GP11
GP1−−GP2
GP8−−GP14
N3−−GP8
N7−−GP5
P4−−P7
P2−−N5
N5−−N7
P5−−N7
P2−−N7
GP11−−GP15
P2−−P4
N5−−GP11
GP11−−GP13
P4−−GP4
N5−−GP12
C−−GP6
N1−−GP3
GP8−−GP12
N1−−GP5
GP14−−GP15
P4−−GP11
GP7−−GP11
GP7−−GP10
P1−−P5
N2−−GP13
GP11−−GP12
P3−−GP11
GP9−−GP15
P6−−GP2
P2−−GP2
GP12−−GP14
P2−−GP10
P1−−P3
P1−−N4
N4−−GP12
GP1−−GP15
N4−−GP6
P6−−GP3
GP13−−GP15
GP4−−GP13
GP8−−GP16
N3−−GP13
P2−−GP14
P4−−GP8
C−−P7
GP1−−GP8
P6−−GP12
GP4−−GP16
P3−−P6
N1−−N4
N5−−GP10
GP1−−GP4
P3−−GP5
GP5−−GP7
GP7−−GP12
GP10−−GP11
N1−−N3
N7−−GP12
P4−−N7
GP4−−GP15
P1−−N7
P4−−P5
P4−−GP15
P5−−P6
P7−−GP12
N6−−GP8
GP9−−GP12
GP5−−GP13
P6−−GP11
C−−P5
P2−−GP12
GP1−−GP9
P7−−GP16
N4−−GP7
N5−−GP5
P1−−GP15
GP3−−GP14
GP6−−GP14
P3−−GP13
P2−−GP9
GP3−−GP11
P3−−GP2
P1−−GP6
P5−−P7
P6−−GP1
P2−−P5
P3−−GP16
N3−−GP5
N6−−GP15
P3−−P7
GP7−−GP9
GP4−−GP5
GP10−−GP16
P6−−GP4
C−−GP12
C−−GP10
C−−P2
N2−−GP15
C−−GP14
N7−−GP16
C−−N1
P1−−P2
GP6−−GP7
N2−−GP16
GP8−−GP10
C−−N2
N5−−GP7
GP4−−GP11
N7−−GP3
P3−−GP3
P7−−GP6
GP9−−GP14
C−−N4
GP8−−GP11
N6−−GP6
N6−−GP12
N3−−GP14
GP1−−GP3
P4−−N5
GP6−−GP10
P3−−GP10
GP7−−GP13
N1−−GP4
N4−−N5
P7−−GP11
P6−−GP8
N4−−GP2
GP5−−GP15
P2−−GP1
N2−−GP3
GP4−−GP6
N3−−N4
P2−−GP5
GP1−−GP13
N1−−GP12
GP2−−GP16
N5−−GP16
GP3−−GP4
P4−−GP5
C−−GP15
P1−−GP13
GP12−−GP15
GP1−−GP7
GP6−−GP16
P6−−N1
GP13−−GP16
P6−−GP6
P7−−N7
P2−−GP8
P7−−GP7
P2−−GP16
P3−−N5
N3−−GP10
GP3−−GP8
N2−−GP4
N5−−GP9
P1−−GP8
P6−−P7
N4−−GP11
N2−−GP6
GP12−−GP13
GP6−−GP13
GP1−−GP6
N7−−GP10
P3−−GP6
P7−−N1
GP3−−GP5
P3−−GP12
N3−−GP12
N2−−GP12
P1−−GP2
P6−−N3
P1−−N1
P3−−GP15
P6−−N2
C−−P6
GP10−−GP12
GP3−−GP13
N1−−N7
P4−−GP3
GP1−−GP16
N4−−GP9
P1−−GP4
N1−−GP6
P5−−GP3
GP3−−GP9
GP14−−GP16
P2−−N2
N1−−GP16
P5−−GP6
P7−−GP1
N4−−N6
N6−−GP11
N3−−GP4
C−−GP9
GP13−−GP14
GP6−−GP11
GP4−−GP14
C−−P4
GP4−−GP8
P1−−GP1
P4−−N4
N7−−GP14
P4−−GP2
P4−−N3
GP2−−GP15
P1−−GP11
N1−−GP10
P3−−N2
N2−−GP10
GP9−−GP16
N6−−GP10
GP5−−GP9
P6−−GP5
P1−−GP12
GP9−−GP10
C−−GP1
GP4−−GP12
GP4−−GP9
P1−−P4
N1−−GP11
GP2−−GP11
GP11−−GP16
C−−P1
GP3−−GP16
P5−−GP8
P7−−N5
P2−−GP15
P5−−GP2
N2−−N6
P1−−GP3
GP1−−GP10
P5−−N3
P1−−N6
P2−−N4
P3−−N6
P5−−GP1
N5−−GP4
N4−−GP14
N6−−GP5
P4−−GP9
GP1−−GP5
P2−−P3
N1−−GP9
P3−−GP1
P3−−N3
P1−−GP5
P4−−GP12
N6−−GP13
N2−−GP1
GP10−−GP14
GP7−−GP15
P4−−GP10
P5−−N2
N1−−GP13
N2−−N7
GP4−−GP10
P7−−GP2
N5−−GP14
P7−−GP5
N7−−GP6
N4−−GP1
N4−−GP13
GP5−−GP8
GP2−−GP10
N2−−GP7
N3−−GP16
N3−−GP9
GP6−−GP9
P5−−GP7
P6−−GP10
N7−−GP2
N5−−GP1
N3−−GP2
P3−−N1
P6−−N7
P2−−GP7
P6−−N5
GP3−−GP15
P5−−GP5
N1−−GP8
N4−−GP10
P2−−P7
P5−−GP15
N5−−GP2
N4−−N7
N7−−GP1
N2−−GP5
N7−−GP7
N1−−GP2
C−−GP13
GP2−−GP13
GP10−−GP13
P6−−GP14
P4−−N2
N5−−GP3
N1−−GP15
N6−−GP14
GP2−−GP7
P1−−P7
GP2−−GP8
GP8−−GP9
P6−−N6
N3−−GP11
N2−−N5
P5−−GP11
GP9−−GP13
GP7−−GP16
N2−−GP8
N1−−GP1
N7−−GP11
GP5−−GP10
P1−−GP7
P5−−GP14
GP1−−GP14
N2−−GP14
N4−−GP4
GP5−−GP16
C−−GP4
N6−−GP1
GP3−−GP10
GP5−−GP14
P7−−N6
P2−−N3
GP12−−GP16
P1−−GP14
GP1−−GP12
C−−GP11
N3−−GP7
N6−−GP3
P7−−N3
GP6−−GP8
P7−−N2
GP5−−GP11
P3−−GP7
N7−−GP13
P3−−GP14
P4−−P6
P2−−P6
P3−−N4
N1−−N5
P1−−GP10
C−−GP2
GP2−−GP12
N7−−GP9
P5−−N4
C−−N3
N2−−GP2
N3−−GP15
GP11−−GP14
P6−−GP13
P7−−GP9
P7−−GP13
P2−−N6
GP2−−GP5
P7−−GP4
P3−−P4
N6−−GP2
P6−−GP7
N7−−GP4
P5−−GP10
GP9−−GP11
P4−−GP16
GP3−−GP7
C−−P3
P4−−GP1
C−−GP16
P3−−P5
N5−−GP8
P5−−GP4
GP4−−GP7
P4−−GP13
GP7−−GP14
P2−−N1
N6−−GP4
GP5−−GP6
N6−−GP9
N2−−GP9
N3−−GP1
P6−−GP9
C−−GP7
P5−−N5
N6−−N7
P2−−GP3
N3−−GP6
P7−−N4
P3−−GP4
P1−−N2
P5−−GP13
C−−GP5
P1−−N3
P1−−N5
N2−−GP11
P5−−GP16
N7−−GP8
N5−−GP13
GP5−−GP12
N6−−GP16
N3−−N5
GP7−−GP8
GP8−−GP15
P3−−GP8
C−−N6
GP2−−GP9
C−−GP3
P7−−GP15
P4−−GP6
N4−−GP5
N4−−GP8
GP2−−GP14
P5−−N6
N4−−GP3
GP6−−GP12
P4−−N6
GP8−−GP13
P1−−GP16
P2−−GP6
C−−GP8
P7−−GP10
GP6−−GP15
C−−N5
P2−−GP4
GP1−−GP11
P6−−GP15
GP3−−GP12
C−−N7
P4−−N1
N1−−GP14
P4−−GP7
P5−−N1
GP10−−GP15
N3−−GP3
P6−−N4
P3−−N7
N5−−GP6
N4−−GP15
N5−−GP15
P7−−GP3
−0.2 0.0 0.2 0.4 0.6
Figure C1 Accuracy of the edge-weights for the current cannabis use and symptomatology network.
The horizonal area within the plot represents the 95% quantile range of the parameter values across
1000 bootstraps. The red dots indicate the sample values, while the black dots indicate the bootstrap
mean values.
Page | 309
edge
P1−−GP9
N2−−N4
N3−−N6
N4−−GP16
P1−−P6
P7−−GP8
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N2−−N3
N6−−GP7
P7−−GP14
P2−−GP13
P6−−GP16
N1−−N2
P4−−GP14
N3−−N7
C_m−−GP12
GP3−−GP6
N5−−N6
N1−−N6
GP15−−GP16
P2−−GP11
P5−−GP9
N3−−GP8
GP1−−GP2
N7−−GP5
GP2−−GP3
GP8−−GP14
P5−−GP12
P4−−P7
P2−−N5
P5−−N7
GP11−−GP15
P2−−N7
C_m−−GP6
N5−−GP11
GP14−−GP15
N5−−N7
N1−−GP5
P2−−P4
GP8−−GP12
N1−−GP3
P4−−GP4
P2−−GP10
GP11−−GP12
GP11−−GP13
P2−−GP2
GP7−−GP10
P4−−GP11
P1−−P5
P6−−GP2
N7−−GP12
N5−−GP12
GP13−−GP15
GP7−−GP11
N3−−GP13
N6−−GP8
N2−−GP13
P3−−GP11
GP4−−GP13
GP9−−GP15
P1−−P3
P6−−GP3
N1−−N4
P1−−N4
GP12−−GP14
P4−−P5
GP1−−GP15
GP8−−GP16
P4−−GP8
GP1−−GP8
GP4−−GP16
P2−−GP14
N4−−GP6
N4−−GP12
GP7−−GP12
GP10−−GP11
C_m−−P5
P4−−N7
P6−−GP12
P1−−GP15
P5−−P6
GP1−−GP4
N5−−GP10
P3−−GP5
GP1−−GP9
P6−−GP11
GP5−−GP13
P3−−P6
GP7−−GP9
P4−−GP15
GP3−−GP11
P2−−GP9
GP4−−GP15
P3−−GP16
N1−−N3
N3−−GP5
P1−−N7
P3−−GP13
GP9−−GP12
P3−−GP3
C_m−−P2
GP3−−GP14
P6−−GP1
GP5−−GP7
C_m−−GP14
P1−−GP6
N5−−GP7
GP6−−GP10
P1−−P2
GP6−−GP7
GP6−−GP14
N7−−GP3
N5−−GP5
N2−−GP15
N4−−GP7
C_m−−N1
P2−−GP12
N7−−GP16
P7−−GP12
C_m−−P7
P3−−P7
P3−−GP2
C_m−−N2
P2−−P5
GP8−−GP10
C_m−−P6
P5−−P7
C_m−−P4
N2−−GP16
P7−−GP16
P6−−GP4
GP4−−GP11
C_m−−GP4
P7−−GP6
GP4−−GP5
GP1−−GP3
N6−−GP12
N6−−GP6
N6−−GP15
GP9−−GP14
P3−−GP10
N3−−GP14
P7−−GP11
C_m−−N3
N3−−N4
GP7−−GP13
P4−−N5
N1−−GP4
GP13−−GP16
P2−−GP1
N2−−GP3
P6−−GP8
N4−−N5
P2−−GP5
GP8−−GP11
GP5−−GP15
N4−−GP2
P1−−GP13
GP3−−GP4
GP1−−GP13
P4−−GP5
GP2−−GP16
GP6−−GP16
GP1−−GP7
GP4−−GP6
N5−−GP16
P2−−GP8
P7−−GP7
N1−−GP12
GP12−−GP15
C_m−−GP10
P2−−GP16
P3−−N5
P7−−N7
GP12−−GP13
N3−−GP10
P6−−N1
GP10−−GP16
P6−−P7
P6−−GP6
N4−−GP11
GP3−−GP8
GP1−−GP6
N2−−GP4
GP3−−GP5
P3−−GP12
GP6−−GP13
GP3−−GP13
P1−−GP8
P7−−N1
N5−−GP9
GP10−−GP12
P3−−GP6
P6−−N2
N7−−GP10
C_m−−N4
P3−−GP15
N2−−GP6
P6−−N3
C_m−−GP9
N1−−N7
P1−−N1
N3−−GP12
GP6−−GP11
N4−−GP9
N1−−GP6
GP1−−GP16
P7−−GP1
P1−−GP2
GP14−−GP16
P5−−GP3
P4−−GP3
P1−−GP4
P2−−N2
C_m−−GP15
N4−−N6
N1−−GP16
GP4−−GP8
N2−−GP10
N2−−GP12
GP3−−GP9
GP13−−GP14
GP2−−GP15
P1−−GP12
P4−−GP2
N3−−GP4
N7−−GP14
P4−−N4
N6−−GP11
GP5−−GP9
P5−−GP6
P1−−GP1
GP4−−GP14
P3−−N2
GP9−−GP10
P2−−GP15
P1−−P4
N6−−GP10
P4−−N3
P1−−N6
N1−−GP11
GP4−−GP9
P1−−GP11
GP9−−GP16
P5−−GP2
N5−−GP4
N1−−GP10
GP3−−GP16
GP1−−GP10
P1−−GP3
GP2−−GP11
P5−−GP8
P5−−N3
GP1−−GP5
GP11−−GP16
P6−−GP5
N4−−GP14
N2−−N6
P2−−N4
P7−−N5
N6−−GP5
GP7−−GP15
GP4−−GP12
GP10−−GP14
P3−−N6
N1−−GP9
P5−−GP1
P1−−GP5
P5−−N2
N2−−GP1
N7−−GP6
P2−−P3
P3−−N3
P3−−GP1
P4−−GP10
P4−−GP9
P7−−GP2
GP4−−GP10
N3−−GP2
P6−−GP10
N2−−GP7
N6−−GP13
C_m−−GP16
P5−−GP7
N5−−GP14
GP2−−GP10
N1−−GP13
N3−−GP9
N4−−GP1
N1−−GP8
N3−−GP16
N7−−GP7
P4−−GP12
P7−−GP5
N4−−N7
N2−−N7
GP5−−GP8
P6−−N7
GP8−−GP9
P3−−N1
P6−−N5
N4−−GP13
N5−−GP1
N5−−GP2
P2−−GP7
GP6−−GP9
P2−−P7
N4−−GP10
N7−−GP2
N1−−GP2
P5−−GP5
GP1−−GP12
N7−−GP1
P5−−GP15
GP3−−GP15
P5−−GP11
P6−−GP14
P1−−P7
N3−−GP11
P4−−N2
GP2−−GP7
N2−−GP5
GP10−−GP13
N1−−GP15
GP9−−GP13
N2−−GP8
N5−−GP3
P6−−N6
GP2−−GP13
N6−−GP14
N4−−GP4
N2−−N5
GP5−−GP10
GP2−−GP8
N1−−GP1
P5−−GP14
P1−−GP14
GP7−−GP16
N6−−GP1
GP5−−GP14
GP1−−GP14
GP3−−GP10
GP5−−GP16
C_m−−GP1
N7−−GP11
N2−−GP14
C_m−−P1
N3−−GP7
P1−−GP7
P7−−N6
N6−−GP3
GP6−−GP8
C_m−−N5
P7−−N3
P2−−N3
P3−−GP14
P7−−N2
N2−−GP2
GP2−−GP12
GP12−−GP16
P3−−GP7
P3−−N4
P2−−P6
C_m−−GP7
P4−−P6
GP5−−GP11
N7−−GP13
GP11−−GP14
N7−−GP9
P2−−N6
P7−−GP9
P7−−GP4
P5−−N4
P7−−GP13
P4−−GP16
N1−−N5
P3−−P4
P1−−GP10
N3−−GP15
P6−−GP13
P6−−GP7
N7−−GP4
N6−−GP4
P4−−GP1
GP9−−GP11
GP2−−GP5
P3−−P5
N5−−GP8
GP3−−GP7
P5−−GP10
GP4−−GP7
N6−−GP9
P2−−N1
GP5−−GP6
N6−−GP2
N3−−GP1
GP7−−GP14
P4−−GP13
P5−−GP4
P2−−GP3
P6−−GP9
N2−−GP9
P5−−GP13
P5−−N5
N6−−N7
P3−−GP4
P7−−N4
N3−−GP6
P1−−N2
P1−−N5
N2−−GP11
P1−−N3
P7−−GP15
N5−−GP13
N7−−GP8
P5−−GP16
P4−−GP6
C_m−−GP5
C_m−−GP13
N6−−GP16
N3−−N5
N4−−GP5
N4−−GP3
GP8−−GP15
GP2−−GP9
P3−−GP8
C_m−−N6
C_m−−GP2
C_m−−GP3
GP5−−GP12
GP2−−GP14
GP7−−GP8
P5−−N6
GP6−−GP12
C_m−−GP11
GP8−−GP13
P2−−GP4
GP6−−GP15
N4−−GP8
P2−−GP6
P7−−GP10
P1−−GP16
P6−−GP15
GP1−−GP11
P4−−GP7
P4−−N6
P4−−N1
P6−−N4
C_m−−P3
GP3−−GP12
GP10−−GP15
C_m−−GP8
N1−−GP14
N5−−GP15
N5−−GP6
N3−−GP3
P5−−N1
N4−−GP15
P3−−N7
P7−−GP3
C_m−−N7
−0.2 0.0 0.2 0.4 0.6
Figure C2 Accuracy of the edge-weights for the intensity of cannabis use in the past 12 months and
symptomatology network. The horizonal area within the plot represents the 95% quantile range of
the parameter values across 1000 bootstraps. The red dots indicate the sample values, while the black
dots indicate the bootstrap mean values.
Page | 310
edge
P1−−GP9
N2−−N4
N3−−N6
N4−−GP16
P7−−GP8
P1−−P6
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N2−−N3
P2−−GP13
N6−−GP7
P7−−GP14
P6−−GP16
N1−−N2
N3−−N7
P4−−GP14
GP15−−GP16
N1−−N6
GP3−−GP6
N5−−N6
GP2−−GP3
GP8−−GP14
C_y−−GP12
GP1−−GP2
P2−−GP11
P5−−GP12
N3−−GP8
P5−−N7
GP11−−GP15
P2−−N5
N5−−N7
P5−−GP9
N5−−GP11
P4−−P7
C_y−−GP6
N7−−GP5
P1−−P5
P2−−P4
GP14−−GP15
GP8−−GP12
GP11−−GP13
P2−−N7
N1−−GP5
P6−−GP2
GP9−−GP15
N1−−GP3
GP11−−GP12
P4−−GP4
P6−−GP3
GP12−−GP14
N2−−GP13
P4−−GP11
N5−−GP12
P2−−GP10
GP7−−GP11
C_y−−P6
P3−−GP11
P2−−GP2
P1−−P3
N4−−GP12
GP7−−GP10
GP1−−GP15
P4−−P5
GP4−−GP13
GP10−−GP11
N3−−GP13
P4−−GP8
GP13−−GP15
C_y−−P2
N5−−GP10
N4−−GP6
GP4−−GP16
N7−−GP12
N4−−GP7
P1−−N4
P6−−GP11
N6−−GP8
GP7−−GP12
P2−−GP14
GP1−−GP4
GP8−−GP16
P4−−N7
C_y−−P5
C_y−−GP14
N3−−GP5
P4−−GP15
GP7−−GP9
P6−−GP12
GP4−−GP15
P3−−GP13
P3−−P6
P5−−P6
P3−−GP5
GP5−−GP13
N5−−GP5
GP9−−GP12
P2−−GP9
N1−−N4
GP6−−GP7
P7−−GP12
GP1−−GP9
N1−−N3
N2−−GP15
N5−−GP7
GP5−−GP7
P1−−GP15
P1−−N7
P3−−GP2
GP3−−GP11
P1−−P2
P3−−GP3
P2−−P5
GP6−−GP10
GP1−−GP8
P3−−GP16
P2−−GP12
N7−−GP16
GP6−−GP14
C_y−−P7
GP8−−GP10
P3−−P7
C_y−−GP10
C_y−−N1
GP3−−GP14
N2−−GP16
P5−−P7
P7−−GP16
P6−−GP1
N7−−GP3
C_y−−GP1
P1−−GP6
GP4−−GP11
P7−−GP6
GP9−−GP14
GP4−−GP5
N6−−GP15
P6−−GP4
N3−−GP14
P3−−GP10
P7−−GP11
GP1−−GP3
N6−−GP12
P4−−N5
C_y−−N2
P2−−GP5
N1−−GP4
P2−−GP1
C_y−−N4
N6−−GP6
N3−−N4
N4−−N5
GP8−−GP11
P6−−GP8
GP7−−GP13
GP2−−GP16
GP1−−GP13
P4−−GP5
N4−−GP2
GP5−−GP15
P1−−GP13
P7−−GP7
N2−−GP3
GP3−−GP4
GP1−−GP7
P2−−GP8
GP6−−GP16
GP12−−GP13
N5−−GP16
GP4−−GP6
GP13−−GP16
P7−−N7
GP12−−GP15
N1−−GP12
GP3−−GP8
P3−−N5
N2−−GP4
N3−−GP10
P6−−N1
GP3−−GP5
P7−−N1
P3−−GP6
P6−−GP6
N5−−GP9
P3−−GP12
P2−−GP16
GP3−−GP13
GP6−−GP13
GP10−−GP16
N4−−GP11
N3−−GP12
N2−−GP6
P3−−GP15
P1−−GP8
N1−−GP6
N7−−GP10
N1−−N7
P5−−GP3
P1−−N1
P6−−P7
P1−−GP4
GP1−−GP6
C_y−−GP4
P6−−N3
N2−−GP12
P1−−GP2
N4−−GP9
P6−−N2
GP10−−GP12
GP13−−GP14
GP6−−GP11
GP14−−GP16
P2−−N2
P4−−GP3
N1−−GP16
GP4−−GP8
P7−−GP1
C_y−−P4
P1−−GP1
GP1−−GP16
GP3−−GP9
N1−−GP10
N6−−GP11
GP9−−GP16
P1−−GP12
N3−−GP4
P4−−N4
P4−−N3
P1−−GP11
GP4−−GP14
N7−−GP14
N2−−GP10
P5−−GP6
GP2−−GP15
P3−−N2
GP9−−GP10
N4−−N6
GP5−−GP9
P1−−P4
P2−−GP15
P4−−GP2
P5−−GP2
N1−−GP11
P1−−N6
GP4−−GP12
GP1−−GP10
C_y−−N6
GP4−−GP9
GP11−−GP16
P6−−GP5
N6−−GP10
P5−−GP8
N2−−N6
GP2−−GP11
N5−−GP4
P7−−N5
GP1−−GP5
P5−−N3
N4−−GP14
P2−−N4
N6−−GP5
P3−−N6
C_y−−GP9
GP3−−GP16
P1−−GP3
P4−−GP9
P1−−GP5
N6−−GP13
GP10−−GP14
P6−−N7
N1−−GP9
P3−−N3
P2−−P3
P5−−GP1
C_y−−GP16
P4−−GP12
P5−−N2
N4−−GP13
GP7−−GP15
P4−−GP10
GP4−−GP10
P5−−GP7
P7−−GP2
P7−−GP5
N2−−GP1
N3−−GP16
N2−−N7
C_y−−GP15
N3−−GP9
N2−−GP7
N1−−GP13
P6−−GP10
P3−−GP1
N5−−GP14
P2−−GP7
GP2−−GP10
N4−−GP1
N3−−GP2
N7−−GP6
GP6−−GP9
P6−−N5
N4−−GP10
N1−−GP8
N5−−GP1
P2−−P7
GP5−−GP8
P5−−GP15
N1−−GP15
P3−−N1
P5−−GP5
N2−−GP5
N5−−GP2
P1−−P7
N4−−N7
GP3−−GP15
N7−−GP1
N1−−GP2
P5−−GP11
GP8−−GP9
N5−−GP3
P4−−N2
N7−−GP2
P6−−N6
GP2−−GP8
N7−−GP7
N7−−GP11
P6−−GP14
GP2−−GP13
P1−−GP14
GP9−−GP13
N2−−N5
N6−−GP14
N6−−GP1
GP2−−GP7
N3−−GP11
GP7−−GP16
GP5−−GP14
N1−−GP1
P2−−N3
GP10−−GP13
P5−−GP14
GP5−−GP10
GP3−−GP10
N2−−GP8
GP5−−GP16
GP1−−GP14
P1−−GP7
N2−−GP14
N4−−GP4
GP1−−GP12
GP12−−GP16
N6−−GP3
N3−−GP7
GP2−−GP12
P3−−GP7
GP6−−GP8
P7−−N3
N2−−GP2
P7−−N6
P7−−N2
N7−−GP13
GP5−−GP11
N7−−GP9
P1−−GP10
P3−−N4
P3−−GP14
C_y−−N3
P4−−P6
GP11−−GP14
P5−−N4
N1−−N5
N3−−GP15
P2−−N6
P2−−P6
P7−−GP13
N7−−GP4
P7−−GP4
P6−−GP7
P6−−GP13
P3−−P4
P4−−GP1
P7−−GP9
N6−−GP4
GP5−−GP6
GP7−−GP14
P4−−GP16
GP2−−GP5
P3−−P5
N6−−GP2
P5−−GP10
N5−−GP8
GP3−−GP7
GP9−−GP11
GP4−−GP7
P2−−N1
P4−−GP13
P5−−GP4
N2−−GP9
N3−−GP1
P2−−GP3
N3−−GP6
P6−−GP9
N6−−GP9
C_y−−P3
C_y−−GP3
N6−−N7
P7−−N4
P5−−N5
P1−−N2
P3−−GP4
C_y−−N5
P5−−GP13
N2−−GP11
P1−−N5
P4−−GP6
GP5−−GP12
C_y−−P1
P1−−N3
N7−−GP8
P5−−GP16
C_y−−GP11
N4−−GP3
N6−−GP16
N4−−GP5
GP2−−GP14
P3−−GP8
GP8−−GP13
GP2−−GP9
C_y−−GP8
P1−−GP16
GP7−−GP8
N3−−N5
P7−−GP15
N5−−GP13
P5−−N6
N4−−GP8
GP1−−GP11
C_y−−GP13
P7−−GP10
GP8−−GP15
GP6−−GP12
P4−−N6
C_y−−N7
GP6−−GP15
P6−−GP15
P4−−N1
P2−−GP6
P4−−GP7
P2−−GP4
P5−−N1
C_y−−GP7
GP10−−GP15
N1−−GP14
GP3−−GP12
P6−−N4
C_y−−GP5
N3−−GP3
N5−−GP6
N5−−GP15
P3−−N7
P7−−GP3
N4−−GP15
C_y−−GP2
−0.2 0.0 0.2 0.4 0.6
Figure C3 Accuracy of the edge-weights for the intensity of cannabis use in the past 3 years and
symptomatology network. The horizonal area within the plot represents the 95% quantile range of
the parameter values across 1000 bootstraps. The red dots indicate the sample values, while the black
dots indicate the bootstrap mean values.
Page | 311
Page | 312
GP16 P1 P2 GP16 P1 P2 GP16 P1 P2
GP15 P3 GP15 P3 GP15 P3
GP14 P4 GP14 P4 GP14 P4
GP13 P5 GP13 P5 GP13 P5
GP12 P6 GP12 0.65 P6 GP12 0.69 P6
GP11 P7 GP11 P7 GP11 0.62 P7

0.96 0.96 0.77
0.63
GP10 N1 GP10 N1 GP10 N1
C C_m C_y
0.64
0.67
GP8 0.65 N3 GP8 0.92 N3 GP8 0.82 N3
0.75
0.71 0.84 0.64
GP7 N4 GP7 N4 GP7 0.8 N4

GP3 GP2 GP1 GP3 GP2 GP1 GP3 GP2 GP1
Figure C4 Bootstrapped edge inclusion probabilities, based on 1000 nonparametric bootstraps. The threshold was set at 60%, displaying the edges that have
been included in over 60% of the bootstraps. C refers to current cannabis use, C_m refers to intensity of cannabis use in the past 12 months, while C_ y refers to
intensity of cannabis use in the past
edge
P1−−GP9
N2−−N4
N3−−N6
P1−−P6
N4−−GP16
P7−−GP8
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N6−−GP7
P7−−GP14
N2−−N3
P2−−GP13
P4−−GP14
GP15−−GP16
N3−−N7
N1−−N2
P6−−GP16
GP3−−GP6
N1−−N6
N5−−N6
P5−−GP9
P5−−GP12
GP2−−GP3
P2−−GP11
GP1−−GP2
GP8−−GP14
N3−−GP8
N7−−GP5
P4−−P7
P2−−N5
N5−−N7
P5−−N7
P2−−N7
GP11−−GP15
P2−−P4
N5−−GP11
GP11−−GP13
P4−−GP4
N5−−GP12
C−−GP6
N1−−GP3
GP8−−GP12
N1−−GP5
GP14−−GP15
P4−−GP11
GP7−−GP11
GP7−−GP10
P1−−P5
N2−−GP13
GP11−−GP12
P3−−GP11
GP9−−GP15
P6−−GP2
P2−−GP2
GP12−−GP14
P2−−GP10
P1−−P3
P1−−N4
N4−−GP12
GP1−−GP15
N4−−GP6
P6−−GP3
GP13−−GP15
GP4−−GP13
GP8−−GP16
N3−−GP13
P2−−GP14
P4−−GP8
C−−P7
GP1−−GP8
P6−−GP12
GP4−−GP16
P3−−P6
N1−−N4
N5−−GP10
GP1−−GP4
P3−−GP5
GP5−−GP7
GP7−−GP12
GP10−−GP11
N1−−N3
N7−−GP12
P4−−N7
GP4−−GP15
P1−−N7
P4−−P5
P4−−GP15
P5−−P6
P7−−GP12
N6−−GP8
GP9−−GP12
GP5−−GP13
P6−−GP11
C−−P5
P2−−GP12
GP1−−GP9
P7−−GP16
N4−−GP7
N5−−GP5
P1−−GP15
GP3−−GP14
GP6−−GP14
P3−−GP13
P2−−GP9
GP3−−GP11
P3−−GP2
P1−−GP6
P5−−P7
P6−−GP1
P2−−P5
P3−−GP16
N3−−GP5
N6−−GP15
P3−−P7
GP7−−GP9
GP4−−GP5
GP10−−GP16
P6−−GP4
P7−−GP15
P4−−GP6
N4−−GP5
N4−−GP8
GP2−−GP14
P5−−N6
N4−−GP3
GP6−−GP12
P4−−N6
GP8−−GP13
P1−−GP16
P2−−GP6
C−−GP8
P7−−GP10
GP6−−GP15
C−−N5
P2−−GP4
GP1−−GP11
P6−−GP15
GP3−−GP12
C−−N7
P4−−N1
N1−−GP14
P4−−GP7
P5−−N1
GP10−−GP15
N3−−GP3
P6−−N4
P3−−N7
N5−−GP6
N4−−GP15
N5−−GP15
P7−−GP3
P7−−GP3
N5−−GP15
N4−−GP15
N5−−GP6
P3−−N7
P6−−N4
N3−−GP3
GP10−−GP15
P5−−N1
P4−−GP7
N1−−GP14
P4−−N1
C−−N7
GP3−−GP12
P6−−GP15
GP1−−GP11
P2−−GP4
C−−N5
GP6−−GP15
P7−−GP10
C−−GP8
P2−−GP6
P1−−GP16
GP8−−GP13
P4−−N6
GP6−−GP12
N4−−GP3
P5−−N6
GP2−−GP14
N4−−GP8
N4−−GP5
P4−−GP6
P7−−GP15
P6−−GP4
GP10−−GP16
GP4−−GP5
GP7−−GP9
P3−−P7
N6−−GP15
N3−−GP5
P3−−GP16
P2−−P5
P6−−GP1
P5−−P7
P1−−GP6
P3−−GP2
GP3−−GP11
P2−−GP9
P3−−GP13
GP6−−GP14
GP3−−GP14
P1−−GP15
N5−−GP5
N4−−GP7
P7−−GP16
GP1−−GP9
P2−−GP12
C−−P5
P6−−GP11
GP5−−GP13
GP9−−GP12
N6−−GP8
P7−−GP12
P5−−P6
P4−−GP15
P4−−P5
P1−−N7
GP4−−GP15
P4−−N7
N7−−GP12
N1−−N3
GP10−−GP11
GP7−−GP12
GP5−−GP7
P3−−GP5
GP1−−GP4
N5−−GP10
N1−−N4
P3−−P6
GP4−−GP16
P6−−GP12
GP1−−GP8
C−−P7
P4−−GP8
P2−−GP14
N3−−GP13
GP8−−GP16
GP4−−GP13
GP13−−GP15
P6−−GP3
N4−−GP6
GP1−−GP15
N4−−GP12
P1−−N4
P1−−P3
P2−−GP10
GP12−−GP14
P2−−GP2
P6−−GP2
GP9−−GP15
P3−−GP11
GP11−−GP12
N2−−GP13
P1−−P5
GP7−−GP10
GP7−−GP11
P4−−GP11
GP14−−GP15
N1−−GP5
GP8−−GP12
N1−−GP3
C−−GP6
N5−−GP12
P4−−GP4
GP11−−GP13
N5−−GP11
P2−−P4
GP11−−GP15
P2−−N7
P5−−N7
N5−−N7
P2−−N5
P4−−P7
N7−−GP5
N3−−GP8
GP8−−GP14
GP1−−GP2
P2−−GP11
GP2−−GP3
P5−−GP12
P5−−GP9
N5−−N6
N1−−N6
GP3−−GP6
P6−−GP16
N1−−N2
N3−−N7
GP15−−GP16
P4−−GP14
P2−−GP13
N2−−N3
P7−−GP14
N6−−GP7
P3−−GP9
GP2−−GP4
N7−−GP15
N1−−GP7
GP2−−GP6
P7−−GP8
N4−−GP16
P1−−P6
N3−−N6
N2−−N4
P1−−GP9
Figure C5 Bootstrapped difference test for edge weights for the current cannabis use and symptom-
atology network. The significance difference testing (α=.05) examines whether edges significantly differ
from each other in strength. The color of the boxes indicates whether there is a significant difference (i.e.,
grey boxes reflect no significant differences and black boxes reflect significant differences).
Page | 313
edge
P1−−GP9
N2−−N4
N3−−N6
N4−−GP16
P1−−P6
P7−−GP8
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N2−−N3
N6−−GP7
P7−−GP14
P2−−GP13
P6−−GP16
N1−−N2
P4−−GP14
N3−−N7
C_m−−GP12
GP3−−GP6
N5−−N6
N1−−N6
GP15−−GP16
P2−−GP11
P5−−GP9
N3−−GP8
GP1−−GP2
N7−−GP5
GP2−−GP3
GP8−−GP14
P5−−GP12
P4−−P7
P2−−N5
P5−−N7
GP11−−GP15
P2−−N7
C_m−−GP6
N5−−GP11
GP14−−GP15
N5−−N7
N1−−GP5
P2−−P4
GP8−−GP12
N1−−GP3
P4−−GP4
P2−−GP10
GP11−−GP12
GP11−−GP13
P2−−GP2
GP7−−GP10
P4−−GP11
P1−−P5
P6−−GP2
N7−−GP12
N5−−GP12
GP13−−GP15
GP7−−GP11
N3−−GP13
N6−−GP8
N2−−GP13
P3−−GP11
GP4−−GP13
GP9−−GP15
P1−−P3
P6−−GP3
N1−−N4
P1−−N4
GP12−−GP14
P4−−P5
GP1−−GP15
GP8−−GP16
P4−−GP8
GP1−−GP8
GP4−−GP16
P2−−GP14
N4−−GP6
N4−−GP12
GP7−−GP12
GP10−−GP11
C_m−−P5
P4−−N7
P6−−GP12
P1−−GP15
P5−−P6
GP1−−GP4
N5−−GP10
P3−−GP5
GP1−−GP9
P6−−GP11
GP5−−GP13
P3−−P6
GP7−−GP9
P4−−GP15
GP3−−GP11
P2−−GP9
GP4−−GP15
P3−−GP16
N1−−N3
N3−−GP5
P1−−N7
P3−−GP13
GP9−−GP12
P3−−GP3
C_m−−P2
GP3−−GP14
P6−−GP1
GP5−−GP7
C_m−−GP14
P1−−GP6
N5−−GP7
GP6−−GP10
P1−−P2
GP6−−GP7
GP6−−GP14
N7−−GP3
GP5−−GP12
GP2−−GP14
GP7−−GP8
P5−−N6
GP6−−GP12
C_m−−GP11
GP8−−GP13
P2−−GP4
GP6−−GP15
N4−−GP8
P2−−GP6
P7−−GP10
P1−−GP16
P6−−GP15
GP1−−GP11
P4−−GP7
P4−−N6
P4−−N1
P6−−N4
C_m−−P3
GP3−−GP12
GP10−−GP15
C_m−−GP8
N1−−GP14
N5−−GP15
N5−−GP6
N3−−GP3
P5−−N1
N4−−GP15
P3−−N7
P7−−GP3
C_m−−N7
C_m−−N7
P7−−GP3
P3−−N7
N4−−GP15
P5−−N1
N3−−GP3
N5−−GP6
N5−−GP15
N1−−GP14
C_m−−GP8
GP10−−GP15
GP3−−GP12
C_m−−P3
P6−−N4
P4−−N1
P4−−N6
P4−−GP7
GP1−−GP11
P6−−GP15
P1−−GP16
P7−−GP10
P2−−GP6
N4−−GP8
GP6−−GP15
P2−−GP4
GP8−−GP13
C_m−−GP11
GP6−−GP12
P5−−N6
GP7−−GP8
GP2−−GP14
GP5−−GP12
N7−−GP3
GP6−−GP14
GP6−−GP7
P1−−P2
GP6−−GP10
N5−−GP7
P1−−GP6
C_m−−GP14
GP5−−GP7
P6−−GP1
GP3−−GP14
C_m−−P2
P3−−GP3
GP9−−GP12
P3−−GP13
P1−−N7
N3−−GP5
N1−−N3
P3−−GP16
GP4−−GP15
P2−−GP9
GP3−−GP11
P4−−GP15
GP7−−GP9
P3−−P6
GP5−−GP13
P6−−GP11
GP1−−GP9
P3−−GP5
N5−−GP10
GP1−−GP4
P5−−P6
P1−−GP15
P6−−GP12
P4−−N7
C_m−−P5
GP10−−GP11
GP7−−GP12
N4−−GP12
N4−−GP6
P2−−GP14
GP4−−GP16
GP1−−GP8
P4−−GP8
GP8−−GP16
GP1−−GP15
P4−−P5
GP12−−GP14
P1−−N4
N1−−N4
P6−−GP3
P1−−P3
GP9−−GP15
GP4−−GP13
P3−−GP11
N2−−GP13
N6−−GP8
N3−−GP13
GP7−−GP11
GP13−−GP15
N5−−GP12
N7−−GP12
P6−−GP2
P1−−P5
P4−−GP11
GP7−−GP10
P2−−GP2
GP11−−GP13
GP11−−GP12
P2−−GP10
P4−−GP4
N1−−GP3
GP8−−GP12
P2−−P4
N1−−GP5
N5−−N7
GP14−−GP15
N5−−GP11
C_m−−GP6
P2−−N7
GP11−−GP15
P5−−N7
P2−−N5
P4−−P7
P5−−GP12
GP8−−GP14
GP2−−GP3
N7−−GP5
GP1−−GP2
N3−−GP8
P5−−GP9
P2−−GP11
GP15−−GP16
N1−−N6
N5−−N6
GP3−−GP6
C_m−−GP12
N3−−N7
P4−−GP14
N1−−N2
P6−−GP16
P2−−GP13
P7−−GP14
N6−−GP7
N2−−N3
P3−−GP9
GP2−−GP4
N7−−GP15
N1−−GP7
GP2−−GP6
P7−−GP8
P1−−P6
N4−−GP16
N3−−N6
N2−−N4
P1−−GP9
Figure C6 Bootstrapped difference test for edge weights for the intensity of cannabis use in the
past 12 months and symptomatology network. The significance difference testing (α=.05) examines
whether edges significantly differ from each other in strength. The color of the boxes indicates wheth-
er there is a significant difference (i.e., grey boxes reflect no significant differences and black boxes
reflect significant differences).
Page | 314
edge
P1−−GP9
N2−−N4
N3−−N6
N4−−GP16
P7−−GP8
P1−−P6
GP2−−GP6
N1−−GP7
N7−−GP15
GP2−−GP4
P3−−GP9
N2−−N3
P2−−GP13
N6−−GP7
P7−−GP14
P6−−GP16
N1−−N2
N3−−N7
P4−−GP14
GP15−−GP16
N1−−N6
GP3−−GP6
N5−−N6
GP2−−GP3
GP8−−GP14
C_y−−GP12
GP1−−GP2
P2−−GP11
P5−−GP12
N3−−GP8
P5−−N7
GP11−−GP15
P2−−N5
N5−−N7
P5−−GP9
N5−−GP11
P4−−P7
C_y−−GP6
N7−−GP5
P1−−P5
P2−−P4
GP14−−GP15
GP8−−GP12
GP11−−GP13
P2−−N7
N1−−GP5
P6−−GP2
GP9−−GP15
N1−−GP3
GP11−−GP12
P4−−GP4
P6−−GP3
GP12−−GP14
N2−−GP13
P4−−GP11
N5−−GP12
P2−−GP10
GP7−−GP11
C_y−−P6
P3−−GP11
P2−−GP2
P1−−P3
N4−−GP12
GP7−−GP10
GP1−−GP15
P4−−P5
GP4−−GP13
GP10−−GP11
N3−−GP13
P4−−GP8
GP13−−GP15
C_y−−P2
N5−−GP10
N4−−GP6
GP4−−GP16
N7−−GP12
N4−−GP7
P1−−N4
P6−−GP11
N6−−GP8
GP7−−GP12
P2−−GP14
GP1−−GP4
GP8−−GP16
P4−−N7
C_y−−P5
C_y−−GP14
N3−−GP5
P4−−GP15
GP7−−GP9
P6−−GP12
GP4−−GP15
P3−−GP13
P3−−P6
P5−−P6
P3−−GP5
GP5−−GP13
N5−−GP5
GP9−−GP12
P2−−GP9
N1−−N4
GP6−−GP7
P7−−GP12
GP1−−GP9
N1−−N3
N2−−GP15
N5−−GP7
GP5−−GP7
P1−−GP15
P1−−N7
P3−−GP2
GP3−−GP11
P1−−P2
P3−−GP3
P2−−P5
GP6−−GP10
GP7−−GP8
N3−−N5
P7−−GP15
N5−−GP13
P5−−N6
N4−−GP8
GP1−−GP11
C_y−−GP13
P7−−GP10
GP8−−GP15
GP6−−GP12
P4−−N6
C_y−−N7
GP6−−GP15
P6−−GP15
P4−−N1
P2−−GP6
P4−−GP7
P2−−GP4
P5−−N1
C_y−−GP7
GP10−−GP15
N1−−GP14
GP3−−GP12
P6−−N4
C_y−−GP5
N3−−GP3
N5−−GP6
N5−−GP15
P3−−N7
P7−−GP3
N4−−GP15
C_y−−GP2
C_y−−GP2
N4−−GP15
P7−−GP3
P3−−N7
N5−−GP15
N5−−GP6
N3−−GP3
C_y−−GP5
P6−−N4
GP3−−GP12
N1−−GP14
GP10−−GP15
C_y−−GP7
P5−−N1
P2−−GP4
P4−−GP7
P2−−GP6
P4−−N1
P6−−GP15
GP6−−GP15
C_y−−N7
P4−−N6
GP6−−GP12
GP8−−GP15
P7−−GP10
C_y−−GP13
GP1−−GP11
N4−−GP8
P5−−N6
N5−−GP13
P7−−GP15
N3−−N5
GP7−−GP8
GP6−−GP10
P2−−P5
P3−−GP3
P1−−P2
GP3−−GP11
P3−−GP2
P1−−N7
P1−−GP15
GP5−−GP7
N5−−GP7
N2−−GP15
N1−−N3
GP1−−GP9
P7−−GP12
GP6−−GP7
N1−−N4
P2−−GP9
GP9−−GP12
N5−−GP5
GP5−−GP13
P3−−GP5
P5−−P6
P3−−P6
P3−−GP13
GP4−−GP15
P6−−GP12
GP7−−GP9
P4−−GP15
N3−−GP5
C_y−−GP14
C_y−−P5
P4−−N7
GP8−−GP16
GP1−−GP4
P2−−GP14
GP7−−GP12
N6−−GP8
P6−−GP11
P1−−N4
N4−−GP7
N7−−GP12
GP4−−GP16
N4−−GP6
N5−−GP10
C_y−−P2
GP13−−GP15
P4−−GP8
N3−−GP13
GP10−−GP11
GP4−−GP13
P4−−P5
GP1−−GP15
GP7−−GP10
N4−−GP12
P1−−P3
P2−−GP2
P3−−GP11
C_y−−P6
GP7−−GP11
P2−−GP10
N5−−GP12
P4−−GP11
N2−−GP13
GP12−−GP14
P6−−GP3
P4−−GP4
GP11−−GP12
N1−−GP3
GP9−−GP15
P6−−GP2
N1−−GP5
P2−−N7
GP11−−GP13
GP8−−GP12
GP14−−GP15
P2−−P4
P1−−P5
N7−−GP5
C_y−−GP6
P4−−P7
N5−−GP11
P5−−GP9
N5−−N7
P2−−N5
GP11−−GP15
P5−−N7
N3−−GP8
P5−−GP12
P2−−GP11
GP1−−GP2
C_y−−GP12
GP8−−GP14
GP2−−GP3
N5−−N6
GP3−−GP6
N1−−N6
GP15−−GP16
P4−−GP14
N3−−N7
N1−−N2
P6−−GP16
P7−−GP14
N6−−GP7
P2−−GP13
N2−−N3
P3−−GP9
GP2−−GP4
N7−−GP15
N1−−GP7
GP2−−GP6
P1−−P6
P7−−GP8
N4−−GP16
N3−−N6
N2−−N4
P1−−GP9
Figure C7 Bootstrapped difference test for edge weights for the intensity of cannabis use in the past
3 years and symptomatology network. The significance difference testing (α=.05) examines whether
edges significantly differ from each other in strength. The color of the boxes indicates whether there
is a significant difference (i.e., grey boxes reflect no significant differences and black boxes reflect
significant differences).
Page | 315
D
Appendix D1. Analysis of network stability

While the employed methodology does not allow for the investigation of the
network stability using traditional techniques (Epskamp, Borsboom, et al.,
2017), the method in itself could be thought of being based on bootstrapping.
Nonetheless, to further ensure this, we carried out an additional stability analysis
using nonparametric bootstrap techniques, as described by Epskamp and colleagues
(2017), but designed for the method described above and not yet implemented
in the bootnet R package (Epskamp, Borsboom, et al., 2017). Specifically, we re-
sampled the observations in the data with replacement, over 1000 iterations and
drew bootstrapped intervals of the edge weights. Due to the estimation method, we
were required to perform the bootstrap analyses on a high-performance computer
cluster, parallelized over 100 multicore units, each running 10 bootstrap samples.
Figure D1 on the next page presents the results of the nonparametric bootstrap.
Page | 318
edge
Depression−−Suicidality
Somatic Concern−−Anxiety
Unusual Thought Content−−Bizarre Behavior
Anxiety−−Depression
Hallucinations−−Unusual Thought Content
Depression−−Guilt
Suspiciousness−−Unusual Thought Content
Depression−−Hallucinations
Suspiciousness−−Hallucinations
Suicidality−−Guilt
Suicidality−−Hallucinations
Bizarre Behavior−−Transition To Clinical Psychosis
Hallucinations−−Affective Speech Illusions
Grandiosity−−Unusual Thought Content
Suicidality−−Grandiosity
Anxiety−−Guilt
Somatic Concern−−Bizarre Behavior
Suicidality−−Bizarre Behavior
Depression−−Suspiciousness
Anxiety−−Suspiciousness
Affective Speech Illusions−−Transition To Clinical Psychosis
Grandiosity−−Transition To Clinical Psychosis
Guilt−−Affective Speech Illusions
Bizarre Behavior−−Affective Speech Illusions
Unusual Thought Content−−Transition To Clinical Psychosis
Grandiosity−−Bizarre Behavior
Depression−−Transition To Clinical Psychosis
Suspiciousness−−Transition To Clinical Psychosis
Anxiety−−Unusual Thought Content
Grandiosity−−Suspiciousness
Guilt−−Bizarre Behavior
Guilt−−Unusual Thought Content
Guilt−−Suspiciousness
Anxiety−−Hallucinations
Somatic Concern−−Suspiciousness
Suicidality−−Unusual Thought Content
Guilt−−Transition To Clinical Psychosis
Anxiety−−Transition To Clinical Psychosis
Somatic Concern−−Suicidality
Somatic Concern−−Transition To Clinical Psychosis
Suicidality−−Transition To Clinical Psychosis
Guilt−−Grandiosity
Somatic Concern−−Guilt
Anxiety−−Bizarre Behavior
Somatic Concern−−Depression
Somatic Concern−−Grandiosity
Suicidality−−Affective Speech Illusions
Depression−−Grandiosity
Somatic Concern−−Hallucinations
Anxiety−−Suicidality
Depression−−Bizarre Behavior
Depression−−Unusual Thought Content
Somatic Concern−−Unusual Thought Content
Somatic Concern−−Affective Speech Illusions
Suicidality−−Suspiciousness
Suspiciousness−−Affective Speech Illusions
Grandiosity−−Affective Speech Illusions
Hallucinations−−Transition To Clinical Psychosis
Suspiciousness−−Bizarre Behavior
Hallucinations−−Bizarre Behavior
Anxiety−−Grandiosity
Anxiety−−Affective Speech Illusions
Depression−−Affective Speech Illusions
Unusual Thought Content−−Affective Speech Illusions
Guilt−−Hallucinations
Grandiosity−−Hallucinations
−0.2 0.0 0.2 0.4 0.6
Figure D1 Bootstrapped edge-weights for the network structure of affective speech illusions, posi-
tive, affective and cognitive symptoms and transition to clinical psychosis. The red line depicts point
estimates of the edge weights, the grey bar 95% bootstrapped confidence intervals.
Page | 319
Appendix D2. Raw and standardized centrality values

Tables D1 and D2 below include all raw and standardized centrality values for
both the affective speech illusions network and the all speech illusions network (i.e.,
the main analyses carried out in the current chapter). Figure D2 below displays the
centrality plot of betweenness and closeness, for both the affective speech illusions
and all speech illusions networks, ordered by betweenness values.
Table D1 Raw and standardized strength centrality values

Standardized Raw Standardized
Raw (affective
Node Name (affective (all speech (all speech
speech illusions)
speech illusions) illusions) illusions)
Depression 1.13 1.86 1.12 1.91
Unusual Thought Content 0.82 0.93 0.81 0.96
Hallucinations 0.87 1.08 0.75 0.77
Suicidality 0.72 0.6 0.71 0.63
Anxiety 0.66 0.42 0.66 0.49
Bizarre Behavior 0.5 -0.08 0.48 -0.05
Guilt 0.45 -0.23 0.44 -0.18
Suspiciousness 0.39 -0.42 0.39 -0.35
Somatic Concern 0.36 -0.52 0.35 -0.47
Grandiosity 0.2 -1 0.19 -0.96
Transition to Clinical
0.11 -1.27 0.11 -1.22
Psychosis
Speech Illusions 0.08 -1.36 0 -1.55
Page | 320
Betweenness Closeness
Depression ● ● ●
Hallucinations ● ● ● ●
Unusual Thought Content ●● ●●
Bizarre Behavior ●● ●●
Anxiety ●● ●●
Suicidality ●● ●●
type
● Affective Speech Illusions
● All Speech Illusions
Transition to Clinical Psychosis ●
● ● ●
Suspiciousness ●
● ● ●
Speech Illusions ●
● ●
Somatic Concern ●
● ● ●
Guilt ●
● ●●
Grandiosity ●
● ● ●
0 1 2 −1 0 1
Figure D2 Centrality Plot displaying the betweenness and closeness centrality for both the affective
speech illusions and all speech illusions networks (Galdos operationalization; Galdos et al., 2011).
Centrality measures are shown as standardized z-scores and are ordered by betweenness. Raw and
standardized centrality scores are included in Table D2 below.
Page | 321
Table D2 Raw and standardized betweenness and closeness centrality values
Standardized Standardized Raw Standardized Standardized
Raw betweenness Raw closeness Raw closeness
betweenness closeness betweenness betweenness closeness
Page | 322
Node Name (affective (affective (all speech
(affective (affective (all speech (all speech (all speech
speech illusions) speech illusions) illusions)
speech illusions) speech illusions) illusions) illusions) illusions)
Depression 64 1.9 0.009 1.2 56 2.12 0.01 1.2
Unusual Thought
Content 42 0.99 0.008 1.13 36 1.1 0.01 1.06

Hallucinations 60 1.73 0.009 1.58 40 1.3 0.01 1.4
Suicidality 8 -0.4 0.007 0.44 8 -0.34 0.008 0.39
Anxiety 20 0.09 0.007 0.2 18 0.17 0.008 0.13
Bizarre Behavior 20 0.09 0.007 0.21 18 0.17 0.008 0.1
Guilt 0 -0.73 0.006 -0.18 0 -0.75 0.007 -0.3
Suspiciousness 0 -0.73 0.006 -0.07 0 -0.75 0.007 -0.23
Somatic Concern 0 -0.73 0.006 -0.55 0 -0.75 0.006 -0.68
Grandiosity 0 -0.73 0.004 -1.27 0 -0.75 0.005 -1.46
Transition to
Clinical Psychosis 0 -0.73 0.004 -1.37 0 -0.75 0.005 -1.6
Speech Illusions 0 -0.73 0.004 -1.31 0 -0.75 NA NA
N.B.: Given that the speech illusion node is disconnected, the raw centrality values calculated for closeness are 0. We have therefore calculated and presented here the raw
closeness values of the giant component (i.e., the network structure without the speech illusions node), based on the functionality described by Costantini et al. (2015).
Appendix D3. Extended network structure of affective speech illusions, posi-
tive, affective and cognitive symptoms and transition to clinical psychosis
Figure D3 below displays the network structure of associations between affective
speech illusions, positive, affective, and cognitive symptoms and transition to clinical
psychosis, as well as the associations between the symptoms themselves, when
conditioning on all other variables in the network structure. The estimation
procedure is the same as described in the Methods section of Chapter 7. One
association was identified between affective speech illusions and mannerism and
posturing. Cognitive symptoms were found to be mostly associated with bizarre
behavior, suspiciousness, and each other, apart from (weak) associations with guilt,
suicidality and depression. The only item in the network structure associated with
later transition to clinical psychosis was bizarre behavior. An elaboration on these
results can be found here, also in comparison to the results presented in Chapter 7.
The items with the highest centrality were depression, bizarre behavior, unusual
thought content, suicidality, and anxiety (see Figure D4 below). In line with the
main analysis of Chapter 7, speech illusions and transition to clinical psychosis
displayed lowest predictability, while the items with highest predictability were
(in increasing order) hallucinations (26%), unusual thought content (30%), anxiety
(32%), suicidality (37%), and depression (48%). The addition of cognitive symptoms
did not increase the predictability with more than 1% for any of the items.
15
Affective Symptoms
16 1: Somatic Concern
14
2: Anxiety
3: Depression
5 4: Suicidality
5: Guilt
Positive Symptoms
12 6: Grandiosity
4 7: Suspiciousness
8: Hallucinations
9: Unusual Thought Content
3
10 10: Bizarre Behavior
2 Cognitive Symptoms
11: Self−neglect
11 12: Disorientation
13: Conceptual Disorganization
14: Mannerisms and Posturing
9 8
1 Speech Illusions
13 15: Affective Speech Illusions
Transition
16: Transition To Clinical Psychosis
Figure D3 Network of affective speech illusions, positive, affective and cognitive symptoms and transition to
clinical psychosis. Symptom groups are differentiated by colors.
Page | 323
Strength
Depression ●
Bizarre Behavior ●
Unusual Thought Content ●
Suicidality ●
Anxiety ●
Hallucinations ●
Self−neglect ●
Conceptual Disorganization ●
Suspiciousness ●
Guilt ●
Mannerisms and Posturing ●
Disorientation ●
Somatic Concern ●
Affective Speech Illusions ●
Grandiosity ●
Transition To Clinical Psychosis ●
−1 0 1 2
Figure D4 Centrality Plot displaying strength centrality for the network of affective speech illusions,
positive, affective and cognitive symptoms and transition to clinical psychosis. Centrality measures
are shown as standardized z-scores in the plot and are ordered by strength. Raw and standardized
centrality scores are included in Table D3 below.
Table D3 Raw and standardized strength centrality values for the extended network structure of affective
speech illusions, positive, affective and cognitive symptoms and transition to clinical psychosis
Node Name Raw Standardized
Depression 1.19 2.24
Bizarre Behavior 0.85 1.04
Unusual Thought Content 0.79 0.84
Suicidality 0.78 0.83
Anxiety 0.67 0.44
Hallucinations 0.63 0.32
Page | 324
Table D3Continued.
Node Name Raw Standardized

Self-neglect 0.59 0.17
Conceptual Disorganization 0.58 0.11
Suspiciousness 0.52 -0.05
Guilt 0.49 -0.20
Mannerism and Posturing 0.47 -0.25
Disorientation 0.41 -0.46
Somatic Concern 0.35 -0.68
Affective Speech Illusions 0.15 -1.37
Grandiosity 0.12 -1.49
Transition to Clinical Psychosis 0.11 -1.51
Appendix D4. Edge variability across the estimation of 1000 networks, using
different seeds for the lambda parameter
As the current analysis focuses on between-domain links, the estimation approach
used aimed to ensure a high specificity. The two network structures were estimated
1000 times, using different seeds for the lambda parameter, and only the edges that
were non-zero in 95% of the cases were retained. Figure D5 below displays how often
(in %) each edge in the network was identified across the 1000 estimations. For
visualization purposes, all edges identified in less than 50% of the cases are hidden.
Figure D5, panel A displays these results for the main network presented in the
current chapter.1 The edge between affective speech illusions and hallucinations was
identified in 98% of the estimated networks, while the edge between transition
to clinical psychosis and bizarre behavior was identified in 98.4% of the networks.
One other edge consistently identified, even though not to a degree that passed
our pre-defined threshold, is the edge between transition to clinical psychosis and
grandiosity (79.9%). Most other edges were identified in almost 100% of the cases
and are thus also included and discussed in the main results described in Chapter 7.
Figure D5, panel B displays these results for the secondary analysis, which
further included cognitive symptoms in the network structure. Similar to the
previous network structure, transition to clinical psychosis was associated to bizarre
behavior in 100% of the network estimations. Other consistently identified edges
1 We focus on the network of affective speech illusions, as this is almost identical to the network of
all speech illusions, with the addition of the hallucinations edge, which is of high interest here.
Page | 325
were between affective speech illusions and hallucinations (77.9%), between affective
speech illusions and mannerism and posturing (95.3%), and between transition to
clinical psychosis and grandiosity (59.1%). Not all edges passing the threshold may
be due to the increased number of nodes in relation to the relatively low sample
size. Compared to the main network which only focused of positive and affective
symptoms, within the more extended network structure more associations below
the 95% threshold emerged, including several negative associations, which may be
indicative of collider structures or false-positive effects.
The main results of both analyses are generally well aligned, and controlling for
cognitive symptoms does not strongly affect the associations between affective speech
illusions and other nodes. However, adding more nodes appears to have resulted in
reduced power and a likely more unstable network structure (i.e., more variability in
edge weights), due to a small sample size in relation to the higher number of nodes.
Thus, we advise caution when interpreting these results and recommend replication
in larger samples of both network structures.
Panel A Panel B
5 15
100% 95.3%
100%
100% 16
2 14
100%
100% 100% 5
3 100% 52.8%
100% 100% 100%
4 100%
12 98.4% 1 100%
12 100% 77.9% 100%
4
100% 100%
100% 100% 98.5%
93% 3
98.4% 100% 10 91%
100% 98.6% 100% 58.5%
92.8% 100%
100% 100%
59.1%
100% 60.3% 89.1% 2
100% 100% 78% 100%
100% 100% 11 100% 97.2%100%
79.9% 8 80.8% 100% 100%
10 98%
100% 9 100%
59.9% 100% 899%
100% 11
100% 1
98.8% 99.9%
13
9 85.3%100% 100% 100%
100% 100% 86.1%
100%
100%
6 7
7 6
Minimum: 50 Maximum: 100
Figure D5 Network visualization of edge variability for Panel A. the network of affective speech
illusions, positive and affective symptoms and transition to psychotic disorder; Panel B. the network
of affective speech illusions, positive, affective, and cognitive symptoms and transition to psychotic
disorder. All edges identified across less than 50% of the estimated network structures are not includ-
ed, for visualization purposes.
Page | 326
Appendix D5. Network structure of all speech illusions (Catalan operational-
ization; Catalan et al., 2014), positive and affective symptoms, and transition
to clinical psychosis
Figure D6 below displays the network structure of associations between all speech
illusions (operationalized according to Catalan; Catalan et al., 2014), positive and
affective symptoms, and transition to clinical psychosis, as well as the associations
between the symptoms themselves, when conditioning on all other variables in
the network structure. The estimation procedure is the same as described in the
Methods section of Chapter 7. The results are generally well-aligned with the main
results presented in the chapter. However, when using the operationalization
of Catalan and colleagues to construct the speech illusions variable, a negative
association emerges between speech illusions and grandiosity. This association is not
present when using the operationalization of Galdos and colleagues (Galdos et al.,
2011), indicating that the way the variable is operationalized may lead to different
results and network structures. This association could either be a real effect or a
false positive effect; of note, the original correlation between the two variables was
also negative, indicating that there was no sign switch when computing the network
structure. Further research should aim to replicate these results.
Affective Symptoms
3
1: Somatic Concern
4 2: Anxiety
1 3: Depression
12 4: Suicidality
5: Guilt
Positive Symptoms
6: Grandiosity
7: Suspiciousness
8: Hallucinations
9: Unusual Thought Content
10: Bizarre Behavior
8 Speech Illusions
11: Speech Illusions
10
11 Transition
12: Transition to Clinical Psychosis
7
Maximum: 0.45
Figure D6 Network of all speech illusions (Catalan operationalization; Catalan et al., 2014), positive,
affective and cognitive symptoms and transition to clinical psychosis. Symptom groups are differenti-
ated by colors.
Page | 327
E
Appendix E1. Accuracy and stability check for the estimated network model
In order to check whether the estimated network connections and centrality
measure were accurate, we carried out bootstrap stability checks, using the
R-package bootnet (Epskamp, Borsboom, et al., 2017). Figure E1 below shows
bootstrap results for edge-weights between PRS and all other nodes, while Figure
E2 shows the average case-drop bootstraps of node-specific betweenness. An
extensive interpretation of each result is included in the respective figure captions.
C42−−PRS 0.99
●
C41−−PRS 0.33 ●
C40−−PRS 0.99
●
C39−−PRS 0.96
●
C38−−PRS 0.86
●
C37−−PRS 0.96
●
C36−−PRS 0.69
●
C35−−PRS 0.98
●
C34−−PRS 0.99
●
C33−−PRS 0.91
●
C32−−PRS 0.84
●
C31−−PRS 0.96
●
C30−−PRS ● 0.25
C29−−PRS 0.99
●
C28−−PRS 0.80
●
C27−−PRS 0.87
●
C26−−PRS 0.98
●
C25−−PRS 0.94
●
C24−−PRS 0.90
●
C23−−PRS 0.94
●
C22−−PRS 0.96
●
C21−−PRS 0.74
●
C20−−PRS 0.98
●
C19−−PRS 0.91
●
C18−−PRS 0.78
●
C17−−PRS 0.86
●
C16−−PRS 0.96
●
C15−−PRS 0.96
●
C14−−PRS 0.88
●
C13−−PRS 0.97
●
C12−−PRS 0.63 ●
C11−−PRS 0.84
●
C10−−PRS 0.48 ●
C9−−PRS 0.70
●
C8−−PRS 0.95
●
C7−−PRS 0.78
●
C6−−PRS 0.98
●
C5−−PRS 0.88
●
C4−−PRS 0.85
●
C3−−PRS 0.96
●
C2−−PRS 0.97
●
C1−−PRS 0.91
●
−0.10 −0.05 0.00 0.05 0.10
Figure E1 Bootstrap results for edge-weights between PRS and all other nodes, based on 1000
nonparametric bootstrap samples. Each horizonal line within the plot represents the 95% quantile
range of the parameter values when the parameter was included in the model. The red dots indicate
the sample values for the analyzed data, while the grey dots indicate the bootstrap mean values. The
values displayed in the center of the plot show the percentage of bootstraps in which an edge was zero;
in addition, the more faded the line, the higher the percentage of bootstraps in which an edge was
zero. The sample values lie within the bootstrapped confidence intervals and the bootstrap confidence
intervals are relatively small, thus indicating accurate estimations. The edges from PRS to C42 and
C30 are the most robust (i.e., showing up in 67% of the bootstraps and 75% of the bootstraps respec-
tively). Notably, all edges are identified in at least 50% of the bootstraps.
Page | 330
20
15
C10
10
mean
C7
C41
5
C12
C14
C5
C28
C31
C33
C1
C2
C32
C9
C21
C42
C17
C24
C39
C29
C11
C27
C26
C36
C35
C18
C8
C30
C20
C3
C40
C13
C25
C23
C38
C15
C22
C4
C34
C16
C37
C6
C19
0 PRS
0.8 0.6 0.4 0.2

prop
Figure E2 Average case-drop bootstraps of node-specific betweenness, based on 1000 iterations. The lines represent how node-specific betweenness (i.e., how
often a node lies on the pathways between two other nodes, of which one is always the PRS) changes for each variable when dropping different proportions of the
data. Overall, the stability is not very reliable, but nodes C10 and C12 retain relatively high node-specific betweenness across case-drops.
Page | 331
EBIC Glasso Unregularized Model Search Partial Correlations (.01)
0.05
Edge Weight
0.00
−0.05
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Connected node
Figure E3 Comparison of edge presence and strength across different estimation techniques. The
lines indicate the presence and strength of the edges identified between the PRS and other nodes in
the network when using different estimation techniques (i.e., the EBICglasso with a tuning param-
eter of .5, unregularized method search, and partial correlations thresholded at alpha .01). The edges
between PRS and nodes C30 (Thought echo) and C42 (Visual hallucinations) are identified by all
estimation methods, while the edges between PRS and nodes C10 (Conspiracy) and C12 (No future)
by two out of three estimation methods.
Table E1 Fit comparison across different estimation methods (Epskamp, Rhemtulla, et al., 2017)
Node Name AIC BIC EBIC RMSEA CFI
Unregularized model search 231618.6 233811.0 236178.6 .015 .990
EBICglasso 231845.0 234216.5 240627.6 .018 .989
Partial correlations α .01 232935.2 233811.0 236178.6 .032 .951
Page | 332
F
Appendix F1. Accuracy and stability checks

Using the R-package bootent version 1.3. (Epskamp & Fried, 2018) we performed
several checks in order to investigate whether the estimated network associations
and centrality measures were stable and accurate. These checks allowed us to
estimate the accuracy of edge weights, as well as investigate the stability of centrality
measures. First, to investigate the stability of the edge weights, participants were
randomly resampled 1000 times, and the bootstrapped confidence intervals (CIs)
of the edge weights were estimated. Second, to investigate whether the strength
centrality was interpretable, we used case-drop bootstrapping, re-estimating the
network with fewer cases. To quantify the stability of strength centrality indices,
we used a correlation stability coefficient (CS), as recommended and described
by Epskamp and colleagues (2018). Because betweenness is generally not a very
stable centrality measure, we used both nonparametric and case-drop bootstraps
to investigate the extent of variability of the node-specific predictive betweenness
variability, as done in previous research (Isvoranu et al., 2019; see Chapter 8).
Further, we programmed an R function, based on the nonparametric bootstrap
procedure described above, to allow investigating the stability of the shortest
pathways. Figures F1, F2, and F3 showcase the bootstrap results, based on 1000
iterations, alongside the interpretation of these results. For further details on the
procedure, please refer to Epskamp and colleagues (2018).
Page | 334
edge
GP1−−GP2
P1−−P2
GP1−−N1
N1−−N2
B2−−B3
P4−−N3
GP4−−B1
P4−−C2
N2−−B3
C2−−E2
GP1−−N2
GP5−−GP8
P1−−P3
GP2−−GP4
N3−−C2
GP8−−C1
GP5−−GP6
N1−−B2
P4−−C1
GP7−−E1
C1−−E1
N3−−C1
N1−−B3
N2−−B2
P3−−B1
E1−−E2
P2−−P4
GP6−−E2
GP8−−N2
GP7−−C1
GP7−−P1
P2−−P3
GP5−−E1
N3−−E1
GP5−−B1
E1−−B1
GP2−−B1
N3−−E2
GP1−−GP5
C1−−B2
GP5−−GP7
GP8−−B2
N2−−C1
GP1−−E1
GP1−−B3
GP2−−P3
E1−−B2
GP6−−B2
GP4−−P1
GP4−−GP7
GP4−−P2
E2−−B2
GP1−−C1
GP8−−P3
N1−−C1
N1−−B1
GP1−−B1
GP6−−C2
GP4−−C1
GP2−−GP7
P3−−P4
GP4−−P4
GP6−−GP8
GP5−−P1
GP5−−B2
P3−−C1
N1−−E1
GP5−−N1
GP1−−GP8
P1−−B3
N3−−B2
P2−−E2
GP1−−P3
GP6−−B1
B1−−B3
GP8−−B3
GP5−−P2
GP5−−B3
GP6−−E1
GP1−−P2
GP2−−N2
GP7−−P3
GP5−−P3
P4−−B3
P2−−E1
N2−−E1
P2−−C1
N3−−B1
GP2−−GP8
GP4−−P3
GP8−−N3
GP7−−P4
P4−−E2
N2−−B1
GP2−−B3
GP2−−GP5
GP1−−P1
P2−−B2
GP6−−P2
GP5−−C1
C2−−B2
P2−−N1
GP4−−GP8
N2−−E2
C1−−C2
N1−−N3
GP2−−N1
P1−−B2
GP5−−P4
GP8−−P2
GP2−−P2
GP1−−GP4
GP4−−GP5
P1−−N1
GP6−−GP7
GP4−−N3
GP7−−N3
P1−−E1
P1−−P4
GP4−−C2
GP6−−P3
GP5−−N2
B1−−B2
C2−−B1
GP8−−N1
GP2−−C1
C1−−B1
GP5−−E2
GP8−−E1
GP7−−C2
GP6−−N3
GP1−−B2
GP4−−GP6
E1−−B3
P2−−B1
GP2−−B2
GP8−−P4
GP7−−GP8
P2−−B3
N2−−N3
GP6−−P4
GP2−−P1
P4−−E1
GP8−−P1
GP6−−P1
GP7−−N2
P1−−C1
P4−−B2
N1−−E2
C1−−B3
P4−−B1
P2−−N2
C2−−B3
C2−−E1
GP6−−C1
GP4−−E1
N3−−B3
GP4−−B2
P3−−C2
E2−−B3
P3−−B3
GP2−−GP6
GP1−−GP7
P2−−C2
GP7−−P2
GP1−−P4
GP7−−N1
GP8−−C2
GP7−−E2
GP4−−N2
P1−−N2
P1−−C2
P3−−N2
GP2−−P4
GP7−−B2
P3−−E1
GP5−−C2
GP6−−N2
P1−−N3
GP2−−E1
E2−−B1
P3−−B2
P3−−N3
GP1−−N3
C1−−E2
GP1−−GP6
P3−−N1
GP6−−N1
GP4−−N1
GP8−−B1
GP1−−E2
P1−−E2
GP5−−N3
P4−−N1
GP7−−B3
P4−−N2
P2−−N3
GP4−−B3
GP2−−N3
GP2−−E2
N2−−C2
P1−−B1
GP8−−E2
P3−−E2
GP6−−B3
GP2−−C2
GP1−−C2
GP4−−E2
N1−−C2
GP7−−B1
0.0 0.2 0.4
Figure F1 Accuracy of the edge-weights. The horizonal area within the plot represents the 95%
quantile range of the parameter values across 1000 bootstraps. The red dots indicate the sample values
for the analyzed data, while the black dots indicate the bootstrap mean values. The sample values
lie within the bootstrapped confidence intervals and the bootstrap mean values are generally well-
aligned with the sample values, overall indicating accurate estimations. Of note, the bootstrapped
confidence intervals are relatively wide, thus some caution is recommended especially when interpret-
ing the presence and strength of weaker edges.
Page | 335
edge
GP1−−GP2
P1−−P2
GP1−−N1
N1−−N2
B2−−B3
P4−−N3
GP4−−B1
P4−−C2
N2−−B3
C2−−E2
GP1−−N2
GP5−−GP8
P1−−P3
GP2−−GP4
N3−−C2
GP8−−C1
GP5−−GP6
N1−−B2
P4−−C1
GP7−−E1
C1−−E1
N3−−C1
N1−−B3
N2−−B2
P3−−B1
E1−−E2
P2−−P4
GP6−−E2
GP8−−N2
GP7−−C1
GP7−−P1
P2−−P3
GP5−−E1
N3−−E1
GP5−−B1
E1−−B1
GP2−−B1
N3−−E2
GP1−−GP5
C1−−B2
GP5−−GP7
GP8−−B2
N2−−C1
GP1−−E1
GP1−−B3
GP2−−P3
E1−−B2
GP6−−B2
GP4−−P1
GP4−−GP7
GP4−−P2
E2−−B2
GP1−−C1
GP8−−P3
N1−−C1
N1−−B1
GP1−−B1
GP6−−C2
GP4−−C1
GP2−−GP7
P3−−P4
GP4−−P4
GP6−−GP8
GP5−−P1
GP5−−B2
P3−−C1
N1−−E1
GP5−−N1
GP1−−GP8
P1−−B3
N3−−B2
P2−−E2
GP1−−P3
GP6−−B1
B1−−B3
GP8−−B3
GP5−−P2
GP5−−B3
GP6−−E1
GP1−−P2
GP2−−N2
GP7−−P3
GP5−−P3
P4−−B3
P2−−E1
N2−−E1
P2−−C1
N3−−B1
GP2−−GP8
GP4−−P3
GP8−−N3
GP7−−P4
P4−−E2
N2−−B1
GP2−−B3
GP2−−GP5
GP1−−P1
P2−−B2
GP6−−P2
GP5−−C1
C2−−B2
P2−−N1
GP4−−GP8
N2−−E2
C1−−C2
N1−−N3
GP2−−N1
P1−−B2
GP5−−P4
GP8−−P2
GP2−−P2
GP1−−GP4
GP4−−GP5
P1−−N1
GP7−−B1
GP7−−B1
P1−−N1
GP4−−GP5
GP1−−GP4
GP2−−P2
GP8−−P2
GP5−−P4
P1−−B2
GP2−−N1
N1−−N3
C1−−C2
N2−−E2
GP4−−GP8
P2−−N1
C2−−B2
GP5−−C1
GP6−−P2
P2−−B2
GP1−−P1
GP2−−GP5
GP2−−B3
N2−−B1
P4−−E2
GP7−−P4
GP8−−N3
GP4−−P3
GP2−−GP8
N3−−B1
P2−−C1
N2−−E1
P2−−E1
P4−−B3
GP5−−P3
GP7−−P3
GP2−−N2
GP1−−P2
GP6−−E1
GP5−−B3
GP5−−P2
GP8−−B3
B1−−B3
GP6−−B1
GP1−−P3
P2−−E2
N3−−B2
P1−−B3
GP1−−GP8
GP5−−N1
N1−−E1
P3−−C1
GP5−−B2
GP5−−P1
GP6−−GP8
GP4−−P4
P3−−P4
GP2−−GP7
GP4−−C1
GP6−−C2
GP1−−B1
N1−−B1
N1−−C1
GP8−−P3
GP1−−C1
E2−−B2
GP4−−P2
GP4−−GP7
GP4−−P1
GP6−−B2
E1−−B2
GP2−−P3
GP1−−B3
GP1−−E1
N2−−C1
GP8−−B2
GP5−−GP7
C1−−B2
GP1−−GP5
N3−−E2
GP2−−B1
E1−−B1
GP5−−B1
N3−−E1
GP5−−E1
P2−−P3
GP7−−P1
GP7−−C1
GP8−−N2
GP6−−E2
P2−−P4
E1−−E2
P3−−B1
N2−−B2
N1−−B3
N3−−C1
C1−−E1
GP7−−E1
P4−−C1
N1−−B2
GP5−−GP6
GP8−−C1
N3−−C2
GP2−−GP4
P1−−P3
GP5−−GP8
GP1−−N2
C2−−E2
N2−−B3
P4−−C2
GP4−−B1
P4−−N3
B2−−B3
N1−−N2
GP1−−N1
P1−−P2
GP1−−GP2
Figure F2 Bootstrapped difference test for edge weights. The significance difference testing (α=.05)
examines whether edges significantly differ from each other in terms of strength. The color of the
boxes indicates whether there is a significant difference (i.e., grey boxes reflect no significant differenc-
es and black boxes reflect significant differences). Here most of the strongest edges generally signifi-
cantly differ from the other edges in the network. The largest edge strength difference was identified
between GP1 (depression) and GP2 (suicidality).
Page | 336
strength
GP1 1.30
N1 1.10
N2 1.00
C1 1.00
B2 0.98
GP5 0.92
E1 0.91
B3 0.85
P4 0.85
B1 0.84
GP2 0.80
N3 0.78
P2 0.77
P1 0.74
GP8 0.73
P3 0.69
GP4 0.68
C2 0.63
E2 0.61
GP7 0.60
GP6 0.48
GP6
GP7
E2
C2
GP4
P3
GP8
P1
P2
N3
GP2
B1
P4
B3
E1
GP5
B2
C1
N2
N1
GP1
Figure F3 Bootstrapped difference test for strength centrality. The significance difference testing
(α=.05) examines whether nodes significantly differ from each other in terms of strength central-
ity. The color of the boxes indicates whether there is a significant difference (i.e., grey boxes reflect
no significant differences and black boxes reflect significant differences). Here most nodes do not
significantly differ from each other in terms of strength centrality. Of note, the node with the highest
strength, GP1 (depression), showed significantly larger node strength than most of the other nodes in
the network.
Page | 337
12.5
Page | 338
10.0
Node
GP1
GP2
GP4
GP5
GP7
GP5
7.5 GP8
P1
P2
P3
P4
Mean
N1
N2
5.0
N3
C1
C2
E1
E2
B1
B2
2.5 E2
B3
B2
GP8
B1
C2
GP1
E1
N1
C1
P1
P4
P2
GP2
GP7
B3
P3
N2
GP4
N3
0.0
1.0 0.8 0.6 0.4 0.2

Proportion
Figure F4 Average case-drop bootstraps of node-specific betweenness, based on 1000 iterations. The lines represent how node-specific betweenness (i.e., how
often a node lies on the pathways between two other nodes, of which one is always the OCS) changes for each variable when dropping different proportions of
the data. Overall, the stability is not very reliable, but node GP5 retains very high node-specific betweenness across all case-drops.
Table F1 Mean and standard deviation of CAARMS* scores for subjects at Clinical High-Risk (N=341)
and controls (N=66)
CHR Controls
Item Label Item Description Mean SD Mean SD
P1 Unusual thought 2.73 1.83 0.23 0.86
P2 Non-bizarre ideas 2.85 1.73 0.33 0.87
P3 Perceptual abnormalities 2.89 1.66 0.47 1.14
P4 Disorganised speech 1.62 1.43 0.09 0.42
N1 Anhedonia 2.87 1.72 0.48 1.17
N2 Avolition 2.91 1.55 0.62 1.33
N3 Alogia 1.17 1.24 0.24 0.63
C1 Subjective cognitive change 2.35 1.18 0.42 0.86
C2 Observed cognitive change 0.78 1.01 0.02 0.13
E1 Emotional disturbance 1.91 1.54 0.36 0.93
E2 Observed blunted affect 0.93 1.22 0.03 0.25
B1 Aggression 2.16 1.61 0.36 0.84
B2 Social isolation 2.47 1.61 0.38 1.20
B3 Impaired role function 2.89 1.82 0.62 1.49
GP1 Depression 3.25 1.35 0.71 1.31
GP2 Suicidality 1.85 1.56 0.17 0.60
GP4 Mood swings 1.44 1.54 0.14 0.55
GP5 Anxiety 3.06 1.60 0.64 1.16
GP6 OCD symptoms 1.35 1.68 0.27 0.78
GP7 Dissociative symptoms 1.29 1.67 0.03 0.25
Impaired tolerance to normal
GP8 2.12 1.79 0.47 1.18
stress
*CAARMS denotes Comprehensive Assessment of At-Risk Mental States (Fusar-Poli, Hobson, et al., 2012)
Page | 339
GP2 GP1: Depression
Page | 340
GP4 GP2: Suicidality
GP4: Mood swings
GP5: Anxiety
GP7: Dissociative symptoms
GP1
P3 GP8: Impaired tolerance to normal stress
N1
B1 OCS
GP6: OCS symptoms
N2 P1 Positive Symptoms
P1: Unusual thought
P2: Non−bizarre ideas
GP8 P3: Perceptual abnormalities
B3 P4: Disorganised speech
GP7 P2
Negative Symptoms
C1 N1: Anhedonia
N2: Avolition
B2 GP5 N3: Alogia
Cognitive Symptoms
E1 C1: Subjective cognitive change
C2: Observed cognitive change
P4
Emotional Symptoms
E1: Emotional disturbance
N3
E2: Observed blunted affect
GP6
Behavioural Symptoms
B1: Aggression
B2: Social isolation
E2 C2 B3: Impaired role function
Figure F5 Network structure of 21 CAARMS symptoms, in the UHR sample only. Item groups are differentiated by colour. The colour of the edge indicates the
size of the association (blue for positive associations; red for negative associations).
Appendix F2. UHR-only network structure of 21 CAARMS symptoms
Figure F5 presents the network structure of the 21 CAARMS symptoms, in the
UHR sample only. The network shows comparable results, with the note that many
of the connections are weaker or missing, indicating a significant loss in power. The
edges in the UHR sample only network were mainly–and as expected–a subset of
the edges in the overall network structure.
Appendix F3. Extended CAARMS network structure

Figure F6 presents the full network structure of the CAARMS, including low
variability items (i.e., items with more than 75% response of 0: never): GP3 (mania),
E3 (observed inappropriate affect), B4 (disorganising, odd, stigmatising behaviour),
M1 (subjective motor change), M2 (observed motor change), M3 (impaired bodily
sensation), and M4 (impaired autonomic functioning). Within the main analysis
of the chapter, these items were excluded in an attempt to ensure more stable
network estimates and avoid potential biased results (e.g., due to severe violations
of normality assumption).
We therefore present the network structure below for reference only and
recommend caution when interpreting this, as well as replication of results in larger
samples. Of note and importance here, the same main associations between the
OCS item and other items were identified (i.e., OCS was strongly associated to
anxiety, observed blunted affect, and social isolation).
Page | 341
M4 B3
GP1: Depression
Page | 342
N2 GP2: Suicidality
N1 GP3: Mania
B2 GP4: Mood swings
GP5: Anxiety
M1 GP7: Dissociative symptoms
GP8: Impaired tolerance to normal stress
GP1 OCS
GP8 GP6: OCS symptoms
M2 GP2
Positive Symptoms
GP6 P1: Unusual thought
P2: Non−bizarre ideas
GP5 P3: Perceptual abnormalities
P4: Disorganised speech
C1 Negative Symptoms
B4 N1: Anhedonia
B1
N2: Avolition
GP4 N3: Alogia
E1 Cognitive Symptoms
C1: Subjective cognitive change
C2: Observed cognitive change
N3 Emotional Symptoms
E1: Emotional disturbance
E3 E2: Observed blunted affect
P3 GP3 E3: Observed inappropriate affect
P4 Behavioural Symptoms
B1: Aggression
C2 GP7 B2: Social isolation
B3: Impaired role function
B4: Disorganising, odd, stigmatising behaviour
E2 Motor Symptoms
P1 M1: Subjective motor change
P2 M2: Observed motor change
M3: Impaired bodily sensation
M4: Impaired autonomic functioning
M3
Figure F6 Extended network structure of 28 CAARMS symptoms. Item groups are differentiated by colour. The colour of the edge indicates the size of the
association (blue for positive associations; red for negative associations).
G
Appendix G1. Accuracy and stability checks

We used the R-package bootnet version 1.4.3 (Epskamp, Borsboom, et al., 2017),
following the procedure described by Epskamp and colleagues (2017). Specifically,
we investigated the accuracy of the edge weights using non-parametric bootstrapping
(i.e., re-estimating the network after resampling) and centrality measures using
case-drop bootstrapping (i.e., re-estimating the network with fewer cases). To
quantify the stability of strength and bridge strength centrality indices, we used a
correlation stability coefficient (CS-coefficient). The CS-coefficient represents the
maximum proportion of cases that can be dropped, such that with 95% probability
the correlation between the original centrality indices and centrality of networks
based on subsets is .7 or higher. Figures G1-G12 below present the results of the
bootstrap analyses. All results are based on 1000 iterations.
Generally, the results show high stability both for the FR and the PD network
structures. The sample values lie within the bootstrapped confidence intervals and
the bootstrap mean values are generally well-aligned with the sample values. The
bootstrapped confidence intervals are slightly wider for the FR network than for
the PD network, but overall, the results indicate accurate estimations. The CS-
coefficients obtained for strength were CS=.59 for the FR sample and CS=.67 for
the PD sample. The CS-coefficients obtained for bridge strength were CS=.59 for
the FR sample and CS=.36 for the PD sample. Most of these are above the preferred
.5 cut-off, and above recommended .25 cut-off, generally indicating stable results.
The nodes with the highest centrality were in general significantly more central
than most other nodes in the network, but not more central than each other (see
Figures G9-G12).
Page | 344
edge
−0.2 0.0 0.2 0.4
Figure G1 Accuracy of the edge-weights for the familial risk (FR) network. The horizonal area
within the plot represents the 95% quantile range of the parameter values across 1000 bootstraps. The
red dots indicate the sample values, while the black dots indicate the bootstrap mean values.
Page | 345
edge
−0.25 0.00 0.25 0.50
Figure G2 Accuracy of the edge-weights for the psychotic disorders (PD) network. The horizonal
area within the plot represents the 95% quantile range of the parameter values across 1000 bootstraps.
The red dots indicate the sample values, while the black dots indicate the bootstrap mean values.
Page | 346
strength
1.0
0.5
0.0
−0.5
−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled cases
Figure G3 Accuracy of strength centrality for the familial risk (FR) network.
Page | 347
strength
1.0
0.5
0.0
−0.5
−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled cases
Figure G4 Accuracy of strength centrality for the psychotic disorders (PD) network.
Page | 348
bridgeStrength
1.0
0.5
0.0
−0.5
−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled cases
Figure G5 Accuracy of bridge strength centrality for the familial risk (FR) network.
Page | 349
bridgeStrength
1.0
0.5
0.0
−0.5
−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled cases
Figure G6 Accuracy of bridge strength centrality for the psychotic disorders (PD) network.
Page | 350
edge
Avolition−−Depression
SocialWithdrawal−−Avolition
SocialSkills−−CommunicationSkills
SocialWithdrawal−−Depression
IndependencePerformance−−RecreationalActivities
Withdrawal−−InterpersonalBehavior
Hallucinations−−MagicalThinking
AttentionSwitching−−CommunicationSkills
BizarreExperiences−−Paranoia
SocialWithdrawal−−AffectiveFlattening
ProSocialActivities−−RecreationalActivities
Paranoia−−Depression
ProSocialActivities−−IndependencePerformance
IndependencePerformance−−IndependenceCompetence
InterpersonalBehavior−−ProSocialActivities
Paranoia−−Avolition
MF_AGE−−RecreationalActivities
SocialSkills−−SocialWithdrawal
SocialSkills−−AttentionSwitching
AffectiveFlattening−−Avolition
BizarreExperiences−−Grandiosity
Withdrawal−−RecreationalActivities
BizarreExperiences−−Hallucinations
Paranoia−−MagicalThinking
Paranoia−−Grandiosity
CommunicationSkills−−AffectiveFlattening
SocialSkills−−Imagination
CommunicationSkills−−Imagination
Paranoia−−AffectiveFlattening
Withdrawal−−IndependencePerformance
Grandiosity−−MagicalThinking
ProSocialActivities−−IndependenceCompetence
Withdrawal−−OccupationEmployment
Grandiosity−−SocialWithdrawal
AffectiveFlattening−−Depression
AttentionSwitching−−Imagination
IndependenceCompetence−−RecreationalActivities
SocialSkills−−Depression
AttentionSwitching−−Avolition
BizarreExperiences−−AffectiveFlattening
AttentionDetail−−AffectiveFlattening
AttentionSwitching−−Depression
Paranoia−−SocialWithdrawal
BizarreExperiences−−SocialWithdrawal
MF_IQ−−OccupationEmployment
BizarreExperiences−−Depression
MagicalThinking−−AffectiveFlattening
BizarreExperiences−−MagicalThinking
Hallucinations−−Paranoia
Hallucinations−−Depression
InterpersonalBehavior−−IndependencePerformance
BizarreExperiences−−Avolition
Hallucinations−−Grandiosity
AttentionDetail−−MagicalThinking
CommunicationSkills−−Paranoia
AttentionSwitching−−BizarreExperiences
Grandiosity−−AffectiveFlattening
AttentionSwitching−−Paranoia
MF_IQ−−AttentionDetail
AttentionDetail−−Paranoia
MF_AGE−−Imagination
Avolition−−IndependenceCompetence
Imagination−−IndependenceCompetence
Avolition−−OccupationEmployment
AffectiveFlattening−−Withdrawal
SocialWithdrawal−−InterpersonalBehavior
Paranoia−−IndependenceCompetence
Avolition−−Withdrawal
Depression−−IndependenceCompetence
Avolition−−InterpersonalBehavior
Imagination−−InterpersonalBehavior
AffectiveFlattening−−InterpersonalBehavior
AttentionSwitching−−ProSocialActivities
Paranoia−−Withdrawal
CommunicationSkills−−IndependencePerformance
Grandiosity−−Withdrawal
AttentionSwitching−−Withdrawal
AttentionSwitching−−IndependenceCompetence
SocialWithdrawal−−Withdrawal
CommunicationSkills−−InterpersonalBehavior
Avolition−−ProSocialActivities
Imagination−−Withdrawal
SocialSkills−−InterpersonalBehavior
CommunicationSkills−−Withdrawal
AttentionSwitching−−InterpersonalBehavior
SocialSkills−−ProSocialActivities
SocialSkills−−Withdrawal
CommunicationSkills−−Withdrawal
Avolition−−ProSocialActivities
AttentionSwitching−−Withdrawal
Grandiosity−−Withdrawal
CommunicationSkills−−IndependencePerformance
AttentionSwitching−−ProSocialActivities
AffectiveFlattening−−InterpersonalBehavior
Avolition−−InterpersonalBehavior
Depression−−IndependenceCompetence
AffectiveFlattening−−Withdrawal
Imagination−−IndependenceCompetence
AttentionDetail−−Paranoia
AttentionSwitching−−Paranoia
AttentionDetail−−MagicalThinking
BizarreExperiences−−Avolition
BizarreExperiences−−Depression
MF_IQ−−OccupationEmployment
BizarreExperiences−−SocialWithdrawal
AttentionDetail−−AffectiveFlattening
SocialSkills−−Depression
IndependenceCompetence−−RecreationalActivities
AffectiveFlattening−−Depression
Grandiosity−−SocialWithdrawal
ProSocialActivities−−IndependenceCompetence
CommunicationSkills−−AffectiveFlattening
Figure G7 Bootstrapped difference test for edge weights for the familial risk (FR) network. The
significance difference testing (α=.05) examines whether edges significantly differ from each other
in strength. The color of the boxes indicates whether there is a significant difference (i.e., grey boxes
reflect no significant differences and black boxes reflect significant differences).
Page | 351
edge
Performance−−IndependenceCompetence
encePerformance−−RecreationalActivities
nceCompetence−−OccupationEmployment
Withdrawal−−ProSocialActivities
oSocialActivities−−OccupationEmployment
MF_IQ−−IndependenceCompetence
AttentionDetail−−Grandiosity
cialActivities−−IndependencePerformance
onalBehavior−−IndependencePerformance
MF_IQ−−InterpersonalBehavior
rpersonalBehavior−−RecreationalActivities
AttentionDetail−−Depression
AttentionSwitching−−AttentionDetail
AttentionDetail−−Hallucinations
Paranoia−−RecreationalActivities
Imagination−−OccupationEmployment
nterpersonalBehavior−−ProSocialActivities
MF_IQ−−Depression
MagicalThinking−−Depression
MF_IQ−−AffectiveFlattening
ncePerformance−−OccupationEmployment
AttentionDetail−−CommunicationSkills
onalBehavior−−IndependenceCompetence
SocialSkills−−AttentionDetail
AttentionDetail−−BizarreExperiences
SocialSkills−−Hallucinations
CommunicationSkills−−SocialWithdrawal
CommunicationSkills−−BizarreExperiences
MagicalThinking−−RecreationalActivities
AttentionSwitching−−SocialWithdrawal
allucinations−−IndependenceCompetence
tentionDetail−−IndependencePerformance
MF_IQ−−CommunicationSkills
Grandiosity−−IndependencePerformance
MagicalThinking−−OccupationEmployment
Experiences−−IndependenceCompetence
AttentionSwitching−−Grandiosity
veFlattening−−IndependenceCompetence
SocialSkills−−Grandiosity
Hallucinations−−Withdrawal
Avolition−−IndependencePerformance
arreExperiences−−OccupationEmployment
BizarreExperiences−−Withdrawal
gicalThinking−−IndependenceCompetence
MF_IQ−−AttentionSwitching
Hallucinations−−OccupationEmployment
nicationSkills−−IndependenceCompetence
Experiences−−IndependencePerformance
MF_IQ−−BizarreExperiences
Imagination−−Depression
tentionDetail−−IndependenceCompetence
Grandiosity−−IndependenceCompetence
MF_IQ−−Hallucinations
Imagination−−Grandiosity
onSwitching−−IndependenceCompetence
Imagination−−IndependencePerformance
MF_AGE−−ProSocialActivities
MF_IQ−−Imagination
mmunicationSkills−−InterpersonalBehavior
MF_IQ−−Imagination
MF_AGE−−ProSocialActivities
Imagination−−IndependencePerformance
Imagination−−Grandiosity
MF_IQ−−Hallucinations
Grandiosity−−IndependenceCompetence
AttentionDetail−−IndependenceCompetence
Imagination−−Depression
MF_IQ−−BizarreExperiences
BizarreExperiences−−IndependencePerformance
CommunicationSkills−−IndependenceCompetence
Hallucinations−−OccupationEmployment
MF_IQ−−AttentionSwitching
MagicalThinking−−IndependenceCompetence
BizarreExperiences−−Withdrawal
BizarreExperiences−−OccupationEmployment
Avolition−−IndependencePerformance
Hallucinations−−Withdrawal
SocialSkills−−Grandiosity
AffectiveFlattening−−IndependenceCompetence
AttentionSwitching−−Grandiosity
BizarreExperiences−−IndependenceCompetence
MagicalThinking−−OccupationEmployment
Grandiosity−−IndependencePerformance
MF_IQ−−CommunicationSkills
AttentionDetail−−IndependencePerformance
Hallucinations−−IndependenceCompetence
AttentionSwitching−−SocialWithdrawal
MagicalThinking−−RecreationalActivities
CommunicationSkills−−BizarreExperiences
CommunicationSkills−−SocialWithdrawal
SocialSkills−−Hallucinations
AttentionDetail−−BizarreExperiences
SocialSkills−−AttentionDetail
InterpersonalBehavior−−IndependenceCompetence
AttentionDetail−−CommunicationSkills
IndependencePerformance−−OccupationEmployment
MF_IQ−−AffectiveFlattening
MagicalThinking−−Depression
MF_IQ−−Depression
Imagination−−OccupationEmployment
Paranoia−−RecreationalActivities
AttentionDetail−−Hallucinations
AttentionSwitching−−AttentionDetail
AttentionDetail−−Depression
InterpersonalBehavior−−RecreationalActivities
MF_IQ−−InterpersonalBehavior
AttentionDetail−−Grandiosity
MF_IQ−−IndependenceCompetence
ProSocialActivities−−OccupationEmployment
Withdrawal−−ProSocialActivities
IndependenceCompetence−−OccupationEmployment
Figure G8 Bootstrapped difference test for edge weights for the psychotic disorders (PD) network.
The significance difference testing (α=.05) examines whether edges significantly differ from each
other in strength. The color of the boxes indicates whether there is a significant difference (i.e., grey
boxes reflect no significant differences and black boxes reflect significant differences).
Page | 352
strength
Avolition 1.100
SocialWithdrawal 1.100
Depression 0.940
Withdrawal 0.900
SocialSkills 0.900
Paranoia 0.790
CommunicationSkills 0.790
ProSocialActivities 0.720
InterpersonalBehavior 0.660
AttentionSwitching 0.650
IndependencePerformance 0.610
RecreationalActivities 0.600
AffectiveFlattening 0.600
BizarreExperiences 0.470
MagicalThinking 0.360
Grandiosity 0.350
IndependenceCompetence 0.340
Hallucinations 0.290
Imagination 0.290
MF_AGE 0.120
OccupationEmployment 0.085
AttentionDetail 0.061
MF_IQ 0.030
MF_IQ
AttentionDetail
OccupationEmployment
MF_AGE
Imagination
Hallucinations
IndependenceCompetence
Grandiosity
MagicalThinking
BizarreExperiences
AffectiveFlattening
RecreationalActivities
IndependencePerformance
AttentionSwitching
InterpersonalBehavior
ProSocialActivities
CommunicationSkills
Paranoia
SocialSkills
Withdrawal
Depression
SocialWithdrawal
Avolition
Figure G9 Bootstrapped difference test for strength centrality for the familial risk (FR) network. The
significance difference testing (α=.05) examines whether nodes significantly differ from each other in
strength centrality. The color of the boxes indicates whether there is a significant difference (i.e., grey
boxes reflect no significant differences and black boxes reflect significant differences).
Page | 353
strength
SocialSkills 1.20
Paranoia 1.20
Depression 1.10
Avolition 1.10
Withdrawal 0.95
Grandiosity 0.85
Imagination 0.82
Hallucinations 0.80
MF_IQ 0.58
MF_AGE 0.25
MF_AGE
AttentionDetail
MF_IQ
AffectiveFlattening
CommunicationSkills
MagicalThinking
Hallucinations
AttentionSwitching
Imagination
Grandiosity
Withdrawal
ProSocialActivities
SocialWithdrawal
Avolition
Depression
Paranoia
SocialSkills
BizarreExperiences
Figure G10 Bootstrapped difference test for strength centrality for the psychotic disorders (PD)
network. The significance difference testing (α=.05) examines whether nodes significantly differ from
each other in strength centrality. The color of the boxes indicates whether there is a significant differ-
ence (i.e., grey boxes reflect no significant differences and black boxes reflect significant differences).
Page | 354
bridgeStrength
Withdrawal 0.5300
SocialSkills 0.4700
Avolition 0.1500
MF_AGE 0.1200
Imagination 0.1100
Depression 0.0900
Paranoia 0.0720
Grandiosity 0.0480
MF_IQ 0.0300
Hallucinations
BizarreExperiences
MagicalThinking
MF_IQ
Grandiosity
AttentionDetail
Paranoia
Depression
Imagination
MF_AGE
AffectiveFlattening
Avolition
SocialWithdrawal
ProSocialActivities
CommunicationSkills
AttentionSwitching
SocialSkills
Withdrawal
Figure G11 Bootstrapped difference test for bridge strength centrality for the familial risk (FR)
network. The significance difference testing (α=.05) examines whether nodes significantly differ from
each other in bridge strength centrality. The color of the boxes indicates whether there is a significant
difference (i.e., grey boxes reflect no significant differences and black boxes reflect significant differ-
ences).
Page | 355
bridgeStrength
MF_IQ 0.580
Imagination 0.530
SocialSkills 0.480
Withdrawal 0.280
Grandiosity 0.260
MF_AGE 0.250
Depression 0.240
Avolition 0.130
Paranoia 0.130
MagicalThinking
AffectiveFlattening
Paranoia
Avolition
Hallucinations
BizarreExperiences
SocialWithdrawal
CommunicationSkills
Depression
MF_AGE
Grandiosity
ProSocialActivities
Withdrawal
AttentionDetail
AttentionSwitching
SocialSkills
Imagination
MF_IQ
Figure G12 Bootstrapped difference test for bridge strength centrality for the psychotic disorders
(PD) network. The significance difference testing (α=.05) examines whether nodes significantly differ
from each other in bridge strength centrality. The color of the boxes indicates whether there is a sig-
nificant difference (i.e., grey boxes reflect no significant differences and black boxes reflect significant
differences).
Page | 356
Table G1 Mean and standard deviation of scores for typical comparisons (TC), familial risk (FR), and
psychotic disorders (PD) samples
Mean (SD)
Variable TC Sample PD Sample
FR Sample
Age 38.51 (10.64) 34 (7.89) 33.38 (7.19)
IQ 115.31 (17.12) 111.61 (17.70) 100.79 (17.90)
Social Skills 1.75 (1.82) 1.97 (1.95) 3.56 (2.34)
Attention Switching 2.79 (2.01) 3.08 (1.97) 4.97 (2.21)
Attention Detail 3.15 (1.93) 3.33 (1.87) 4.02 (2.20)
Communication Skills 1.82 (1.64) 1.97 (1.68) 3.27 (1.96)
Imagination 2.93 (1.77) 3.11 (1.80) 4.17 (1.99)
Bizarre Experiences 0.15 (0.74) 0.17 (0.66) 2.67 (3.80)
Hallucinations 0.06 (0.46) 0.04 (0.26) 1.34 (2.20)
Paranoia 0.85 (1.29) 1.00 (1.34) 3.04 (2.77)
Grandiosity 0.21 (0.68) 0.24 (0.67) 1.05 (1.48)
Magical Thinking 0.25 (0.70) 0.28 (0.78) 1.22 (1.57)
Social Withdrawal 2.02 (1.79) 2.27 (2.02) 3.90 (2.59)
Affective Flattening 0.71 (1.21) 0.93 (1.42) 2.32 (2.16)
Avolition 3.02 (2.56) 3.51 (3.06) 6.01 (4.14)
Depression 3.83 (3.29) 4.06 (3.33) 7.07 (4.75)
Withdrawal 13.27 (1.83) 12.99 (1.92) 10.73 (2.60)
Interpersonal Behavior 8.83 (0.59) 8.66 (0.85) 7.58 (1.69)
Prosocial Activities 27.04 (8.86) 26.22 (9.72) 20.32 (10.01)
Independence Performance 34.75 (3.90) 34.03 (4.32) 30.92 (5.83)
Independence Competence 38.62 (1.48) 38.70 (1.13) 36.40 (3.93)
Recreational Activities 27.25 (5.63) 25.86 (5.92) 22.61 (6.33)
Occupation Employment 9.58 (1.38) 9.46 (1.47) 6.64 (3.07)
Page | 357
H
Appendix H: Tutorial
In the following, we illustrate the presented norms using two example datasets.
Both datasets are available online, provided together with the R code.2 The dataset
used in Example 1 below has been included in a prior publication discussing
the results (Burger, Stroebe, et al., 2020), while Example 2 has not yet, to our
knowledge, been used in prior network analyses.
Here, the focus lies on reporting relevant results for a specific research question,
rather than interpreting them in regards to the given research question. The two
examples cover different aspects of the analysis-specific routine discussed in this
chapter; for an overview of these aspects see Table H1 below. Of note, the type of
analysis-specific routine very much relies on the specific research question at hand.
In principle, one could also showcase other aspects of the routine not applied here,
using the same datasets. We present the examples in the style of a typical network
contribution for an APA manuscript, and include the headers as instructed in the
main body of the chapter, for clarity.
Table H1 Overview of routines covered in the two examples.

Example 1 Example 2
Data description Data and results from Data and results from
Bereavement or breakup: an online offering of the
Differences in networks of Taylor Manifest Anxiety
depression (Burger et al., 2020) Scale (Taylor, 1953)
Methods: General analysis routine ✓ ✓
Methods: Group comparison ✓
Methods: Centrality ✓
Methods: Differences between edges ✓
Methods: Clustering
Results: General analysis routine ✓ ✓
Results: Network visualization ✓ ✓
Results: Global network properties ✓
Results: Local network properties ✓
Results: Group comparisons ✓
2 The R-code is available at https://osf.io/p9wn2/
Page | 360
Example 1: Separation and widowhood in elderly individuals
Data for the first example stem from the Swiss longitudinal study Relationships
in later life, which followed widowed and separated individuals after their loss
experience, and collected information on their psychosocial functioning, including
depressive symptomatology. The data and project description can be found online
(https://www.kpp.psy.unibe.ch/forschung/projekte/nccrlives/index_ger.html), and
the results have been discussed previously (Burger, Stroebe, et al., 2020). The main
research interest here lies in comparing depressive symptom networks between
the widowed and separated individuals, specifically comparing how strongly they
are connected and the overall structure of the two networks. Next to the general
analysis routine, we therefore focus on group comparison (methods and results),
network visualization (results), and global network properties (results).
Page | 361
Methods
General Analysis Routine
Features of the data
Sample, For this analysis, we included data collected on the German version of the Center
instrument, for Epidemiologic Studies Depression scale (CES-D; Radloff, 1977; German:
and processing Allgemeine Depressions-Skala, ADS-K; Hautzinger & Geue, 2016).
choices The dataset consists of 1276 married, 566 widowed, and 971 separated individuals.
Participants were contacted via post mail and filled in a pen-and-paper questionnaire.
To circumvent the issue that participants might be at different stages of adaptation
to the adverse life event, we only included participants with a maximum distance of
two years to the event (widowhood/ separation). This resulted in 145 widowed and
217 separated individuals. To be able to include widowhood/ separation as node in
the network, we added 145 married controls to the widowed sample, and 217 married
controls to the separated sample.3 This way, widowhood/ separation is included as
a binary node, indicating the presence versus absence of the respective life event.
In order to investigate conceptual overlap between variables, we examined bivariate
correlations between all variables, and combined items if their content suggested
strong conceptual similarity, and their bivariate correlation was r ≥ .50. Accordingly,
we combined the original items mood, upset, and depressed (new item ‘mood’), as
well as the items happy and enjoy (new item ‘happy’). This resulted in 12 variables,
each rated on an ordinal scale with four answer categories: 1=rarely or none of the
time (less than 1 day), 2=some or a little of the time (1–2 days), 3=occasionally or a
moderate amount of time (3–4 days), 4=most or all of the time (5–7 days).
Features of the statistical method
Estimation We estimated partial correlation networks for both samples, the widowed and
method separated sample, using the glasso regularization and a tuning-parameter gamma
set to .5 (Foygel & Drton, 2010). Due to the ordinal nature of the data, we used
Spearman’s rank-correlation, and pairwise complete observations to handle missing
data. In total, of all variables included in the network analysis, 6.6% of the ratings
were missing in the widowed/ married sample, and 5.1% in the separated /married
sample. Here, we assume that these ratings are missing at random (Rubin, 1976).
Accuracy and To assess accuracy of the edge estimates, we conducted the routine implemented
stability of in the bootnet package (Epskamp, Borsboom, et al., 2017), using nonparametric
edge-estimates bootstrapping with 1000 bootstrap samples.
Statistical The analyses have been conducted using R version 3.5.2 on October 8th, 2020. For
packages network estimation, we used the estimateNetwork function in the bootnet package
(Epskamp, Borsboom, et al., 2017). Networks have been visualized using the qgraph
package (Epskamp et al., 2012).
Analysis-specific Routine
Group Groups were compared by obtaining the difference in global strength using the
comparisons Network Comparison Test (NCT; van Borkulo et al., 2017), with seed set to 123.
This test assesses if the two networks differ in their overall level of connectivity.
Since we are primarily interested in global differences in network connectivity,
other tests available within the NCT were disregarded in the present analyses.
Additionally, we correlated the adjacency matrices of the two networks as an
additional measure of similarity between the networks.
Page | 362
Results
Final The widowed network included 290 individuals (145 widowed and 145
sample size married controls), and the separated network included 434 individuals
(217 separated and 217 married controls).
Results of accuracy and In general, the confidence intervals were rather broad and overlapping.
stability checks The order of edge estimates should therefore be interpreted with caution
(see Figure H2).
Note: Results of accuracy routines are commonly included as part of the
supplementary materials. The figures presented below are figures from the
original article (Burger, Stroebe, et al., 2020) and are included here for
reference.
Network visualization Note: Visualization of networks does not necessarily require explicit
descriptions. To facilitate interpretability, here we used the colorblind-
theme in qgraph (Epskamp et al., 2012), fixed the average layout between
the two network plots, curved edges that would otherwise cross nodes, and
made negative edges dashed (useful if printed without colors; see Figure
H1). Any exploratory reporting of findings, such as relevant edges, will
be specific to the given research context. The figures presented below are
figures from the original article (Burger, Stroebe, et al., 2020).
Global network Since we are interested in comparing the two networks in regards to their
properties connectivity, we computed the density of the two networks as the ratio of
detected edges to the total number of edges in a fully connected network.
The network of widowed/ married individuals had a density of .615 (48/78
edges), the separated network had a density of .744 (58/78 edges).
Group comparisons While the global invariance test within the Network Comparison Test
procedure indicated that there were some differences in the overall level
of connectivity between the widowed and separated network (p=.003), the
adjacency matrices showed a rather large correlation (r=.750), indicating
that the overall structure between the networks was similar. This shows
that the networks differed in how strongly connected they are (sum of
absolute edge weights, connectivity), while edges that were detected
showed a similar pattern across the two networks (correlation of edges),
i.e., edges that were large (small) in the separated network, were generally
also large (small) in the widowed network.
3 Note that adding control participants and including group membership in the network is only but
one way to approach group comparisons. Many other techniques have been discussed recently, such as
moderated network analysis (Haslbeck et al., 2019), or Bayesian approaches (Williams et al., 2020). For
more detailed information on the approach used here, we advise to consider the original publication.
Page | 363
unfriend unfriend
Page | 364
exhaust exhaust
sleep getgo afraid sleep getgo afraid

concentr concentr
failure failure
mood mood
happy happy
lonely lonely
sad sad
talk separation talk widowhood
Figure H1 Regularized partial-correlation networks (tuning-parameter gamma=.5) for the separated (left) and the widowed sample (right). Solid-blue edges
represent positive, regularized partial-correlations, dashed-red edges represent negative, regularized partial-correlations.
Page | 365
Figure H2 Nonparametric bootstrapping results with 1000 samples for the separated (left) and the widowed network (right).
Example 2: Taylor Manifest Anxiety Scale

Data for the second example stem from the openpsychometrics.org project, using
the Taylor Manifest Anxiety Scale (Taylor, 1953). The data and project description
can be found online (https://openpsychometrics.org/tests/TMAS). Let’s assume
the main research interests here lie in the general network structure of anxiety,
edge differences in the network structure, as well as in which items play a more
central role in the network. Next to the general analysis routine, we therefore focus
on centrality results (methods and results), edge differences (methods and results),
network visualization (results), and local network properties (results).
Page | 366
Methods
Features of the data
Sample, For this analysis, we included data collected on the Taylor Manifest Anxiety
instrument, Scale (Taylor, 1953). This data was collected online; at the end of the test users
and processing were asked if their answers were accurate and could be used for research. 76%
choices said yes and data have been published on the openpsychometrics.org project. The
dataset consisted of 5410 individuals.
The network model included all questions from the Taylor Manifest Anxiety
Scale (Taylor, 1953), thus resulting in 50 nodes. Each item was rated on a binary
scale with two answer categories (0=FALSE,1=TRUE). In addition, we used
listwise deletion for missing data points, as the chosen estimation algorithm
does not allow for missing data. Data were assumed to be missing completely
at random.
Features of the statistical method
Estimation We estimated the network structure using an Ising model (van Borkulo,
method Borsboom, et al., 2014). An Ising model is represented using log liner
relationships and pairwise interactions, similar to partial correlation
coefficients in a Gaussian Graphical Model (GGM; Epskamp, Waldorp,
et al., 2018). To control for potential spurious associations, the estimation
procedure here uses a penalty approach, specifically the eLasso penalty based
on the Extended Bayesian Information Criterion (Ravikumar et al., 2010).
Default values as set in the package were used, with the EBIC hypertuning
parameter set to 0.25.
Accuracy and To assess accuracy of the edge weight estimates, we conducted the routine
stability of edge- as recommended by Epskamp and colleagues (2017), using nonparametric
estimates bootstrapping based on 1000 bootstrap samples.
Statistical The analyses have been conducted using R version 3.5.2 on October 12th,
packages 2020. For network estimation, we used the estimateNetwork function in
the bootnet package (Epskamp et al., 2018), using the IsingFit package (van
Borkulo, Epskamp, et al., 2014). The accuracy of estimates has been assessed
using the bootnet function. Networks have been visualized using the qgraph
package (Epskamp et al., 2012).
Centrality Indices To further quantify how well a node is directly connected to other nodes in the
network structure, we investigated strength as a centrality measure (Costantini
et al., 2015; Opsahl et al., 2010).
Accuracy and To assess accuracy of the strength centrality estimates, we conducted the
stability of routine implemented in the bootnet package (Epskamp et al., 2018), using
centrality- case-drop bootstrapping based on 1000 bootstrap samples. Further, to ensure
estimates interpretable differences in centrality, we used the bootstrapped difference-test
in the bootnet package.
Differences Finally, as we were interested in an exploratory fashion whether certain edges were
between edges stronger and stood out in the network structure, we carried out a bootstrapped
difference-test using the R package bootnet (Epskamp et al., 2018).
Page | 367
Results
Final Following removal of missing data, 4474 subjects were included in the
sample size current analyses.
Results of accuracy In general, the confidence intervals were very narrow, indicating stable
and stability checks results (see Figure H5). In addition, strength centrality estimates were
stable, with a centrality stability coefficient of .75, indicating that 75% of
the data could be dropped to retain with 95% certainty a correlation of .7
with the original dataset. Of note, while the most central items were more
central than most other items in the network, they were not more central
than each other (see Figure H7).
Note: Results of accuracy routines are commonly included as part of the
supplementary materials. The figures presented below are included here for
reference.
Network visualization Note: Visualization of networks does not necessarily require explicit
descriptions. The network visualization is presented in Figure H3. To
facilitate interpretability, here we used the colorblind-theme in qgraph
(Epskamp et al., 2012), included a legend with the description of each item,
and used a cut value of 0. The layout used was the automatically generated
layout based on the Fruchterman-Reingold algorithm (Fruchterman &
Reingold, 1991). Any exploratory reporting of findings, such as relevant
edges, will be specific to the given research context.
Local network Figure H4 below presents the results of the centrality analyses. In addition,
properties: Centrality Table H2 presents the standardized and raw centrality indices. The 3 most
indices central items were: Q27, Q31, and Q48. Of note, while these were more
central than many other items in the network, they were not more central
than each other (see Figure H7).
Local network Figure H8 presents the results of the edge difference test. The labels are
properties: Edge omitted for clarity. In general, the stronger edges in the network were
Comparison significantly different than most other edges in the network. Of note, the
two strongest edges in the current network structure were significantly
different from each other and all other edges in the network. These are
the edge between Q6 and Q41 and between Q40 and Q46.
Page | 368
Q1: I do not tire quickly
Q2: I am troubled by attacks of nausea
Q3: I believe I am no more nervous than most others
Q4: I have very few headaches
Q5: I work under a great deal of tension
Q7 Q22 Q23 Q6: I cannot keep my mind on one thing
Q7: I worry over money and business
Q8: I frequently notice my hand shakes
when I try to do something
Q14 Q9: I blush no more often than others
Q6
Q34 Q35 Q10: I have diarrhea once a month or more
Q11: I worry quite a bit over possible misfortunes
Q20 Q12: I practically never blush
Q41 Q24 Q13: I am often afraid that I am going to blush
Q11 Q14: I have nightmares every few nights
Q15: My hands and feet are usually warm
Q5 Q16: I sweat very easily even on cool days
Q36 Q17: Sometimes when embarrassed, I break out in a sweat
Q37 Q18: I hardly ever notice my heart pounding
Q33
Q27 Q21 and I am seldom short of breath
Q19: I feel hungry almost all the time
Q10 Q20: I am very seldom troubled by constipation
Q45 Q21: I have a great deal of stomach trouble
Q8 Q22: I have had periods in which I lost sleep over worry
Q31 Q23: My sleep is fitful and disturbed
Q48 Q24: I dream frequently about things that
Q28 are best kept to myself
Q49 Q39 Q2 Q25: I am easily embarrassed
Q40 Q26: I am more sensitive than most other people
Q4 Q27: I frequently find myself worrying about something
Q32 Q28: I wish I could be as happy as others seem to be
Q29: I am usually calm and not easily upset
Q46 Q19 Q30: I cry easily
Q38
Q44 Q31: I feel anxiety about something or
Q29 someone almost all the time
Q3 Q32: I am happy most of the time
Q18 Q33: It makes me nervous to have to wait
Q34: I have periods of such great restlessness
that I cannot sit long I a chair
Q43 Q30 Q35: Sometimes I become so excited that I find
Q47 Q1 it hard to get to sleep
Q36: I have sometimes felt that difficulties
Q17 Q16 were piling up so high that I could not overcome them
Q50 Q37: I must admit that I have at times been
Q26 worried beyond reason over something that really did not matter
Q38: I have very few fears compared to my friends
Q13 Q39: I have been afraid of things or people
Q25 that I know could not hurt me
Q42 Q40: I certainly feel useless at times
Q15 Q41: I find it hard to keep my mind on a task or job
Q12 Q42: I am usually self−conscious
Q43: I am inclined to take things hard
Q44: I am a high−strung person
Q9 Q45: Life is a trial for me much of the time
Q46: At times I think I am no good at all
Q47: I am certainly lacking in self−confidence
Q48: I sometimes feel that I am about to go to pieces
Q49: I shrink from facing crisis of difficulty
Q50: I am entirely self−confident
Page | 369
Figure H3 Ising Model. Blue edges represent positive relations, red edges represent negative relations.
Strength
Q27
Q31
Q48
Q47
Q46
Q25
Q45
Q11
Q36
Q32
Q28
Q41
Q49
Q13
Q29
Q21
Q23
Q43
Q40
Q22
Q50
Q39
Q33
Q26
Q37
Q34
Q17
Q38
Q14
Q12
Q30
Q24
Q6
Q4
Q42
Q2
Q44
Q35
Q18
Q8
Q16
Q7
Q1
Q10
Q9
Q5
Q3
Q19
Q20
Q15
0 2 4 6
Figure H4 Centrality plot denoting strength centrality results. The order is set from the strongest to
the weakest item.
Page | 370
edge
−2 −1 0 1 2
Figure H5 Accuracy of the edge-weights for the estimated network model. The horizonal area within
the plot represents the 95% quantile range of the parameter values across 1000 bootstraps. The red
dots indicate the sample values, while the black dots indicate the bootstrap mean values.
Page | 371
strength
Page | 372
1.0
0.5
0.0
−0.5

−1.0
90% 80% 70% 60% 50% 40% 30%

Sampled cases
Figure H6 Accuracy of strength centrality.
strength
Q27 7.50
Q31 7.30
Q48 7.00
Q47 6.80
Q46 6.70
Q25 6.40
Q45 5.80
Q11 5.70
Q36 5.60
Q32 5.30
Q28 5.20
Q41 5.00
Q49 5.00
Q13 4.90
Q29 4.90
Q21 4.50
Q23 4.50
Q43 4.40
Q40 4.30
Q22 4.20
Q50 4.20
Q39 4.20
Q33 4.00
Q26 4.00
Q37 4.00
Q34 4.00
Q17 4.00
Q38 3.90
Q14 3.90
Q12 3.80
Q30 3.50
Q24 3.50
Q6 3.40
Q4 3.20
Q42 3.10
Q2 3.00
Q44 2.80
Q35 2.80
Q18 2.70
Q8 2.70
Q16 2.60
Q7 2.40
Q1 2.40
Q10 2.10
Q9 2.00
Q5 2.00
Q3 1.50
Q19 0.85
Q20 0.82
Q15 0.30
Q15
Q20
Q19
Q3
Q5
Q9
Q10
Q1
Q7
Q16
Q8
Q18
Q35
Q44
Q2
Q42
Q4
Q6
Q24
Q30
Q12
Q14
Q38
Q17
Q34
Q37
Q26
Q33
Q39
Q50
Q22
Q40
Q43
Q23
Q21
Q29
Q13
Q49
Q41
Q28
Q32
Q36
Q11
Q45
Q25
Q46
Q47
Q48
Q31
Q27
Figure H7 Strength bootstrapped difference test (α=.05). Grey boxes reflect no significant differences
and black boxes reflect significant differences.
Page | 373
edge
Figure H8 Edge bootstrapped difference test for all non-zero edges in the network structure (α=.05).
Grey boxes reflect no significant differences and black boxes reflect significant differences. Labels are
omitted for clarity.
Page | 374
Table H2 Standardized and unstandardized strength centrality values
Standardized strength values Unstandardized strength values
Q1 -0.93 2.42
Q2 -0.58 3
Q3 -1.47 1.51
Q4 -0.45 3.22
Q5 -1.2 1.97
Q6 -0.31 3.45
Q7 -0.92 2.43
Q8 -0.75 2.71
Q9 -1.16 2.03
Q10 -1.15 2.05
Q11 1.02 5.67
Q12 -0.13 3.76
Q13 0.54 4.88
Q14 -0.06 3.87
Q15 -2.19 0.3
Q16 -0.85 2.55
Q17 -0.01 3.95
Q18 -0.74 2.73
Q19 -1.86 0.85
Q20 -1.88 0.82
Q21 0.33 4.53
Q22 0.14 4.21
Q23 0.31 4.49
Q24 -0.28 3.5
Q25 1.47 6.44
Q26 0.01 3.99
Q27 2.13 7.55
Q28 0.75 5.22
Q29 0.53 4.86
Q30 -0.28 3.51
Q31 1.98 7.3
Q32 0.77 5.27
Q33 0.01 3.99
Page | 375
Table H2Continued.
Standardized strength values Unstandardized strength values
Q34 -0.01 3.96
Q35 -0.71 2.78
Q36 0.99 5.63
Q37 0 3.97
Q38 -0.06 3.88
Q39 0.13 4.19
Q40 0.2 4.3
Q41 0.64 5.04
Q42 -0.5 3.14
Q43 0.24 4.37
Q44 -0.67 2.85
Q45 1.1 5.81
Q46 1.63 6.71
Q47 1.69 6.8
Q48 1.78 6.95
Q49 0.62 5.02
Q50 0.14 4.2
Page | 376
I
Appendix I1. R-function

# This function can be used to write the following files:
# - Pearson _ listwise.csv - Listwise pearson correlations
# - Pearson _ pairwise.csv - Pairwise pearson correlations
# - Spearman _ listwise.csv - Listwise Spearman correlations
# - Spearman _ pairwise.csv - Pairwise Spearman correlations
# - descriptives.txt - Some general descriptive measures
getCorrelations <- function(data){
# Check if this is a matrix or data frame:
if (!is.matrix(data) && !is.data.frame(data)){
stop("Input is not a matrix or data frame.")
}
# If it is a matrix, make it a data frame:

if (is.matrix(data)){
data <- as.data.frame(data)
}
# Descriptives:
nSample _ listwise <- sum(apply(data,1,function(x)all(!is.
na(x))))
nSample _ full <- sum(apply(data,1,function(x)any(!is.na(x))))
# Compute average sample size for pairwise correlations:

nomisdata <- !is.na(as.matrix(data))
nMat <- t(nomisdata) %*% nomisdata
nSample _ pairwise <- mean(nMat[lower.tri(nMat,diag=FALSE)],na.
omit=TRUE)
# Means:
means <- colMeans(data,na.rm = TRUE)
# SDs:
SDs <- sapply(data,sd,na.rm = TRUE)
# Number of levels:
Page | 378
nLevels <- sapply(data, function(x)length(unique(x)[!is.
na(unique(x))]))
# Set names:
if (is.null(colnames(data))){
colnames(data) <- paste0("V",seq _ len(ncol(data)))
}
# Write these to a file:

descriptivesFile <- paste0(getwd(),"/descriptives.txt")
write(paste0(
"Sample size (full): ", nSample _ full, "\n",
"Sample size (listwise): ", nSample _ listwise, "\n",
"Sample size (pairwise average): ", nSample _ pairwise, "\n",
"Name: ", paste0(colnames(data), collapse = "; "), "\n",
"Means: ", paste0(means, collapse = "; "), "\n",
"Standard deviations: ", paste0(SDs, collapse = "; "), "\n",
"Number of levels: ", paste0(nLevels, collapse = "; ")
), file = descriptivesFile)
# Correlations:
try({
pearsonCorsFile _ listwise <- paste0(getwd(),"/Pearson _
listwise.csv")
w r i t e.c s v(c o r(d a t a , u s e = " c o m p l e t e.o b s "),
file = pearsonCorsFile _ listwise)
})
try({
pearsonCorsFile _ pairwise <- paste0(getwd(),"/Pearson _
pairwise.csv")
write.csv(cor(data, use = "pair wise.complete.obs"),
file = pearsonCorsFile _ pairwise)
})
try({
Page | 379
spearmanCorsFile _ listwise <- paste0(getwd(),"/Spearman _

listwise.csv")
w r i t e .c s v(c o r( d a t a , u s e = " c o m p l e t e .o b s ",
method = "spearman"), file = spearmanCorsFile _ listwise)
})
try({
spearmanCorsFile _ pairwise <- paste0(getwd(),"/Spearman _
pairwise.csv")
write.csv(cor(d ata, use = "pair wise.com plete.obs",
method = "spearman"), file = spearmanCorsFile _ pairwise)
})
cat("Done! Please mail us the following files:\n\n1.

",pearsonCorsFile _ listwise,
"\n2. ",pearsonCorsFile _ pairwise,
"\n3. ",spearmanCorsFile _ listwise,
"\n4. ",spearmanCorsFile _ pairwise,
"\n5. ",descriptivesFile,"\n\nThank you for your assistance!")
}
Page | 380
Table I1 Node descriptions and alternative descriptions mapped to the nodes included
DSM-IV ID Label Examples of alternative descriptions DSM-IV description
B1 Intrusive Thoughts Intrusive recollections; Thought about it when Recurrent and intrusive distressing recollections of the event,
didn’t mean to including images, thoughts, or perceptions. Note: In young
children, repetitive play may occur in which themes or aspects of
the trauma are expressed
B2 Nightmares Dreams, Traumatic dreams, Distressing dreams; Recurrent distressing dreams of the event
Has upsetting dreams; Had dreams about it
B3 Flashbacks Re-experiencing; Reminders brought back Acting or feeling as if the traumatic event were recurring (includes
feelings; Pictures popped into mind a sense of reliving the experience, illusions, hallucinations, and
dissociative flashback episodes, including those that occur on
awakening or when intoxicated). Note: In young children, trauma-
specific re-enactment may occur
B4 Psychological Upset at reminder of trauma; Heightened emotional Intense psychological distress at exposure to the internal or external
reactivity reactivity; Avoid letting themselves getting upset cues that symbolize or resemble an aspect of the traumatic event
when thinking or being reminded of it
B5 Physiological Physiological cue reactivity; Physiological Physiological reactivity on exposure to internal or external cues
reactivity reaction on exposure that symbolize or resemble an aspect of the traumatic event
C1 Internal avoidance Avoiding Thoughts/Feelings Efforts to avoid thoughts, feelings, or conversations associated
with the trauma
C3 Amnesia Inability remembering; felt as if it hadn’t Inability to recall important aspect of the trauma
happened or wasn’t real
C4 Loss of interest Anhedonia Markedly diminished interest or participation in significant
activities
C5 Feeling detached Feeling distant or cut off from others; Feelings of detachment or estrangement from others
Difficulties feeling close to others
Page | 381
Table I1Continued.
DSM-IV ID Label Examples of alternative descriptions DSM-IV description
Page | 382
C6 Emotional numbing Numbness Happiness/Love; Restricted Affect Restricted range of affect (e.g., unable to have loving feelings)
D2 Irritability / anger Anger; Irritability Irritability or outbursts of anger
D4 Hypervigilant Overly Alert; Watchful / On-guard Hyper vigilance
D5 Easily startled Exaggerated startle, Exaggerated startle response Exaggerated startle response
D3 Difficulty Concentration Difficulty concentrating

concentrating
D1 Sleep disturbance Difficulty falling or staying asleep; Trouble Difficulty falling or staying asleep
staying asleep; Trouble falling asleep
C7 Hopelessness Future foreshortening; Feeling plans won’t come Sense of a foreshortened future (e.g., does not expect to have a
true career, marriage, children, or a normal life span).
Table I2 Numeric results
Implied Random-
Variable 1 Variable 2 Edge SE p value
Correlation effect SD
nightmares intrusive thoughts .229 0.016 <.001 .548 0.133
flashbacks intrusive thoughts .235 0.023 <.001 .550 0.133
psychological reactivity intrusive thoughts .174 0.025 <.001 .529 0.155
physiological reactivity intrusive thoughts .075 0.017 <.001 .505 0.099
internal avoidance intrusive thoughts .105 0.014 <.001 .448 0.117
external avoidance intrusive thoughts .023 0.012 .051 .413 0.118
amnesia intrusive thoughts -.014 0.014 .314 .256 0.129
loss of interest intrusive thoughts .016 0.013 .217 .354 0.124
feeling detached intrusive thoughts -.001 0.015 .948 .361 0.120
emotional numbing intrusive thoughts .025 0.015 .093 .346 0.118
irritability anger intrusive thoughts -.010 0.013 .437 .325 0.119
hypervigilant intrusive thoughts .043 0.012 <.001 .373 0.114
easily startled intrusive thoughts .009 0.013 .483 .388 0.113
difficulty concentrating intrusive thoughts .046 0.013 <.001 .390 0.099
sleep disturbance intrusive thoughts .027 0.017 .114 .394 0.109
hopelessness intrusive thoughts .061 0.019 .001 .368 0.108
flashbacks nightmares .211 0.020 <.001 .511 0.140
psychological reactivity nightmares -.009 0.023 0.694 .416 0.151
physiological reactivity nightmares .147 0.023 <.001 .493 0.134
Page | 383
Table I2Continued.
Implied Random-
Page | 384
internal avoidance nightmares .016 0.015 .260 .373 0.125
external avoidance nightmares .033 0.013 .010 .372 0.133
amnesia nightmares .003 0.017 .872 .233 0.142
loss of interest nightmares .021 0.016 .174 .320 0.139

feeling detached nightmares -.028 0.016 .080 .314 0.129
emotional numbing nightmares .009 0.014 .538 .302 0.115
irritability anger nightmares -.018 0.020 .362 .298 0.118
hypervigilant nightmares .023 0.011 .038 .341 0.108
easily startled nightmares .026 0.014 .069 .371 0.140
difficulty concentrating nightmares .006 0.018 .724 .352 0.120
sleep disturbance nightmares .234 0.025 <.001 .472 0.141
hopelessness nightmares .005 0.015 .749 .310 0.099
psychological reactivity flashbacks .118 0.021 <.001 .472 0.153
physiological reactivity flashbacks .088 0.031 .004 .470 0.144
internal avoidance flashbacks -.005 0.014 .739 .362 0.120
external avoidance flashbacks .038 0.011 .001 .372 0.135
amnesia flashbacks .063 0.014 <.001 .268 0.127
loss of interest flashbacks -.012 0.016 .452 .302 0.133
feeling detached flashbacks -.000 0.018 .992 .318 0.130
Table I2Continued.
Implied Random-
emotional numbing flashbacks .016 0.014 .245 .307 0.115
irritability anger flashbacks .032 0.017 .070 .307 0.124
hypervigilant flashbacks .016 0.013 .227 .330 0.120
easily startled flashbacks .041 0.014 .002 .364 0.128
difficulty concentrating flashbacks -.007 0.016 .666 .327 0.114
sleep disturbance flashbacks -.018 0.014 .196 .336 0.129
hopelessness flashbacks .050 0.016 .002 .329 0.116
physiological reactivity psychological reactivity .249 0.025 <.001 .567 0.140
internal avoidance psychological reactivity .108 0.021 <.001 .461 0.148
external avoidance psychological reactivity .077 0.016 <.001 .446 0.149
amnesia psychological reactivity .035 0.015 .021 .286 0.130
loss of interest psychological reactivity .023 0.014 .109 .366 0.115
feeling detached psychological reactivity .041 0.020 .037 .388 0.120
emotional numbing psychological reactivity -.004 0.019 .831 .343 0.121
irritability anger psychological reactivity .065 0.016 <.001 .367 0.122
hypervigilant psychological reactivity .017 0.013 .208 .366 0.126
easily startled psychological reactivity .050 0.018 .006 .409 0.123
difficulty concentrating psychological reactivity .010 0.014 .468 .380 0.108
sleep disturbance psychological reactivity -.005 0.012 .682 .357 0.102
Page | 385
Table I2Continued.
Implied Random-
Page | 386
hopelessness psychological reactivity .035 0.017 .036 .354 0.103
internal avoidance physiological reactivity .056 0.018 .002 .444 0.123
external avoidance physiological reactivity .096 0.018 <.001 .455 0.135
amnesia physiological reactivity .025 0.019 .177 .283 0.125

loss of interest physiological reactivity .000 0.014 .990 .364 0.122
feeling detached physiological reactivity .032 0.026 .224 .391 0.123
emotional numbing physiological reactivity .011 0.021 .587 .351 0.133
irritability anger physiological reactivity .044 0.015 .003 .369 0.126
hypervigilant physiological reactivity .016 0.016 .309 .385 0.126
easily startled physiological reactivity .105 0.019 <.001 .450 0.128
difficulty concentrating physiological reactivity .048 0.018 .006 .408 0.124
sleep disturbance physiological reactivity .046 0.014 .001 .406 0.132
hopelessness physiological reactivity .008 0.018 .660 .346 0.115
external avoidance internal avoidance .332 0.027 <.001 .560 0.177
amnesia internal avoidance .079 0.016 <.001 .307 0.140
loss of interest internal avoidance .036 0.014 .011 .369 0.126
feeling detached internal avoidance .010 0.015 .513 .361 0.132
emotional numbing internal avoidance .037 0.013 .003 .339 0.119
irritability anger internal avoidance .011 0.013 .412 .318 0.121
hypervigilant internal avoidance .052 0.012 <.001 .362 0.107
Table I2Continued.
Implied Random-
easily startled internal avoidance -.001 0.014 .949 .363 0.123
difficulty concentrating internal avoidance .057 0.017 .001 .376 0.129
sleep disturbance internal avoidance .043 0.014 .003 .355 0.120
hopelessness internal avoidance -.005 0.014 .709 .318 0.107
amnesia external avoidance .074 0.015 <.001 .303 0.130
loss of interest external avoidance .086 0.013 <.001 .380 0.122
feeling detached external avoidance .025 0.012 .046 .360 0.121
emotional numbing external avoidance -.021 0.015 .160 .313 0.139
irritability anger external avoidance .001 0.013 .948 .307 0.133
hypervigilant external avoidance .065 0.016 <.001 .373 0.123
easily startled external avoidance .062 0.021 .003 .388 0.128
difficulty concentrating external avoidance -.021 0.014 .127 .341 0.112
sleep disturbance external avoidance .011 0.014 .434 .335 0.121
hopelessness external avoidance .040 0.018 .024 .327 0.106
loss of interest amnesia .053 0.018 .003 .289 0.135
feeling detached amnesia .027 0.018 .125 .289 0.104
emotional numbing amnesia .065 0.016 <.001 .290 0.120
irritability anger amnesia .012 0.013 0.348 .233 0.134
hypervigilant amnesia -.000 0.014 .981 .224 0.122
easily startled amnesia .023 0.011 .042 .250 0.106
Page | 387
Table I2Continued.
Implied Random-
Page | 388
difficulty concentrating amnesia .036 0.014 .011 .273 0.127
sleep disturbance amnesia -.005 0.016 .772 .223 0.142
hopelessness amnesia .055 0.017 .001 .270 0.125
feeling detached loss of interest .243 0.023 <.001 .552 0.145

emotional numbing loss of interest .146 0.016 <.001 .489 0.125
irritability anger loss of interest .075 0.014 <.001 .396 0.127
hypervigilant loss of interest .017 0.017 .294 .328 0.125
easily startled loss of interest .003 0.013 .845 .347 0.131
difficulty concentrating loss of interest .129 0.018 <.001 .454 0.137
sleep disturbance loss of interest .032 0.017 .067 .363 0.119
hopelessness loss of interest .055 0.023 .016 .400 0.152
emotional numbing feeling detached .299 0.017 <.001 .580 0.120
irritability anger feeling detached .087 0.018 <.001 .427 0.139
hypervigilant feeling detached .011 0.015 .472 .345 0.137
easily startled feeling detached .045 0.016 .006 .379 0.132
difficulty concentrating feeling detached .076 0.013 <.001 .457 0.124
sleep disturbance feeling detached .071 0.018 <.001 .393 0.134
hopelessness feeling detached .137 0.017 <.001 .464 0.098
irritability anger emotional numbing .098 0.017 <.001 .402 0.142
hypervigilant emotional numbing .006 0.014 .661 .312 0.134
Table I2Continued.
Implied Random-
easily startled emotional numbing .016 0.014 .267 .334 0.121
difficulty concentrating emotional numbing .050 0.015 .001 .415 0.115
sleep disturbance emotional numbing .011 0.020 .586 .343 0.127
hopelessness emotional numbing .160 0.017 <.001 .454 0.115
hypervigilant irritability anger .066 0.017 <.001 .347 0.131
easily startled irritability anger .051 0.016 .002 .367 0.138
difficulty concentrating irritability anger .121 0.017 <.001 .425 0.155
sleep disturbance irritability anger .122 0.021 <.001 .391 0.147
hopelessness irritability anger .024 0.020 .222 .336 0.147
easily startled hypervigilant .357 0.024 <.001 .555 0.164
difficulty concentrating hypervigilant .041 0.015 .007 .380 0.130
sleep disturbance hypervigilant .037 0.014 .007 .351 0.108
hopelessness hypervigilant .068 0.015 <.001 .327 0.104
difficulty concentrating easily startled .128 0.016 <.001 .436 0.135
sleep disturbance easily startled .070 0.012 <.001 .390 0.119
hopelessness easily startled -.037 0.020 .071 .304 0.143
sleep disturbance difficulty concentrating .131 0.019 <.001 .433 0.128
hopelessness difficulty concentrating .106 0.014 <.001 .408 0.097
hopelessness sleep disturbance .037 0.016 .019 .335 0.110
Page | 389
Cumulative Per year
25
20
15
Number of papers
10
2015 2016 2017 2018 2019
Figure I1 Overview of the cumulative number of articles, and overall number of articles published
per year.
Page | 390
Pearson − Listwise Pearson − Pairwise
intrusive thoughts 49 45 39 40 33 47 49 39 34 43 43 44 48 48 39 38 23 49 45 39 40 33 47 49 39 34 43 43 44 48 48 39 38 23
nightmares 45 46 37 36 33 45 46 36 34 43 44 40 46 46 35 35 23 45 46 37 36 33 45 46 36 34 43 44 40 46 46 35 35 23
flashbacks 39 37 40 35 32 40 40 38 34 35 36 38 40 40 38 38 23 39 37 40 35 32 40 40 38 34 35 36 38 40 40 38 38 23
psychological reactivity 40 36 35 40 32 39 40 34 30 35 35 39 39 39 35 34 19 40 36 35 40 32 39 40 34 30 35 35 39 39 39 35 34 19
physiological reactivity 33 33 32 32 33 32 33 32 30 30 31 32 33 33 31 31 19 33 33 32 32 33 32 33 32 30 30 31 32 33 33 31 31 19
internal avoidance 47 45 40 39 32 48 48 38 34 43 44 42 48 48 38 38 23 47 45 40 39 32 48 48 38 34 43 44 42 48 48 38 38 23
external avoidance 49 46 40 40 33 48 50 39 34 44 44 44 49 49 39 38 23 49 46 40 40 33 48 50 39 34 44 44 44 49 49 39 38 23
amnesia 39 36 38 34 32 38 39 39 34 34 35 39 39 39 38 38 23 39 36 38 34 32 38 39 39 34 34 35 39 39 39 38 38 23
loss of interest 34 34 34 30 30 34 34 34 34 34 34 34 34 34 34 34 23 34 34 34 30 30 34 34 34 34 34 34 34 34 34 34 34 23
feeling detached 43 43 35 35 30 43 44 34 34 44 43 39 43 43 35 34 23 43 43 35 35 30 43 44 34 34 44 43 39 43 43 35 34 23
emotional numbing 43 44 36 35 31 44 44 35 34 43 44 39 44 44 35 35 23 43 44 36 35 31 44 44 35 34 43 44 39 44 44 35 35 23
irritability anger 44 40 38 39 32 42 44 39 34 39 39 44 43 43 39 38 23 44 40 38 39 32 42 44 39 34 39 39 44 43 43 39 38 23
hypervigilant 48 46 40 39 33 48 49 39 34 43 44 43 49 49 38 38 23 48 46 40 39 33 48 49 39 34 43 44 43 49 49 38 38 23
easily startled 48 46 40 39 33 48 49 39 34 43 44 43 49 49 38 38 23 48 46 40 39 33 48 49 39 34 43 44 43 49 49 38 38 23
difficulty concentrating 39 35 38 35 31 38 39 38 34 35 35 39 38 38 39 38 23 39 35 38 35 31 38 39 38 34 35 35 39 38 38 39 38 23
sleep disturbance 38 35 38 34 31 38 38 38 34 34 35 38 38 38 38 38 23 38 35 38 34 31 38 38 38 34 34 35 38 38 38 38 38 23
hopelessness (DSM IV only) 23 23 23 19 19 23 23 23 23 23 23 23 23 23 23 23 23 23 23 23 19 19 23 23 23 23 23 23 23 23 23 23 23 23
Spearman − Listwise Spearman − Pairwise

intrusive thoughts 44 40 32 37 30 42 44 32 27 38 38 37 43 43 32 31 18 44 40 32 37 30 42 44 32 27 38 38 37 43 43 32 31 18
nightmares 40 40 29 33 30 39 40 29 27 37 38 33 40 40 28 28 18 40 40 29 33 30 39 40 29 27 37 38 33 40 40 28 28 18
flashbacks 32 29 32 32 29 32 32 31 27 27 28 31 32 32 31 31 18 32 29 32 32 29 32 32 31 27 27 28 31 32 32 31 31 18
psychological reactivity 37 33 32 37 29 36 37 31 27 32 32 36 36 36 32 31 18 37 33 32 37 29 36 37 31 27 32 32 36 36 36 32 31 18
physiological reactivity 30 30 29 29 30 29 30 29 27 27 28 29 30 30 28 28 18 30 30 29 29 30 29 30 29 27 27 28 29 30 30 28 28 18
internal avoidance 42 39 32 36 29 42 42 31 27 37 38 35 42 42 31 31 18 42 39 32 36 29 42 42 31 27 37 38 35 42 42 31 31 18
external avoidance 44 40 32 37 30 42 44 32 27 38 38 37 43 43 32 31 18 44 40 32 37 30 42 44 32 27 38 38 37 43 43 32 31 18
amnesia 32 29 31 31 29 31 32 32 27 27 28 32 32 32 31 31 18 32 29 31 31 29 31 32 32 27 27 28 32 32 32 31 31 18
loss of interest 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 18 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 18
feeling detached 38 37 27 32 27 37 38 27 27 38 37 32 37 37 28 27 18 38 37 27 32 27 37 38 27 27 38 37 32 37 37 28 27 18
emotional numbing 38 38 28 32 28 38 38 28 27 37 38 32 38 38 28 28 18 38 38 28 32 28 38 38 28 27 37 38 32 38 38 28 28 18
irritability anger 37 33 31 36 29 35 37 32 27 32 32 37 36 36 32 31 18 37 33 31 36 29 35 37 32 27 32 32 37 36 36 32 31 18
hypervigilant 43 40 32 36 30 42 43 32 27 37 38 36 43 43 31 31 18 43 40 32 36 30 42 43 32 27 37 38 36 43 43 31 31 18
easily startled 43 40 32 36 30 42 43 32 27 37 38 36 43 43 31 31 18 43 40 32 36 30 42 43 32 27 37 38 36 43 43 31 31 18
difficulty concentrating 32 28 31 32 28 31 32 31 27 28 28 32 31 31 32 31 18 32 28 31 32 28 31 32 31 27 28 28 32 31 31 32 31 18
sleep disturbance 31 28 31 31 28 31 31 31 27 27 28 31 31 31 31 31 18 31 28 31 31 28 31 31 31 27 27 28 31 31 31 31 31 18
hopelessness (DSM IV only) 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18 18
intrusive thoughts
nightmares
flashbacks
internal avoidance
external avoidance
amnesia
loss of interest
feeling detached
emotional numbing
irritability anger
hypervigilant
easily startled
sleep disturbance
intrusive thoughts
nightmares
flashbacks
internal avoidance
external avoidance
amnesia
loss of interest
feeling detached
emotional numbing
irritability anger
hypervigilant
easily startled
sleep disturbance
Figure I2 Number of samples for each pair of variables for which different types of correlations were available.
Page | 391
Figure I3 Estimated edge weights in the pooled MAGNA and 95% confidence regions based on the
estimated standard. The α=.0004 level corresponds to a Bonferroni corrected α level of .05 rounded to
four digits.
Page | 392
Edge weights saturated network
Shared DSM−IV & DSM−5 symptoms DSM−IV symptoms
Dependent samples collapsed Dependent samples not collapsed Dependent samples collapsed Dependent samples not collapsed
Pearson Spearman Pearson Spearman Pearson Spearman Pearson Spearman
listwise pairwise listwise pairwise listwise pairwise listwise pairwise listwise pairwise listwise pairwise listwise pairwise listwise pairwise
sleep_disturbance −− psychological_reactivity
sleep_disturbance −− physiological_reactivity
sleep_disturbance −− nightmares
sleep_disturbance −− loss_of_interest
sleep_disturbance −− irritability_anger
sleep_disturbance −− intrusive_thoughts
sleep_disturbance −− internal_avoidance
sleep_disturbance −− hypervigilant
sleep_disturbance −− flashbacks
sleep_disturbance −− feeling_detached
sleep_disturbance −− external_avoidance
sleep_disturbance −− emotional_numbing
sleep_disturbance −− easily_startled
sleep_disturbance −− difficulty_concentrating
sleep_disturbance −− amnesia
psychological_reactivity −− nightmares
psychological_reactivity −− intrusive_thoughts
psychological_reactivity −− flashbacks
physiological_reactivity −− psychological_reactivity
physiological_reactivity −− nightmares
physiological_reactivity −− intrusive_thoughts
physiological_reactivity −− flashbacks
nightmares −− intrusive_thoughts
loss_of_interest −− psychological_reactivity
loss_of_interest −− physiological_reactivity
loss_of_interest −− nightmares
loss_of_interest −− intrusive_thoughts
loss_of_interest −− internal_avoidance
loss_of_interest −− flashbacks
loss_of_interest −− external_avoidance
loss_of_interest −− amnesia
irritability_anger −− psychological_reactivity
irritability_anger −− physiological_reactivity
irritability_anger −− nightmares
irritability_anger −− loss_of_interest
irritability_anger −− intrusive_thoughts
irritability_anger −− internal_avoidance
irritability_anger −− flashbacks
irritability_anger −− feeling_detached
irritability_anger −− external_avoidance
irritability_anger −− emotional_numbing
irritability_anger −− amnesia
internal_avoidance −− psychological_reactivity
internal_avoidance −− physiological_reactivity
internal_avoidance −− nightmares
internal_avoidance −− intrusive_thoughts
internal_avoidance −− flashbacks
hypervigilant −− psychological_reactivity
hypervigilant −− physiological_reactivity
hypervigilant −− nightmares
hypervigilant −− loss_of_interest
hypervigilant −− irritability_anger
hypervigilant −− intrusive_thoughts
hypervigilant −− internal_avoidance
hypervigilant −− flashbacks
hypervigilant −− feeling_detached
hypervigilant −− external_avoidance
hypervigilant −− emotional_numbing
hypervigilant −− amnesia
hopelessness −− sleep_disturbance
hopelessness −− psychological_reactivity
hopelessness −− physiological_reactivity
hopelessness −− nightmares
hopelessness −− loss_of_interest
hopelessness −− irritability_anger
hopelessness −− intrusive_thoughts
hopelessness −− internal_avoidance
hopelessness −− hypervigilant
hopelessness −− flashbacks
hopelessness −− feeling_detached
hopelessness −− external_avoidance
hopelessness −− emotional_numbing
hopelessness −− easily_startled
hopelessness −− difficulty_concentrating
hopelessness −− amnesia
flashbacks −− nightmares
flashbacks −− intrusive_thoughts
feeling_detached −− psychological_reactivity
feeling_detached −− physiological_reactivity
feeling_detached −− nightmares
feeling_detached −− loss_of_interest
feeling_detached −− intrusive_thoughts
feeling_detached −− internal_avoidance
feeling_detached −− flashbacks
feeling_detached −− external_avoidance
feeling_detached −− amnesia
external_avoidance −− psychological_reactivity
external_avoidance −− physiological_reactivity
external_avoidance −− nightmares
external_avoidance −− intrusive_thoughts
external_avoidance −− internal_avoidance
external_avoidance −− flashbacks
emotional_numbing −− psychological_reactivity
emotional_numbing −− physiological_reactivity
emotional_numbing −− nightmares
emotional_numbing −− loss_of_interest
emotional_numbing −− intrusive_thoughts
emotional_numbing −− internal_avoidance
emotional_numbing −− flashbacks
emotional_numbing −− feeling_detached
emotional_numbing −− external_avoidance
emotional_numbing −− amnesia
easily_startled −− psychological_reactivity
easily_startled −− physiological_reactivity
easily_startled −− nightmares
easily_startled −− loss_of_interest
easily_startled −− irritability_anger
easily_startled −− intrusive_thoughts
easily_startled −− internal_avoidance
easily_startled −− hypervigilant
easily_startled −− flashbacks
easily_startled −− feeling_detached
easily_startled −− external_avoidance
easily_startled −− emotional_numbing
easily_startled −− amnesia
difficulty_concentrating −− psychological_reactivity
difficulty_concentrating −− physiological_reactivity
difficulty_concentrating −− nightmares
difficulty_concentrating −− loss_of_interest
difficulty_concentrating −− irritability_anger
difficulty_concentrating −− intrusive_thoughts
difficulty_concentrating −− internal_avoidance
difficulty_concentrating −− hypervigilant
difficulty_concentrating −− flashbacks
difficulty_concentrating −− feeling_detached
difficulty_concentrating −− external_avoidance
difficulty_concentrating −− emotional_numbing
difficulty_concentrating −− easily_startled
difficulty_concentrating −− amnesia
amnesia −− psychological_reactivity
amnesia −− physiological_reactivity
amnesia −− nightmares
amnesia −− intrusive_thoughts
amnesia −− internal_avoidance
amnesia −− flashbacks
amnesia −− external_avoidance
individual (averaged)
individual (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
pooled (averaged)
pooled (per_study)
Figure I4 Multiverse plot displaying results for distinct MAGNA estimation procedures and data
variants. Each box shows the edge weight estimated using different settings and different data sources
in MAGNA.
Page | 393
strength closeness betweenness
intrusive thoughts
nightmares
flashbacks
internal avoidance
external avoidance
amnesia
loss of interest
feeling detached
emotional numbing
irritability anger
hypervigilant
easily startled
sleep disturbance

0.0
0.3
0.6
0.9
1.2
0.000
0.001
0.002
0.003
0.004
0.005
20
40
Figure I5 Multiverse analysis of centrality indices. Each line corresponds to the centrality results of
one of the variants reported in Figure I4.
Page | 394
Birkland et al. (2020) MAGNA network Duek et al. (2020)
psychological psychological psychological
reactivity reactivity reactivity
physiological physiological physiological
internal reactivity internal reactivity internal reactivity
avoidance avoidance avoidance
flashbacks flashbacks flashbacks

external external external
avoidance avoidance avoidance
intrusive intrusive intrusive
thoughts thoughts thoughts
easily easily easily
hypervigilant startled hypervigilant startled hypervigilant startled
nightmares nightmares nightmares
difficulty difficulty difficulty

concentrating concentrating concentrating
sleep sleep sleep

disturbance disturbance disturbance
amnesia irritability amnesia irritability amnesia irritability
anger anger anger
hopelessness hopelessness hopelessness
feeling loss of feeling loss of feeling loss of

detached interest detached interest detached interest
emotional emotional emotional

numbing numbing numbing
Figure I6 This figure compares the pooled MAGNA network (middle panel) to results from the literature review presented by Birkland and colleagues (2020,
top panel) and a recent large-scale PTSD symptom network estimated by Duek and colleagues (2020, bottom panel) on.a sample of 158,139 veterans with PTSD.
The top panel is a visual representation of the table presented in Figure 3 of Birkeland et al. (2020).
Page | 395
MAGNA Duek et al. (2020)
Strength Expected Influence R2 (Predictability) Closeness Betweenness
intrusive thoughts
nightmares
flashbacks
internal avoidance
external avoidance
amnesia
loss of interest
feeling detached
emotional numbing
irritability anger
hypervigilant
easily startled
sleep disturbance

0.0
0.5
1.0
0.0
0.3
0.6
0.9
1.2
0.0
0.2
0.4
0.6
0.000
0.001
0.002
0.003
0.004
0.005
0
10
20
30
Figure I7 Centrality coefficients of the pooled MAGNA network compared to the results from Duek
and colleagues (2020).
Page | 396
References
Summary
Nederlandse Samenvatting
Funding, Publications, and Short CV
Acknowledgements - Dankwoord
References
References
Abacioglu, C. S., Isvoranu, A.–M., Verkuyten, M., Thijs, J., & Epskamp, S. (2019). Exploring multicultural
classroom dynamics: A network analysis. Journal of School Psychology, 74, 90–105.
Addington, J., & Duchak, V. (1997). Reasons for substance use in schizophrenia. Acta Psychiatrica
Scandinavica, 96(5), 329–333.
Addington, J., Piskulic, D., Liu, L., Lockwood, J., Cadenhead, K. S., Cannon, T. D., Cornblatt, B. A.,
McGlashan, T. H., Perkins, D. O., Seidman, L. J., Tsuang, M. T., Walker, E. F., Bearden, C. E.,
Mathalon, D. H., & Woods, S. W. (2017). Comorbid diagnoses for youth at clinical high risk of
psychosis. Schizophrenia Research, 190, 90–95.
Albert, U., Tomassi, S., Maina, G., & Tosato, S. (2018). Prevalence of non–psychotic disorders in ultra–
high risk individuals and transition to psychosis: A systematic review. Psychiatry Research, 270, 1–12.
Aleman, A., Böcker, K. B. E., Hijman, R., De Haan, E. H. F., & Kahn, R. S. (2003). Cognitive basis of
hallucinations in schizophrenia: Role of top–down information processing. Schizophrenia Research,
64(2–3), 175–185.
Alves–Pinto, A., Rus, O. G., Reess, T. J., Wohlschläger, A., Wagner, G., Berberich, G., & Koch, K.
(2019). Altered reward–related effective connectivity in obsessive–compulsive disorder: An fMRI
study. Journal of Psychiatry and Neuroscience, 44(6), 395–406.
American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders (4th ed.).
American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (5th ed.).
American Psychological Association. (2020). Publication Manual of the American Psychological
Association (7th ed.).
Ansell, E. B., Laws, H. B., Roche, M. J., & Sinha, R. (2015). Effects of marijuana use on impulsivity and
hostility in daily life. Drug and Alcohol Dependence, 148, 136–142.
Appelbaum, M., Cooper, H., Kline, R. B., Mayo–Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal
article reporting standards for quantitative research in psychology: The APA publications and
Communications Board task force report. American Psychologist, 73(1), 3–25.
Archie, S., & Gyomorey, K. (2009). First episode psychosis, substance abuse and prognosis: A systematic
review. Current Psychiatry Reviews, 5(3), 153–163.
Armour, C., Fried, E. I., Deserno, M. K., Tsai, J., & Pietrzak, R. H. (2017). A network analysis of DSM–5
posttraumatic stress disorder symptoms and correlates in U.S. military veterans. Journal of Anxiety
Disorders, 45, 49–59.
Armour, C., Mullerová, J., & Elhai, J. D. (2016). A systematic literature review of PTSD’s latent structure
in the Diagnostic and Statistical Manual of Mental Disorders: DSM–IV to DSM–5. Clinical
Psychology Review, 44, 60–74.
Arseneault, L., Cannon, M., Poulton, R., Murray, R., Caspi, A., & Moffitt, T. E. (2002). Cannabis use
in adolescence and risk for adult psychosis: longitudinal prospective study. BMJ (Clinical Research
Ed.), 325(7374), 1212–1213.
Bak, M., Drukker, M., Hasmi, L., & Van Jim, O. S. (2016). An n=1 clinical network analysis of symptoms
and treatment in psychosis. PLoS ONE, 11(9), e0162811.
Ballenger, J. C. (2004). Remission rates in patients with anxiety disorders treated with paroxetine.
Journal of Clinical Psychiatry, 65(12), 1696–1707.
Barlati, S., Deste, G., Gregorelli, M., & Vita, A. (2019). Autistic traits in a sample of adult patients with
schizophrenia: Prevalence and correlates. Psychological Medicine, 49(1), 140–148.
Page | 398
Baron–Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001). The Autism–Spectrum
Quotient (AQ): Evidence from Asperger Syndrome/High–Functioning Autism, males and females,
scientists and mathematicians. Journal of Autism and Developmental Disorders, 31(1), 5–17.
Barrat, A., Barthélemy, M., Pastor–Satorras, R., & Vespignani, A. (2004). The architecture of complex
weighted networks. Proceedings of the National Academy of Sciences of the United States of America,
101(11), 3747–3752.
Beard, C., Millner, A. J., Forgeard, M. J. C., Fried, E. I., Hsu, K. J., Treadway, M., Leonard, C. V, Kertz,
S., & Björgvinsson, T. (2016). Network analysis of depression and anxiety symptom relations in a
psychiatric sample. Psychological Medicine, 46(16), 3359–3369.
Bebbington, P., Jonas, S., Kuipers, E., King, M., Cooper, C., Brugha, T., Meltzer, H., McManus, S., &
Jenkins, R. (2011). Childhood sexual abuse and psychosis: Data from a cross–sectional national
psychiatric survey in England. British Journal of Psychiatry, 199(1), 29–37.
Bechi, M., Agostoni, G., Buonocore, M., Bosinelli, F., Spangaro, M., Bianchi, L., Cocchi, F., Guglielmino,
C., Cavallaro, R., & Bosia, M. (2020). The Influence of premorbid adjustment and autistic traits
on social cognitive dysfunction in schizophrenia. Journal of the International Neuropsychological
Society, 26(3), 276–285.
Ben–Zeev, D., Ellington, K., Swendsen, J., & Granholm, E. (2011). Examining a cognitive model of
persecutory ideation in the daily life of people with schizophrenia: A computerized experience
sampling study. Schizophrenia Bulletin, 37(6), 1248–1256.
Benet–Martínez, V., & John, O. P. (1998). Los Cinco Grandes across cultures and ethnic groups:
Multitrait multimethod analyses of the Big Five in Spanish and English. Journal of Personality and
Social Psychology, 75(3), 729–750.
Bentall, R. P., Jackson, H. F., & Pilgrim, D. (1988). Abandoning the concept of “schizophrenia”: Some
implications of validity arguments for psychological research into psychotic phenomena. British
Journal of Clinical Psychology, 27(4), 303–324.
Bentall, R. P., Wickham, S., Shevlin, M., & Varese, F. (2012). Do specific early–life adversities lead to
specific symptoms of psychosis? A study from the 2007 the Adult Psychiatric Morbidity Survey.
Schizophrenia Bulletin, 38(4), 734–740.
Bernstein, D. P., & Fink, L. (1998). Childhood trauma questionnaire: A retrospective self–report: Manual.
Psychological Corporation.
Bernstein, D. P., Stein, J. A., Newcomb, M. D., Walker, E., Pogge, D., Ahluvalia, T., Stokes, J., Handelsman,
L., Medrano, M., Desmond, D., & Zule, W. (2003). Development and validation of a brief screening
version of the Childhood Trauma Questionnaire. Child Abuse and Neglect, 27(2), 169–190.
Berntsen, D., & Rubin, D. C. (2014). Involuntary memories and dissociative amnesia: Assessing key
assumptions in posttraumatic stress disorder research. Clinical Psychological Science, 2(2), 174–186.
Bersani, G., Orlandi, V., Kotzalidis, G. D., & Pancheri, P. (2002). Cannabis and schizophrenia: Impact
on onset, course, psychopathology and outcomes. European Archives of Psychiatry and Clinical
Neuroscience, 252(2), 86–92.
Birchwood, M., Meaden, A., Trower, P., Gilbert, P., & Plaistow, J. (2000). The power and omnipotence
of voices: subordination and entrapment by voices and significant others. Psychological Medicine,
30(2), 337–344.
Birchwood, M., Smith, J., Cochrane, R., Wetton, S., & Copestake, S. (1990). The Social Functioning Scale.
The development and validation of a new scale of social adjustment for use in family intervention
programmes with schizophrenic patients. British Journal of Psychiatry, 157(6), 853–859.
Birkeland, M. S., Greene, T., & Spiller, T. R. (2020). The network approach to posttraumatic stress
disorder: a systematic review. European Journal of Psychotraumatology, 11(1), 1700614.
Page | 399
References
Blanken, T. F., van der Zweerde, T., van Straten, A., van Someren, E. J. W., Borsboom, D., & Lancee,
J. (2019). Introducing network intervention analysis to investigate sequential, symptom–specific
treatment effects: A demonstration in co–occurring insomnia and depression. Psychotherapy and
Psychosomatics, 88(1), 52–54.
Bleuler, E. (1911). Dementia praecox oder die Gruppe der Schizophrenien. Deuticke.
Boccaletti, S., Latora, V., Moreno, Y., Chavez, M., & Hwang, D.–U. (2006). Complex networks:
Structure and dynamics. Physics Reports, 424(4–5), 175–308.
Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology,
64(9), 1089–1108.
Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 16(1), 5–13.
Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: an integrative approach to the structure
of psychopathology. Annual Review of Clinical Psychology, 9, 91–121.
Borsboom, D., Cramer, A. O. J., & Kalis, A. (2019). Brain disorders? Not really: Why network structures
block reductionism in psychopathology research. Behavioral and Brain Sciences, 42(e2), 1–63.
Borsboom, D., Cramer, A. O. J., Schmittmann, V. D., Epskamp, S., & Waldorp, L. J. (2011). The small
world of psychopathology. PLoS ONE, 6(11), e27407.
Borsboom, D., Fried, E. I., Epskamp, S., Waldorp, L. J., van Borkulo, C. D., van der Maas, H.
L. J., & Cramer, A. O. J. A. (2017). False alarm? A comprehensive reanalysis of “evidence that
psychopathology symptom networks have limited replicability” by Forbes, Wright, Markon, and
Krueger (2017). Journal of Abnormal Psychology, 126(7), 989–999.
Borsboom, D., Robinaugh, D. J., Rhemtulla, M., & Cramer, A. O. J. (2018). Robustness and replicability
of psychopathology networks. World Psychiatry, 17(2), 143–144.
Bos, F. M., Snippe, E., de Vos, S., Hartmann, J. A., Simons, C. J. P., van der Krieke, L., de Jonge, P., & Wichers,
M. (2017). Can we jump from cross–sectional to d0ynamic interpretations of networks? Implications for
the network perspective in psychiatry. Psychotherapy and Psychosomatics, 86(3), 175–177.
Boschloo, L., Schoevers, R. A., van Borkulo, C. D., Borsboom, D., & Oldehinkel, A. J. (2016). The
network structure of psychopathology in a community sample of preadolescents. Journal of
Abnormal Psychology, 125(4), 599–606.
Boschloo, L., van Borkulo, C. D., Borsboom, D., & Schoevers, R. A. (2016). A prospective study on how
symptoms in a network predict the onset of depression. Psychotherapy and Psychosomatics, 85(3), 183–184.
Boschloo, L., van Borkulo, C. D., Rhemtulla, M., Keyes, K. M., Borsboom, D., & Schoevers, R. A. (2015).
The network structure of symptoms of the diagnostic and statistical manual of mental disorders.
PLoS ONE, 10(9), e0137621.
Bourgeois, J., & Mistry, H. (2010). Migraine–associated psychosis and subsequent renal transplant.
Psychosomatics, 51(1), 77–79.
Bovasso, G. B. (2001). Cannabis abuse as a risk factor for depressive symptoms. The American Journal
of Psychiatry, 158(12), 2033–2037.
Boyette, L.–L., Isvoranu, A.–M., Schirmbeck, F., Velthorst, E., Simons, C. J. P., Barrantes–Vidal, N.,
Bressan, R., Kempton, M. J., Krebs, M.–O., McGuire, P., Nelson, B., Nordentoft, M., Riecher–
Rössler, A., Ruhrmann, S., Rutten, B. P., Sachs, G., Valmaggia, L. R., van der Gaag, M., Borsboom,
D., … van Os, J. (2020). From speech illusions to onset of psychotic disorder: Applying network
analysis to an experimental measure of aberrant experiences. Schizophrenia Bulletin Open, 1(1),
sgaa025.
Brandes, U. (2008). On variants of shortest–path betweenness centrality and their generic computation.
Social Networks, 30(2), 136–145.
Page | 400
Bringmann, L. F., Elmer, T., Epskamp, S., Krause, R. W., Schoch, D., Wichers, M., Wigman, J. T. W.,
& Snippe, E. (2019). What do centrality measures measure in psychological networks? Journal of
Abnormal Psychology, 128(8), 892–903.
Bringmann, L. F., Lemmens, L. H. J. M., Huibers, M. J. H., Borsboom, D., & Tuerlinckx, F. (2015).
Revealing the dynamic network structure of the Beck Depression Inventory–II. Psychological
Medicine, 45(04), 747–757.
Bringmann, L. F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., Peeters, F., Borsboom, D.,
& Tuerlinckx, F. (2013). A network approach to psychopathology: new insights into clinical
longitudinal data. PLoS One, 8(4), e60188.
Brown, S. (1997). Excess mortality of schizophrenia. A meta–analysis. British Journal of Psychiatry,
171(6), 502–508.
Buist–Bouwman, M. A., De Graaf, R., Vollebergh, W. A. M., & Ormel, J. (2005). Comorbidity of
physical and mental disorders and the effect on work–loss days. Acta Psychiatrica Scandinavica,
111(6), 436–443.
Burger, J., Stroebe, M. S., Perrig–Chiello, P., Schut, H. A., Spahni, S., Eisma, M. C., & Fried, E. I. (2020).
Bereavement or breakup: Differences in networks of depression. Journal of Affective Disorders, 267, 1–8.
Burger, J., van der Veen, D. C., Robinaugh, D. J., Quax, R., Riese, H., Schoevers, R. A., & Epskamp, S.
(2020). Bridging the gap between complexity science and clinical practice by formalizing idiographic
theories: A computational model of functional analysis. BMC Medicine, 18, 1–18.
Buyukdura, J. S., McClintock, S. M., & Croarkin, P. E. (2011). Psychomotor retardation in depression:
biological underpinnings, measurement, and treatment. Progress in Neuro–Psychopharmacology &
Biological Psychiatry, 35(2), 395–409.
Carpenter, L. L., Tyrka, A. R., Ross, N. S., Khoury, L., Anderson, G. M., & Price, L. H. (2009). Effect
of childhood emotional abuse and age on cortisol responsivity in adulthood. Biological Psychiatry,
66(1), 69–75.
Catalan, A., Simons, C. J. P., Bustamante, S., Drukker, M., Madrazo, A., De Artaza, M. G., Gorostiza,
I., van Os, J., & Gonzalez–Torres, M. A. (2014). Novel evidence hat attributing affectively salient
signal to random noise is associated with psychosis. PLoS ONE, 9(7), e102520.
Chandrasekhar, T., Copeland, J. N., Spanos, M., & Sikich, L. (2020). Autism, psychosis, or both?
Unraveling complex patient presentations. Child and Adolescent Psychiatric Clinics of North America,
29(1), 103–113.
Chapman, L. J., Chapman, J. P., Numbers, J. S., Edell, W. S., Carpenter, B. N., & Beckfield, D. (1984).
Impulsive nonconformity as a trait contributing to the prediction of psychotic–like and schizotypal
symptoms. The Journal of Nervous and Mental Disease, 172(11), 681–691.
Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large
model spaces. Biometrika, 95(3), 759–771.
Cheung, M. W. L. (2015). Meta–analysis: A structural equation modeling approach. John Wiley & Sons, Inc.
Cheung, M. W. L., & Chan, W. (2005). Meta–analytic structural equation modeling: A two–stage
approach. Psychological Methods, 10(1), 40–64.
Chisholm, K., Lin, A., Abu–Akel, A., & Wood, S. J. (2015). The association between autism and
schizophrenia spectrum disorders: A review of eight alternate models of co–occurrence. Neuroscience
and Biobehavioral Reviews, 55, 173–183.
Choi, K. W., Batchelder, A. W., Ehlinger, P. P., Safren, S. A., & O’Cleirigh, C. (2017). Applying network
analysis to psychological comorbidity and health behavior: Depression, PTSD, and sexual risk in sexual
minority men with trauma histories. Journal of Consulting and Clinical Psychology, 85(12), 1158.
Page | 401
References
Chong, S. A., Abdin, E., Sherbourne, C., Vaingankar, J., Heng, D., Yap, M., & Subramaniam, M.
(2012). Treatment gap in common mental disorders: The Singapore perspective. Epidemiology and
Psychiatric Sciences, 21(2), 195–202.
Chong, S. A., Abdin, E., Vaingankar, J. A., Heng, D., Sherbourne, C., Yap, M., Lim, Y. W., Wong, H.
B., Ghosh–Dastidhar, B., Kwok, K. W., & Subramaniam, M. (2012). A population–based survey of
mental disorders in Singapore. Annals of the Academy of Medicine Singapore, 41(2), 49–66.
Colombo, D., & Maathuis, M. H. (2015). Order–independent constraint–based causal structure
learning. Journal of Machine Learning Research, 15(1), 3741–3782.
Contreras, A., Nieto, I., Valiente, C., Espinosa, R., & Vazquez, C. (2019). The study of psychopathology from
the network analysis perspective: A systematic review. Psychotherapy and Psychosomatics, 88(2), 71–83.
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., & Cramer, A. O.
J. (2015). State of the aRt personality research: A tutorial on network analysis of personality data
in R. Journal of Research in Personality, 54, 13–29.
Costantini, G., & Perugini, M. (2016). The network of conscientiousness. Journal of Research in
Personality, 65, 68–88.
Craddock, N., & Owen, M. J. (2005). The beginning of the end for the Kraepelinian dichotomy. British
Journal of Psychiatry, 186(5), 364–366.
Cramer, A. O. J. (2012). The pathoplasticity of dysphoric episodes: differential impact of stressful life events
on the pattern of depressive symptom inter–correlations. Psychological Medicine, 42(5), 957–965.
Cramer, A. O. J., van der Sluis, S., Noordhof, A., Wichers, M., Geschwind, N., Aggen, S. H., Kendler, K.
S., & Borsboom, D. (2012). Dimensions of normal personality as networks in search of equilibrium:
You can’t like parties if you don’t like people. European Journal of Personality, 26(4), 414–431.
Cramer, A. O. J., Waldorp, L. J., van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: a network
perspective. The Behavioral and Brain Sciences, 33(2–3), 137–193.
D’Souza, D. C., Abi–Saab, W. M., Madonick, S., Forselius–Bielen, K., Doersch, A., Braley, G., Gueorguieva,
R., Cooper, T. B., & Krystal, J. H. (2005). Delta–9–tetrahydrocannabinol effects in schizophrenia:
Implications for cognition, psychosis, and addiction. Biological Psychiatry, 57(6), 594–608.
Daalman, K., Verkooijen, S., Derks, E. M., Aleman, A., & Sommer, I. E. C. (2012). The influence of semantic
top–down processing in auditory verbal hallucinations. Schizophrenia Research, 139(1–3), 82–86.
Dai, J., Du, X., Yin, G., Zhang, Y., Xia, H., Li, X., Cassidy, R., Tong, Q., Chen, D., Teixeira, A. L., Zheng,
Y., Ning, Y., Soares, J. C., He, M. X., & Zhang, X. Y. (2018). Prevalence, demographic and clinical
features of comorbid depressive symptoms in drug naïve patients with schizophrenia presenting
with first episode psychosis. Schizophrenia Research, 193, 182–187.
Dannlowski, U., Stuhrmann, A., Beutelmann, V., Zwanzger, P., Lenzen, T., Grotegerd, D., Domschke, K.,
Hohoff, C., Ohrmann, P., Bauer, J., Lindner, C., Postert, C., Konrad, C., Arolt, V., Heindel, W., Suslow,
T., & Kugel, H. (2012). Limbic scars: long–term consequences of childhood maltreatment revealed by
functional and structural magnetic resonance imaging. Biological Psychiatry, 71(4), 286–293.
Dazzi, F., Shafer, A., & Lauriola, M. (2016). Meta–analysis of the Brief Psychiatric Rating Scale –
Expanded (BPRS–E) structure and arguments for a new version. Journal of Psychiatric Research,
81, 140–151.
de Artaza, M. G., Catalan, A., Angosto, V., Valverde, C., Bilbao, A., van Os, J., & Gonzalez–Torres,
M. A. (2018). Can an experimental white noise task assess psychosis vulnerability in adult healthy
controls? PLoS ONE, 13(2), 1–8.
de Crescenzo, F., Postorino, V., Siracusano, M., Riccioni, A., Armando, M., Curatolo, P., & Mazzone,
L. (2019). Autistic symptoms in schizophrenia spectrum disorders: A systematic review and meta–
analysis. Frontiers in Psychiatry, 10(78), 1–11.
Page | 402
de Haan, L., & Zink, M. (2015). Clinical presentation of obsessive–compulsive symptoms in patients
with psychotic disorders. Psychopathological concepts, differential diagnosis, and symptom
presentation. In L. de Haan, F. Schirmbeck, & M. Zink (Eds.), Obsessive–Compulsive Symptoms in
Schizophrenia (pp. 33–45). Springer International Publishing.
de Ron, J., Fried, E. I., & Epskamp, S. (2019). Psychological networks in clinical populations:
Investigating the consequences of Berkson’s bias. Psychological Medicine, 51(1), 168–176.
Dekker, N., Linszen, D. H., & De Haan, L. (2009). Reasons for cannabis use and effects of cannabis use
as reported by patients with psychotic disorders. Psychopathology, 42(6), 350–360.
Deserno, M. K., Borsboom, D., Begeer, S., & Geurts, H. M. (2016). Multicausal systems ask for
multicausal approaches: A network perspective on subjective well–being in individuals with autism
spectrum disorder. Autism, 21(8), 960–971.
Deste, G., Barlati, S., Gregorelli, M., Lisoni, J., Turrina, C., Valsecchi, P., & Vita, A. (2018). Looking
through autistic features in schizophrenia using the PANSS Autism Severity Score (PAUSS).
Psychiatry Research, 270, 764–768.
Deste, G., Vita, A., Nibbio, G., Penn, D. L., Pinkham, A. E., & Harvey, P. D. (2020). Autistic symptoms
and social cognition predict real–world outcomes in patients with schizophrenia. Frontiers in
Psychiatry, 11, 524.
Deste, G., Vita, A., Penn, D. L., Pinkham, A. E., Nibbio, G., & Harvey, P. D. (2020). Autistic symptoms
predict social cognitive performance in patients with schizophrenia. Schizophrenia Research, 215, 113–119.
Digman, J. M. (1989). Five robust trait dimensions: Development, stability, and utility. Journal of
Personality, 57(2), 195–214.
Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik,
1(1), 269–271.
Djelantik, A. A. A. M. J., Robinaugh, D. J., Kleber, R. J., Smid, G. E., & Boelen, P. A. (2020).
Symptomatology following loss and trauma: Latent class and network analyses of prolonged grief
disorder, posttraumatic stress disorder, and depression in a treatment–seeking trauma–exposed
sample. Depression and Anxiety, 37(1), 26–34.
Doherty, A. M., & Gaughran, F. (2014). The interface of physical and mental health. Social Psychiatry
and Psychiatric Epidemiology, 49(5), 673–682.
Drton, M., & Perlman, M. D. (2004). Model selection for Gaussian concentration graphs. Biometrika,
91(3), 591–602.
Dudbridge, F. (2013). Power and predictive accuracy of polygenic risk scores. PLoS Genetics, 9(3),
e1003348.
Duek, O., Spiller, T. R., Pietrzak, R. H., Fried, E. I., & Harpaz–Rotem, I. (2020). Network analysis of PTSD
and depressive symptoms in 158,139 treatment–seeking veterans with PTSD. Depression and Anxiety, 1–9.
Efron, B. (1979). Computers and the theory of statistics: Thinking the unthinkable. SIAM Review,
21(4), 460–480.
Epskamp, S. (2017). Network Psychometrics (Doctoral dissertation, University of Amsterdam). UvA–Dare.
https://pure.uva.nl/ws/files/9870201/Epskamp_FullDissertation.pdf
Epskamp, S. (2018). New features in qgraph 1.5. http://psychosystems.org/qgraph_1.5
Epskamp, S. (2019). Reproducibility and replicability in a fast–paced methodological world. Advances
in Methods and Practices in Psychological Science, 2(2), 145–155.
Epskamp, S. (2020a). Psychometric network models from time–series and panel data. Psychometrika,
85(1), 206–231.
Epskamp, S. (2020b). Psychonetrics. In CRAN (0.6). https://cran.r–project.org/web/packages/
psychonetrics/index.html
Page | 403
References
Epskamp, S. (2020c). Software updates: Bootnet 1.3 and psychonetrics 0.5. http:// psychonetrics.
org/2020/01/26/software–updates–bootnet–1–3–and–psychonetrics–0–5/#New_default_behavior
Epskamp, S., Borsboom, D., & Fried, E. I. (2017). Estimating psychological networks and their accuracy:
A tutorial paper. Behavior Research Methods, 50(1), 195–212.
Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph : Network
Visualizations of Relationships in Psychometric Data. Journal of Statistical Software, 48(4), 1–18.
Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological
Methods, 23(4), 617–634.
Epskamp, S., Isvoranu, A.–M., & Cheung, M. W. L. (2020). Meta–analytic Gaussian Network
Aggregation. PsyArXiv (Preprint), https://psyarxiv.com/236w8/.
Epskamp, S., Kruis, J., & Marsman, M. (2017). Estimating psychopathological networks: Be careful
what you wish for. PLoS ONE, 12(6), E0179891.
Epskamp, S., Maris, G. K. J., Waldorp, L. J., & Borsboom, D. (2018). Network Psychometrics. In P.
Irwing, D. Hughest, & T. Booth (Eds.), The Wiley Handbook of Psychometric Testing, 2 Volume
Set: A Multidisciplinary Reference on Survey, Scale and Test Development. (pp. 953–986). Wiley.
Epskamp, S., Rhemtulla, M., & Borsboom, D. (2017). Generalized Network Psychometrics: Combining
network and latent variable models. Psychometrika, 82(4), 904–927.
Epskamp, S., van Borkulo, C. D., Isvoranu, A.–M., Riese, H., & Cramer, A. O. J. (2018). Personalized
network modeling in psychopathology: The importance of contemporaneous and temporal
connections. Clinical Psychological Science, 6(3), 416–427.
Epskamp, S., Waldorp, L. J., Mõttus, R., & Borsboom, D. (2018). The Gaussian graphical model in
cross–sectional and time–series data. Multivariate Behavioral Research, 53(4), 453–480.
Euesden, J., Lewis, C. M., & O’Reilly, P. F. (2015). PRSice: Polygenic risk score software. Bioinformatics,
31(9), 1466–1468.
Feigenson, K. A., Gara, M. A., Roché, M. W., & Silverstein, S. M. (2014). Is disorganization a feature
of schizophrenia or a modifying influence: Evidence of covariation of perceptual and cognitive
organization in a non–patient sample. Psychiatry Research, 217(1–2), 1–8.
Fisher, A. J., & Boswell, J. F. (2016). Enhancing the personalization of psychotherapy with dynamic
assessment and modeling. Assessment, 23(4), 496–506.
Fisher, H. L., Craig, T. K., Fearon, P., Morgan, K., Dazzan, P., Lappin, J., Hutchinson, G., Doody,
G. A., Jones, P. B., Mcguffin, P., Murray, R. M., Leff, J., & Morgan, C. (2011). Reliability and
comparability of psychosis patients’ retrospective reports of childhood abuse. Schizophrenia Bulletin,
37(3), 546–553.
Flake, J. K., & Fried, E. I. (2019). Measurement schmeasurement: Questionable measurement practices
and how to avoid them. Psychological Science, 3(4), 456–465.
Fonseca–Pedrero, E., Ortuño, J., Debbané, M., Chan, R. C. K., Cicero, D., Zhang, L. C., Brenner, C.,
Barkus, E., Linscott, R. J., Kwapil, T., Barrantes–Vidal, N., Cohen, A., Raine, A., Compton, M. T.,
Tone, E. B., Suhr, J., Inchausti, F., Bobes, J., Fumero, A., … Fried, E. I. (2018). The network structure
of schizotypal personality traits. Schizophrenia Bulletin, 44(suppl_2), S468–S479.
Fontenelle, L. F., Lin, A., Pantelis, C., Wood, S. J., Nelson, B., & Yung, A. R. (2011). A longitudinal
study of obsessive–compulsive disorder in individuals at ultra–high risk for psychosis. Journal of
Psychiatric Research, 45(9), 1140–1145.
Fontenelle, L. F., Lin, A., Pantelis, C., Wood, S. J., Nelson, B., & Yung, A. R. (2012). Markers of
vulnerability to obsessive–compulsive disorder in an ultra–high risk sample of patients who
developed psychosis. Early Intervention in Psychiatry, 6(2), 201–206.
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. (2019a). Quantifying the reliability and
replicability of psychopathology network characteristics. Multivariate Behavioral Research, 1–19.
Page | 404
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2017a). Evidence that psychopathology
symptom networks have limited replicability. Journal of Abnormal Psychology, 126(7), 969.
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2017b). Further evidence that
psychopathology networks have limited replicability and utility: Response to Borsboom et al. and
Steinley et al. Journal of Abnormal Psychology, 126(7), 1011–1016.
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2019b). The network approach to
psychopathology: promise versus reality. World Psychiatry, 18(3), 272–273.
Forbes, M. K., Wright, A. G. C., Markon, K. E., & Krueger, R. F. (2021). On unreplicable inferences
in psychopathology symptom networks and the importance of unreliable parameter estimates.
Multivariate Behavioral Research, 1–9.
Fowler, I. L., Carr, V. J., Carter, N. T., & Lewin, T. J. (1998). Patterns of current and lifetime substance
use in schizophrenia. Schizophrenia Bulletin, 24(3), 443–455.
Foygel, R., & Drton, M. (2011). Bayesian model choice and information criteria in sparse generalized
linear models. ArXiv (Preprint), 1–37. http://arxiv.org/abs/1112.5635
Foygel, R., & Drton, M. (2010). Extended Bayesian information criteria for Gaussian graphical models.
Advances in Neural Information Processing Systems, 604–612.
Franić, S., Dolan, C. V., Borsboom, D., Hudziak, J. J., van Beijsterveldt, C. E. M., & Boomsma, D. I.
(2013). Can genetics help psychometrics? Improving dimensionality assessment through genetic
factor modeling. Psychological Methods, 18(3), 406–433.
Freeman, D., Garety, P. A., Kuipers, E., Bebbington, P. E., Fowler, D., & Dunn, G. (2004). Why do
people with delusions fail to choose more realistic explanations for their experiences? An empirical
investigation. Journal of Consulting and Clinical Psychology, 72(4), 617–680.
Frewen, P. A., Allen, S. L., Lanius, R. a, & Neufeld, R. W. J. (2012). Perceived causal relations: novel
methodology for assessing client attributions about causal associations between variables including
symptoms and functional impairment. Assessment, 19(4), 480–493.
Fried, E. I. (2015). Problematic assumptions have slowed down depression research: why symptoms, not
syndromes are the way forward. Frontiers in Psychology, 6(309), 1–11.
Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., Cramer, A. O. J., Epskamp, S.,
Tuerlinckx, F., Carr, D., & Stroebe, M. (2015). From loss to loneliness: The relationship between
bereavement and depressive symptoms. Journal of Abnormal Psychology, 124(2), 256–265.
Fried, E. I., & Cramer, A. O. J. (2017). Moving forward: Challenges and direction for psychopathological
network theory and methodology. Perspectives on Psychological Science, 12(6), 999–1020.
Fried, E. I., Eidhof, M. B., Palic, S., Costantini, G., Huisman–van Dijk, H. M., Bockting, C.
L. H., Engelhard, I., Armour, C., Nielsen, A. B. S., & Karstoft, K. I. (2018). Replicability and
generalizability of posttraumatic stress disorder (PTSD) networks: A cross–cultural multisite study
of PTSD symptoms in four trauma patient samples. Clinical Psychological Science, 6(3), 335–351.
Fried, E. I., Epskamp, S., Nesse, R. M., Tuerlinckx, F., & Borsboom, D. (2015). What are “good”
depression symptoms? Comparing the centrality of DSM and non–DSM symptoms of depression
in a network analysis. Journal of Affective Disorders, 189, 314–320.
Fried, E. I., & Nesse, R. M. (2014). The impact of individual depressive symptoms on impairment of
psychosocial functioning. PloS One, 9(2), e90311.
Fried, E. I., & Nesse, R. M. (2015). Depression is not a consistent syndrome: An investigation of unique
symptom patterns in the STAR*D study. Journal of Affective Disorders, 172, 96–102.
Fried, E. I., Nesse, R. M., Zivin, K., Guille, C., & Sen, S. (2014). Depression is more than the sum score of its
parts: Individual DSM symptoms have different risk factors. Psychological Medicine, 44(10), 2067–2076.
Page | 405
References
Fried, E. I., van Borkulo, C. D., Cramer, A. O. J., Boschloo, L., Schoevers, R. A., & Borsboom, D.
(2016). Mental disorders as networks of problems: a review of recent insights. Social Psychiatry and
Psychiatric Epidemiology, 58(12), 7250–7257.
Fried, E. I., van Borkulo, C. D., & Epskamp, S. (2020). On the importance of estimating parameter
uncertainty in Network Psychometrics: A response to Forbes et al. (2019). Multivariate Behavioral
Research, 1–6.
Fried, E. I., van Borkulo, C. D., Epskamp, S., Schoevers, R. A., Tuerlinckx, F., & Borsboom, D. (2016).
Measuring depression over time…or not? Lack of unidimensionality and longitudinal measurement
invariance in four common rating scales of depression. Psychological Assessment, 38(11), 1354–1367.
Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical
lasso. Biostatistics, 9(3), 432–441.
Fruchterman, T. M. J., & Reingold, E. M. (1991). Graph drawing by force‐directed placement. Software:
Practice and Experience, 21(11), 1129–1164.
Fuller, G. N., Marshall, A., Flint, J., Lewis, S., & Wise, R. J. S. (1993). Migraine madness: Recurrent
psychosis after migraine. Journal of Neurology Neurosurgery and Psychiatry, 56(5), 416–418.
Fusar–Poli, P., Bonoldi, I., Yung, A. R., Borgwardt, S., Kempton, M. J., Valmaggia, L., Barale, F.,
Caverzasi, E., & McGuire, P. (2012). Predicting psychosis: Meta–analysis of transition outcomes
in individuals at high clinical risk. Archives of General Psychiatry, 63(3), 220–229.
Fusar–Poli, P., Cappucciati, M., Bonoldi, I., Hui, L. M. C., Rutigliano, G., Stahl, D. R., Borgwardt, S.,
Politi, P., Mishara, A. L., Lawrie, S. M., Carpenter, W. T., & McGuire, P. K. (2016). Prognosis of
brief psychotic episodes a meta–analysis. JAMA Psychiatry, 73(3), 211–220.
Fusar–Poli, P., Hobson, R., Raduelli, M., & Balottin, U. (2012). Reliability and validity of the
Comprehensive Assessment of the at Risk Mental State, Italian version (CAARMS–I). Current
Pharmaceutical Design, 18(4), 386–391.
Fusar–Poli, P., Nelson, B., Valmaggia, L., Yung, A. R., & McGuire, P. K. (2014). Comorbid depressive
and anxiety disorders in 509 individuals with an at–risk mental state: Impact on psychopathology
and transition to psychosis. Schizophrenia Bulletin, 40(1), 120–131.
Fusar–Poli, P., Salazar De Pablo, G., Correll, C. U., Meyer–Lindenberg, A., Millan, M. J., Borgwardt,
S., Galderisi, S., Bechdolf, A., Pfennig, A., Kessing, L. V., Van Amelsvoort, T., Nieman, D. H.,
Domschke, K., Krebs, M. O., Koutsouleris, N., McGuire, P., Do, K. Q., & Arango, C. (2020).
Prevention of psychosis: Advances in detection, prognosis, and intervention. JAMA Psychiatry,
77(7), 755–765.
Fusar–Poli, P., Tantardini, M., De Simone, S., Ramella–Cravaro, V., Oliver, D., Kingdon, J., Kotlicka–
Antczak, M., Valmaggia, L., Lee, J., Millan, M. J., Galderisi, S., Balottin, U., Ricca, V., & McGuire,
P. (2017). Deconstructing vulnerability for psychosis: Meta–analysis of environmental risk factors
for psychosis in subjects at ultra high–risk. European Psychiatry, 40, 65–75.
Galderisi, S., Rucci, P., Kirkpatrick, B., Mucci, A., Gibertoni, D., Rocca, P., Rossi, A., Bertolino, A.,
Strauss, G. P., Aguglia, E., Bellomo, A., Murri, M. B., Bucci, P., Carpiniello, B., Comparelli,
A., Cuomo, A., De Berardis, D., Dell’Osso, L., Di Fabio, F., … Maj, M. (2018). Interplay among
psychopathologic variables, personal resources, context–related factors, and real–life functioning
in individuals with schizophrenia a network analysis. JAMA Psychiatry, 75(4), 396–404.
Galdos, M., Simons, C., Fernandez–Rivas, A., Wichers, M., Peralta, C., Lataster, T., Amer, G., Myin–
Germeys, I., Allardyce, J., Gonzalez–Torres, M. A., & van Os, J. (2011). Affectively salient meaning
in random noise: A task sensitive to psychosis liability. Schizophrenia Bulletin, 37(6), 1179–1186.
Gates, K. M., & Molenaar, P. C. M. (2012). Group search algorithm recovers effective connectivity maps
for individuals in homogeneous and heterogeneous samples. NeuroImage, 63(1), 310–319.
Page | 406
Gilbar, O. (2020). Examining the boundaries between ICD–11 PTSD/CPTSD and depression and
anxiety symptoms: A network analysis perspective. Journal of Affective Disorders, 262, 429–439.
Glaser, J. P., van Os, J., Portegijs, P. J. M., & Myin–Germeys, I. (2006). Childhood trauma and
emotional reactivity to daily life stress in adult frequent attenders of general practitioners. Journal
of Psychosomatic Research, 61(2), 229–236.
Goldberg, L. R. (1990). An alternative “description of personality”: The Big–Five factor structure.
Journal of Personality and Social Psychology, 59(6), 1216–1229.
Goldberg, L. R. (1993). The Structure of phenotypic personality traits. American Psychologist, 48(1), 26–34.
Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the
number of dimensions in psychological research. PLoS ONE, 12(6), e0174035.
Goodman, W. K., Price, L. H., Rasmussen, S. A., Mazure, C., Fleischmann, R. L., Hill, C. L., Heninger,
G. R., & Charney, D. S. (1989). The Yale–Brown Obsessive Compulsive Scale: I. Development, use,
and reliability. Archives of General Psychiatry, 46(11), 1006–1011.
Goodwin, R. D., Jacobi, F., & Thefeld, W. (2003). Mental disorders and asthma in the community.
Archives of General Psychiatry, 60(11), 1125–1130.
Goswami, S., Singh, G., Mattoo, S. K., & Basu, D. (2003). Courses of substance use and schizophrenia in
the dual–diagnosis patients: Is there a relationship? Indian Journal of Medical Sciences, 57(8), 338–346.
Gottesman, I. I., McGuffin, P., & Farmer, a E. (1987). Clinical genetics as clues to the “real” genetics of
schizophrenia (a decade of modest gains while playing for time). Schizophrenia Bulletin, 13(1), 23–47.
Gottesman, I. I., & Shields, J. (1972). Schizophrenia and genetics. A Twin Study Vantage Point. Academic Press.
Green, B., Young, R., & Kavanagh, D. (2005). Cannabis use and misuse prevalence among people with
psychosis. British Journal of Psychiatry, 187(4), 306–313.
Greenwood, T. A., Braff, D. L., Light, G. A., Cadenhead, I. S., Calkins, M. E., Dobie, D. J., Freedman,
R., Green, M. F., Gur, R. E., Gur, R. C., Mintz, J., Nuechterlein, K. H., Olincy, A., Radant, A. D.,
Seidman, L. J., Siever, L. J., Silverman, J. M., Stone, W. S., Swerdlow, N. R., … Schork, N. J. (2007).
Initial heritability analyses of endophenotypic measures for schizophrenia: The Consortium on the
Genetics of Schizophrenia. Archives of General Psychiatry, 64(11), 1242–1250.
Griffiths, M., Wardle, H., Orford, J., Sproston, K., & Erens, B. (2010). Gambling, alcohol, consumption,
cigarette smoking and health: Findings from the 2007 British Gambling Prevalence Survey.
Addiction Research and Theory, 18(2), 208–223.
Grisham, J. R., Fullana, M. A., Mataix–Cols, D., Moffitt, T. E., Caspi, A., & Poulton, R. (2011).
Risk factors prospectively associated with adult obsessive–compulsive symptom dimensions and
obsessive–compulsive disorder. Psychological Medicine, 41(12), 2495–1506.
Guillem, F., Satterthwaite, J., Pampoulova, T., & Stip, E. (2009). Relationship between psychotic and
obsessive compulsive symptoms in schizophrenia. Schizophrenia Research, 115(2–3), 358–362.
Guloksuz, S., Pries, L.–K., & van Os, J. (2017). Application of network methods for understanding
mental disorders: pitfalls and promise. Psychological Medicine, 47(16), 2743–2752.
Guloksuz, S., van Nierop, M., Bak, M., de Graaf, R., ten Have, M., van Dorsselaer, S., Gunther, N., Lieb, R.,
van Winkel, R., Wittchen, H.–U., & van Os, J. (2016). Exposure to environmental factors increases
connectivity between symptom domains in the psychopathology network. BMC Psychiatry, 16(1), 1–10.
Guloksuz, S., van Nierop, M., Lieb, R., Winkel, R. Van, Wittchen, H., & van Os, J. (2015). Evidence that
the presence of psychosis in non– psychotic disorder is environment–dependent and mediated by
severity of non–psychotic psychopathology. Psychological Medicine, 45(11), 2389–2401.
Guloksuz, S., & van Os, J. (2018). The slow death of the concept of schizophrenia and the painful birth
of the psychosis spectrum. Psychological Medicine, 48(2), 229–244.
Page | 407
References
Hallquist, M. N., Wright, A. G. C., & Molenaar, P. C. M. (2019). Problems with centrality measures
in psychopathology symptom networks: Why Network Psychometrics cannot escape psychometric
theory. Multivariate Behavioral Research, 56(2), 199–223.
Hambrecht, M., Häfner, H. T., & Apophänie, A. (1993). Ist Conrads Phasenmodel empirisch
begründbahr? Fortschritte Neurologie Psychiatrie, 61, 418–423.
Hanssen, M., Bak, M., Bijl, R., Vollebergh, W., & van Os, J. (2005). The incidence and outcome of
subclinical psychotic experiences in the general population. British Journal of Clinical Psychology,
44(2), 181–191.
Harvey, P. D., Deckler, E., Jones, M. T., Jarskog, L. F., Penn, D. L., & Pinkham, A. E. (2019). Autism
symptoms, depression, and active social avoidance in schizophrenia: Association with self–reports
and informant assessments of everyday functioning. Journal of Psychiatric Research, 115, 36–42.
Harvey, P. D., Koren, D., Reichenberg, A., & Bowie, C. R. (2005). Negative symptoms and cognitive
deficits: What is the nature of their relationship? Schizophrenia Bulletin, 32(2), 250–258.
Hasan, A., von Keller, R., Friemel, C. M., Hall, W., Schneider, M., Koethe, D., Leweke, F. M., Strube, W.,
& Hoch, E. (2020). Cannabis use and psychosis: a review of reviews. European Archives of Psychiatry
and Clinical Neuroscience, 270(4), 403–412.
Haslbeck, J. M. B. (2020). Estimating group differences in network models using moderation analysis.
PsyArXiv (Preprint), https://psyarxiv.com/926pv.
Haslbeck, J. M. B., Borsboom, D., & Waldorp, L. (2019). Moderated network models. Multivariate
Behavioral Research, 56(2), 256–287.
Haslbeck, J. M. B., Bringmann, L. F., & Waldorp, L. J. (2020). A tutorial on estimating Time–varying
vector autoregressive models. Multivariate Behavioral Research, 56(1), 120–149.
Haslbeck, J. M. B., & Fried, E. I. (2017). How predictable are symptoms in psychopathological networks?
A reanalysis of 18 published datasets. Psychological Medicine, 47(16), 2767–2776.
Haslbeck, J. M. B., Ryan, O., & Dablander, F. (2020). The sum of all fears: Comparing networks based
on symptom sum–scores. PsyArXiv (Preprint), https://psyarxiv.com/ 3nxu9.
Haslbeck, J. M. B., & Waldorp, L. J. (2018). How well do network models predict observations? On
the importance of predictability in network models. Behavior Research Methods, 50(2), 853–861.
Haslbeck, J. M. B., & Waldorp, L. J. (2020). MGM: Estimating time–varying mixed graphical models
in high–dimensional data. Journal of Statistical Software, 93(8), 1–46.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). Elements of Statistical Learning (2nd ed.). Springer
series in statistics.
Hautzinger, M., & Geue, K. (2016). Allgemeine Depressionsskala. Diagnostische Verfahren in Der
Psychotherapie, 1, 33.
Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (1999). Acceptance and commitment therapy: An
experiential approach to behavior change.
Heeren, A., & McNally, R. J. (2016). An integrative network approach to social anxiety disorder: The
complex dynamic interplay among attentional bias for threat, attentional control, and symptoms.
Journal of Anxiety Disorders, 42, 95–104.
Heim, C., Nater, U. M., Maloney, E., Boneva, R., Jones, J. F., & Reeves, W. C. (2009). Childhood trauma
and risk for chronic fatigue syndrome: association with neuroendocrine dysfunction. Archives of
General Psychiatry, 66(1), 72–80.
Hennig, C. (2015). What are the true clusters? Pattern Recognition Letters, 64, 53–62.
Herdman, M., Gudex, C., Lloyd, A., Janssen, M., Kind, P., Parkin, D., Bonsel, G., & Badia, X. (2011).
Development and preliminary testing of the new five–level version of EQ–5D (EQ–5D–5L).
Quality of Life Research, 20(10), 1727–1736.
Page | 408
Herman, J. L., Perry, J. C., & van der Kolk, B. A. (1989). Childhood trauma in borderline personality
disorder. The American Journal of Psychiatry, 146(4), 490–495.
Highton–Williamson, E., Priebe, S., & Giacco, D. (2015). Online social networking in people with
psychosis: A systematic review. International Journal of Social Psychiatry, 61(1), 92–101.
Hoekstra, R. H. A., Epskamp, S., & Borsboom, D. (2020). Heterogeneity in individual network analysis:
Reality or illusion? Masuscript Submitted for Publication, Department of Psychology, University
of Amsterdam.
Hoffman, R. E., Woods, S. W., Hawkins, K. A., Pittman, B., Tohen, M., Preda, A., Breier, A., Glist,
J., Addington, J., Perkins, D. O., & McGlashan, T. H. (2007). Extracting spurious messages from
noise and risk of schizophrenia–spectrum disorders in a prodromal population. British Journal of
Psychiatry, 191(04), 355–356.
Hovens, J. G. F. M., Giltay, E. J., Wiersma, J. E., Spinhoven, P., Penninx, B. W. J. H., & Zitman, F. G.
(2012). Impact of childhood life events and trauma on the course of depressive and anxiety disorders.
Acta Psychiatrica Scandinavica, 126(3), 198–207.
Howes, O. D., & Murray, R. M. (2014). Schizophrenia: An integrated sociodevelopmental–cognitive
model. The Lancet, 383(9929), 1366–1387.
Hoyle, R. H., & Isherwood, J. C. (2013). Reporting results from structural equation modeling analyses
in Archives of Scientific Psychology. Archives of Scientific Psychology, 1(1), 14.
Hugdahl, K. (2009). “hearing voices”: Auditory hallucinations as failure of top–down control of
bottom–up perceptual processes. Scandinavian Journal of Psychology, 50(6), 553–560.
Insel, T. R., & Cuthbert, B. N. (2015). Brain disorders? Precisely. Science, 348(6234), 499–500.
Ising, E. (1925). Beitrag zur theorie des ferromagnetismus. Zeitschrift Für Physik, 31(1), 253–258.
Isvoranu, A.–M., Borsboom, D., van Os, J., & Guloksuz, S. (2016). A network approach to environmental
impact in psychotic disorder: Brief theoretical framework. Schizophrenia Bulletin, 42(4), 870–873.
Isvoranu, A.–M., Boyette, L.–L., Guloksuz, S., & Borsboom, D. (2020). Symptom network models of
psychosis. In C. A. Tamminga, E. I. Ivleva, U. Reininghaus, & J. van Os (Eds.), Psychotic Disorders:
Comprehensive Conceptualization and Treatments. Oxford University Press.
Isvoranu, A.–M., & Epskamp, S. (2021). Continuous and ordered categorical data in Network
Psychometrics: Which estimation method to choose? Deriving guidelines for applied researchers.
PsyArXiv (Preprint), https://psyarxiv.com/mbycn/.
Isvoranu, A.–M., Epskamp, S., & Cheung, M. W. L. (2020). Network models of post–traumatic stress
disorder: A meta–analysis. PsyArXiv (Preprint), https://psyarxiv.com/8k4u6.
Isvoranu, A.–M., Guloksuz, S., Epskamp, S., van Os, J., Borsboom, D., & GROUP. (2019). Toward incorporating
genetic risk scores into symptom networks of psychosis. Psychological Medicine, 50(4), 636–643.
Isvoranu, A.–M., van Borkulo, C. D., Boyette, L.–L. L., Wigman, J. T. W. T. W., Vinkers, C. H. H.,
Borsboom, D., Kahn, R., De Haan, L., Van Os, J., Wiersma, D., Bruggeman, R., Cahn, W., Meijer,
C., & Myin–Germeys, I. (2017). A network approach to psychosis: Pathways between childhood
trauma and psychotic symptoms. Schizophrenia Bulletin, 43(1), 187–196.
Jablensky, A. (2010). The diagnostic concept of schizophrenia: Its history, evolution, and future prospects.
Dialogues in Clinical Neuroscience, 12(3), 271–287.
Jakubowska, A., Kaselionyte, J., Priebe, S., & Giacco, D. (2019). Internet use for social interaction
by people with psychosis: A systematic review. Cyberpsychology, Behavior, and Social Networking,
22(5), 336–343.
James, W. (1890). The Principles of Psychology (2nd ed.). Henry Holt and Co.
Jamshidian, M., Jalal, S., & Jansen, C. (2014). MissMech: An R package for testing homoscedasticity,
multivariate normality, and missing completely at random (MCAR). Journal of Statistical Software,
56(6), 1–31.
Page | 409
References
Janssen, I., Krabbendam, L., Bak, M., Hanssen, M., Vollebergh, W., de Graaf, R., & van Os, J. (2004). Childhood
abuse as a risk factor for psychotic experiences. Acta Psychiatrica Scandinavica, 109(1), 38–45.
Jones, P. J. (2017). networktools: Assorted tools for identifying important nodes in networks. R package
version 1.0.0. In CRAN (1.0.0). https://cran.r–project.org/web/packages/ networktools/index.html
Jones, P. J., Mair, P., & McNally, R. J. (2018). Visualizing psychological networks: A tutorial in R.
Frontiers in Psychology, 9(1742), 1–12.
Kalisch, M., & Bühlmann, P. (2007). Estimating high–dimensional directed acyclic graphs with the
PC–algorithm. Journal of Machine Learning Research, 8(3), 613–636.
Kalisch, M., Fellinghauer, B. A. G., Grill, E., Maathuis, M. H., Mansmann, U., & Bühlmann, P.
(2010). Understanding human functioning using graphical models. BMC Medical Research
Methodology, 10(1), 1–10.
Kalisch, M., Mächler, M., Colombo, D., Maathuis, M. H., & Bühlmann, P. (2012). Causal inference
using graphical models with the R package pcalg. Journal of Statistical Software, 47(11), 1–26.
Kan, K. J., van der Maas, H. L. J., & Levine, S. Z. (2019). Extending psychometric network analysis:
Empirical evidence against g in favor of mutualism? Intelligence, 73, 52–62.
Kapur, S. (2003). Psychosis as a state of aberrant salience: A framework linking biology, phenomenology,
and pharmacology in schizophrenia. American Journal of Psychiatry, 160(1), 13–23.
Kapur, S., Phillips, A. G., & Insel, T. R. (2012). Why has it taken so long for biological psychiatry to
develop clinical tests and what to do about it? Molecular Psychiatry, 17(12), 1174–1179.
Katon, W., & Ciechanowski, P. (2002). Impact of major depression on chronic medical illness. Journal
of Psychosomatic Research, 53(4), 859–863.
Katz, G., Durst, R., Shufman, E., Bar–Hamburger, R., & Grunhaus, L. (2010). Cannabis abuse
and severity of psychotic and affective disorders in Israeli psychiatric inpatients. Comprehensive
Psychiatry, 51(1), 37–41.
Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for
schizophrenia. Schizophrenia Bulletin, 13(2), 261–276.
Kaymaz, N., Drukker, M., Lieb, R., Wittchen, H. U., Werbeloff, N., Weiser, M., Lataster, T., & van
Os, J. (2012). Do subthreshold psychotic experiences predict clinical outcomes in unselected non–
help–seeking population–based samples? A systematic review and meta–analysis, enriched with
new results. Psychological Medicine, 42(11), 2239–2253.
Kendler, K. S. (2016). The nature of psychiatric disorders. World Psychiatry, 15(1), 5–12.
Kendler, K. S., Zachar, P., & Craver, C. (2011). What kinds of things are psychiatric disorders?
Psychological Medicine, 41(06), 1143–1150.
Keshavan, M. S., Nasrallah, H. A., & Tandon, R. (2011). Schizophrenia, “Just the Facts” 6. Moving
ahead with the schizophrenia concept: From the elephant to the mouse. Schizophrenia Research,
173(1–3), 3–13.
Kessler, R. C., & Üstün, B. B. (2004). The World Mental Health (WMH) survey initiative version of
the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI).
International Journal of Methods in Psychiatric Research, 13(2), 93–121.
Kessler, R. C., & Zhao, S. (2010). The prevalence of mental illness. In T. L. Scheid & T. N. Brown (Eds.),
A handbook for the study of mental health: Social contexts, theories, and systems (2nd ed., pp. 46–53).
Cambridge University Press.
Khantzian, E. J. (1985). The self medication hypothesis of addictive disorders. American Journal of
Psychiatry, 142(11), 1259–1264.
Kim, N. S. (2002). Clinical psychologists’ theory–based representations of mental disorders affect their
diagnostic reasoning and memory. Journal of Experimental Psychology: General, 131(4), 451–476.
Page | 410
Kincaid, D. L., Doris, M., Shannon, C., & Mulholland, C. (2017). What is the prevalence of autism
spectrum disorder and ASD traits in psychosis? A systematic review. Psychiatry Research, 250, 99–105.
Klaiber, J., Epskamp, S., & van der Maas, H. (2015). Estimating Ising models on complete and incomplete
psychometric data. Unpublished Master’s Thesis.
Klippel, A., Viechtbauer, W., Reininghaus, U., Wigman, J., van Borkulo, C. D., Myin–Germeys, I., &
Wichers, M. (2018). The cascade of stress: A network approach to explore differential dynamics in
populations varying in risk for psychosis. Schizophrenia Bulletin, 44(2), 328–337.
Konings, M., Bak, M., Hanssen, M., van Os, J., & Krabbendam, L. (2006). Validity and reliability of
the CAPE: A self–report instrument for the measurement of psychotic experiences in the general
population. Acta Psychiatrica Scandinavica, 114(1), 55–61.
Korver, N., Quee, P. J., Boos, H. B., Simons, C. J., & de Haan, L. (2012). Genetic Risk and Outcome
of Psychosis (GROUP), a multi–site longitudinal cohort study focused on gene–environment
interaction: objectives, sample characteristics, recruitment and assessment methods. International
Journal of Methods in Psychiatric Research, 21(3), 205–221.
Kossakowski, J. J., Epskamp, S., Kieffer, J. M., Borkulo, C. D., Rhemtulla, M., & Borsboom, D. (2015). The
application of a network approach to Health–Related Quality of Life (HRQoL): introducing a new method
for assessing HRQoL in healthy adults and cancer patients. Quality of Life Research, 25(4), 781–792.
Kovasznay, B., Bromet, E., Schwart, J. E., Ram, R., Lavelle, J., & Brandon, L. (1993). Substance abuse
and onset of psychotic illness. Hospital and Community Psychiatry, 44(6), 567–571.
Kraan, T. C., Velthorst, E., Themmen, M., Valmaggia, L., Kempton, M. J., McGuire, P., van Os, J.,
Rutten, B. P. F., Smit, F., de Haan, L., van der Gaag, M., Valmaggia, L. R., Calem, M., Tognin,
S., Modinos, G., Burger, N., Van Dam, D. S., Barrantes–Vidal, N., Domínguez–Martínez, T., …
Krebs, M. O. (2018). Child maltreatment and clinical outcome in individuals at Ultra–High Risk
for psychosis in the EU–GEI High Risk Study. Schizophrenia Bulletin, 44(3), 584–592.
Krabbendam, L., Myin–Germeys, I., Bak, M., & van Os, J. (2005). Explaining transitions over the
hypothesized psychosis continuum. Australian and New Zealand Journal of Psychiatry, 39(3), 180–186.
Krabbendam, L., Myin–Germeys, I., Hanssen, M., Bijl, R. V., De Graaf, R., Vollebergh, W., Bak, M., &
Van Os, J. (2004). Hallucinatory experiences and onset of psychotic disorder: Evidence that the risk
is mediated by delusion formation. Acta Psychiatrica Scandinavica, 110(4), 264–272.
Kraemer, H. C. (1992). Evaluating medical tests: Objective and quantitative guidelines. SAGE
Publications.
Kroeze, R., van der Veen, D., Servaas, M., Bastiaansen, J., Oude Voshaar, R., Borsboom, D., Ruhe, E.,
Schoevers, R., & Riese, H. (2017). Personalized feedback on symptom dynamics of psychopathology:
A proof–of–principle study. Journal of Person–Oriented Research, 3(1), 1–11.
Kupfer, D. J., Frank, E., Perel, J. M., Cornes, C., Mallinger, A. G., Thase, M. E., Mceachran, A. B., &
Grochocinski, V. J. (1992). Five–year outcome for maintenance therapies in recurrent depression.
Archives of General Psychiatry, 49(10), 767–773.
Lafit, G., Tuerlinckx, F., Myin–Germeys, I., & Ceulemans, E. (2019). A partial correlation screening approach
for controlling the false positive rate in sparse Gaussian graphical models. Scientific Reports, 9(1), 1–24.
Langley, D. J., & Epskamp, S. (2015). Should I get that jab ? Exploring influence to encourage vaccination
via online social media. ECIS 2015, Research–in–Progress Papers, Paper 64.
Larkin, W., & Read, J. (2003). Childhood trauma and psychosis: evidence, pathways, and implications.
Journal of Postgraduate Medicine, 54(4), 287–293.
Lataster, T., Dina, C., Lardinois, M., Valmaggia, L., van Os, J., & Myin–Germeys, I. (2010). On the
pathway from stress to psychosis. Schizophrenia Research, 117(2), 182.
Lauritzen, S. L. (1996). Graphical models (17th ed.). Clarendon.
Page | 411
References
Lauritzen, S. L., & Wermuth, N. (1989). Graphical models for associations between variables, some
of which are qualitative and some quantitative. In The Annals of Statistics (Vol. 17, Issue 4, pp.
1916–1916).
Lazarov, A., Suarez–Jimenez, B., Levy, O., Coppersmith, D. D. L., Lubin, G., Pine, D. S., Bar–Haim, Y.,
Abend, R., & Neria, Y. (2019). Symptom structure of PTSD and co–morbid depressive symptoms
– A network analysis of combat veteran patients. Psychological Medicine, 50(13), 2154–2170.
Leask, S. J. (2004). Environmental influences in schizophrenia: the known and the unknown. Advances
in Psychiatric Treatment, 10(5), 323–330.
Lesieur, H. R., & Blume, S. B. (1987). The South Oaks Gambling Screen (SOGS): A new instrument
for the identification of pathological gamblers. American Journal of Psychiatry, 144(9), 11841188.
Lev–Ran, S., Roerecke, M., Le Foll, B., George, T. P., McKenzie, K., & Rehm, J. (2014). The association
between cannabis use and depression: A systematic review and meta–analysis of longitudinal studies.
Leverich, G. S., McElroy, S. L., Suppes, T., Keck, P. E., Denicoff, K. D., Nolen, W. A., Altshuler, L. L.,
Rush, A. J., Kupka, R., Frye, M. A., Autio, K. A., & Post, R. M. (2002). Early physical and sexual
abuse associated with an adverse course of bipolar illness. Biological Psychiatry, 51(4), 288–297.
Lim, J., Rekhi, G., Rapisarda, A., Lam, M., Kraus, M., Keefe, R. S. E., & Lee, J. (2015). Impact of
psychiatric comorbidity in individuals at Ultra High Risk of psychosis – Findings from the
Longitudinal Youth at Risk Study (LYRIKS). Schizophrenia Research, 164(1–3), 8–14.
Liu, D., Epskamp, S., & Isvoranu, A.–M. (2020). Network Analysis of physical and psychiatric symptoms
of hospital discharged patients infected with COVID–19. OSF (Preprint).
Liu, H., Lafferty, J., & Wasserman, L. (2009). The nonparanormal: semiparametric estimation of high
dimensional undirected graphs. The Journal of Machine Learning Research, 10, 2295–2328.
Longden, E., Sampson, M., Read, J., Longden, E., Sampson, M., & Sciences, P. (2016). Childhood adversity
and psychosis: generalised or specific effects ? Epidemiology and Psychiatric Sciences, 25(4), 349–359.
López–Pelayo, H., Batalla, A., Balcells, M. M., Colom, J., & Gual, A. (2015). Assessment of cannabis
use disorders: A systematic review of screening and diagnostic instruments. Psychological Medicine,
45(6), 1121–1133.
Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of
the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories.
Behaviour Research and Therapy, 33(3), 335–343.
Lugnegard, T., Hallerbäck, M. U., & Gillberg, C. (2015). Asperger syndrome and schizophrenia: Overlap
of self–reported autistic traits using the Autism–spectrum Quotient (AQ). Nordic Journal of
Psychiatry, 69(4), 268–274.
Ma, S. H., & Teasdale, J. D. (2004). Mindfulness–based cognitive therapy for depression: Replication
and exploration of differential relapse prevention effects. Journal of Consulting and Clinical
Psychology, 72(1), 31–40.
Madley–Dowd, P., Hughes, R., Tilling, K., & Heron, J. (2019). The proportion of missing data should
not be used to guide decisions on multiple imputation. Journal of Clinical Epidemiology, 110, 63–73.
Maher, B. A. (1988). Anomalous experience and delusional thinking: The logic of explanations. In T. F.
Oltmanns & B. A. Maher (Eds.), Delusional beliefs (pp. 15–33). John Wiley & Sons, Inc.
Malgaroli, M., Maccallum, F., & Bonanno, G. A. (2018). Symptoms of persistent complex bereavement
disorder, depression, and PTSD in a conjugally bereaved sample: A network analysis. Psychological
Medicine, 48(14), 2439–2448.
Mancini, A. D., Littleton, H. L., Grills, A. E., & Jones, P. J. (2019). Posttraumatic stress disorder near
and far: Symptom networks from 2 to 12 months after the Virginia Tech campus shootings. Clinical
Psychological Science, 7(6), 1340–1354.
Page | 412
Mansueto, A. C., Wiers, W. R., van Weert, J. C. M., Schouten, B. C., & Epskamp, S. (2020). Investigating
the feasibility of idiographic network models. PsyArXiv (Preprint), https:// psyarxiv.com/hgcz6/.
Marden, J. R., Walter, S., Tchetgen Tchetgen, E. J., Kawachi, I., & Glymour, M. M. (2014). Validation of a
polygenic risk score for dementia in black and white individuals. Brain and Behavior, 4(5), 687–697.
Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., Maas, H. L. J. va. der,
& Maris, G. (2018). An introduction to Network Psychometrics: Relating Ising network models to
item response theory models. Multivariate Behavioral Research, 53(1), 15–35.
McCrae, R. R., & Costa, P. T. (1997). Personality trait structure as a human universal. The American
Psychologist, 52(5), 509–516.
McGrath, D. S., & Barrett, S. P. (2009). The comorbidity of tobacco smoking and gambling: A review
of the literature. Drug and Alcohol Review, 28(6), 676–681.
Mcintyre, R. S., Woldeyohannes, H. O., Soczynska, J. K., Maruschak, N. A., Wium–Andersen, I. K.,
Vinberg, M., Cha, D. S., Lee, Y., Xiao, H. X., Gallaugher, L. A., Dale, R. M., Alsuwaidan, M. T.,
Mansur, R. B., Muzina, D. J., Carvalho, A. F., Jerrell, J. M., & Kennedy, S. H. (2016). Anhedonia
and cognitive function in adults with MDD: Results from the International Mood Disorders
Collaborative Project. CNS Spectrums, 21(5), 362–366.
McNally, R. J., Robinaugh, D. J., Wu, G. W. Y., Wang, L., Deserno, M., & Borsboom, D. (2015).
Mental disorders as causal systems: A network approach to posttraumatic stress disorder. Clinical
Psychological Science, 3(6), 836–849.
Meinshausen, N., Meier, L., & Bühlmann, P. (2009). p–Values for high–dimensional regression. Journal
of the American Statistical Association, 104(488), 1671–1681.
Merckelbach, H., & Van de Ven, V. (2001). Another White Christmas: Fantasy proneness and reports
of “hallucinatory experiences” in undergraduate students. Journal of Behavior Therapy and
Experimental Psychiatry, 32(3), 137–144.
Merikangas, K. R., Calkins, M. E., Burstein, M., He, J. P., Chiavacci, R., Lateef, T., Ruparel, K., Gur, R.
C., Lehner, T., Hakonarson, H., & Gur, R. E. (2015). Comorbidity of physical and mental disorders
in the neurodevelopmental genomics cohort study. Pediatrics, 135(4), e927–e938.
Merikangas, K. R., He, J. P., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., Benjet, C., Georgiades,
K., & Swendsen, J. (2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results
from the national comorbidity survey replication–adolescent supplement (NCS–A). Journal of the
American Academy of Child and Adolescent Psychiatry, 49(10), 980–989.
Mishara, A. L. (2010). Klaus Conrad (1905–1961): delusional mood, psychosis, and beginning
schizophrenia. Schizophrenia Bulletin, 36(1), 9–13.
Miyamoto, S., Miyake, N., Jarskog, L. F., Fleischhacker, W. W., & Lieberman, J. A. (2012).
Pharmacological treatment of schizophrenia: A critical review of the pharmacology and clinical
effects of current and future therapeutic agents. Molecular Psychiatry, 17(12), 1206–1227.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., Antes, G., Atkins, D., Barbour, V.,
Barrowman, N., Berlin, J. A., Clark, J., Clarke, M., Cook, D., D’Amico, R., Deeks, J. J., Devereaux, P.
J., Dickersin, K., Egger, M., Ernst, E., … Tugwell, P. (2009). Preferred reporting items for systematic
reviews and meta–analyses: The PRISMA statement. PLoS Medicine, 8(5), 336–341.
Moore, T. H., Zammit, S., Lingford–Hughes, A., Barnes, T. R., Jones, P. B., Burke, M., & Lewis, G.
(2007). Cannabis use and risk of psychotic or affective mental health outcomes: a systematic review.
Lancet, 370(9584), 319–328.
Moritz, S., & Woodward, T. S. (2007). Metacognitive training in schizophrenia: from basic research to
knowledge translation and intervention. Current Opinion in Psychiatry, 20(6), 619–625.
Musshoff, F., & Madea, B. (2006). Review of biologic matrices (urine, blood, hair) as indicators of recent
or ongoing cannabis use. Therapeutic Drug Monitoring, 28(2), 155–163.
Page | 413
References
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and
continuous latent variable indicators. Psychometrika, 49(1), 115–132.
Myin–Germeys, I., & van Os, J. (2007). Stress–reactivity in psychosis: evidence for an affective pathway
to psychosis. Clinical Psychology Review, 27(4), 409–424.
Myles, N., Newall, H., Nielssen, O., & Large, M. (2012). The association between cannabis use and
earlier age at onset of schizophrenia and other psychoses: Meta–analysis of possible confounding
factors. Current Pharmaceutical Design, 18(32), 5055–5069.
Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian networks in R: with applications in systems
biology. Springer.
Nanni, V., Uher, R., & Danese, A. (2012). Childhood maltreatment predicts unfavorable course of illness
and treatment outcome in depression: A meta–analysis. American Journal of Psychiatry, 169(2), 141–151.
Negrete, J., Knapp, W. P., Douglas, D. E., & Smith WB. (1986). Cannabis affects the severity of
schizophrenic symptoms: Results of a clinical survey. Psychological Medicine, 16(3), 515–520.
Newman, M. (2004). Analysis of weighted networks. Physical Review E – Statistical Physics, Plasmas,
Fluids, and Related Interdisciplinary Topics, 70(5), 636–643.
Newman, M. (2010). Networks: An introduction. Oxford University Press.
Niendam, T. A., Berzak, J., Cannon, T. D., & Bearden, C. E. (2009). Obsessive compulsive symptoms
in the psychosis prodrome: Correlates of clinical and functional outcome. Schizophrenia Research,
108(1–3), 170–175.
Ochoa, S., Usall, J., Cobo, J., Labad, X., & Kulkarni, J. (2012). Gender differences in schizophrenia and
first–episode psychosis: A comprehensive literature review. Schizophrenia Research and Treatment,
2012, 1–9.
Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient.
Psychometrika, 44(4), 441–460.
Ooi, H., de Vries, A., & Microsoft. (2020). checkpoint: Install packages from snapshots on the Checkpoint
server for reproducibility. In CRAN. https://cran.r–project.org/web/ packages/checkpoint/
Oorschot, M., Lataster, T., Thewissen, V., Lardinois, M., Van Os, J., Delespaul, P. A. E. G., & Myin–
Germeys, I. (2012). Symptomatic remission in psychosis and real–life functioning. British Journal
of Psychiatry, 201(3), 215–220.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science,
349(6251), 943–951.
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010). Node centrality in weighted networks: Generalizing
degree and shortest paths. Social Networks, 32(3), 245–251.
Overall, J. E., & Gorham, D. R. (1962). The Brief Psychiatric Rating Scale. Psychological Reports, 10(3),
799–812.
Pardiñas, A. F., Holmans, P., Pocklington, A. J., Escott–Price, V., Ripke, S., Carrera, N., Legge, S. E.,
Bishop, S., Cameron, D., Hamshere, M. L., Han, J., Hubbard, L., Lynham, A., Mantripragada,
K., Rees, E., MacCabe, J. H., McCarroll, S. A., Baune, B. T., Breen, G., … Walters, J. T. R. (2018).
Common schizophrenia alleles are enriched in mutation–intolerant genes and in regions under
strong background selection. Nature Genetics, 50(3), 381–389.
Patrick, D. L., Kinne, S., Engelberg, R. A., & Pearlman, R. A. (2000). Functional status and perceived
quality of life in adults with and without chronic conditions. Journal of Clinical Epidemiology,
53(8), 779–785.
Patton, G. C., Coffey, C., Carlin, J. B., Degenhardt, L., Lynskey, M., & Hall, W. (2002). Cannabis use and
mental health in young people: cohort study. BMJ (Clinical Research Ed.), 325(7374), 1195–1198.
Patton, G. C., Coffey, C., Carlin, J. B., Sawyer, S. M., & Lynskey, M. (2005). Reverse gateways? Frequent
cannabis use as a predictor of tobacco initiation and nicotine dependence. Addiction, 100(10), 1518–1525.
Page | 414
Pearl, J. (2000). Causality: Models, reasoning and inference. Cambridge University Press.
Peralta, V., & Cuesta, M. J. (1992). Influence of cannabis abuse on schizophrenic psychopathology. Acta
Psychiatrica Scandinavica, 85(2), 127–130.
Petry, N. M., Stinson, F. S., & Grant, B. F. (2005). Comorbidity of DSM–IV pathological gambling and
other psychiatric disorders: Results from the national epidemiologic survey on alcohol and related
conditions. Journal of Clinical Psychiatry, 66(5), 564–574.
Pietrzak, R. H., Morasco, B. J., Blanco, C., Grant, B. F., & Petry, N. M. (2007). Gambling level and
psychiatric and medical disorders in older adults: Results from the national epidemiologic survey
on alcohol and related conditions. American Journal of Geriatric Psychiatry, 15(4), 301–313.
Poyurovsky, M., Kriss, V., Weisman, G., Faragian, S., Schneidman, M., Fuchs, C., Weizman, A., &
Weizman, R. (2005). Familial aggregation of schizophrenia–spectrum disorders and obsessive–
compulsive associated disorders in schizophrenia probands with and without OCD. American
Journal of Medical Genetics – Neuropsychiatric Genetics, 130(3), 214–226.
Price, M., Legrand, A. C., Brier, Z. M. F., & Hébert–Dufresne, L. (2019). The symptoms at the center:
Examining the comorbidity of posttraumatic stress disorder, generalized anxiety disorder, and
depression with network analysis. Journal of Psychiatric Research, 109, 52–58.
Pries, L. K., Guloksuz, S., Menne–Lothmann, C., Decoster, J., Winkel, R. van, Collip, D., Delespaul, P.,
Hert, M. De, Derom, C., Thiery, E., Jacobs, N., Wichers, M., Simons, C. J. P., Rutten, B. P. F., & van
Os, J. (2017). White noise speech illusion and psychosis expression: An experimental investigation
of psychosis liability. PLoS ONE, 12(8), 1–10.
Pushkarskaya, H., Sobowale, K., Henick, D., Tolin, D. F., Anticevic, A., Pearlson, G. D., Levy, I., Harpaz–
Rotem, I., & Pittenger, C. (2019). Contrasting contributions of anhedonia to obsessive–compulsive,
hoarding, and post–traumatic stress disorders. Journal of Psychiatric Research, 109, 202–213.
Quattrone, D., Ferraro, L., Tripoli, G., La Cascia, C., Quigley, H., Quattrone, A., Jongsma, H. E., Del
Peschio, S., Gatto, G., Gayer–Anderson, C., Jones, P. B., Kirkbride, J. B., La Barbera, D., Tarricone,
I., Berardi, D., Tosato, S., Lasalvia, A., Szöke, A., Arango, C., … Di Forti, M. (2020). Daily use
of high–potency cannabis is associated with more positive symptoms in first–episode psychosis
patients: The EU–GEI case–control study. Psychological Medicine, 1–9.
Quigley, H., & MacCabe, J. H. (2019). The relationship between nicotine and psychosis. Therapeutic
Advances in Psychopharmacology, 9, 2045125319859969.
R Core Team. (2015). R: A language and environment for statistical computing. https://www.r–project.org/
Radloff, L. S. (1977). The CES–D Scale. Applied Psychological Measurement, 1(3), 385–401.
Ravikumar, P., Wainwright, M. J., & Lafferty, J. D. (2010). High–dimensional ising model selection
using ℓ1–regularized logistic regression. Annals of Statistics, 38(3), 1287–1319.
Read, John, Agar, K., Argyle, N., & Aderhold, V. (2003). Sexual and physical abuse during childhood
and adulthood as predictors of hallucinations, delusions and thought disorder. Psychology and
Psychotherapy: Theory, Research and Practice, 76(1), 1–22.
Read, John, & Argyle, N. (1999). Hallucinations, delusions, and thought disorder among adult
psychiatric inpatients with a history of child abuse. Psychiatric Services, 50(11), 1467–1472.
Read, John, & Beavan, V. (2013). Gender and psychosis. In J. Read & J. Dillon (Eds.), Models of madness:
Psychological, social and biological approaches to psychosis (2nd ed.). (pp. 210–219). Routledge/ Taylor
& Fracis Group.
Read, John, Fosse, R., Moskowitz, A., Perry, B., Moskowitz, A., & Perry, B. (2001). The traumagenic
neurodevelopmental model of psychosis revisited. Neuropsychiatry, 4(2014), 65–79.
Read, John, van Os, J., Morrison, A. P., & Ross, C. A. (2005). Childhood trauma, psychosis and
schizophrenia: A literature review with theoretical and clinical implications. Acta Psychiatrica
Scandinavica, 112(5), 330–350.
Page | 415
References
Reed, G. (1972). The Psychology of anomalous experience: A cognitive approach. Hutchinson and Co.
Renkema, T. C., de Haan, L., Schirmbeck, F., Alizadeh, B. Z., van Amelsvoort, T., Bartels–Velthuis,
A. A., van Beveren, N. J., Bruggeman, R., Cahn, W., Delespaul, P., Luykx, J. J., Myin–Germeys, I.,
Kahn, R. S., Simons, C. J. P., van Os, J., & van Winkel, R. (2020). Childhood trauma and coping in
patients with psychotic disorders and obsessive–compulsive symptoms and in un–affected siblings.
Child Abuse and Neglect, 99, 104243.
Revelle, W. (2015). Package “psych” – Procedures for psychological, psychometric and personality
research. In CRAN. https://cran.r–project.org/web/packages/psych/
Rhemtulla, M., Fried, E. I., Aggen, S. H., Tuerlinckx, F., Kendler, K. S., & Borsboom, D. (2016). Network
analysis of substance abuse and dependence symptoms. Drug and Alcohol Dependence, 161, 230–237.
Rimvall, M. K., Clemmensen, L., Munkholm, A., Rask, C. U., Larsen, J. T., Skovgaard, A. M., Simons, C.
J. P., van Os, J., & Jeppesen, P. (2016). Introducing the White Noise task in childhood: Associations
between speech illusions and psychosis vulnerability. Psychological Medicine, 43(13), 1–10.
Robinaugh, D. J., Haslbeck, J. M. B., Ryan, O., Fried, E. I., & Waldorp, J. (2020). Invisible hands and
fine calipers: A call to use formal theory as a toolkit for theory construction. PsyArXiv (Preprint),
https://psyarxiv.com/ugz7y.
Robinaugh, D. J., Hoekstra, R. H. A., Toner, E. R., & Borsboom, D. (2020). The network approach
to psychopathology: A review of the literature 2008–2018 and an agenda for future research.
Robinaugh, D. J., Millner, A. J., & McNally, R. J. (2016). Identifying highly influential nodes in the
complicated grief network. Journal of Abnormal Psychology, 125(6), 747–757.
Robinson, D. G., Woerner, M. G., McMeniman, M., Mendelowitz, A., & Bilder, R. M. (2004).
Symptomatic and functional recovery from a first episode of Schizophrenia or Schizoaffective
disorder. American Journal of Psychiatry, 161(3), 473–479.
Rodebaugh, T. L., Tonge, N. A., Piccirill, M. L., Fried, E. I., Horenstein, A., Morrison, A. S., Goldin,
P., Gross, J. J., Lim, M. H., Fernandez, K. C., Blanco, C., Schneier, F. R., Bogdan, R., Thompson, R.
J., & Heimberg, R. G. (2018a). Does centrality in a cross–sectional network suggest intervention
targets for social anxiety disorder? Journal of Consulting and Clinical Psychology, 86(10), 831–844.
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software,
48(2), 1–36.
Rubin, D. C. (1976). Inference and missing data. Biometrika, 63(3), 581–592.
Rubin, D. C., Berntsen, D., & Bohni, M. K. (2008). A memory–based model of posttraumatic stress
disorder: Evaluating basic assumptions underlying the PTSD diagnosis. Psychological Review, 115(4),
985–1011.
Rutigliano, G., Valmaggia, L., Landi, P., Frascarelli, M., Cappucciati, M., Sear, V., Rocchetti, M., De
Micheli, A., Jones, C., Palombini, E., McGuire, P., & Fusar–Poli, P. (2016). Persistence or recurrence
of non–psychotic comorbid mental disorders associated with 6–year poor functional outcomes in
patients at ultra high risk for psychosis. Journal of Affective Disorders, 203, 101–110.
Ruzzano, L., Borsboom, D., & Geurts, H. M. (2015). Repetitive behaviors in autism and obsessive–
compulsive disorder: new perspectives from a network analysis. Journal of Autism and Developmental
Disorders, 45(1), 192–202.
Santos, H., Fried, E. I., Asafu–Adjei, J., & Jeanne Ruiz, R. (2017). Network structure of perinatal
depressive symptoms in latinas: Relationship to stress and reproductive biomarkers. Research in
Nursing and Health, 40(3), 218–228.
Saricoban, H. E., Ozen, A., Harmanci, K., Razi, C., Zahmacioglu, O., & Cengizlier, M. R. (2011).
Common behavioral problems among children with asthma: Is there a role of asthma treatment?
Annals of Allergy, Asthma and Immunology, 106(3), 200–204.
Page | 416
Sartorius, N. (2018). Comorbidity of mental and physical disorders: A key problem for medicine in the
21st century. Acta Psychiatrica Scandinavica, 137, 369–370.
Scheffer, M., Bascompte, J., Brock, W. A., Brovkin, V., Carpenter, S. R., Dakos, V., Held, H., van Nes,
E. H., Rietkerk, M., & Sugihara, G. (2009). Early–warning signals for critical transitions. Nature,
461(7260), 53–59.
Schepers, E., Lousberg, R., Guloksuz, S., Pries, L. K., Delespaul, P., Kenis, G., Luykx, J. J., Lin, B. D.,
Richards, A. L., Akdede, B., Binbay, T., Altınyazar, V., Yalınçetin, B., Gümüş–Akay, G., Cihan, B.,
Soygür, H., Ulaş, H., Cankurtaran, E. Ş., Kaymak, S. U., … van Os, J. (2019). White noise speech
illusions: A trait–dependent risk marker for psychotic disorder? Frontiers in Psychiatry, 10(676), 1–10.
Schepers, E., van Os, J., & Lousberg, R. (2019). White noise speech illusions in the general population:
The association with psychosis expression and risk factors for psychosis. PLoS ONE, 14(2), 1–10.
Schirmbeck, F., Konijn, M., Hoetjes, V., Zink, M., & de Haan, L. (2019). Obsessive–compulsive
symptoms in psychotic disorders: longitudinal associations of symptom clusters on between– and
within–subject levels. European Archives of Psychiatry and Clinical Neuroscience, 269(2), 245–255.
Schirmbeck, F., Swets, M., Meijer, C. J., Zink, M., de Haan, L., Kahn, R. S., van Os, J., Bruggeman, R.,
Cahn, W., Bartels–Velthuis, A. A., & Myin–Germeys, I. (2018). Obsessive–compulsive symptoms
and overall psychopathology in psychotic disorders: longitudinal assessment of patients and siblings.
European Archives of Psychiatry and Clinical Neuroscience, 268(3), 279–289.
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D.
(2013). Deconstructing the construct: A network perspective on psychological phenomena. New
Ideas in Psychology, 31(1), 43–53.
Schneider, M., Reininghaus, U., van Nierop, M., Janssens, M., Myin–Germeys, I., Alizadeh, B., Bartels–
Velthuis, A. A., Bruggeman, R., Cahn, W., de Haan, L., Delespaul, P., Kahn, R. S., Meijer, C. J.,
Myin–Germeys, I., Simons, C., van Haren, N., van Os, J., & van Winkel, R. (2017). Does the Social
Functioning Scale reflect real–life social functioning? An experience sampling study in patients
with a non–affective psychotic disorder and healthy control individuals. Psychological Medicine,
47(16), 2777–2786.
Schofield, D., Tennant, C., Nash, L., Degenhardt, L., Cornish, A., Hobbs, C., & Brennan, G. (2006).
Reasons for cannabis use in psychosis. Australian and New Zealand Journal of Psychiatry, 40(6–7),
570–574.
Schreiber, J. B., Stage, F. K., King, J., Nora, A., & Barlow, E. A. (2006). Reporting structural equation
modeling and confirmatory factor analysis results: A review. Journal of Educational Research, 99(6),
323–338.
Schreuder, M. J., Schirmbeck, F., Meijer, C., & de Haan, L. (2017). The associations between childhood
trauma, neuroticism and comorbid obsessive–compulsive symptoms in patients with psychotic
disorders. Psychiatry Research, 254, 48–53.
Scott, K. M., Bruffaerts, R., Tsang, A., Ormel, J., Alonso, J., Angermeyer, M. C., Benjet, C., Bromet, E.,
de Girolamo, G., de Graaf, R., Gasquet, I., Gureje, O., Haro, J. M., He, Y., Kessler, R. C., Levinson,
D., Mneimneh, Z. N., Oakley Browne, M. A., Posada–Villa, J., … Von Korff, M. (2007). Depression–
anxiety relationships with chronic physical conditions: Results from the World Mental Health
surveys. Journal of Affective Disorders, 103(1–3), 113–120.
Scutari, M. (2010). Learning Bayesian networks with the bnlearn R Package. Journal of Statistical
Software, 35(3), 1–22.
Seddon, J. L., Birchwood, M., Copello, A., Everard, L., Jones, P. B., Fowler, D., Amos, T., Freemantle, N.,
Sharma, V., Marshall, M., & Singh, S. P. (2016). Cannabis use is associated with increased psychotic
symptoms and poorer psychosocial functioning in first–episode psychosis: A report from the UK
National EDEN study. Schizophrenia Bulletin, 42(3), 619–625.
Page | 417
References
Shevlin, M., Houston, J. E., Dorahy, M. J., & Adamson, G. (2008). Cumulative traumas and psychosis:
an analysis of the national comorbidity survey and the British Psychiatric Morbidity Survey.
Sim, K., Mahendran, R., Siris, S. G., Heckers, S., & Chong, S. A. (2004). Subjective quality of life
in first episode schizophrenia spectrum disorders with comorbid depression. Psychiatry Research,
129(2), 141–147.
Simons, J. S., Simons, R. M., Walters, K. J., Keith, J. A., O’Brien, C., Andal, K., & Stoltenberg, S. F.
(2019). Nexus of despair: A network analysis of suicidal ideation among veterans. Archives of Suicide
Research, 24(SUP1), 314–336.
Smeets, F., Lataster, T., Dominguez, M. D. G., Hommes, J., Lieb, R., Wittchen, H. U., & van Os, J.
(2012). Evidence that onset of psychosis in the population reflects early hallucinatory experiences
that through environmental risks and affective dysregulation become complicated by delusions.
Smeets, F., Lataster, T., van Winkel, R., de Graaf, R., ten Have, M., & van Os, J. (2012). Testing the
hypothesis that psychotic illness begins when subthreshold hallucinations combine with delusional
ideation. Acta Psychiatrica Scandinavica, 127(1), 34–47.
Smeets, F., Lataster, T., Viechtbauer, W., Delespaul, P., Bruggeman, R., Cahn, W., de Haan, L., Kahn, R.
S., Meijer, C. J., Myin–Germeys, I., van Os, J., & Wiersma, D. (2015). Evidence that environmental
and genetic risks for psychotic disorder may operate by impacting on connections between core
symptoms of perceptual alteration and delusional ideation. Schizophrenia Bulletin, 41(3), 687–697.
Soyata, A. Z., Akışık, S., İnhanlı, D., Noyan, H., & Üçok, A. (2018). Relationship of obsessive–
compulsive symptoms to clinical variables and cognitive functions in individuals at ultra high risk
for psychosis. Psychiatry Research, 261, 323–337.
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a
multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712.
Stevens, L., & Rodin, I. (2011). Antipsychotic drugs. In Psychiatry: An Illustrated Colour Text (pp.
24–25). Elsevier Health Sciences.
Stroup, T. S., & Gray, N. (2018). Management of common adverse effects of antipsychotic medications.
World Psychiatry, 17(3), 341–356.
Subramaniam, M., Vaingankar, J., Heng, D., Kwok, K. W., Lim, Y. W., Yap, M., & Chong, S. A. (2012).
The Singapore Mental Health Study: An overview of the methodology. International Journal of
Methods in Psychiatric Research, 21(2), 149–157.
Sullivan, P. F. (2008). Schizophrenia genetics: The search for a hard lead. Current Opinion in Psychiatry,
21(2), 157–160.
Sullivan, S., Rai, D., Golding, J., Zammit, S., & Steer, C. (2013). The association between autism
spectrum disorder and psychotic experiences in the Avon Longitudinal Study of Parents and
Children (ALSPAC) birth cohort. Journal of the American Academy of Child and Adolescent
Psychiatry, 52(8), 806–814.
Swets, M., Dekker, J., van Emmerik–van Oortmerssen, K., Smid, G. E., Smit, F., de Haan, L., &
Schoevers, R. A. (2014). The obsessive compulsive spectrum in schizophrenia, a meta–analysis and
meta–regression exploring prevalence rates. Schizophrenia Research, 152(2–3), 458–468.
Szechtman, H., & Woody, E. Z. (2006). Obsessive–compulsive disorder as a disturbance of security
motivation: Constraints on comorbidity. Neurotoxicity Research, 10(2), 103–112.
Tandon, R., Keshavan, M. S., & Nasrallah, H. A. (2008a). Schizophrenia, “Just the Facts”, what we
know in 2008. Part 1. Overview. Schizophrenia Research, 100(1–3), 4–19.
Tandon, R., Keshavan, M. S., & Nasrallah, H. A. (2008b). Schizophrenia, “Just the Facts”, what we know
in 2008. Part 2. Epidemiology and etiology. Schizophrenia Research, 102(1–3), 1–18.
Page | 418
Taylor, J. A. (1953). A personality scale of manifest anxiety. Journal of Abnormal and Social Psychology,
48(2), 285–290.
Therman, S., & Ziermans, T. B. (2015). Confirmatory factor analysis of psychotic–like experiences in
a general population sample. Psychiatry Research, 235, 197–199.
Thewissen, V., Bentall, R. P., Oorschot, M., A Campo, J., van Lierop, T., van Os, J., & Myin–Germeys,
I. (2011). Emotions, self–esteem, and paranoid episodes: an experience sampling study. The British
Journal of Clinical Psychology / the British Psychological Society, 50(2), 178–195.
Thombs, B. D., Bernstein, D. P., Lobbestael, J., & Arntz, A. (2009). A validation study of the Dutch
Childhood Trauma Questionnaire–Short Form: Factor structure, reliability, and known–groups
validity. Child Abuse and Neglect, 33(8), 518–523.
Thorup, A., Albert, N., Bertelsen, M., Petersen, L., Jeppesen, P., Le Quack, P., Krarup, G., Jørgensen, P.,
& Nordentoft, M. (2014). Gender differences in first–episode psychosis at 5–year follow–up–two
different courses of disease? Results from the OPUS study at 5–year follow–up. European Psychiatry,
29(1), 45–51.
Tibshirani, R. (1994). Regression delection and dhrinkage via the lasso. Journal of the Royal Statistical
Society B, 58(1), 267–288.
Tio, P., Epskamp, S., Noordhof, A., & Borsboom, D. (2016). Mapping the manuals of madness:
Comparing the ICD–10 and DSM–IV–TR using a network approach. International Journal of
Methods in Psychiatric Research, 25(4), 267–276.
Tolin, D. F., & Foa, E. B. (2006). Sex differences in trauma and posttraumatic stress disorder: a
quantitative review of 25 years of research. Psychological Bulletin, 132(6), 959–992.
Tsuang, M. T., Stone, W. S., & Faraone, S. V. (2001). Genes, environment and schizophrenia. British
Journal of Psychiatry, 178(S40), s18–s24.
van Borkulo, C. D., Borsboom, D., Epskamp, S., Blanken, T. F., Boschloo, L., Schoevers, R. A., &
Waldorp, L. J. (2014). A new method for constructing networks from binary data. Scientific Reports,
4(1), 1–10.
van Borkulo, C. D., Boschloo, L., Borsboom, D., Brenda, W. J. H. P., Waldorp, L. J., & Schoevers, R. A.
(2016). Association of symptom network structure with the course of depression. JAMA Psychiatry,
72(12), 1219–1226.
van Borkulo, C. D., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D., & Waldorp,
L. J. (2017). Comparing network structures on three aspects: A permutation test. Masuscript
Submitted for Publication, Department of Psychology, University of Amsterdam.
van Borkulo, C. D., Epskamp, S., & Robitzsch, A. (2014). IsingFit: Fitting Ising models using the eLasso
method. In CRAN. https://cran.r–project.org/web/packages/IsingFit/
van Buuren, S., & Groothuis–Oudshoorn, K. (2011). mice: Multivariate imputation by chained
equations in R. Journal of Statistical Software, 45(3), 1–67.
van Dael, F., van Os, J., De Graaf, R., Ten Have, M., Krabbendam, L., & Myin–Germeys, I. (2011). Can
obsessions drive you mad? Longitudinal evidence that obsessive–compulsive symptoms worsen the
outcome of early psychotic experiences. Acta Psychiatrica Scandinavica, 123(2), 136–146.
van Dam, D. S., van Nierop, M., Viechtbauer, W., Velthorst, E., van Winkel, R., Genetic, R., Outcome
of Psychosis, I., Bruggeman, R., Cahn, W., de Haan, L., Kahn, R. S., Meijer, C. J., Myin–Germeys,
I., van Os, J., & Wiersma, D. (2015). Childhood abuse and neglect in relation to the presence and
persistence of psychotic and depressive symptomatology. Psychological Medicine, 45(7), 1363–1377.
Page | 419
References
van de Leemput, I. A., Wichers, M., Cramer, A. O. J., Borsboom, D., Tuerlinckx, F., Kuppens, P., van
Nes, E. H., Viechtbauer, W., Giltay, E. J., Aggen, S. H., Derom, C., Jacobs, N., Kendler, K. S., van der
Maas, H. L. J., Neale, M. C., Peeters, F., Thiery, E., Zachar, P., & Scheffer, M. (2014). Critical slowing
down as early warning for the onset and termination of depression. Proceedings of the National
Academy of Sciences, 111(1), 87–92.
van der Feltz–Cornelis, C. M., Biemans, H., & Timmer, J. (2012). Hearing voices: Does it give your
patient a headache? A case of auditory hallucinations as acoustic aura in migraine. Neuropsychiatric
Disease and Treatment, 8, 105–111.
van der Heijden, P., van den Bos, P., Mol, B., & Kessels, R. P. (2013). Structural validity of the Dutch–language
version of the WAIS–III in a psychiatric sample. Applied Neuropsychology: Adult, 20(1), 41–46.
van Gastel, W. A., MacCabe, J. H., Schubart, C. D., Vreeker, A., Tempelaar, W., Kahn, R. S., & Boks, M.
P. M. (2013). Cigarette smoking and cannabis use are equally strongly associated with psychotic–
like experiences:A cross–sectional study in 1929 young adults. Psychological Medicine, 43(11),
2393–2401.
van Heijst, B. F. C., Deserno, M. K., Rhebergen, D., & Geurts, H. M. (2020). Autism and depression are
connected: A report of two complimentary network studies. Autism, 24(3), 680–692.
van Loo, H. M., van Borkulo, C. D., Peterson, R. E., Fried, E. I., Aggen, S. H., Borsboom, D., & Kendler,
K. S. (2017). Robust symptom networks in recurrent major depression across different levels of
genetic and environmental risk. Journal of Affective Disorders, 227, 313–322.
van Nierop, M., Lataster, T., Smeets, F., Gunther, N., Van Zelst, C., De Graaf, R., Ten Have, M., Van
Dorsselaer, S., Bak, M., Myin–Germeys, I., Viechtbauer, W., van Os, J., & Van Winkel, R. (2014).
Psychopathological mechanisms linking childhood traumatic experiences to risk of psychotic
symptoms: Analysis of a large, representative population–based sample. Schizophrenia Bulletin,
40(SUPPL. 2), 123–130.
van Os, J., & Kapur, S. (2009). Schizophrenia. The Lancet, 374(9690), 635–645.
van Os, J., Kenis, G., & Rutten, B. P. F. (2010). The environment and schizophrenia. Nature, 468(7321),
203–212.
van Os, J., & Linscott, R. J. (2012). Introduction: The extended psychosis phenotype – relationship with
schizophrenia and with ultrahigh risk status for psychosis. Schizophrenia Bulletin, 38(2), 227–230.
van Os, J., Rutten, B. P., Myin–Germeys, I., Delespaul, P., Viechtbauer, W., van Zelst, C., Bruggeman, R.,
Reininghaus, U., Morgan, C., Murray, R. M., Di Forti, M., McGuire, P., Valmaggia, L. R., Kempton,
M. J., Gayer–Anderson, C., Hubbard, K., Beards, S., Stilo, S. A., Onyejiaka, A., … Mirjanic, T.
(2014). Identifying gene–environment interactions in schizophrenia: Contemporary challenges
for integrated, large–scale investigations. Schizophrenia Bulletin, 40(4), 729–736.
van Os, J., van der Steen, Y., Islam, M. A., Gülöksüz, S., Rutten, B. P., & Simons, C. J. (2017). Evidence
that polygenic risk for psychotic disorder is expressed in the domain of neurodevelopment, emotion
regulation and attribution of salience. Psychological Medicine, 47(14), 2421–2437.
van Rooijen, G., Isvoranu, A.–M., Kruijt, O. H., van Borkulo, C. D., Meijer, C. J., Wigman, J. T.
W., Ruhé, H. G., de Haan, L., Bruggeman, R., Cahn, W., de Haan, L., Kahn, R. S., Meijer, C.,
Myin–Germeys, I., van Os, J., & Bartels–Velthuis, A. A. (2018). A state–independent network of
depressive, negative and positive symptoms in male patients with schizophrenia spectrum disorders.
Schizophrenia Research, 193, 232–239.
van Rooijen, G., Isvoranu, A.–M., Meijer, C. J., van Borkulo, C. D., Ruhé, H. G., & de Haan, L. (2017).
A symptom network structure of the psychosis spectrum. Schizophrenia Research, 189, 75–83.
Vanzhula, I. A., Calebs, B., Fewell, L., & Levinson, C. A. (2019). Illness pathways between eating
disorder and post–traumatic stress disorder symptoms: Understanding comorbidity with network
analysis. European Eating Disorders Review, 27(2), 147–160.
Page | 420
Vaskinn, A., & Abu–Akel, A. (2019). The interactive effect of autism and psychosis severity on theory
of mind and functioning in schizophrenia. Neuropsychology, 33(2), 195–202.
Ventura, J., Nuechterlein, K. H., Subotnik, K., & Gilbert, E. (1995). Symptom dimensions in recent
onset schizophrenia: The 24–item expanded BPRS. Schizophrenia Research, 15(1), 22.
Vercammen, A., & Aleman, A. (2010). Semantic expectations can induce false perceptions in
hallucination–prone individuals. Schizophrenia Bulletin, 36(1), 151–156.
Vinkers, C. H., van Gastel, W. A., Schubart, C. D., Van Eijk, K. R., Luykx, J. J., Van Winkel, R., Joëls,
M., Ophoff, R. A., & Boks, M. P. M. (2013). The effect of childhood maltreatment and cannabis use
on adult psychotic symptoms is modified by the COMT Val158Met polymorphism. Schizophrenia
Research, 150(1), 303–311.
Weinberger, D. R., & Harrison, P. J. (2010). Schizophrenia. Wiley–Blackwell.
Wells, K. B., Golding, J. M., & Burnam, M. A. (1989). Affective, substance use, and anxiety disorders
in persons with arthritis, diabetes, heart disease, high blood pressure, or chronic lung conditions.
General Hospital Psychiatry, 11(5), 320–327.
Whitfield, C. L., Dube, S. R., Felitti, V. J., & Anda, R. F. (2005). Adverse childhood experiences and
hallucinations. Child Abuse & Neglect, 29(7), 797–810.
Whitton, A. E., Treadway, M. T., & Pizzagalli, D. A. (2015). Reward processing dysfunction in major
depression, bipolar disorder and schizophrenia. Current Opinion in Psychiatry, 28(1), 7–12.
Wichers, M. (2014). The dynamic nature of depression: a new micro–level perspective of mental disorder
that meets current challenges. Psychological Medicine, 44(7), 1349–1360.
Wiersma, J. E., Hovens, J. G. F. M., Van Oppen, P., Giltay, E. J., Van Schaik, D. J. F., Beekman, A. T. F.,
& Penninx, B. W. J. H. (2009). The importance of childhood trauma and childhood life events for
chronicity of depression in adults. Journal of Clinical Psychiatry, 70(7), 983–989.
Wigboldus, D. H. J., & Dotsch, R. (2016). Encourage playing with data and discourage questionable
reporting practices. Psychometrika, 81(1), 27–32.
Wigman, J. T. W., van Os, J., Borsboom, D., Wardenaar, K. J., Epskamp, S., Klippel, A., Viechtbauer, W.,
Myin–Germeys, I., & Wichers, M. (2015). Exploring the underlying structure of mental disorders:
Cross–diagnostic differences and similarities from a network perspective using both a top–down
and a bottom–up approach. Psychological Medicine, 45(11), 2375–2387.
Williams, D. R. (2018). Bayesian estimation for Gaussian graphical models: Structure learning,
predictability, and network comparisons. PsyArXiv (Preprint), https://psyarxiv.com/ x8dpr/.
Williams, D. R. (2020). Learning to live with sampling variability: Expected replicability in partial
correlation networks. PsyArXiv (Preprint), https://psyarxiv.com/fb4sa/.
Williams, D. R. (2021). Beyond Lasso: A survey of nonconvex regularization in Gaussian graphical
models. PsyArXiv (Preprint), https://psyarxiv.com/ad57p/.
Williams, D. R., & Mulder, J. (2019). BGGM : A R Package for Bayesian Gaussian graphical models.
PsyArXiv (Preprint), https://psyarxiv.com/3b5hf/.
Williams, D. R., & Mulder, J. (2020). BGGM: Bayesian Gaussian graphical models in R. Journal of
Open Source Software, 5(51), 2111.
Williams, D. R., & Rast, P. (2020). Back to the basics: Rethinking partial correlation network
methodology. Journal of Mathematical and Statistical Psychology, 73(2), 187–212.
Williams, D. R., Rast, P., Pericchi, L. R., & Mulder, J. (2020). Comparing Gaussian graphical models
with the posterior predictive distribution and Bayesian model selection. Psychological Methods,
25(5), 653–672.
Williams, D. R., Rhemtulla, M., Wysocki, A. C., & Rast, P. (2019). On Nonregularized Estimation of
Psychological Networks. Multivariate Behavioral Research, 54(5), 719–750.
Page | 421
References
Winston, C., Cheng, J., Allaire, J., Xie, Y., & McPherson, J. (2017). Shiny: Web application framework
for R. In CRAN. https://cran.r–project.org/package=shiny
Wittchen, H. U., Perkonigg, A., Lachner, G., & Nelson, C. B. (1998). Early developmental stages of
psychopathology study (EDSP): Objectives and design. European Addiction Research, 4(1–2), 18–27.
Wüsten, C., Schlier, B., Jaya, E. S., Fonseca–Pedrero, E., Peters, E., Verdoux, H., Woodward, T. S.,
Ziermans, T. B., & Lincoln, T. M. (2018). Psychotic experiences and related distress: A cross–national
comparison and network analysis based on 7141 participants from 13 countries. Schizophrenia
Bulletin, 44(6), 1185–1194.
Wykes, T., Steel, C., Everitt, B., & Tarrier, N. (2008). Cognitive behavior therapy for schizophrenia:
Effect sizes, clinical models, and methodological rigor. Schizophrenia Bulletin, 34(3), 523–537.
Yonkers, K. A., Dyck, I. R., Warshaw, M., & Keller, M. B. (2000). Factors predicting the clinical course
of generalised anxiety disorder. British Journal of Psychiatry, 176(6), 544–549.
Young, E. A., Abelson, J. L., Curtis, G. C., & Nesse, R. M. (1997). Childhood adversity and vulnerability
to mood and anxiety disorders. Depression and Anxiety, 5(2), 66–72.
Yung, A. R., McGorry, P. D., McFarlane, C. A., Jackson, H. J., Patton, G. C., & Rakkar, A. (1996). Monitoring
and care of young people at incipient risk of psychosis. Schizophrenia Bulletin, 22(2), 283–303.
Yung, A. R., Nelson, B., Thompson, A., & Wood, S. J. (2010). The psychosis threshold in Ultra High
Risk (prodromal) research: Is it valid? Schizophrenia Research, 120(1–3), 1–6.
Yung, A. R., Stanford, C., Cosgrave, E., Killackey, E., Phillips, L., Nelson, B., & McGorry, P. D. (2006).
Testing the Ultra High Risk (prodromal) criteria for the prediction of psychosis in a clinical sample
of young people. Schizophrenia Research, 84(1), 57–66.
Yung, A. R., Yuen, H. P., McGorry, P. D., Phillips, L. J., Kelly, D., Dell’Olio, M., Francey, S. M., Cosgrave,
E. M., Killackey, E., Stanford, C., Godfrey, K., & Buckby, J. (2005). Mapping the onset of psychosis:
The Comprehensive Assessment of At–Risk Mental States. Australian and New Zealand Journal
of Psychiatry, 39(11–12), 964–971.
Zhou, H. yu, Yang, H. xue, Gong, J. bo, Cheung, E. F. C., Gooding, D. C., Park, S., & Chan, R. C. K.
(2019). Revisiting the overlap between autistic and schizotypal traits in the non–clinical population
using meta–analysis and network analysis. Schizophrenia Research, 212, 6–14.
Ziermans, T. B., Schirmbeck, F., Oosterwijk, F., Geurts, H. M., & De Haan, L. (2020). Autistic traits
in psychotic disorders: Prevalence, familial risk, and impact on social functioning. Psychological
Medicine, 1–10.
Zink, M., Schirmbeck, F., Rausch, F., Eifler, S., Elkin, H., Solojenkina, X., Englisch, S., Wagner, M.,
Maier, W., Lautenschlager, M., Heinz, A., Gudlowski, Y., Janssen, B., Gaebel, W., Michel, T. M.,
Schneider, F., Lambert, M., Naber, D., Juckel, G., … Bechdolf, A. (2014). Obsessive–compulsive
symptoms in at–risk mental states for psychosis: associations with clinical impairment and cognitive
function. Acta Psychiatrica Scandinavica, 130(2), 214–226.
Page | 422
Page | 423
Summary
Summary
Recent years have seen a rise in the modeling of mental disorders as networks of
interacting components. The centerpiece of network modeling lies in the idea that
symptoms are active causal agents in producing disorder states, and that the study
of their interaction is central to progress in understanding and treating mental
disorders. While standard approaches to the psychosis spectrum conceptualize
psychotic disorders as latent conditions, with symptoms being merely the effects of
such a latent condition, the core of the current thesis is to move toward a network
conceptualization of psychosis. Rather than investigating syndromes, this thesis
zooms into concrete symptoms and their interrelations, as well as their interactions
with common risk factors. In addition, the book aims to serve as a guide to applied
researchers and practitioners, introducing guidelines, developments and new
methodologies in the field of network psychometrics.
The book begins with an introduction to novel methodological
conceptualizations to mental health in general, and in particular to psychosis
(Part I). Specifically, Chapter 2 discusses differences between standard approaches
to psychotic disorders and network models, providing a framework to study risk
factors from a network approach and discussing how network models can be
integrated into treatment approaches. This chapter introduces the visualization of
a network structure, with variables (e.g., symptoms, risk factors) being represented
as nodes and the presence of an edge between any two nodes implying the existence
of a statistical association, which does not vanish upon controlling for all of the
other nodes in the network. In standard visualizations as also used in this book,
blue edges indicate positive connections and red edges indicate negative connections
between nodes. Further, comorbidity from a network perspective is discussed in
Chapter 3, focusing on a wide array of mental and physical conditions. A key
conclusion of this first part is that node activation may ‘spread’ faster within the
same domain before reaching other domains, though this finding is generally less
prominent for general psychopathology, where symptoms of varying domains are
also strongly linked to each other.
The second part of the book (Part II) expands on the first chapters, by
investigating individual symptoms and their interactions not only with respect to
each other, but also with respect to genetic and environmental risk factors. Chapter
4 briefly introduces a network–analysis based framework to study environmental
impact in psychotic disorder, focusing on three common environmental risk factors
Page | 424
for psychosis: cannabis use, developmental trauma, and urban environment. The
results indicate the existence of specific paths between environmental risk factors
and symptomatology, as well as more strongly connected networks for individuals
exposed to environmental risk factors. Chapter 5 zooms into a commonly identified
risk factor for psychosis: childhood trauma. The chapter investigates how different
types of traumatic childhood experience relate to symptoms of psychotic disorders
and, by introducing shortest path analysis, aims to identify pathways between
childhood trauma and symptomatology. The findings indicate anxiety may be a
main connecting component between childhood trauma and psychosis, but also
point to other potential paths between trauma and psychosis, and re–emphasize
questions regarding the specificity of trauma as a risk factor to psychosis. Chapter
6 investigates the well–known association between cannabis use and psychotic
symptomatology. The findings of this chapter suggest positive psychotic symptoms
and depression may be exacerbated by cannabis use, while cannabis use may aid
in reducing negative and general psychopathology symptoms. In addition, longer
intensity of cannabis use may potentially lead to smoking, suggesting nicotine
dependence could be an important health consequence of early frequent cannabis
use. Chapter 7 focuses on aberrant perceptional experiences as a potential early
marker of psychosis development, with findings suggesting that affective, but not all
speech illusions are associated with hallucinatory experiences. Chapter 8 proposes
that the identification of genetic variation underlying psychotic disorders may have
suffered due to issues in the psychometric conceptualization of the phenotype and
opens up a new line of research into the genetics of mental disorders by explicitly
incorporating genes into symptom networks. The results indicate that a polygenic
risk score for psychosis is directly linked to the spectrum of positive and depressive
symptoms. The chapter introduces the predictive path diagram, in an attempt to
display how the connectivity of edges is related across genetic and symptomatic
levels of analysis.
The third part of the book (Part III) focuses on psychotic disorders and
comorbid features. Chapter 9 investigates the network structure of obsessive–
compulsive symptoms and symptoms of the at risk mental state for psychosis and
identifies a central role of depression in the network, as well as strong associations
between obsessive–compulsive symptoms and anxiety, social isolation, and blunted
affect, with anxiety being a key item in bridging obsessive–compulsive symptoms
and psychosis. Chapter 10 provides insight into the link between autistic symptoms,
social functioning, and psychosis, in individuals both with and without a psychotic
Page | 425
Summary
disorder diagnosis. The findings suggest that the presence of autistic symptoms is
negatively associated with social functioning across different psychosis vulnerability,
but in the psychotic disorders network symptoms appear to reinforce each other
more easily.
Overall, the key findings of part II and part III strongly support the existence
of an affective pathway to psychosis. Anxiety is identified as the main connecting
component between childhood trauma and paranoia, delusions, and hallucinations,
as well as between obsessive–compulsive symptoms and other psychotic
symptomatology. Further, depression is often a highly central symptom both in
at–risk populations, as well as in patient populations, with findings showing that
depression may further be exacerbated by use of cannabis. Overall, risk factors may
lead to different pathways to the onset and progress of psychotic disorders, party
explaining why some environmental risk factors are often shared between distinct
mental disorder categories. Treatment extended to emotional disturbances, as well
as interventions based on every individual’s vulnerabilities may be essential.
The fourth part of the book moves beyond the field of psychosis, to
methodological advances in the field of network psychometrics. Chapter 11
introduces reporting standards for network models. In Chapter 12 a novel
methodology that allows conducting meta–analysis of network models is presented,
together with findings from a large network meta–analysis of post–traumatic stress
disorder. The meta–analysis shows that no single or small set of nodes clearly plays
a more central role than other nodes, as well as large between–study heterogeneity,
but overall similar network structures between single sample network models and
the pooled meta–analytic network model. Chapter 13 finalizes the thesis with a
large simulation study designed to compare the performance of several estimation
algorithms used in network psychometrics, in an aim to guide applied researchers
toward choosing suitable estimators for their research question(s). The findings of
the simulation study are summarized through an online platform which allows
manually exploring the results. The estimation method is best chosen in light of
each research question, and the results indicate that an exchange between sensitivity
and specificity is to be expected.
In sum, this PhD thesis aims to provide a new framework of thought toward
approaching psychopathology, with a focus on psychosis, and highlights the
importance of moving beyond a single syndrome. The thesis is concluded with
a discussion covering both clinical and methodological considerations of this
research, alongside challenges of current approaches and future research directions.
Page | 426
In the coming years, many more promising research avenues remain to be pursued,
including pre–registration and hypothesis testing of network models. One potential
way to investigate new hypotheses may be through the use of theoretical modeling
and, based on findings from this thesis, the discussion proposes a comprehensive
theoretical model of a imaginary person, George, whose case conceptualization
shows how findings from applied research could serve as the foundation for
a mathematical model. There are many challenges yet to be overcome in a still
maturing field, but many advances have been made in the past years and novel
methodological developments are underway.
Page | 427
Nederlandse samenvatting
In de afgelopen jaren is er een sterke opkomst van modellen waarin psychische
stoornissen worden benaderd als netwerken van op elkaar inwerkende
componenten. De kern van deze netwerkbenadering ligt in het idee dat symptomen
actieve oorzakelijke factoren zijn, die bijdragen aan het ontstaan van stoornissen.
Dit contrasteert met standaardbenaderingen, die psychische stoornissen
conceptualiseren als latente aandoeningen; in deze benadering zijn symptomen niet
meer dan de effecten van een dergelijke latente aandoening. De netwerkbenadering
stelt dat de studie van interacties tussen symptomen essentieel is om psychische
stoornissen beter te begrijpen en te behandelen. Dit proefschrift introduceert een
netwerkconceptualisatie van psychose. In plaats van syndromen te onderzoeken,
focust dit proefschrift op concrete symptomen, hun onderlinge relaties, en hun
interacties met veelvoorkomende risicofactoren. Daarnaast dient dit boek als een
gids voor toegepaste onderzoekers en therapeuten door richtlijnen, ontwikkelingen
en nieuwe methodologieën op het gebied van netwerkpsychometrie te introduceren.
Het boek begint met een inleiding in nieuwe methodologische conceptualiseringen
van geestelijke gezondheid in het algemeen en psychose in het bijzonder
(Deel I). Hoofdstuk 2 bespreekt verschillen tussen standaardbenaderingen en
netwerkbenaderingen van psychotische stoornissen, biedt een kader om risicofactoren
vanuit een netwerkbenadering te bestuderen, en bespreekt hoe netwerkmodellen
kunnen worden geïntegreerd in de behandeling van psychose. Dit hoofdstuk
introduceert de netwerkvisualisatie, waarbij variabelen (bijv. symptomen en
risicofactoren) worden weergegeven als knopen. Een verbinding tussen twee knopen
impliceert een statistische associatie die niet verklaard kan worden door andere
knopen in het netwerk; blauwe verbindingen indiceren positieve relaties en rode
verbindingen negatieve relaties. Hoofdstuk 3 bespreekt verder comorbiditeit vanuit
een netwerkperspectief, waarbij de nadruk ligt op een breed scala aan mentale en
fysieke aandoeningen. Een belangrijke conclusie van Deel I is dat symptomen elkaar
eerder lijken te kunnen activeren binnen hetzelfde domein, voordat een ander domein
wordt bereikt. Deze bevinding is over het algemeen minder prominent voor algemene
psychopathologie, waar symptomen van verschillende domeinen ook sterk met elkaar
in verband staan.
Het tweede deel van het boek (Deel II) borduurt voort op de eerste
hoofdstukken door niet alleen interacties tussen individuele symptomen te
onderzoeken, maar ook interacties tussen deze symptomen en genetische
Page | 428
factoren en omgevingsrisicofactoren te bestuderen. Hoofdstuk 4 introduceert
kort een op netwerkanalyse gebaseerd raamwerk voor het bestuderen van de
impact van omgevingsfactoren bij psychotische stoornissen, met een nadruk op
drie veelvoorkomende omgevingsrisicofactoren voor psychose: cannabisgebruik,
ontwikkelingstrauma en urbanisatie. De resultaten wijzen op het bestaan van
specifieke paden tussen omgevingsrisicofactoren en symptomatologie, evenals
op sterker verbonden netwerken voor individuen die zijn blootgesteld aan
omgevingsrisicofactoren. Hoofdstuk 5 zoomt in op een vaak geïdentificeerde
risicofactor voor psychose: jeugdtrauma. Het hoofdstuk onderzoekt hoe
verschillende soorten traumatische ervaringen uit de kindertijd verband houden
met symptomen van psychotische stoornissen. Door de introductie van de
kortstepadanalyse probeert het hoofdstuk verder de paden tussen kindertrauma
en symptomatologie te identificeren. De bevindingen wijzen erop dat angst een
belangrijke verbindende component kan zijn tussen traumatische ervaringen in
de kindertijd en een latere psychose, maar wijzen ook op andere mogelijke paden
tussen trauma en psychose. Verder worden opnieuw vragen over de specificiteit van
trauma als een risicofactor voor psychose benadrukt. Hoofdstuk 6 onderzoekt de
associatie tussen cannabisgebruik en psychotische symptomen. De bevindingen
van dit hoofdstuk suggereren dat positieve psychotische symptomen en depressie
kunnen worden verergerd door cannabisgebruik, terwijl cannabisgebruik juist
kan helpen bij het verminderen van negatieve en algemene psychopathologische
symptomen. Bovendien kan een langdurig en intensief cannabisgebruik mogelijk
leiden tot een hogere frequentie van roken in het algemeen, wat suggereert dat
nicotineverslaving een belangrijk gezondheidseffect kan zijn van vroeg frequent
cannabisgebruik. Hoofdstuk 7 richt zich op afwijkende waarnemingservaringen
als een mogelijk vroeg teken van psychose–ontwikkeling en beschrijft dat
affectieve spraakillusies geassocieerd kunnen zijn met hallucinatoire ervaringen,
terwijl dit niet voor typen illusies geldt. Hoofdstuk 8 stelt dat problemen in de
psychometrische conceptualisering van het fenotype mogelijk de identificatie van
genetische variatie die ten grondslag ligt aan psychotische stoornissen in de weg
staat. Het hoofdstuk opent een nieuwe onderzoeksrichting naar de genetica van
psychische stoornissen door genen expliciet op te nemen in symptoomnetwerken.
De resultaten geven aan dat een polygenetische risicoscore voor psychose direct
verband houdt met het spectrum van positieve en depressieve symptomen. Het
hoofdstuk introduceert verder het voorspellende paddiagram, dat de connectiviteit
tussen genetische en symptomatische analyseniveaus weergeeft.
Page | 429
Het derde deel van het boek (Deel III) richt zich op psychotische stoornissen
en comorbide kenmerken. Hoofdstuk 9 onderzoekt de netwerkstructuur van
obsessief–compulsieve symptomen en symptomen van de mentale risicotoestand
met betrekking tot psychose en identificeert een centrale rol van depressie in het
netwerk, evenals sterke associaties tussen obsessief–compulsieve symptomen en
angst, sociaal isolement en afgestompt affect, waarbij angst een belangrijk aspect is
bij het overbruggen van obsessief–compulsieve symptomen en psychose. Hoofdstuk
10 geeft inzicht in het verband tussen autistische symptomen, sociaal functioneren
en psychose, zowel bij mensen met als zonder diagnose psychotische stoornis.
De bevindingen suggereren dat de aanwezigheid van autistische symptomen in
negatief verband staat tot sociaal functioneren. Deze verbanden zijn zichtbaar op
elk niveau van kwetsbaarheid voor psychose. Symptomen lijken elkaar makkelijker
te beïnvloeden in een netwerk gebaseerd op personen met een diagnose psychotische
stoornis dan in een netwerk gebaseerd op personen zonder deze diagnose.
Over het algemeen ondersteunen de belangrijkste bevindingen van deel II en deel
III het bestaan van een affectieve route naar psychose. Angst wordt geïdentificeerd als
de belangrijkste verbindende component tussen trauma uit de kindertijd en paranoia,
wanen en hallucinaties, evenals tussen obsessief–compulsieve symptomen en andere
psychotische symptomatologie. Verder is depressie vaak een zeer centraal symptoom,
zowel in risicopopulaties als in patiëntenpopulaties, waarbij bevindingen aantonen
dat depressie verder kan worden verergerd door het gebruik van cannabis. Over het
algemeen kunnen risicofactoren leiden tot verschillende wegen naar het ontstaan
en de voortgang van psychotische stoornissen, wat verklaart waarom sommige
omgevingsrisicofactoren vaak samenvallen met verschillende categorieën van
psychische stoornissen. Behandeling van emotionele stoornissen, evenals interventies
op basis van de kwetsbaarheden van elk individu, kunnen essentieel zijn.
Het vierde deel van het boek (Deel IV) gaat voorbij het gebied van psychose en
focust op methodologische vooruitgangen op het gebied van netwerkpsychometrie.
Hoofdstuk 11 introduceert rapportagestandaarden voor netwerkmodellen.
Hoofdstuk 12 presenteert een nieuwe methodologie die het mogelijk maakt om
een meta–analyse van netwerkmodellen uit te voeren, samen met bevindingen
van een grote meta–analyse van netwerkmodellen voor posttraumatische
stressstoornis. Uit deze meta–analyse blijkt dat er geen groep symptomen kan
worden geïdentificeerd die duidelijk een meer centrale rol speelt. Verder legt de
meta–analyse een grote mate van heterogeniteit tussen de studies bloot. Echter
laat het hoofdstuk ook zien dat netwerken gebaseerd op aparte studies resultaten
Page | 430
opleveren, die vergelijkbaar zijn met het gepoolde meta–analytische netwerkmodel.
Hoofdstuk 13 rondt het proefschrift af met een grote simulatiestudie waarin
verschillende schattingsalgoritmen die worden gebruikt in netwerkpsychometrie
worden vergeleken, met als doel toegepaste onderzoekers te begeleiden bij het
kiezen van een algoritme dat past bij hun onderzoeksvraag. De bevindingen van
de simulatiestudie zijn samengevat in een online platform dat het mogelijk maakt
om de resultaten handmatig te verkennen. De schattingsmethode kan het beste
worden gekozen in het licht van de afzonderlijke onderzoeksvraag, en de resultaten
geven aan dat sensitiviteit en specificiteit vaak ten koste van elkaar gaan.
Kortom, dit proefschrift heeft tot doel een nieuw denkkader te bieden voor
het benaderen van psychopathologie met een focus op psychose en benadrukt
het belang om verder te kijken dan naar één enkel syndroom. Het proefschrift
wordt afgesloten met een discussie over zowel klinische als methodologische
overwegingen van dit onderzoek, kritische overwegingen van de gebruikte
methode, en toekomstige onderzoeksrichtingen. In de komende jaren moeten nog
veel veelbelovende onderzoekspaden worden bewandeld, waaronder preregistratie
en hypothesetesten van netwerkmodellen. Een mogelijke manier om een nieuwe
hypothese te onderzoeken is door het gebruik van theoretische modellering. Op
basis van bevindingen uit dit proefschrift stelt de discussie een uitgebreid theoretisch
model voor van een imaginaire persoon, George, die dient als voorbeeld om te laten
zien hoe bevindingen uit toegepast onderzoek kunnen dienen als basis voor een
wiskundig model. Er zijn nog veel uitdagingen die moeten worden overwonnen in
een nog steeds rijpend gebied, maar er is de afgelopen jaren veel vooruitgang geboekt
en nieuwe methodologische ontwikkelingen zijn gaande.
Page | 431
Funding
This thesis project was supported by the Netherlands Organisation for Scientific
Research (NWO), grant number 406.16.516.
Publication List
Edited Books
Isvoranu, A. M., Epskamp, S., Waldorp, L. J., & Borsboom, D. (Eds.). (in press).
Network Psychometrics with R: A Guide for Behaviorial and Social Scientists.
Routledge, Taylor & Francis Group.
First Author
Isvoranu, A. M., Borsboom, D., van Os, J., & Guloksuz, S. (2016). A Network
Approach to Environmental Impact in Psychotic Disorder: Brief Theoretical
Framework. Schizophrenia Bulletin, 42(4), 870–873.
Isvoranu, A. M., van Borkulo, C. D., Boyette, L. L., Wigman, J. T. W., Vinkers,
C. H., Borsboom, D., & GROUP Investigators (2017). A Network Approach
to Psychosis: Pathways Between Childhood Trauma and Psychotic Symptoms.
Isvoranu, A. M., Guloksuz, S., Epskamp, S., van Os, J., Borsboom, D., & GROUP
Investigators (2019). Toward Incorporating Genetic Risk Scores into Symptom
Networks of Psychosis. Psychological Medicine, 50(4), 636–643.
Isvoranu, A. M., Boyette, L. L., Guloksuz, S., & Borsboom, D. Network Models
of Psychosis. In: Tamminga, C.A., Ivleva, E.I., Reininghas, U., van Os, J. (2020).
Psychotic Disorders: Comprehensive Conceptualization and Treatments. New York,
NY: Oxford University Press.
Boyette, L. L*., Isvoranu, A. M.*, Schirmbeck F., et al. (2020). From Speech Illusions
to Onset of Psychotic Disorder: Applying Network Analysis to an Experimental
Measure of Aberrant Experiences. Schizophrenia Bulletin Open.
Isvoranu, A. M., Abdin, E., Chong, S. A., Vaingankar, J., Borsboom, D., &
Subramaniam, M. (2021). Extended Network Analysis of Symptomatology: From
Psychopathology to Chronic Illness. BMC Psychiatry, 21(1), 1–9.
Page | 432
Isvoranu, A. M., Boyette, L. L, van Borkulo, C., de Haan, L., & Borsboom,
D. (submitted). Cannabis (Ab)use in Psychosis: From Short– to Long–Term
Symptomatology.
Isvoranu, A. M. & Epskamp, S. (2021). Which Estimation Method to Choose
in Network Psychometrics? Deriving Guidelines for Applied Researchers.
Isvoranu, A. M., Epskamp, S., & Cheung, M. (2021). Network Models of Post–
traumatic Stress Disorder: A Meta–analysis. Journal of Abnormal Psychology.
Ong, H. L*., Isvoranu, A. M.*, Schirmbeck, F., et al. (2021). Obsessive–Compulsive
Symptoms and Other Symptoms of the At Risk Mental State for Psychosis: A
Network Perspective. Schizophrenia Bulletin.
Isvoranu, A. M.*, Ziermans, T.*, Schirmbeck, F., Borsboom, D., Geurts, H.,
de Haan, L., & GROUP Investigators (2021). Autistic Symptoms and Social
Functioning in Psychosis: A Network Approach. Schizophrenia Bulletin.
Burger, J.*, Isvoranu, A. M.*, Lunansky, G., Haslbeck, J. M. B. Epskamp, S.,
Hoekstra, R. H. A., Fried, E. I., Borsboom, D., & Blanken, T. F. (under review).
Reporting Standards for Psychological Network Analyses in Cross–sectional Data.
Isvoranu, A. M. & Epskamp, S. (in press). Constructing and Drawing Networks
in qgraph. In Isvoranu, A. M., Epskamp, S., Waldorp, L. J., & Borsboom, D.
(Eds.), Network Psychometrics with R: A Guide for Behaviorial and Social Scientists.
Routledge, Taylor & Francis Group.
Last Author
Köhne, A. & Isvoranu, A. M. (2021). A Network Perspective on the Comorbidity

of Personality Disorders and Mental Disorders: An Illustration of Depression and
Borderline Personality Disorder. Frontiers in Psychology, 12, 1–9.
Lunansky, G., Epskamp, S., & Isvoranu, A. M. (in press). Short Introduction to R.
In Isvoranu, A. M., Epskamp, S., Waldorp, L. J., & Borsboom, D. (Eds.), Network
Psychometrics with R: A Guide for Behaviorial and Social Scientists. Routledge,
Taylor & Francis Group.
Page | 433
Collaborations
van Rooijen, G., Isvoranu, A. M., Meijer, C. J., van Borkulo, C.D., , Ruhé H. G.,
de Haan, L., & GROUP Investigators (2017). A Symptom Network Structure of
the Psychosis Spectrum. Schizophrenia Research, 189, 75–83.
Abacioglu, C. S., Isvoranu, A. M., Epskamp, S., Verkuyten, M., & Thijs, J. (2018).
Exploring Multicultural Classroom Dynamics: A Network Analysis. Journal of
School Psychology, 74, 90–105.
Epskamp, S., van Borkulo, C. D., van der Veen, D. C., Servaas, M. N., Isvoranu,
A. M., Riese, H., & Cramer, A. O. J (2018). Personalized network modeling in
psychopathology: The importance of contemporaneous and temporal connections.
Clinical Psychological Science, 6(3), 416–427.
Fonseca–Pedrero, E., Ortuño, J., Debbané, M., Chan, R. C., Cicero, D., Zhang, L.
C., ... , Isvoranu, A. M., Epskamp, S., & Fried, E. I. (2018). The network structure
of schizotypal personality traits. Schizophrenia Bulletin, 44(suppl_2), S468–S479.
van Rooijen, G., Isvoranu, A. M., Kruijt, O. H., van Borkulo, C. D., Meijer, C. J.,
Wigman, J. T. W., Ruhé, H. G., de Haan, L., & GROUP Investigators (2018). A state–
independent network of depressive, negative and positive symptoms in male patients
with schizophrenia spectrum disorders. Schizophrenia Research, 193, 232–239.
Borsboom, D., Deserno, M. K., Rhemtulla, M., Epskamp, S., Fried, E. I., McNally,
R. J., Robinaugh, D. J., Perugini, M., Dalege, J., Costantini, G., Isvoranu, A.
M., Wysocki, A. C., van Borkulo, C. D., van Bork, R., & Waldorp, L. J. (2021).
Network Analysis of Multivariate Data in Psychological Science. Nature Reviews
Methods Primers, 1(58), 1–18.
Epskamp, S., Isvoranu, A. M., Cheung, M. (2021). Meta–analytic Gaussian
Network Aggregation. Psychometrika.
Liu, D., Epskamp, S., & Isvoranu, A. M. (2021). Network Analysis of Physical and
Psychiatric Symptoms of Hospital Discharged Patients Infected with COVID–19.
Journal of Affective Disorders, 294, 707–713.
Moura, B.M., Isvoranu, A. M., Kovacs, V., van Rooijen, G., van Amelsvoort,
T., Simons, C. J. P., Bartels–Ve;thuis, A. A., Bakker, P. B., Marcelis, M., de
Haan, L., Schirmbeck, F. (under review). The Puzzle of Functional Recovery
in Schizophrenia–Spectrum Disorders – Revisiting a Network Analysis Study.
Schizophrenia Bulletin.
Page | 434
Blanken, T. F., Isvoranu, A. M., & Epskamp, S. (in press). Estimating Network
Structures using Model Selection. In Isvoranu, A. M., Epskamp, S., Waldorp, L.
J., & Borsboom, D. (Eds.), Network Psychometrics with R: A Guide for Behaviorial
and Social Scientists. Routledge, Taylor & Francis Group.
Deserno, M. K., Isvoranu, A. M., Epskamp, S., & Blanken, T. F. (in press).
Descriptive Analysis of Network Structures. In Isvoranu, A. M., Epskamp, S.,
Waldorp, L. J., & Borsboom, D. (Eds.), Network Psychometrics with R: A Guide
for Behaviorial and Social Scientists. Routledge, Taylor & Francis Group.
Epskamp, S., Haslbeck, J. M. B., Isvoranu, A. M., & van Borkulo, C. D. (in press).
Pairwise Markov Random Fields. In Isvoranu, A. M., Epskamp, S., Waldorp, L.
J., & Borsboom, D. (Eds.), Network Psychometrics with R: A Guide for Behaviorial
and Social Scientists. Routledge, Taylor & Francis Group.
*denotes shared first authorship
Page | 435
Short CV
Grants
James S. McDonnell Foundation (JSMF) Postdoctoral Fellowship Award

September 2019, Amsterdam, The Netherlands
NWO Talent Grant: four-year PhD grant

September 2016, Amsterdam, The Netherlands
Research Priority Area Yield: four-year PhD grant (declined for NWO)
August 2016, Amsterdam, The Netherlands
Awards
Best Presentation Award Winter Conference Interuniversity Graduate School of

Psychometrics and Sociometrics (IOPS)
December 2019, Leiden, The Netherlands
Unilever Research Prize

December 2017, Vlaardigen, The Netherlands
Poster Award: presentation Research Master Graduate Conference

October 2016 & October 2015, Amsterdam, The Netherlands
Adela-Maria Isvoranu was born in Bucharest, Romania on August 14, 1992.

She completed her Bachelor’s degree in Psychology at the University of Leeds, in
the United Kingdom. Following her Bachelor’s degree, she obtained a Research
Master’s degree in Psychology at the University of Amsterdam, in The Netherlands,
with a specialization in Psychological Research Methods and a minor in Clinical
Psychology. In 2016, she received the NWO Talent Grant to start her PhD project,
under the supervision of Prof. Denny Borsboom (University of Amsterdam) and
Prof. Jim van Os (Utrecht University). Between August 2019 and October 2019,
Adela visited the Institute of Mental Health in Singapore, where she worked on
Chapter 3 of this thesis.
Page | 436
Page | 437
Acknowledgements – Dankwoord
My PhD journey was a wonderful experience, with support from many old and
new friends. Thank you all for these great years!
Denny, thank you so much for being my supervisor and accompanying me through
this journey, starting with my master’s and following with my grant applications and
PhD research. I don’t think I ever got the chance to tell you, but when I approached
you as a student to ask if you would be interested in applying for an NWO talent
grant with me, I never expected that you would say ‘yes’. It all happened a bit last
minute and so many people at the time told me how busy you are and how you could
only support one student at a time, so when you agreed, I was extremely happy. And,
of course, even more so when I learned the positive outcome and that I would get
to spend the next four years doing a PhD in Amsterdam. While I often hear people
around me struggling throughout their PhD, I regard my PhD years to be some of
the best years of my life and much of this is thanks to you. You always gave me the
freedom to choose the research I was interested in doing, trusted me to venture into
many collaborations, organize my own time, and travel the world attending so many
conferences I found interesting. This taught me so much and gave me a lot of self-
confidence, allowing me not only to develop as a scientist, but also as a person. No
matter how busy, you were always there when I needed help, advice, and guidance, but
never pressured me and always worked with me on my own pace. I found this to be
a great privilege and I really appreciate it. I also really enjoyed all the cigarette breaks
outside Diamantbeurs, the G building, CREA, and Kriterion. I find it great that you
always join the student group chats, and are always so open and friendly. You create a
wonderful academic environment and I am so happy to have had the chance to carry
out my research under your supervision.
Jim, thank you so much for being my co-promotor! Irrespective of how busy
you were, you always gladly said ‘yes’ to any meeting requests, and were always so
happy to discuss potential new ideas. After every single one of our talks, I had so
many pages full of notes and so many new projects I could pick up. I learned so
much from your expertise and ideas!
While presenting one of my posters at a SIRS conference, I was approached by
a researcher and the first thing he told me was: “I saw your supervisors are Denny
Borsboom and Jim van Os, that is so amazing, how did you manage this?”. This
brought such a big smile on my face and I feel so fortunate to have had you both as PhD
supervisors. Thank you both once again for these wonderful years and fantastic PhD!
Page | 438
I would especially like to thank my family for their support (in Romanian this time).
Mami, mulțumesc pentru tot sprijinul acordat. Datorită ție am devenit persoana care
sunt și sper că știi cât de mult te apreciez. Mereu ai crezut în mine și m-ai încurajat să
îmi urmez calea, m-ai susținut când am decis să plec din țară și ai fost alături de mine
cu fiecare pas pe care l-am făcut. Îți sunt și voi fi mereu recunoscătoare pentru tot ce
ai făcut pentru mine, ești o persoană extraordinară! Mia, mereu ai fost un exemplu
pentru mine și sora pe care nu am avut-o. Mulțumesc pentru că ai fost alături de mine,
mulțumesc pentru vizite, sfaturi, și pentru excursiile și aventurile pe care le-am trăit
împreună. Flori, Costel, mulțumesc pentru că ne-ați oferit întotdeauna o familie mare
și unită și am crescut înconjurate de zâmbete și fericire. Mamaie, tataie, mulțumesc
că m-ați crescut cu atâta dragoste și răbdare. Tataie, tati, îmi lipsiți enorm, dar mereu
veți ocupa un loc special în sufletul meu.
Sacha, if I were to write down all the things I would like to thank you for, I
might end up writing down more pages than I already wrote down for my thesis…
Thank you for all the years spent together and for all the absolutely amazing trips
we did throughout the world, from Europe, to America, to Asia. Thank you for
being the best diving buddy and for teaching me to tolerate spicy food! Thank you
for always being a great research collaborator and co-organizer of the Summer and
Winter schools. I had so much fun writing papers together, teaching workshops,
going on writing weeks, and visiting so many new places. Thank you for always
cooking for me, thank you for always celebrating every event (and enjoying my
cheesy-ness), from Valentine’s day to all kinds of anniversaries. Thank you for
indulging me in playing computer games all night long, and for starting to celebrate
Christmas and agreeing to transform our house in a Christmas wonderland! Thank
you for introducing me to Singapore and for the wonderful three months spent
there, with the loveliest marriage proposal. Thank you for being the best pandemic
partner, and for making our house such a cozy place during this scary time. Thank
you for discovering all the small cute places in our neighborhood and for bringing
me breakfast in bed every morning during pregnancy. Thank you for always being
there for me, during happy and difficult times. Last, but not least, thank you for
being the best husband and father to Ollie. I look forward to our future together
and the many more adventures that await us.
Ollie, it may be a long time before you can read this, but you have been there
with me during every moment of the final steps of my PhD. As somebody put it,
“my greatest creations in life in one year”. Thank you for showing me a kind of
love I never experienced before and for being the cutest baby in the whole world!
Page | 439
Hugo, thank you for being my first cat and being a cuddly moral support during the
difficult days of the pandemic and the lazy days of Netflix-ing on the sofa. Thank
you Tilly and Tiny for being the best fluffies and joining us in all of our road trips!
I would very much like to thank my paranymphs for the wonderful years of
friendship, starting with the research master until today. Ceren, I had so much fun
in our early days in the master’s, hanging out together working through endless
assignments, drinking cheap red wine, and smoking on the small balcony of my
studio. You are always such a happy presence, teaching everyone how to enjoy life
and always bringing so many smiles! Maien, thank you for always being a such
a caring friend, cooking delicious dinners and cakes, and never forgetting any
important dates! I really enjoyed our Fridays at CREA, our coffee breaks, and
long walks during the pandemic. I feel so happy and lucky to have you both in my
life and as my paranymphs.
There are so many people and colleagues I met over the years and listing them
all down is so difficult. Mijke, thank you for being a wonderful supervisor in my
master’s, for the interesting research visit in Davis, and for the lovely baby-welcome
gift. Lindy, thank you for trusting a first-year research master student with starting
(and finishing) a large research project, for always being a great collaborator, for the
wonderful days spent together in Florence, and for the lovely Oliver bib. Frederike,
thank you for all the many collaborations and interesting projects we worked on
together. Mythili, thank you for a fantastic research visit in Singapore and I hope
we will collaborate again in the future. Mike, thank you for offering me a space in
Experimental Cubicle 6 and for the great collaboration. I look forward to working
together on more projects.
Gaby thank you for always joining in our trips, from Greece, to Paris, to
Thailand, Singapore, and Romania, and of course for enjoying watching Eurovision
just as much as we do! Jonas, thank you for the fun trips, always encouraging a
glass of Gin Tonic, and hopefully a soon-to-be trip to Santa Fe. Ria, thank you for
the happy office vibes and for all the fun years we had organizing the Summer and
Winter schools together. Eiko, thank you for always being a kind and inspiring
person, for encouraging me to follow the Network Analysis class as a student, and
for all the fun boardgame evenings. Jonas, thank you for being my desk neighbor,
helping out with mgm-questions, for all the visits and boardgame nights trying
to defeat the pandemic! Zara, thank you for your friendship and for always being
such a unique person, you are very much missed. Johnny, Damiano, Ruben, Ale,
Eliza, Joris, thank you for all the fun parties and DND adventures. Leonie, thank
Page | 440
you for your friendship, all the fun IOPS conferences, and visits to Amsterdam.
Giulio, thank you for the fun conference hang-outs and for getting us excited about
Formula 1! Sanne, thank you for being the best student advisor and for the balcony
chats during the pandemic, we miss you as a neighbor! Hui-Lin, thank you for
becoming a friend, introducing me to the IMH, and showing us around Singapore!
Dominique and Lucy, thank you for the fun evenings in Singapore and hope we’ll
see each other soon again! Thank you to all the people I met while traveling to
quite a number of (inter)national conferences and to all the colleagues from IOPS,
which made every conference interesting and inspiring. Thank you to everyone in
the Pyschosystems and the PML group I did not yet get to mention, Riet, Claudia,
Marie, Tessa, Julian, Lisa, Angelika, Quentin, Akash, Alexandra, Alexander, Udo,
Fabian, Don, Nathan, Ravi, Jonathon, Dora, Angélique, Jolanda, Maarten, Claire,
Lilian, Lourens, EJ, Han. Thank you for always keeping such a great and friendly
work atmosphere!
I would especially like to thank my committee members, Claudi, Lindy, Marie,
Richard, Han, and Lieuwe, for accepting to be part of this final step and taking the
time to read through my thesis.
Thank you as well to my Dutch family, Ellen, Piet, Yori, Oma, Hans for
welcoming me into your family and introducing me to the Dutch culture.
I would also like to thank all my friends who are not part of the UvA. Vebby,
thank you so much for always being there for me and joining me in all my life
adventures, from binge-watching anime, to long clubbing nights, visits to the UK
and Singapore, and finally to taking the decision to move to Amsterdam. I am so
lucky to have you as a friend. Stef, thank you for being the best housemate during
our studies in the UK. I loved how we used to always hang out in the kitchen till
late at night, with a glass of wine sharing life stories, or playing Plants vs Zombies. I
really miss having you around all the time, our dinners eating smothered chicken at
the Slug & Lettuce, late in the night chicken wraps, and evenings at Rock of Ages!
Miruna, thank you for your happy vibes, always being the soul of the party and a
great friend, and never missing an important event. Renze, Sitara, thank you for the
fun visits in Utrecht and walks staring at our not-yet-ready house. I look forward
to being your neighbors! Puf, thank you for your many years of friendship, sharing
a desk with me in high-school, seaside trips and long nights in Kulturhaus. Thank
you for answering all my pregnancy-related questions and worries and for already
being a wonderful godmother to Ollie.
Page | 441

PHD Thesis Adela Is Voran U

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PHD Thesis Adela Is Voran U

Uploaded by

Copyright:

Available Formats

F rom syndromes to Symptoms:

Network Models of Psychosis and beyond

© 2021 Adela-Maria Isvoranu

This research was supported by an NWO talent grant,

Cover design: @alphacreative.des

Promotor: prof. dr. D. Borsboom Universiteit van Amsterdam

Promotor: Prof. dr. D. Borsboom

Faculteit: Faculteit der Maatschappij- en Gedragswetenschappen

Unlike common mental disorders uncovered since the period of antiquity,

This chapter is adapted from:

framework. We concentrate on how (environmental and genetic) risk factors can be

2.2 Psychosis: disease model versus network model

With this framework in mind, extensive research has focused on identifying

have seen an emergence of network models of mental disorders, in which disorders

2.3 Toward incorporating environmental and

Widespread research has focused on identifying biological and environmental

External Field External Field

Hos Del Hos Del

risk factors–such as a polygenic risk score–and symptoms, (3) both environmental

2.4 Conceptualizing treatment from a network

(partly) personalized network model capturing the person’s most important

2.5 Concluding Remarks

This chapter is adapted from:

therefore be useful tools to study both within-diagnoses and between-diagnoses

3.2.3 Statistical analysis

Table 3.1 Study measures

3.3.2 Network analysis

To summarize the results of our analysis, we will first highlight within-domain

Table 3.2 Demographic profile distribution

Oppositional Defiant Disorder

9 6 Functioning and Health−related Quality of Life

Social Phobia Chronic Conditions

Oppositional Defiant Disorder

Sad, Empty, Depressed7 ●

Loss of Interest (e.g., work, hobbies)9 ●

Losing Temper (childhood or adolescence)

Breaking Rules (childhood or teenage years)

Really Shy with People

Problems with Concentration as Child

Anger Attack: Hit or Tried to Hit Someone6 ●

Feeling Upset when Separated from Family Members (adulthood)

Anger Attack: Broke or Smashed Something5 ●

Being Fidgety (as a child)

Feeling Upset when Separated from Mother

High Blood Sugar/ Diabetes

3.3.4 Network replicability and robustness

most chronic conditions were associated with items related to functioning/

by zooming into and bringing together a multitude of health-related domains.

3.5 Concluding remarks

This chapter is adapted from:

4.2 Psychotic disorder – standard approaches and a

4.3 Network analysis of general population

(a) Interrelations Environmental Exposure & Symptomatology

Anx Environmental Exposure

(b) No Environmental Exposure (c) Environmental Exposure Cannabis

Sen OCD Sen OCD

This chapter is adapted from:

other depression symptoms (Fried, Bockting, et al., 2015). In addition, non-DSM

5.2.3 Childhood trauma

5.2.4 Network construction

5.2.5 Network analysis

Item label Domain color Item description

GP12 N1 GP7 GP12