Journal of Fluency Disorders 67 (2021) 105823

Contents lists available at ScienceDirect

Journal of Fluency Disorders

journal homepage: www.elsevier.com/locate/jfludis

Evaluating three stuttering assessments through network analysis,

random forests and cluster analysis
David Ward a, *, Ronan Miller a, Alexandre Nikolaev b
a
School of Psychology and Clinical Language Sciences, University of Reading, Reading, RG6 6AL, UK
b
School of Languages and Cultures, University of Sheffield, Sheffield S3 7RA, UK, and Department of Languages, University of Helsinki, Helsinki FI-
00014, Finland

A R T I C L E I N F O A B S T R A C T

Keywords: Purpose: In stuttering, cognitive and behavioural variables interact in nonlinear fashion. These
Stuttering assessment variables can be assessed by instruments which evaluate perceived impact of stuttering and
Network science stuttering severity. We applied three statistical methods in combination to the analysis of three
Random forests
assessment protocols to discover relationships within and between the tests to better understand
Cluster analysis
variations in behavioural and social aspects of stuttering.
Methods: Scores from Stuttering Severity Index (SSI-IV), Overall Assessment of the Speaker’s
Experience of Stuttering scale (OASES), and Unhelpful Thoughts and Beliefs About Stuttering
scale (UTBAS), collected from 26 participants were compared using three statistical methods:
network analysis, random forests, and cluster analysis.
Results: Network analysis demonstrated that SSI-IV only weakly interacts with a quality of life
index (OASES) and a self-perception and belief systems index (UTBAS). Random forest analyses
revealed the last two measures relate strongly to each other. The results from cluster analysis
suggest a) a possible regrouping of OASES items and b) a possible use of one UTBAS scale instead
of the three.
Conclusion: A combination of three statistical methods allowed us to evaluate the three assess­
ments in more depth. The lack of interaction between the SSI-IV on the one hand, and OASES and
UTBAS on the other, suggests that the network of the three commonly used stuttering assessments
may be fractured in a non-productive way. A potential gap may exist for an assessment tool that
would link behavioural and social aspects of stuttering.

1. Introduction

Within the field of fluency disorders, a range of assessment measures have been developed to evaluate various aspects of stuttering.
For example, the Stuttering Severity Index (SSI-IV, Riley, 2009) is a clinician administered measure used to assess the severity of
frequency, duration, and physical concomitants of stuttering. The SSI-IV claims its construct validity to be stuttering severity, although
questions have been raised as to its inter and intra judge reliability (Davidow & Scott, 2017). In the same vein, the Overall Assessment
of the Speaker’s Experience of Stuttering (OASES, Yaruss & Quesal, 2010) is a self-report measure designed to be used alongside
clinician administered instruments to collect information regarding the impact of various aspects of stuttering. It contains 100-items

* Corresponding author.
E-mail addresses: d.ward@reading.ac.uk (D. Ward), r.l.miller@reading.ac.uk (R. Miller), alexandre.nikolaev@sheffield.ac.uk (A. Nikolaev).

https://doi.org/10.1016/j.jfludis.2020.105823
Received 14 May 2020; Received in revised form 1 November 2020; Accepted 2 December 2020
Available online 9 December 2020
0094-730X/© 2020 Elsevier Inc. All rights reserved.
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

across 4 sections (I General information, II Reactions to stuttering, III Communication in daily situations, and IV Quality of life), scored
against a 5-point Likert scale. Similarly, the Unhelpful Thoughts and Beliefs about Stuttering scale (UTBAS, Iverach et al., 2011) is a
self-report measure that returns information on the frequency of unhealthy self-perceptions and belief systems that contribute to
speech-related anxiety. At a meta-level, these assessments share their construct validity, namely stuttering.1 SSI-IV (particularly the
stuttering frequency component), OASES, and UTBAS have been widely employed in stuttering research and clinical assessment
(Alameer, Meteyard, & Ward, 2017). It is important to explore these relationships to better understand the (often nonlinear) rela­
tionship between stuttering severity, quality of life and belief systems. For example, many people who stutter at what would be
regarded as low levels may experience acute distress because of their fear of stuttering as much as the act of stuttering itself. The more
we understand about these relationships, the better we can focus our therapy (Sønsterud, Halvorsen, Feregen, Kirmess, & Ward, 2020).
Recent statistical tools, such as network analysis (NA, which originates from graph theory, Newman, 2010), random forests (RF, a
machine learning method developed by Breiman, 2001), in combination with cluster analysis (CA) allow us to shed more light on the
relationships between these different tests/scales.
Fluency clinicians and researchers recognise the need to consider both behavioural and cognitive changes and relatedly, quality of
life when assessing efficacy. These are parameters which cannot be assessed within a single instrument. The three assessments we
examine reflect the need to cover these areas (e.g. Alameer et al., 2017; Freud & Amir, 2020; Sønsterud et al., 2019).
The nature of the present study is exploratory, not confirmatory. That is why we do not state a priori hypotheses. We base our study
within a theoretical framework of Network analysis accompanying it with the two complementary analyses mentioned above and
discussed later in this section. Contrary to latent variable models which mainly focus on what is common among different scales and
measures, we focus our attention on what is specific to these scales and measures in the context provided by other scales and measures
in the network.
These methods are particularly well-suited to deal with the problems that typically accompany large data sets collected from a
small sample of population. Assessing stuttering typically results in a large number of variables. We aim to study each of them in the
context provided by all other scales and measures collected. Thus, correlating a large number of variables (e.g., scores for a range of
measurements and scales) to each other can easily lead to overparameterization of the model, which in turn increases a risk of false
positive findings. A small sample size seems to be a loose description, which has not yet been well-defined (Epskamp, Kruis, &
Marsman, 2017). A general rule would be to have at least as many participants as the number of parameters (which can be calculated
with the following equation: P(P − 1)/2, where P = number of variables collected). However, large data sets with a small sample size
often violate this general rule. One solution is to increase a sample size. However, recruiting large numbers of people who stutter (PWS)
is often a problem in stuttering research (Iverach & Rapee, 2014). Thus, when recruiting more participants is problematic, one can
apply statistical methods such as RF or NA, each of which deals with the so called “small n large p” problem by using different solutions
to minimise the likelihood of a false positive outcome (see Section 2.3).
Network analysis research is attracting increasing attention in the psychological sciences due to its ability to reveal complex re­
lations between many variables (Borgatti, Mehra, Brass, & Labianca, 2009; Neal & Neal, 2017). Networks consist of nodes (typically
depicted by circles) and edges (lines) that connect nodes. Nodes could represent people in social networks, or they could represent
variables, e.g., aggregates of OASES and UTBAS scores collected from a group of PWS. If the two nodes (e.g., OASES total and UTBAS
total) are connected, but there is no information about the intensity of their relation, then this network is unweighted. However, if
there is some information about the intensity of their correlation, then it is usually visualized by the thickness and the length of the
edge between these two nodes. Positive associations between the variables could be depicted by, e.g., blue edges, while negative ones
by red. When the relationship between the variables is symmetrical, the edges are undirected (as in the present study), however, if the
relationship is not symmetrical, edges could be directed (depicted by arrowheads; they could represent, e.g., temporal dependencies,
when one variable precedes another). Typically, networks are built based on partial correlation of variables. The difference between
classic and partial correlation is that in the latter the edge between the two nodes is drawn only after removing the variance that these
two nodes share with all other nodes in the network (see more detailed discussion regarding this topic in Costantini & Perugini, 2020).
Thus, networks consist of nodes representing observed variables (e.g., behaviour measures), connected by edges estimated from
data and thus representing statistical relationships (Epskamp, Borsboom, & Fried, 2018). As such, this approach would appear well
suited to the analysis of stuttering, where a wide range of cognitive and behavioural variables can be seen to interact in nonlinear
fashion (Smith & Weber, 2017), for example, the relationship between stuttering severity (as measured by frequency and duration) and
perceived impact of the stutter by the individual concerned (Douglass, Constantino, Alvarado, Verrastro, & Smith, 2019). We use this
method to visualize relations between three different instruments routinely used in the assessment of stuttering and to evaluate the role
of each variable (node) in the network. In addition to drawing nodes and edges, we also report the so-called centrality indices, which
typically quantify the relative importance of the node in relation to other nodes in the network (Costantini & Perugini, 2020). We
report strength (the strength of the interactions that a node has with its neighbours), expected influence on the network, betweenness (the
number of times a node lies on the shortest path between two other nodes), and closeness (how well a node is directly or indirectly
connected to other nodes). In each of these definitions of centrality, a node can be somewhere on the continuum from central to pe­
ripheral (Costantini & Perugini, 2020).
In order to evaluate the importance of each question in OASES and UTBAS assessments, we use Random Forest, a popular machine

1
OASES, UTBAS and SSI-1V each measure different parameters of stuttering. However, the condition is perhaps unique in speech and language
pathology with regard to the extent that the core behaviors might not be the greatest issues themselves, but rather their cognitive and affective
ramifications; hence the concept of the tests sharing construct validity at a higher level.

2
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

learning method. RF can handle nonlinear relationships in the data by aggregating the predictions made by many regression trees
trained on a subset of the data called the bootstrapped dataset (Tagliamonte & Baayen, 2012). RF works through the data to establish
by trial and error whether each explanatory variable (e.g., answers to the three sets of 66 questions of UTBAS, which measure fre­
quency of negative cognitions associated with stuttering, strength of conviction in these beliefs, and anxiety provoked by them) is a
useful predictor of a dependent (response) variable (e.g. the total values of OASES). Finding questions in one assessment that have the
strongest predictive power of the total values of other assessments might shed more light on the nature of qualitative similarities
between these two tests. UTBAS was developed after the OASES, which implies that UTBAS would tackle some questions that OASES
left unanswered. The aim of our RF analysis is to test whether these two measures overlap, and – if they do – to reveal the area of
questions in which these two assessments overlap the most. Alternatively, RF technique could be also used to reduce the number of
items in the test which would lead to faster and easier assessment (cf. to Iverach et al., 2016, in which the authors reduced 66 UTBAS
items to only 6 calculating internal consistency reliability, inter-item correlations, item-total correlations, item-rest correlations, and
item-set multicollinearity).
As mentioned earlier, the nature of the relationship between key variables is important in stuttering, and in this context, the
concept of linearity. A linear association describes a straight-line homogenous relationship between two variables. Nonlinear asso­
ciations on the other hand do not follow this principle of homogeneity, and changes in one variable may disproportionally affect
changes in another – a phenomenon that is sometimes referred to as chaos theory, and the idea that apparently random events in
complex systems are, in fact, rule governed. Both linear and nonlinear systems have been used to describe dynamic relationships
between variables in stuttering at a meta-level, such as the behavioural symptoms and their cognitive consequences (Douglass et al.,
2019) as well as at the more fundamental level of articulatory control. For example, Ward (1997) examined articulatory timing re­
lationships between upper lip, lower lip and jaw in a group of PWS and control speakers using both linear and nonlinear perspectives.
Results showed that coordination between the three articulators was better explained as a nonlinear phenomenon; a finding that has
been replicated by other researchers (Van Lieshout, 1997) and taken forward to drive theories of stuttering (Smith & Weber, 2017; Van
Lieshout, Ben-David, Lipski, & Namasivayam, 2014).
Hierarchical cluster analysis traces groups in numerical tables, and it is the name for a family of techniques for clustering data and
displaying them in a tree-like format (Baayen, 2008). CA is typically exploratory, meaning that it is performed without a precisely
formulated ultimate objective. We employ cluster analysis to test whether row scores (i.e., answers on a 5-point Likert scale) of our
participants tend to cluster accordingly to four different sections of OASES: General information, Reactions to stuttering, Communi­
cation in daily situations, and Quality of life. The authors of OASES developed 100 items divided into four sections based on the World
Health Organization’s International Classification of Functioning, Disability, and Health (ICF, World Health Organization, 2001). Our
only assumption for CA is that the data may contain distinct groups, but we do not postulate how many groups there are or what their
meaning is. If clustering of row scores of our participants will not reflect that of OASES sections, this would not suggest a further
revision of the four OASES sections that reflect constructs developed within the ICF framework, because the goal of CA is not to
challenge the ICF framework. However, this could suggest a possible regrouping of 100 OASES items between the four sections, which
could be done by using, e.g., Factor analysis (FA) based on a large number of participants. Using CA, we likewise aim to test the division
of 198 UTBAS questions into three different sections. The authors suggest that 198 UTBAS questions/thoughts should cluster in three
major sections based on their 1) frequency, 2) accuracy, or 3) anxious feelings that co-occur with these questions/thoughts. However,
alternatively, the questions could cluster in 66 minor clusters, which is more plausible based on the fact that in UTBAS, a participant is
asked to answer 66 questions, each from the three different perspectives. Should this be the case, it would warrant a more careful
consideration of a relation between the three separate scales of UTBAS: how informative/useful this separation is in the first place.
Network analysis has been used previously to detect relationships between different aspects of the stuttering experience as assessed
by the OASES (Siew, Pelczarski, Yaruss, & Vitevitch, 2017). However, to the best of our knowledge, NA, CA and RF have not yet been
utilised to assess relationships between different variables present in the SSI-IV, OASES, and UTBAS, nor to identify potential overlaps
between the three instruments, both internally, or when used in conjunction with each other.

2. Method

2.1. Participants

We recruited 26 PWS (aged 20–64, mean 33.5, SD 10.9, 7 females). 13 participants were native English speakers, the remaining 13
were non-native multilingual speakers who used English daily in professional or educational settings. Non-native speakers were
speakers of several different languages (Arabic, Spanish, Vietnamese, Dutch, Portuguese, Hindi, Yoruba, and Polish). Individuals were
recruited via posts to stuttering community groups online. Prior to assessment, participants were interviewed by a fluency specialist
who judged all to present stuttering and stuttering behaviours. Based on extended conversations in these interviews, as well as written
correspondence via email, participants were judged to have at least “functional English language skills” (Blumgart, Tran, Scott Yaruss,
& Craig, 2012, p. 85), that is to say, there were no concerns regarding their ability to understand and respond to the outcome measures
used in this study, nor participate in a wider study assessing stuttering intervention conducted in English. This judgment was supported
by an additional experienced clinician who provided a second opinion on language levels. No formal assessment of participants’
English language proficiency was carried out. The research was reviewed by the University of Reading’s ethical research committee
and given a favourable opinion for ethical conduct.

3
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

2.2. Assessment (SSI-IV, UTBAS, OASES)

We analysed SSI-IV (Riley, 2009), OASES (Yaruss & Quesal, 2010), and UTBAS (Iverach et al., 2011; St Clare et al., 2009). SSI-IV is a
clinician administered measure that assesses the severity of overt stuttering behaviours, divided into subsections of frequency,
duration, and physical concomitants of stuttering displayed by the speaker. For assessment, two speech samples are collected during a
short reading and a spontaneous passage of speaking. For accurate severity scores, each sample must contain at least 200 syllables
(Todd et al., 2014). The SSI-IV (and its preceding versions) have been used extensively with children and adults in clinical assessment
and research contexts to assess overt symptoms of stuttering (Riley, 2009). However, concerns regarding the inter and intra judge
reliability of the instrument have been raised (Davidow & Scott, 2017). We collected samples containing at least 300 syllables during
videocalls between participants and a specialist fluency clinician who recorded all calls and performed an initial evaluation according
to SSI-IV conventions. There is recent evidence to show that videolink assessment by SSI-IV produces similar outcomes to those
completed face-to-face (Aldukair, 2018). To ensure reliability, 50 % of recordings were evaluated by a second specialist clinician and
an agreement was reached between both professionals on evaluation. The other 50 % of recordings were then re-evaluated by the first
clinician to ensure accurate evaluation. Subsequently, a final SSI-IV score was established for each participant. Normative data for the
SSI-IV is based on native speakers of English. Half of our participant sample were multilingual and spoke English in addition to native
languages. Normative data for the SSI-IV when used with non-native English populations is scarce. However, the instrument has been
used to assess dysfluency in other languages (e.g., Boey, Wuyts, van de Heyning, Heylen, & de Bodt, 2009) and has recently been
translated into Persian and found to be reliable (Tahmasebi, Shafie, Karimi, & Mazaheri, 2018). While all participants in the current
study were judged to have at least functional English language skills, it is possible that the variety of language backgrounds in our
participant sample represents an additional variable that could influence our findings.
The OASES is a 100-item self-report measure designed to be used alongside clinician administered assessments to evaluate the
impact of stuttering across various aspects of an individual’s life. The instrument contains 4 sections (I General information, II Re­
actions to stuttering, III Communication in daily situations, and IV Quality of life), scored against a 5-point Likert scale. Responses
provide an overall score and a score for each section. The instrument was developed and validated by Yaruss and Quesal (2006), who
collected normative data from 173 adults (ages 18–70) in North America. Normative data has subsequently been collected from adults
who stutter using the English version of the OASES in Australia, and translated versions in Japan, Sweden, and Iran (Blumgart et al.,
2012; Lindström et al., 2019; Sakai, Chu, Mori, & Yaruss, 2017; Yadegari et al., 2018; Yaruss & Quesal, 2006). 26 % of the participant
sample from which the Australian normative data was drawn were from non-native populations (Blumgart et al., 2012). Normative
data from these contexts were collected from adults aged between 18− 85. Comparison of data collected from different countries
suggests similarities, but that societal attitudes toward stuttering may influence the personal experience of PWS (Lindström et al.,
2019; Yadegari et al., 2018). The OASES is now widely used and has been celebrated as an instrument that examines numerous aspects
of the stuttering experience that can affect PWS (Briley, Wright, O’Brien, & Ellis, 2020).
UTBAS was developed to measure speech-related anxiety related to stuttering (St Clare et al., 2009). The scale allows PWS to
self-report the frequency of unhealthy cognitions, the veracity of these thoughts, and related sensations of anxiety (UTBAS I, II, and III).
Each subscale contains 66 items scored against a 5-point Likert scale (example item: “I won’t be able to keep a job if I stutter”). The
total sum of the three sections provides an overall UTBAS score. Normative data was collected from adults who stutter (ages 18–73) in
Australia and New Zealand and the UTBAS was reported to be valid when used to assess negative cognitions in this population (Iverach
et al., 2011). Translated versions of the UTBAS have been used in Japan and Turkey to collect normative data from different contexts.
Aydın Uysal and Ege (2020) administered the UTBAS assessment with 100 Turkish speaking adults (ages 17–41) and found that UTBAS
overall scores are significantly lower than those from adults in Australia and New Zealand or Japan (ages 18–74) (Chu, Sakai, Mori, &
Iverach, 2017). In line with the explanation proposed by Lindström et al. (2019) regarding cross-cultural differences, Aydın Uysal and
Ege (2020) explain their finding in terms of social anxiety level, which is lower in general in developing countries compared to
developed ones (see also Stein et al., 2010).

2.3. Data analysis

All the analyses were produced in the statistical software R (R Core Team, 2019). The data and analysis code that we used in this
study are available on the Open Science Framework platform at the following link: https://osf.io/bynfp/?view_
only=d57a1aa478834754b0b996bd1e0e9c97.
Each statistical tool we use here has its own aim. Network analysis has been developed and then adopted by psychometricians as a
different approach to disorders. Unlike classical (categorical and dimensional) approaches, which strive to find a common dependence
of symptoms on a latent entity that causes emergence of symptoms, NA holds that, e.g., a speech disorder such as stuttering, can be
viewed as rather a network of interacting, possibly self-reinforcing symptoms than an underlying entity producing those symptoms (cf.,
NA of posttraumatic stress disorder, McNally et al., 2015).
Another analytic tool we use, Random Forest, is a recent, widely used nonlinear machine learning algorithm (see, e.g., Fernán­
dez-Delgado, Cernadas, Barro, & Amorim, 2014, for a report on RF performance in comparison with numerous other techniques).
Unlike its popular alternatives, classical linear and logistic regression models, RF can deal well with the stuttering assessment data such
as OASES or UTBAS, where predictor variables are strongly correlated or involved in complex interactions. RF complements our NA
analysis. The aim of RF is to shed light on the areas between different stuttering assessments when these assessments overlap (as NA
will reveal below). Potentially it could also be used as a tool to decrease the number of items in assessments such as UTBAS, however,
this is outside the scope of the present study.

4
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

The aim of the third analytic tool we use, Cluster analysis, is also to complement NA by paying closer attention to the internal
structure of OASES and UTBAS assessments. As NA, CA does not seek for underlying latent entities. CA is used for data classification. By
using CA, following the authors of these assessments, we assume that OASES or UTBAS questions form groups/clusters. However,
unlike the authors of these assessments, we do not specify how many of those clusters should be or what they are.
Summa summarum, the aim of using these three statistical tools is to gain an enriched view of the stuttering assessment data.

2.3.1. Network analysis

Since partial correlation between any two nodes (variables) in the network is a result of extraction of correlations of all remaining
variables, these multiple correlations could lead to overfitting the data and hence to unstable estimates. To avoid this and reliably
estimate a large number of parameters in relatively small datasets (and thus achieve a low false positive outcome), we used the least
absolute shrinkage and selection operator (LASSO) (Tibshirani, 1996) to obtain a conservative (sparse) network model with only a
relatively small number of edges (connections between nodes) to explain the covariation structure in the data (Jankova & van de Geer,
2018). A tuning parameter was selected by minimizing the extended Bayesian information criterion (EBIC) (Chen & Chen, 2008). For
each participant we used his or her total scores of OASES, UTBAS (I, II, III), reading and speaking tasks, frequency, duration and
physical concomitants scores of SSI-IV. We also added participants’ age. NA was calculated in the package bootnet (Epskamp et al.,
2018) by using function estimateNetwork.

2.3.2. Random forests

RF is a relatively recent, flexible tool for a nonparametric data analysis that makes no assumption about the distribution of the
population, and hence fits well with the data showing a great deal of interindividual and intraindividual variation or unbalanced
designs (Tagliamonte & Baayen, 2012). An example of an intraindividual variation could be any PWS completing, e.g., OASES, since he
or she contributes more than one observation to the data. Unlike logistic regression models that choose predictors using a mathe­
matical equation, RF does that by trial and error, and is immune to the problem of collinearity of predictors (Strobl, Malley, & Tutz,
2009; Tagliamonte & Baayen, 2012). Likewise, RF does not suffer from overfitting when one has too many predictors relative to
number of observations, which is crucial for the OASES or UTBAS -type of data when collected from a relatively small number of PWS.
In order to improve predictive ability of our models, we produced numerous regression trees and combined the results using ran­
domForest function (from the package randomForest, Liaw & Wiener, 2002; alternatively, the function cforest from the package party
could be used, Hothorn, Bühlmann, Dudoit, Molinaro, & Van Der Laan, 2006), which implements Breiman’s random forest algorithm
(Breiman, 2001) for regression. First, total scores of UTBAS I, II, and III were used as the dependent variables and explained by a set of
100 OASES variables. Then OASES total scores were used as the dependent variable which was explained by three sets of 66 UTBAS
variables. A conditional inference tree provides estimates of the likelihood of the value of the dependent variable (e.g., OASES total
score) based on the values of explanatory variables (e.g., 198 UTBAS scores), splitting them into subsets where justified, and then
recursively considering each of the subsets, until further splitting is not justified. After each step of this recursive process, a test of
independence between the explanatory (certain UTBAS question) and dependent (OASES total score) variables is carried out. Inde­
pendence would indicate that an explanatory variable is useless, otherwise the variable is useful. If there is more than one useful
explanatory variable, the one with the strongest association would be selected and the p-value would be recorded (Tagliamonte &
Baayen, 2012). To ensure that every input row was predicted at least a few times we set the number of trees to grow to 1000. In
bootstrap aggregation an individual tree was built on a random sample of the data, roughly corresponding to two-thirds of the total
observations; the remaining one-third is typically called out-of-bag (OOB). For each tree a training set was then paired with these OOB
observations to compare accuracy of predictions. The variable importance measure (%IncMSE, see Fig. 2) was computed from
permuting data: for each tree, the prediction error (mean squared error, MSE) on the OOB portion of the data was recorded in order to
evaluate how useful each explanatory variable is as a predictor. In other words, the variable importance measure is the difference in
prediction accuracy before and after permuting the explanatory variable (averaged over all trees) (Breiman, 2001; Tagliamonte &
Baayen, 2012).

2.3.3. Cluster analysis

The goal of CA is to find groups in data. Clustering assumes that objects inside a cluster, e.g. students that passed a class, are in some
sense similar to each other, and differ to some extent from objects in other clusters, e.g. students that failed the class (even though
students from both clusters may represent a continuum of skill levels and some passing students may be more similar to the failing
students than to the top students in the class) (Hennig et al., 2015). Therefore using CA we can locate two clusters that represent, e.g.
students from the class, but we cannot claim based on CA that the class contains two different populations. Dissimilarities between
objects can be obtained in numerous ways, however, the most widely used way is the Euclidean distance (Hennig et al., 2015). For each
of the two assessments (OASES and UTBAS) we calculated distance matrices by using Euclidean distance between the two vectors to
compute the distances between the variables in the data. We then applied the hierarchical cluster analyses (when each cluster is further
partitioned into subclusters adding new levels to the hierarchy) using Ward’s minimum variance method (or error sum of squares,
Ward, 1963) in function hclust on sets of dissimilarity structures produced by the distance matrices by using function dist.
Dissimilarity-based techniques do not have specific computational issues with many variables because the size of the dissimilarity
matrix does not depend on the dimension of the data space from which it is computed (Hennig et al., 2015), which is crucial for the
present study (given a large number of analysed variables in OASES and UTBAS datasets).

5
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

3. Results

3.1. Network analysis

Fig. 1a shows a network and Fig. 1b shows centrality indices for all participants.
The network included 10 nodes and 11 edges (Fig. 1a). The shape of the network suggests three clusters: 1) assessments measuring
participants’ opinions (OASES; UTBAS I, II, and III), 2) variables measuring behaviour (SSI-IV Reading and Speaking task scores, and
their total value, Frequency), and 3) variables measuring behaviour (SSI-IV Duration and Physical concomitants score). UTBAS I and
SSI-IV Frequency scores are positively related (although weakly), thus connecting two clusters (opinion and behaviour). Age is
negatively related to Frequency score and positively to Speaking task scores in the SSI-IV cluster. Reading and Speaking task scores in
SSI-IV do not have edges to Duration and Physical concomitants scores in SSI-IV. We did not include overall SSI-IV score in our analysis,
since it would be expected that duration, frequency and physical concomitants (as composites of that overall score) would be strongly
related to that score.
The nodes for SSI-IV Frequency scores and UTBAS I have high betweenness indices showing their higher importance in the average
path between other nodes. The node UTBAS II also has a somewhat higher betweenness index.

Fig. 1. A (upper plot): Estimated network in all participants. Blue edges (lines) indicate positive while red edges – negative relations. The size and
the colour intensity of edges show the intensity of the relationship. B: Centrality indices: Strength (how well a node is directly connected to other
nodes); Expected Influence (nodes are ordered according to their expected influence on the network, e.g., the z-score of the node Age is less than -2 on
the x-axis, indicating that this node has the least expected influence on the network); Betweenness (shows how important a node is in the average
path between other nodes); Closeness (how well a node is indirectly connected to other nodes). (For interpretation of the references to colour in this
figure legend, the reader is referred to the web version of this article).

6
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

3.2. Random forests

In order to see relations in the OASES/UTBAS cluster depicted in the network analysis (see previous section) in more detail, we ran
random forest models using the UTBAS I, II, and III total values as the dependent variables and participants’ answers to the 100
questions of OASES questionnaire as explanatory variables. Fig. 2a, b, and c identify a list of the most important predictors of UTBAS I,
II, and III total values respectively from all of these 100 explanatory variables. Figures show that the same three questions from the
OASES assessment (#78 “How much is your overall quality of life negatively affected by other people’s reactions to your stuttering?”,
#97 “Overall, how much does stuttering interfere with your overall health and physical well-being?”, and #56 “How difficult is it for
you to communicate in the following general situations? Talking with people you do not know well (e.g., strangers)”) are the most
important in predicting total values of UTBAS I, II, and III assessments. In the same vein, answers to some of the UTBAS questions, such
as U.I.37 (“People won’t like me because I won’t be able to talk [How FREQUENTLY I have this thought]”), U.II.2 (“It’s impossible to
be really successful in life if you stutter [How much I BELIEVE this thought]”), or U.I.35 (“I don’t want to go - people won’t like me
[How FREQUENTLY I have this thought]”) would predict the total value of OASES assessment (see Fig. 2D).

3.3. Cluster analysis

We applied cluster analyses to both questionnaire assessments (UTBAS and OASES). Answers to the 100 questions on OASES tend to
form many clusters. However, after dividing all of these smaller clusters into the major four clusters, we notice their distribution does
not fit with OASES four major sections (I: General information, II: Reactions to stuttering, III: Communication in daily situations, and
IV: Quality of life). In other words, most of the questions in the four clusters do not overlap with this division. The only exception is one
cluster that combines the following questions: #90, How much does stuttering interfere with your ability to advance in your career?;
#92, How much does stuttering interfere with your ability to earn as much as you feel you should?; #93, How much does stuttering
interfere with your sense of self-worth or self-esteem?; #94, How much does stuttering interfere with your overall outlook on life?;
#95, How much does stuttering interfere with your confidence in yourself?; #96, How much does stuttering interfere with your
enthusiasm for life?; #99, How much does stuttering interfere with your sense of control over your life?, All of which belong to the
fourth section of OASES (Quality of life). However, our analyses indicate that other questions in this section cluster together with
questions from sections I, II, and III (see Fig. 3a).
Similarly to OASES, cluster analysis applied to UTBAS questionnaire seems to not represent the division of each claim/thought into
three subgroups: UTBAS I, II, and III. Rather, answers to UTBAS questionnaire tend to form multiple small three-member-clusters that

Fig. 2. Variable importance for the random forest with all predictors. A: “UTBAS I ~ OASES” means that the total value of UTBAS I is used as the
dependent variable explained by the raw data (all variables) of OASES. X-axes indicate the variable importance measure, %IncMSE (Increase in
Mean Squared Error of predictions); predictors with the rightmost points on the horizontal dashed lines are the most important. B: Dependent
variable is the total value of UTBAS II, explanatory variables are raw data of OASES. C: Dependent variable is the total value of UTBAS III,
explanatory variables are raw data of OASES. D: Dependent variable is the total value of OASES, explanatory variables are raw data of UTBAS I, II,
and III.

7
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

Fig. 3. A: Results of the hierarchical cluster analysis for OASES. Due to a large number of nodes, to better visualise the four sections of OASES we
marked each node with a line of the following symbols: +++++++ 1. General information, >>>>>> 2. Reactions to stuttering, ————— 3.
Communication in daily situations, and ======= 4. Quality of life). Gray boxes indicate division into four sections according to the cluster
analysis results. B: Results of the hierarchical cluster analysis for UTBAS. Each node indicates to which of the three sections of UTBAS it belongs
(+++++++ UTBAS I. how frequently I have these thoughts, >>>>>> UTBAS II. how much I believe these thoughts, and ————— III. how
anxious these thoughts make me feel). Online readers can zoom in on the figures in order to see nodes more clearly.

usually include the same claim/thought split into the tree criteria (I, II, and III). This result is also supported by the random forest
models (see Section 3.2) showing that the three different sets of 66 UTBAS questions are not that different, since the total values of each
set could be predicted by the same set of OASES questions. Thus, the difference within each set of 66 UTBAS questions seems to be more
diverse than the difference between these three sets.

3.4. Discussion

Using two recent (NA, RF) and one classic (CA) statistical tools, we have gained an enriched view of the stuttering assessment data.
New statistical tools allowed us to investigate different assessments of stuttering as a network that does not assume underlying (latent)
entities. The present study was exploratory. However, we speculated that a speech disorder such as stuttering could be viewed as a
network of interacting, self-reinforcing symptoms, and NA is a suitable tool to study phenomena like this.

3.4.1. Behavioural vs social concepts of stuttering

We have identified three distinct clusters in NA that the outcomes from the SSI-IV, OASES, and UTBAS assessments tend to form.
However, these clusters do not confirm to three concepts to which we could discretely assign the labels SSI-IV, OASES, and UTBAS. The
SSI-IV assessment seems to measure two somewhat different behavioural concepts of stuttering presented here by the variables
Reading/Speaking task scores vs. Physical concomitants/Duration, which is unsurprising given that physical concomitants usually
reflect strategies to avoid or escape from struggle which is more strongly related to duration of stuttering rather than frequency. On the

8
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

other hand, both OASES and UTBAS seem to measure the same social concept of stuttering. The fact that these two concepts of
stuttering, behavioural (SSI-IV findings) and social, (OASES, UTBAS), have only a weak relation to each other may indicate that some
of these three assessments that we used in the current study do not fulfil their intended purpose properly (to assess stuttering). An
alternative interpretation, and perhaps a more viable one given existing evidence on the social consequences of even very mild
stuttering (Blomgren, 2013), is that our findings reflect the likelihood that behavioural and social aspects of stuttering indeed have no
relation or only a weak relation to each other in PWS.

3.4.2. Do OASES and UTBAS assess the same social concept of stuttering?
While NA suggests a strong relation between OASES and UTBAS, RF identifies sets of questions in which the two assessments
overlap and answers to which may serve as predictors of the impact scores obtained via these evaluative measures. Thus, inspection of
the importance of the OASES or UTBAS questions as predictors of the total values of UTBAS or OASES respectively suggests small sets of
key questions for the assessments’ outcomes. These sets of questions that have higher predictive power require further (qualitative)
analysis which is beyond the scope of the current study. However, the results of RF emphasize similarities between these two as­
sessments suggesting that, to some extent, they share their construct validity.

3.4.3. Internal structure of the OASES and UTBAS assessments

In the last step of the analysis, we investigated whether the OASES or UTBAS data could be sorted in different groups and how well
these groups align with those suggested by the authors of assessments. Directions for the application of the OASES indicate that scores
for each of the four subsections are intended to provide clarity to the speaker’s experience of stuttering and the individual’s overall
impact rating (Yaruss & Quesal, 2006). CA of OASES showed that its four sections (I General information, II Reactions to stuttering, III
Communication in daily situations, and IV Quality of life) do not align with the responses of our participants, when divided into four
distinct clusters (Fig. 3a). Our findings are somewhat in line with those from Siew et al. (2017: 7, see their Table 1) who used a
community detection analysis in NA, and found that the division of all OASES items into four clusters (communities) does not align
with four sections of OASES. The authors used a large sample size (183 participants), however, Siew et al. only used one assessment
(OASES). The present study and Siew et al. (2017) do not necessarily undermine the conceptual organization of OASES: the observed
clusters in our study and in that of Siew et al. (2017) could alternatively indicate that questions from different subsections of OASES are
related in meaningful ways. Findings from large confirmatory studies would be needed to explore this possibility.
The three different sections of UTBAS questionnaire are also intended to be used separately, as total scores of each of these three
sections (I: how frequently I have these thoughts; II: how much I believe these thoughts, and III: how anxious these thoughts make
me feel). However, CA of UTBAS reveals that most of the UTBAS questions/thoughts tend to split into numerous three-member clusters
rather than into three large clusters that would correspond to sections I, II, or III. These small triplets typically consist of one statement/
thought in its three reincarnations (frequency, believe, and anxiousness) showing that these three scales are not necessarily that
different. There are though some inconsistencies in this pattern that we consider as exceptions, rather than a rule. For instance, the
UTBAS question #13 (represented in Fig. 3b with nodes U.I.13, U.II.13, and U.III.13) asks a participant to confirm with the following
statement: “I’m stupid”. Nodes U.I.13 and U.II.13 ask how frequently one has this thought and how much one believes this thought is
correct. Both of them cluster together, however, the third member of this triplet (U.III.13, asking how anxious this thought makes one
feel), can be found two clusters further to the right forming a cluster with the UTBAS question #64, “People who stutter are stupid”
(represented with nodes U.I.64 and U.II.64 how frequently one has this thought and how much one believes this thought is correct). In
addition, RF supports CA results that the three UTBAS scales are not that different since the same set of OASES questions would reliably
predict each scale of the UTBAS assessment (see Fig. 2).

3.4.4. Future directions

Whilst we have focused on three commonly used assessments for stuttering, our approach could equally be applied to better un­
derstand the relationship between factors within other assessments as well as the associations between other stuttering (and poten­
tially cluttering) assessments. Our approach may also have value for studies which explore relationships between cognitive/emotional
aspects of stuttering and its behavioural presentation. From an aetiological perspective, Van Lieshout et al.’s (2014) speech motor skill
model which factors in emotional state variables to the motor speech consequences of stuttering and the dual diathesis-stressor model
presented by Walden et al. (2012) provide two such examples. Studies which explore similar relationships from a clinical/therapeutic
perspective (for example, the study by Menzies et al., 2008, comparing outcomes from a fluency shaping program to those from a
cognitive-behavioural therapy approach) could also benefit.

3.4.5. Limitations
We report on a relatively small sample size. Subsequently, participants with low SSI-IV scores (that may be considered very mild),
women, and elderly people are underrepresented in our study. While normative data for UTBAS and OASES has been collected in
different contexts (Blumgart et al., 2012; Chu et al., 2017; Sakai et al., 2017), the instruments used in this study have not been
standardized or validated with all the participant groups represented in our sample. The diverse language backgrounds of our
participant sample may represent an additional variable that could influence our findings. Further inquiry should seek to attend to
these limitations in participant sampling.

9
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

4. Conclusion

Multiple measures from a small sample of participants set challenges for the reliability of statistical inferences in the fields of
research where recruiting large numbers of participants may be difficult. We used a machine learning method (random forests) and the
least absolute shrinkage and selection operator (LASSO) in the network analysis in order to overcome these problems while analysing
results collected from a small sample of PWS using three different stuttering measurements: SSI-IV, OASES, and UTBAS. Our results
demonstrate that the stuttering severity index (SSI-IV) only weakly interacts with a quality of life index (OASES) and a self-perception
and belief systems index (UTBAS). The lack of interaction and many similarities (overlaps) between the last two indices suggest that
the network of the three commonly used stuttering assessments may be fractured in a non-productive way. In other words, there might
be a missing key player: an assessment that would strongly link behavioural and social aspects of stuttering. Confirmation of key
factors of this kind of assessment awaits corroboration from further (qualitative) analyses with larger participant numbers.

Financial disclosure

None.

Non-financial disclosure

None.

Acknowledgements

We thank all our participants. This study was supported by the Innovate UK Open Grant Funding Programme, project number:
26405.

References

Alameer, M., Meteyard, L., & Ward, D. (2017). Stuttering generalization self-measure: Preliminary development of a self-measuring tool. Journal of Fluency Disorders,
53, 41–51.
Aldukair, L. (2018). Telepractice application for the clinical assessment of school-age children who stutter. Doctoral thesis. UK: University of Reading.
Aydın Uysal, A., & Ege, P. (2020). Reliability and validity of the UTBAS-TR (The Unhelpful Thoughts and Beliefs Scale-the Turkish version) in the Turkish population.
International Journal of Speech-language Pathology, 22(1), 24–29.
Baayen, R. H. (2008). Analyzing linguistic data: A practical introduction to statistics using R. Cambridge: Cambridge University Press.
Blomgren, M. (2013). Behavioral treatments for children and adults who stutter: A review. Psychological Research and Behavior Management, 19, 9–19.
Blumgart, E., Tran, Y., Scott Yaruss, J. S., & Craig, A. (2012). Australian normative data for the overall assessment of the speaker’s experience of stuttering. Journal of
Fluency Disorders, 37(2), 83–90.
Boey, R. A., Wuyts, F. L., van de Heyning, P. H., Heylen, L., & de Bodt, M. S. (2009). Characteristics of stuttering in Dutch-speaking individuals. Clinical Linguistics &
Phonetics, 23(4), 241–254.
Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323(April), 892–896.
Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.
Briley, P., Wright, H., O’Brien, K., & Ellis, C. (2020). Relative contributions to overall impact of stuttering in adults using the overall assessment of the Speaker’s
Experience of Stuttering (OASES). Journal of Fluency Disorders. https://doi.org/10.1016/j.jfludis.2020.105775.
Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95(3), 759–771.
Chu, S. Y., Sakai, N., Mori, K., & Iverach, L. (2017). Japanese normative data for the Unhelpful Thoughts and Beliefs about Stuttering (UTBAS) Scales for adults who
stutter. Journal of Fluency Disorders, 51, 1–7.
Costantini, G., & Perugini, M. (2020). 17 Network analysis for psychological situations. In J. F. Rauthmann, R. Sherman, & D. C. Funder (Eds.), The Oxford handbook of
psychological situations (pp. 269–286). Oxford University Press.
Davidow, J. H., & Scott, K. A. (2017). Intrajudge and interjudge reliability of the stuttering severity instrument - fourth edition. American Journal of Speech-language
Pathology, 26(4), 1105–1119.
Douglass, J. E., Constantino, C., Alvarado, J., Verrastro, K., & Smith, K. (2019). Qualitative investigation of the speech-language therapy experiences of individuals
who covertly stutter. Journal of Fluency Disorders, 61(August), Article 105713.
Epskamp, S., Kruis, J., & Marsman, M. (2017). Estimating psychopathological networks: Be careful what you wish for. PloS one, 12(6), Article e0179891.
Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212.
Fernández-Delgado, M., Cernadas, E., Barro, S., & Amorim, D. (2014). Do we need hundreds of classifiers to solve real world classification problems? Journal of
Machine Learning Research, 15(1), 3133–3181.
Freud, D., & Amir, O. (2020). Resilience in people who stutter: Association with covert and overt characteristics of stuttering. Journal of Fluency Disorders. https://doi.
org/10.1016/j.jfludis.2020.105761.
Hennig, C., Meila, M., Murtagh, F., & Rocci, R. (Eds.). (2015). Handbook of cluster analysis. CRC Press.
Hothorn, T., Bühlmann, P., Dudoit, S., Molinaro, A., & Van Der Laan, M. J. (2006). Survival ensembles. Biostatistics, 7(3), 355–373.
Iverach, L., & Rapee, R. M. (2014). Social anxiety disorder and stuttering: Current status and future directions. Journal of Fluency Disorders, 40, 69–82.
Iverach, L., Menzies, R., Jones, M., O’Brian, S., Packman, A., & Onslow, M. (2011). Further development and validation of the Unhelpful Thoughts and Beliefs about
Stuttering (UTBAS) scales: Relationship to anxiety and social phobia among adults who stutter. International Journal of Language & Communication Disorders, 46(3),
286–299.
Iverach, L., Heard, R., Menzies, R., Lowe, R., O’Brian, S., Packman, A., et al. (2016). A brief version of the unhelpful thoughts and beliefs about stuttering scales: The
UTBAS-6. Journal of Speech Language and Hearing Research, 59(5), 964–972.
Jankova, J., & van de Geer, S. (2018). Inference for high-dimensional graphical models. In M. Drton, M. Maathuis, S. Lauritzen, & M. Wainwright (Eds.), In: Handbook
of graphical models. Boca Raton, Florida, USA: CRC Press.
Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R News, 2(3), 18–22.
Lindström, E., Nilsson, E., Nilsson, J., Schödin, I., Strömberg, N., Österberg, S., et al. (2019). Swedish outcomes of the overall assessment of the speaker’s experience of
stuttering in an international perspective. Logopedics Phoniatrics Vocology, 0(0), 1–9.

10
D. Ward et al. Journal of Fluency Disorders 67 (2021) 105823

McNally, R. J., Robinaugh, D. J., Wu, G. W., Wang, L., Deserno, M. K., & Borsboom, D. (2015). Mental disorders as causal systems: A network approach to
posttraumatic stress disorder. Clinical Psychological Science, 3(6), 836–849.
Menzies, R., O’Brian, S., Onslow, M., Packman, A., St. Clare, T., & Block, S. (2008). An experimental clinical trial of a cognitive-behavior therapy package for chronic
stuttering. Journal of Speech Language and Hearing Research, 51(6), 1451–1464.
Neal, Z. P., & Neal, J. W. (2017). Network analysis in community psychology: Looking back, looking forward. American Journal of Community Psychology, 60(1–2),
279–295.
Newman, M. E. J. (2010). Networks: An introduction. Oxford, UK: Oxford University Press.
R Core Team. (2019). R: A language and environment for statistical computing. URL. Vienna, Austria: R Foundation for Statistical Computing https://www.R-project.org/.
Riley, G. D. (2009). SSI-4 stuttering severity instrument. Austin, TX: Pro-Ed.
Sakai, N., Chu, S. Y., Mori, K., & Yaruss, J. S. (2017). The Japanese version of the overall assessment of the speaker’s experience of stuttering for adults (OASES-A-J):
Translation and psychometric evaluation. Journal of Fluency Disorders, 51, 50–59.
Siew, C. S., Pelczarski, K., Yaruss, J. S., & Vitevitch, M. (2017). Using the OASES-A to illustrate how network analysis can be applied to understand the experience of
stuttering. Journal of Communication Disorders, 65, 1–9.
Smith, A., & Weber, C. (2017). How stuttering develops: The multifactorial dynamic pathways theory. Journal of Speech Language and Hearing Research, 60(9),
2483–2505.
Sønsterud, H., Kirmess, M., Howells, K., Ward, D., Feragen, K., & Halvorsen, M. (2019). The working alliance in stuttering treatment: a neglected variable?
International Journal of Language & Communication Disorders, 54(4), 606–619.
Sønsterud, H., Halvorsen, M., Feregen, K., Kirmess, M., & Ward, D. (2020). What works for whom? Multidimensional individualized stuttering therapy (MIST). Journal
of Communication Disorders. https://doi.org/10.1016/j.jcomdis.2020.106052.
Stein, D. J., Ruscio, A. M., Lee, S., Petukhova, M., Alonso, J., Andrade, L. H. S. G., et al. (2010). Subtyping social anxiety disorder in developed and developing
countries. Depression and Anxiety, 27(4), 390–403.
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees,
bagging, and random forests. Psychological Methods, 14(4), 323–348.
St Clare, T., Menzies, R., Onslow, M., Packman, A., Thompson, R., & Block, S. (2009). International Journal of Language & Communication Disorders, 44(3), 338–351.
Tagliamonte, S. A., & Baayen, R. H. (2012). Models, forests, and trees of York English: Was/were variation as a case study for statistical practice. Language Variation
and Change, 24(2), 135–178.
Tahmasebi, N., Shafie, B., Karimi, H., & Mazaheri, M. (2018). A Persian-version of the stuttering severity instrument-version four (SSI-4): How the new additions to
SSI-4 complement its stuttering severity score? Journal of Communication Disorders, 74(September 2017), 1–9.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288.
Todd, H., Mirawdeli, A., Costelloe, S., Cavenagh, P., Davis, S., & Howell, P. (2014). Scores on Riley’s stuttering severity instrument versions three and four for samples
of different length and for different types of speech material. Clinical Linguistics & Phonetics, 28(12), 912–926.
Van Lieshout, P. H. H. M. (1997). Higher and lower order influences on the stability of the dynamic coupling between articulators. In H. F. M. Peters, W. Hulstijn, &
P. H. H. M. van Lieshout (Eds.), Speech production: motor control and fluency disorders (pp. 161–170). Amsterdam, The Netherlands: Elsevier Scientific.
Van Lieshout, P. H. H. M., Ben-David, B., Lipski, M., & Namasivayam, A. (2014). The impact of threat and cognitive stress on motor speech control in people who
stutter. Journal of Fluency Disorders, 40, 93–109.
Walden, T. A., Frankel, C. B., Buhr, A. P., Johnson, K. N., Conture, E. G., & Karrass, J. M. (2012). Dual diathesis-stressor model of emotional and linguistic
contributions to developmental stuttering. Journal of Abnormal Child Psychology, 40(4), 633–644.
Ward, D. (1997). Intrinsic and extrinsic timing in stutterers’ speech: Data and implications. Language and Speech, 40, 289–310.
Ward, J. H., Jr. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58, 236–244.
World Health Organization. (2001). The international classification of functioning, disability & health. Geneva: World Health Organization.
Yadegari, F., Shirazi, T. S., Howell, P., Nilipour, R., Shafiei, M., Shafiei, B., et al. (2018). Persian overall assessment of the Speaker’s Experience of stuttering for adults:
The impact of stuttering on the Persian-speaking adults who stutter. Iranian Rehabilitation Journal, 16(2), 131–138.
Yaruss, J. S., & Quesal, R. W. (2006). Overall Assessment of the Speaker’s Experience of Stuttering (OASES): Documenting multiple outcomes in stuttering treatment.
Journal of Fluency Disorders, 31(2), 90–115.
Yaruss, J. S., & Quesal, R. W. (2010). OASES: Overall assessment of the speaker’s experience of stuttering. Bloomington, MN: Pearson.

David Ward, PhD, is Director of the Speech Research Laboratory at the University of Reading, UK, and a specialist fluency clinician for Oxford Health within the NHS.
He has published extensively on both aetiological and clinical aspects of disorders of fluency. The second edition of his textbook Stuttering and Cluttering was published in
2017. He is also co-editor of Cluttering: A handbook of Research, Intervention and Education (2011) and co-author of Managing Cluttering (2013).

Ronan Miller, PhD, is a research assistant in the School of Psychology and Clinical Language Sciences at the University of Reading, his research interests include anxiety
and self-related constructs in people who stutter in foreign language learning.

Alexandre Nikolaev, PhD, is a research associate in the School of Language and Cultures at the University of Sheffield. He has experience conducting research in the
area of language decline in people with dementia. In his doctoral thesis (2011), Nikolaev studied the morphology of the Finnish language and then morphological
decline in people with Alzheimer’s disease or mild cognitive impairment.

11