ARTICLE IN PRESS

Classifying and Identifying Motor Learning Behaviors in

Voice-Therapy Clinician-Client Interactions: A Proposed Motor
Learning Classification Framework
*Catherine Madill, *Anna McIlwaine, *Rosanne Russell, †Nicola J. Hodges, and *Patricia McCabe, *Australia, and
yCanada

Summary: Purpose. We studied whether concepts in motor skill learning could be operationalized to identify
clinical interactions and behaviors in a voice therapy setting. Our aim was to test the feasibility of measuring
these behaviors in the prepractice phase so that we could eventually evaluate and apply principles of motor learn-
ing and skill acquisition to Speech-Language Pathology. Four general categories of behaviors that have been
identified in the client-clinician prepractice phase were identified: motivation, modeling, verbal information, and
feedback. All variables were extracted from a proposed Motor Learning Classification Framework.
Method. Nine participants categorized clinician behaviors in three voice therapy training videos into specific,
described, prepractice variables.
Results. Good intrarater reliability was shown across viewings. Inter-rater reliability was high for modeling and
verbal information, but raters were not consistent when identifying behaviors classified as motivation and feed-
back. Raters responded positively to the classification exercise and the categories encompassed nearly all noted
behaviors.
Conclusion. Behaviors described within the motor learning literature can be identified in the initial stages of
voice therapy, providing evidence that motor learning concepts can be used to identify interactions and behaviors
in clinical settings. Disagreement in classification among raters was influenced by differences in implicit and
explicit interpretations of verbal information. This suggests that greater clarity in specific concepts is needed to
support teaching of motor learning principles and implementation of these principles in clinical practice for the
treatment of speech-language pathology.
Key Words: Motor Learning−Voice−Therapy−Practice.

INTRODUCTION been operationalized, identified, and used in clinical settings

Motor learning theory in speech-language pathology
Motor learning theory is an area of expanding interest in
Speech-Language Pathology (SLP) as researchers seek to
identify more effective treatments and processes for learning
the motor skills required for verbal communication. Investi-
gations into the understanding and application of motor
learning theory to voice, motor speech disorders, stuttering,
and swallowing have resulted in recommendations for goal
setting (eg1), use of instructions and feedback (eg,2), and
novel treatments (eg, ReST3). The majority of these studies
have focused on specific motor learning variables, such as
motivation, verbal information, modeling, and feedback.
To understand how the findings from this body of research
might apply to SLP, a broader, more comprehensive and
contextualized study of motor learning is needed to help
train therapists and improve treatment and training out-
comes. One step in this direction is to determine how con-
cepts described in motor learning research and theory have
Accepted for publication March 22, 2019.
From the *Voice Research Laboratory, Discipline of Speech Pathology, Faculty of
Health Sciences, The University of Sydney, Australia; and the ySchool of Kinesiol-
ogy, Faculty of Education, University of British Columbia, Canada.
Address correspondence and reprint requests to Catherine Madill, PO Box 170,
Lidcombe, NSW 1825 Australia. E-mail: cate.madill@sydney.usyd.edu.au
Journal of Voice, Vol. &&, No. &&, pp. &&−&&
0892-1997
© 2019 The Authors. Published by Elsevier Inc. on behalf of The Voice Founda-
tion. This is an open access article under the CC BY-NC-ND license.
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
https://doi.org/10.1016/j.jvoice.2019.03.014
and teaching of SLP students.
There has been a specific interest in motor learning the-
ory as it applies to voice training.4−9 A number of research-
ers have investigated the effects of feedback on acquisition
and learning in experimental studies with an SLP focus,
including focus of attention in feedback,4 modeling and
instruction without feedback,5 self-controlled feedback,7
knowledge of results feedback8 and biophysical feedback.9
Practice organization has also been studied, but to a lesser
degree (eg, massed vs spaced practice in vocology train-
ing6). Voice therapy (the remediation of the disordered
voice) provides a well-defined context to investigate
whether interactions between, and behaviors of, clinicians
and clients can be identified and analyzed against the back-
drop of motor learning theory and pertinent concepts.
Such an analysis would contribute to the understanding of
what components of voice therapy involve motor skill
learning principles more generally, such as use of feedback
and demonstrations and whether these are provided in
ways that are thought to be most optimal. This will also
help identify what aspects are not applied, not applied well
or are not associated with motor skill learning at all.
Active ingredients in direct voice therapy
In recent years, significant efforts have been made to investi-
gate the “active ingredients” of voice therapy.10 These ingre-
dients have been identified as falling into two categories:
 ARTICLE IN PRESS
2 Journal of Voice, Vol. &&, No. &&, 2019

pedagogic strategies (how we learn) and voice techniques or

exercises (what we learn).11 Identification of these categories
has been made difficult by the fact that even voice treatment
studies with the highest levels of evidence have not isolated
effects due to voice technique or pedagogic strategies.12 A
theoretical model of active ingredients in voice therapy
has been developed, where a wider range of classifications
has been proposed to explain efficacy of direct voice therapy
interventions.13 This includes differentiating so-called
extrinsic (clinician applied) versus intrinsic/patient-applied
delivery methods, counseling, and the analysis of direct
interventions into different subcategories.13 Many of these
elements include behaviors described in motor learning the-
ory, for example modeling, augmented feedback, and self-
evaluation. This is not unexpected given the need to learn,
relearn or in some other way alter the habitual movements of
the vocal system with the aim of producing an optimal voice.
A clearer overview of motor learning behaviors that are used
across different therapies and modes of delivery would facili-
tate further development of an active ingredient model of
voice therapy.

Motor learning theory

Two phases of motor learning interventions and activities
have been considered in research: prepractice and prac-
tice.14 Prepractice is the phase before the learner or client
begins to rehearse or change their behaviors. This is the cli-
nician-client period (or coach-athlete time), when there may
be time constraints and the primary aim is to convey to the
learner what needs to be practiced. The practice phase, by
contrast, may be unsupervised, at least in the rehabilitation
context, and this is the time when the learner is building rep-
etitions and putting into practice the knowledge acquired in
the prepractice phase. In the field of voice and motor
speech, there has been significant research into the later
practice phase, with a focus on improving skill retention
past the completion of intervention (see Maas et al, 2008 for
review). Significantly less research exists in SPL on the pre-
practice phase. The prepractice phase ensures that the client
understands the target movement (ie, builds a reference-of-
correctness15) and can produce it under optimal condi-
tions.16 Treatment effectiveness is the focus in practice.
However, treatment efficiency, that is the speed with which
the components required for practice are understood or
acquired, is a key concern in the prepractice phase. In pre- FIGURE 1. A proposed Motor Learning Classification Frame-
practice, the focus is on factors that allow a particular skill work. The main variables appear on the left of the model and are
to be accurately performed quickly, with the lowest degree ordered somewhat in relation to how they are delivered chronolog-
of effort and cost, and with minimized risk of injury.17 This ically. To the right of these variables are classifications of how
is especially salient in voice therapy as the target movement these variables can be considered along with more specific exam-
may be difficult to acquire and detect due to its transient ples. Note KR, Knowledge of results; KP, Knowledge of perfor-
and abstract nature, and the lack of visual, auditory and mance.
somatosensory feedback.
is motivated to change behavior is essential for learning.14
Motivation can be established by conveying the importance
Prepractice variables of the task to the client16 or by setting goals with the cli-
The prepractice variables include motivation, verbal infor- ent.18 Verbal information is a broad category of prepractice
mation, and modeling (Figure 1).14 Ensuring that the client behaviors within which instructions, explanations, and
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
perceptual training can be considered subcategories.14 The
client must possess an understanding of the target move-
ment in order to accurately produce the movement pat-
tern.19 Thus, instructions and explanations about the
movement and associated sensations may be necessary in
the prepractice phase.14 For the acquisition of speech and
voice tasks, auditory-perceptual and somatosensory train-
ing, especially discrimination training, is one way of alerting
to a to-be-acquired movement.20 This prepractice percep-
tual discrimination training has been shown to activate pre-
motor cortical areas21 and lead to motor learning in the
absence of movement practice.22 Modeling involves physi-
cal demonstration of the target movement. This allows the
client to observe aspects of the skill that cannot be verbally
explained (eg,18,23,24). Modeling may be visual or auditory.
Feedback provided for initial movement attempts during
prepractice is also essential to ensure that the client is aware
of the target movement and can approximate this before
beginning practice (eg,16). Of course, some attempt to try
the target movement is needed to check if the client is able
to make the correct movement,16 even though in the pre-
practice phase, attempts are typically limited. Attempts
may be made independently or with guidance from the
clinician. Because in voice therapy, incorrect movement
attempts may cause phonotrauma,25 guidance in the initial
stages of intervention may be appropriate. Feedback on the
accuracy of initial movement attempts is essential to
encourage awareness and change.26 Clinicians provide
information about the outcome of movement (knowledge of
results) or about the nature of the movement pattern
(knowledge of performance) (eg,27). In SLP contexts, feed-
back can relate to auditory, kinesthetic or visual aspects of
the movement and may be provided during or after move-
ment attempts.28
Within these five broad categories are numerous subcate-
gories which have been studied in the motor learning litera-
ture and have shown to impact motor skill acquisition as
detailed in Figure 1. We focus primarily on the four preprac-
tice variables of motivation, verbal information, modeling,
and feedback in this study as they are most relevant in the ini-
tial stages of intervention and therefore most likely to be iden-
tifiable in the clinician-client interaction. To simplify the
identification analysis for this first investigation, we only look
at subcategories associated with the types of verbal informa-
tion (ie, instructions, explanations or perceptual training).
Why operationalize and apply motor learning
concepts to therapy?
Generally, practice conditions that increase task demands
result in improved retention and generalization of motor
skills.29,30 However, clients can benefit from reduced task
demands in the initial stages of intervention, that is, in the
prepractice phase, when learning inherently requires effort
and has high information processing demands.17 According
to the challenge point framework,29 the task difficulty and
the client's skill level interact to determine the optimal
3
amount of information, or difficulty, for each client. In pre-
practice, the client's skill level is generally low and it is there-
fore appropriate for the clinician to reduce the difficulty of
the task by providing relevant cues, prompts, and supports.
If a learner appears to be struggling to achieve the target
action (eg, clear and effortless voicing), due to too little
information or decreased motivation, then increasing the
amount of guidance or information may be warranted.
The challenge for the SLP in prepractice is to identify
suitable instructional and feedback behaviors to reduce
the task demands and assist the client to achieve the task
goal efficiently.31 While some proprietary programs such
as PROMPT32 or Voicecraft33 recommend specific instruc-
tional strategies, overall there is relatively limited guidance
in the SLP literature regarding whether these are appropri-
ate instructional behaviors to expedite completion of the
necessary components of the prepractice phase. By investi-
gating the general motor learning literature it is possible to
identify strategies which instructors can use to assist learners
to acquire any new motor task. For example, instructions
that are short and simple, help cue individuals to key infor-
mation, and/or provide an external focus on the learning
outcome (ie, the sound or the feeling during sound produc-
tion), rather than a detailed description of the technique,
have been shown to benefit learning.19,34,35
Adoption of techniques which work to bring about a
change in behavior through more implicit means have also
been shown to be useful ways for promoting learning, whilst
additionally keeping cognitive demands on the learner low
(eg,36−39). For example, use of analogies, or changes to
physical constraints (eg, posture) can bring about behaviors
as a consequence, rather than as an explicitly prescribed
aim.40,41 Asking singers to focus on letting the breath flow
easily has been shown to be more effective in raising volume
than focusing on using breath support.42 Similarly, move-
ment demonstrations rather than explicit instructions can
help to bring about motor skills more implicitly.43,44 Skill
acquisition and movement-form retention are most optimal
when learners are exposed to several episodes of modeling
(or indeed multiple models)42 rather than just a single epi-
sode (or model) (Figure 2).45−48
Identifying activities and behaviors in training and
therapy settings
Content analysis is one method for analyzing written or ver-
bal texts such as clinical interactions. It provides a research
technique which can be used for objective, systematic, and
quantitative description of communication and behavior.49
To achieve objective analysis of instructional behavior, valid,
and reliable classification systems must be developed.50,51 A
number of classification systems for sports-based instruc-
tional behaviors have been developed which meet these crite-
ria. The Motor Teaching Principles Taxonomy, for example,
has been used to reliably analyze physical therapists’ instruc-
tional behaviors across multiple therapists and therapy
sessions.52
 ARTICLE IN PRESS
4 Journal of Voice, Vol. &&, No. &&, 2019
FIGURE 2. Prepractice variables within the Motor Learning
Classification Framework.
Systematic observation of sports coaches during training
sessions has been undertaken to investigate the practice
activities and instructional behaviors used in an applied set-
ting (eg,50,53). These studies have shown that coach behavior
is often in conflict with principles of best practice based on
motor learning theory and hence promote better knowledge
dissemination of evidence-based practice among practi-
tioners (see also54). Such observational research is also
essential in SLP to determine what clinicians actually do in
clinical practice so that the match between theory and prac-
tice can be made and appropriate interventions targeted
based on this knowledge.55
For clinicians to apply motor learning principles and
thereby improve the efficacy of the prepractice phase, as
well as the overall motor learning process, including prac-
tice, there is a need for: (1) an overarching classification
framework that incorporates all of the motor learning varia-
bles demonstrated to facilitate learning and (2) a means by
which clinicians can recognize and classify these elements in
real-life therapy contexts. The overall aim would be to char-
acterize, identify, evaluate, and test these various preprac-
tice and practice variables and then use this information to
train speech-language clinicians. Such a framework would
include numerous variables referenced in the motor learning
literature. We have attempted to detail these motor learning
variables in Figure 1.
Purpose of the study
We aim to study whether motor learning concepts can be
operationalized and then reliably used to identify and
classify prepractice variables (motivation, verbal informa-
tion, modeling, and feedback) in clinical interactions in
voice therapy.
MATERIAL AND METHODS
Participants
Participants were nine female students with an age range of
21−36 years, in their final semester of a Master of Speech-
Language Pathology degree at the University of Sydney,
Australia. All were fluent speakers of English. All passed a
hearing screening of 20 dB at 0.5, 1, 2, and 4 kHz.56 Partici-
pants will be referred to as “raters” throughout this study.
Ethical approval was obtained through Human Research
Ethics Committee of the University of Sydney (08-2009/
11958). Direct consent was obtained from all raters.
Procedure
Stimuli
Test stimuli were selected samples from three commercially
available voice therapy training videos: Lessac Madsen Res-
onant Voice Therapy (LMRVT),57 Yell Well,58 and Vocal
Function Exercises (VFE).59
LMRVT57 is used for the treatment of hyper- and hypofunc-
tional voice disorders. The aim of LMRVT is for the client to
produce a vocal tone in a resonant and easy manner such that
the vocal folds are barely adducted or barely abducted result-
ing in reduced impact pressure between the vocal folds.60
Yell Well58 is a treatment for vocal nodules in children.
The program teaches children how to yell safely via a step
wise series of cues to induce release of laryngeal constric-
tion, stabilize the laryngeal framework for optimal activa-
tion of intrinsic laryngeal musculature, and shape the
resonatory tract for amplification of the vocal signal.
VFEs59 is a series of vocal exercises that are claimed to
promote physiologic balance across respiratory, phonatory,
and resonance subsystems. The goal of the exercises is to
strengthen the intrinsic laryngeal and auxiliary musculature,
and enhance the vocal tract resonance characteristics.61
A pilot study to determine the most user-friendly classifi-
cation task and checklist format was run with four indepen-
dent raters. None of the raters from the pilot study were
included in any further research. An additional prepractice
behavior category, physical attempts, was included in the
pilot and excluded from the checklist in the reported study
as it was a client behavior and reliability was very high.
Classification protocol
One therapy sample was selected from each video recording.
Each sample was selected on the basis that it represented a
prepractice or acquisition sequence, that is, the clinician
was introducing a new vocal task to the client, the sample
included one-on-one interaction between the presenter of
the video and a patient with a voice disorder, and the sam-
ple was considered by the clinician authors to be typical of a
standard therapy session. In these interactions, the client's
voice disorder was addressed using the motor training tech-
niques recommended by the specific therapy approach.
Each of the three samples was 3 minutes in duration. An
additional 2-minute sample from LMRVT57 was used as a
training stimulus. The video therapy samples were presented
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
three times each. First the entire video was shown (ie,
3 minutes), then the video was shown in segments of 10 clini-
cian behaviors interspersed with 20 second pauses between
segments, and finally the entire video was repeated. This pro-
cedure was repeated for each video to give raters time to
classify each of the 10 behaviors. Raters were provided with
a copy of the Motor Learning Classification Framework
(MLCF) (Figure 1) and written definitions of the general
and specific prepractice variables (see Appendix A). An elec-
tronic version of the observational checklist (Appendix B)
was provided to raters and included a written transcription
of the video samples, with each spoken sentence and action
in the therapy sample transcribed by the researcher (AM) on
a separate line of the spreadsheet. Raters were instructed to
type a “1” in the column(s) of the identified prepractice vari-
able(s) identified in columns beside each transcribed behav-
ior. Raters could identify a single behavior as more than one
variable (eg, “feedback” and “explanation”), however, they
were advised to differentiate between the variables as much
as possible. An additional category other was included for
raters to identify clinically relevant behaviors that could not
be classified according to the current framework.
The classifications were conducted over two classification
sessions where all raters attended, and watched and scored
the videos together. The first 2-hour classification session
included both a training component and the first classifica-
tion task. The second 1-hour session occurred 1 week after
the first session and included a reclassification and evalua-
tion task. The observational checklists and evaluations were
deidentified before data analysis.
Session 1-Training. The researcher verbally explained
the classification task and answered any questions. Raters
then jointly watched the 2-minute training sample, which
illustrated a range of prepractice behaviors. Viewing the
training video segment conformed to the protocol used later
in the classification sessions. That is, during the first view-
ing, raters watched the whole sample without classifying the
behaviors, to orient them to the whole task, the types of
behaviors and their rate of occurrence. Throughout the sec-
ond and third viewings they had the opportunity to now
identify and classify the behaviors. For the training video
only, raters compared their classifications to the researcher's
and had the opportunity to discuss and clarify any discrep-
ancies or difficulties with the framework and classification
task.
Session 1-Classification. Following training, all raters
viewed the three 3-minute therapy samples (one from each
therapy program) and classified the clinician behaviors
according to the prepractice variables. A 5-minute break
was provided between each therapy sample.
Session 2-Reclassification. A second classification ses-
sion was conducted in which raters rerated the middle
1-minute segment from each 3-minute therapy sample to
assess intrarater reliability. The middle segment of the
5
sample was chosen to avoid recency and primacy effects.62
The same procedure was used as in the first classification
session. At the end of Session 2, raters were asked to evalu-
ate the framework and classification task. Raters were
invited to give written comments on potential applications
of the framework and provide suggestions for improving the
framework or classification task.
Data analysis
The accuracy of data entry was maximized by having raters
enter classifications directly into the spreadsheets used for
data analysis. A frequency count of the general behaviors as
a function of the three different types of therapies was first
calculated. A frequency score for the general verbal informa-
tion variable was calculated for each behavior by counting
each time at least one of the specific verbal information vari-
ables was selected.
Intrarater reliability was measured by calculating percent
agreement between each participant's frequency score on
the first and second session for each behavior in each video
segment.63 Inter-rater reliability was measured by calculat-
ing percent agreement between all participants for each var-
iable and each behavior for each video segment.63 Only
eight of the nine participants were included in the intra- and
inter-rater analyses; one rater was excluded as she did not
demonstrate clear understanding of the framework or learn-
ing from the training provided. Percent agreement for pre-
practice variables was calculated by counting the number of
times one or more people classified a behavior in the same
way and dividing this number by the potential number of
agreements and then multiplying by 100. If for example,
one person classified seven behaviors as motivation, there
would be a potential total agreement of 56 (seven behaviors
classified the same by eight people). If the actual agreements
for this behavior were 17, then the % agreement would be
17/56 £ 100 = 30.4%.
As there were eight response categories and eight partici-
pants, the chance of complete agreement on a category for a
single behavior was one in 64 (1.56%), that is, agreement by
chance was low. An estimate of guessing could not be calcu-
lated as participants were able to provide more than one
classification for each behavior.
RESULTS
Frequency of prepractice variables
Table 1 shows the total number of times (#) each variable
was identified for each video segment across all eight partic-
ipants included in the inter-rater analyses. The total n values
were different across the video segments. More variables
were identified and classified in LMRVT than VFE and
Yell Well.
Intrarater reliability
High intrarater reliability was demonstrated across all three
video segments. Agreement ranged from 68.8% to 100%,
 ARTICLE IN PRESS
6 Journal of Voice, Vol. &&, No. &&, 2019

TABLE 1.
Frequency Counts for Each Variable in Each Therapy Approach
LMRVT (n = 120) Yell Well (n = 67) VFE (n = 73)
# # #
General variables
Motivation 19 18 21
Modeling 411 59 148
Verbal Information 484 333 362
Feedback 73 164 56
Other 0 0 10
Specific variables (within the category verbal information)
Instructions 291 214 110
Explanations 88 49 198
Perceptual training 118 75 50
Other verbal information 5 44 18
Note: n, number of clinician behaviors within each therapy approach identified by the researchers. Multiplying this number by 8 would give the total number
of behaviors; #, number of clinician behaviors given by the raters for each prepractice variable summed across all 8 raters.

M = 88.7% (SD = 7.18). Intrarater percent agreements for
each rater as a function of therapy session are shown in
Figure 3.
Inter-rater reliability for general variables
Participants’ percent agreement for each general prepractice
variable within each video segment is represented in
Figure 4.
The level of agreement for each therapy approach within
each prepractice variable was compared. Participants dem-
onstrated relatively high agreement across all video seg-
ments for modeling (LMRVT = 87.5%; Yell Well = 67.0%;
VFE = 87.5%) and verbal information (LMRVT = 87.0%;
Yell Well = 77.80%; VFE = 75.6%). Feedback was rated
with high agreement in Yell Well (74%) but less so in VFE
(46.4%) and LMRVT (22.4%). Participants did not show
good agreement on the motivation variable across all ther-
apy approaches (LMRVT = 35.4%; Yell Well = 10.2%;
VFE = 14.6%). The other category was not used by partici-
pants in LMRVT or Yell Well but was used on 10 occasions
when classification VFE. Three participants consistently
rated the clinician asking “Ok?” after instructions or
explanations as other. They each described this behavior
as “checking/confirming the client understands.” One rater
also identified two additional behaviors as other and
described them as “building rapport.”
Inter-rater reliability for specific prepractice variables
Participants’ percent agreements for the specific verbal infor-
mation variables within each therapy approach are repre-
sented in Figure 5 below.
Participants demonstrated highest agreement across all
therapy approaches for instructions (LMRVT = 81.8%; Yell
Well = 69.0%; VFE = 49.5%). There was less agreement
when classification explanations (LMRVT = 41.1%; Yell
Well = 35.3%; VFE = 55.5%) and perceptual training was

FIGURE 4. Inter-rater reliability for each general prepractice

variable within each therapy approach. General variables are moti-
FIGURE 3. Percent agreement for each rater within each therapy vation (MOT), modeling (MOD), general verbal information (VI-
approach. SUM), feedback (FB), and other. Error bars denote Standard
R, rater. Deviation.
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
FIGURE 5. Inter-rater reliability for each specific prepractice
variable within each therapy approach. Specific variables are
instructions (IN), explanations (EX), perceptual training (PT) and
other verbal information (Other VI).
rated relatively high for agreement in LMRVT (50.9%) but
not in Yell Well (33.9%) and VFE (18.5%). The other verbal
information category was not used by participants in either
in LMRVT or Yell Well but was used on 16 occasions for
VFE with 14.8% agreement.
Behavior analysis
Due to the relatively low inter-rater agreement for some
prepractice variables, we conducted a further behavioral
analysis to identify which behaviors had full agreement or
incomplete agreement. Complete agreement (100%) was
achieved when all participants categorized a behavior the
same (eg, all eight participants categorized the clinician
producing “m” as modeling). Incomplete agreement was
anything less than 100%. A summary of these analyses sub-
divided into general categories is given in Table 2.
For behaviors on which participants did not agree, we
identified specific behaviors which are displayed in Table 3.
Rater evaluations
Analysis of written comments provided at the end of the ses-
sions revealed that participants thought the MLCF was a
TABLE 2.
Summary Table Showing the Number and Percent Agreement for Behaviors Which had Complete Agreement or Incom-
plete Agreement in Each Therapy Approach
LMRVT (n = 120)
# %
Complete agreement
General variables only 91
General and specific variables 79
Incomplete agreement 28
Note: n, the total number of clinician behaviors within each therapy approach; #, the number of clinician behaviors for which there was complete agreement
or outlying ratings.
7
useful tool. Rater's evaluations of the framework and classi-
fication task are summarized in Table 4 below.
Relative frequency of prepractice variables across
different treatments
Only classifications that had complete agreement were
assessed to ascertain if there were differences in the ratio of
prepractice behaviors in each of the different video seg-
ments. There were observed differences in the amount of
modeling, instructions and feedback as shown in Figure 6.
DISCUSSION
We investigated whether motor learning concepts, specifi-
cally prepractice variables, could be reliably identified and
thus operationalized in clinical interactions in voice therapy.
Evidence of good intrarater reliability was shown, but inter-
rater reliability was not as high. In general, there was rela-
tively high inter-rater percent agreement for the identifica-
tion of modeling and verbal information, but lower
agreement for motivation and feedback. Within the category
verbal information (instructions, explanations, and percep-
tual training), inter-rater reliability was also typically low.
However, there were some consistent patterns in disagree-
ment, indicating that classification disagreement was mostly
due to difficulties categorizing motivational and conversa-
tional statements. The participants were generally positive
about the MLCF as a way of evaluating client-clinician
interactions in voice therapy.
Overall intrarater and inter-rater reliability
The high intrarater agreement (M = 88.7%) suggests partici-
pants developed internal standards for each prepractice var-
iable and used these standards to classify the clinician
behaviors in each classification session. Inter-rater reliability
across all general and specific variables was relatively low.
However, there was complete agreement between partici-
pants on the general variables for »77% of behaviors in
LMRVT, »69% in Yell Well, and »66% in VFE. Given the
number of potential response categories for the eight partic-
ipants, these numbers suggest that on a general level,
Yell Well (n = 67) VFE (n = 73)
# % # %
76.7 46 68.7 48 65.8
65.8 31 46.3 31 42.5
23.3 16 23.9 17 23.3
 ARTICLE IN PRESS
8 Journal of Voice, Vol. &&, No. &&, 2019

TABLE 3.
Patterns of Disagreement for Clinician Behaviors With Low Inter-rater Reliability
Behavior Type
Indirect feedback ie, instruction
provided immediately after
an attempt
Explanations to motivate client
Hand and facial gestures
Explanations
Instructions with sensory
terms
Knowledge of performance
feedback
Conversational behavior/
checking client’s knowledge
or understanding
Instruction regarding sound to
be produced
Example
“I’m going to stop you”
“You’re doing something very
good for your vocal folds”
Moves index finger back and
forth in front of body.
“I’m going to tell you how to
make the biggest shout.”
“Now see if you can feel your
giggle on your own hand.”
“Your neck’s disappeared.”
“Remember that?” “Ok?”
“With an ‘m’”
Rating Pattern
 All agree verbal information
 R3 & R8 also rated it as feedback
 Split between verbal information (explanations)
and motivation
 Some rate as both
 Most agree this is modeling
 R4 & R8 consistently rated as feedback
 Occasionally no rating from one or more raters
 Split between instructions and explanations
 Raters also showed lower intrarater agreement
for these behaviors
 Split between instructions and explanations
 Split between feedback and verbal information
(explanations or perceptual training).
 Split between other, other verbal information,
explanations, feedback and motivation
 Most agree verbal information (instructions)
 Some rated as modeling

participants were relatively consistent at discriminating
between the behaviors.
Inter-rater reliability for prepractice variables
Participants demonstrated relatively high agreement for
modeling (67%−88%) and verbal information (76%−87%).
This suggests that participants found it easy to identify
doing (modeling) and talking (verbal information) behav-
iors. The more reliably identified behaviors were also the
most frequently exhibited.
The low frequency use of the other variable was another
significant finding supporting the utility of the MLCF. This
category was never used in classifications of LMRVT and
Yell Well and only 10 times across all participants and
behaviors in VFE. This demonstrates that clinicians’

TABLE 4.
Evaluation of Framework and Rating Task
Evaluation Theme
The framework was initially over-
whelming for some raters.
The framework was comprehensive
and useful.
It was difficult to understand some
framework variables.
More training was required for the
rating task.
More time was required for rating
task.
The observation checklist was well
formatted.
The framework could be used for
self-evaluation.
The framework could be used as a
training tool for student education
and evaluation.
Example
“The framework is a bit overwhelming at first but once I became more familiar
with it, it was actually very clear to understand.”
“The framework is an excellent way to conceptualize the components of ther-
apy. It’s comprehensive but not confusing.”
“. . .some variables were harder than others.”
“I was confused about how to rate a few utterances and I think a little more
training and explanation time may have helped this.”
“Also, perhaps by “pausing” a little more frequently... I found that I was not
keeping up with the audio.”
“The checklist is very useful, especially since all behaviors/utterances can be
coded under the headings included. It’s quick and easy to use.”
“I really like it because it is quite a ‘meta’ way of looking at your own clinical
behavior.”
“It would be useful to do a similar task, at uni, across the 5 areas (eg, speech/
voice/stuttering, etc) to help us realize the importance of giving motivation/
instructions/ explanations, etc., in therapy, and how to deliver it.”
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
FIGURE 6. Average number of identified behaviors in 3 minutes.
behaviors can be operationalized and classified using the
MLCF, along with clarification of the framework categories
through a pretraining session and an example look-up table.
It will be important to demonstrate that such observations
improve with practice (and feedback).
Disagreement analysis
Motivation and feedback and the specific prepractice varia-
bles instructions, explanations, and perceptual training had
the lowest inter-rater agreements. The use of both general
categories (ie, verbal information) and specific variables
(ie, instructions, explanations, and perceptual training)
within a single classification task, which is common to
many similar checklists eg,64,65 may have improved speci-
ficity at the expense of accuracy. Distinguishing between
different types of verbal information may be difficult due
to the often implicit meaning of some statements and the
wider context in which they might be uttered.
Multiple classifications of specific behaviors
Analysis of the patterns of disagreement showed that the
tendency to give multiple classifications to one specific
behavior was unusual, and only shown by a few participants
(reducing overall reliability). Some participants picked up
on more subtle meanings of phrases whilst other partici-
pants only classified based on what was explicitly said. In
the VFE video segment, the client attempts the movement
and then the clinician says, “I'm going to stop you.” Whilst
the explicit instruction is to stop, feedback that the move-
ment was wrong is also implied by that instruction. All par-
ticipants identified the explicit meaning (instruction) and
two participants also identified the more implicit meaning
(feedback) in their classification. Thus, the reliability for
feedback was reduced by “reinstruction” behaviors.66 Reli-
ability for motivation was similarly reduced for explanations
about the importance of the task where all participants
9
identified the explicit meaning (explanation) and a few iden-
tified the more implicit meaning (motivation).
Mutual exclusivity and verbal information
subcategories
The lower reliability for the subcategories of verbal infor-
mation was most likely caused by a lack of clear definitions
of the three subcategories. There was consistent confusion
between explanations and instructions due to the subtle dif-
ference in definitions for these variables. Explanations typi-
cally involve a task description whereas instructions involve
task description and an imperative to perform. A similar
difficulty in distinguishing behaviors that included informa-
tion about the task but did not require the client to act was
noted in classification of motor learning behaviors in a reha-
bilitation setting with children with coordination disor-
ders.52 They called this variable “sharing knowledge” in
order to clearly distinguish it from both feedback and
instructions.
When operationalizing the variables for the classification
task, there was difficulty in clearly defining each of the spe-
cific variables. As these variables have typically been inves-
tigated independently, there is some overlap between their
definitions.67 For example, auditory perceptual training
may also be considered as auditory modeling. One way to
increase mutual exclusivity would be to pay careful consid-
eration to the terminology. For example, renaming percep-
tual training as “attention to sensory effects” may assist to
more clearly differentiate between modeling and perceptual
training.
Conversational behaviors
Other behaviors which showed high disagreement were con-
versational-type behaviors. Classifications for behaviors
such as the clinician asking, “OK?” were often split between
multiple categories (eg, other, other verbal information,
 ARTICLE IN PRESS
10
feedback). Excluding these conversational behaviors from
analysis would have increased the reliability in applying the
MLCF. Conversational behavior may, however, be consid-
ered a variable in and of itself. Whilst not specific to motor
learning,52 conversational behaviors are a clinical reality
and it is possible that these behaviors serve an important
but as yet undescribed function in motor acquisition. For
example, they may establish and maintain motivation for
therapy, trust in the clinician through personal rapport and
/or provide necessary breaks from practice. These types of
conversational behaviors may be better understood using a
systemic functional linguistics analysis61 rather than a
motor learning behavior analysis.
Motor learning behaviors as differentiating
ingredients of different therapies
Analysis of the frequency of different prepractice variables in
the video segments of the three therapies showed high variabil-
ity. This variability may be due to inherent differences in the
therapy approaches, individual clinician's differences, individ-
ual patient differences or the production decisions made in the
video segments used in this study. All videos chosen were mar-
keted as demonstration videos in which the therapy is demon-
strated by the author of the therapy technique using a real
patient. This suggests that these videos can be considered
exemplars of the specific therapy approach. An analysis of the
therapy using the MLCF may provide insight in future
research as to whether the motor learning behaviors used in a
therapy might be considered as an “active ingredient” as much
as the underlying skill being taught.
Clinical implications
The aim of developing the MLCF was to provide clinicians
with a means of identifying behaviors and interactions in ther-
apeutic and training interventions. This would allow them to
consciously and more accurately follow explicit guidelines for
the different stages of intervention. It appears that the MLCF
could be both refined and expanded. For example, it could be
used to categorize and systematically analyze the instructional
behaviors used by clinicians to prepare their client for prac-
tice. As such the MCLF could play an important role in edu-
cating students and clinicians on the various behaviors that
can be utilized across the learning process.
Identification of different variables in the three voice ther-
apy approaches might speak to inherent differences in the
process of the specific voice therapy approach, or it may
just reflect differences in the style of the individual clinician
and/or client. The use of the MCLF presents an opportunity
to further understand differences between voice therapy
techniques and provides insight into which approach may
be more or less efficacious depending on the clinician's and
client's individual characteristics. This may ultimately help
us understand what the critical “active ingredients” are for
successful therapy. Whether it will be possible to apply this
classification analysis to real-life clinical interactions is yet
to be explored.
Journal of Voice, Vol. &&, No. &&, 2019
Limitations and future directions
The current study was undertaken as an initial exploration
of the use of the MLCF in helping to identify clinically rele-
vant behaviors for effective learning. This study was, how-
ever, limited by the small number of participants and the
lack of diversity in the participants. Because we only col-
lected frequency data, we were also limited by the types of
analysis we could perform. In future, it will be important to
replicate and extend this work, comparing across more par-
ticipants, and across participants of varying ability/experi-
ence. Online technologies can be used in future work to
facilitate ease and accuracy of classification (for example,
Bridge2Practice68). Application of the MCLF to other areas
of voice training (eg, singing and spoken voice training) and
SLP practice (eg, motor speech, stuttering, dysphagia, etc)
should be investigated to evaluate if there are specific motor
learning concepts that are operationalized differently
depending on context. Similarly, the MLCF can be applied
to the teaching of any motor or perceptual skills in disci-
plines such as Medicine, Surgery, Physiotherapy, Occupa-
tional Therapy, Medical Radiation Sciences, and Exercise
and Sports Physiology.
CONCLUSIONS
We provided some initial evidence supporting the use of a
proposed MLCF as a tool for analyzing the prepractice
phase of voice-therapy interventions. The general variables
modeling and verbal information were identified reliably,
however, participants were less reliable in identifying moti-
vation and feedback. Clearer instructions to participants
regarding the depth of analysis required for these behaviors,
along with a few more examples and further practice,
should help increase reliability for these variables. The
reported difficulty in identifying specific prepractice varia-
bles such as explanations and perceptual training suggests
the need for clearer definitions and distinctions between
these variables and further research to determine if they are
separable and involve different motor learning principles.
There has been a recent move to explore motor learning
principles in more applied settings. This initial analysis of
the use and discrimination of motor learning variables in
voice therapy sessions paves the way for future research to
better evaluate behaviors in SLP settings, with an eye to
analyzing whether principles of good motor learning are
being adhered in such settings and whether there are associ-
ated consequences for treatment outcomes when they are
not. Although we only provide a simple discrimination, fur-
ther development of the MLCF would allow for a more in-
depth analysis of SLP interventions, ultimately helping to
identify key behaviors for success such that clinicians can
become more effective teachers of voice motor skills.
Acknowledgments
Our sincere thanks to the participants, to Dr Robert Heard
for advice regarding research design and statistics, to San-
tino Di Bartolo for assistance with video editing, and to Dr
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
Duong Nguyen for manuscript preparation. This study was
supported by the Dr Liang Voice Program at The Univer-
sity of Sydney.
APPENDIX A: WRITTEN DEFINITIONS OF
PREPRACTICE VARIABLES
General prepractice variables: Definitions
(participants)
1. Motivation
The clinician engages in behavior which aims to increase
the client's motivation to participate in the therapy task.
Examples:
 Outlining the importance or benefits of the task.
 Setting goals for therapy.
2. Verbal Information
The clinician verbally communicates what the target
movement is and what they need to do to achieve the target.
Common ways that clinicians provide this information is
through instructions, explanations, and perceptual training.
2 a. Instructions: The clinician provides information to
the client about what they should do to produce the target
movement. Instructions occur before or during the client's
movement attempt.
Instructions may be:
 Short and simple.
 Long and complex.
 Refer to part of the movement.
 Refer to the movement holistically, eg, using meta-
phors (“Open your throat like when you yawn”).
Note: Instructions may include perceptual terms, how-
ever, the focus of the utterance is what the client should do
or the action to perform (eg, “I want you to slide into that
“m” so that it is a nice, smooth sound”).
2 b. Explanation: The clinician provides information and
descriptions to the client to help them understand some
aspect of the target movement or something related to the
task. Explanations may occur before or after the movement.
Examples:
 Information about the mechanical or biomechani-
cal properties (eg, vocal fold vibration).
 Elaboration of an instruction.
2 c. Perceptual Training: Perceptual training involves rais-
ing the client's awareness of sensations to develop under-
standing of the target movement. The client notices and
reflects on what they see, feel, or hear during or after they
attempt a new movement. Perceptual training may occur
before, during or after the client's movement attempts.
11
Examples:
 Asking the client to attend to a sensation
(eg, “I want you to notice if you feel more
vibrations.”).
 Asking the client to remember or self-evaluate a
sensation
(eg, “How did that feel/sound?”)
 Contrasting one sensation with another
(eg, “Which way felt easier?” using an old way/new
way).
3. Modeling
A physical demonstration of the movement is provided to
the client to assist them to develop an understanding of the
target movement. Demonstrations may be provided before
or during the client's movement attempts.
The demonstration can be:
 Part of a movement.
 The whole movement.
 Correct or incorrect movement (as in negative prac-
tice).
 A small hand gesture modeling some part of the
movement (eg, point to the upper lip to indicate
that the client needs to project the sound forward).
4. Feedback
The clinician makes a judgment about the client's move-
ment attempt. Feedback can occur during or after a move-
ment.
Examples:
 Knowledge of Results (ie, correct/ incorrect or
clear/husky).
 Knowledge of Performance (eg, “You brought the
vocal folds together too hard that time.”).
Note: Feedback may include perceptual / sensory terms,
however the focus of the utterance is that the clinician is
making a judgement (eg, “That sounded clearer.”).
5. Other-please explain
Use this code if you cannot classify any behavior or phrase
according to the suggested variables. Please include a brief
description of the clinical relevance of the behavior/phrase.
Examples:
 Acknowledgment of a client utterance (not an attempt)
 Consolation of the client
Multiple classifications: You are able to rate one utter-
ance or behavior as more than one variable. However, you
are encouraged to differentiate between the variables as
much as possible.
 ARTICLE IN PRESS
12
APPENDIX B: OBSERVATION CHECKLIST
REFERENCES
1. Langmore SE, Pisegna JM. Efficacy of exercises to rehabilitate dysphagia:
a critique of the literature. Int J Speech Lang Pathol. 2015;17:222–229.
2. Maas, Butalla, Farinella. Feedback frequency in treatment for child-
hood apraxia of speech. Am J Speech Lang Pathol. 2012;21:239–257.
3. Murray E, McCabe P, Ballard KJ. A randomized controlled trial for
children with childhood apraxia of speech comparing rapid syllable
transition treatment and the Nuffield Dyspraxia Programme−Third
Edition. J Speech Lang Hear Res. 2015;58:669–686.
4. Joscelyne-May C, Madill CJ, Thorpe W, et al. The effect of clinician
feedback type on the acquisition of a vocal siren. Folia Phoniatr Logop.
2015;67:57–67.
5. Look C, McCabe P, Heard R, et al. Show and tell: video modeling and
instruction without feedback improves performance but is not sufficient
for retention of a complex voice motor skill. J Voice. 2019;33:239–249.
6. Meerschman I, Van Lierde K, Van Puyvelde C, et al. Massed versus
spaced practice in vocology: effect of a short-term intensive voice train-
ing versus a longer-term traditional voice training. Int J Lang Commun
Disord. 2018;53:393–404.
7. Ma EPM, Yiu GKY, Yiu EML. The effects of self-controlled feedback
on learning of a “relaxed phonation task”. J Voice. 2013;27:723–728.
8. Steinhauer K, Grayhack JP. The role of knowledge of results in perfor-
mance and learning of a voice motor task. J Voice. 2000;14:137–145.
9. Yiu EML, Verdolini K, Chow LPY. Electromyographic study of
motor learning for a voice production task. J Speech Lang Hear Res.
2005;48:1254–1268.
10. Roy N. Optimal dose-response relationships in voice therapy. Int J
Speech Lang Pathol. 2012;14:419–423.
Journal of Voice, Vol. &&, No. &&, 2019
11. Mathieson L. Greene and Mathieson's the Voice and Its Disorders. (6th
ed.) Chichester, UK: Wiley and Sons Ltd; 2001.
12. Eastwood C, Madill C, McCabe P. The behavioural treatment of mus-
cle tension voice disorders: a systematic review. Int J Speech Lang
Pathol. 2015;17:287–303.
13. Van Stan JH, Roy N, Awan S, et al. A taxonomy of voice therapy. Am
J Speech Lang Pathol. 2015;24:101–125.
14. Schmidt RA, Lee TD. Motor Control And Learning: A Behavioural
Emphasis. (4th ed.) Champaign, Illinois: Human Kinetics; 2005.
15. Adams JA. A closed-loop theory of motor learning. J Motor Behav.
1971;3:111–150.
16. Maas E, Robin DA, Austermann Hula SN, et al. Principles of motor
learning in treatment of motor speech disorders. Am J Speech Lang
Pathol. 2008;17:277–298.
17. Wulf G, Shea CH. Principles derived from the study of simple skills do
not generalize to complex skill learning. Psychonom Bullet Rev.
2002;9:185–211.
18. McNeil MR. Clinical management of sensorimotor speech disorders.
2nd ed Thieme New York; 2009.
19. Hodges NJ, Franks IM. Modelling coaching practice: the role of
instruction and demonstration. J Sports Sci. 2002;20:793–811.
20. Schumann A, Liebscher T, Hoppe U. Phoneme discrimination training
with experienced cochlear implant listeners. HNO. 2016;64:751–758.
21. Brown S, Martinez MJ. Activation of premotor vocal areas during
musical discrimination. Brain Cogn. 2007;63:59–69.
22. Meegan DV, Aslin RN, Jacobs RA. Motor timing learned without
motor training. Nat Neurosci. 2000;3:860.
 ARTICLE IN PRESS
Catherine Madill, et al Classifying Motor Learning Behaviors in Voice Therapy
23. Granados C, Wulf G. Enhancing motor learning through dyad prac-
tice: contributions of observation and dialogue. Res Quart Exerc Sport.
2007;78:197–203.
24. Ram N, Riggs S, Skaling S, et al. A comparison of modelling and
imagery in the acquisition and retention of motor skills. J Sports Sci.
2007;25:587–597.
25. Behrman A, Rutledge J, Hembree A, et al. Vocal hygiene education,
voice production therapy, and the role of patient adherence: a treat-
ment effectiveness study in women with phonotrauma. J Speech Lang
Hear Res. 2008;51:350–366.
26. Hodges NJ, Chua R, Franks IM. The role of video in facilitating per-
ception and action of a novel coordination movement. J Sport Exercise
Psychol. 2002;24:69–70.
27. Steinhauer K, Grayhack JP. The role of knowledge of results in perfor-
mance and learning of a voice motor task. J Voice. 2000;14:137–145.
28. Swinnen SP. Information feedback for motor skill learning: a review.
Adv Motor Learn Control. 1996:37–66.
29. Guadagnoli MA, Lee TD. Challenge point: a framework for conceptu-
alizing the effects of various practice conditions in motor learning.
J Motor Behav. 2004;36:212–224.
30. Lee TD, Swinnen SP, Serrien DJ. Cognitive effort and motor learning.
Quest. 1994;46:328–344.
31. Gentile AM. Movement science: implicit and explicit processes during
acquisition of functional skills. Scand J Occup Ther. 1998;5:7–16.
32. Bose A, Square PA, Schlosser R, et al. Effects of PROMPT therapy on
speech motor function in a person with aphasia and apraxia of speech.
Aphasiology. 2001;15:767–785.
33. Bagnall A. Voicecraft workshop manual. Adelaide, SA: Voicecraft
International; 1997.
34. Schmidt RA, Lee TD. Motor Control And Learning: A Behavioral
Emphasis. 5th ed. Champaign, IL: Human kinetics; 2011.
35. Wulf G, Prinz W. Directing attention to movement effects enhances
learning: a review. Psychonom Bullet Rev. 2001;8:648–660.
36. Buszard T., Masters R.S. A culture of striving augments use of work-
ing memory? Implications for attention control. Progress Brain Res.
2017;232:197−200.
37. Masters RS. Knowledge, knerves and knowhow: the role of explicit
versus implicit knowledge in the breakdown of a complex motor skill
under pressure. Br J Psychol. 1992;83:343–358.
38. Poolton JM, Zhu FF, Malhotra N, et al. Multitask training promotes
automaticity of a fundamental laparoscopic skill without compromis-
ing the rate of skill learning. Surg Endosc. 2016;30:4011–4018.
39. Verdolini-Marston K, Balota DA. Role of elaborative and perceptual
integrative processes in perceptual-motor performance. J Exp Psychol
Learn Mem Cogn. 1994;20:739–749.
40. Davids KW. The constraints-based approach to motor learning: impli-
cations for a non-linear pedagogy in sport and physical education. In:
Renshaw I, Davids KW, Savelsbergh GJP, eds. Motor Learning in
Practice: A Constraints-Led Approach. London: Routledge (Taylor &
Francis Group); 2010:3–16.
41. Ara ujo D, Davids K, Bennett SJ, et al. 19. Emergence of sport skills
under constraints. Skill Acquisition in Sport: Research, Theory and
Practice. UK: Routledge; 2004:409–433.
42. Kim S. The effect of two different singing instructions on subglottal
pressure, sound pressure level, and airflow rate. The Voice Foundation
46th Annual Symposium. 2017. Philadelphia, PA.
43. Hodges NJ, Franks IM. Modelling coaching practice: the role of
instruction and demonstration. J Sports Sci. 2002;20:793–811.
44. Beek PJ. Toward a theory of implicit learning in the perceptual-motor
domain. Int J Sport Psychol. 2000;31:547–554.
13
45. Hodges, Ste-Marie. Observation as an instructional method. Develop-
ing Sport Expertise: Researchers and Coaches Put Theory into Practice.
(2nd ed) 2013. p. 115−128.
46. Andrieux M, Proteau L. Mixed observation favors motor learning
through better estimation of the model's performance. Exp Brain Res.
2014;232:3121–3132.
47. Rohbanfard H, Proteau L. Learning through observation: a combina-
tion of expert and novice models favors learning. Exp Brain Res.
2011;215:183–197.
48. Weeks DL, Anderson LP. The interaction of observational learning
with overt practice: effects on motor skill learning. Acta Psychologica.
2000;104:259–271.
49. Potter W.J., LevineDonnerstein D.. Rethinking validity and reliabil-
ity in content analysis. J Appl Commun Res 1999;27:258−284.
50. Cushion CJ, Jones RL. A systematic observation of professional top-
level youth soccer coaches. J Sport Behav. 2001;24:354–376.
51. Niemeijer AS, Schoemaker MM, Smits-Engelsman BC. Are teaching
principles associated with improved motor performance in children
with developmental coordination disorder? A pilot study. Phys Ther.
2006;86:1221–1230.
52. Niemeijer AS, Smits-Engelsman BC, Reynders K, et al. Verbal actions
of physiotherapists to enhance motor learning in children with DCD.
Hum Mov Sci. 2003;22:567–581.
53. Ford P, Yates I, Williams A. An analysis of activities and instruc-
tional behaviours used by coaches during practice in English youth
soccer: exploring the link between theory and practice. J Sport Sci.
2010;28:483–495.
54. Williams AM, Hodges NJ. Practice, instruction and skill acquisition in
soccer: Challenging tradition. J Sport Sci. 2005;23:637–650.
55. Potrac P, Jones R, Cushion C. Understanding power and the coach's
role in professional English soccer: a preliminary investigation of coach
behaviour. Soccer Soc. 2007;8:33–49.
56. ASHA. Guidelines for manual pure-tone threshold audiometry. 2005.
57. Abbott KV. Lessac-Madsen Resonant Voice Therapy: Overview (Vid-
eorecording). San Diego: Plural Publishing; 2008.
58. Bagnall AD, Sprod R. Yell Well: AB Voice International; 1995.
59. Stemple J. Vocal Function Exercises (Videorecording). San Diego:
Communicare Publishing; 2002.
60. Verdolini K, Druker DG, Palmer PM, et al. Laryngeal adduction in
resonant voice. J Voice. 1998;12:315–327.
61. Angadi V, Croake D, Stemple J. Effects of vocal function exercises: a
systematic review. J Voice. 2019;33. 124.e13−124.e34.
62. Frensch PA. Composition during serial learning: a serial position
effect. J Exp Psychol Learn Mem Cogn. 1994;20:423–442.
63. Portney LG, Watkins MP. Foundations of Clinical Research: Applica-
tions to Practice. 3rd ed. Upper Saddle River, NJ: Pearson/Prentice
Hall; 2009.
64. Damico JS. Clinical discouse analysis: a functional approach to lan-
guage assessment. Communication Skills and Classroom Success. 1985
p 165−204.
65. Prutting CA, Kittchner DM. A clinical appraisal of the pragmatic
aspects of language. J Speech Hear Disord. 1987;52:105–119.
66. Lacy AC, Darst PW. Evolution of a systematic observation system:
the ASU coaching observation instrument. J Teach Phys Educ.
1984;3:59–66.
67. Verschuren PJ. Holism versus reductionism in modern social science
research. Qual Quant. 2001;35:389–405.
68. Madill C, So T, Corcoran S. Bridge2practice: Translating theory into
practice 2019. Available at: https://bridge2practice.com/. Accessed
March 7, 2019.