Behavioral Interventions - 2023 - Kamana - Large Scale Evaluation of Staff Training in Programs For Adults With

Received: 17 November 2022 Revised: 24 July 2023 Accepted: 27 July 2023
DOI: 10.1002/bin.1971
RESEARCH ARTICLE
Large-scale evaluation of staff training in programs

for adults with intellectual and developmental
disabilities
Bertilde U. Kamana1 | Claudia L. Dozier2 | Nicole A. Kanaman3 |

Florence DiGennaro Reed2 | Stephanie M. Glaze2 | Ali M. Markowitz2 |
Marcella M. Hangen4 | Kelley L. Harrison2 | Alec M. Bernstein2 |
Rachel L. Jess3 | Tyler G. Erath5
The May Institute, Santa Cruz, California, USA

1
2
Department of Applied Behavioral Science,
Abstract
University of Kansas, Lawrence, Kansas, USA Behavioral skills training and on-the-job feedback are effec-
GoodLife Innovations, Lenexa, Kansas, USA
3
tive in changing staff behavior as evidenced by years of
4
Department of Disability and
staff-training research. However, community programs for
Psychoeducational Studies, University of
Arizona, Tucson, Arizona, USA adults with intellectual and developmental disabilities (IDD)
5
Vermont Center on Behavior and Health, often do not utilize these best-practice training methods.
Burlington, Vermont, USA
The purpose of the current study was to train four empir-
Correspondence ically derived practices to staff who work with adults with
Bertilde U. Kamana.
IDD. We trained the staff to provide positive interactions,
Email: bkamana@thebayschool.org
provide effective instructions, provide correct responses
to problem behavior, and promote consumer engagement
with items and activities. We used behavioral skills training
and on-the-job feedback to increase staff implementation of
these practices on a large scale in a community-based organ-
ization despite some barriers such as high staff turn over
rates. Overall, results showed that our training procedure
was effective in increasing staff implementation of the four
practices in many homes and programs.
KEYWORDS
adults with disabilities, behavior skills training, community-based
intervention, on-the-job feedback, staff training
Behavioral Interventions. 2023;1–22. wileyonlinelibrary.com/journal/bin © 2023 John Wiley & Sons Ltd. 1
1099078x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/bin.1971 by Behavior Analyst Certification Board, Wiley Online Library on [20/10/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2 KAMANA et al.
1 | INTRODUCTION
The late 1960s are partly notable for the deinstitutionalization movement that resulted in the relocation of many
adults with intellectual and developmental disabilities (IDD) from institutions to community-based homes and day
programs (Bouras & Jacobson, 2002; Fulone et al., 2021). The transition from institutions to community-based
programs for many adults with IDD has resulted in more choice, access to integration, and notably, an opportunity for
caregivers (e.g., staff and families) to be involved in service delivery for adults with IDD (Bouras & Jacobson, 2002).
Federal regulations require that programs for adults with IDD provide active treatment, which refers to “a continuous,
aggressive, and consistent implementation of a program of specialized and generic training, treatment, and health or
related services, directed toward helping the enrollee function with as much self-determination and independence
as possible” (Code of Federal Regulations, 2011; Medicaid, 2019). However, federal and state regulations on active
treatment do not clarify the specific types of services that constitute active treatment in these settings.
Both early and contemporary researchers have attempted to define some specific staff behaviors that may
constitute active treatment for adults with IDD (e.g., Parsons et al., 1989; Parsons & Reid, 1993). Some of these
behaviors include staff positive interactions and rapport building, choice provision, delivery of effective instructions
(e.g., use of prompts), and promotion of appropriate consumer engagement in activities (Cooper & Browder, 2001;
Fleming & Sulzer-Azaroff, 1992; Jones et al., 1999; McLaughlin & Carr, 2005; Parsons et al., 1989, 2004; Realon
et al., 2002; Reid et al., 2001; Repp et al., 1981; Weinberg et al., 2000). However, several researchers have reported
lack of active treatment in various environments serving adults with IDD (Chan & Yau, 2002; Felce & Emerson, 2001;
Parsons et al., 1989, 2004, 2013; Reid et al., 2001; Repp et al., 1981). For example, with respect to positive interac-
tions, Chan and Yau (2002) found that interactions between staff and consumers were absent in approximately 62%
of intervals during observations, and most of the interactions provided were centered around custodial or health care.
Behavior challenges are common in individuals with IDD (Colombo et al., 2021; Lowe et al., 2007). There-
fore, staff who work with individuals with IDD are often trained on the common functions of challenging behavior
(Hagopian et al., 2013) and why it is important to create treatment environments that provide frequent access to
meaningful social reinforcers (noncontingent reinforcement; Carr et al., 2002). Furthermore, staff who work with
individuals with IDD may be trained to implement procedures that attempt to eliminate the response–reinforcer
relationship between challenging behavior and common social reinforcers (e.g., Ala'i-Rosales et al., 2018). Some ways
to create environments for adults with IDD that take the common functions of challenging behavior into account
involve providing frequent access to meaningful social reinforcers (e.g., providing attention and access to tangibles),
attempting to eliminate evocative situations that may lead to challenging behavior (e.g., providing prompts; Ebanks &
Fisher, 2003), and using differential reinforcement to increase appropriate behavior and decrease challenging behav-
ior (Carr & Durand, 1985). Overall, the functional behavior assessment and treatment literature also points to the
importance of active treatment in environments for adults with IDD.
The lack of active treatment in some programs for adults with IDD may partly be influenced by the quality of staff
training in these settings. Behavior skills training (BST; Parsons et al., 2012) is a best-practice training method that
involves instruction, modeling, rehearsal, and feedback. Effective staff training methods (e.g., BST), in combination
with on-the-job support and feedback, have been found to be necessary for the maintenance of staff skills on the
job. On-the-job support and feedback (referred to OJF in this paper) refers to the monitoring of staff skills through
observations on the job and providing feedback and ongoing training as needed to maintain and improve staff skills
(Van OOrsouw et al., 2009). Many researchers have used these best-practice training methods to train staff working
with adults with IDD in community-based programs (Collins et al., 2009); however, according to a recent study (Erath
et al., 2019), these best-practice training methods are not consistently being utilized by programs for adults with
IDD. In fact, research has suggested there is a research-to-practice gap in selecting training procedures that incor-
porate the essential components of effective training in human service industries (Fixsen et al., 2010). Thus, if staff
who work in organizations that serve individuals with IDD are not effectively trained, then adults with IDD in these
programs may not be accessing active treatment. It is important to note some barriers to effective training practices
KAMANA et al. 3
in programs for adults with IDD including lack of access to experts who can provide effective training and lack of
resources needed to provide training in programs that are across multiple communities (Cox et al., 2015; DiGennaro
Reed et al., 2013; Hewitt & Larson, 2007; Rotholz et al., 2013; Shapiro & Kazemi, 2017; Wood et al., 2007). Further-
more, researchers have reported that BST and OJF requires time (Parsons et al., 2013), which is only increased for
community-based programs that are often scattered across various locations that require travel time for trainers (e.g.,
Erath et al., 2019). Finally, high staff turnover and staffing shortages (Hewitt & Larson, 2007), lack of funding for adult
services (e.g., Bottos, et al., 2001), and lack of clear federal regulations and guidelines for behavior support practices
(Anderson et al., 2012; Rotholz et al., 2013) make establishing active treatment challenging. Therefore, there is a need
for a demonstration of the effectiveness of best-practice staff training methods in programs for adults with IDD that
have some of the barriers discussed.
Decades of research, albeit mostly in institutional settings (Harchik & Campbell, 1998; Wood et al., 2007), clari-
fied ways to improve staff provision of various aspects of active treatment using best-practice staff training methods
in the organizational and behavior management literature (e.g., BST & OJF; e.g., Harchik & Campbell, 1998; Harchik
et al., 1992; Parsons et al., 1989; Reid et al., 2011; Van OOrsouw et al., 2009). However, the focus of these studies
has been on improving one or two aspects of active treatment (e.g., promoting positive interactions and increasing
consumer engagement). Thus, research that systematically addresses multiple staff behaviors for increasing active
treatment is warranted. To the best of our knowledge, there has not been a study that has utilized BST and OJF to
train staff in programs for adults with IDD on multiple skills, on a large scale. In this study, we evaluated the effects
of BST and OJF (Parsons et al., 2012; van Oorsouw et al., 2009) to train approximately 150 staff to implement four
practices which included: provide positive interactions, provide effective instructions, provide correct responses to
problem behavior, and promote consumer engagement with items and activities.
2 | METHOD
2.1 | Participants and setting
Participants were approximately 150 staff and various consumers from 16 group homes and 3-day programs in a
large organization serving adults with IDD. Staff were at least 19 years old and had at least a high school diploma or
General Equivalency Degree. A large number of staff were included due to various factors. First, staff included day and
night staff, weekday and weekend staff, and regular and substitute staff who filled in when regular staff were absent.
Second, new staff were included at various times throughout the study given the relatively high staff-turnover rates
in some homes and programs. Thus, staff were not necessarily consistent within a home or program across phases.
All staff participated in a mandatory 5-day, new-hire training prior to working for the company. This new-hire train-
ing included training on using strategies for occasioning appropriate behavior, recognizing antecedents and conse-
quences of problem behavior, and implementing general problem behavior management strategies using a systematic
crisis management program.
Consumers who participated in this study were adults with IDD who were between 18 and 60 years old and lived
in the community-based homes (i.e., group homes) or attended the day programs where we conducted the study.
Most consumers were reported to engage in minor problem behavior (e.g., inappropriate verbal behavior), severe
problem behavior (e.g., physical aggression), or both, for which they had behavior support plans.
Graduate student consultants conducted all trainings, observations, and feedback sessions with staff. All
consultants were enrolled as full-time doctoral students studying behavior analysis; six of them were Board Certified
Behavior Analysts (BCBAs)® and three were receiving supervised field experience to fulfill the requirements for
becoming BCBAs. All consultants had prior experience in the assessment and treatment of problem behavior and
function-based interventions. Each consultant had a caseload of homes or programs and kept records of all staff
training, observations, and feedback throughout the duration of the study. Two doctoral-level behavior analysts
4 KAMANA et al.
(second and fourth authors) trained and supervised the consultants. All consultant training and supervision took
place during weekly, 2-h meetings and consisted of the doctoral-level behavior analysts modeling the training and
feedback process to be used with staff.
All initial training, observations, and feedback took place in the homes between 6 a.m. and 9 p.m., Monday
through Sunday, and in the day programs between 9 a.m. and 3 p.m., Monday through Friday. Initial training sessions
took place in a quiet corner or location within the homes or programs with one or two staff on shift. Observa tions took
place in the common areas of homes (e.g., living room) during times in which staff and consumers were scheduled to
be in the homes and were engaged in activities in these common areas (e.g., meals and leisure activities). Observa-
tions also took place in various large and small rooms at the day programs anytime during operational hours when
various classes (e.g., music class), work-related tasks (e.g., recycling) and leisure activities (e.g., card games) occurred.
Staff feedback was provided after observations with the staff on shift, and all feedback occurred after staff had
received initial training. Specifically, feedback took place in a quiet corner of the homes or programs or in a small staff
office. Initial training sessions were approximately 15 min. All observations lasted 15 min. Feedback sessions were
approximately 5 min, depending upon how much feedback was required.
2.2 | Response measurement, interobserver agreement, and data analysis
We created four competency checklists (described below in detail) that were used to both collect data and provide
staff with feedback on their performance on each of the four practices. Trained graduate and undergraduate students
collected data on staff implementation of each practice.
2.2.1 | Positive interactions
We collected data on positive interactions in 16 homes and 3 day programs. Observers used a checklist (see Appen-
dix A in Supporting Information S1) to collect data on a number of different positive interactions delivered by staff
to each consumer present in the common areas during 5-min intervals in the 15-min observation. We trained staff
to provide positive interactions at least once every 5 min to consumers in their vicinity, which is why we used 5-min.
Prior to observations, observers determined which staff to observe (based on which staff was providing supervision
in the common areas of the homes or programs) and which of the four consumers (maximum) present in the common
areas of the home or program to include in the observation. If a home or program had more than four consumers
present in the common areas, observers picked the four consumers they saw first to include in the observation.
During observations, data collectors scored whether the target staff provided any of the six types of positive inter-
actions (i.e., give a compliment, converse with consumer, greet consumer, provide appropriate physical interaction,
provide expression of care, and provide praise) to each target consumer present during each 5-min interval. However,
the consumer had to be present for at least half of the interval (2.5 min) for the interval to count. Data collectors only
scored positive interactions if they were delivered with a pleasant facial expression (i.e., staff were not frowning or
grimacing).
The main dependent variable was the percentage of 5-min intervals of overall positive interactions for each
observation. This was calculated by dividing the number of intervals for all target consumers present in which the
target staff provided a positive interaction by the total number of intervals across consumers present. We also
analyzed the mean percentage of intervals of overall positive interactions for applicable individual staff across phases
(baseline and BST + OJF). That is, for staff for whom we have both baseline and posttraining data, we conducted a
pre–post comparison of their baseline and BST + OJF performance regarding their percentage of intervals of overall
positive interactions to determine the effects of our training at the individual level.
A second independent observer collected data during at least 23% of observations in homes and programs across
all phases to determine interobserver agreement (IOA). We used an interval-by-interval agreement method for each
KAMANA et al. 5
type of interaction. We scored an agreement if both observers agreed whether a particular type of positive interac-
tion occurred in an interval. We calculated IOA by dividing the number of agreement intervals for each interaction
by the total number of intervals and multiplying by 100%. Mean IOA for positive interactions for all homes and
programs was 96% (range, 90%–99%).
2.2.2 | Effective instructions
We collected data on effective instructions in 15 homes and by 3 day programs. Observers used a checklist (see
Appendix B in Supporting Information S1) to collect data on each instruction delivered by a staff and whether that
instruction was delivered with a pleasant tone and facial expression, phrased as a “do” request, and included a tell/
prompt instruction sequence (i.e., if the consumer did not comply, staff provided some form of prompt for compli-
ance before delivering a new instruction) during observations. Prior to the observation, data collectors determined
which target staff to observe (based on which staff was providing supervision in the common areas of the homes
or programs). During observations, we observed staff providing instructions to any number of consumers in the
common areas of the programs. Data collectors scored an instruction by writing down each instruction delivered by
staff. An instruction was defined as staff requiring a specific behavior from the consumers by delivering a directive
or command to consumers (e.g., “Put your jacket on.”). A new instruction was scored only when the staff specified a
different task or behavior to be completed. Therefore, rephrasing an instruction, which specified the same behavior
or task was not considered a new instruction.
The main dependent variable was the percentage of correct instructions, which was defined as an instruction
delivered using a pleasant voice tone and facial expression, phrased as “do” requests, and delivered as a tell/prompt
instruction. The percentage of correct instructions was calculated by dividing the number of correct instructions
by the total number of instructions delivered by the staff in each 15-min observation. We also analyzed the mean
percentage of correct instructions for applicable individual staff across phases (baseline and BST + OJF). That is, for
staff for whom we had baseline and post-training data, we conducted a pre–post comparison of their baseline and
BST + OJF performance regarding their percentage of correct instructions to determine the effects of our training at
the individual level.
A second independent observer collected data during at least 25% of observations in homes and programs
across all phases to determine IOA. We used two calculations to determine IOA. First, a total IOA calculation
method was used to determine observers' agreement on the number of instructions provided by staff. Second, an
instruction-by-instruction method for each of the three elements of an effective instruction was used to determine
observers' agreement on whether each instruction met the criterion for that element. Only instructions both observ-
ers agreed to have occurred were included in this IOA calculation method. Mean IOA for effective instructions for all
homes and programs was 93% (range, 84%–100%).
2.2.3 | Responding to problem behavior
We collected data on responding to problem behavior in 14 homes and by 3 day programs. Observers used a check-
list (see Appendix C in Supporting Information S1) to collect data on whether staff responded correctly to minor
disruptive behavior and severe problem behavior during 3-min intervals in the 15-min observation. Minor disruptive
behavior included inappropriate verbal behavior (e.g., screaming, teasing, arguing, and complaining), as well as any
other nonharmful problem behavior (i.e., behavior that may disrupt the environment but could not result in harm to
self, others, or property). Severe problem behavior included any behavior that could result in harm to self, others, or
property (e.g., physical aggression, self-injurious behavior, and property destruction). Prior to observations, observ-
ers determined which staff to observe (based on which staff was providing supervision in the common areas of the
6 KAMANA et al.
homes or programs) and which four consumers (maximum) present in the common areas of the home or program to
include in the observation. If a home or program had more than four consumers present, then observers picked the
first four consumers they saw to include in the observation. The consumer could be present for any duration of the
interval for the interval to count. Correct responses to minor disruptive behavior were scored if the target staff with-
held commenting on minor disruptive behavior throughout the interval. However, a correct response was scored if
staff provided choices to the consumer (e.g., “Would you like to keep reading that book, or would you like to do this
puzzle?”), redirected consumers to the ongoing activity (e.g., “Wow, those pictures are cool in your book.”), continued
with a demand (e.g., “Let's finish setting the table together”), or engaged in other interactions with the consumer that
did not involve talking about or commenting on the disruptive behavior. Correct responses to severe problem behavior
were scored if the target staff withheld commenting on severe problem behavior throughout the interval and with-
held all attention and items or activities until at least 10 s without the occurrence of severe problem behavior. The
only exception was when staff needed to intervene for safety (e.g., response blocking); however, when they did so, it
was only scored as correct if they did not make eye contact or say anything to the consumer.
The main dependent variable was the percentage of 3-min intervals of overall staff correct responses to problem
behavior (both minor disruptive behavior and severe problem behavior). This was calculated by dividing the number
of intervals for all target consumers in which staff responded correctly to all problem behavior that occurred (both
minor disruptive behavior and severe problem behavior) by the total number of intervals in which problem behavior
occurred. We also analyzed the mean percentage of intervals of staff overall correct responses to problem behavior
for applicable individual staff across baseline and BST + OJF phases. That is, staff for whom we have baseline and
posttraining data, we conducted a pre–post comparison of their baseline and BST + OJF performance regarding their
percentage of overall correct responses to problem behavior to determine the effects of our training at the individual
level.
A second independent observer collected data during at least 30% of observations in homes and programs across
all phases to determine IOA. We used an interval-by-interval agreement method for each type of staff response (i.e.,
correct responses to minor disruptive behavior and correct responses to severe problem behavior). Mean IOA for
responding to problem behavior across all homes and programs was 95% (range, 89%–100%).
2.2.4 | Consumer engagement
We collected data on consumer engagement in 15 homes and by 3 day programs. Observers used a checklist (see
Appendix D in Supporting Information S1) to collect data on whether each consumer was appropriately engaged with
an item or in an activity at any point during 3-min intervals in the 15-min observation. Prior to observations, observ-
ers determined which consumers to observe. Consumers were included in the observation (up to four maximum) if
they were present in the common areas at the beginning of the observation period. If a home or program had more
than four consumers present, observers picked which four to include in the observation. However, for the interval to
count, the consumer had to be present for more than half of the interval (1.5 min). Consumer activity engagement was
defined as the consumer attending to, looking at, or manipulating an item in the way it was intended. This included
looking at the TV, swinging on a swing in the yard, or turning the pages while looking at a magazine. This did not
include engaging in problem behavior such as repetitive behavior while holding an item (e.g., flapping their hands
while holding a magazine).
The main dependent variable was the percentage of 3-min intervals of consumer activity engagement, which was
calculated by dividing the number of intervals for all target consumers present in which consumers were engaged by
the total number of intervals across consumers present. A second independent observer collected data during at least
22% of observations in homes and programs across all phases to determine IOA. We used an interval-by-interval
agreement method in which an agreement was scored if both observers agreed whether a consumer was engaged in
an interval. Mean IOA for consumer engagement across all homes and programs was 95% (range, 91%–99%).
KAMANA et al. 7
T A B L E 1 Summary of training and on-the-job observation procedures in the homes/programs by each of the
four practices.
Responding to Consumer
Positive interactions Effective instructions problem behavior engagement
# homes/programs 16 homes; 3 day 15 homes; 3 day 14 homes; 3 day 15 homes; 3 day
programs programs programs programs
Skills trained Provide positive Deliver instruction Refrain from Provide consumers
interaction to each w/pleasant facial commenting with choices
target consumer expression; on minor and to engage
in common area at present “do” severe problem with items and
least once every request; use behavior; could activities when
5 min prompts when provide choice, not engaged
needed redirect, etc.,
following minor
problem behavior;
should withhold
attention/items
for at least 10 s
following severe
problem behavior
Session information 15-min observations; 15-min observations 15-min observations; 15-min observations;
5-min intervals 3-min intervals 3-min intervals
Consumer Any target consumers Staff could deliver Any target consumers Any target
participation (max, four) in instructions to (max, four) in consumers (max,
common area of any number of common area of four) in common
home/program consumers in home/program area of home/
common area of program
home/program
Inclusion/exclusion Consumer(s) had to NA: Data Consumer(s) could be Consumer(s) had to
criteria of intervals be present for determined by present for any be present for
for data analysis at least half of # of instructions duration of the more than half
interval (2.5 min) delivered by target 3-min interval for of the interval
for interval to staff the interval to (1.5 min) for it to
count count count
Main dependent Percentage of Percentage of correct Percentage of 3-min Percentage of
variable 5-min intervals instructions intervals with staff 3-min intervals
with positive delivered by staff correct responses of consumer
interactions to problem engagement with
behavior items/activities
2.3 | General procedures
Table 1 depicts a summary of procedures for each practice. For each practice, we conducted (a) baseline observa-
tions, (b) initial training (BST), and (c) observations with OJF. Baseline, initial training, and observations with OJF
were conducted separately for each practice in the following order: provide positive interactions, provide effective
instructions, provide correct responses to problem behavior, and promote consumer engagement. Thus, a new base-
line, training, and observations with OJF for a particular practice did not begin until all sessions were completed for
a previous practice. This was done due to the large number of homes and programs in which the intervention was
implemented. Additionally, the consultants were being trained to implement the training, observations, and feedback
in a systematic fashion.
We used AB designs in which we collected repeated measures across baseline and BST + OJF phases across
homes and programs, resulting in approximately 70 AB designs. As suggested by Lanovaz et al. (2018), AB designs are
8 KAMANA et al.
likely sufficient in practical instances in which withdrawals or reversals are not possible. Furthermore, it is important
to note that within the context of our evaluation, several naturalistic multiple baselines across homes/programs and
practices occurred. However, we could not use a systematic multiple baseline design because we were unable to
stagger our training for each practice because the same staff often worked across different homes and programs.
2.3.1 | Baseline
During baseline for each practice, consultants and data collectors conducted 15-min observations at the home or
program using the relevant checklist (Appendices A–D in Supporting Information S1). Prior to baseline observations,
the consultant informed the staff that they were going to conduct a 15-min observation and informed the staff to
continue doing their work as they usually would. Following baseline observations, the consultant thanked the staff
but provided no programmed consequences for their behavior during the observation.
2.3.2 | BST and OJF
Once baseline observations were completed for a practice, the consultant began training staff in the home or program
on that practice using BST. After BST was completed with a staff on a practice, the consultant began conducting post-
training observations, in which the consultant observed staff on-the-job and provided feedback on their performance
using an on-the-job feedback protocol specific to each practice (see the second [back] page of each checklist in
Appendices A–D for the on-the-job feedback protocol in Supporting Information S1).
Initial training involved using BST to train each practice, which included training individual staff on the implemen-
tation of the practice using instructions, modeling, role play, and feedback. Specifically, for each practice, staff were
instructed on the practice by the consultant who reviewed a PowerPoint presentation specific to that practice. The
presentation included (a) a brief description of the practice, (b) how to implement the practice (with video examples),
(c) when to implement the practice, and (d) why it is important to implement the practice. Each PowerPoint presenta-
tion was scripted to increase the likelihood of uniform training across consultants. After reviewing the presentation,
the consultant modeled the practice and then had the staff rehearse the practice with the staff playing themselves
and the consultant playing the role of a consumer. The consultant then provided positive feedback for correct imple-
mentation of the practice and corrective feedback for incorrect implementation of the practice during rehearsal until
the staff displayed correct implementation of the practice.
On-the-job observations and feedback sessions were conducted after the staff had experienced BST (initial
training) on a practice. These observations were conducted like baseline observations, except that after the obser-
vation, the consultant immediately provided target staff (one or more, depending on the practice) on-the-job feed-
back in a quiet area of the home using an on-the-job feedback protocol created for that practice. Across practices,
on-the-job feedback included the same steps but were tailored to feedback for a specific practice. First, the consult-
ants reviewed the checklist outcomes for the practice with the target staff (ensuring to show the checklist to the staff
as they were reviewing). Next, the consultant provided the staff with behavior-specific praise for correct implementa-
tion of the practice and corrective feedback for incorrect implementation of the practice. Consultants were trained to
use a supportive voice tone and facial expression (e.g., refrain from reprimanding the staff or frowning at the staff) to
provide corrective feedback. If applicable, the consultant also described how the staff could improve implementation
of the practice in the future and then implemented BST. That is, the consultant modeled the practice, had the staff
role play the practice with the consultant or with consumers, who were present (if feasible), and provided feedback
until the staff correctly implemented the practice. Finally, consultants answered any questions and clarified any
procedures based on staff inquiries. The on-the-job feedback protocol for each practice was on the back of the data
sheet for that practice and was tailored to be a checklist for the consultant to follow to ensure they implemented the
feedback correctly. Below is additional information on the initial training for each practice.
KAMANA et al. 9
Because providing positive interactions was the first practice we trained, the PowerPoint for this practice also
included an introduction to problem behavior, common reasons why individuals might engage in problem behav-
ior, and a list of the four practices (see Appendix E, slides 1–3 in Supporting Information S1). After the consultant
reviewed this information, they began the PowerPoint training on positive interactions (named “Provide Positive
Interactions” in the PowerPoint slides [see Appendix E, slides 4–12 in Supporting Information S1]), which included
training staff on different types of positive interactions (e.g., compliments, conversation, praise) and to provide those
interactions at least once every 5 min to consumers present. Furthermore, the presentation included a review of why
providing consumers with frequent positive interactions is important (e.g., to build rapport, to decrease likelihood of
problem behavior to gain access to attention).
Training on effective instructions began with reviewing the PowerPoint for this practice (named “Provide Effective
Instructions” in the PowerPoint slides [see Appendix E, slides 15–18 in Supporting Information S1]). This presentation
included training on how to deliver effective instructions including (a) using a pleasant voice tone and facial expres-
sion, (b) presenting instructions using simple and clear demands (e.g., breaking down demands into smaller steps), (c)
using “do” rather “don't” requests, and (d) using two-step prompting (i.e., tell and show), as well as providing help when
necessary. The presentation also included training staff to provide effective instructions during any instructional,
task, or chore context throughout the day. Finally, the presentation included a review of why provid ing consumers
with effective instructions is important (e.g., to decrease task difficulty, increase compliance, and decrease likelihood
problem behavior in instructional contexts).
2.3.5 | Correct responses to problem behavior
Prior to training this practice (during the baseline data-collection period), we reviewed all behavior plans of consum-
ers in each program to ensure consultants assigned to each program were aware of any procedures that may be
contradictory to the content trained in this practice (e.g., behavior plans directing staff to deliver attention following
severe problem behavior). Behavior plan review outcomes showed that six programs had at least one behavior plan,
which contained procedures that involved the delivery of attention for problem behavior and were contradictory
to the training. Therefore, in consultation with administration and home or program clinical teams, the consultants
worked to change the behavior plans to be in line with our training on how to respond to problem behavior (i.e.,
no longer provide attention immediately following severe problem behavior). This change to these behavior plans
occurred prior to our training phase but during baseline data collection. Therefore, for these 6 homes, these changes
may have influenced our outcomes.
Training on correct responses to problem behavior began with reviewing the PowerPoint for this practice (named
“Good practices following problem behavior” in the PowerPoint slides [see Appendix E, slides 19–21 in Supporting
Information S1]). This presentation included training staff on what constituted minor disruptive behavior and severe
problem behavior. In addition, it included training on refraining from commenting on minor disruptive behavior but
included explanation that other interactions (e.g., providing choices) following these behaviors were acceptable.
Furthermore, it included training on refraining from commenting on severe problem behavior at any point in time and
withholding attention and access to preferred items and activities for at least 10 s following occurrence of severe
problem behavior. However, the staff were informed that if physical intervention was necessary for consumer safety
(e.g., blocking), they were to implement it with the minimal attention (e.g., no eye contact and no talking to the
consumer).
10 KAMANA et al.
During our baseline observations in many of the homes for this practice, we observed a lack of items and activities in
the common areas with which consumers could engage. Furthermore, we observed that the items and activities that
were present may not have been preferred by consumers or were not in useable condition (e.g., electronics missing
batteries and broken items). Therefore, after baseline and before our intervention phase, the consultants conducted
formal staff and consumer interviews to determine items and activities that would be preferred by consumers in the
homes using the Reinforcer Inventory for Adults. After these interviews were conducted, the consultants worked
with administration to purchase new items and activities for each home based on the list of potential preferred items
and activities. All new items and activities were stored in common areas for consumer access. Therefore, part of the
intervention for these homes included environmental enrichment (EE; Horner, 1980) in which access to preferred
items activities was made available in the common areas of the home. Additional items and activities were not
purchased for the 3 day programs because observations suggested there was a large variety of items and activities
in good condition available.
Training and on-the-job observations for this phase began once new items and activities had been purchased
for relevant homes. Initial training included reviewing the PowerPoint on promoting consumer engagement (named
“Provide access to preferred items/activities” in the PowerPoint slides [see Appendix E, slides 12–14] in Supporting
Information S1). This presentation included training the staff to provide consumers with choices of preferred things
to do throughout the day, particularly during leisure periods or when staff were occupied with other tasks within
the home and programs. The presentation also included a discussion of why providing consumers with choices of
preferred items and activities is important (e.g., to promote rapport and decrease the likelihood of problem behavior
maintained by positive reinforcement).
3 | RESULTS
3.1 | Staff practices (all homes and programs)
Representative data depicting the major outcomes for each practice across homes and programs are depicted
in Figures 1–4 (refer to Supporting Information S1: Appendix F for all graphs for all homes and programs across
practices).
Figure 1 depicts representative data for the percentage of intervals of overall positive interactions for 19 homes and
programs in which we trained this practice. The left panel depicts representative data for seven homes and programs
that showed relatively low but variable levels of positive interactions in baseline but higher and in some cases, more
stable levels in the BST + OJF phase. The right panel depicts representative data for six homes and programs that
showed an increase in level, stability, or both from baseline to the BST + OJF phase. The bottom panel depicts
repre sentative data for the remaining six homes and programs that showed no clear difference between baseline and
BST + OJF; however, responding either occurred at very high levels in baseline or relatively high but variable levels
in baseline.
Figure 2 depicts representative data for the percentage of correct staff instructions for 18 homes and programs
in which we trained this practice. The numbers on top of the data points depict the number of staff instructions
KAMANA et al. 11
F I G U R E 1 Depicts representative data for the percentage intervals of overall positive interactions for 19
homes and programs. The left panel (D-1) depicts representative data for seven homes and programs that showed
relatively low levels of positive interactions in baseline and increased levels of positive interactions in BST + OJF;
the right panel (P-1) depicts representative data for six homes and programs that had an increase in level, stability,
or both from baseline to BST + OJF; the bottom panel (F-2) depicts representative data for six homes that showed
no clear differences from baseline to BST + OJF. BST, behavior skills training; OJF, on-the-job feedback.
provided in that observation. The asterisks on the bottom of each graph depict observations in which the staff did
not deliver any instructions. The top panel depicts representative data for nine homes and programs that showed
low, variable, or a consistently decreasing percentage of correct staff instructions in baseline but higher and consist-
ently more stable levels in BST + OJF. The bottom panel depicts representative data for nine homes and programs
that showed relatively high levels or increasing levels of correct staff instructions in baseline that were maintained
at similar high levels in BST + OJF. In these homes, there was no clear difference between baseline and BST + OJF,
mostly because baseline levels were already high.
3.1.3 | Responding to problem behavior
Figure 3 depicts representative data for the percentage of intervals of correct responses to problem behavior for 17
homes and programs in which we trained this practice. The top panel depicts representative data for 11 homes and
programs that showed low or decreasing percent intervals of staff correct responses to problem behavior during
baseline and high and stable percent intervals of staff correct responses to problem behavior during BST + OJF. The
bottom panel depicts representative data for six homes and programs that showed similar levels of staff correct
responses to problem behavior across baseline and BST + OJF. In these homes, responding correctly was either high
but variable across both phases, or high and variable in baseline to high and stable in BST + OJF.
12 KAMANA et al.
F I G U R E 2 Depicts representative data for the percentages of correct instructions for 18 homes and programs.
The numbers on each data point depict the number of instructions in the observation. The asterisks at the
bottom of some graphs depict sessions in which the staff did not provide instructions. The top panel (L-4) depicts
representative data for nine homes and programs that showed low or consistently decreasing levels of correct
instructions in baseline and higher, more stable levels of correct instructions in BST + OJF. The bottom panel (T-1)
depicts representative data for nine homes that showed high or increasing levels of correct instructions in baseline
and maintained high levels in BST + OJF. BST, behavior skills training; OJF, on-the-job feedback.
Figure 4 depicts representative data for consumer engagement for 18 homes and programs in which we imple-
mented this practice. The top left panel depicts representative data for 12 homes and programs that showed
increases in percent intervals of consumer engagement from baseline to EE + BST + OJF. The top right panel depicts
representative data for six homes and programs that did not show robust differences in consumer engagement
across baseline and EE + BST + OJF. In these homes, there were increases in consumer engagement from baseline
to EE + BST + OJF; however, most of these programs showed high or increasing levels of consumer engagement
in baseline, and those levels either stabilized at high levels following the intervention or were maintained at high
KAMANA et al. 13
F I G U R E 3 Depicts representative data for the percent intervals of correct responses to problem behavior for
17 homes and programs. Sessions missing data points are those in which the staff did not have the opportunity
to respond correctly or incorrectly to problem behavior. The top panel (F-17) depicts representative data for 11
homes and programs whose correct responses to problem behavior are low or decreasing in baseline and high and
stable in BST + OJF. The bottom panel (F-6) depicts representative data for six homes and programs whose correct
responses to problem behavior are low or decreasing in baseline and high and stable in BST + OJF. BST, behavior
skills training; OJF, on-the-job feedback.
levels following the intervention. The bottom panel depicts the only program (F6) in which consumer engagement
decreased from baseline to EE + BST + OJF.
3.2 | Individual staff data for applicable staff
We also conducted additional analyses for applicable individual staff for the first three practices. That is, for positive
interactions, effective instructions, and correct responses to problem behavior, we analyzed staff mean performance
in baseline and BST + OJF phases for staff who were involved in baseline and post-training phases of our evalua-
tion. We did not conduct this analysis for the consumer engagement practice because more than one staff could be
observed in a single observation. These outcomes are depicted in Figures 5–7.
14 KAMANA et al.
F I G U R E 4 Depicts representative data for consumer engagement for 18 homes and programs. The left panel
(F-3) depicts representative data for 12 homes and programs whose consumer engagement increased from baseline
to EE + BST + OJF. The right panel (G1) depicts representative data for five homes and programs whose consumer
engagement data did not show a robust increase from baseline to EE + BST + OJF. The bottom panel (F-6) depicts
representative data for the only program whose consumer engagement decreased from baseline to EE + BST + OJF.
BST, behavior skills training; EE, environmental enrichment; OJF, on-the-job feedback.
Figure 5 depicts two panels of pre–post mean percentage intervals of overall positive interactions for 36
staff from 19 homes and programs. Overall, 25 staff showed an increase in positive interactions from baseline to
BST + OJF. The mean increase for these 25 staff was 46% (range, 8%–100%). Furthermore, five staff (Alexandra in
the top panel and Libby, Rogelio, Bruce, and Linda in the bottom panel) did not show an increase or decrease positive
interactions from baseline to BST + OJF; however, for all five staff, positive interactions were already occurring at
high levels (range, 83%–100%). Finally, six staff (Adelaide in the top panel and Jane, Mark, Brad, Poppy, and Breanne
in the bottom panel) showed a decrease in positive interactions from baseline to BST + OJF. The mean decrease was
14% (range, 5%–29%). For these six staff, decreases were minimal, and positive interactions were already occurring
at high levels in baseline.
Figure 6 depicts two panels for pre–post mean percentages of correct instructions for 30 staff from 16 homes
and programs. Overall, 23 staff showed an increase in correct instructions from baseline to BST + OJF. The mean
increase of correct instructions for these 23 staff was 40% (range, 10%–100%). Furthermore, five staff (Courtney
in the top panel and Debra, Cameron, Lani, and Sid in the bottom panel) did not show an increase or decrease the
mean percentage of correct instructions from baseline to BST + OJF. It is important to note that for four of these staff
(Courtney in top panel and Debra, Cameron, and Lani in the bottom panel), correct instructions were already occur-
ring at 100% in baseline; however, for one staff (Sid in the bottom panel), both baseline and BST + OJF levels were at
50%. Thus, the intervention was not effective for increasing correct instructions for Sid. Finally, two staff (Rogelio and
KAMANA et al. 15
F I G U R E 5 Depicts the mean percentage intervals of overall positive interactions for individual staff in baseline
and BST + OJF. The top and bottom panels depict data for 36 staff in 19 homes and programs. The upside triangles
depict increases, and the downside triangles depict decreases from baseline to BST + OJF. BST, behavior skills
training; OJF, on-the-job feedback.
Christa in the bottom panel) showed a decrease in the percentage of correct instructions from baseline to BST + OJF.
The mean decrease was 16% (range, 9%–23%). For these two staff, decreases were minimal and correct instructions
were already occurring at high levels in baseline.
Figure 7 depicts two panels for pre–post mean percentage intervals of correct responses to problem behavior for
26 staff from 15 homes and programs. Overall, 20 staff showed an increase in correct responses to problem behavior
from baseline to BST + OJF. The mean percent increase for these 20 staff was 61% (range, 10%–100%). Further-
more, five staff (Jesse in the top panel and Brianne, Cameron, Hailey, and Jaxon in the bottom panel) did not show
an increase or decrease in the mean percentage intervals of correct responses to problem behavior from baseline
to BST + OJF. However, for four of these staff (Breanne, Cameron, Hailey, and Jaxon in the bottom panel), correct
responses to problem behavior were already occurring at high levels in baseline. For one of these staff (Jesse in the
16 KAMANA et al.
F I G U R E 6 Depicts the mean percentage intervals of effective instructions for individual staff in baseline and
BST + OJF. The top and bottom panels depict data for 30 staff in 16 homes and programs. The upside triangles
depict increases, and the downside triangles depict decreases from baseline to BST + OJF. BST, behavior skills
training; OJF, on-the-job feedback.
top panel), correct responses occurred at moderate levels (i.e., 67%) in both baseline and BST + OJF. Finally, one staff
(Amy in the bottom panel) showed a decrease in the percentage of intervals of correct responses to problem behavior
from baseline to BST + OJF. However, this decrease was minimal and correct responses to problem behavior were
already occurring at high levels in baseline.
4 | DISCUSSION
The purpose of the current study was to use BST and OJF to provide large-scale staff training on four empirically
derived practices (e.g., Parsons et al., 2004; Realon et al., 2002). Like outcomes of previous research (e.g., Harchik &
KAMANA et al. 17
F I G U R E 7 Depicts the mean percentage intervals of correct responses to problem behavior for individual
staff in baseline and BST + OJF. The top and bottom panels depict data for 26 staff in 15 homes and programs. The
upside triangles depict increases, and the downside triangles depict decreases from baseline to BST + OJF. BST,
behavior skills training; OJF, on-the-job feedback.
Campbell, 1998; Reid et al., 2015; van OOrsouw et al., 2009), we were able to change staff behavior by using perfor-
mance management procedures in a community setting, which provides services to adults with IDD. We extended
staff training literature in programs for adults with IDD by training on multiple skills; conducting observations in
multiple community-based programs during days and evenings, weekdays, and weekends; and in both structured
and unstructured contexts in group homes and day programs. Furthermore, we were able to provide this training on
a large scale by training at least 150 staff who each worked with at least four consumers at a time, and all phases of
the training were implemented on the job. In doing this, our study attempted to fill some of the research-to-practice
gaps which exist regarding lack of the use of best-practice staff training methods (i.e., BST; Parsons et al., 2004) in
programs for adults with IDD (Fixsen et al., 2010). Additionally, our study also addressed the need to use best-practice
18 KAMANA et al.
training methods to train complex or multiple skills in these programs (Erath et al., 2019). Overall, this study adds to
the staff training literature in community programs for adults with IDD that often have multiple barriers to effective
staff training (e.g., high staff-turnover rates; traveling across multiple programs to conduct training on the job).
Most of our baseline data showed that staff in most homes and programs were not implementing interactions
and procedures that have been shown to be effective aspects of best practice in service provision for adults with
IDD (i.e., active support and treatment; Parsons et al., 2004; Reid et al., 2001). These data are in line with both older
and contemporary research that evaluated the occurrence of important behaviors such as staff positive interactions,
consumer engagement, and staff provision of choices to consumers (e.g., Chan & Yau, 2002; Parsons et al., 2004; Repp
et al., 1981). Thus, our baseline data underscores the continued need for research focusing on important staff behav-
ior change in community programs for adults with IDD. Overall, the outcomes of our study showed that BST + OJF
were effective for behavior change across the different practices and across homes and programs. Specifically, our
overall outcomes suggest that most homes and programs increased positive interactions, increased effective instruc-
tions, and increased appropriate responses to problem behavior. Additionally, consumers from different homes and
programs increased appropriate engagement with preferred items and activities. Furthermore, our pre–post analyses
for applicable staff showed that individual staff increased their implementation of three practices (i.e., provide posi-
tive interactions, provide effective instructions, and provide correct responses to problem behavior). Specifically, we
observed increases in positive interactions for 25 of 36 staff, increases in effective instructions for 23 of 30 staff, and
increases in correct responses to problem behavior for 20 of 26 staff.
Our study had some methodological and analytical limitations that are worth noting. First, the staff were some-
times different across baseline and BST + OJF phases and across the different practices. Although the presence of
different staff across phases is a limitation of the study, it was a variable we could not control given (a) the fluctuating
staff schedules and turnover rates, and (b) the major purpose of the evaluation, which was to train important skills
to staff who work with adults with IDD. However, we conducted pre–post analyses of staff who participated in
both baseline and BST + OJF phases to circumvent this limitation of our study. Future research might be conducted
with individual staff as the unit of measurement as compared to our approach, which used the homes and programs
as the major unit of measurement. Second, we typically conducted training with only one staff member at a time
because all phases of the study were conducted with the staff on shift. This individualized training may have had
some benefits; however, it resulted in a large time commitment for the consultants, which underscores the need to
determine ways to make staff training more efficient (Parsons et al., 2013). Thus, future research might look at the
efficacy of conducting the initial training in small or large group formats to increase the efficiency of the training.
Furthermore, it may be beneficial to use a pyramidal training model (Erath et al., 2019; Harchik & Campbell, 1998;
Page et al., 1982; Parsons et al., 2013; Shore et al., 1995) to train program supervisors, who would then focus their
training on their assigned program(s). The pyramidal model of training may also allow for the monitoring of the
maintenance of staff skills. That is, given that our study was conducted with various staff within and across phases
and in various homes and programs during a relatively short period of time, we were not able to determine whether
the effects of our intervention would maintain over time in the absence of intervention. Along these lines, observer
reactivity may have influenced some of our outcomes. It's possible that staff implementation of the practices only
occurred during the intervention phase when the consultants were conducting observations, particularly given what
we know about the influence of reactivity (Brackett et al., 2007; Kazdin, 1979). Future research might use technology
(audio–video technology; DiGennaro Reed & Reed, 2013) to conduct unobtrusive observations of staff to determine
whether they continue to engage in the practices.
There are also other variables that may have influenced the outcomes of our study. For example, for staff
correct responses to problem behavior, consultants observed during baseline that some staff were implementing
procedures in behavior intervention plans, which were contradictory to our training (i.e., providing attention or
preferred stimuli to consumers contingent on the occurrence of problem behavior). We made changes to these
behavior plans such that they were in line with our training, but these changes were made during baseline; thus,
outcomes may have been influenced by the changes in the behavior plans in addition to our training for some
KAMANA et al. 19
homes. Although it was not ideal to make these changes to behavior plans at this time, these observations were
made during baseline and the team of consultants had a clinical obligation to change behavior plans to reflect
more applicable procedures for problem behavior. Furthermore, following baseline for promoting consumer
engagement, we made changes in the environment by working with the company to purchase items and activities,
which were reported to be preferred for consumers in the homes, such that there were more items and activities
with which the consumers could engage. Thus, this variable alone may have influenced consumer engagement
levels.
Furthermore, we did not collect consultant procedural integrity data. Although the consultants followed a
specific on-the-job feedback protocol to increase the likelihood that they provided feedback to staff in a consistent
and systematic manner, we did not collect integrity data regarding the consultant's implementation of OJF. Proce-
dural integrity may even be more warranted if a pyramidal training model is used given that individuals who are
not experts in the field would be conducting the training and providing staff with feedback. Furthermore, although
various administrators and staff commented on positive aspects of the practices, staff training, and on-the-job staff
feedback and support, we did not conduct formal social validity assessments of the practices and training. However,
some aspects that support the acceptance of the practices and training are that (a) the administration chose to begin
training the four practices to all newly hired staff as part of initial staff training; (b) the administration asked us to
create a modified, one-page, supervisor-friendly version of the observation sheet such that home and program super-
visors could observe and provide feedback on all four practices in a single observation; and (c) the company adopted
a crisis prevention and management system that was more in line with our training and is based on behavior analytic
approaches to preventing and managing problem behavior in persons with IDD.
An important direction for future research may be to include data collection of consumer behavior across all
practices. Researchers have suggested the importance of collecting data not only on staff behavior to determine
whether they acquired important skills but also to measure consumer behavior as an ancillary measure on the utility
of staff behavior change (Harchik & Campbell, 1998). Future research should be conducted to determine the associ-
ation between the implementation of the four staff practices over an extended period and analyze data for problem
behavior and appropriate behavior to assess whether changes in staff behavior are associated with concomitant
positive outcomes for consumers. As suggested by previous research, behaviors other than problem behavior could
be measured, which suggest positive effects of these practices such as measuring consumer indices of happiness
(e.g., affect; Green & Reid, 1996).
ACKNOWLE DG ME NTS
This research is based on the dissertation submitted by the first author, under the supervision of the second author,
in partial fulfillment of the doctoral degree for the Department of Applied Behavioral Science at the University of
Kansas. We would like to thank Dr. Derek Reed, Dr. Pamela Neidert, and Dr. Jason Travers for their comments on a
previous version of this manuscript.
CO N FLI CT OF I NTE RE ST STATE M E N T

This study was conducted in affiliation with GoodLife Innovations, Inc. Claudia L. Dozier is a consultant for GoodLife
Innovations, Inc. Claudia L. Dozier is on the board of directors of GoodLife Innovations, Inc. and receives no compen-
sation as a member of the board.
DATA AVAI LABI LI TY STATE M E N T

The data that support the findings of this study are available from the corresponding author upon reasonable request.
O RC ID
Bertilde U. Kamana https://orcid.org/0000-0003-2941-9342
Claudia L. Dozier https://orcid.org/0000-0001-6631-0472
20 KAMANA et al.
R EF ERE NCE S
Ala’i-Rosales, S., Cihon, J. H., Currier, T. D. R., Ferguson, J. L., Leaf, J. B., Leaf, R., McEachin, J., & Weinkauf, S. M. (2018). The
big four: Functional assessment research informs preventative behavior analysis. Behavior Analysis in Practice, 12(1),
222–234. https://doi.org/10.1007/s40617-018-00291-9
Anderson, K. A., Dabelko-Schoeny, H. I., & Tarrant, S. D. (2012). A constellation of concerns. Home Health Care Management
& Practice, 24(3), 132–139. https://doi.org/10.1177/1084822311424595
Bottos, M., Feliciangeli, A., Sciuto, L., Gericke, C., & Vianello, A. (2001). Functional status of adults with cerebral palsy
and implications for treatment of children. Developmental Medicine and Child Neurology, 43(8), 516–528. https://doi.
org/10.1111/j.1469-8749.2001.tb00755.x
Bouras, N., & Jacobson, J. (2002). Mental health care for people with mental retardation: A global perspective. World Psychi-
atry, 1, 162–165.
Brackett, L., Reid, D. H., & Green, C. W. (2007). Effects of reactivity to observations on staff performance. Journal of Applied
Behavior Analysis, 40(1), 191–195. https://doi.org/10.1901/jaba.2007.112-05
Carr, E. G., Dunlap, G., Horner, R. H., Koegel, R. L., Turnbull, A. P., Sailor, W., Anderson, J. L., Albin, R. W., Koegel, L. K., & Fox, L.
(2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions, 4(1), 4–16.
https://doi.org/10.1177/109830070200400102
Carr, E. G., & Durand, V. M. (1985). Reducing behavior problems through functional communication training. Journal of Applied
Behavior Analysis, 18(2), 111–126. https://doi.org/10.1901/jaba.1985.18-111
Chan, J. S., & Yau, M. K. (2002). A study on the nature of interactions between direct-care staff and persons with devel-
opmental disabilities in institutional care. British Journal of Developmental Disabilities, 48(94), 39–51. https://doi.
org/10.1179/096979502799104274
Code of Federal Regulations, 42 C.F.R §483.440. (2011). Condition participation: Active treatment services §§ Section
483.440.
Collins, S., Higbee, T. S., Salzberg, C. L., & Carr, J. (2009). The effects of video modeling on staff implementation of a
problem-solving intervention with adults with developmental disabilities. Journal of Applied Behavior Analysis, 24(4),
849–854. https://doi.org/10.1901/jaba.2009.42-849
Colombo, R. A., Taylor, R. S., & Hammond, J. L. (2021). State of current training for severe problem behavior: A survey. Behav-
ior Analysis Practice, 14(1), 11–19. https://doi.org/10.1007/s40617-020-00424-z
Cooper, K. J., & Browder, D. M. (2001). Preparing staff to enhance active participation of adults with severe disabilities by
offering choice and prompting performance during a community purchasing activity. Research in Developmental Disabil-
ities, 22(1), 1–20. https://doi.org/10.1016/s0891-4222(00)00065-2
Cox, A. D., Dube, C., & Temple, B. (2015). The influence of staff training on challenging behaviour in individuals with intellec-
tual disability. Journal of Intellectual Disabilities, 19(1), 69–82. https://doi.org/10.1177/1744629514558075
DiGennaro Reed, F., Hirst, J., & Howard, V. (2013). Empirically supported staff selection, 105 training, and management
strategies. In D. D. Reed, F. D. DiGennaro Reed, & J. K. Luiselli (Eds.), Handbook of crisis intervention and developmental
disabilities (pp. 71–85). Springer.
DiGennaro Reed, F. D., & Reed, D. D. (2013). HomeLink support technologies at community living opportunities. Behavior
Analysis in Practice, 6(1), 80–81. https://doi.org/10.1007/bf03391794
Ebanks, M. E., & Fisher, W. W. (2003). Altering the timing of academic prompts to treat destructive behavior maintained by
escape. Journal of Applied Behavior Analysis, 36(3), 355–359. https://doi.org/10.1901/jaba.2003.36-355
Erath, T. G., DiGennaro Reed, F. D., Sundermeyer, H. W., Brand, D., Novak, M. D., Harbison, M. J., & Shears, R. (2019). Enhanc-
ing the training integrity of human service staff using pyramidal behavioral skills training. Journal of Applied Behavior
Analysis, 53(1), 449–464. https://doi.org/10.1002/jaba.608
Felce, D., & Emerson, E. (2001). Living with support in a home in the community: Predictors of behavioral development and
household and community activity. Mental Retardation and Developmental Disabilities Research Reviews, 7(2), 75–83.
https://doi.org/10.1002/mrdd.1011
Fixsen, D. L., Blasé, K. A., Duda, M. A., Naoom, S. F., & Van Dyke, M. (2010). Implementation of evidence-based treatments
for children and adolescents: Research findings and their implications for the future. In J. R. Weisz & A. E. Kazdin (Eds.),
Evidence-based psychotherapies for children and adolescents (pp. 435–450). Guilford Press.
Fleming, R., & Sulzer-Azaroff, B. (1992). Reciprocal peer management: Improving staff instruction in a vocational training
program. Journal of Applied Behavior Analysis, 25(3), 611–620. https://doi.org/10.1901/jaba.1992.25-611
Fulone, I., Barreto, J. O. M., Barberato-Filho, S., De Cássia Bergamaschi, C., Silva, M. T., & Lopes, L. C. (2021). Improving care
for deinstitutionalized people with mental disorders: Experiences of the use of knowledge translation tools. Frontiers in
Psychiatry, 12. https://doi.org/10.3389/fpsyt.2021.575708
Green, C. W., & Reid, D. H. (1996). Defining, validating, and increasing indices of happiness among people with profound
multiple disabilities. Journal of Applied Behavior Analysis, 29(1), 67–78. https://doi.org/10.1901/jaba.1996.29-67
Hagopian, L. P., Dozier, C. L., Rooker, G., & Jones, B. A. (2013). Assessing and treating severe problem behavior. In G. J.
Madden (Ed.), APA handbook of behavior analysis. American Psychological Association.
KAMANA et al. 21
Harchik, A. E., & Campbell, A. R. (1998). Supporting people with developmental disabilities in their homes in the community.
Journal of Organizational Behavior Management, 18(2– 3), 83–101. https://doi.org/10.1300/j075v18n02_05
Harchik, A. E., Sherman, J. A., Sheldon, J. B., & Strouse, M. C. (1992). Ongoing consultation as a method of improving perfor-
mance of staff members in a group home. Journal of Applied Behavior Analysis, 25(3), 599–610. https://doi.org/10.1901/
jaba.1992.25-599
Hewitt, A., & Larson, S. (2007). The direct support workforce in community supports to individuals with developmental disa-
bilities: Issues, implications, and promising practices. Mental Retardation and Developmental Disabilities Research Reviews,
13(2), 178–187. https://doi.org/10.1002/mrdd.20151
Horner, R. D. (1980). The effects of an environmental “enrichment” program on the behavior of institutionalized profoundly
retarded children. Journal of Applied Behavior Analysis, 13(3), 473–491. https://doi.org/10.1901/jaba.1980.13-473
Jones, E., Perry, J., Lowe, K., Felce, D., Toogood, S., Dunstan, F., Allen, D., & Pagler, J. (1999). Opportunity and the promotion of
activity among adults with severe intellectual disability living in community residences: The impact of training staff in active
support. Journal of Intellectual Disability Research, 43(3), 164–178. https://doi.org/10.1046/j.1365-2788.1999.00177.x
Kazdin, A. E. (1979). Unobtrusive measures in behavioral assessment. Journal of Applied Behavior Analysis, 12(4), 713–724.
https://doi.org/10.1901/jaba.1979.12-713
Lanovaz, M. J., Turgeon, S., Cardinal, P., & Wheatley, T. L. (2018). Using single-case designs in practical settings: Is
within-subject replication always necessary? Perspectives on Behavior Science, 42(1), 153–162. https://doi.org/10.1007/
s40614-018-0138-9
Lowe, K., Allen, D., Jones, E., Brophy, S., Moore, K., & James, W. (2007). Challenging behaviours: Prevalence and topographies.
Journal of Intellectual Disability Research, 51(8), 625–636. https://doi.org/10.1111/j.1365-2788.2006.00948.x
McLaughlin, D. M., & Carr, E. G. (2005). Quality of rapport as a setting event for problem behavior: Assessment and interven-
tion. Journal of Positive Behavior Interventions, 7(2), 68–91. https://doi.org/10.1177/10983007050070020401
Medicaid. (2019). Intermediate care facilities for individuals with intellectual disability. Retrieved from https://www.medicaid.
gov/medicaid/ltss/institutional/icfid/index.html
Page, T. J., Iwata, B. A., & Reid, D. H. (1982). Pyramidal training: A large-scale application with institutional staff. Journal of
Applied Behavior Analysis, 15(3), 335–351. https://doi.org/10.1901/jaba.1982.15-335
Parsons, M. B., Cash, V. B., & Reid, D. H. (1989). Improving residential treatment services: Implementation and norm-referenced
evaluation of a comprehensive management system. Journal of Applied Behavior Analysis, 22(2), 143–156. https://doi.
org/10.1901/jaba.1989.22-143
Parsons, M. B., & Reid, D. H. (1993). Evaluating and improving residential treatment during group leisure situ-
ations: A program replication and refinement. Research in Developmental Disabilities, 14(1), 67–85.
https://doi.org/10.1016/0891-4222(93)90006-6
Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2004). Improving day-treatment services for adults with severe disabilities: A
norm-referenced application of outcome management. Journal of Applied Behavior Analysis, 37(3), 365–377. https://doi.
org/10.1901/jaba.2004.37-365
Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis
in Practice, 5(2), 2–11. https://doi.org/10.1007/bf03391819
Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2013). Teaching practitioners to conduct behavioral skills training: A pyramidal
approach for training multiple human service staff. Behavior Analysis in Practice, 6(2), 4–16. https://doi.org/10.1007/
bf03391798
Realon, R. E., Bligen, R. A., La Force, A., Helsel, W. J., & Goldman, V. (2002). The effects of the positive environment program
(PEP) on the behaviors of adults with profound cognitive and physical disabilities. Behavioral Interventions, 17(1), 1–13.
https://doi.org/10.1002/bin.102
Reid, D. H., O’Kane, N. P., & Macurik, K. M. (2011). Staff training and management. In W. W. Fisher, C. C. Piazza, & H. S. Roane
(Eds.), Handbook of applied behavior analysis (pp. 281–294). Guilford Press.
Reid, D. H., Parsons, M. B., & Green, C. W. (2001). Evaluating the functional utility of congregate day treatment activities for
adults with severe disabilities. American Journal of Mental Retardation, 106(5), 460–469. https://doi.org/10.1352/0895
8017(2001)106<0460:ETFUOC>2.0.CO;2
Reid, D. H., Parsons, M. B., & Jensen, J. M. (2015). Maintaining staff performance following a training intervention: Suggestions
from a 30-year case example. Behavior Analysis in Practice, 10(1), 12–21. https://doi.org/10.1007/s40617-015-0101-0
Repp, A. C., Barton, L. E., & Brulle, A. R. (1981). Correspondence between effectiveness and staff use of instructions for severely
retarded persons. Applied Research in Mental Retardation, 2(3), 237–245. https://doi.org/10.1016/0270-3092(81)90017-5
Rotholz, D. A., Moseley, C. R., & Carlson, K. B. (2013). State policies and practices in behavior supports for persons with intel-
lectual and developmental disabilities in the United States: A national survey. Intellectual and Developmental Disabilities,
51(6), 433–445. https://doi.org/10.1352/1934-9556-51.6.433
Shapiro, M., & Kazemi, E. (2017). A review of training strategies to teach individuals implementation of behavioral interven-
tions. Journal of Organizational Behavior Management, 37(1), 32–62. https://doi.org/10.1080/01608061.2016.1267066
22 KAMANA et al.
Shore, B. A., Iwata, B. A., Vollmer, T. R., Lerman, D. C., & Zarcone, J. R. (1995). Pyramidal staff training in the extension of
treatment for severe behavior disorders. Journal of Applied Behavior Analysis, 28(3), 323–332. https://doi.org/10.1901/
jaba.1995.28-323
van Oorsouw, W. M. W. J., Embregts, P. J. C. M., Bosman, A. M. T., & Jahoda, A. (2009). Training staff serving clients with intel-
lectual disabilities: A meta-analysis of aspects determining effectiveness. Research in Developmental Disabilities, 30(3),
503–511. https://doi.org/10.1016/j.ridd.2008.07.011
Weinberg, M., Parenti, K., & Powell, S. (2000). Relationship building as a best practices approach in treatment planning.
Behavior Analyst Today, 1(2), 39–40. https://doi.org/10.1037/h0099872
Wood, A. L., Luiselli, J. K., & Harchik, A. E. (2007). Training instructional skills with paraprofessional service
providers at a community-based habilitation setting. Behavior Modification, 31(6), 847–855. https://doi.
org/10.1177/0145445507302893
SUP P ORTI NG I NFORM ATI O N

Additional supporting information can be found online in the Supporting Information section at the end of this
article.
How to cite this article: Kamana, B. U., Dozier, C. L., Kanaman, N. A., DiGennaro Reed, F., Glaze, S.
M., Markowitz, A. M., Hangen, M. M., Harrison, K. L., Bernstein, A. M., Jess, R. L., & Erath, T. G. (2023).
Large-scale evaluation of staff training in programs for adults with intellectual and developmental
disabilities. Behavioral Interventions, 1–22. https://doi.org/10.1002/bin.1971

Behavioral Interventions - 2023 - Kamana - Large Scale Evaluation of Staff Training in Programs For Adults With

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Behavioral Interventions - 2023 - Kamana - Large Scale Evaluation of Staff Training in Programs For Adults With

Uploaded by

Copyright:

Available Formats

Received: 17 November 2022 Revised: 24 July 2023 Accepted: 27 July 2023

Large-scale evaluation of staff training in programs

Bertilde U. Kamana1 | Claudia L. Dozier2 | Nicole A. Kanaman3 |

The May Institute, Santa Cruz, California, USA

2.1 | Participants and setting

2.2 | Response measurement, interobserver agreement, and data analysis

2.2.1 | Positive interactions

2.2.2 | Effective instructions

2.2.3 | Responding to problem behavior

2.2.4 | Consumer engagement

2.3 | General procedures

2.3.2 | BST and OJF

2.3.3 | Positive interactions

2.3.4 | Effective instructions

2.3.5 | Correct responses to problem behavior

2.3.6 | Consumer engagement

3.1 | Staff practices (all homes and programs)

3.1.1 | Positive interactions

3.1.2 | Effective instructions

3.1.3 | Responding to problem behavior

3.1.4 | Consumer engagement

3.2 | Individual staff data for applicable staff

CO N FLI CT OF I NTE RE ST STATE M E N T

DATA AVAI LABI LI TY STATE M E N T

SUP P ORTI NG I NFORM ATI O N

You might also like