Research Article

British Journal of Occupational Therapy

2020, Vol. 83(6) 363–374
Establishing and measuring treatment fidelity of ! The Author(s) 2020
Article reuse guidelines:
a complex cognitive rehabilitation intervention: sagepub.com/journals-permissions
DOI: 10.1177/0308022619898091
The multicontext approach journals.sagepub.com/home/bjot

Joan Toglia1,2 , Alyson Lee2,3, Chelsea Steinberg2, Amiya Waldman-Levi4

Abstract
Introduction: This article describes the process of establishing and measuring treatment fidelity of a complex cognitive rehabil-
itation intervention that uses a metacognitive strategy framework (the multicontext approach). We adapted treatment guidelines
for use within an inpatient rehabilitation setting for people with acquired brain injury; explored training methods and therapist
perspectives; developed and examined inter-rater reliability of a fidelity measure; and examined adherence to and proficiency of
treatment methods.
Method: Therapist perspectives of the intervention and training were obtained from written questionnaires and reflections. Inter-
rater reliability of 21 treatment components across 22 video-recorded treatment sessions was assessed by examining absolute
agreement between two raters using Cohen’s kappa. The proportion of treatment components implemented in individual sessions
as well as average proficiency ratings was calculated.
Results: The fidelity measure demonstrated good inter-rater agreement, ranging from 91–100% for treatment adherence and
kappa of .77 to .94 for therapist proficiency. Adherence and proficiency varied between therapists but increased to nearly proficient
levels with adjustments to training and procedures. Therapists highlighted the critical importance of video review as a
training tool.
Conclusion: The fidelity measure provides a clinical tool for therapist self-reflection, supervision and training, as well as a tool for
future research.

Keywords
Treatment fidelity, assessment, brain injury, cognition, evidence-based practice, metacognitive strategy approach, occupa-
tional therapy

Received: 8 June 2019; accepted: 5 December 2019

Introduction
The importance of establishing treatment fidelity has
been increasingly recognized as a crucial step in examin-
ing the effectiveness of interventions in occupational
therapy practice (Breckenridge and Jones, 2015; Hand
et al., 2018; Hildebrand et al., 2012). Treatment fidelity
refers to the extent to which an intervention is imple-
mented as it was intended (Bellg et al., 2004). Two key
aspects of treatment fidelity are (1) treatment integrity or
adherence to treatment protocols and quality of inter-
vention delivery, and (2) treatment differentiation or the
degree to which core treatment components differ from
other treatment conditions. Treatment fidelity increases
the reliability and internal validity of a study so that
results can be interpreted with confidence and
evidence-based conclusions about an intervention can
be drawn. Therefore, fidelity has been identified as an
essential component of research and is included within
the Consolidated Standards of Reporting Trials
(CONSORT) guidelines (Schulz et al., 2010).
The establishment of treatment fidelity begins with
treatment design or the development of an intervention
framework including theory, outline of treatment com-
ponents, procedures and activities (Dunn et al., 2018;
Gearing et al., 2011). Once treatment is specified, a ther-
apist training program and a system of assessing and
monitoring adherence, competence or quality in imple-
menting the treatment procedures is developed.
Treatment fidelity measures can provide a clear under-
standing of what therapeutic strategies have been
1
School of Health and Natural Sciences, Mercy College, New York, USA
2
Department of Rehabilitation Medicine, New York-Presbyterian/Weill
Cornell Medical Center, New York, USA
3
MOTION Sports Medicine, Bronx, New York, USA
4
Department of Occupational Therapy, Long Island University, Brooklyn,
New York, USA
Corresponding author:
Joan Toglia, Mercy College, 555 Broadway, Dobbs Ferry, NY 10522, USA.
Email: jtoglia@mercy.edu
364
implemented by therapists and how (Dunn et al., 2018;
Gearing et al., 2011). The results of fidelity assessments
can be used to provide feedback, supervision and further
training to therapists, which in turn improves treatment
adherence and quality (Hildebrand, 2019; Sanches et al.,
2018). Fidelity measures can also identify treatment pro-
cedures that need further specification and clarify inter-
vention ingredients that create change in targeted client
outcomes (Borrelli, 2011; Des Jarlais et al., 2004). These
three components of fidelity (treatment design, training
and intervention monitoring) have been described by
Gearing et al. (2011), and are included as part of the
core and fundamental elements of treatment fidelity
based on the National Institutes of Health (NIH) behav-
ioral change consortium (Bellg et al., 2004). We focus on
these aspects of fidelity and illustrate how treatment
design, training and fidelity measurement are closely
intertwined with each other in establishing fidelity.
Treatment fidelity has been identified as an important
consideration in both occupational therapy research and
clinical practice (Hildebrand et al., 2012). There are very
few articles, however, that describe the process of estab-
lishing fidelity, including operationalizing a complex
intervention, examining training and developing a fidel-
ity measure (Dunn et al., 2018; Hildebrand et al., 2012).
A handful of fidelity assessment tools have been identi-
fied for specific occupational therapy interventions; how-
ever, the majority of specific interventions used in
occupational therapy do not have a measure of fidelity
available (Hand et al., 2018; Parham et al., 2011;
Parvaneh et al., 2015).
We discuss the process of further operationalizing the
multicontext approach (MC approach) for application
within inpatient rehabilitation, examining training
needed and developing a fidelity measure for this
approach to monitor the adherence, consistency and
quality of MC intervention delivery. While such a fidel-
ity measure is essential for research, it is also intended to
be used to guide the accurate and consistent use of the
MC approach in clinical practice.
Metacognitive strategy interventions
Metacognitive strategy interventions have been identi-
fied as an evidence-based treatment to optimize execu-
tive function, self-awareness and functional performance
(Cicerone et al., 2019; Radomski et al., 2016). Although
there are several occupational therapy interventions that
focus on strategy use and metacognition, there are
important differences across interventions in the way
strategies and metacognitive principles are used or inte-
grated (Dawson et al., 2009; Kaizerman-Dinerman et al.,
2018; Toglia, 2018). The variability and ambiguity in
what constitutes metacognitive strategy training and
how it is carried out make it difficult to know if what
is characterized as metacognitive strategy training is the
same across studies. They also make it difficult for prac-
titioners to implement a metacognitive strategy training
approach in practice.
British Journal of Occupational Therapy 83(6)
Although treatment fidelity assessment has been men-
tioned in research reports within occupational therapy
metacognitive strategy interventions (Fleming et al.,
2017; Skidmore et al., 2015), the details or procedures
of such assessments are lacking. There are no published
treatment fidelity tools that clearly define, describe or
measure the implementation of metacognitive strategy
training techniques within rehabilitation, making it dif-
ficult to know if what is described as metacognitive strat-
egy training in one approach is the same as it is in
another approach. There is a need for occupational ther-
apy interventions that use a metacognitive strategy train-
ing approach to more clearly define, measure and
monitor the actual methods and techniques utilized by
therapists.
The multicontext approach is a complex multicompo-
nent intervention that utilizes an over-arching metacog-
nitive strategy approach. A key aspect of the MC
approach is its focus on promoting self-awareness, self-
monitoring skills, strategy generation and effective strat-
egy use across a wide range of functionally relevant
activities that are tailored to an optimal level of cogni-
tive challenge. The key treatment components of this
approach are outlined in Table 1.
The MC approach is based on the dynamic interac-
tional model of cognition and provides general guide-
lines for metacognitive strategy training that involves
collaborating with a client to determine single or multi-
ple strategies that can be applied across a broad range of
tasks (Toglia, 2018). Each treatment session uses a con-
sistent metacognitive strategy framework that includes
mediation or guided questions before, during and imme-
diately after each treatment activity (see Table 1) to
enhance the ability to generate and use strategies as
well as to anticipate, monitor and self-assess perfor-
mance (Toglia et al., 2010). Mediation is differentiated
from cues because cues provide increasing levels of assis-
tance or direct the person on what to do and/or how to
do it. In contrast, mediation involves posing guided
questions that help the person figure out what to do
themselves (Toglia and Foster, in press).
Throughout treatment there is a focus on the transfer
and generalization of learning from each session to the
next as well as to a person’s everyday life. Transfer and
generalization require the ability to detect similarities
between situations and recognize that a previously
learned strategy is applicable to a new situation
(Toglia, 1991; Toglia and Foster, in press). To assist
individuals in making connections and applying strate-
gies across situations, treatment activities are structured
along a horizontal continuum, representing different
levels of transfer. Treatment progresses in a sideways
manner from activities that look alike or are alternate
activities (near transfer) to those that are different in
appearance (far transfer). The underlying cognitive
skills, however, remain consistent across activities
(Toglia, 1991, 2018). Table 1 presents an example of
four different activities involving the use of a list.
Although the materials and contexts vary, all activities
Toglia et al.
Table 1. The multicontext (MC) approach: a summary of key treatment components.
Treatment components
1. Metacognitive strategy framework
System of guided questions and mediated learning techniques
used before, during and immediately after activities within
each session, as well as across different sessions
(See supplemental material 1–2 for examples)
2. Functional cognitive treatment activities are structured
horizontally
3. Therapeutic support and methods to enhance self-efficacy
365
Description of MC treatment components
Pre-activity phase:
- Guided anticipation of challenges
- Guided strategy generation
During activity: performance is interrupted with brief periods of
mediation or stop, check and review periods as needed, to help
the client self-recognize and manage errors or re-assess the
effectiveness of strategies by posing questions that encourage
self-assessment and strategic thinking.
Post-activity phase:
- Session self-evaluation: structured self-assessment methods or
self-check lists to allow self-discovery of errors. Guided
questions focus on “awareness of performance” or identifi-
cation of challenges encountered.
- Strategy reflection: questions are posed to help the client
identify task methods or strategies that contribute to success,
to understand why the strategies worked or to identify alter-
nate methods that could be used.
- Strategy application: questions are posed that explicitly help the
client to make connections between strategies used in the
treatment session and past, present and future life activities
(strategy bridging) and identify alternate strategies.
OR
- Structured journaling: used to summarize and self-evaluate
performance; strategies used; reflect on what was learned;
connect or bridge the strategy to other activities and identify
what could be done differently next time; set goals for next
session.
Horizontal transfer continuum:
- The horizontal continuum structures activities from near to far
transfer. Physical similarity between activities is gradually
decreased along the continuum, while underlying cognitive
demands remain similar. For example, all activities below
involve identifying similarities or discrepancies between lists
of items and other information.
Comparing a list of

Food items to those on a brunch menu

Food items to those on a dinner menu or food circular (very
similar)

Ingredients from a recipe to items that are in a kitchen

Items to pack for a trip to items in a bathroom/bedroom
(different)
Variability and practice
- Repeated experiences with a variety of activities that present
common cognitive challenges are used to provide the client
with the opportunity to self-discover performance errors.
- Focus on strategy practice and application across a variety of
activities that are at similar levels of challenge (sideways
learning), prior to increasing difficulty.
Treatment activities are functionally relevant
Optimal challenge point:
- Activities are not too easy and not too hard.
(Strategies are not needed if the activity is too easy and are too
effortful or ineffective if the activity is too hard.)
Therapeutic methods to enhance self-efficacy
- Figuring out problems: encourages client to figure out perfor-
mance problems themselves or identify strategies that might
help prevent/control cognitive symptoms.
- Supportive questions: guided questions are posed using a
neutral, non-confrontational or non-threatening tone.
- Focus on methods to enhance success: there is an emphasis on
the methods that lead to success and on empowering the
person to understand and manage their cognitive symptoms.
The focus is on staying a step ahead, or on what the person
can do to optimize performance, rather than on limitations.
- Positive, supportive atmosphere: use of positive reinforcement,
(continued)
366 British Journal of Occupational Therapy 83(6)

Table 1. Continued
Treatment components Description of MC treatment components

praise, encouragement or validation, with a focus on the

process.
- Reflection of observations: assists client in self-reflection.
Restates/rephrases and reframes client statements, or shares
observations and interpretations when appropriate and
highlights positive task methods.

require the ability to follow a list, search for and locate 2a. Explore the type of training and supervision
information, identify discrepancies and keep track of needed.
items that are located.
Activities with similar cognitive demands result in 3. Establish a fidelity procedure and assessment tool, focus-
similar performance errors. This provides an opportuni- ing on metacognitive strategy intervention techniques to
ty for the person to repeatedly practice recognizing and optimize treatment adherence and proficiency.
managing the same error across different functional
activities. Treatment activities are not increased in com- 3a. Determine inter-rater reliability of the fidelity
plexity until there is evidence of strategy transfer, or measure.
3b. Determine therapist adherence and proficiency
application of a strategy across different activities or
across treatment sessions.
contexts at the same difficulty level (Toglia, 2018). MC
treatment components and case applications are further
described in detail in other sources (Steinberg and
Method
Zlotnick, 2019; Toglia, 2018; Toglia and Foster, in
press; Toglia et al., 2010). We examined treatment adherence and quality in imple-
menting the MC approach with adults with acquired
brain injury in an inpatient rehabilitation setting
Study rationale within the context of routine occupational therapy treat-
ment sessions. The study was approved by the institu-
The MC approach, including the metacognitive frame-
tional review board (IRB) and all participants provided
work, has been demonstrated to be effective in several
written consent. Videos of treatment sessions from a
studies that have included people with acquired brain
pilot feasibility and preliminary efficacy study (Jaywant
injury or Parkinson’s disease (Foster et al., 2017;
et al., manuscript submitted) were used for supervision
Goverover et al., 2007; Toglia et al., 2010). Although
as well as to refine and further develop a fidelity
key treatment components of the metacognitive frame-
measure.
work used within the MC approach have been described
In this study, we report methods to ensure the con-
and examined previously, only a single experienced
sistency of the intervention. Two therapists video-
research-trained therapist delivered the interventions in
recorded their own MC treatment sessions with either
outpatient or community settings. We sought to deter-
a tripod or assistance from an occupational therapy
mine how the same treatment protocol could be success-
graduate student, when available. Four clients with
fully used within an inpatient rehabilitation unit for
acquired brain injury, two from each therapist, were
people with acquired brain injury and replicated by
selected from the pilot feasibility study based on a min-
less experienced therapists within the context of every-
imum of five recorded treatment sessions. Clients ranged
day clinical care.
in age from 48 to 73 years. Conditions included those
As a first step toward establishing intervention fidelity
with stroke (2), traumatic brain injury and status post
of the MC approach, we focused our efforts on interven-
brain tumor resection. There were 5–7 treatment sessions
tion delivery (treatment design, intervention training and
video-recorded for each client, with a total of 22 sessions
monitoring intervention delivery) as described by
that were available for fidelity ratings. MC treatment
Gearing et al. (2011), with a primary goal of developing
sessions were 1–2 times a day for a minimum of five
a measure of fidelity for the MC approach.
consecutive sessions and were conducted as part of occu-
Our aims were the following:
pational therapy clinical care. Therapists, raters and cli-
1. Further operationalize, specify and refine the MC treat- ents were selected through convenience sampling.
ment guidelines for replication by others and for appli- Clients were assigned to therapists upon admission to
cation to an inpatient rehabilitation setting. the rehabilitation inpatient unit based on availability
2. Explore implementation of the MC approach within the within the therapist schedule, as per standard clinical
context of clinical care, from the therapist perspective. procedure. Recorded treatment sessions of the first and
Toglia et al.
last clients for each therapist, who also met the minimum
criteria of five videos, were selected for fidelity ratings.
Refining MC treatment guidelines
Prior to initiation of the feasibility study or videotaped
MC treatment sessions, five therapists on an inpatient
rehabilitation unit volunteered to review the previous
treatment protocol used with people with chronic TBI,
including video case examples of MC intervention
(Toglia et al., 2010, 2011). Therapists had between 1
and 15 years of experience and all had attended in-
service training on the MC approach that included
videos of case presentations of persons with chronic
TBI. This group of therapists provided initial input on
the application and potential challenges of implementing
the established treatment protocol within the current
inpatient rehabilitation setting through a focus group
discussion. Therapists agreed that all treatment compo-
nents were applicable to the current setting but identified
the following potential challenges: (1) time constraints
that could interfere with discussion before and after an
activity; (2) short length of stay and minimal time to
address both physical and cognitive deficits; (3) lack of
easy access to a wide range of short functional cognitive
activities (10–15 minutes) that specifically target the cog-
nitive symptoms typically present in an inpatient setting
(keep track of information; use simple lists to locate, find
or gather information or materials); and (4) difficulty
with structuring activities along a horizontal continuum.
To address these needs, a series of structured inpa-
tient cognitive functional activity kits were constructed,
tested and revised based on therapist feedback. Activity
kits included everyday materials such as menus, sched-
ules or food circulars, and involved the use of functional
contexts such as gathering items on a list from kitchen
cabinets, checking inventory in a supply closet or pack-
ing items on a list for a weekend getaway. Therapists
were able to choose from directions that were similar
across functional activities and that placed different
demands on cognitive skills depending on the client’s
cognitive performance errors and needs. In addition,
activities could be easily adjusted to the person’s cogni-
tive and motor abilities, completed within 10–15 minutes
and presented across a horizontal continuum (Toglia,
2017). It was determined that a minimum of six struc-
tured cognitive functional activities would be used across
a minimum of three sessions. Therapists were provided
with the option of using other personalized structured
activities with similar cognitive demands, if it was appro-
priate. Similar to the previous treatment protocol
(Toglia et al., 2010), as self-awareness emerged, or
after experience with structured activities, clients could
be asked to select their own activities.
Therapist training and supervision
After initial treatment guidelines were modified for the
inpatient setting based on group input, two therapists
367
participated in the feasibility study sequentially and
implemented videotaped MC treatment sessions follow-
ing initial training as described below. One therapist had
been part of the initial group treatment review and the
other joined the project 9 months after it began. Both
therapists had an entry-level master’s degree in occupa-
tional therapy, with 2 years of experience at the start of
their participation. Each therapist participated in initial
training with the first author, prior to the implementa-
tion of MC treatment sessions that included a review of
previous publications, treatment protocols and discus-
sion of treatment videos that modeled metacognitive
strategy intervention techniques. Updated treatment
guidelines including an outline of the metacognitive
framework, treatment fidelity checklist and sample ques-
tions were provided.
Once MC treatment began, treatment videos were
reviewed with the two therapists, along with the fidelity
checklist, on a weekly basis during intervention, by the
first author. In addition to video reviews, the first author
provided ongoing supervision through phone conferen-
ces, email correspondence and on-site training/feedback.
Some of the lessons learned during supervision with
therapist 1 were integrated into the initial training for
therapist 2. For example, in addition to the above, ther-
apist 2 had the benefit of more specific guidelines for
mediation. Role-playing of the metacognitive frame-
work with the primary author and with colleagues was
also implemented with therapist 2 prior to seeing clients.
This was a recommendation from therapist 1.
Therapist perceptions of the training for and imple-
mentation of the approach were obtained through a
written survey and therapist reflections after the treat-
ment project was completed. Questions included: What
stands out from this experience? What did you learn as a
therapist? What challenges did you experience in trying
to implement the MC approach? What components of
the training helped you the most? How did your clients
respond to this approach? What recommendations do
you have for other therapists who want to learn and
use this approach within the context of everyday
practice?
Development of the treatment fidelity
measurement tool
An initial fidelity checklist was developed that focused
on adherence to key components of the metacognitive
framework (pre-activity, during activity and post-
activity questioning) described in previous literature
(Toglia, 2018; Toglia et al., 2010). The initial checklist
only rated adherence or the implementation of each
treatment component. The fidelity checklist served as a
useful guide for discussion and supervision during video
reviews, but it quickly became apparent that it was insuf-
ficient for fully examining the quality and skills needed
for implementing treatment as intended. For example,
quality or level of competence in carrying out each treat-
ment component was not included. Video review
368
illustrated that full implementation of a treatment com-
ponent does not indicate that the treatment component
was carried out as intended. As a result of these obser-
vations, the original components were preserved but
major changes were made to item content specification
and scoring, including the addition of proficiency
ratings.
The final fidelity measure included 16 items across
three categories that included the key components of
MC treatment with examples for each core ingredient.
An accompanying scoring guide that specified criteria
and proficiency levels was developed from video reviews
(see online supplemental material 1). The metacognitive
category, subdivided into questions used before, during
and after the activity, included seven items rated accord-
ing to adherence (treatment component is present (2) or
absent (1)) and six items rated according to proficiency
or quality (0 ¼ absent, 1 ¼ little evidence, 2 ¼ emerging/
adequate, 3 ¼ proficient or skilled). The second category,
general therapeutic techniques, included five items rated
according to proficiency level. The third category includ-
ed characteristics of treatment activities and consisted of
four items, which were rated according to adherence. In
total there were 21 ratings for each treatment session.
Raters included the supervising occupational thera-
pist (first author) and a second independent rater that
had not been involved in the pilot feasibility project. The
second rater has a terminal degree with advanced
research training with an emphasis on instrumentation,
video analysis and intervention effectiveness. She
attended a 2-day workshop on the MC approach pre-
sented by the first author and reviewed additional read-
ing materials and resources. This rater initially reviewed
other MC treatment videos with the first author prior to
rating clients using the fidelity measure.
After the rater’s training phase was completed, each
rater observed and rated video treatment sessions sepa-
rately. Each rater identified examples where they were
unsure of the score because the scoring criteria were
ambiguous. This led to further clarity and re-definition
of the scoring criteria. Once this was completed and the
scoring criteria guide was finalized, all videos were rated
independently and compared at the college lab. Formal
ratings of fidelity on the final measure occurred after the
pilot feasibility project had concluded and included the
same videos that were obtained during the pilot feasibil-
ity study. Inter-rater reliability of the final treatment
fidelity measure was established by comparing 462 rat-
ings by each independent rater. This included 21 ratings
of individual treatment components within 22 treatment
sessions, across four clients.
Data analysis
Inter-rater agreement for the final fidelity measure was
assessed by examining both percentage of absolute
agreement, and Cohen’s kappa for binary variables
and weighted kappa for ordinal variables (proficiency
level) for each treatment component. Cohen’s kappa
British Journal of Occupational Therapy 83(6)
provides an index of inter-rater reliability and adjusts
for the likelihood of agreement by chance. Acceptable
level of agreement was defined as 75% of absolute agree-
ment or Cohen’s kappa statistic of above .60
(Chaturvedi and Shweta, 2015; McHugh, 2012). We
also examined the correlation between the two raters
for each treatment component using Kendall’s tau coef-
ficient. Fidelity was examined by calculating percentage
of treatment components that were present in individual
sessions (adherence) across each therapist. In addition,
total average ratings for proficiency in the metacognitive
and therapeutic sections of the fidelity measure were
averaged across raters and compared across therapists.
Results
Refining MC treatment guidelines
Prior to implementing MC treatment by therapists 1 and
2, existing treatment guidelines and activities used in
prior studies were modified for the inpatient rehabilita-
tion setting, based on initial group feedback. The meta-
cognitive framework was simplified by shortening
questions and eliminating self-ratings previously used
for self-evaluation. Strategy reflection and application
discussion questions were designed to substitute for the
use of a structured written journal (see Table 1) due to
time constraints; however, the core MC treatment com-
ponents and guidelines remained unchanged (Toglia
et al., 2010).
After procedures and activities for the inpatient set-
ting were finalized, and initial training was provided, the
MC treatment sessions were initiated by therapists 1 and
2. Video reviews of the initial treatment sessions of ther-
apist 1 led to greater specification of treatment guide-
lines. For example, although metacognitive questions
and video examples were reviewed prior to the start of
videotaped treatment sessions, initial video analysis
during the supervision of therapist 1 indicated inade-
quate probing of responses, an over-focus on task out-
comes, and a tendency to predominantly use direct cues,
yes/no questions or questions focused on the task rather
than on the process. Online supplemental material 2
presents selected examples and analysis from video tran-
scripts in a table format to demonstrate the different
levels of proficiency that were observed. Guided ques-
tions that do and do not align with the principles of
metacognitive strategy techniques used within the MC
approach are illustrated. This is an example of informa-
tion that emerged during video reviews that was used to
further specify and refine treatment guidelines as well as
rating criteria for the fidelity measure.
Therapist training and supervision: perceptions of
training and intervention delivery
Several themes emerged from therapist reflections on
their experiences with the project through the question-
naires they completed. This included learning to shift
Toglia et al. 369

Table 2. Therapist perspectives on training and implementing the multicontext (MC) approach.
Topics Therapist quotes

Learning to shift clinical practice
Knowing is different from “doing”
Video reviews
Therapist recommendations
“We’re often task oriented. We don’t take the time to stop and ask questions and examine the client’s
perceptions of performance. It doesn’t matter if the task is not finished. It’s more about the process
than the end product and it took me a long time to fully understand how valuable that was.”
“Until I had completed this project, I viewed cognition very differently. I’ve learned that providing
patients with cognitive strategies is not effective if they do not understand their errors, and/or
cannot identify when and how strategies will be useful. Understanding how a patient detects his/
her own error is equally important to teaching them how to implement a strategy. Previously I
didn’t focus on these crucial components of cognitive dysfunction.”
“I understood the concept of providing guided questions without direct cueing; however, I quickly
found that it is often hard to alter your way of assisting and cueing a client once it has become a
habit. I had to learn to sit back, keep quiet and to simply observe performance, without jumping in
to cue.”
“Video and self-reflection on my own performance helped me the most. It helped me recognize when
I was doing more direct cueing. It showed me that there were points in time that I didn’t need to
jump in and I could have stood back a little. It helped me use my own self-monitoring strategies
during treatment.”
“Through constant review of these videos, I was able to self-evaluate my own skills and ability to
facilitate performance. Using the video as self-reflection is probably where I learned the most. It
has helped me grow as a therapist.”
“Don’t be afraid to review the videos two, three or more times over. We learn more and more about
ourselves as providers of feedback, the more we can witness how we deliver that feedback.”
“It takes practice . . . It’s not something that you can just read about and do . . . Clinicians have to
prepare and spend time outside of sessions mastering this.”

perspectives on cognition, clinical practice and the treat-
ment methods typically used; the importance of video
review as a training tool; challenges in implementation;
recommendations and client responses to intervention.
Therapist quotes in these areas are presented in Table 2.
therapist indicated that once she felt comfortable, she
began to naturally use the same type of metacognitive
questions before and after a wide range of different
activities across treatment, including transfers, upper
extremity exercises and other activities of daily living.

Learning to shift clinical practice. Both therapists indicat-
ed that they had a deeper understanding of cognition
after using the MC activities and implementing this
approach. They discussed a shift in clinical practices
from focusing on task outcomes or providing direct
cues and instruction to focusing on and analyzing the
process and methods used. Therapists described how the
treatment methods required a different perspective and
an adjustment of methods used in conventional practice.
Challenges in implementation: knowing is different from
“doing”. Both therapists expressed that it was more
challenging than they initially thought it would be to
implement this approach. Therapists indicated that
although they knew and understood the treatment pro-
cedures, it was challenging for them to use their knowl-
edge when they were “in the moment.” One therapist
found that it was most difficult to “limit my natural
tendency to jump in and cue” and observe, analyse
and mediate all at the same time.
Other challenges identified included knowing when or
how much to intervene, keeping activities at the right
level and staying process oriented rather than task ori-
ented. Therapists also acknowledged that finding time to
implement this approach in the acute rehab setting was
particularly difficult in the beginning. It took time and a
lot of practice to feel comfortable and confident in using
this approach and integrating it into practice. One
Video reviews as a powerful training tool. Both therapists
agreed that the video reviews were a critical component
in training, learning and skill development. The process
of video review helped therapists become aware of meth-
ods that they didn’t realize they had used and promoted
reflective thinking about their clinical skills.
Therapist recommendations. Therapist recommendations
to others who want to learn this approach included
hands-on practice, role-playing, creating cue cards for
oneself and video review as a means of self-reflection
and critique.
Client response. Both therapists indicated that clients
responded positively to the MC approach regarding
improvement in strategy use, function, and participation
and engagement in therapy sessions. Therapists com-
mented that clients actively participated in discussion
before and after activities and appeared to benefit from
self-assessing their own performance. One therapist indi-
cated that clients often gave more elaborate insight into
their thought process and what strategies they had
thought of or used, above and beyond what the therapist
had been aware of or had expected. The second therapist
indicated that ensuring that activities were at an optimal
level of challenge and functionally relevant was a key to
client engagement and motivation. Therapists appreciat-
ed the flexibility of having structured functional activity
370 British Journal of Occupational Therapy 83(6)

Table 3. Inter-rater agreement of the metacognitive section of the proficiency scale.

Metacognitive items % agreement Kendall’s tau Weighted kappa Z p SE CI

Anticipation 85% .85 .89 4.8 .000 .07 .76–1.0

n ¼ 20 p ¼ .000
Strategy generation 95% .91 .94 5.5 .00 .06 .81–1.1
n ¼ 20 p ¼ .000
Mediation 75%
n¼8
Self-assessment 86% .81 .77 4.3 .00 .12 .53–1.0
n ¼ 22 p ¼ .000
Strategy use 86% .82 .84 5.1 .00 .10 .65–1.0
n ¼ 22 p ¼ .000
Strategy application 86% .89 .90 5.1 .000 .06 .78–1.0
n ¼ 21 p ¼ .000

kits as well as the option to use personalized activities

when appropriate. A disadvantage noted by one thera-
pist was that it was difficult to implement with individ-
uals with language deficits or those with strong denial of
their deficits.
Fidelity measurement tool
We first report inter-rater agreement results of the fidel-
ity adherence scale, proficiency scale and general thera-
peutic techniques and then discuss therapist adherence
and proficiency across treatment sessions. Our sample
size included 22 treatment sessions, each with 21 treat-
ment components, yielding a total of 462 items scored by
each rater.
Fidelity adherence scale. There was complete (100%)
agreement between raters in identifying if treatment
components were present or absent within the metacog-
nitive framework section. Similarly, raters agreed that
activities were varied at the same or optimal level of
challenge. This was expected because pre-made activity
sets were used the majority of the time (68%). Absolute
agreement regarding the functional relevance of activi-
ties was 91%.
The percentage of time that treatment components
were missing appropriately (followed guidelines or lack
of opportunity) was examined. Questions regarding
anticipation, strategy generation and strategy generali-
zation were each missed on one occasion (5%).
Mediation during the activity was missed appropriately
64% (14/22) of the time; however, both raters agreed
100% that this component was missing appropriately.
Mediation depends on the needs of the client and was
only used 36% of the time.
Proficiency scale. The level of agreement in scoring pro-
ficiency or treatment quality across the metacognitive
section is presented in Table 3 and ranged from 75–
95%. The kappa statistic was statistically significant
and ranged from .77 to .94, reflecting moderate
to good agreement, over and above chance. Proficiency
was not rated if a treatment component was
Table 4. Fidelity measure: inter-rater agreement of general
therapeutic technique (n ¼ 22).
% Agreement Kappa p SE CI
Figure out problems 96% .87 .00 .10 .67–1.1
Questions supportive 100% 1.00 .
Focus on success 86% .65 .001 .18 .30–.99
Support/positive 91% .62 .002 .24 .16–1.1
Reflects on observations 86% .64 .003 .19 .27–1.0
missing; therefore, the number of sessions for the
kappa statistic varies between 20 and 22, with the excep-
tion of mediation. The kappa statistic was not calculated
for mediation due to the inadequacy of the sample size
(n ¼ 8). Analysis of ratings indicated that disagreements
were related to the distinction between emerging compe-
tency and proficiency. Raters consistently agreed with
ratings of little evidence of proficiency. Overall there
were significant, positive correlations between ratings,
as indicated by Kendall’s tau coefficients.
General therapeutic techniques. Level of agreement for
items used to build a positive atmosphere that fosters
self-efficacy ranged from 86–100% absolute agreement,
with Cohen’s kappa ranging from .62–1.00, indicating
acceptable levels of agreement. The strongest areas of
agreement and highest level of proficiency involved
asking questions in a non-threatening and supportive
tone. In general, both therapists demonstrated high
levels of proficiency in therapeutic support, with average
ratings of 2.8 and 2.9 respectively (see Table 4).
Therapist adherence and proficiency across treatment
sessions. Treatment integrity involves delivering treat-
ment with adequate levels of adherence. Four out of
six components were implemented 90% of the time or
more when appropriate. Table 5 illustrates overall
adherence as well as changes in adherence across ses-
sions. Overall results suggest that the metacognitive
components of “identification of challenges before the
task” and “discussion on strategy application after the
task (generalization)” showed the least adherence and
Toglia et al. 371

Table 5. Adherence to metacognitive key treatment components and changes over sessions.
Sessions 1–7 Sessions 8–12 Sessions 13–17 Sessions 18–22
Therapist 1 Therapist 1 Therapist 2 Therapist 2
Overall fidelity or (client 1) (client 2) (client 3) (client 4)
adherence % present % present % present % present

Orientation 100% 100% 100% 100% 100%

n ¼ 22 (22/22) (7/7) (5/5) (5/5) (5/5)
Anticipation 75% 17% 100% 100% 100%
n ¼ 20 (15/20) (1/6) (5/5) 4/4 (5/5)
Strategy generation 90% 71% 100% 100% 100%
n ¼ 21 (19/21) (5/7) (5/5) 4/4 (5/5)
Mediation 100% 100% 100% 100%
n¼8 (8/8) (2/2) (3/3) 0/0 (3/3)
Self-assessment 95% 86% 100% 100% 100%
n ¼ 22 (21/22) (6/7) (5/5) (5/5) (5/5)
Strategy use 77% 29% 100% 100% 100%
n ¼ 22 (17/22) (2/7) (5/5) (5/5) (5/5)
Strategy application 67% 17% 80% 100% 80%
n ¼ 21 (14/21) (1/6) (4/5) (5/5) 4/5

were only implemented 75% (15/20) and 67% (14/21) of
the time respectively, despite opportunity to implement
these components. Closer inspection revealed that this
was primarily related to the first seven sessions con-
ducted within the project. These sessions reflected low
adherence and quality (client 1; see Table 5).
Changes over time were observed with therapist 1 and
from therapist 1 to therapist 2 as the treatment protocol
was further specified. The first seven sessions represented
the ratings of therapist 1, who had the least amount of
training. Comparison of the first seven sessions (thera-
pist 1) to subsequent sessions could be considered an
example of treatment differentiation as it involves the
contrast between a therapist untrained and trained in
this approach. During and after these initial sessions,
additional training took place and adherence for thera-
pist 1 increased to 100% for all components except for
strategy generalization, which increased to 80%. This
pattern was also reflected in proficiency ratings. For
example, for therapist 1 the average proficiency rating
was 1.4 for sessions 1 to 7 (1 ¼ little evidence,
2 ¼ emerging/adequate) compared to an average profi-
ciency rating of 2.4 (3 ¼ proficient) for the same therapist
for sessions 8–12, which is approaching proficiency.
Discussion
The current article describes the process of refining the
MC approach within the context of everyday occupa-
tional therapy practice in an acute rehabilitation inpa-
tient setting for people with acquired brain injury.
Complex and multicomponent interventions such as
the MC approach are particularly challenging to oper-
ationalize and measure. Many of the treatment processes
and therapist responses are individualized and depend
on client needs or responses. The therapist often has to
make “on the spot” decisions or adjust methods and
activities within treatment sessions. Treatment is
therefore dependent on intervention delivery by the ther-
apist and therapist–client interactions (Hildebrand,
2019).
Therapist training and skill is therefore a critical
aspect of intervention delivery. Our initial training pro-
gram included a series of presentations with video treat-
ment examples, background publications and written
treatment guidelines, including a treatment fidelity
checklist. However, video reviews of initial treatment
sessions illustrated that these methods were insufficient
to achieve treatment fidelity, particularly in implement-
ing the metacognitive treatment framework. This may be
because we only provided examples of skilled interven-
tion delivery, so that it may have appeared “easy” to
implement. Without a structured system for the rating
or observation of videos, a therapist who is initially
using this approach may not fully grasp key elements
or relationships.
Therapist perceptions and quotes described in this
article highlight the challenges faced in attempting to
shift clinical practices. They also point out the need to
go beyond modeling and written procedures in training.
Changes in treatment methods require alterations in
therapist habits and procedural skills. This takes time,
conscious effort and practice. Within rehabilitation set-
tings, therapists typically identify strategies or methods
to address problem areas and directly instruct clients on
these methods. If errors are observed, the client is pro-
vided with feedback. The use of guided questioning
before and after an activity to facilitate strategy use
and self-monitoring skills was not within the scope of
conventional treatment methods. Under the pressures of
everyday clinical practice, therapists are likely to revert
back to the habitual procedures or techniques that they
are most comfortable with, without realizing it.
Knowledge of a procedure and how to carry it out is
different from implementing it consistently or accurately
within clinical practice. Therapists described a parallel
process of learning to use their own metacognitive
372
abilities to monitor, adjust and reflect on their ability to
use guided questions effectively, while at the same time
helping clients to monitor and regulate performance.
This is similar to the findings of Hildebrand et al.
(2012), who also found that the implementation of new
practices needed to go beyond a treatment manual.
Video recording and review of treatment is not
common in occupational therapy supervision or train-
ing, but it is often used in psychotherapy (Haggerty and
Hilsenroth, 2011). Video review of MC treatment ses-
sions emerged as a powerful tool. Therapists learned to
become more conscious of the way they used questions
or cues. It helped them identify areas that they needed to
monitor or alter in future interactions with clients.
Similar to benefits described in psychotherapy, video
review helped increase therapists’ self-reflection, clinical
reasoning and self-awareness (McCullough et al., 2011;
Topor et al., 2017).
In addition, video analysis helped us further delineate
criteria for measuring the fidelity of the MC treatment
approach and at the same time clarify treatment guide-
lines. Development of the MC treatment fidelity measure
emerged as a reciprocal and iterative process. The pro-
cess of examining MC treatment fidelity through video
reviews led to an in-depth look at intervention ingre-
dients and helped us to differentiate levels of proficiency
in implementation, as well as identify ambiguity within
treatment elements, not previously recognized. The
value of fidelity in directing attention to treatment ingre-
dients and improving treatment procedures has also
been observed by others (Breckenridge and Jones,
2015; Hildebrand et al., 2012).
The final MC fidelity measure provides a structure for
the microanalysis of the integrity and quality of a meta-
cognitive strategy intervention. The perfect agreement
achieved in determining the presence or absence of the
metacognitive treatment components reflects the clear
criteria of the treatment elements. The tool identified
low adherence and quality within the first seven sessions,
resulting in the removal of the first client from the pilot
intervention study and leading to additional clarification
of treatment guidelines and revised training methods.
The fidelity tool identified that nearly all metacognitive
components (identification of challenges, strategy gener-
ation, mediation, self-assessment, strategy discussion)
were utilized between sessions 8 and 22, when it was
appropriate to do so as per the treatment guidelines.
Mediation within the activity was used less than
expected; however, this appears to be related to the
brief nature of this early intervention, time constraints
within an inpatient setting, or the early phase of recov-
ery. Strategy-bridging techniques were also used less
than expected. Less utilization of this component is
likely related to time constraints, typical within an inpa-
tient setting as this component is typically addressed at
the end of a session.
Inter-rater agreement for treatment proficiency
reached acceptable levels; however, there were areas of
disagreement. Ratings for proficiency or quality of
British Journal of Occupational Therapy 83(6)
treatment are more subjective and, in addition to written
criteria, it may require more examples and training. The
kappa statistic levels in general therapeutic techniques
are borderline (.62–.65) in some areas; however, there
is a discrepancy between the measure of absolute agree-
ment (86% and above), the high correlations and the
lower kappa statistic (.62–.65). The kappa statistic can
be misleadingly low if a large majority of ratings are at
the highest level, and this was the case with most of these
items (Chaturvedi and Shweta, 2015). Levels of compe-
tency for both therapists were high (2.8 and 2.9 out of 3),
suggesting that the criteria for these items may need to
be further refined to better differentiate between skill
levels.
Limitations and recommendations for future
research
Additional research with this tool is needed across dif-
ferent raters, therapists and clients to further validate
this tool. Although 465 items were rated across 22 video-
taped sessions, a long and arduous process, there were
only two raters, two therapists and four clients. In addi-
tion, the videos used for inter-rater agreement had been
previously viewed and discussed with the principal inves-
tigator (PI) in relation to the training and development
of the fidelity checklist, and this could have influenced
the results of the inter-rater reliability ratings.
Therapist participants were interested and highly
motivated to integrate these methods into practice, and
this likely contributed to positive results. Additionally,
fidelity was measured under conditions of ongoing
supervision and support. The extent that treatment fidel-
ity is maintained over time when ongoing support is
withdrawn needs to be examined. Use of this tool to
rate therapy sessions with a comparison of ratings to
those of an expert, prior to treatment implementation,
should be investigated as part of training. However, even
with this type of training, our experience and that of the
therapist participants indicated that actual practice and
self-assessment through video review appears to be an
essential component of training.
Conclusion
In summary, we report fidelity procedures that focus on
the intervention design, training and development of an
intervention fidelity tool for monitoring the quality and
consistency of intervention delivery. Formal assessment
of treatment fidelity was also helpful in refining treat-
ment guidelines and specifying key elements of the inter-
vention itself (Gearing et al., 2011; Morrison et al.,
2017). Our final MC treatment fidelity tool has good
inter-rater agreement and can provide data on treatment
adherence and competency for research as well as pro-
vide a useful clinical tool for therapist supervision, train-
ing and self-reflection. Since the fidelity tool specifies the
key components of the MC approach along with exam-
ples of guided questions and probes for the
Toglia et al.
metacognitive framework, it provides guidance for accu-
rate implementation of this approach in clinical practice
as well. In the area of research, the MC fidelity tool can
be used to monitor treatment and ensure that it is deliv-
ered as intended across different therapists and detect or
prevent therapist drift away from treatment guidelines,
so that research results can be reliably interpreted.
Although the need for reliable treatment fidelity
measures has been acknowledged, this is the first such
measure reported within occupational therapy cognitive
intervention. This article describes procedures and pro-
cesses that may be helpful for the further development
and integration of other complex occupational therapy
interventions used in clinical practice. The processes
involved in “unpacking” a complex intervention so
that it can be used consistently by others, during both
clinical care and research, is quite challenging. A gran-
ular, in-depth look at intervention was facilitated by the
analysis of treatment videos taken within the context of
occupational therapy clinical practice. The fidelity mea-
sure provided in the online supplementary material pro-
vides clinicians with a tool that guides training in,
supervision of and clinical practice of the MC approach.
At the same time, it provides a tool for researchers that
can be used to study the effectiveness of the MC
approach.
Key findings
• The multicontext approach fidelity form is a useful
clinical tool for training occupational therapy clini-
cians on this approach as well as for use in effective-
ness studies.
• Video review is a powerful tool to use in occupational
therapy supervision to elicit self-reflection and facili-
tate clinical reasoning.
What the study has added
This study demonstrates the need for and usefulness
of establishing a reliable treatment fidelity measure to
(1) monitor the adherence to and treatment quality of
an occupational therapy cognitive rehabilitation
intervention, so that treatment results and outcomes
can be confidently interpreted; (2) further refine inter-
vention guidelines and (3) facilitate therapist self-
reflection, supervision and training.
Acknowledgments
Special thanks and appreciation are extended to Andrea
Mastrogiovanni MA, OTR/L, for her assistance with this project,
and the clients for their participation and contributions to this
study. We also thank Michel O’Dell, MD, for his support with
this study, Erin Foster, OTR/L, PhD, for her thoughtful comments
and feedback on the entire manuscript and Caroline Pelaez, grad-
uate occupational therapy student, for her assistance in manuscript
preparation.
373
Research ethics
Ethical approval was obtained from Weill Cornell Medicine
Institutional Review Board, Protocol #1308014257R005, approved
December 30 2015.
Consent
All participants provided written informed consent for participa-
tion in this research, including agreement to videotape occupational
therapy treatment sessions.
Declaration of conflicting interests
The authors declare no potential conflicts of interest with respect to
the authorship, research or publication of this article.
Funding
The authors disclosed receipt of the following financial support for
the research, authorship and/or publication of this article: Amiya
Waldman-Levi received consultation funds to support her work on
this project. Other authors received no financial support for the
research, authorship and/or publication of this article.
Contributorship
Joan Toglia conceptualized the project, designed the treatment
guidelines, applied for ethical approval, rated videos, analyzed
results and wrote the initial draft of the manuscript. All authors
contributed to content within sections of the manuscript. Alyson
Lee and Chelsea Steinberg contributed to the methodology of the
project, including subject recruitment, intervention implementation
and refining the treatment protocol. Amiya Waldman-Levi assisted
with the literature review, coded and rated videos, and assisted in
analyzing and interpreting results. All authors reviewed and edited
the manuscript and approved the final version.
ORCID iD
Joan Toglia https://orcid.org/0000-0002-6902-6853
Supplemental material
Supplemental material for this article is available online.
References
Bellg AJ, Borrelli B, Resnick B, et al. (2004) Enhancing treat-
ment fidelity in health behavior change studies: Best practi-
ces and recommendations from the NIH Behavior Change
Consortium. Health Psychology 23(5): 443–451.
Borrelli B (2011) The assessment, monitoring, and enhance-
ment of treatment fidelity in public health clinical trials.
Journal of Public Health Dentistry 71(s1): S52–S63.
Breckenridge J and Jones D (2015) Evaluation in everyday
occupational therapy practice: Should we be thinking
about treatment fidelity? British Journal of Occupational
Therapy 78(5): 331–333.
Chaturvedi HK and Shweta RC (2015) Evaluation of inter-
rater agreement and inter-rater reliability for observational
data: An overview of concepts and methods. Journal of the
Indian Academy of Applied Psychology 41(3): 20–27.
Cicerone KD, Goldin Y, Ganci K, et al. (2019) Evidence-based
cognitive rehabilitation: Systematic review of the literature
from 2009 through 2014. Archives of Physical Medicine and
Rehabilitation 100(8): 1515–1533.
Dawson DR, Gaya A, Hunt A, et al. (2009) Using the cognitive
orientation to occupational performance (CO-OP) with
adults with executive dysfunction following traumatic
374
brain injury. Canadian Journal of Occupational Therapy
76(2): 115–127.
Des Jarlais DC, Lyles C, Crepaz N, et al. (2004) Improving the
reporting quality of nonrandomized evaluations of behav-
ioral and public health interventions: The TREND state-
ment. American Journal of Public Health 94(3): 361–366.
Dunn W, Little LM, Pope E, et al. (2018) Establishing fidelity
of occupational performance coaching. OTJR: Occupation,
Participation and Health 38(2): 96–104.
Fleming J, Ownsworth T, Doig E, et al. (2017) The efficacy of
prospective memory rehabilitation plus metacognitive skills
training for adults with traumatic brain injury: Study pro-
tocol for a randomized controlled trial. Trials 18(3): 1–11.
Foster ER, Spence D and Toglia J (2017) Feasibility of a cog-
nitive strategy training intervention for people with
Parkinson’s disease. Disability and Rehabilitation 40(10):
1127–1134.
Gearing R, El-Bassel N, Ghesquiere A, et al. (2011) Major
ingredients of fidelity: A review and scientific guide to
improving quality of intervention research implementation.
Clinical Psychology Review 31(1): 79–88.
Goverover Y, Johnston MV, Toglia J, et al. (2007) Treatment
to improve self-awareness in persons with acquired brain
injury. Brain Injury 21(9): 913–923.
Haggerty G and Hilsenroth MJ (2011) The use of video in
psychotherapy supervision. British Journal of
Psychotherapy 27(2): 193–210.
Hand BN, Darragh AR and Persch AC (2018) Thoroughness
and psychometrics of fidelity measures in occupational and
physical therapy: A systematic review. American Journal of
Occupational Therapy 72(5): 7205205050p1–7205205050p10.
Hildebrand MW (2019) Treatment fidelity. British Journal of
Occupational Therapy 82(5): 261–262.
Hildebrand MW, Host HH, Binder EF, et al. (2012) Measuring
treatment fidelity in a rehabilitation intervention study.
American Journal of Physical Medicine & Rehabilitation
91(8): 715–724.
Jaywant A, Steinberg CJ, Lee A, et al. (manuscript submitted)
Preliminary efficacy and patient satisfaction of multicontext
treatment to improve self-regulation and executive function-
ing in acute inpatient rehabilitation.
Kaizerman-Dinerman A, Roe D and Josman N (2018) An effi-
cacy study of a metacognitive group intervention for people
with schizophrenia. Psychiatry Research 270: 1150–1156.
McCullough L, Bhatia M, Ulvenes P, et al. (2011) Learning
how to rate video-recorded therapy sessions: A practical
guide for trainees and advanced clinicians. Psychotherapy
48(2): 127–137.
McHugh ML (2012) Interrater reliability: The kappa statistic.
Biochemia Medica 22(3): 276–282.
Morrison JD, Becker H and Stuifbergen AK (2017) Evaluation
of intervention fidelity in a multisite clinical trial in persons
with multiple sclerosis. The Journal of Neuroscience
Nursing: Journal of the American Association of
Neuroscience Nurses 49(6): 344–348.
Parham LD, Smith Roley S, May-Benson TA, et al. (2011)
Development of a fidelity measure for research on the
British Journal of Occupational Therapy 83(6)
effectiveness of the Ayres Sensory IntegrationVR interven-
tion. American Journal of Occupational Therapy 65(2):
133–142.
Parvaneh S, Cocks E, Buchanan A, et al. (2015) Development
of a fidelity measure for community integration pro-
grammes for people with acquired brain injury. Brain
Injury 29(3): 320–328.
Radomski MV, Anheluk M, Bartzen P, et al. (2016)
Effectiveness of interventions to address cognitive
impairments and improve occupational performance after
traumatic brain injury: A systematic review. American
Journal of Occupational Therapy 70(3):
7003180050p1–7003180050p9.
Sanches SA, Swildens WE, van Busschbach JT, et al. (2018)
FiRe: Evaluation of a fidelity measure to promote imple-
mentation of evidence-based rehabilitation. Psychiatric
Rehabilitation Journal 41(1): 46–54.
Schulz KF, Altman DG, Moher D, et al. (2010) CONSORT
2010 statement: Updated guidelines for reporting parallel
group randomised trials. BMC Medicine 8(18): 1–9.
Skidmore ER, Whyte EM, Butters MA, et al. (2015) Strategy
training during inpatient rehabilitation may prevent apathy
symptoms after acute stroke. PM & R: The Journal of
Injury, Function, and Rehabilitation 7(6): 562–570.
Steinberg CJ and Zlotnick S (2019) The multicontext approach.
In: Wolf TJ, Edwards DF and Giles GM (eds) Functional
Cognition and Occupational Therapy: A Practical Approach
to Treating Individuals with Cognitive Loss. Bethesda, MD:
American Occupational Therapy Association, Inc.,
219–237.
Toglia J (1991) Generalization of treatment: A multicontext
approach to cognitive perceptual impairment in adults
with brain injury. American Journal of Occupational
Therapy 45(6): 505–516.
Toglia J (2017) Schedule Activity Module: Functional Cognitive
Rehabilitation Activities & Strategy Based Intervention.
New York: MC CogRehab Resources, LLC.
Toglia J (2018) The dynamic interactional model and the multi-
context approach. In: Katz N and Toglia J (eds) Cognition,
Occupation, and Participation Across the Lifespan. Bethesda,
MD: AOTA Press, 355–385.
Toglia J and Foster E (in press) The Multicontext approach:
A metacognitive strategy-based intervention for functional
cognition. Available from the author.
Toglia J, Goverover Y, Johnston MV, et al. (2011) Application
of the multicontextual approach in promoting learning and
transfer of strategy use in an individual with TBI and exec-
utive dysfunction. OTJR: Occupation, Participation and
Health 31(1): S53–S60.
Toglia J, Johnston MV, Goverover Y, et al. (2010) A multi-
context approach to promoting transfer of strategy use and
self regulation after brain injury: An exploratory study.
Brain Injury 24(4): 664–677.
Topor DR, AhnAllen CG, Mulligan EA, et al. (2017) Using
video recordings of psychotherapy sessions in supervision:
Strategies to reduce learner anxiety. Academic Psychiatry
41(1): 40–43.