Monitoring client progress, Monitoring treatment
integrity/fidelity,
Data-based decision making for effectiveness, Data-based decision
making for treatment revision
We have discussed previously
about:
• How to define a behavior
• How to collect data about the behavior
• How to make measurable goals
• What strategies can be used for behavior change
• But now we need to:
• Find out how to monitor the progress of the planned intervention
LETS DISCUSS
• What is the monitoring of progress?
• Why the monitoring of progress is important?
• What do you monitor?
• When do we monitor?
• How to monitor the progress?
WHAT IS MONITORING
MONITORING is a process of continuous collection of data
about the target behaviour during the implementation of the
intervention plan, so the intervention effects can be
evaluated.
Key elements of monitoring
• Behaviour support plans should receive continuous, ongoing evaluation (supervision must be
consistent and regular).
• The monitoring process is based on data collection systems.
• During monitoring a specific, measurable and observable behaviour (dependent variable) is under
our attention and specific data is collected.
• During the monitoring, a specific strategy (independent variable) is implemented and observed
how is implemented.
• Data is compared continuously or at specific times
Behaviour intervention plans should indicate:
(O’Neil et al., 1997)
The system that is used for data collection.
The process of data review (How often and by whom).
What can be monitored with Data
Collection
Systems
• Monitoring skills/general goals
• (ex. Receptive identification of objects, Matching non-identical pictures, Dressing, Tacting objects
on pictures etc.)
• Monitoring specific targets within the skill/goal
• (ex. Clap hands, stomp feet, turn around, sit down etc.)
• Monitoring daily implementation of targets
• (ex. Frequency of behaviour per session, duration of the behaviour per day, percentage of correct
responses etc. )
Monitoring Skills/
General Skills
•
• Elaborate a comprehensive list of
skills/goals necessary for
monitoring that you plan to
implement with a client (a real
example can be used).
Exercise 1
Monitoring Specific Progress
Electronic data Collection
Platforms
• Over the last years professionals started
to use also electronic data collection
systems.
• These systems can be used through the
mobile phones or computer.
• These systems can be used to support
decisions about current goals,
automatically can generate graphs, parent
and supervisor notifications, generating
reports, generating bills etc.
•
• What are your pros and cons of using
electronic platforms for data collection?
Monitoring Target Behaviors-
Problem Behaviors
The goal of the intervention:
Definition of the behaviour:
Collected measures:
(Data sheet can be found in annex)
Description of the strategy:
(Add function of the B if necessary)
When the data is reviewed:
Reviewed by:
Monitoring Target Problem Behavoiors
Exercise 2: Complete the Table
The goal of the intervention:
Definition of the behaviour:
Collected measures:
(Data sheet can be found in annex)
Description of the strategy:
(Add function of the B if necessary)
When the data is reviewed:
Reviewed by:
Monitoring Daily
Implementation of
Targets
• System must allow rapid and reliable data
collection
• System must allow multiple-responses data
collection
• It must allow to collect different measures of
behavior
• It should be easy to use
• It shouldn’t be expensive
• Data in the system should be easy
understandable by others
Monitoring Daily Implementation of
Targets
Monitoring
Daily
Implementati
on of Targets
Monitoring
Daily
Implementati
on of
Targetsnitori
n Daily
Implementati
on of Targets
Using graphs to monitor
the progress
• Graphs – a simple format for a visual display of
the relationship between a series of
measurements and relevant variables (Cooper,
Heron and Heward, 2020)
• Once the behaviour is observed and recorded, the
obtained data is
• the only source about the behaviour in the
past.
• Collecting data at different data points allows a
comparative analysis of the data.
• Comparing different data points guides
practitioners towards conclusions about
how the target behaviour is changed due to
manipulations of independent variable.
Treatment integrity/procedural fidelity
(Cooper, Heron and Heward, 2020)
Treatment integrity - refers to the extent to which the
independent variable is implemented as planned and no other
variables are affecting the treatment.
Procedural fidelity – often used interchangeably with
treatment integrity more accurately refers to the extent to which
procedures in all conditions of an experiment, including
baseline, are correctly implemented.
• Difficult or
impossible to
interpret
Low • the results of the
• In low treatment
integrity, the
intervention.
Treatment Carroll et al., 2013
treatment is
administered
Integrity incorrectly,
applied
inconsistently,
delivered in
underdose or
overdose, other
variables were
applied instead or
at the same time.
Low treatment Integrity Conclusions
• False positive – claiming a functional relation between variables when this does not exist.
• Example: Andy was engaging in aggressive behaviour (hitting) every time when his father used
the phone in his presence. Every occurrence of hitting was followed by giving him the phone.
Behaviour analyst implemented with family an Extinction procedure (access to tangible) and his
behaviour decreased to relative zero. At the same time with
• Extinction procedure, Andy’s mother started gluten free diet. Now family is claiming that
aggressive behavior decreased due to his gluten free diet.
Methods for ensuring
treatment integrity
• Precise operational definitions
• Intervention/Independent variable must be explicit, no
ambiguities, understandable and as a result easy replicable.
• Example:
1. The child is playing with toys for 5 minutes (a timer is used).
Then he is told that playing is finished and toys are removed.
2. Extinction access to tangibles - If the child starts the tantrum,
therapist is continuing removal of the toys and waiting until
the child will calm down.
3. Extinction attention - At the same time NO any verbal comments
about the child’s behaviour (Ex: please calm down, Shhhh! Etc.)
Try to avoid any visual contact during the problem behaviour.
4. Child is CALM for 1 minute (use the timer), then toys are given
back to the him.
Carroll et al.,
2013
Exercise 3
• Elaborate precise operational definitions for Andy’s case
describe previously:
• “ Andy was engaging in aggressive behaviour (hitting)
every time when his father used the phone in his
presence. Every occurrence of hitting was followed by
giving him the phone. Behaviour analyst implemented
with family an Extinction procedure (access to tangible).
When the behaviour analyst was present Extinction was
applied, but other times father was giving the phone
when Andy was hitting him.
• After 6 days of the implementation of “Extinction”
family made the conclusion that this strategy is not
effective with Andy”.
Methods for Ensuring
Treatment Integrity
• Simplify, Standardize and Automate
• Treatments that a brief, simple and
precise require little effort and it is more
likely that will be implemented as it is.
• More complex is the strategy more
probability for errors and low
• treatment integrity.
• When complex strategies are
implemented a script could improve the
accurate implementation of the strategy.
• Training, Practice and Feedback
• Reading the instructions for intervention is not
enough to ensure the accurate implementation of
the treatment.
• A combination of verbal instructions, modelling, and
rehearsal lead to higher level of treatment integrity
(Mueller et al., 2003).
• Howard and DiGennaro Reed (2014) found that
written instruction produced little or no
improvement in trainers implementation of a dog
training. A single viewing of a video showing of
correct and
• incorrect examples of the routine “produced
marked improvement” in correct implementation.
One to one training with modelling, descriptive
paise and feedback until 100% integrity was
obtained in a role play lead to satisfactory levels of
treatment integrity.
Howard
and
DiGenarro
, 2014
Measuring treatment integrity
Involves observing and recording the behaviour of those who
implement the intervention.
The target behaviours are selected and specified in written
protocols.
Summary of the accuracy of the implementation of all phases is
calculated.
During the observation can be found that there are external
factors that are intruding during treatment conditions.
Exercise 4: Watch the video and measure the treatment integrity (2 min /10 trials).
Ready behaviour/No problem behaviour
Eye contact with teacher/material
Clear instruction
Present the instruction once
Praise correct response
Prompt within 3 sec of incorrect response
/ - correct response 0 – incorrect
response
Video DTT
Source: https://www.youtube.com/watch?v=BSwBcGnWvTs
• Look through the data.
Take the decision:
To continue the intervention
Exercise To stop the intervention
To revise the intervention
To go to the next step
5: Data
Based • Explain your decision.
decisions
References:
• Carroll, R., Kodak, T., Fisher, W. (2013). An evaluation of programmed treatment-integrity errors
during discrete-trial instruction. Journal of Applied Behavior Analysis. 46, 379–394.
• Cooper, J., O., Heron, T., E., & Heward, W., L. (2020). Applied Behavioral Analysis (3rd Edition).
Harlow: Pearson Education.
• Frazer, T.J. (2018). ABLLS-R Skill Acquisition Program Manual. New York.
• Gravetter, F., J., & Forzano, L., B. (2006) Research Methods for the Behavioral Scinces (2nd Edition).
Belmont, CA: Wadsworth/Thomson Learning.
• Johnston, J. M., Pennypacker, H. S., & Green, G. (2020). Strategies and Tactics of Behavioral
Research and Practice (4th Edition). New York, NY: Routledge.
• Leaf, R & McEachin, J. (1999). A work in progress. New York.
• O’Neil, R., Horner, R., Albin, R., Sprague, J., Storey, K., and Newton., S. (1997). Functional
Assessment and Program Development for Problem Behavior: A Practical Hanbook (2nd ed). USA.
• Sundberg, M. & Partington, J. (1998). Teaching Language to Children with Autism or Other
Developmental Disabilities. AVB Press.
