You are on page 1of 7

Research Worksheet Quantitative

By: Brianna Pupp, Faith Wilkins and Michelle Wilson

Research Question
Is vocational training at a job site more effective compared to a simulated work site for
individuals with developmental disabilities preparing to enter the workforce?
APA Reference
Ahearn, W. (2006). Enhancing job-site training of supported workers with autism: A
reemphasis on simulation. Journal of Applied Behavior Analysis, 39(1), 91102.
How does this study relate to your research question?
Opportunity Village offers vocational skills training through their Job Discovery Program
to adolescents ages 18-22 with developmental disabilities in the Clark County School District.
This program is designed to provide students the vocational training needed to become further
employed in their community. Unfortunately, often times these individuals do not get employed
after turning 22 years of age, but they are no longer eligible for this training program as they
have aged out.
This article combines vocational training in a naturalistic environment while at the same
having additional training in a simulated work environment. This correlates to our fieldwork and
research question in that if the Job Discovery Program (JDP) at Opportunity Village also
incorporates training in a simulated work environment, will the number of students being
employed at 22 years old increase after leaving JDP?
What was the purpose of the study?
The purpose of this study is to compare job-site plus simulation training for teaching job
skills to supported workers with autism to provision of training exclusively on the job (Ahearn,
What are the research questions/hypothesis?
Does combining vocational training in a naturalistic environment while at the same time
having additional training in a simulated work environment to individuals with autism increase
their ability to be successful at a part-time job position?
Does the literature review justify the need for this study?
The literature review identifies the need for individuals with disabilities to have
supported work training. But research has shown that offering exclusively supported work is not
successful for individuals with more severe disabilities. This article focuses on nonverbal
individuals on the autism spectrum who were involved in both job training in a supported work
environment as well as a simulated work environment. The literature review emphasizes the
importance of combining both simulated job training with on-site training to individuals with
severe disabilities.

What is the study design/type of study? What is the level of evidence?

The study design of this article is a Case Study - Level IV.
How many participants? all together and in each group if applicable
This study included four participants.
How were the participants recruited and selected?
These individuals were selected for the study because they all worked part-time for the
same company and were about to begin new tasks. They were employed part-time at a
publishing company where job responsibilities included clerical tasks and office cleaning. When
they werent scheduled to work, they attended an adult day program.
If applicable, how were participants assigned to groups?
The group of four individuals were divided into two groups of two. It did not discuss into
further detail as to how or why they were divided. Two of the participants, Mr. Gray and Mr.
Mack, were being trained on preparing envelopes and emptying trash cans. Mr. Jones and Mr.
Russ were being trained to prepare packing paper as well as preparing envelopes. Each of these
tasks included 18 steps.
How are the participants described demographics e.g. diagnosis, age, gender, race
All participants were nonverbal and had been diagnosed with Autism or profound
intellectual disability. The participants range in age from 29-40 years old. Mr. Mack, the
youngest participant at 29 years old, was diagnosed with Fragile X syndrome. The other
participants included Mr. Jones who was 30 years old, Mr. Russ age 32, and Mr. Gray who was
40 years old. Mr. Mack, Mr. Russ, and Mr. Jones were able to respond to vocal directions
accompanied by manual signs and gestures from the job coach. Mr. Gray had severe hearing loss
and responded to simple vocal directions.
What are the variables? Independent and dependent if applicable.
The dependent variable was the percentage of task steps (out of 18) that were performed
independently. In order to complete a task independently, the task step must be completed
correctly by the worker without any prompting.

What measures were used?

The job coach observed each of the four workers independently. He/she would set up the
environment for each participant by placing materials needed for each activity in the workers
view. This would not count as a prompt, but if the worker did not get to work within 10 seconds
and needed additional prompting, then it would not be counted as independent.
If applicable, what is the intervention?
Of the two groups of participants, one individual from each group received simulated
training in addition to work-site training. The other two individuals only received job-site
What statistical analyses were used?

Our article, Enhancing Job-Site Training of Supported Workers with Autism: A

Reemphasis on Simulation, used a multiple probe design to compare the relative effects of the
two teaching approaches. Baseline data was collected for each pair of workers on two different
work tasks. After baseline data was taken, the pair of adults that worked two days a week
received the intervention. Mr. Gray received job-site plus simulation training while Mr. Mack
received job-site training only on one task. For the second task, positions were reversed. The
second pair of adults, Mr. Jones and Mr. Russ, did not receive the same sequence of interventions
until Mr. Gray and Mr. Mack completed the intervention phases.
Probes were then taken during the routine work day on independent step completion
during the intervention phases. The data collected was then compared side by side to look for any
differences between the two training approaches. The data showed either a higher level of task
performance or more rapid progress for the worker who received job-site plus simulation
training. The differences in performance between baseline and intervention data was shown in
percentages. The researchers looked at the average amount of time that the workers met the
criterion set for increased job performance.
What are the findings?
In general, the researchers found that the two pairs of workers showed a higher level of
task performance or more rapid progress for the worker who received job-site plus simulation
training. However, there were differences noted between tasks. During the envelope-preparation
part of task one, the researchers found that the two workers that received job-site training and
simulation, Mr. Gray and Mr. Russ, increased their performance and met the 80% criterion
within three probes. Once they met the criterion, their job performance was always above the
80% criterion, averaging 88% independently completed steps during the regular job routine.
Whereas Mr. Mack who received job-site training only took eight probes to meet the criterion
and Mr. Jones who also only received job-site training was unable to meet the criterion.
However, for the second part of task one, trash removal, there was little difference
detected between job-site training and job-site training plus simulation training. Neither worker
showed consistent progress during baseline, but did complete more steps of the trash removal
task independently. They received averages of 57% and 58%, but during subsequent training
both Mr. Mack and Mr. Gray made progress and were able to meet the 80% criterion. Mr. Mack,
received job-site training plus simulation met the criterion in two probes and Mr. Gray met the
criterion in three probes even though he had only received job-site training. They continued to
perform above the criterion level during the routine work day.
Differences were also detected between job-site training plus simulation and job-site
training only during task two. Task two consisted of preparing packing paper and preparing
envelopes for mailing books. Mr. Jones was provided job-site training plus simulation and was
able to meet the criterion of independent completion of 80% of the job task analysis in four
probes. Mr. Jones continued to meet and exceed the criterion during regular job routine probes.
Mr. Russ was unable to meet the criterion during eight probes, but he had only received job-site
Do these findings support the hypothesis?

The researchers wanted to investigate and compare the effects of job-site training and jobsite training plus simulation training on the acquisition of new job skills among supported
workers with autism and severe or profound mental retardation. Due to their literature review,
they believed the research to imply that job-site training supplemented with simulation training
may be effective when teaching supported adult workers new tasks for the job-site.
The researchers findings do support their hypothesis in most cases. For both task two and
the envelope preparation part of task one, job-site training plus simulation training increased the
workers overall performance as well as the speed in which they were able to meet and exceed the
criterion of 80% independence when completing job-site task steps. The supported workers that
reached the criterion were also able to consistently outperform it during the regular work routine
after the intervention phase.
The second part of task one, trash removal, did show improvement, but baseline data was
not low to begin with. While the supported workers met and exceeded the criterion, baseline data
suggest the task in of itself is not an overly complicated task for Mr. Jones and Mr. Russ. There
was little difference in the performance of Mr. Jones who had received job-site training plus
simulation training and Mr. Russ who only received job-site training. After the intervention
phase, they both continuously performed above the 80% criterion.
How do the findings relate to previous research as described in the literature review?
Within the literature review, the authors discuss how there is evidence that supports adults
within a supported work environment rather than a sheltered work or prevocational program.
They also state that regardless of this evidence many adults with special needs do not get to
participate or benefit from a supported employment and work in sheltered settings instead. The
articles regarding supported work environments have revealed difficulties encountered by the
adults with severe disabilities and therefore call for more research examining more effective
ways to provide supports to these individuals.
As current training efforts in supported work environments focuses on job-site training,
or training in the natural environment the individual will be completing the task in, there is
evidence to support these efforts as well as discuss the failures. There also was a considerable
amount of research on training community survival skills to individuals with disabilities in
simulated situations. This set of research concluded that instruction in simulated settings with
supplemental instruction in a classroom produced generalization of skills learned when these
individuals were probed in the community. These two sets of data review was the basis for this
research article. The authors wanted to provide adult workers with disabilities simulated
instruction as well as the job-site instruction to determine if simulation instruction is an effective
Does the author state any clinical implications for the findings?
The researchers state that it seems job-site training plus simulation training is a more
effective way to train workers with severe disabilities. Workers that received job-site training
plus simulation training were able to meet the 80% independent step completion criterion
quicker than the workers that just received job-site training. However, the researchers admit that
further research on simulated training in a supported work situation is warranted. Research

including evaluation of the amount of time spent in simulation training and the amount of time
spent with on the job training.
In light of the results the researchers found at the conclusion of their own study and
results they read about during their literature review, the use of simulation training with the
promotion of generalizing skills may contribute to workers with disabilities picking up new skills
quicker. This thought and hopeful actuality is dependent on the delivery of the simulation
training and whether it is adequate.
What are the limitations that the author identifies?
The amount of time allotted for training at work was limited, whereas, time allotted for
training at the simulated center was not. Therefore, limiting the generalizability of simulated
training as one could argue if there were no time limits at work, it could potentially be just as
effective as the time constraint-free simulation activities. The sample size was a noted
limitation as well, as well as, limited diagnoses were represented in this participant pool.
Does the author discuss implications for future research?
The author does believe that continued research on simulation training with job-site
training is needed. Further investigations could research the relative amounts of simulation and
on the job training combinations to find the most effective pair. The author specifically states this
idea for future research as they did not evaluate the exclusive use of simulation training. Other
research articles, some covered in the literature review have had success with generalization of
skills with behavioral simulation training of community skills. The author believes these skills
may be trained exclusively in simulation once the skills have been identified.
What would you say about the sample size? Do you think it is adequate?
The sample size was only 4 adults with special needs working in the same facility. This
sample size is too small to generalize the results to all people with disabilities. Further research
would need to be conducted with more individuals from different job-sites throughout the
country to see if the experiment and results can be recreated.
If the researcher did not find a significant difference between the groups, is it possible that
this is due to a Type II error? If so, why do you think so?
The researchers found significant difference between all results except for one job task.
However, the task was trash removal and the researchers state this task may have been an easy
task for both Mr. Russ and Mr. Jones. As this task would have been too easy for both individuals,
baseline data was higher than with that of the other tasks, so even when the individuals met the
criterion, there was not much difference.
Is there a control or comparison group? If so, is the control or comparison group
comparable to the experimental group on key features?
There is a comparison group. Of the four participants, 2 received on-site job training
along with simulated training and the other two only received on-site job training. As previously
stated all participants were nonvocal and had been diagnosed with Autism or profound
intellectual disability. All participants were assigned tasks with similar difficulties and steps.

Are those administering the outcome measures blind to group assignment?

No, in fact this was a bias of the study. The same job coach was present for all
individuals scheduled work and schedule simulation. This was to ensure rapport was commonly
established between all participants, and to provide similar teaching patterns.
Are the participants blind to group assignment?
The article did not state if the workers not receiving the simulation training were privy to
the concept that others workers were receiving simulated instruction. However, the members of
the group that received the simulation knew they were receiving additional support for work.
Does the researcher account for drop-outs in the study? Could drop outs have influenced
the outcomes?
There was no mention of drop-outs in this study. This study would have been greatly
impacted if a participant would have dropped out, as the numbers were so few to begin with.
This research design relied heavily on the comparison factor, and had someone dropped they
would have not been able to compare adequately.
Does the researcher report reliability and validity of the outcome measures? Are there
questions about the outcome measures chosen?
The researchers do not report the reliability and validity of the chosen outcome measures.
However, they do elaborate on the process they used to develop measurable observations. They
used an interobserver agreement method, that checked the data recorded by the two observers. If
there were no discrepancies in performance, observed data was considered to be a 100%
agreement; if there were discrepancies in performance, observed data was considered a
nonoccurrence agreement. This method was not clearly defined, and it seems there could have
been room for subjectivity in data collection.
What confounding factors could contribute to or influence the study outcomes?
The same job coach was present during the research data collection; however, was not
present at all work times. This could have led to bias, as the students may have performed
differently when the particular job coach/researcher was present. The participants age ranged
from 29-40, but the paper does not mention how long these individuals have been in the
workforce. Perhaps the older individual has been working and training on similar tasks for some
years now, which could impact this research generalizability.
What are the major strengths of this study? (list 3)
The major strengths of this study are as follows:
1. This research brings to light important aspects of job training.
2. The participants have profound disabilities.
3. While not generalizable, it did support both on-site and simulated training for the
individuals, suggesting more research in this area should be pursued.
What are the major weaknesses of the study? (list 3)
The major weaknesses are as follows:
1. Limited sample size of only four participants

2. Tasks may not have been motivating to the participants. Meaning the occupation
performance could have been better or worse depending on participants interest.
3. Overall lack of generalizability to similar populations
How would you use this article as a therapist?
This article is great for therapist in that it reiterates important behavioral learning
concepts along with current techniques of on-site training. The current era of job transitioning is
more heavily focused on train and place. However, this article brings to topic the concept of
teaching and simulating job task skills in a calmer, demand-free environment. Job coaches are
often in a hard spot as they have the responsibility of teaching job tasks, along with ensuring
workers complete specified requirements by the job place supervisor. Frequently, job coaches
are forced to decide between taking longer to teach the task to workers risking workplace
dissatisfactory, or just completing the task for the worker thereby, not effectively helping the
worker learn how to complete the task. This is where OTs can be influential in training
individuals for work. We as, future OTs, can focus on the simulations, training and teaching of
skills needed to complete tasks. While OTs are overqualified to be a job coach, they can be
invaluable in the beginning teaching necessary and foundational skills to individuals with
developmental disabilities preparing to enter the workforce.
How does this article support/not support participation in occupation and the field of
occupational therapy?
There is no mention of Occupational Therapy within this paper; however, the paper
revolves around four individuals partaking in work, a meaningful instrumental activity of daily
living. This article focuses on finding the best way to train individuals in their respective
occupations. With a basic understanding of the fundamental values and foundational concepts of
Occupational Therapy, it is logical to see the correlation between this research and professions
with the OT field. As a healthcare profession, OTs strive to better the lives of their clients. OT
is unique in that it focuses on the individual and what they want, need and have to do. By
understanding this research, and similar research, OTs can be prepared and well equipped to
meet the needs of individual clients looking to pursue work and their respective learning needs.