Professional Documents
Culture Documents
Behavioral caregiver training programs are extensively used interventions, based on empirical research
and applied concepts of behavior. As these interventions become more widespread in use across settings
for various populations, the availability of efficient methods to evaluate program effectiveness is crucial.
Currently, there is a lack of widely used and psychometrically sound measurement tools to assess change
in caregivers’ knowledge of behavioral principles following participation in such training. In the current
study, we assessed change in caregivers’ knowledge of behavioral principles following participation in a
caregiver training based on the principles of Applied Behavior Analysis. Twenty caregivers participated
in the evaluation, which included pre- and post-Brief Behavioral Knowledge Questionnaire (BBKQ)
completed before and after participation in a brief behavioral training (BBT). The measurement was
found to have good internal consistency. Results also show that following participation in the BBT,
caregivers’ initial knowledge of behavioral principles increased significantly. The results suggest
that the BBKQ may be a useful tool in detecting changes in caregiver’s knowledge of behavior prin-
ciples following BBT. Limitation and future directions are discussed. Copyright © 2016 John Wiley
& Sons, Ltd.
*Correspondence to: Melissa González, Ph.D., BCBA-D, Department of Behavioral Psychology, Kennedy Krieger
Institute, Baltimore, MD 21205, USA. E-mail: GonzalezM@kennedykrieger.org
Karin Stern is now at the ABA certification program at Kibbutzim College and the Access for All Program at Tel Aviv
University. We would like to thank Maureen A. Gamache for her contribution to this paper.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 37
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
38 K. Stern and M. L. González
DeVellis (1991), in his work on scale construction, asserts that scale reliability is
fundamental to the development of such measures. An important aspect of scale reli-
ability is internal consistency. Internal consistency is the extent to which the items are
highly intercorrelated. Higher inter-item and item-scale correlations suggest that
items are measuring the same thing. Coefficient alpha, an indicator of internal consis-
tency, is the proportion of variance in the scale that is attributed to the true score.
Second, the adequacy of a scale at measuring the construct it aims to measure is a
question of a measure’s validity. One way to assess validity is to demonstrate that
scale items can differentiate between members of a group based on scale scores.
The purpose of the current study was to determine: (i) the initial scale reliability of
the Brief Behavioral Knowledge Questionnaire (BBKQ) by examining the inter-item,
item-scale correlations and the internal consistency, and (ii) to assess the initial valid-
ity of this scale in differentiating between pre-training and post-training measures of
caregiver knowledge.
METHOD
Setting
The current study was conducted in an intensive hospital-based pediatric feeding
disorders program in Baltimore, Maryland that provides services aimed to establish
appropriate feeding responses for children who present with a variety of feeding
difficulties. The program involves professionals from the fields of medicine,
nutrition/dietetics, behavior psychology, occupational therapy, speech language pa-
thology, and social work to support patient progress toward developmentally
appropriate eating. The BBT was implemented as an optional service provided to
caregivers of children admitted to the 6 to 8-week intensive pediatric feeding dis-
orders program.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 39
otherwise) are associated with different responses from caregivers such as a variation
in the amount or quality of caregiver attention provided (i.e., coaxing or comforting)
or the demands that are placed (i.e., amount or type of food presented or duration of
the meal; Piazza et al., 2003; Babbitt et al., 1994). Given the learning experiences that
take place in the meal context, it should not be surprising that similar contingencies
resulting in challenging behaviors outside of meals are also common. As such, the in-
tensive feeding program where this study was conducted provides caregivers general
training in behavioral principles via a BBT to increase positive child–caregiver inter-
actions, to promote generalization of skills across contexts, and adherence to
recommendations (Glogower & Sloop, 1976). This general behavioral training was
provided in addition to the individualized training to implement the feeding-specific
protocol (similar to methods described by Mueller et al., 2003).
PARTICIPANTS
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
40 K. Stern and M. L. González
PROCEDURE
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 41
The first session of the BBT was dedicated to presenting basic information related
to applied behavior analysis. The purpose of this session was to teach the caregivers
what behavior is and that each behavior, challenging or not, serves a purpose. Topics
taught in this session included: (i) why children engage in challenging behavior; (ii)
how to identify the function of the challenging behavior; (iii) the four main functions
behavior: attention, escape, access to tangible and automatic/sensory stimulation; (iv)
the Antecedent–Behavior–Consequence (A–B–C) context of behavior; (v) how to
operationally define challenging behaviors; (vi) what is reinforcement; (vii) what is
punishment, and (viii) how to design an effective reward system (e.g., immediacy,
frequency, magnitude, and satiation). Specific examples related to the meal sessions
as well as outside of meal context (e.g., developmental playroom) were provided.
First caregivers were presented with a short lecture, and then they engaged in inter-
active activities so that they could practice newly learned skills (i.e., visuals
depicting everyday situations were presented and caregivers were asked to identify
the function of the behavior). In addition, caregivers were provided with copies of
A–B–C data collection forms and explanation of how to collect data using this
format.
The second session was designed to teach caregivers how to address challenging
behaviors that are maintained by access to attention. Main emphasis during this
session was on: (i) recognizing common triggers for challenging behaviors that are
maintained by access to attention; (ii) identifying common consequences; (iii) teach-
ing effective antecedent interventions such as child-directed play and non-contingent
attention; and (iv) providing examples of effective interventions that could decrease
inappropriate behavior by using differential attention for appropriate behavior (e.g.,
providing specific praise as a strategy to increase appropriate behavior) and
minimizing attention (e.g., effective strategies for using active ignoring).
The third session was devoted to explaining how to address behaviors maintained
by negative reinforcement in the form of escape. During this session, methods for in-
creasing compliance as well as guidelines for effective instruction were reviewed.
Caregivers were also provided with a flow chart explaining the three-step guided
compliance prompting sequence (i.e., least-to-most prompting). A role-play was also
utilized so that caregivers could practice and receive immediate feedback and
coaching. During the role-play, caregivers were presented with a variety of prompts
and were asked to practice the use of the three-step guided compliance sequence to
complete each of the tasks.
The fourth (last) session was dedicated to teaching caregivers how to address
behaviors that are maintained by positive reinforcement in the form of access to pre-
ferred toys/activities (tangible function). First, antecedent interventions and effective
consequences that could be helpful in decreasing tangible-maintained challenging
behaviors were reviewed. Next, information regarding effective strategies for using
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
42 K. Stern and M. L. González
time-out (e.g., how to transition to timeout, what to do during timeout, and how to
end a timeout) was reviewed and demonstrated.
During all sessions, information was disseminated and practiced via verbal instruc-
tion and examples, role-play, interactive activities, and distribution of written
materials related to the topic of each session. The caregivers were also provided with
the third edition of ‘SOS Help for Caregivers’ book (Clark, 2005), and specific chap-
ters were assigned at the end of each session. Caregivers were encouraged to share
experiences, questions, and concerns from their own interactions with their children
during the meeting. Additional individualized training sessions and consultation
was provided as needed to address individualized caregiver needs. However, this
additional consultation occurred following participation in the training group and
completion of the BBKQ.
MEASURE
BBKQ
Caregivers completed a pre- and post-questionnaire as a measure of their knowl-
edge of the training content. The BBKQ was composed of 15 questions, and was
developed from an initial pilot questionnaire. The pilot version of the questionnaire
included 15 yes/no questions related to common behavioral functions. Each of the
yes/no questions was presented as a scenario related to a child engaging in a problem
behavior. Each scenario described the setting/antecedents and behavior of concern.
Caregivers were asked to indicate whether a possible caregiver response would or
would not be appropriate in order to decrease the problem behavior. Correct re-
sponses indicate a healthy contingency for the behavioral function targeted by the
question (e.g., planned ignoring for an attention seeking inappropriate behavior).
These questions were generated by the authors based on common childhood chal-
lenging behavior and caregiver responses (e.g., tantrum while caregiver is on the
phone, tantrum while in store, refusal to comply with picking up toys). The questions
were reviewed and revised by four behavior analysts as well as two staff persons
working directly with young children in the hospital setting. Finally, the questions
were trialed with four caregivers who provided feedback (who were not part of the
pilot sample). Thirty-six caregivers completed the pilot version; 20 of them com-
pleted both pre- and post-questionnaires. Items were then analyzed by examining
the percentage of participants who answered the item correctly at pre-test. There
was a ceiling effect (i.e., >90% of participants answering correct at pre-test) noted
for seven items. Six of these items were removed and one item was revised based
on caregiver feedback on the pilot questionnaire. Six multiple-choice questions were
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 43
added related to the social determinants of behavior (information presented in the first
training session).
The final BBKQ was composed of six multiple-choice questions related to the
social determinants of behavior and 9 yes/no questions related to three behavioral
functions: escape, attention, and tangible. Data analysis and results described below
solely refer to the revised version of the questionnaire (see Appendix A).
DATA ANALYSIS
Data analysis was conducted using Statistical Package for the Social Sciences
(SPSS). An important step in scale construction and evaluation is examining the ex-
tent to which items are appropriate for the assessment measure in question. Therefore,
an item analysis of the measurement was conducted via inter-item and item-scale cor-
relations. As the correlation among the items increases, the reliability of the
individual items and the scale as a whole also increase (DeVellis, 1991). To ensure
that the items discriminate among individuals, the item-scale mean, variance, and in-
ternal consistency were examined. Good estimates of internal consistency are those
with alph ≥ .80 (Clark & Watson, 1995; Cicchetti, 1994).
In order to determine the sample size needed, an a priori power analysis was con-
ducted using G*Power 3 software (Faul, Erdfelder, Lang, & Buchner, 2007). For a
paired samples t-test, the specified parameters included: two-tailed test, large effect size
of 0.80, Type 1 error probability of α = 0.05, and power of 0.80. Based on these param-
eters, a sample size of 16 was required. The final sample included 20 participants.
The BBKQ was analyzed in the following manner: a score of 1 was assigned to each
response considered correct. A score of 0 was assigned for incorrect answers, selection
of more than one response of the multiple choice questions, and missing answers.
Change in knowledge was evaluated by calculating the difference between post-
test and pre-test scores on the BBKQ. Individual data was analyzed via visual inspec-
tion. Group data was analyzed by a paired sample t-test in SPSS and calculation of
the effect size.
RESULTS
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
44 K. Stern and M. L. González
1 1.00 .19 .22 .25 .28 .04 .04 .20 .30 .03 .10 .03 .06 .23 .11
2 1.00 .04 .04 .00 .10 .16 .04 .10 .04 .27 .13 0.01 .14 .13
3 1.00 .10 .14 .35 .51 .48 .12 .08 .20 .32 .02 .51 .15
4 1.00 .21 .46 .15 .35 .30 .20 .05 .17 .01 .17 .13
5 1.00 .05 .32 .48 .30 .07 .07 .17 .02 .05 .24
6 1.00 .18 .17 .17 .15 .03 .11 .04 .18 .18
7 1.00 .48 .50 .25 .17 .65 .47 .26 .50
8 1.00 .33 .02 .18 .22 .07 .38 .21
9 1.00 .40 .13 .50 .41 .24 .65
10 1.00 .41 .26 .54 .19 .43
11 1.00 .33 .07 .38 .11
12 1.00 .45 .28 .51
13 1.00 .31 .45
14 1.00 .19
15 1.00
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 45
Figure 1. Graph depicting raw changes in caregivers’ knowledge (in bars), individual participants pre-
and post-scores (in open and closed data paths), and group’s average pre- and post-scores (in dashed
lines).
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
46 K. Stern and M. L. González
DISCUSSION
The purpose of the current study was to evaluate the initial psychometric properties
of a brief behavioral questionnaire on changes in caregiver knowledge following
BBT. Initial psychometrics calculated for the measurement showed good internal
consistency (Kline, 2000). In addition, results from a paired-sample t-test showed a
significant increase in caregivers’ knowledge (with a large effect) following partici-
pation in BBT, suggesting this may be a useful measure for detecting changes in
caregivers’ knowledge following training.
Behavioral training administrators should systematically examine changes in
knowledge of training participants, to evaluate the effectiveness of their interven-
tion program. Prinz and Sanders (2007) and Foster, Prinz, Sanders, and Shapiro
(2008) call for an increased use of behavioral caregiver training interventions
given the empirical support for these interventions and the potential consequences
of the failure to intervene on a child’s social, emotional, and behavioral function-
ing. With a wider use of caregiver training across settings and the likelihood of
adaptations to these trainings, effective and efficient methods of monitoring train-
ing effectiveness are needed (Shapiro, Prinz, & Sanders, 2015). Currently, a valid
measurement of caregivers’ knowledge of behavioral principles is not available.
The measure proposed in this study can potentially be used to assess a variety
of behavioral training programs and target populations. Systematic detection of
changes in caregivers’ knowledge can offer additional support for the usefulness
of behavioral training and inform clinicians of the effectiveness of their interven-
tion. Based on the results of this study, this brief questionnaire shows adequate
internal consistency and sensitivity to caregiver’s change in knowledge as a result
of training. Additional studies will be needed to further demonstrate the psy-
chometric properties of this measurement of caregivers’ knowledge of behavioral
principles.
More work will be needed to examine additional psychometric properties includ-
ing other forms of reliability, validity, and generalizability of the measurement to
other populations, such as caregivers of children with other forms of challenging be-
havior who do not present with feeding problems, and with other behavioral
caregiver training programs. The design used in the current study lacks a control
group, thus we were unable to account for any changes in caregivers’ knowledge that
may be a result of other ongoing intervention provided to caregivers during their
child’s admission to the hospital. While participation in the BBT and completion of
BBKQ typically preceded any feeding-specific caregiver training (involving methods
described by Mueller and colleagues, 2003), further investigation is needed to
conclude whether the measured changes in knowledge can be contributed solely to
participation in the brief behavioral caregiver training.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 47
The current study demonstrated that this measure detects immediate changes in
caregivers’ knowledge following training. Additional work is needed to determine
if this is a valid and efficient measure of knowledge over longer periods of time. In
addition, future studies should determine the effectiveness of this measure in moni-
toring short and long-term treatment outcomes of behavioral caregiver training. It
should also be determined if changes in knowledge are related to changes in care-
givers’ behaviors/interactions in the presence of their children and to what extent a
change in knowledge is a prerequisite or an essential consideration for long-term
changes in caregiver-child interactions. Given the call to employ behavioral caregiver
training to larger masses (Shapiro, Prinz, & Sanders, 2015), effective, efficient, and
low-cost measures of change resulting from behavioral caregiver trainings are
essential to inform clinicians of the effectiveness of their intervention.
REFERENCES
Allen, S. L., Smith, I. M., Duku, E., Vaillancourt, T., Szatmari, P., Bryson, S., & Georgiades, S. (2015).
Behavioral pediatrics scale in young children with autism spectrum disorder: Psychometrics and asso-
ciations with child and parent variables. Journal of Pediatric Psychology, 40, 581–590. doi:10.1093/
jpepsy/jsv006.
Allen, K. D., & Warzak, W. J. (2000). The problem of parental nonadherence in clinical behavior anal-
ysis: Effective treatment is not enough. Journal of Applied Behavior Analysis, 33, 373–391.
doi:10.1901/jaba.2000.33-373.
Babbitt, R. L., Hoch, T. A., & Coe, D. A. (1994). Behavioral assessment and treatment of pediatric
feeding disorders. Journal of Developmental Behavioral Pediatrics, 15, 278–291. doi:10.1097/
00004703-199408000-00011.
Baillargeon, R. H., Morisset, A., Keenan, K., Normand, C. L., Séguin, J. R., Japel, C. & Cao, G. (2012).
Development of disruptive behaviors in young children: A prospective population-based cohort study.
Infant Mental Health Journal, 33, 633–650. doi:10.1002/imhj.21353.
Baker, B. L., Heifetz, L. J., & Murphy, D. M. (1980). Behavioral training for parents of mentally
retarded children: One-year follow-up. American Journal of Mental Deficiency, 85, 31–38 Retrieved
from http://psycnet.apa.org/psycinfo/1980-28450-001.
Barlow, J., Smailagic, N., Ferriter, M., Bennett, C., & Jones, H. (2010). Group-based caregiver-training
programmes for improving emotional and behavioral adjustment in children from birth to three years
old. Cochrane Database of Systematic Reviews, (3). doi:10.1002/14651858.CD003680.pub2.
Beernink, A. C., Swinkels, S. H. N., & Buitelaar, J. K. (2007). Problem behavior in a community
sample of 14- and 19-month-old children: Common versus uncommon behaviors, structure, and
stability. Journal of European Child & Adolescent Psychiatry, 16, 271–280. doi:10.1007/s00787-
007-0600-9.
Blum, N. J., & Friman, P. C. (2000). Behavioral pediatrics: The confluence of applied behavior analysis
and pediatric medicine. In J. Austin, & J. E. Carr (Eds.), Handbook of applied behavior analysis.
Reno, Nevada: Context press Retrieved from http://eds.a.ebscohost.com/ehost/detail/detail?
vid=37&sid=51dea812-48be-4717-a249-6443cd4382d7%
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
48 K. Stern and M. L. González
40sessionmgr4003&hid=4205&bdata=JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=2001-
16421-007&db=psyh.
Borrego, J., & Burrell, T. L. (2010). Behavioral parent training to treat disruptive behavior in young
children: A how-to approach using video clips. Cognitive and Cognitive and Behavioral
PracticeBehavioral Practice, 17, 25–34. doi:10.1016/j.cbpra.2009.10.003.
Brennan, L. M., Shaw, D. S., Dishion, T. J., & Wilson, M. (2012). Longitudinal predictors of school-age
academic achievement: unique contributions of toddler-age aggression, oppositionality, inattention
and hyperactivity. Journal of Abnormal Child Psychology, 40, 1289–1300. doi:10.1007/s10802-
012-9639-2.
Briesmeister, J. M., & Schaefer, C. E. (1998). Handbook of parent training. New York: Parents as Co-
Therapists for Children’s Behavior Problems Retrieved from http://eds.a.ebscohost.com/ehost/detail/
detail?vid=42&sid=51dea812-48be-4717-a249-6443cd4382d7%40sessionmgr4003&hid=4205&-
bdata=JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=1997-36836-000&db=psyh.
Brightman, R. P., Baker, B. L., Clark, D. B., & Ambrose, S. A. (1982). Effectiveness of alternative
parent training formats. Journal of Behavior Therapy and Experimental Psychiatry, 13, 113–117.
doi:10.1016/0005-7916(82)90051-9.
Bulotsky-Shearer, R. J., Dominguez, X., Bell, E. R., Rouse, H. L., & Fantuzzo, J. W. (2010). Relations
between behavior problems in classroom social and learning situations and peer social competence in
head start and kindergarten. Journal of Emotional and Behavioral Disorders, 18, 195–210.
doi:10.1177/1063426609351172.
Calhoun, S. L., Fernandez-Mendoza, J., Vgontzas, A. N., Liao, D., & Bixler, E. O. (2014). Prevalence of
insomnia symptoms in a general population sample of young children and preadolescents: Gender
effects. Sleep Medicine, 15, 91–95. doi:10.1016/j.sleep.2013.08.787.
Caspi, A., & Moffitt, T. E. (1995). The continuity of maladaptive behavior: From description to under-
standing in the study of antisocial behavior. In Developmental psychopathology, Vol. 2: Risk, disorder,
and adaptation (pp. 472–511), Wiley series on personality processes. Oxford, England: John Wiley &
Sons Retrieved from http://eds.a.ebscohost.com/ehost/detail/detail?vid=45&sid=51dea812-48be-
4717-a249-6443cd4382d7%40sessionmgr4003&hid=4205&bdata=JnNpdGU9ZWhvc3QtbGl2ZSZ-
zY29wZT1zaXRl#AN=1995-97697-013&db=psyh.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized
assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-
3590.6.4.284.
Clark, L. (2005). SOS: Help for parents (3rd ed.). Kentucky, USA: SOS Programs & Parents Press.
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issue in scale development. Psycholog-
ical Assessment, 7, 309–319. doi:10.1037/1040-3590.7.3.309.
Daly, D. L., Friman, F. C., Larzelere, R. E., Smith, G., Osgood, D. W., & Thompson, R. W. (1997).
Educating out-of-home adolescents: A longitudinal analysis of length of stay in residential care.
Boys Town, NE: Boys Town National Research Institute for Child and Family Studies (Tech.
Rep. No. 010-97).
DeVellis, R. F. (1991). Scale development: Theory and applications. Newbury Park, California: Sage
Publications Retrieved fromhttp://eds.b.ebscohost.com/ehost/detail/detail?vid=14&sid=5b76ae2e-
2bec-45ab-b6d3-34bc69075ed2%40sessionmgr104&hid=122&bdata=JnNpdGU9ZWhvc3QtbGl2Z-
SZzY29wZT1zaXRl#AN=1991-98125-000&db=psyh.
Dunlap, G., Strain, P. S., Fox, L., Carta, J. J., Conroy, M., Smith, B. J., Kern, L., … (2006).
Prevention and intervention with young children’s challenging behavior: Perspectives regarding
current knowledge. Behavioral Disorders, 32, 29–45 Retrieved fromhttp://www.jstor.org/stable/
23890617.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 49
Equit, M., P lmke, M., Becker, N., Moritz, A., Becker, S., & von Gontard, A. (2013). Eating problems in
young children—A population-based study. Acta Paediatrica, 102, 149–155. doi:10.1111/apa.12078.
Eyberg, S. M., Boggs, S. R., & Algina, J. (1995). Parent–child interaction therapy: A psychosocial
model for the treatment of young children with conduct problems and their families. Psycho-
pharmacology Bulletin, 31, 83–91 Retrieved fromhttp://eds.b.ebscohost.com/ehost/detail/detail?
vid=24&sid=5b76ae2e-2bec-45ab-b6d3-34bc69075ed2%40sessionmgr104&hid=122&bdata=JnNpd-
GU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=1996-12228-001&db=psyh.
Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial treatments for
children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent
Psychology, 37, 215–237. doi:10.1080/15374410701820117.
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power
analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods,
39, 175–191. doi:10.3758/BF03193146.
Feil, E. G., Frey, A., Walker, H. M., Small, J. W., Seeley, J. R., Golly, A., & Forness, S. R. (2014). The
efficacy of a home-school intervention for preschoolers with challenging behaviors: A randomized
controlled trial of Preschool First Step to Success. Journal of Early Intervention, 36, 151–170.
doi:10.1177/1053815114566090.
Feldman, M. A., & Werner, S. E. (2002). Collateral effects of behavioral caregiver training on families
of children with developmental disabilities and behavior disorders. Behavioral Interventions, 17,
75–83. doi:10.1002/bin.111.
Foster, E. M., Prinz, R. J., Sanders, M. R., & Shapiro, C. J. (2008). The cost of a public health infrastruc-
ture for delivering parenting and family support. Children and Youth Services Review, 30, 493–501.
doi:10.1016/j.childyouth.2007.11.002.
Furtkamp, E., Giffort, D., & Schiers, W. (1982). In-class evaluation of behavior modification
knowledge: Parallel tests for use in applied settings. Journal of Behavioral Therapy and Experimental
Psychiatry, 13(2), 131–134. doi:10.1016/0005-7916(82)90054-4.
Glogower, F., & Sloop, E. W. (1976). Two strategies of group training of parents as effective behavior
modifiers. Behavior Therapy, 7, 177–184. doi:10.1016/S0005-7894(76)80273-0.
González, M. L., & Stern, K. (2015). Co-occurring behavioral difficulties in children with severe feeding
problems: A descriptive study. Manuscript in preparation.
Greer, A. J., Gulotta, C. S., Masler, E. A., & Laud, R. B. (2008). Caregiver stress and outcomes of
children with pediatric feeding disorders treated in an intensive interdisciplinary program. Journal
of Pediatric Psychology, 33, 612–620. doi:10.1093/jpepsy/jsm116.
Gregory, A. M., Caspi, A., Eley, T. C., Moffitt, T. E., O’Connor, T. G., & Poulton, R. (2005). Prospec-
tive longitudinal associations between persistent sleep problems in childhood and anxiety and depres-
sion disorders in adulthood. Journal of Abnormal Child Psychology, 33, 157–163. doi:10.1007/
s10802-005-1824-0.
Helfer, M. E., Kempe, R. S., & Krugman, R. D. (1997). The battered child (5th ed., rev. & exp.).
Chicago, IL, US: University of Chicago Press Retrieved fromhttp://eds.a.ebscohost.com/ehost/de-
tail/detail?vid=53&sid=51dea812-48be-4717-a249-6443cd4382d7%40sessionmgr4003&hid=4205&-
bdata=JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=1997-36259-000&db=psyh.
Jansen, P. W., Roza, S. J., Jaddoe, V. W. V., Mackenbach, J. D., Hein, R., Albert, H., Verhulst, F. C., …
(2012). Children’s eating behavior, feeding practices of parents and weight problems in early child-
hood: Results from the population-based Generation R study. The International Journal of Behavioral
Nutrition and Physical Activity, 9, 1–11. doi:10.1186/1479-5868-9-130.
Johnson, C. R., Turner, K., Stewart, P. A., Schmidt, B., Shui, A., Macklin, E., & Courtney, P. M.
(2014). Relationships between feeding problems, behavioral characteristics and nutritional quality
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
50 K. Stern and M. L. González
in children with ASD. Journal of Autism and Developmental Disorders, 1–10. doi:10.1007/s10803-
014-2095-9.
Kaminski, J. W., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of compo-
nents associated with caregiver training program effectiveness. Journal of Abnormal Child Psychol-
ogy, 36, 567–589. doi:10.1007/s10802-007-9201-9.
Kline, P. (2000). The handbook of psychological testing (2nd ed.). London: Routledge.
Lavigne, J. V., Bryant, F. B., Hopkins, J., & Gouze, K. R. (2015). Dimensions of oppositional defiant
disorder in young children: Model comparisons, gender, and longitudinal invariance. Journal of
Abnormal Child Psychology, 43, 423–239. doi:10.1007/s10802-014-9919-0.
Lichtenstein, P., & Annas, P. (2000). Heritability and prevalence of specific fears and phobias in
childhood. Journal of Child Psychology and Psychiatry, 41, 927–937. doi:10.1111/1469-7610.00680.
Lundahl, B., Risser, H. J., & Lovejoy, M. C. (2006). A meta-analysis of caregiver training:
Moderators and follow-up effects. Clinical Psychology Review, 26, 86–104. doi:10.1016/j.
cpr.2005.07.004.
Matson, J. L., Mahan, S., & LoVullo, S. V. (2009). Caregiver training: A review of methods for children
with developmental disabilities. Research in Developmental Disabilities, 30, 961–968. doi:10.1016/j.
ridd.2009.01.009.
Maughan, D. R., Christiansen, E., Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent
training as a treatment for externalizing behaviors and disruptive behavior disorders: A meta-
analysis. School Psychology Review, 34, 267–286 Retrieved from http://search.proquest.com/
docview/219654976?accountid=11752.
McClintock, K., Hall, S., & Oliver, C. (2003). Risk markers associated with challenging behavior in
people with intellectual disabilities. Journal of Intellectual Disability Research, 47, 405–416.
doi:10.1046/j.1365-2788.2003.00517.x.
McKillop, J. (1994). A brief instrument to assess knowledge of behavioral principles. Journal of Behav-
ioral Therapy and Experimental Psychiatry, 25(3), 251–253. doi:10.1016/0005-7916(94)90025-6.
Mueller, M. M., Piazza, P. C., Moore, J. W., Kelley, M. E., Bethke, S. A., Pruett, A. E., & Layer, S. A.
(2003). Training parents to implement pediatric feeding protocols. Journal of Applied Behavior Anal-
ysis, 36, 545–562. doi:10.1901/jaba.2003.36-545.
O’Dell, S. L., Tarler-Benlolo, L., & Flynn, J. M. (1979). An instrument to measure knowledge of behav-
ioral principles as applied to children. Journal of Therapy and Experimental Psychiatry, 10, 29–34.
doi:10.1016/0005-7916(79)90033-8.
Piazza, C. C., Fisher, W. W., & Brown, K. A. (2003). Functional analysis of inappropriate mealtime
behaviors. Journal of Applied Behavior Analysis, 36, 187–204. doi:10.1901/jaba.2003.36-187.
Postorino, V., Sanges, V., Giovagnoli, G., Fatta, L. M., De Peppo, L., Armando, M., Vicari, S., …
(2015). Clinical differences in children with autism spectrum disorder with and without food selectiv-
ity. Appetite, 92, 126–132. doi:10.1016/j.appet.2015.05.016.
Prinz, R. J., & Sanders, M. R. (2007). Adopting a population-level approach to parenting and family
support interventions. Clinical Psychology Review, 27, 739–749. doi:10.1016/j.cpr.2007.01.005.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation—A systematic approach (7th ed.).
Thousand Oaks: Sage Publications.
Sanders, M. R., Kirby, J. N., Tellegen, C. L., & Day, J. J. (2014). The triple P-positive parenting
program: A systematic review and meta-analysis of a multi-level system of parenting support. Clinical
Psychology Review, 34, 337–357. doi:10.1016/j.cpr.2014.04.003.
Sandler, I. N., Schoenfelder, E. N., Wolchik, S. A., & MacKinnon, D. P. (2011). Long-term impact of
prevention programs to promote effective parenting: Lasting effects but uncertain processes. Annual
Review of Psychology, 62, 299–329. doi:10.1146/annurev.psych.121208.131619.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 51
Schmitt, B. D. (1987). Seven deadly sins of childhood: Advising parents about difficult developmental
phases. Child Abuse and Neglect, 11, 421–432. doi:10.1016/0145-2134(87)90015-9.
Shapiro, C. J., Prinz, R. J., & Sanders, M. R. (2015). Sustaining use of an evidence-based parenting
intervention: Practitioner perspectives. Journal of Child and Family Studies, 24, 1615–1624.
doi:10.1007/s10826-014-9965-9.
Smidts, D. P., & Jaap, O. (2007). How common are symptoms of ADHD in typically developing
preschoolers? A study on prevalence rates and prenatal/demographic risk factors. Cortex: A Journal
Devoted to the Study of the Nervous System and Behavior, 43, 710–717. doi:10.1016/S0010-9452
(08)70500-8.
Tellegen, C. L., & Sanders, M. R. (2013). Stepping stones triple P-positive parenting program for
children with disability: A systematic review and meta-analysis. Research in Developmental
Disabilities, 34, 1556–1571. doi:10.1016/j.ridd.2013.01.022.
Verlinden, M., Jansen, P. W., Veenstra, R., Jaddoe, V. W. V., Hofman, A., Verhulst, F. C., & Henning,
T. (2015). Preschool attention-deficit/hyperactivity and oppositional defiant problems as antecedents
of school bullying. Journal of the American Academy of Child & Adolescent Psychiatry, 54, 571–579.
doi:10.1016/j.jaac.2015.05.002.
Wichstrøm, L., Berg-Nielsen, T. S., Angold, A., Egger, H. L., Solheim, E., & Sveen, T. H. (2012).
Prevalence of psychiatric disorders in preschoolers. Journal of Child Psychology and Psychiatry,
53, 605–705. doi:10.1111/j.1469-7610.2011.02514.x.
Wierson, M., & Forehand, R. (1994). Parent behavioral training for child noncompliance: Rationale,
concepts and effectiveness. Current Directions in Psychological Science, 3, 146–150. doi:10.1111/
1467-8721.ep10770643.
Zucker, N., Copeland, W., Franz, L., Carpenter, K., Keeling, L., Angold, A., & Egger, H. (2015).
Psychological and psychosocial impairment in preschoolers with selective eating. Pediatrics, 136,
e582–2590. doi:10.1542/peds.2014-2386.
Dear caregiver,
This short questionnaire assesses the quality and usefulness of the materials and
methods used during the parent group. Thank you for your cooperation!
Please circle the number beside the best answer:
1. Majority of problem behaviors young children present are probably:
1. A reflection of deeper emotional problems.
2. Result of disorders such as ADHD and autism.
3. May be accidentally taught by the child’s family.
4. Because of to a stage which the child will outgrow.
2. When should a child who is just learning to wash his hands be praised the first
time?
1. When he first opens the faucet.
2. When he finish washing his hands.
3. When he asks to do it himself without assistance.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
52 K. Stern and M. L. González
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin
Brief Behavioral Knowledge Questionnaire 53
Johnny’s mom wants the crying to stop, what do you think she should do?
7. Should mom hang up the phone and go see what happened? YES / NO
8. Should mom ignore the crying and continue talking on the phone? YES / NO
9. Should mom hang up the phone and go talk to Johnny, explain to him that he
is a big boy and he should be quiet and let mommy talk on the phone? YES /
NO
4 year-old Hanna and her dad are going to the supermarket. As usual, Hanna gets
very excited, and wants to buy different candies. When her dad says no, she immedi-
ately bursts in to tears, and goes into a very loud temper tantrum in the middle of the
store. Many shoppers stare at the little girl and her dad.
What do you think Hanna’s dad should do?
10. Should dad leave the store and put Hanna in time-out? YES / NO
11. Should dad ignore her temper tantrum (and the other shoppers) and continue
shopping? YES / NO
12. Should dad quickly pick Hanna up and hug her, explain that daddy loves her
but candy is bad for her health, and also screaming is not nice because it gets
the other people really worried? YES / NO
David’s dad walks into 5-year-old David’s room and sees that his toys are all over
the room. He calls David to come back to his room and pick up his toys. David says:
‘I don’t want to pick them up; I like it just the way it is.’
What do you think David’s dad should do?
13. Should dad say: ‘This is unacceptable, pick your toys on the count of three or
I will send you to time-out! 1, 2, 3…’ and put David in time-out if he does
not comply? YES / NO
14. Should dad say: ‘David you need to pick up your toys’, and show him how to
pick them up. If David is not picking the toys up independently, should dad
physically guide him to pick up his toys? YES / NO
15. Should dad immediately explain to David why it is important to clean up af-
ter he’s done playing, and that he thinks he is a big boy that should be
responsible for keeping his toys in place each time it happens? YES / NO
* Item 2 was removed from the questionnaire following the initial item analysis.
Copyright © 2016 John Wiley & Sons, Ltd. Behav. Intervent. 32: 35–53 (2017)
DOI: 10.1002/bin