Issues in Mental Health Nursing

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/imhn20

Examination of Telemental Health Practices in

Caregivers of Children and Adolescents with
Mental Illnesses: A Systematic Review

Zümra Ülker Dörttepe & Zekiye Çetinkaya Duman

To cite this article: Zümra Ülker Dörttepe & Zekiye Çetinkaya Duman (2022) Examination
of Telemental Health Practices in Caregivers of Children and Adolescents with Mental
Illnesses: A Systematic Review, Issues in Mental Health Nursing, 43:7, 625-637, DOI:
10.1080/01612840.2021.2013366

To link to this article: https://doi.org/10.1080/01612840.2021.2013366

Published online: 12 Jan 2022.

Submit your article to this journal

Article views: 509

View related articles

View Crossmark data

Citing articles: 2 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=imhn20
Issues in Mental Health Nursing
2022, VOL. 43, NO. 7, 625–637
https://doi.org/10.1080/01612840.2021.2013366

Examination of Telemental Health Practices in Caregivers of Children and

Adolescents with Mental Illnesses: A Systematic Review
Zümra Ülker Dörttepe, PhD Candidatea and Zekiye Çetinkaya Duman, PhDb
a
Medical Services and Techniques Department, Vocational School of Health Sciences, Uşak University, Uşak, Turkey; bNursing Faculty,
Psychiatric Nursing Department, Dokuz Eylül University, İzmir, Turkey

ABSTRACT
In this systematic review, effects of telemental health (TMH) practices’ on caregivers of children/
adolescents with mental illnesses were investigated. The literature review included databases, and
reference lists of published studies. All studies published until September 2021 were reviewed.
Eleven studies were included. Several services were provided via TMH: education, cognitive
behavioral therapy (CBT), parent training, caregiver behavior training, family CBT. The effectiveness
of TMH interventions on caregivers varied from low to high. Most reproducible findings were on
caregivers’ satisfaction, stress, therapeutic alliance and caregiver burden. Studies had a low to high
bias risk. Most studies had small samples. Results built on the small but growing literature support
TMH interventions’ promising role in caregivers of children with mental illness. Future studies
should estimate outcomes with medium to low effect size. Other caregiver groups, rarely considered
in previous studies, should be included. Bias risk should be minimized. Larger, more methodologically
rigorous studies should be conducted.

Introduction The use of TMH with caregivers of children with mental

Most mental disorders that start in childhood can lead to
the loss of function over time, and can continue in adulthood
(National Institue of Mental Health, 2021). Most of the chil-
dren and adolescents with mental illnesses cannot sufficiently
meet and maintain their needs, and thus they need a care-
giver. Caregivers face various obstacles and suffer stress while
performing these roles (Chang et al., 2020; Mendenhall &
Mount, 2011). Therefore, they benefit from professional sup-
port and assistance (American Psychological Association,
2016; Dalrymple et al., 2020). Providing such support is dif-
ficult in many countries and rural communities due to the
lack of mental health professionals. Telemental health (TMH),
is an evolving approach that support caregivers who cannot
readily access traditional services (Pruitt et al., 2014).
TMH is a digital health solution that connects the patient,
caregiver and healthcare provider through real-time audio
and video technology, and can be used as an alternative to
traditional in-person care delivery (American Medical
Association, 2020; Centers for Medicare & Medicaid Services,
2020). Through TMH interventions, individuals with a men-
tal illness and their caregivers can receive services such as
remote monitoring and support, counseling, consultation,
and therapy (Myers et al., 2017; Uslu et al., 2019; World
Health Organization, 2011). TMH applications can be car-
ried out individually or in groups (Doyen et al., 2018; Haley
et al., 2011; Ozkan et al., 2013).
illnesses has not been widely reported (McHugh & Barlow,
2010; Mendenhall & Mount, 2011). However, individuals
face many barriers to accessing care, including distance,
provider shortage, issues of transportation, stigma and cost
(Comer & Myers, 2016; Jefee-Bahloul, 2014; Madigan et al.,
2021). In addition, for families in rural areas or for those
who live far away from specialty clinics, access to health
care is significantly lacking (Strasser et al., 2016). With
TMH, caregivers can participate in real-time services con-
ducted by experts, regardless of their geographic proximity
to an expert in a mental health facility. In addition, TMH
offers an avenue to meet some of the emotional, behavioral,
and developmental needs of caregivers by expanding assess-
ment and treatment options to underserved populations in
urban and rural areas (Hepburn et al., 2016; Nelson et al.,
2017). Therefore, TMH interventions make mental health
services available to caregivers in their own communities
and helping caregivers helps them and their children (Chou
et al., 2016; Dalrymple et al., 2020).
There has been no comprehensive systematic review on
TMH interventions regarding the caregivers of children and
adolescents with mental illnesses. A systematic review of
the limited available studies to be conducted within this
context will help to plan future empirical studies testing the
effectiveness of TMH interventions with caregivers. The
present study reports a critical review of the available

CONTACT Zümra Ülker Dörttepe zumra.dorttepe@usak.edu.tr Medical Services and Techniques Department, Vocational School of Health Sciences, 1
Eylül Kampüsü MA3 Blok Kat:1 İzmir Yolu 8.Km. 64200 Merkez, Uşak University, Uşak, Turkey.
© 2022 Taylor & Francis Group, LLC
626 Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN
studies. Research question created in line with the purpose
of the systematic review is as follows: "What are the effects
of TMH interventions on the caregivers of children and
adolescents with a mental illness?”
Materials and methods
Research strategy
While this systematic review was conducted, the researchers,
utilized the “Centers for Reviews and Dissemination 2009”
guidelines developed by York University National Health
Research Institute (Center for Reviews & Dissemination,
2009). Literature screening methods, eligibility criteria of the
studies and the results to be examined were determined
beforehand. In the literature review, databases, reference lists
of published studies and expert opinions were used. Databases
of PubMed, CINAHL, Cochrane, Google Scholar, EBSCHO,
Clinical Trials, Centers for Reviews and Dissemination and
ProQuest were screened using the X University Library service
provider, and all studies published until September 2021 were
examined. The reviews were carried out by the researchers
individually, and after each researcher reviewed all the studies,
they all agreed on the findings. The keywords determined as
“telemedicine, videoconference, mental disorders, child, ado-
lescent, caregivers, parents” were selected from the MeSH
(Medical Subject Headings) index.
Inclusion and exclusion criteria
The review included full-text articles of experimental or
quasi-experimental studies written in English. The inclusion
criteria were determined in accordance with the PICO (P:
population, I: intervention, C: comparison, O: outcome)
questions. According to this, the studies in which:
• participants were the caregivers of children or ado-
lescents with a mental illness,
• at least one of the psychosocial interventions (i.e.,
parent training, interventions on behavioral modi-
fication, teaching skills, social-communication skills
interventions, therapies, cognitive behavioral thera-
pies) was performed by TMH,
• the intervention was performed by videoconferenc-
ing (VC) technology (synchronous and performed
individually or in group),
• TMH interventions where assessments or measure-
ments about caregivers were made were included in
the systematic review.
Qualitative and descriptive studies were not included in
the systematic review.
Results
The studies accessed (N = 1,955,363) after the screenings
were analyzed in terms of their titles, abstracts and con-
tents. Studies complying with the purpose of the study
were determined (n = 40). Twenty-nine studies did not meet
the inclusion criteria of this review, because they were
reviews too (n = 3), they were mentioned in an abstract
for a congress, with not enough details to evaluate the
outcome of the study (n = 4), they were duplicated studies
(n = 2), they were study protocols (n = 3), they did not meet
the inclusion criteria (n = 3), were case reports (n = 4) and
they did not assess TMH effects in caregivers (n = 10).
Articles which met the inclusion criteria were analyzed
within the scope of the systematic review (n = 11). The
selection process of the studies as given in Figure 1, the
characteristic features of the studies examined are presented
in Table 1. The content of the features given in Table 1
is presented in the Results section of the article in detail
whereas the characteristics of the psychosocial interventions
applied in the studies are presented as sample sizes, loca-
tion and date of the studies, information about the care-
giver, technology considerations and the evaluation of
bias status.
Characteristics of the included studies
The main characteristics of the included studies are reported
in Table 1. In six studies, a randomized controlled research
design was employed and in the other five studies, a
quasi-experimental design was utilized. While in two of the
quasi-experimental design studies a controlled design was
used, in three of them an uncontrolled design was used. In
this section, where the characteristics of the studies are
analyzed in detail, the interventions applied with the TMH
method, the sample characteristics of the studies, the num-
ber and duration of sessions held in the studies, the mea-
surement tools used and the times of the measurements
(pretest/posttest/retest) are presented.
Parent training programs
Six studies were parent training programs which lasted
8-12 sessions held once a week. One of them had eight
sessions performed with the Group Triple P Positive
Parenting Program, provided by VC technology (Reese
et al., 2012). Families with a child diagnosed with Attention
Deficit Hyperactivity Disorder (ADHD) attended the pro-
gram. The study was aimed at assessing differences between
pre- and post-measurements in parenting self-efficacy and
parent depression, anxiety and stress. Changes in parenting
self-efficacy were assessed with the Being a Parent Scale
(BPS; Johnston & Mash, 1989) and, parent depression,
anxiety and stress were evaluated with the Depression
Anxiety Stress Scales-21 (DASS-21; Lovibond &
Lovibond, 1995).
Another study (Xie et al., 2013) was a 10 sessions of
weekly group parent training conducted through either VC
or in-person delivery using a manualized treatment based
on that of Barkley (Barkley, 2013). Parents of children with
ADHD between 6 and 14 years of age were recruited. At
the baseline and end of the program, participants were also
tested on parent-child relations with the Parent Child
Relationship Questionnaire for Child and Adolescent

Figure 1. Identification and selection of the studies.
(PRQ-CA; Kamphaus & Reynolds, 2006), and the child’s
social skills with the Social Skills Rating System (SSRS;
Elliott & Gresham, 1987). In addition, parents’ perceptions
of VC and the training program were evaluated after the
program with the Likert scale created by the authors.
The other study was a group Internet-delivered
Parent-Child Interaction Therapy (I-PCIT) comparing
VC to remotely deliver behavioral parent training at the
home setting with a therapist (Comer et al., 2017a).
Racially/ethnically diverse children aged 3-5 years with
disruptive behaviors and their caregivers participated in
the study comparing I-PCIT to standard clinic-based
PCIT. Major assessments were conducted at the baseline,
mid-treatment, posttreatment and 6-month follow-up.
Assessments included caregiver burden measured with
the Eyberg Child Behavior Inventory-problem score for
caregivers (ECBI; Eyberg & Pincus, 1999), barriers to
treatment measured with the Barriers to Treatment
Participation Scale (BTPS; Kazdin et al., 1997), and treat-
ment satisfaction with Client Satisfaction Questionnaire
(CSQ-8; Larsen et al., 1979) and Therapy Attitude
Inventory (TAI; Brestan et al., 1999).
Issues in Mental Health Nursing 627
Marino et al.’s (2020) study was a 12-session tele-assisted
and in-person comparing group parent training intervention
for caregivers of children aged between 30 months and
10 years with Autism Spectrum Disorder (ASD) based on
applied behavior analysis treatment (ABA). The study was
carried out in three phases and the results of the phase
three were published. This phase included tele-assisted and
in-person intervention results. Major assessments were made
before and after the phase three with the Home Situations
Questionnaire (HSQ-ASD; Chowdhury et al., 2016)and PSI.
The study of the Breaux and their colleagues (2021) was
an eight sessions of group-based social-emotional interven-
tion (i.e., Regulating Emotions Like An eXpert (RELAX)
program) targeted to improve emotion dysregulation, inter-
personal conflict, teaching adolescent and caregivers coping,
communication, conflict management and emotion social-
ization skills. Racially/ethnically diverse adolescents aged
11-16 years with ADHD and their caregivers participated in
the study comparing in-person and telehealth groups. The
effect of intervention performed within the scope of the
study on caregiver emotion dysregulation and
parent-adolescent conflict using the Difficulties in Emotion
Table 1. Characteristics of the studies included. 628

Reference Study Design N (E/C) Caregivers Intervention Instruments Follow-up Results

Reese et al., Quasi-experimental 8 • Age (M = 33.8) • Group Triple P Positive • BPS Pre-post • Parents’ stress decreased after intervention (d=-0.34).
2012 (USA) study (single • Female (71.4%) Parenting Education • DASS-21 • Parents’ self-efficacy improved.
group design) • Low socio-economic
status (100 %)
Xie et al., 2013 RCT 22 (9/13) • Male (68%)/Female (32%) • Experimental group: • PRQ-CA Pre-post • The difference between the VC and in-person groups was
(USA) • Education level Group parenting • SSRS not significant (p = 0.25).
(M = 14.5 year) program • LS • Discipline subscale of parenting relationship significantly
• Control group: In-person increased after training (p = 0.006, ES = 0.93).
group parenting • Parent-rated social skills showed an improvement (p = 0.13,
program ES = 0,57).
• The Likert scale of parent perceptions of the VC and the
training program showed no group difference (p = 0.675).
Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN

Tse et al., 2015 RCT 37 (12/25) • Education level (59.4% • Experimental group: • PHQ Pre-post • Caregivers showed comparable satisfaction with the two
(USA) some college and high Group caregiver • PSI treatment modalities.
school) behavior training • CSQ-ADHD • Caregivers’ parenting stress in the in-person group
• Underserved • Control group: In-person • CGSQ improved pre-to post-intervention (p < 0.05).
communities group-based caregiver • FES • Caregivers in the in-person group supported better
behavior training outcomes regarding caregiver strain and empowerment
(p < 0.05).
Hepburn et al., Quasi-experimental 33 (17/16) • Only mothers • Experimental group: • PSOC Pre-post • Parents’ sense of competence statistically changed pre- to
2016 (USA) study (control • Education level (57.5% Multiple family group post-intervention (p = 0.03).
group design) some college and high CBT • Satisfaction with the intervention content, delivery method
school) • Control group: No and alliance with therapist was high (92.9%).
• Urban area (15%)/Rural intervention
area (85%)
Comer et al., RCT 22 (11/11) • Maternal age (M = 38.1)/ • Experimental group: • CSQ-8 Pre-post-post • Mothers’ satisfaction in both group was high after
2017b (USA) Paternal age (M = 39.6) Group-based family CBT • FAS-PR (6 months intervention.
• Education level (90.9% • Control group: In-person later) • Family accomodation increased in both group with high
completed college) group-based family CBT effect sizes (d = 1.38 and d = 1.41).
• High socio-economic
status (63.6%)
Comer et al., RCT 40 (20/20) • Maternal age (M = 37.5) • Experimantal group: • ECBI-problem Pre-post-post • In the study, treatment satisfaction was high in both
2017a (USA) • Low socio-economic I-PCIT score for (6 months groups (Minternet = 30.1 versus Mclinic = 28.5) and the TAI
status (56%) • Control group: caregivers later) (Minternet = 45.9 versus Mclinic = 45.1).
clinic-based PCIT • CSQ-8 • Caregivers in I-PCIT had less barriers about treatment
• (Group-based) • TAI participation than clinic-based PCIT [F(3.29)=7.62,
• BTPS p<.001); βcondition = 8.40, p=.01, Minternet = 50.1 versus
Mclinic = 58.5].
• Caregivers’ burden from pre-to-post and from pre-to-follow
up were all “large” to “very large” for I-PCIT and “medium”
to “very large” for clinic-based PCIT
Sibley et al., Quasi-experimental 20 • 1 father/19 mother • STAND therapy (i.e., • Likert Scale Pre-post • Parents’ perceived importance (M = 8.55, SD = 1.91),
2017 (USA) study • Education level (55% individual family therapy) for confidence (M = 8.95, SD = 1.64), and readiness (M = 7.90,
(single group Master’s degree or motivation SD = 1.83) were strong at baseline and did not change
design) higher) to change significantly over time.
• Marital status (80% • TBS • Parents perceived therapeutic alliance was strong at
married/20% single) • PAMS baseline (M = 3.53, SD=.39) and did not change over time.
• Urban area (100%) • STAND-SQ • Parents’ academic management results indicated a
significant positive linear effect of time for frequency of
parent homework checks ((baseline M = 1.56, SD = 1.31, b =
.17, SE = .06, p = .008, d= .98), and contingent provision
of home privileges (baseline M = .74, SD = .82, b = .22, SE
= .06, p < .001, d = 2.41).
• Satisfaction of parents was well above the neutral point of
the scale (M = 3.92, SD=.68).
Carpenter et al., Quasi-experimental 13 • Only mothers • Group-based TMH-FCBT • BTPS Pre-post-post • Mothers’ barriers to TMH-FCBT participation were very low
2018 (USA) study (2/4/7) • Age (M = 44.2) • WAI (3 months (M = 50.1, SD = 6.5).
(multiple baseline • Education level (38.4% • CSQ-8 later) • Mothers’ alliance ratings was very high across treatment
design) complete college/46.1% • FACLIS (M = 245.6, SD = 5.9).
graduate degree) • Mothers’ satisfaction was very high across the treatment
• High socio-economic (M = 30.7, SD = 2.4).
status (69.2%), urban • Significant reductions in interference related to parental
area (100%) accomodation were observed from Baseline II to
• Marital status (77% posttreatment (Cohen’s d =-2.17), and from Baseline II to
married/23% divorced or 3-month follow-up (Cohen’s d =-2.12).
not married)
Marino et al., RCT 74 • 23 female, 19 male • Experimental group: • HSQ-ASD Pre-post • Tele-assisted group had significant improvement in parental
2020 (37/37) Group tele-assisted • PSI stress (p<.001).
(Italy) parent training • Post-tests total scores of the HSQ-ASD decreased by 20%.
• Control gorup: Group • Control group had no significant improvement over time.
in-person parent training
Breaux et al., Quasi-experimental 32 • 25 mothers, six fathers, • Experimental group: • DERS Pre-post • There were no significant differences about caregiver’s
2021 study (control (18/14) one grandmother Group-based telehealth • CCNES emotion dysregulation between groups (η2=.03).
(USA) group design) • Age range (33-66 years) RELAX program • CBQ-44 • Caregiver and adolescent conflict did not improve
• Education level (59.4% • Control group: In-person significantly (η2 =0.13 and η2=0.03) across the entire
Bachelor’s degree or group-based RELAX sample at pre/post measures.
higher/25% high program
school/15.6% Associate’s
degree or some college)
• Single family home
(62.70%)
Comer et al., RCT 40 • 29 female, 11 male • Experimental group: • DASS-21 Pre-post-post • Folow-up tests found that family accommodation was
2021 (20/20) • Age (M = 38.0) group-based İCALM • CSQ-8 (6 month significant in the prediction of improvements in parental
(USA) • Etnicity/race (65% • Control group: No • FASA follow-up) depression (t=-2.17, p=.03, B=-.04), and parental stress
ethnic/racial minority/ intervention (after study (t=-2.03, p=.046, B=-.06).
35% Non-Hispanic enrolled to İCALM) • Caregiver’s satisfaction was high (Score: 29.9) after the
Caucasian treatment (highest possible scale score 32.0).
• Medium socio-economic
status (42.5%)
N: Sample Size, E: Experimental Group, C: Control Group, TMH: Telemental health intervention. RCT: Randomized controlled trial, CBT: Cognitive Behavioral Therapy, PCIT: Parent- Child Interaction
Therapy, I-PCIT: Internet-delivered Parent-Child Interaction Therapy, STAND: Supporting Teens’ Autonomy Daily, TMH-FCBT: Telemental Health-delivered Family Cognitive Behavioral Therapy, RELAX:
Regulating Emotions Like An eXpert program. OCD: Obsessive-compulsive disorder, ADHD: Attention Deficit and Hyperactivity Disorder, N/A: Not available, ES: Effect Size. BPS: Being a Parent Scale,
DASS-21: Depression Anxiety Stress Scales-21, PRQ-CA: Parent Child Relationship Questionnaire, SSRS: Social Skills Rating System, LS: Likert Scale created by authors, PHQ: Patient Health Questionnaire,
PSI: Parenting Stress Index, CSQ-8: Client Satisfaction Questionnaire, CSQ-ADHD: Client Satisfaction Questionnaire, CGSQ: Caregiver Strain Questionnaire, FES: Family Empowerment Scale, PSOC:
Parenting Sense of Competence Scale, Working Alliance Inventory, FAS-PR: Family Accommodation Scale-Parent Report, ECBI: Eyberg Child Behavior Inventory-problem score for caregivers, TAI: Therapy
Attitude Inventory, BTPS: Barriers to Treatment Participation Scale, TBS: Therapist Bond Scale, PAMS: Parent Academic Management Scale, STAND-SQ: STAND satisfaction questionnaire, BTPS: Barriers
to Treatment Participation Scale, WAI: Working Alliance Inventory, FACLIS: Family Accommodation Checklist and Interference Scale, DERS: Difficulties in Emotion Regulation Scale, CCNES: Children’s
Negative Emotions Scale, CBQ-44: Conflict Behavior Questionnaire, FASA: Family Accommodation Scale-Anxiety, HSQ-ASD: Home Situations Questionnaire.
Issues in Mental Health Nursing
629
630 Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN
Regulation Scale (DERS; Gratz & Roemer, 2004) and Conflict
Behavior Questionnaire (CBQ-44; Prinz et al., 1979)
respectively.
The other study (Comer et al., 2021) was a 12 session
group-based family-based behavioral parenting intervention
(i.e., Internet Coaching Approach Behavior and Leading by
Modeling (İCALM Telehealth Program)) made with care-
givers of children (3-8 years) with a diagnosis social anxiety
disorder who were randomly assigned ICALM or waitlist.
Of the major assessments regarding the intervention, care-
givers’ negative affect was measured with the DASS-21
(Lovibond & Lovibond, 1995), treatment satisfaction was
measured with the CSQ-8 (Larsen et al., 1979) and
family accommodation was measured with the Family
Accommodation Scale–anxiety.
Caregiver behavior training programs
The study of Tse et al. (2015) consists of six sessions per-
formed at 3-4 weeks intervals in 25 weeks. The study is a
manualized group-based caregiver behavior training inter-
vention based on the reviews of the evidence for treating
youth with ADHD (Chronis et al., 2006). In addition, the
study is a sub-study of a larger clinical trial (Children’s
ADHD Telemental Health Treatment Study (CATTS); Myers
et al., 2015). Caregivers of children with ADHD (5.5-12 years
of age) in the current study consisted of 37 of the 223
participants in the CATTS. This component of the study
was conducted through either VC or in-person delivery
caregiver behavior training. At the baseline and end of the
program, the caregivers were tested on parenting distress
(Parenting Stress Index; Abidin, 1983), depression (Patient
Health Questionnaire; Wittkampf et al., 2009), caregiver
strain (Caregiver Strain Questionnaire; Brannan et al., 1997),
family empowerment (Family Empowerment Scale; Koren
et al., 1992) and caregivers’ acceptability of, or satisfaction
with services they received and their perception of the cli-
nician’s understanding of their children’s treatment needs
(Client Satisfaction Questionnaire-ADHD; Attkisson &
Zwick, 1982).
Therapies
Another study which included an interactive multi-family
therapy group was conducted 1-h per week and lasted
10 weeks (Hepburn et al., 2016). The study is a pilot study
of a telehealth version of the Facing Your Fears (FYF;
Table 2. Risk of bias according to the Cochrane risk of bias tool.
Xie et al., Tse et al., Comer
Type of risk of bias 2013 2015 et al., 2017 et al., 2020
Selection bias (Random sequence High risk High risk Low risk
generation)
Selection bias (Allocation concealment) High risk Unclear risk Low risk
Performance bias (Blinding of Unclear risk Low risk Low risk
participants and Assessors)
Attrition bias (Incomplete outcome Unclear risk Unclear risk Low risk
data)
Reporting bias (Selective reporting) Low risk Low risk Low risk
Other bias (Anything else) Low risk Low risk Low risk
The risk of bias has been expressed as the number of bias items presented by each study
Reaven et al., 2011), a manualized, evidenced-based psy-
chosocial intervention for anxiety in youth with Autism
Spectrum Disorder (ASD). Young people with ASD (between
the ages of 7 and 19 years) and their parents were recruited.
In the VC FYF sample, 17 families from rural and frontier
communities participated in the study and 16 families were
selected for the wait-list of the FYF program for the com-
parison group. The study is is aimed at investigating the
feasibility and efficacy of the VC FYF program. Pre-to
post-changes in anxiety symptoms in youth were examined
with the parent report of the Screen for Anxiety and Related
Emotional Disorders in Children (SCARED; Birmaher et al.,
1999), and parent self-efficacy was examined with the
Parenting Sense of Competence Scale (PSOC; Johnston &
Mash, 1989).
The study of Comer et al. (2017b) was an RCT compar-
ing VC-delivered family-based CBT (FB-CBT; Freeman &
Garcia, 2008) versus clinic-based FB-CBT in the treatment
of children aged 4-8 years with obsessive-compulsive disorder
(OCD) (N = 22). Parents are trained as coaches for their
children. Internet-delivered FB-CBT follows the same
14-week program as clinic-based FB-CBT. Pretreatment,
posttreatment and 6-month follow-up assessments were
included. Parents’ measurements made on satisfaction at
posttreatment were evaluated with the CSQ-8 (Larsen et al.,
1979) and family accommodation at pre-, post- and 6-month
follow-ups were evaluated with the Family Accommodation
Scale-Parent Report (FAS-PR; Flessner et al., 2011).
The study of Sibley et al. (2017) was an uncontrolled
pilot evaluation of Supporting Teens Autonomy Daily
(STAND; Sibley et al., 2016b) delivered via VC to 20 ado-
lescents with ADHD and their parents. STAND consists of
ten 60-min manualized family therapy sessions attended by
the parent and teenager. Assessments made on caregivers
were as follows: (1) therapeutic alliance at post-treatment
with Therapist Bond Scale (Shirk & Saiz, 1992), (2) moti-
vation to change at post-treatment with a Likert-type Scale
created by authors, (3) frequency of parents monitor, assist,
and reinforce adolescent skills at pre- and post-treatments
with the Parent Academic Management Scale (PAMS; Sibley
et al., 2016a), (4) parents’ treatment satisfaction at
post-treatment using a satisfaction questionnaire developed
for STAND (Sibley et al., 2016b).
In the study of Carpenter et al. (2018) which was a pilot
quasi-experimental study of TMH-Family-based CBT (FCBT;
Howard et al., 2000) included 13 anxious youth aged 7 to
14 years and their mothers. They used multiple-baseline
Marino Comer Comer
et al., 2017 et al., 2021 Low Unclear High
High risk Low risk Unclear 33% 17% 50%
High risk Low risk High risk 33% 17% 50%
Unclear Low risk Low risk 67% 33% 0%
High risk Low risk Low risk 50% 33% 17%
Low risk Low risk Low risk 100% 0% 0%
Low risk Low risk Low risk 100% 0% 0%
and as a percentage across all the included studies.

design in which the participants were randomly assigned to
one of three baseline intervals to clarify changes occurred
after the initiation of the treatment, and not during the
baseline interval. After Baseline II, families received 16 ses-
sions of TMH-FCBT. The intervention was delivered in
weekly, 1-hour sessions for 16 weeks through VC. Major
assessments were conducted at mid-treatment, posttreatment
and 3 month follow-ups with Barriers to Treatment
Participation Scale (BTPS; Kazdin et al., 1997), Working
Alliance Inventory, Parent-Report (WAI; Horvath &
Greenberg, 1989), CSQ-8 (Attkisson & Zwick, 1982), and
Family Accommodation Checklist and Interference Scale
(FACLIS; Thompson-Hollands et al., 2014).
Sample sizes
The sample size of the studies ranged from 8 to 74. Of the
11 studies reviewed, 4 were pilot studies. Randomized stud-
ies contained the largest number of participants (range =
22-74), followed by uncontrolled studies (range = 8-33).
Dropout rates fluctuated from 0% to 56.7% in randomized
studies, and from 0% to 15% in uncontrolled studies. The
total number of the participants in the 11 studies was 341.
In most of the studies (n = 7), child/adolescent and caregiver
groups were included together.
Location and date
The studies included in the review were conducted between
2012 and 2021. The majority of them (n = 7) were published
in 2017 and beyond. Nearly all of them were conducted in
the USA (n = 10), except for one study carried out in Italy.
Caregivers
In most of the studies reviewed within the scope of the
present study, the characteristics of the caregivers were not
given in detail. In other words, while in some studies, data
on caregivers’ characteristics such as age, sex, socio-economic
status, and place of residence were included, in others, they
were not given in detail. In the present study, the sampling
characteristics and demographic features of the studies were
examined in detail and given in Table 1. Most of the care-
givers (6/11) in the reviewed studies were mothers. The
mean age of the participants varied between 33.8 and
61 years. Most of the studies in which information about
education level of caregivers was given, most of them had
college or high school education (4/7). In addition, in two
studies (Hepburn et al., 2016; Reese et al., 2012), TMH
interventions were carried out with caregivers living in rural
areas and underserved areas.
Technological considerations
In eight of the studies reviewed, VC sessions were held in
the caregivers’ homes (Breaux et al., 2021; Carpenter et al.,
Issues in Mental Health Nursing 631
2018; Comer et al., 2017a; 2017b; 2021; Hepburn et al.,
2016; Marino et al., 2020; Sibley et al., 2017). VC sessions
were also held in the university conference room (Xie et al.,
2013), public schools located in the caregivers’ places of
residence (Reese et al., 2012), regional medical centers
(Reese et al., 2012) and local clinics (Tse et al., 2015). The
equipment required for the VC was provided for the care-
givers within the scope of the project in all the studies,
except for three studies (Breaux et al., 2021; Carpenter
et al., 2018; Comer et al., 2021; Xie et al., 2013). Technology
disturbances stemmed from a range of sources: mostly not
being able to hear clearly (Hepburn et al., 2016), internet
service delivery difficulties (Hepburn et al., 2016; Reese
et al., 2012; Sibley et al., 2017), equipment difficulties
(Carpenter et al., 2018; Hepburn et al., 2016; Sibley et al.,
2017) and time delays for the onset of sessions (Hepburn
et al., 2016; Sibley et al., 2017). In addition, situations
arising from the home environment (for example, having
sessions in the family room at home, incoming phone calls,
interruptions by other family members) were also stated as
problems (Hepburn et al., 2016; Sibley et al., 2017).
Risk of bias
The Cochrane Collaboration’s tool for assessing risk of bias
in randomized trials was used for the evaluation of this
stage (Higgins et al., 2011). Overall, the randomized con-
trolled studies had a low to high risk of bias (Table 2). The
randomization procedure was rarely described. Two studies
only provided details on allocation concealment. The
blinding of participants and assessors was claimed but never
tested, except in Tse et al.’s (2015) study, for the participants
only, and in Comer et al.’s (2017a; 2017b; 2021) studies who
also tested the experimenter. All studies reported data by
group according to the randomization, but it was unclear
whether there were missing outcome data. In many cases,
outcomes were presented in the graphic form, or the authors
only provided statistical tests and p values, which prevented
the numerical summarization of the outcomes.
Other biases
No study provided a priori power analysis. No study had a
sample size large enough to detect a priori medium effect
size, which might be detected still by chance in small sample
studies. Except the studies of Xie et al. (2013), Comer et al.
(2017a; Comer et al., 2017b) and Marino et al. (2020) none
reported any information on the psychometric properties of
the tools used to assess the outcomes. The occurrence of
dropouts was specified in the studies of Xie et al. (2013),
Comer et al. (2017a; Comer et al., 2017b; 2021), Sibley et al.
(2017), and Carpenter et al. (2018).
Effect of interventions
Sometimes studies had multiple outcomes, which unfortu-
nately, were too disparate to allow a systematic review of
the findings. In line with the purpose of the study, this
632 Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN
section includes the measurement results related to caregiv-
ers and effects of the interventions implemented with TMH.
Parent training programs effects
In six studies, the effects of parenting program on parenting
self-efficacy, stress, parent-child relationship, caregiver’s emo-
tion dysregulation, caregiver and adolescent conflict and
children’s social skills were evaluated. Reese et al. (2012)
found a statistically significant reduction in parent distress
(d = −0.34) after the intervention. Xie et al. (2013) found
no statistical change between the VC and in-person groups.
Parent-rated social skills of children statistically increased
after the intervention (p = 0.13, Effect Size = 0.57). In Comer
et al.’s (2017) study, the effects of I-PCIT and PCIT were
evaluated. In the study, treatment satisfaction was high in
both groups (CSQ-8: Minternet = 30.1 versus Mclinic = 28.5
and TAI: Minternet = 45.9 versus Mclinic = 45.1). The parents
treated with I-PCIT perceived fewer barriers to their treat-
ment participation then did the parents treated with
clinic-based PCIT. In addition, caregivers’ burden from
pre-to-post and from pre-to-follow up were all “large” to
“very large” for I-PCIT and “medium” to “very large” for
clinic-based PCIT. In the other six studies caregiver behavior
training, CBT and other therapies were examined. Marino
et al. (2020) evaluated tele-assisted and in-person ABA based
parent training. Both groups’ measures were evaluated sep-
arately. The tele-assisted group made significant improve-
ment in parental stress (p<.001) and the total scores of
HSQ-ASD decreased by 20%. However, there was no sig-
nificant change in the control group over time. In Breaux
et al.’s (2021) study, effects of in-person and telehealth
RELAX programs were evaluated. There were no significant
differences about caregiver’s emotion dysregulation between
the groups (η2=.03). Caregiver and adolescent conflict did
not improve significantly (η2 =0.13 and η2=0.03) across the
entire sample at pre/post measures. However, caregiver
report of conflict from pre- to post- treatment was signifi-
cantly different between in-person and telehealth groups
(F = 20.91, p < .001). Comer et al. (2021) evaluated ICALM
program on caregiver’s depression, anxiety, stress and inter-
vention satisfaction at pre/post and 6-month follow-ups. In
addition, at baseline parental accommodation was evaluated
and linked with pre/post and 6-month follow-up measure-
ments. ICALM was particularly effective in reducing life
impairments and parental stress among families with higher,
relative to lower, levels of baseline parental accommodation.
In the follow-up tests, it was found that family accommo-
dation was significant in the prediction of improvements in
parental depression (t=-2.17, p=.03, B=-.04), and parental
stress (t=-2.03, p=.046, B=-.06). The caregivers’ satisfaction
was high (Score:29.9) after the treatment.
Caregiver behavior training programs effects
Tse et al. (2015) evaluated the effects of caregiver behavior
training on acceptability of the treatment, caregivers’ distress,
strain and family empowerment. The caregivers showed
comparable satisfaction with the care received through the
two treatment modalities (VC and in-person group).
Caregivers in the in-person group reported improved scores
pre-to post-intervention on the parenting stress, caregiver
strain, and empowerment (p<.05).
Therapies effects
Two studies evaluated the effects of CBT on caregivers.
Hepburn et al. (2016) found statistically significant change
between pre- and post-treatments in terms of parenting
sense of competence (p = 0.03). In addition, parents’ satis-
faction with the VC FYF intervention content, delivery
method and alliance with therapist were high. In the study
of Comer et al. (2017), mothers in both groups (VC and
in-person) reported very high satisfaction at posttreatment.
In addition, family accommodation statistically increased
in the internet-delivered FB-CBT and in-person FB-CBT
with high effect sizes (d = 1.38 and d = 1.41, respectively).
In Sibley et al.’s (2017) pilot study, the effects of STAND
were evaluated. In the results of the study, parents’ per-
ceived importance (M = 8.55, SD = 1.91), confidence
(M = 8.95, SD = 1.64), and readiness (M = 7.90, SD = 1.83)
were strong at the baseline and did not change significantly
over time. Therapeutic alliance was strong at the baseline
(M = 3.53, SD=.39) and did not change over time. Parents’
academic management results indicated a significant posi-
tive linear effect of time for frequency of parent homework
checks ((baseline M = 1.56, SD = 1.31, b=.17, SE=.06, p=.008,
d=.98), and contingent provision of home privileges (base-
line M=.74, SD=.82, b=.22, SE=.06, p<.001, d = 2.41).
Satisfaction of parents was well above the neutral point of
the scale (M = 3.92, SD=.68). Another was Carpenter et al.’s
(2018) study in which the effects of TMH-FCBT on moth-
ers’ satisfaction with the intervention, therapeutic alliance
with the therapist, parental accommodation and barriers to
TMH-FCBT participation were evaluated. In this
multiple-baseline study, Baseline II to posttreatment changes
are interpreted as treatment-related changes. In addition,
maintenance of treatment-related changes was assessed by
comparing the mean values obtained at the Baseline II and
3-month follow-up. The mothers’ perceived barriers to
TMH-FCBT participation were very low (M = 50.1, SD = 6.5).
Therapeutic alliance (M = 245.6, SD = 5.9) and satisfaction
(M = 30.7, SD = 2.4) were very high across the treatment.
Significant reductions in interference related to parental
accommodation were observed from Baseline II to post-
treatment (Cohen’s d=–2.17), and from Baseline II to
3-month follow-up (Cohen’s d=–2.12).
Discussion
In the present study, the authors’ aim was to investigate the
effects of TMH interventions on caregivers of children/
adolescents with mental illnesses. Our study results demon-
strated that TMH interventions had positive effects on the
caregivers of children/adolescents with mental illnesses, and
provided information on the application process and pro-
cedure of TMH interventions in caregivers. The effects of

TMH interventions applied to caregivers in studies examined
within the scope of the present study were positive.
Compared with in-person psychosocial interventions, the
effectiveness of TMH interventions on caregivers varied
from low to large effect size. Most findings were of low and
medium effect size. Most reproducible findings were on
caregivers’ satisfaction (Carpenter et al., 2018; Comer et al.,
2017a; 2017b; 2021; Hepburn et al., 2016; Sibley et al., 2017;
Tse et al., 2015), stress (Comer et al., 2021; Marino et al.,
2020; Reese et al., 2012; Tse et al., 2015), therapeutic alliance
(Carpenter et al., 2018; Sibley et al., 2017) and caregiver
burden (Comer et al., 2017b; Tse et al., 2015). Effects of
studies conducted on TMH in the literature were as follows:
(1) opportunity to access care and continuity in access to
care, and positive outcomes in the patient and caregiver
(Hilty et al., 2013), (2) validity and reliability (Nelson et al.,
2017) and (3) the implementation of similar interventions
as in in-person interventions (Gloff et al., 2015). Whether
TMH provides the same effect as in-person therapies do is
constantly being considered (Jenkins-Guarnieri et al., 2015).
In the present study, in six of the 11 studies examined,
in-person interventions and TMH interventions were com-
pared, and in the other five studies, only the effects of the
applied TMH interventions were evaluated. In comparative
studies, the effects of TMH interventions in caregivers were
determined as more significant or similar. Only in Tse et al.’s
(2015) study, in-person therapy yielded results which were
more effective. Explanations regarding the cause of this
situation were not specified in the study. It is thought that
confounding factors such as not homogenous distribution
of the experimental and control groups and not examining
the effect of this situation may have had an effect. The
effects of other single-group TMH interventions were also
positive in the caregivers in the pre- and post-comparisons.
The risk of bias was low to high in many studies. No
study was large enough to detect small effect sizes.
Generalization of the evidence was impossible because of
the heterogeneity of the cumulating results. The quality of
the studies may be judged moderate at the most.
Methodological limitations in study designs and small sam-
ple sizes contributed to the lower quality. Only six of the
11 studies were RCTs. There was some indication of impre-
cision in the findings (no effect size could be calculated for
some outcomes), and publication bias was possible. Effect
sizes were rarely large, and confounding factors (sex, age)
were rarely taken into account. There is a general need for
further investigation on the topic, but the findings are
promising.
More than half of the studies mentioned in the present
study included children/adolescents and caregivers altogether.
In addition, the fact that some of the studies reviewed were
randomized controlled studies indicated that studies with
high evidence level could be conducted with TMH. The
sizes of the samples indicate that the studies were conducted
with a fewer number of participants compared to traditional
treatments. In the literature, there are TMH studies con-
ducted with larger samples including adults and children
with mental illnesses (Luo et al., 2020; Luxton et al., 2016).
Issues in Mental Health Nursing 633
Among the factors affecting this finding are that the use of
TMH interventions in children/adolescents with mental ill-
nesses and their caregivers is at an early stage (Hilty et al.,
2015), that the number of mental health professionals work-
ing in this area is not many (Nelson et al., 2017; Rebello
et al., 2014), that a health system addressing both children/
adolescents and their families is not available (Comer &
Barlow, 2014), and that people adhere to the traditional
methods (Rebello et al., 2014). Therefore, conducting a great
number of experimental studies on TMH with larger samples
in the future, and measuring the effects of TMH on care-
givers will contribute to the developments in this area.
It was determined that the characteristics of the caregivers
were not presented in detail in most of the studies reviewed.
In most of the studies [except for Reese et al. (2012), Xie
et al. (2013), Marino et al. (2020) and Comer et al.’s (2021)
studies], TMH interventions were applied to both the chil-
dren with mental illnesses and their caregivers. Therefore,
there is a need for studies in which caregivers are only/also
included. The responsibility of providing care for a child
with a mental illness is usually assumed by mothers
(Mendenhall & Mount, 2011; Möller-Leimkühler & Wiesheu,
2012). The fact that most of the participants in the present
study were mothers supports this case. In addition, most of
the caregivers in the present study were in the middle age
and older group. These individuals who are considered as
"Generation Y" are more familiar with new technologies
because they were born in an age when the use of computers
and internet became widespread (Dimock, 2019). Therefore,
as stated in the literature (Belangee & Griffith, 2020; Hilty
et al., 2013), this situation is thought to affect the high
levels of satisfaction and acceptance regarding TMH. In
addition, most of the participants in the reviewed studies
are caregivers living in urban areas. However, in Hepburn
et al. (2016) and Reese et al. (2012) studies, caregivers who
lived in rural areas and could not receive enough health
care were included as well. TMH studies yielded positive
results on these groups. The achievement of such a result
is promising for this group. Quite a number of individuals
cannot receive mental health services worldwide, especially
in developing and underdeveloped countries (Jefee-Bahloul,
2014; Strasser et al., 2016). These groups, who were disad-
vantaged from several aspects, could be enabled to access
mental health services with TMH. In this respect, TMH
interventions provide great advantages (e.g. provision of the
necessary assistance in environments with the most prob-
lems, elimination of location problems, being economical,
reducing stigma) for groups who cannot access mental
health services (Pruitt et al., 2014).
The contribution of technological developments to the
field of TMH is quite high. In our review, while the number
of technological problems encountered in the studies con-
ducted in the early years was higher (Hepburn et al., 2016),
the number of such problems experienced in recent studies
is fewer (Comer et al., 2017a; Sibley et al., 2017). Most of
the TMH implementations were carried out in caregivers’
homes (Breaux et al., 2021; Carpenter et al., 2018; Comer
et al., 2017a; 2017b; 2021; Hepburn et al., 2016; Marino
634 Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN
et al., 2020; Sibley et al., 2017), and in places such as local
medical centers, schools or universities where the Internet
infrastructure is better (Reese et al., 2012; Tse et al., 2015;
Xie et al., 2013). Especially in the early days of TMH inter-
ventions, studies were carried out outside home. Because
the technological infrastructure and internet network con-
nections in the past years were not as wide as they are
today, the implementation of TMH interventions was per-
formed in these environments (Howard & Mazaheri, 2009).
In most of the reviewed studies, the necessary equipment
was provided within the scope of the project, which con-
tributed to technological problems less. In Hepburn et al.’s
(2016) study which included many participants especially
from rural areas, detailed examination of technological pos-
sibilities leading to positive results showed that TMH inter-
ventions would provide an advantage in terms of the
continuity of the health care that these groups should
receive. Caregivers’ Internet and technology usage charac-
teristics also affect the implementation of TMH interven-
tions. Such characteristics of the participants were investigated
only in Sibley et al.’s (2017) study, and then orientation
training on this issue was given. Such applications can be
further expanded in future studies on this group, and TMH
can be more beneficial. In addition, gradual increases in
technological developments, which have a great part in every
moment of individuals’ lives, and expansion of internet net-
works will yield better results in TMH interventions in the
coming years.
In our review, it was observed that CBT and parent train-
ing program were used more as a TMH intervention in
caregivers. In this respect, interventions similar to traditional
treatments were used in children with mental illnesses and
their caregivers (American Academy of Pediatrics, 2019). In
addition, the number of sessions, and the duration and
implementation of TMH interventions appear to be similar
to those of traditional treatments. Therefore, these inter-
ventions are likely to be implemented via TMH. The analysis
of the time intervals of the studies reviewed demonstrate
that more studies have been performed in recent years.
Because the general implementation and dissemination of
TMH have also gained momentum in recent years, the fact
that its use in the caregiver group has increased recently is
not surprising. However, caregivers are often overlooked in
both TMH interventions and family interventions. This is
probably due to the low number of mental health profes-
sionals, provision of healthcare service varying from one
country to another, and political issues (Comer & Myers,
2016; Jefee-Bahloul, 2014; Madigan et al., 2021). All these
factors may cause caregivers to be kept more in the back-
ground in accessing services. However, it should be kept in
mind that, providing these services to caregivers makes a
significant contribution to the achievement and maintenance
of success in child mental health (Becker et al., 2018).
Tables 1 and 2 summarize the obtained results, which
suggests the following: (1) There is a growing interest for
TMH interventions in caregivers of children with mental
illnesses, as most of the studies were performed lately. (2)
Although significant advances have been achieved on care-
givers, more efforts should be made. For example, TMH
studies with larger samples in different regions (such as
underdeveloped and developing countries where more men-
tal health care is needed) can be conducted. (3) Technological
considerations are the most addressed, as seven papers of
11 studies treat the technological dimension. Given these
aspects, technological, administrative, security, and hardware
considerations for telemental health are considered elsewhere
(Chou et al., 2016; Comer et al., 2015). At the same time,
it is necessary to make and develop the necessary changes
in terms of environmental, economic, social and political
aspects for this dimension. (4) This systematic review can
be regarded as a good starting point because it provides an
overall perspective for them.
Limitations
This systematic review has some limitations. First, the num-
ber of studies in this systematic review was lower than
expected. However, this study specify a focus on caregivers
only. Secondly, studies whose language was not English were
excluded, and thirdly only eight databases were screened,
and that the gray literature was not screened.
Implications for practice
Psychosocial interventions are important in the provision
of mental health services for the caregivers of children and
adolescents with mental illnesses. These interventions, espe-
cially those performed on caregivers through TMH practices,
have a positive effect on both the child and the caregiver.
The results of these 11 studies indicate that TMH interven-
tions may produce potentially promising effects in caregivers’
outcome measures. In this respect, TMH interventions can
be an alternative to the strengthening of caregivers and their
receiving the help they need. In general, the positive effects
of TMH interventions on caregivers reviewed in this sys-
tematic review indicate that TMH interventions in caregivers
are well tolerated. Results built on the small but growing
literature support the promising role of TMH interventions
in caregivers of children with mental illnesses. Less evidence
is available on this topic. Conducting high-evidence exper-
imental studies with a larger sample will contribute to the
generalization of the results. In addition, future studies
should be better powered to estimate outcomes with medium
to low effect size and the authors of those studies should
try to enroll other caregiver groups (such as fathers, grand-
mothers/grandfathers, relatives or non-relative caregivers),
who were rarely taken into account in previous studies. Risk
of bias should be minimized. What is required in terms of
the provision of mental health services is to ensure that all
individuals can constantly access the mental health care they
need. However, in the current situation, the delivery of such
services is not as required. TMH is an important component
for the provision of such services. Caregivers of children
with mental illnesses are often overlooked. Therefore, the
implementation of TMH will also be beneficial in this
respect. Providing these services to caregivers will be ben-
eficial both to them and to their children. Performing

widespread studies and their active use in the mental health
system will contribute to the determining of their effects
on caregivers more clearly.
Conclusion
TMH interventions conducted on caregivers of children/
adolescents with mental illnesses yielded potentially prom-
ising findings in measures of caregivers’ outcomes like strain,
TMH intervention satisfaction, therapeutic alliance, sense
of competence, and parenting stress. It has been observed
that interventions similar to those used in traditional treat-
ments can also be carried out with TMH. Both experimental
and quasi-experimental studies indicate that TMH is well
tolerated in caregivers, but generally evidence comes from
samples including mothers. Future studies should be better
powered to estimate outcomes with medium to low effect
sizes, and should attempt to enroll fathers and other care-
giver groups as well. The risk of bias should be minimized,
with better attention paid to allocation concealment and
blinding of assessors and participants. In the studies
reviewed, the technological preparation phase is also con-
sidered an important component of TMH. Taking this com-
ponent into consideration will facilitate the implementation
of TMH. Despite the requirements and positive effects of
TMH on caregivers, it is important to note that future work
is needed on this topic. Undoubtedly, in the coming years,
TMH interventions will be more advantageous in meeting
mental health care needs and will be used more in the
provision of quality care.
Contributions of the authors
Z. U. D.: Determining the subject, literature review, plan-
ning, collection and analysis of the data, and writing out
the article; Z. C. D.: Determining the subject, screening of
the literature, planning, analyzing the data, and revising the
article critically.
Acknowledgements
No external or intramural funding was received.
Disclosure statement
The authors (Zümra Ülker Dörttepe and Zekiye Çetinkaya Duman)
report no actual, financial or potential conflicts of interest.
References
Abidin, R. R. (1983). The parenting stress index (PSI) manual. Pediatric
Psychology Press.
American Academy of Pediatrics (2019). Evidence-based child and
adolescent psychosocial interventions. 1–4. https://www.aap.org/en-us/
documents/crpsychosocialinterventions.pdf.
American Medical Association (2020). Telehealth implementation play-
b o o k . ht t p s : / / w w w. a m a - a s s n . o r g / s y s t e m / f i l e s / 2 0 2 0 - 0 4 /
ama-telehealth-playbook.pdf.
Issues in Mental Health Nursing 635
American Psychological Association (2016). Report of the working group
on psychotropic medications for children and adolescents:
Psychopharmacological, psychosocial, and combined interventions for
childhood disorders: evidence base, contextual factors, and future
directions. https://www.apa.org/pi/families/resources/child-
medications.pdf.
Attkisson, C. C., & Zwick, R. (1982). The client satisfaction question-
naire. Psychometric properties and correlations with service utili-
zation and psychotherapy outcome. Evaluation and Program
Planning, 5(3), 233–237. https://doi.org/10.1016/0149-7189(82)90074-X
Barkley, R. A. (2013). Defiant children: A clinician’s manual for assess-
ment and parent training. Guilford Press.
Becker, K. D., Boustani, M., Gellatly, R., & Chorpita, B. F. (2018).
Forty years of engagement research in children’s mental health ser-
vices: Multidimensional measurement and practice elements. Journal
of Clinical Child and Adolescent Psychology: The Official Journal for
the Society of Clinical Child and Adolescent Psychology, American
Psychological Association, Division 53, 47(1), 1–23. https://doi.org/1
0.1080/15374416.2017.1326121
Belangee, S. E., & Griffith, M. J. (2020). Bridging the generational
technology divide in counseling training, practice, and supervision.
In J. N. Williamson, & D. G. Williamson (Eds.), Distance counseling
and supervision: A guide for mental health clinicians. (p. 45–55).
Wiley.
Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., &
Baugher, M. (1999). Psychometric properties of the Screen for Child
Anxiety Related Emotional Disorders (SCARED): A replication
study. Journal of the American Academy of Child and Adolescent
Psychiatry, 38(10), 1230–1236. https://doi.org/10.1097/00004583-
199910000-00011
Brannan, A. M., Heflinger, C. A., & Bickman, L. (1997). The Caregiver
Strain Questionnaire: Measuring the impact on the family of living
with a child with serious emotional disturbance. Journal of Emotional
and Behavioral Disorders, 5(4), 212–222. https://doi.org/10.1177/
106342669700500404
Breaux, R., Shroff, D. M., Cash, A. R., Swanson, C. S., Carlton, C.,
Bertollo, J. R., & Dahiya, A. V. (2021). Telehealth delivery of the
RELAX intervention for families of adolescents diagnosed with
ADHD: Preliminary treatment outcomes and evidence of accept-
ability and feasibility. Evidence-Based Practice in Child and Adolescent
Mental Health, In press, 1–15. https://doi.org/10.1080/23794925.2021.
1970053
Brestan, E., Jacobs, J., Rayfield, A., & Eyberg, S. M. (1999). A con-
sumer satisfaction measure for parent-child treatments and its re-
lationship to measures of child behavior change. Behavior Therapy,
30(1), 17–30. https://doi.org/10.1016/S0005-7894(99)80043-4
Carpenter, A. L., Pincus, D. B., Furr, J. M., & Comer, J. S. (2018).
Working from home: An initial pilot examination of
videoconferencing-based cognitive behavioral therapy for anxious
youth delivered to the home setting. Behavior Therapy, 49(6), 917–
930. https://doi.org/10.1016/j.beth.2018.01.007
Center for Reviews and Dissemination. (2009). Systematic reviews:
CRD’s guidance for undertaking reviews in healthcare. University of
York NHS Centre for Reviews & Dissemination.
Centers for Medicare & Medicaid Services. (2020). Telehealth. https://
www.medicaid.gov/medicaid/benefits/telemedicine/index.html
Chang, C. C., Chen, Y. M., Liu, T. L., Hsiao, R. C., Chou, W. J., &
Yen, C. F. (2020). Affiliate stigma and related factors in family
caregivers of children with attention-deficit/hyperactivity disorder.
International Journal of Environmental Research and Public Health,
17(2), 576–590. https://doi.org/10.3390/ijerph17020576
Chou, T., Comer, J. S., Turvey, C. L., Karr, A., & Spargo, G. (2016).
Technological considerations for the delivery of real-time child
telemental healthcare. Journal of Child and Adolescent
Psychopharmacology, 26(3), 192–197. https://doi.org/10.1089/
cap.2015.0043
Chowdhury, M., Aman, M. G., Lecavalier, L., Smith, T., Johnson, C.,
Swiezy, N., McCracken, J. T., King, B., McDougle, C. J., Bearss, K.,
Deng, Y., & Scahill, L. (2016). Factor structure and psychometric
properties of the revised Home Situations Questionnaire for autism
636 Z. Ü. DÖRTTEPE AND Z. Ç. DUMAN
spectrum disorder: The home situations questionnaire-autism spec-
trum disorder. Autism: The International Journal of Research and
Practice, 20(5), 528–537. https://doi.org/10.1177/1362361315593941
Chronis, A. M., Jones, H. A., & Raggi, V. L. (2006). Evidence-based
psychosocial treatments for children and adolescents with
attention-deficit/hyperactivity disorder. Clinical Psychology Review,
26(4), 486–502. https://doi.org/10.1016/j.cpr.2006.01.002
Comer, J. S., & Barlow, D. H. (2014). The occasional case against broad
dissemination and implementation: Retaining a role for specialty
care in the delivery of psychological treatments. The American
Psychologist, 69(1), 1–18. https://doi.org/10.1037/a0033582
Comer, J. S., Furr, J. M., Cooper-Vince, C., Madigan, R. J., Chow, C.,
Chan, P., Idrobo, F., Chase, R. M., McNeil, C. B., & Eyberg, S. M.
(2015). Rationale and considerations for the internet-based delivery
of Parent-Child Interaction Therapy. Cognitive and Behavioral
Practice, 22(3), 302–316. https://doi.org/10.1016/j.cbpra.2014.07.003
Comer, J. S., Furr, J. M., Del Busto, C., Silva, K., Hong, N., Poznanski,
B., Sanchez, A., Cornacchio, D., Herrera, A., Coxe, S., Miguel, E.,
Georgiadis, C., Conroy, K., & Puliafico, A. (2021). Therapist-led,
internet-delivered treatment for early child social anxiety: A
waitlist-controlled evaluation of the iCALM Telehealth Program.
Behavior Therapy, 52(5), 1171–1187. https://doi.org/10.1016/j.
beth.2021.01.004
Comer, J. S., Furr, J. M., Kerns, C. E., Miguel, E., Coxe, S., Elkins, R.
M., Carpenter, A. L., Cornacchio, D., Cooper-Vince, C. E., DeSerisy,
M., Chou, T., Sanchez, A. L., Khanna, M., Franklin, M. E., Garcia,
A. M., & Freeman, J. B. (2017a). Internet-delivered, family-based
treatment for early-onset OCD: A pilot randomized trial. Journal
of Consulting and Clinical Psychology, 85(2), 178–186. https://doi.
org/10.1037/ccp0000155
Comer, J. S., Furr, J. M., Miguel, E. M., Cooper-Vince, C. E.,
Carpenter, A. L., Elkins, R. M., Kerns, C. E., Cornacchio, D.,
Chou, T., Coxe, S., DeSerisy, M., Sanchez, A. L., Golik, A.,
Martin, J., Myers, K. M., & Chase, R. (2017). Remotely deliver-
ing real-time parent training to the home: An initial randomized
trial of Internet-delivered parent–child interaction therapy
(I-PCIT). Journal of Consulting and Clinical Psychology, 85(9),
909–917.
Comer, J. S., & Myers, K. (2016). Future directions in the use of
telemental health to improve the accessibility and quality of chil-
dren’s mental health services. Journal of Child and Adolescent
Psychopharmacology, 26(3), 296–300. https://doi.org/10.1089/
cap.2015.0079
Dalrymple, R. A., McKenna Maxwell, L., Russell, S., & Duthie, J.
(2020). NICE guideline review: Attention deficit hyperactivity dis-
order: Diagnosis and management (NG87). Archives of Disease in
Childhood. Education and Practice Edition, 105(5), 289–293. https://
doi.org/10.1136/archdischild-2019-316928
Dimock, M. (2019). Defining generations: Where millennials end and
generation Z begins. Pew Research Center, 17(1), 1–7.
Doyen, C. M., Oreve, M.-J., Desailly, E., Goupil, V., Zarca, K.,
L’Hermitte, Y., Chaste, P., Bau, M.-O., Beaujard, D., Haddadi, S.,
Bibay, A., Contejean, Y., Coutrot, M.-T., Crespin, L., Frioux, I.,
Speranza, M., Francois, N., & Kaye, K. (2018). Telepsychiatry for
children and adolescents: A review of the PROMETTED project.
Telemedicine Journal and e-Health: The Official Journal of the
American Telemedicine Association, 24(1), 3–10. https://doi.
org/10.1089/tmj.2017.0041
Elliott, S. N., & Gresham, F. M. (1987). Children’s social skills:
Assessment and classification practices. Journal of Counseling &
Development, 66(2), 96–99. https://doi.org/10.1002/j.1556-6676.1987.
tb00808.x
Eyberg, S. M., & Pincus, D. (1999). ECBI & SESBI-R: Eyberg child be-
havior inventory and Sutter-Eyberg student behavior inventory-revised:
Professional manual. Psychological Assessment Resources.
Flessner, C. A., Sapyta, J., Garcia, A., Freeman, J. B., Franklin, M. E.,
Foa, E., & March, J. (2011). Examining the psychometric properties
of the family accommodation scale-parent-report (FAS-PR). Journal
of Psychopathology and Behavioral Assessment, 33(1), 38–46. https://
doi.org/10.1007/s10862-010-9196-3
Freeman, J. B., & Garcia, A. M. (2008). Family based treatment for
young children with OCD: Therapist guide. Oxford University Press.
https://doi.org/10.1093/MED:PSYCH/9780195373639.001.0001
Gloff, N. E., LeNoue, S. R., Novins, D. K., & Myers, K. (2015).
Telemental health for children and adolescents. International Review
of Psychiatry (Abingdon, England), 27(6), 513–524. https://doi.org/1
0.3109/09540261.2015.1086322
Gratz, K. L., & Roemer, L. (2004). Multidimensional assessment of
emotion regulation and dysregulation: Development, factor structure,
and initial validation of the difficulties in emotion regulation scale.
Journal of Psychopathology and Behavioral Assessment, 26(1), 41–54.
https://doi.org/10.1023/B:JOBA.0000007455.08539.94
Haley, C., O’Callaghan, E., Hill, S., Mannion, N., Donnelly, B., Kinsella,
A., Murtagh, A., & Turner, N. (2011). Telepsychiatry and carer
education for schizophrenia. European Psychiatry: The Journal of the
Association of European Psychiatrists, 26(5), 302–304. https://doi.
org/10.1016/j.eurpsy.2009.12.021
Hepburn, S. L., Blakeley-Smith, A., Wolff, B., & Reaven, J. A. (2016).
Telehealth delivery of cognitive-behavioral intervention to youth
with autism spectrum disorder and anxiety: A pilot study. Autism:
The International Journal of Research and Practice, 20(2), 207–218.
https://doi.org/10.1177/1362361315575164
Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D.,
Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., & Sterne, J. A.
C. (2011). The Cochrane Collaboration’s tool for assessing risk of
bias in randomised trials. BMJ (Clinical Research ed.), 343, d5928
https://doi.org/10.1136/bmj.d5928
Hilty, D. M., Ferrer, D. C., Parish, M. B., Johnston, B., Callahan, E.
J., & Yellowlees, P. M. (2013). The effectiveness of telemental health:
A 2013 review. Telemedicine Journal and e-Health: The Official
Journal of the American Telemedicine Association, 19(6), 444–454.
https://doi.org/10.1089/tmj.2013.0075
Hilty, D., Yellowlees, P. M., Parrish, M. B., & Chan, S. (2015).
Telepsychiatry: Effective, evidence-based, and at a tipping point in
health care delivery? The Psychiatric Clinics of North America, 38(3),
559–592. https://doi.org/10.1016/j.psc.2015.05.006
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation
of the working alliance inventory. Journal of Counseling Psychology,
36(2), 223–233. https://doi.org/10.1037/0022-0167.36.2.223
Howard, B., Chu, B. C., Krain, A. L., Marrs-Garcia, A. L., & Kendall,
P. C. (2000). Cognitive behavioral family therapy for anxious children:
Therapist manual. (2nd ed.). Workbook Publishing.
Howard, P. N., & Mazaheri, N. (2009). Telecommunications reform,
Internet use and mobile phone adoption in the developing world.
World Development, 37(7), 1159–1169. https://doi.org/10.1016/j.
worlddev.2008.12.005
Jefee-Bahloul, H. (2014). Telemental health in the Middle East:
Overcoming the barriers. Frontiers in Public Health, 2, 86 https://
doi.org/10.3389/fpubh.2014.00086
Jenkins-Guarnieri, M. A., Pruitt, L. D., Luxton, D. D., & Johnson, K.
(2015). Patient perceptions of telemental health: Systematic review
of direct comparisons to in-person psychotherapeutic treatments.
Telemedicine Journal and e-Health: The Official Journal of the
American Telemedicine Association, 21(8), 652–660. https://doi.
org/10.1089/tmj.2014.0165
Johnston, C., & Mash, E. J. (1989). A measure of parenting satisfaction
and efficacy. Journal of Clinical Child Psychology, 18(2), 167–175.
https://doi.org/10.1207/s15374424jccp1802_8
Kamphaus, R. W., & Reynolds, C. R. (2006). Parenting relationship
questionnaire (PRQ). In Behavior assessment system for children.
(2nd ed.). PsychCorp.
Kazdin, A. E., Holland, L., Crowley, M., & Breton, S. (1997). Barriers
to treatment participation scale: Evaluation and validation in the
context of child outpatient treatment. Journal of Child Psychology
and Psychiatry, and Allied Disciplines, 38(8), 1051–1062. https://doi.
org/10.1111/j.1469-7610.1997.tb01621.x
Koren, P. E., DeChillo, N., & Friesen, B. J. (1992). Measuring empow-
erment in families whose children have emotional disabilities: A
brief questionnaire. Rehabilitation Psychology, 37(4), 305–321. https://
doi.org/10.1037/h0079106

Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D.
(1979). Assessment of client/patient satisfaction: Development of a
general scale. Evaluation and Program Planning, 2(3), 197–207.
https://doi.org/10.1016/0149-7189(79)90094-6
Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative
emotional states: Comparison of the depression anxiety stress scales
(DASS) with the Beck depression and anxiety inventories. Behaviour
Research and Therapy, 33(3), 335–343. https://doi.org/10.1016/0005-
7967(94)00075-U
Luo, C., Sanger, N., Singhal, N., Pattrick, K., Shams, I., Shahid, H.,
Hoang, P., Schmidt, J., Lee, J., Haber, S., Puckering, M., Buchanan,
N., Lee, P., Ng, K., Sun, S., Kheyson, S., Chung, D. C.-Y., Sanger,
S., Thabane, L., & Samaan, Z. (2020). A comparison of
electronically-delivered and face to face cognitive behavioural ther-
apies in depressive disorders: A systematic review and meta-analysis.
EClinicalMedicine, 24, 100442. https://doi.org/10.1016/j.
eclinm.2020.100442
Luxton, D. D., Pruitt, L. D., Wagner, A., Smolenski, D. J.,
Jenkins-Guarnieri, M. A., & Gahm, G. (2016). Home-based telebe-
havioral health for US military personnel and veterans with depres-
sion: A randomized controlled trial. Journal of Consulting and
Clinical Psychology, 84(11), 923–934. https://doi.org/10.1037/
ccp0000135
Madigan, S., Racine, N., Cooke, J. E., & Korczak, D. J. (2021).
COVID-19 and telemental health: Benefits, challenges, and future
directions. Canadian Psychology/Psychologie Canadienne, 62(1), 5–11.
https://doi.org/10.1037/cap0000259
Marino, F., Chilà, P., Failla, C., Crimi, I., Minutoli, R., Puglisi, A.,
Arnao, A. A., Tartarisco, G., Ruta, L., Vagni, D., & Pioggia, G.
(2020). Tele-assisted behavioral intervention for families with chil-
dren with autism spectrum disorders: A randomized control trial.
Brain Sciences, 10(9), 649–661. https://doi.org/10.3390/brains-
ci10090649
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and im-
plementation of evidence-based psychological treatments. A review
of current efforts. The American Psychologist, 65(2), 73–84. https://
doi.org/10.1037/a0018121
Mendenhall, A. N., & Mount, K. (2011). Parents of youth with men-
tal illness: A look at caregiver strain & caregiver-focused interven-
tions. Families in Society: The Journal of Contemporary Social
Services, 92(2), 183–190. https://doi.org/10.1606/1044-3894.4097
Möller-Leimkühler, A. M., & Wiesheu, A. (2012). Caregiver burden
in chronic mental illness: The role of patient and caregiver charac-
teristics. European Archives of Psychiatry and Clinical Neuroscience,
262(2), 157–166. https://doi.org/10.1007/s00406-011-0215-5
Myers, K., Nelson, E.-L., Rabinowitz, T., Hilty, D., Baker, D., Barnwell,
S. S., Boyce, G., Bufka, L. F., Cain, S., Chui, L., Comer, J. S.,
Cradock, C., Goldstein, F., Johnston, B., Krupinski, E., Lo, K.,
Luxton, D. D., McSwain, S. D., McWilliams, J., … Bernard, J. (2017).
American telemedicine association practice guidelines for telemen-
tal health with children and adolescents. Telemedicine Journal and
e-Health: The Official Journal of the American Telemedicine
Association, 23(10), 779–804. https://doi.org/10.1089/tmj.2017.0177
Myers, K., Vander Stoep, A., Zhou, C., McCarty, C. A., & Katon, W.
(2015). Effectiveness of a telehealth service delivery model for treat-
ing attention-deficit/hyperactivity disorder: A community-based
randomized controlled trial. Journal of the American Academy of
Child & Adolescent Psychiatry, 54(4), 263–274. https://doi.
org/10.1016/j.jaac.2015.01.009
National Institue of Mental Health (2021). Children and Mental Health.
Is this just a stage? (Revised). USA: NIMH Publications https://www.
nimh.nih.gov/sites/default/files/documents/health/publications/
children-and-mental-health/21-mh-8085-childrenandmentalhealth.
pdf
Nelson, E. L., Cain, S., & Sharp, S. (2017). Considerations for con-
ducting telemental health with children and adolescents. Child and
Adolescent Psychiatric Clinics of North America, 26(1), 77–91. https://
doi.org/10.1016/j.chc.2016.07.008
Ozkan, B., Erdem, E., Ozsoy, S. D., & Zararsiz, G. (2013). Effect of
psychoeducation and telepsychiatric follow up given to the caregiv-
er of the schizophrenic patient on family burden, depression and
Issues in Mental Health Nursing 637
expression of emotion. Pakistan Journal of Medical Sciences, 29(5),
1122. https://doi.org/10.12669/pjms.295.2613
Prinz, R. J., Foster, S., Kent, R. N., & O’Leary, K. D. (1979). Multivariate
assessment of conflict in distressed and nondistressed
mother-adolescent dyads. Journal of Applied Behavior Analysis, 12(4),
691–700. https://doi.org/10.1901/jaba.1979.12-691
Pruitt, L. D., Luxton, D. D., & Shore, P. (2014). Additional clinical
benefits of home-based telemental health treatments. Professional
Psychology: Research and Practice, 45(5), 340–346. https://doi.
org/10.1037/a0035461
Reaven, J., Blakeley-Smith, A., Nichols, S., & Hepburn, S. (2011). Facing
your fears: Group therapy for managing anxiety in children with
high-functioning autism spectrum disorders. Brookes.
Rebello, T. J., Marques, A., Gureje, O., & Pike, K. M. (2014). Innovative
strategies for closing the mental health treatment gap globally.
Current Opinion in Psychiatry, 27(4), 308–314. https://doi.
org/10.1097/YCO.0000000000000068
Reese, R. J., Slone, N. C., Soares, N., & Sprang, R. (2012). Telehealth
for underserved families: An evidence-based parenting program.
Psychological Services, 9(3), 320–322. https://doi.org/10.1037/
a0026193
Shirk, S. R., & Saiz, C. C. (1992). Clinical, empirical, and develop-
mental perspectives on the therapeutic relationship in child psy-
chotherapy. Development and Psychopathology, 4(4), 713–728. https://
doi.org/10.1017/S0954579400004946
Sibley, M. H., Campez, M., Perez, A., Morrow, A. S., Merrill, B. M.,
Altszuler, A. R., Coxe, S., & Yequez, C. E. (2016a). Parent manage-
ment of organization, time management, and planning deficits among
adolescents with ADHD. Journal of Psychopathology and Behavioral
Assessment, 38, 216–228. https://doi.org/10.1007/s10862-015-9515-9
Sibley, M. H., Comer, J. S., & Gonzalez, J. (2017). Delivering parent-teen
therapy for ADHD through videoconferencing: A preliminary in-
vestigation. Journal of Psychopathology and Behavioral Assessment,
39(3), 467–485. https://doi.org/10.1007/s10862-017-9598-6
Sibley, M. H., Graziano, P. A., Kuriyan, A. B., Coxe, S., Pelham, W.
E., Rodriguez, L., Sanchez, F., Derefinko, K., Helseth, S., & Ward,
A. (2016b). Parent-teen behavior therapy + motivational interviewing
for adolescents with ADHD. Journal of Consulting and Clinical
Psychology, 84(8), 699–712. https://doi.org/10.1037/ccp0000106
Strasser, R., Kam, S. M., & Regalado, S. M. (2016). Rural health care
access and policy in developing countries. Annual Review of Public
Health, 37, 395–412. https://doi.org/10.1146/annurev-publhealt
h-032315-021507
Thompson-Hollands, J., Kerns, C. E., Pincus, D. B., & Comer, J. S.
(2014). Parental accommodation of child anxiety and related symp-
toms: Range, impact, and correlates. Journal of Anxiety Disorders,
28(8), 765–773. https://doi.org/10.1016/j.janxdis.2014.09.007
Tse, Y. J., McCarty, C. A., Stoep, A. V., & Myers, K. (2015). Teletherapy
delivery of caregiver behavior training for children with
attention-deficit hyperactivity disorder. Telemedicine Journal and
e-Health: The Official Journal of the American Telemedicine
Association, 21(6), 451–458. https://doi.org/10.1089/tmj.2014.0132
Uslu, E., Buldukoğlu, K., & Beebe, L. H. (2019). A telenursing practice
for care of people with schizophrenia: Telephone intervention prob-
lem solving. Journal of Psychiatric Nursing, 10(2), 131–136. https://
doi.org/10.14744/phd.2019.75768
Wittkampf, K., van Ravesteijn, H., Baas, K., van de Hoogen, H., Schene,
A., Bindels, P., Lucassen, P., van de Lisdonk, E., & van Weert, H.
(2009). The accuracy of Patient Health Questionnaire-9 in detecting
depression and measuring depression severity in high-risk groups
in primary care. General Hospital Psychiatry, 31(5), 451–459. https://
doi.org/10.1016/j.genhosppsych.2009.06.001
World Health Organization. (2011). Videoconferencing system, telemed-
icine. WHO. 2011. https://www.who.int/medical_devices/innovation/
videoconferencing_telemedicine.pdf?ua=1
Xie, Y., Dixon, J. F., Yee, O. M., Zhang, J., Chen, Y. A., Deangelo, S.,
Yellowlees, P., Hendren, R., & Schweitzer, J. B. (2013). A study on
the effectiveness of videoconferencing on teaching parent training
skills to parents of children with ADHD. Telemedicine Journal and
e-Health: The Official Journal of the American Telemedicine
Association, 19(3), 192–199. https://doi.org/10.1089/tmj.2012.0108