Research

Journal of Telemedicine and Telecare

1–5
Harnessing the power of telemedicine to © The Author(s) 2021
Article reuse guidelines:
accomplish international pediatric outcome sagepub.com/journals-permissions
DOI: 10.1177/1357633X211063166
research during the COVID-19 pandemic journals.sagepub.com/home/jtt

Sarah B Mulkey1,2,3 , Margarita Arroyave-Wessel4,

Colleen Peyton5, Emily Ansusinha6 , Corina Gutierrez7,
Andrea Sorkar7, Andres Cure7, Yhina Samper7, Daniela Cure7,
8 7
AQ1 Michael E Msall , and Carlos Cure
¶
GQ1
¶

GQ2 Abstract
¶
GQ4 The COVID-19 pandemic occurred during planned neurodevelopmental follow-up of Colombian children with antenatal
¶
GQ5 Zika-virus exposure. The objective of the study was to leverage the institution’s telemedicine infrastructure to support
¶
international clinical child outcome research. In a prospective cohort study of child neurodevelopment (NCT04398901),
we used synchronous telemedicine to remotely train a research team and perform live observational assessments of chil-
dren in Sabanalarga, Colombia. An observational motor and conceptional standardized tool kit was mailed to Colombia;
other materials were translated and emailed; team training was done virtually. Children were recruited by team on the
ground. Synchronous activities were video-recorded directly to two laptops, each with a telehealth Zoom link to allow
simultaneous evaluation of “table” and “standing” activities, and backup recordings were captured directly on the device in
Colombia. The U.S. team attended live over Zoom from four states and five distinct locations, made observational notes,
and provided real-time feedback. Fifty-seven, 3–4-year-old children with Zika-virus exposure and 70 non-exposed con-
trols were studied during 10 daytrips. Direct laptop recording ensured complete record of child activities due to internet
outages. Telemedicine can be used to successfully perform international neurodevelopmental outcome research in chil-
dren during the COVID-19 pandemic. Telemedicine can benefit global health studies.

Keywords
Telemedicine, Zika virus, Colombia, child outcome, virtual
Date received: 5 October 2021; Date accepted: 7 November 2021

Introduction
In February 2016 the World Health Organization declared
Zika-virus (ZIKV) a public health emergency of international
concern due to recognition that ZIKV causes a congenital infec-
tion that can result in Congenital Zika Syndrome with infant
microcephaly.1 Central and South America were heavily
impacted by the ZIKV epidemic and devastating rate of
infants born with microcephaly and other birth defects caused
by antenatal ZIKV infection.1 Fortunately, many infants born
to mothers with ZIKV infection in pregnancy did not have
Congenital Zika Syndrome nor severe birth defects.2,3
However, we showed in our longitudinal cohort established
during pregnancy that normocephalic infants with antenatal
ZIKV exposure can have lower neurodevelopmental scores to
18 months of age.3,4 Thus, longitudinal neurodevelopmental
assessments to older ages are essential in order to understand
the full effects of the ZIKV epidemic on child health, even in
the absence of ZIKV-related birth defects.5
1
Prenatal Pediatrics Institute, Children’s National Hospital, Washington,
DC, USA
2
Department of Neurology, The George Washington, University School
of Medicine and Health Sciences, Washington, DC, USA
3
Department of Pediatrics, The George Washington, University School of
Medicine and Health Sciences, Washington, DC, USA
4
Telemedicine, Children’s National Hospital, Washington, DC, USA
5
Department of Physical Therapy and Human Movement Sciences,
Northwestern University, Chicago, IL, USA
6
Division of Pediatric Infectious Disease, Children’s National Hospital,
Washington, DC, USA
7
BIOMELab, Barranquilla, Colombia
8
Kennedy Research Center on Intellectual and Neurodevelopmental
Disabilities, University of Chicago Medicine, Chicago, IL, USA
Corresponding author:
Sarah B. Mulkey, Prenatal Pediatrics Institute, Children’s National Hospital;
Neurology and Pediatrics, The George Washington, University School of
Medicine and Health Sciences, 111 Michigan Ave., NW, Washington, DC
20010, USA.
Email: sbmulkey@childrensnational.org
2 Journal of Telemedicine and Telecare 0(0)
The COVID-19 pandemic occurred at an important time
when children with antenatal ZIKV exposure were of age
for scheduled neurodevelopmental evaluations in early
childhood. Multiple international cohorts of pregnant
women, infants, and children with antenatal ZIKV infection
were established between 2016 and 2018; however, those
with planned follow-up after 2 years of age are few.6
Cohort studies with longitudinal follow-up planned to
older ages are essential in order to provide key outcome
data on child outcomes since many measures of child neu-
rodevelopment can only be assessed at older ages.7 The
intersection of the current pandemic on a Colombian com-
munity still recovering from a major health epidemic and
effects on their children presents enormous challenges.
Like many countries, Colombia placed a quarantine limit-
ing travel into the country and between regions, which
prompted a need to develop alternative strategies to
follow child outcomes for an international research team.
Almost overnight with the spread of SARS-CoV-2, teleme-
dicine use expanded rapidly in an effort to protect patients and
care providers, but as a way to provide clinical care during the
COVID-19 pandemic.8–10 Previously, telemedicine was
mostly used for clinical patient visits and it is only beginning
to be utilized in clinical research.11–13 Telemedicine can also
support medical care in resource-limited international set-
tings.14 We previously used recorded videos to score infant
motor development for our cohort of infants in Colombia
with antenatal ZIKV exposure, so had experience with
remote video-based scoring methods.4 However, the current
pandemic forced a shift in our research methods to fully
utilize telemedicine to train our international research team
and perform live observational assessments of child develop-
mental activities. The objective of this study is to describe the
use of telemedicine for international evaluations of children
with antenatal ZIKV exposure using a synchronous telemedi-
cine approach.
Methods
Participants
We performed a prospective cohort study of child neurode-
velopment in Sabanalarga, a rural town, in Colombia using
synchronous telemedicine from September 2020 to
February 2021. Children born between August 1, 2016,
and November 30, 2017, with antenatal ZIKV exposure pre-
viously studied to 18 months of age who did not have CZS
were eligible.4 The COVID-19 pandemic occurred during
planned in-person neurodevelopmental follow-up of this
cohort when the children were 3–4 years of age. Children
age 4–5 years, born before March 31, 2016, without ante-
natal ZIKV exposure were recruited in the same town to
serve as non-exposed controls. Due to the rural location of
the study, children were scheduled by both in-person and
phone contact. The study was approved by the Children’s
National Institutional Review Board (IRB, Pro00012759),
Washington, DC, by the CIRCIE IRB, Barranquilla,
Colombia, and has trial registration NCT04398901 (https://
clinicaltrials.gov/ct2/show/NCT04398901).
Team training by telemedicine
The neurodevelopmental assessments were selected due to
being culturally appropriate, available in English and
Spanish languages, and for their ability to lend themselves
to virtual training. An observational motor and conceptional
standardized tool kit (Movement-ABC-2 and Bracken
School Readiness Assessment – 3rd Edition [Pearson])
and questionnaires were mailed to Colombia; other materi-
als were translated into Spanish and emailed. Remote team
training was performed virtually over three days using
HIPAA compliant Zoom telehealth accounts. Each assess-
ment was demonstrated by the U.S. team and real-time
Spanish interpretation was provided. The Colombian team
had the developmental testing kits and questionnaires avail-
able during the trainings. They then practiced and demon-
strated learned skills to the U.S. team virtually over two
days using non-research children of similar age.
Child evaluations
Evaluations of the children in Colombia were performed by
the team in-person with the children with synchronous teleme-
dicine connection to the U.S. team via two Zoom telehealth
connected devices (Figure 1). Children were brought for
their in-person assessments to either a community center or
public school, both with space to perform the activities,
where the team connected using a portable WIF antenna.
Children were evaluated by the Movement-ABC-2 which
includes measures of manual dexterity, aiming and catching,
and balance and by the Bracken School-Readiness
Assessment. The later includes developmental concepts of
colors, letters, numbers, shapes, and comparisons. The chil-
dren rotated between performing the standing activities and
the table activities to maintain child engagement and effi-
ciency for the research team. Written portions of the test
including the children’s drawings, were scanned and scored
remotely. The team in Colombia wore masks, handwashed,
and cleaned items between children.
The child activities were synchronously video recorded
directly to two laptops, each with a telehealth Zoom link
to allow simultaneous evaluation of “table” and “standing”
activities (Figure 1). Back-up recordings were captured by
the research team in Colombia. The U.S. team attended
live over Zoom from four states and five distinct locations,
made live qualitative observational notes of the child’s
movements and performance on the different measures,
and provided real-time feedback in Spanish and English
to the Colombian team. The teams communicated using
WhatsApp to clarify questions or provide feedback when
Mulkey et al.
Figure 1. Schematic representation of telemedicine approach for child neurodevelopmental evaluations. Schematic figure showing
synchronous telemedicine connection of the U.S. team from four states using laptops and tablets with Zoom telemedicine recording
ability and real-time connection to children in Sabanalarga, Colombia, performing standing and table activities to assess
neurodevelopment.
necessary and to keep track of the order of participants.
Video recordings will be used for qualitative assessment
and scoring.
Data analysis
Child age is reported as mean and standard deviation. The
cost of the team training and evaluations done using teleme-
dicine was compared to the original budget that did not
include telemedicine.
Results
Fifty-seven of 70 (81%) eligible children with antenatal
ZIKV-exposure with a mean (SD) age of evaluation of 4.0
(0.4) years were evaluated during five daytrips in
Sabanalarga, Colombia. Day 1 did not have a working internet
connection. Day 2 (community center) and days 3–5 (school)
had an internet connection for live telemedicine Zoom but
had multiple brief outages despite conducting prior connectiv-
ity testing. Following the evaluation days of the children with
ZIKV-exposure, 70 non-exposed control children (100% of
goal enrollment) with mean age of 5.4 (0.2) years were evalu-
ated over five days with synchronous telemedicine connection
to the U.S. team. These visits occurred at the school and had
3
internet connection with brief disruptions. For all study days,
direct laptop recording of the children was necessary to
ensure a complete video record of child activities. Recorded
videos by the laptop were saved and utilized for evaluation
and scoring during internet outages that affected the Zoom tele-
medicine recordings. The telemedicine approach enabled the
U.S. team to provide important feedback based on live assess-
ment observations of the children and make real-time adjust-
ments to the performance of the neurodevelopmental tests.
The original study budget included $6240 for travel from
the U.S. to Colombia for four team members to accomplish
training of the Colombian research team and to see initial
participants together to verify skills. During the travel, the
neurodevelopmental testing kits and record forms would
be brought by the team as a luggage to avoid additional
costs. Using the telehealth approach, neurodevelopmental
testing kits and forms were air-mailed in two boxes, for a
total cost of $1440: 77% less than the budget for travel
using the non-telehealth approach. The cost of telehealth
Zoom at our institution is $16 per month per user.
Discussion
Telemedicine infrastructure enabled synchronous interna-
tional research collaboration to train a research team,
4 Journal of Telemedicine and Telecare 0(0)
evaluate child outcomes in a rural international setting and
enroll study controls during the COVID-19 pandemic. By
utilizing telemedicine to train the study team, enrollment
goals and timepoints of assessment were able to be main-
tained at the appropriate child age for the original study
objective, without significant delay from the travel restric-
tions imposed by the COVID-19 pandemic. The careful
choice of testing materials combined with live telemedicine
observation allowed multidisciplinary child assessments by
trained rehabilitational, neurological, and psychological
professionals to observe both gross motor, fine motor, and
verbal and nonverbal conceptual skills. The children were
able to perform in their community environment and there
was minimal difficulty in establishing rapport with asses-
sors. Most importantly, this approach-maintained collabor-
ation of the international team at a time when international
travel was not possible.
Virtual training using a Zoom telemedicine platform was
found to be effective with the excellent acquisition of skills
by the research team. Telemedicine has been successfully
used for international clinical neurodevelopmental evalua-
tions and team training.15 The choice of culturally appropri-
ate assessments available in Spanish likely contributed to
the successful use of the toolkit by the Colombian research
team.16 The use of telemedicine to provide international
expertise for training and child assessment was also less
expensive than having an international team travel to be
on-site. This method may be a more efficient way to
sustain international research collaborations during the pan-
demic and beyond.17
Evaluation of child motor neurodevelopment can be
done by observation and thus lend itself to work well in a
telemedicine setting. Building upon our prior study that
used video scoring of the observational motor assessment,
the Alberta Infant Motor Scale, we utilized the
Movement-ABC-2 (Pearson) to assess child motor develop-
ment and function.4 For telemedicine child neurodevelop-
mental evaluations, it is important to choose assessments
that allow observation of child skills as a main measure.
Much can be learned by keen observation of a child and
this can be readily utilized by telemedicine. Even in a clin-
ical setting, home videos of infant and child movements can
help evaluate for neurodevelopmental disabilities and
abnormalities of movement.18
One of the great potentials of telemedicine is that it
can bring health resources to remote, isolated areas of the
world, areas often with significant disparities in the opp-
ortunity for specialty medical care.19 The telemedicine
tool is reliant on available technology and internet conn-
ectivity which in some remote places may be a barrier to
this method of care. An additional benefit for telemedicine
is cost savings, and reduced travel time expense for patients
and providers. To our knowledge, the cost savings benefit
to clinical child outcome research has not been pre-
viously reported. By utilizing telemedicine to accomplish
evaluations of child neurodevelopmental outcome, our
study saved in travel costs and had the additional major
benefit of enabling the U.S. research team to attend all of
the child visit days live and provide direct feedback to the
research team on the ground. By this method, the children
benefitted by having a child neurologist, developmental
pediatrician, and physical therapist virtually observe child
evaluations. This team of three pediatric specialists is not
available in the community where these children live, so
participation in the research was a benefit to them and
their families. In turn, the research team benefitted by live
presence enabling quality observational assessments for
the research study and the ability of the U.S. and
Colombian research team members to train and work
together to accomplish the child evaluations.
One limitation encountered of a telemedicine approach
for our study was difficulty maintaining internet connection
through all child assessments in a rural setting. Internet
outages triggered breaks in recordings on Zoom telehealth
devices and they were overcome by having a direct record-
ing to the laptop at the site as a backup to the telemedicine
recordings. Adequate internet connection should be tested
prior to research activities and direct video recording
should be available as a backup. Future studies may
benefit from having satellite internet capabilities, especially
in rural research sites, which could potentially improve the
quality and experience. Our institution did not have a preex-
isting telemedicine research policy and procedure. Based on
our experience and the growing experience of others, the
use of telemedicine for accomplishing outcome research
is likely to dramatically expand and will require policies
for use in research settings.
While our study was not originally designed to use tele-
medicine in such a significant way, through this process, we
found that synchronous telemedicine connected the
research team and enabled international child outcome
research during the pandemic which would not have other-
wise been possible. The use of this technology for interna-
tional research will likely continue past the pandemic due to
enhanced efficiency, lower cost, and ability to maintain col-
laborations between multiple study sites.
Acknowledgements
The authors appreciate the support of the Telemedicine
Department at Children’s National Hospital, Washington, DC,
and the Children’s National Telemedicine Director, Clarence
Williams. The authors appreciate Jorge Manotas Reyes for his
help with study logistics and patient scheduling in Sabanalarga,
Colombia.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with GQ3
¶
respect to the research, authorship, and/or publication of this
article.
 Mulkey et al.
Funding
The author(s) disclosed receipt of the following financial support
for the research, authorship, and/or publication of this article:
AQ2 This work was supported by the Eunice Kennedy Shriver
¶ 9. Xie J, Prahalad P, Lee TC, et al. Pediatric subspecialty adop-
National Institute of Child Health and Human Development
(grant number R01HD102445).
ORCID iDs
GQ6 Sarah B Mulkey https://orcid.org/0000-0002-8084-526X
¶ 11. Bunnell BE, Sprague G, Qanungo S, et al. An exploration of
Emily Ansusinha https://orcid.org/0000-0003-2866-686X
References
1. Moore CA, Staples JE, Dobyns WB, et al. Characterizing the
pattern of anomalies in congenital Zika syndrome for pediatric
clinicians. JAMA Pediatr 2017; 171: 288–295.
2. Hoen B, Schaub B, Funk AL, et al. Pregnancy outcomes after
ZIKV infection in French territories in the Americas. N Engl J
Med 2018; 378: 985–994.
3. Mulkey SB, Bulas DI, Vezina G, et al. Sequential neuroima-
ging of the fetus and newborn with in utero Zika virus expos-
ure. JAMA Pediatr 2019; 173: 52–59.
4. Mulkey SB, Arroyave-Wessel M, Peyton C, et al.
Neurodevelopmental abnormalities in children with in utero
Zika virus exposure without congenital Zika syndrome.
JAMA Pediatr 2020; 174: 269–276.
5. Mulkey SB and DeBiasi RL. Do not judge a book by its cover:
Critical need for longitudinal neurodevelopmental assessment
of in utero Zika-exposed children. Am J Trop Med Hyg 2020;
102: 913–914.
6. Zika Virus Individual Participant Data C. The Zika virus indi-
vidual participant data consortium: A global initiative to esti-
mate the effects of exposure to Zika virus during pregnancy
on adverse fetal, infant, and child health outcomes. Trop
AQ3 Med Infect Dis 2020; 5.
¶ 19. Bashshur RL, Mandil SH and Shannon GW. Telemedicine/
7. Muller WJ and Mulkey SB. Lessons about early neurodeve-
lopment in children exposed to ZIKV in utero. Nat Med
2019; 25: 1192–1193.
5
8. Demaerschalk BM, Blegen RN and Ommen SR. Scalability of
telemedicine services in a large integrated multispecialty
health care system during COVID-19. Telemed J E Health
2021; 27: 96–98.
tion of telemedicine amidst the COVID-19 pandemic: An
early descriptive analysis. Front Pediatr 2021; 9: 648631.
10. Smith AC, Thomas E, Snoswell CL, et al. Telehealth for
global emergencies: Implications for coronavirus disease
2019 (COVID-19). J Telemed Telecare 2020; 26: 309–313.
useful telemedicine-based resources for clinical research.
Telemed J E Health 2020; 26: 51–65.
12. Burke BLJr and Hall RW and Section on Telehealth Care.
Telemedicine: Pediatric applications. Pediatrics. 2015; 136:
e293–e308.
13. Tuckson RV, Edmunds M and Hodgkins ML. Telehealth.
N Engl J Med 2017; 377: 1585–1592.
14. DeWyer A, Scheel A, Kamarembo J, et al. Establishment of a
cardiac telehealth program to support cardiovascular diagno-
sis and care in a remote, resource-poor setting in Uganda.
PLoS One 2021; 16: e0255918.
15. Pearl PL, Sable C, Evans S, et al. International telemedicine
consultations for neurodevelopmental disabilities. Telemed J
E Health 2014; 20: 559–562.
16. Connery AK, Berrios-Siervo G, Arroyave P, et al.
Responding to the Zika epidemic: Preparation of a neurodeve-
lopmental testing protocol to evaluate young children in rural
Guatemala. Am J Trop Med Hyg 2019; 100: 438–444.
17. Thomas EE, Haydon HM, Mehrotra A, et al. Building on the
momentum: Sustaining telehealth beyond COVID-19.
J Telemed Telecare 2020: 1357633X20960638. AQ4
¶
18. Saini L, Madaan P, Bhagwat C, et al. Home-videos for neuro-
developmental follow-Up of high-risk infants during
COVID-19 pandemic: A simple and inexpensive tool.
J Trop Pediatr 2021; 67.
telehealth: An international perspective. Executive
summary. Telemed J E Health 2002; 8: 95–107.