RESEARCH/Original Article

Journal of Telemedicine and Telecare

0(0) 1–7
Video interpretation and diagnosis of ! The Author(s) 2019
Article reuse guidelines:
pediatric amblyopia and eye disease sagepub.com/journals-permissions
DOI: 10.1177/1357633X19864823
journals.sagepub.com/home/jtt

Kourosh Sabri1,2,3 , Prima Moinul1, Nasrin Tehrani4,

Rick Wiggins5, Natalie Fleming3 and Forough Farrokhyar2,6,7

Abstract
Aim: The aim of this study was to assess the potential of using video screening to interpret the results of paediatric eye
examinations.
Design: Prospective multi-centred, blinded study.
Methods: Children aged 5 months to 11 years referred to a paediatric ophthalmology centre were enrolled in the
study. Outcome measures included the degree of agreement between examiners for assessment of various aspects of
paediatric eye examination. In Phase 1, children were individually assessed in the clinic by three different examiners to
determine the level of agreement. In Phase 2 a video recording was made of the first ophthalmologist examining the
children. The other two examiners viewed the video recordings to make their diagnoses. Areas of assessment included
lid function, pupillary function, ocular motility, strabismus, nystagmus, torticollis and facial asymmetry. Agreement
between examiners was measured using Gwet’s agreement coefficient (AC1).
Results: A total of 27 patients in Phase 1 (mean age 4.0 years) and 160 children in Phase 2 (mean age 4.8 years)
underwent clinical and video-recorded screening. In Phase 1, all but one area of ocular examination (heterotropia)
achieved
84% agreement between three examiners. In Phase 2, there was greater variation between direct clinical
examination and interpretation of video findings, ranging from 55–100% agreement.
Conclusion: Using experienced clinicians and changing only one variable in Phase 2 (the method of assessment – direct
examination versus video interpretation), the results show the possible usefulness of video-recorded screening as a
means of assessing children. Further research is indicated to assess the accuracy of ophthalmologists interpreting video
recordings of eye examinations performed by trained non-eye-care professionals.

Keywords
Tele-ophthalmology, telemedicine
Date received: 14 May 2019; Date accepted: 28 June 2019

Introduction 1
Department of Surgery, Division of Ophthalmology, McMaster
Despite high healthcare standards and the dawn of tele- University, Canada
2
medicine,1 29% of Canadians report difficulty access- Department of Surgery, McMaster Paediatric Eye Research
Group, Canada
ing timely care attributed to long wait times and 3
Department of Surgery, McMaster Pediatric Surgery Research
difficulty securing an appointment with specialists.2 Collaborative, Canada
This is likely due to the fact that 21% of Canada’s 4
Department of Surgery, Division of Ophthalmology, University of
population lives in rural, isolated communities where Toronto, Canada
5
Department of Optometry, University of Waterloo, Canada
only 2.4% of specialists are situated.2 In Ontario, 30% 6
Department of Health Research Methods, Evidence and Impact,
of the population live in rural communities and must McMaster University, Canada
travel up to an average of 178 km to see an ophthal- 7
Office of Surgical Research Services, Department of Surgery, McMaster
mologist.2 In Yukon, Nunavut and the Northwest University, Canada
Territories, patients travel an average of 995 km to
Corresponding author:
see an ophthalmologist.2 Kourosh Sabri, Division of Ophthalmology, McMaster University,
Screening in children to detect and treat amblyopia Hamilton, 1200 Main Street East, Ontario L8N 3Z5, Canada.
is of paramount importance.3 Recent studies have Email: sabrik@mcmaster.ca
2 Journal of Telemedicine and Telecare 0(0)
found the incidence of amblyopia to be 4.5% and 8.6%
in Ethiopia and India, respectively.4,5 Children growing
up with one amblyopic eye have almost three times the
risk (compared to non-amblyopes) of losing vision in
their fellow eye.6 In 1998, Bamford and colleagues rec-
ommended that preschool children must be screened
for strabismus and amblyopia by the age of 3–4 years
to prevent vision loss.3 However, nationally less than
14% of children under 6 years old receive an eye exam-
ination.7 In Canada, most children do not receive
vision screening or photo-screening by their paediatri-
cian or family physician. In 2004, a survey of two
Ontarian cities revealed that 10–15% of Ontario’s
preschool children showed visual acuity deficits.8 The
gap between the need for and delivery of eye care for
children is further increased by the lack of any telemed-
icine programmes that offer screening for amblyopia in
Canada. Given the importance of detecting amblyo-
genic eye disease and that younger children are signif-
icantly more responsive to treatment for amblyopia
than older children,9 access to timely ocular assessment
is imperative.
Except for diagnosing retinopathy of prematurity,
almost all the use of telemedicine is focused on adult
eye disease.10–14
In this study, we assess the potential of video screen-
ing as a tool for interpreting clinical paediatric eye
examinations both for screening and as a means of
detecting abnormal findings.
Methods
This prospective multi-centred cohort study involved
collaboration between eye-care professionals from
three Canadian centres: two children’s hospitals and
one university. One clinician from each site collaborat-
ed in the study: two of the examiners were experienced
paediatric ophthalmologists and the third was an expe-
rienced paediatric optometrist. All children were
recruited from and examined at McMaster Children’s
Hospital. The study was carried out following the
tenets of the Declaration of Helsinki and the Good
Clinical Practice guidelines and was approved by rele-
vant local research boards.
Study population
Enrolment occurred between February 2015 and
March 2016. All children attending the paediatric eye
clinic at the McMaster Children’s Hospital during this
period with suspected amblyopia met the inclusion cri-
teria. Informed parental consent was obtained prior to
patient enrolment.
The clinicians examining the children directly (Phase
1) or indirectly (Phase 2) assessed upper lid function,
presence of ptosis, pupillary function (relative afferent
pupillary defect (RAPD) and anisocoria), ocular motil-
ity (ductions), strabismus (using a cover test in the pri-
mary position), nystagmus in primary position,
torticollis and presence of facial asymmetry. As the
cooperation of the child and their level of fatigue can
affect measurement of heterophoria and angle of het-
erotropia, for this study only the strabismus assessment
used was the absence or presence of heterotropia in the
primary position.
Phase 1. In total, 27 patients underwent examination
three times on the same day, once by each of the three
study clinicians. Each examiner completed their respec-
tive diagnostic reports, after having performed a set of
standardized tests. Examiners were blinded to each
other’s assessments. These data were used to assess the
level of agreement between the three examiners.
Phase 2. Overall, 160 children underwent direct clinical
examination at the ophthalmology clinic by the first
ophthalmologist from the previous phase (examiner
1). This was considered the gold standard for compar-
ison to examination via telemedicine, as to date direct
clinical examination still provides the most accurate
method of examining children’s eyes. Research assis-
tants video recorded the examinations during the
assessment by examiner 1 using the available ambient
light provided by the clinic’s exam rooms. Recordings
were made from the clinician’s perspective using a
Camcorder-Canon high-definition (HD) Vixia HF
R200 with 20X Optical Zoom and 28X Advanced
Zoom and a tripod. The video recordings were
encrypted and uploaded securely on to a password-
protected server as compressed 1080i video files with
standard 30 fps frame rate, which could only be
accessed by the three study examiners. Examiners 2
and 3 downloaded the recorded videos of the eye
examinations to their respective computers, which
had the hardware capability to show HD videos and
used the video player of their choice to complete their
assessments and evaluations. The video provided one
view of the child and did not have the option to pan,
zoom or change the viewpoint in any way. Brightness,
contrast and colour could all be changed per the limi-
tations of the computer screens. There was no limita-
tion on how many times examiners 2 and 3 could view
each video.
Statistical analysis
The level of inter-observer and inter-method (in-clinic
versus video screening) agreement between the clini-
cians was measured for each area of ocular examina-
tion assessed.
Sabri et al.
The Gwet agreement coefficient (AC1) was used to
assess level of agreement between examiners 1 and 2
and examiners 1 and 3 in Phases 1 and 2 of the study.
Agreement was assessed independently for the ophthal-
mologist and optometrist to avoid expertise bias. Gwet
AC1 was preferred over Cohen’s Kappa as it has been
shown to provide a more stable agreement coefficient
by correcting for chance when there is a high expected
level of agreement between expert raters and is less
affected by prevalence and marginal probability of
the condition under study.17
For Phase 2, the level of agreement was estimated
using Gwet AC1, sensitivity and specificity for each
measured ocular feature between examiner 1 against
examiners 2 and 3, respectively. Gwet AC1, sensitivity
and specificity, with 95% confidence interval (CI), are
reported. StatsDirect statistical software (www.statsdir
ect.com) and AgreeStat for Excel for Windows (www.
agreestat.com) were used for the Gwet and diagnostic
test analyses.
Sample-size calculation
Phase 1. Level of agreement could vary from 0 to 100%
between raters. Agreement of <20% is considered
poor, 20–40% is fair, 40–60% is moderate, 60–80% is
substantial and >80% is almost perfect.15 The prede-
fined acceptable agreement criterion between clinicians
was set to a minimum of 80% in each domain of the
clinical assessment, indicating a high level of agree-
ment. In doing so, any observed differences in Phase
2 in the interpretation of the examination findings
between the video examination (by examiners 2 and
3) and direct examination (by examiner 1) could be
confidently attributed to limitations of the video exam-
ination tool and not inter-observer variability. The
minimum of 27 participants in Phase 1 was required
to achieve a minimum agreement coefficient of 0.8
(substantial agreement) assuming an alpha error of
0.05 and beta error of 0.2 on a dichotomous diagnosis
of positive finding on ocular examination by the
three raters.16
Phase 2. The sample size was calculated based on esti-
mates of sensitivity and specificity of 95%. For an
alpha error of 0.05 and assuming that 50% of the
patients will be diagnosed with a positive finding on
ocular examination that may be amblyogenic, a total
of 146 patients of whom at least 50% should have a
positive finding on ocular examination, were required
to achieve a confidence interval of þ/ 5% around the
sensitivity and specificity of 95%.
3
Table 1. Patient demographic data for Phases 1 and 2.
Phase 1 ¼ 27 Phase 2 ¼ 160
children children
Gender
Male 12 (44%) 77 (48%)
Female 14 (52%) 83 (52%)
Unknown 1 (4%) 0
Age (years)
Mean (standard deviation) 4.0 (3.0) 4.8 (2.5)
Median (range) 4.0 (7–8) 5.0 (5–11)
Results
There were 27 children enrolled in Phase 1 and 160
children in Phase 2. Patient demographics are included
in Table 1.
Phase 1. Table 2 reports the level of agreement in Phase
1 between examiner 1 and the other two examiners. The
column labelled ‘number of positive cases’ shows the
number of patients with the said diagnosis where appli-
cable. Overall, except for the agreement in the diagno-
sis of heterotropia between examiners 1 and 3, there
was a high level of agreement (>84%) in all areas of
assessment indicating more than substantial levels of
agreement between examiners 1 and 2 and examiners
1 and 3, respectively. The level of agreement between
examiners 1 and 3 for the diagnosis of heterotropia was
69%. Possible reasons for this somewhat lower level of
agreement will be explored in the discussion sec-
tion below.
Phase 2. In Phase 2, the findings obtained from video
screening (by examiners 2 and 3) were compared to the
gold-standard clinical assessment (by examiner 1). The
overall level of agreement was greater than 90% in all
areas of assessment except strabismus for both exam-
iners 2 and 3. The specificity for both examiners 2 and 3
was 90% or higher for all domains of ocular examina-
tion assessed. Overall agreement for diagnosing heter-
otropia in the primary position was 63% for examiner
2 and 55% for examiner 3. Looking at subsets of stra-
bismus, agreement for diagnosing esotropia was 73%
for examiner 2 and 65% for examiner 3 whereas the
agreement for diagnosing exotropia was the highest:
95% for examiner 2 and 93% for examiner 3.
Reviewing the results in Table 3, the sensitivity
ranged from 33% to 100% for examiner 2 and from
0 to 86% for examiner 3.
Discussion
In this study, all variables except the method of exam-
ination (direct versus interpretation of video) had to
4 Journal of Telemedicine and Telecare 0(0)

Table 2. Phase 1: Assessment of inter-rater reliability when performing direct clinical examination (N ¼ 27 children).

Number of cases Number of cases Reliability between Reliability between

with positive finding with no positive examiners 1 and 2 examiners 1 and 3
Ocular examination (abnormal) finding (normal) (%) (95% CI) (%) (95% CI)

Upper lid function 0 54 eyes 92 (80–100) 96 (99–100)

Presence of ptosis 0 54 eyes 96 (88–100) 100
Presence of RAPD 0 27 children 100 100
Presence of anisocoria 2 25 children 92 (76–100) 96 (87–100)
Assessment of abduction 0 54 eyes 92 (80–100) 87 (72–100)
Assessment of adduction 0 54 eyes 100 100
Assessment of elevation 0 54 eyes 100 96 (88–100)
Assessment of depression 0 54 eyes 100 100
Presence of nystagmus 0 54 eyes 100 100
Presence of torticollis 0 27 children 92 (80–100) 100
Presence of facial asymmetry 0 27 children 100 96 (88–100)
Presence of heterotropia 14 13 children 92 (71–100) 69 (40–98)
Presence of esotropia 10 17 children 100 84 (48–100)
Presence of exotropia 4 23 children 100 84 (48–100)
CI: confidence interval; RAPD: relative afferent pupillary defect.

remain the same. If more than one variable was
changed in Phase 2, then any observed difference in
ocular assessment between direct clinical examination
and video interpretation could not be confidently
assigned to the use of video recording alone. That is
why in Phase 2 both the direct examination and video
interpretation were carried out by the same experienced
clinicians from Phase 1.
Video interpretation for the assessment of upper lid
function, presence of ptosis and RAPD, anisocoria,
assessment of ductions and presence of nystagmus, tor-
ticollis and facial asymmetry showed a very high level
of agreement for both examiners 2 and 3 when com-
pared to direct clinical examination by examiner 1. The
relatively small numbers of positive cases for condi-
tions such as presence of RAPD (one child), anisocoria
(two children) and exotropia (nine children) meant that
any ‘missed’ positive cases by video interpretation
would lead to a large drop in the sensitivity levels.
Furthermore, for some areas of assessment such as
ocular ductions and facial asymmetry, there were no
positive (abnormal cases), which will have to be
addressed in future studies in this field.
Video-oculography has been proven to be effective
for diagnosing exotropia.18 However, the exclusion of
children younger than 4 years old and of all ocular
diagnoses other than exotropia prevented this study
from looking at a large population of young patients.
Cheung et al.19 used video screening to assess stra-
bismus but focused on an adult population. Dawson
et al.20 assessed the accuracy of using a remote telemed-
icine link to diagnose strabismus in adults and
highlighted the challenges in using this technique for
assessing children. Furthermore, the teleconferencing
technology used for assessment by the remote examiner
introduced logistical difficulties as the patient, camera
technician and the remote examiner all had to be avail-
able at the same time. Although Tan et al. have shown
that real-time telemedicine, via live videoconferencing
in adult patients, is comparable to clinical assessments
of ophthalmic conditions,21 they did not include any
children in their study. However, we have shown
there is the potential for video screening alone to be
comparable to clinical examination without real-time
consultation or photographs, even in paediatric
patients. A meta-analysis of real-time teleconferencing
versus clinical examination further reinforced the diag-
nostic accuracy of this technology and highlighted the
reduced burden of economic cost and service coverage
needed for face-to-face consultation.22 However,
Cheung and Dawson also highlighted that strictly per-
forming paediatric strabismus examinations is very
challenging using real-time videoconferencing.19,20
This study showed a high level of specificity but low
sensitivity in diagnosing anisocoria and torticollis for
both examiners performing video interpretation and
for diagnosing ptosis and RAPD for examiner 3. The
same is true for diagnoses of heterotropia and esotro-
pia: in areas of assessment where the number of posi-
tive (abnormal) cases was low, every positive case
missed by the clinicians interpreting the video exami-
nation will result in a large drop in sensitivity.
Although there was a high level of overall agreement
between examiners, the high specificity indicates
increased agreement when findings are within normal
limits. The low sensitivity for some areas of assessment
may be due to a combination of factors such as the low
number of positive cases (therefore skewing the
 Sabri et al.

Table 3. Phase 2: Assessment of inter-rater reliability, comparing findings from direct clinical assessment (examiner 1) with video interpretation (examiners 2 and 3)
(N ¼ 160 children).

Reliability of Reliability of
Number of cases Number of cases video screening video screening
with positive with no positive for examiner 2 Sensitivity Specificity for examiner 3 Sensitivity (%) Specificity (%)
Ocular Examination finding (abnormal) finding (normal) (%) (%) (95% CI) (%) (95% CI) (%) (95% CI) (95% CI)

Upper lid function 0 320 eyes 100 NA 100 (98–100) 98 (98–100) NA 100 (98–100)
(abnormal)
Presence of ptosis 2 318 eyes 100 100 (16–10) 100 (98–100) 99 (98–100) 50 (1–98) 100 (98–100)
Presence of RAPD 1 159 children 100 100 (97–100) 100 (97–100) 99 (98–100) 0 (0–97) 100 (97–100)
Presence of anisocoria 2 158 children 98 (96, 100) 50 (1–98) 99 (96–98) 98 (96–100) 0 (0–84) 99 (96–100)
Assessment of abduction 0 320 eyes 96 (94–99) 83 (43–97) 97 (94–98) 95 (93–100) 12 (3–52) 98 (95–99)
Assessment of adduction 0 320 eyes 98 (97–100) 100 (16–100) 99 (96–99) 98 (97–100) 0 (0–84) 99 (97–100)
Assessment of elevation 0 320 eyes 99 (98–100) NA 100 (98–100) 100 NA 100 (99–100)
Assessment of depression 0 320 eyes 99 (98–100) NA 99 (97–99) 100 NA 100 (99–100)
Presence of nystagmus 7 313 eyes 98 (96–100) 85 (42–99) 99 (96–100) 98 (96–100) 86 (42–99) 99 (96–100)
Presence of torticollis 9 151 children 94 (89–98) 33 (7–70) 99 (95–99) 94 (89–98) 10 (0–44) 100 (97–100)
Presence of facial 0 160 children 98 (97–100) NA 95 (90–98) NA NA 99 (95–99)
asymmetry
Presence of 76 84 children 63 (52–76) 72 (61–81) 93 (84–97) 55 (41–68) 64 (5–74) 90 (81–95)
heterotropia (any)
Presence of esotropia 67 93 children 73 (61–89) 74 (62–84) 97 (90–99) 65 (52–77) 60 (47–92) 97 (90–99)
Presence of exotropia 9 151 children 95 (91–99) 44 (13–79) 99 (95–99) 93 (89–98) 70 (34–93) 96 (91–98)
CI: confidence interval; RAPD: relative afferent pupillary defect.
NA: Not applicable in cases where there were no positive (abnormal) cases.
5
6 Journal of Telemedicine and Telecare 0(0)
sensitivity disproportionately) as well as technical dif-
ficulties with the video recording such as lighting, focus
and angle of view affecting the quality of the view for
the remote examiners. Furthermore, although both
examiners 2 and 3 had over 20 years of clinical experi-
ence, the observed differences between the two of them
in terms of reliability and sensitivity in Phase 3 may in
part be due to the fact that not all individuals are
equally tech savvy and able to use computer software.
Therefore, future emphasis needs to be placed on train-
ing the clinicians in how to best use the necessary com-
puter software to optimise their ability to make
diagnoses based on viewing video recordings.
The diagnosis of esotropia had relatively lower
agreement in Phase 2 of the study, whereas on the con-
trary, the diagnosis of exotropia achieved a much
higher level of agreement. The explanation for this
may be due to several factors including reduced patient
cooperation, as the strabismus assessment was the very
last part of the ocular examination performed on the
child and therefore the child may have become increas-
ingly restless and uncooperative by that stage, affecting
the quality of the video recording. Another reason
may be technical difficulties with the viewing angle
of the video recording, not allowing examiners 2
and 3 to view the ocular alignment in the true primary
position.
This study has limitations. Most patients in Phase 1
did not have any ocular motility, pupillary, or upper lid
abnormalities. Nonetheless, it is important to ensure
high specificity in all cases to minimize false-positive
diagnoses. Agreement on the normal examinations is
essential as many children referred to eye-care profes-
sionals may have only one ophthalmic abnormality and
therefore agreement on pertinent negatives is important
to avoid false positives. Silbert et al. highlighted this by
showing that 42.3% of patients screened for amblyopia
from a random selection of 521 patients at a paediatric
ophthalmology clinic did not have amblyopia.23
Technical difficulties that can affect the quality of the
video recording and hence the ability of viewers to
make accurate diagnoses include reduced patient coop-
eration (bearing in mind children can become restless
very easily), motion artefact,19 glare, optical aberration
or blocked view by the clinical examiner’s movements.
Furthermore, as assessments required only a ‘yes’ or
‘no’ answer, the severity of conditions such as ocular
motility defects or heterotropia could not be assessed.
Finally, due to the low number of cases with an RAPD
and absence of any cases with duction deficits, further
studies with positive cases of these abnormalities are
required to test the sensitivity of this technique in diag-
nosing such abnormalities.
Conclusion
To the best of our knowledge, this is the only study to
date that has looked at the feasibility of telemedicine to
diagnose abnormal findings on a clinical examination
in a large paediatric population. Future, larger-scale
studies with a greater number of patients with positive
diagnoses are required to assess the sensitivity of this
technique. Once sensitivity reaches an acceptable level,
next steps can look at using trained non-eye-care pro-
fessionals to perform a direct examination that is
recorded. An experienced eye-care professional can
then view the video recording remotely. The possibility
of training a novice to conduct the direct eye examina-
tion during the recording is an interesting option that
deserves further research. Hrynchak et al.24 showed
that ocular motility testing, assessed by alternative
cover test, did not show any clinically significant dif-
ference between an experienced and novice examiner.
Such a screening tool will not only help deliver care to
those children with limited or no access to secondary-
or tertiary-level eye care but will also help reduce costs
and morbidity related to unnecessary referrals to ter-
tiary ophthalmology centres.
Declaration of Conflicting Interest
The authors declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of
this article.
Funding
The authors disclosed receipt of the following financial sup-
port for the research, authorship, and/or publication of this
article: This study was funded by a Physician Service
Incorporated (PSI) grant.
ORCID iD
Kourosh Sabri https://orcid.org/0000-0001-7565-287X
References
1. Ferguson EW, Doarn CR and Scott JC. Survey of global
telemedicine. J Med Syst 1995; 19: 35–46.
2. Pong R and Pitblado J. Geographic Distribution of
Physicians in Canada: Beyond How Many and Where.
Ottawa, www.cihi.ca (2005, accessed 2 May 2019).
3. Bamford J, Davis A, Boyle J, et al. Preschool hearing,
speech, language, and vision screening. Qual Health Care
1998; 7: 240–247.
4. Gupta M, Rana SK, Mittal SK, et al. Profile of
Amblyopia in School going (5–15 years) Children at
State Level Referral Hospital in Uttarakhand. J Clin
Diagnostic Res 2016; 10: SC09–SC11.
5. Darge HF, Shibru G, Mulugeta A, et al. The prevalence
of visual acuity impairment among school children at
Sabri et al.
Arada Subcity Primary Schools in Addis Ababa,
Ethiopia. J Ophthalmol 2017; 2017: 1–7.
6. van Leeuwen R, Eijkemans MJC, Vingerling JR, et al.
Risk of bilateral visual impairment in individuals with
amblyopia: the Rotterdam study. Br J Ophthalmol
2007; 91: 1450–1451.
7. The Canadian Association of Optometrists. The Cost of
Children’s Vision Loss, https://opto.ca/news/the-cost-of-
childrens-vision-loss (2014, accessed 22 April 2019).
8. Feightner J. Routine preschool screening for visual and
hearing problems. Ottawa, http://www.hc-sc.gc.ca/hppb/
healthcare/pdf/clinical_preventive/s2c27e.pdf.
9. Holmes JM, Lazar EL, Melia BM, et al. Effect of age on
response to amblyopia treatment in children. Arch
Ophthalmol 2011; 129: 1451.
10. Lamminen H, Voipio V, Ruohonen K, et al.
Telemedicine in ophthalmology. Acta Ophthalmol Scand
2003; 81: 105–109.
11. Ng M, Nathoo N, Rudnisky CJ, et al. Improving access
to eye care: Teleophthalmology in Alberta, Canada.
J Diabetes Sci Technol 2009; 3: 289.
12. Ells AL, Holmes JM, Astle WF, et al. Telemedicine
approach to screening for severe retinopathy of prematu-
rity. Ophthalmology 2003; 110: 2113–2117.
13. Zimmer-Galler IE, Kimura AE, Gupta S. Diabetic reti-
nopathy screening and the use of telemedicine. Curr Opin
Ophthalmol 2015; 26: 167–172.
14. Wedekind L, Sainani K, Pershing S. Supply and per-
ceived demand for teleophthalmology in triage and con-
sultations in California emergency departments. JAMA
Ophthalmol 2016; 134: 537.
15. Viera Aj and Garrett JM. Understanding interobserver
agreement: The kappa statistic. Fam Med 2005;
37: 360–363.
7
16. Walter SD, Eliasziw M and Donner A. Sample size and
optimal designs for reliability studies. Stat Med 1998;
17: 101–110.
17. Wongpakaran N, Wongpakaran T, Wedding D, et al.
A comparison of Cohen’s Kappa and Gwet’s AC1
when calculating inter-rater reliability coefficients:
A study conducted with personality disorder samples.
BMC Med Res Methodol 2013; 13: 61.
18. Park N, Park B, Oh M, et al. A quantitative analysis
method for comitant exotropia using video-oculography
with alternate cover. BMC Ophthalmol 2018; 18: 80.
19. Cheung JC, Dick PT, Kraft SP, et al. Strabismus exam-
ination by telemedicine. Ophthalmology 2000;
107: 1999–2005.
20. Dawson E, Kennedy C, Bentley C, et al. The role of
telemedicine in the assessment of strabismus. J Telemed
Telecare 2002; 8: 52–55.
21. Tan JC, Eugenie Wei TP, Sanjay S, et al. A pilot trial of
tele-ophthalmology for diagnosis of chronic blurred
vision. J Telemed Telecare 2013; 19: 65–69.
22. Tan IJ, Dobson LP, Bartnik S, et al. Real-time teleoph-
thalmology versus face-to-face consultation: A systematic
review. J Telemed Telecare 2017; 23: 629–638.
23. Silbert DI, Matta NS, Burkholder D, et al. Clinical accu-
racy of the AAPOS pediatric vision screening referral
criteria. J Am Assoc Pediatr Ophthalmol Strabismus
2012; 16: 361–364.
24. Hrynchak PK, Herriot C and Irving EL. Comparison of
alternate cover test reliability at near in non-strabismus
between experienced and novice examiners. Ophthalmic
Physiol Opt 2010; 30: 304–309.