Prospective, Randomized Trial Comparing
Simulator-based versus Traditional Teaching of
Direct Ophthalmoscopy for Medical Students
GRANT L. HOWELL, GERMÁN CHÁVEZ, COLIN A. MCCANNEL, PETER A. QUIROS, SABA AL-HASHIMI, FEI YU,
SIMON FUNG, CHRISTOPHER M. DEGIORGIO, YUE MING HUANG, BRADLEY R. STRAATSMA,
CLARENCE H. BRADDOCK, AND GARY N. HOLLAND
• OBJECTIVE: To compare results of simulator-based vs
traditional training of medical students in direct ophthal-
moscopy.
• DESIGN: Randomized controlled trial.
• METHODS: First-year medical student volunteers com-
pleted 1 hour of didactic instruction regarding direct oph-
thalmoscopes, fundus anatomy, and signs of disease. Stu-
dents were randomized to an additional hour of training
on a direct ophthalmoscope simulator (n = 17) or super-
vised practice examining classmates (traditional method,
n = 16). After 1 week of independent student prac-
tice using assigned training methods, masked ophthalmol-
ogist observers assessed student ophthalmoscopy skills
(technique, efficiency, and global performance) during
examination of 5 patient volunteers, using 5-point Lik-
ert scales. Students recorded findings and lesion location
for each patient. Two masked ophthalmologists graded
answer sheets independently using 3-point scales. Stu-
dents completed surveys before randomization and after
assessments. Training groups were compared for grades,
observer- and patient-assigned scores, and survey re-
sponses.
• RESULTS: The simulator group reported longer practice
times than the traditional group (P = .002). Observers
assigned higher technique scores to the simulator group
after adjustment for practice time (P = .034). Combined
grades (maximum points = 20) were higher for the sim-
Supplemental Material available at AJO.com.
Accepted for publication November 9, 2021.
UCLA Stein Eye Institute, David Geffen School of Medicine at UCLA,
University of California, Los Angeles; Department of Ophthalmology,
David Geffen School of Medicine at UCLA, University of California, Los
Angeles; UCLA Simulation Center, David Geffen School of Medicine
at UCLA, University of California, Los Angeles; Doheny Eye Institute,
Pasadena; Departments of Neurology, Cardiology, and Neurobiology,
David Geffen School of Medicine at UCLA, University of California, Los
Angeles, California, USA; Department of Anesthesiology and Periopera-
tive Medicine, David Geffen School of Medicine at UCLA, University of
California, Los Angeles, California, USA; Office of the Dean, David Gef-
fen School of Medicine at UCLA, University of California, Los Angeles,
California, USA; Department of Biostatistics, UCLA Jonathan and Karin
Fielding School of Public Health, Los Angeles
Inquiries to Gary N. Holland, UCLA Stein Eye Institute, 100
Stein Plaza, UCLA, Los Angeles, CA 90095-7000, USA.; e-mail:
uveitis@jsei.ucla.edu
© 2021 THE AUTHORS. PUBLISHED BY ELSEVIER INC.
0002-9394/$36.00 THIS IS AN OPEN ACCESS ARTICLE UNDER THE CC BY-NC-ND LICENSE
https://doi.org/10.1016/j.ajo.2021.11.016 (HTTP://CREATIVECOMMONS.ORG/LICENSES/BY-NC-ND/4.0/).
ulator group (median: 5.0, range: 0.0-11.0) than for the
traditional group (median: 4.0, range: 0.0-9.0), although
the difference was not significant. The simulator group
was less likely to mistake the location of a macular scar
in 1 patient (odds ratio: 0.28, 95% confidence interval:
0.056-1.35, P = .013).
• CONCLUSIONS: Direct ophthalmoscopy is difficult, re-
gardless of training technique, but simulator-based train-
ing has apparent advantages, including improved tech-
nique, the ability to localize fundus lesions, and a foster-
ing of interest in learning ophthalmoscopy, reflected by
increased practice time. (Am J Ophthalmol 2022;238:
187–196. © 2021 The Authors. Published by Elsevier
Inc.
This is an open access article under the CC BY-NC-ND
license
(http://creativecommons.org/licenses/by-nc-nd/4.0/))
D
irect ophthalmoscopy allows clinicians, regardless
of specialty, to visualize fundus findings, includ-
ing diabetic retinopathy and papilledema, that are
critical for the evaluation of patients with systemic disor-
ders. The Association of University Professors of Ophthal-
mology recommends that all medical students learn to ap-
preciate anatomic landmarks and signs of disease through
direct ophthalmoscopy.1 Nevertheless, studies report that
medical students, nonophthalmology residents, and prac-
ticing physicians lack confidence in their ophthalmoscopy
skills.2-4 Both educators and trainees attribute the lack
of skill, in part, to inadequate training during medical
school.5 , 6 In a survey of residency program directors in pri-
mary care specialties, respondents felt that medical schools
were not training students adequately to perform direct
ophthalmoscopy once they are in residencies.6
Traditionally, teaching direct ophthalmoscopy skills in-
volves student-on-student (or student-on-patient) practice;
however, such instruction involves many challenges that
limit its educational value, including an inability to pro-
vide students with accurate feedback about what they have
seen in the fundi of examinees.7 Studies have evaluated al-
ternative teaching techniques, including use of “teaching”
ophthalmoscopes, which provide instructors with images of
187
what students see; fundus photography; and direct ophthal-
moscope simulators.8-12 Simulation has been shown to im-
prove surgical skills among residents,13-15 but evidence is
limited regarding the value of simulation for teaching direct
ophthalmoscopy to students.12 In this article, we describe
the results of a randomized controlled trial that compared
simulator-based vs traditional teaching of direct ophthal-
moscopy to first-year medical students.
METHODS
This prospective, randomized, controlled trial was con-
ducted at the David Geffen School of Medicine at UCLA.
First-year medical student volunteers were recruited in May
2019 to participate in the study via e-mail and social media
announcements. Students were not eligible for participa-
tion in the trial if they had had more than an introduc-
tory demonstration of the direct ophthalmoscope or had
practiced use of a direct ophthalmoscope independently.
Meals were provided at study sessions, and students were
given $50 Amazon gift cards on study completion. Five pa-
tient volunteers with various fundus findings were recruited
from faculty practices to participate in the assessment phase
of the study. Patients were given $100 Amazon gift cards
for participation. Seven ophthalmologists with experience
in medical student instruction volunteered to assess stu-
dent performance; 5 evaluated students as they examined
patients (see Acknowledgments) and 2 (P.A.Q., S.A.-H.)
graded answer sheets on which students recorded their
findings. Fundus imaging of the study eye of each patient
had been performed with photography or optical coher-
ence tomography to assist in grading student answer sheets
(Figure 1). All assessments were masked to each student’s
training group. The study was approved by the UCLA Insti-
tutional Review Board before participant recruitment and
data collection.
• DATA COLLECTION AND STUDY PROCEDURES: The
study was conducted in 3 phases: training, practice, and
skills assessment. Students completed a pretraining survey
that collected the following information: prior direct oph-
thalmoscopy experience (yes vs no), prior study of eye dis-
ease (yes vs no), and the presence or absence of pre-existing
eye conditions (refractive error corrected by glasses or con-
tact lenses, color-blindness, amblyopia). Investigators not
involved in training or assessments randomized (F.Y.) and
enrolled (G.C.) student volunteers.
Training was conducted in 3 identical sessions on con-
secutive days. The session in which each student could par-
ticipate was dictated by other school responsibilities. Train-
ing began with a 1-hour didactic session on fundus anatomy,
signs of retinal and optic disc disease, and basic principles of
direct ophthalmoscopy (ophthalmoscope controls, proper
188 AMERICAN JOURNAL OF OPHTHALMOLOGY
technique, and common mistakes made by students learn-
ing to use the ophthalmoscope). Signs of retinal disease in-
cluded examples similar to those in study eyes of patient
volunteers, but were not limited to those signs; examples
included papilledema, large optic disc cups, chorioretinal
scars, drusen, signs of vasculopathies and ischemia (hem-
orrhage, cotton-wool spots, exudates), and nevi. Students
in each session had been randomized 1:1 into simulator
and traditional training groups using block randomization,
but assignments for each day were revealed after didactic
instruction. All participants on a given day attended the
same didactic session, regardless of randomization assign-
ment. Didactic sessions were conducted by the same in-
structor each day (G.N.H.). Training groups ranged in size
from 4 to 8 students.
The simulator group was introduced to the Eyesi Direct
Ophthalmoscope Simulator (Model EDO491 #03 × 0127,
Platform 2.1, Software v1.8.0.113443; VRmagic GmbH,
Mannheim, Germany) during an additional hour by 1 fac-
ulty instructor (C.A.M.) and a manufacturer’s representa-
tive. The simulator contains 4 “modules,” each with numer-
ous “cases” that are scored on the operator’s ability to com-
plete tasks.16 The first 2 modules focus on finding geomet-
ric shapes superimposed on a simulated fundus. The simu-
lator registers whether the operator successfully locates the
shape. A cross hair indicating the position of gaze must be
held over the shape for 1 second for the simulator to regis-
ter it as having been “seen.” The simulator also records and
displays a map of the total fundus area that has been viewed,
thereby promoting efficient scanning of the entire posterior
pole. Each student in the simulator group used the first mod-
ule during the training hour. The third and fourth modules
display simulated pathologic retinal and optic disc lesions.
The fourth (examination) module evaluates the operator’s
ability to identify lesions and was not available to students
during the study. The simulator also addresses some aspects
of the direct ophthalmoscopy technique. Some cases re-
quire operators to adjust “lenses” within the ophthalmo-
scope hand piece to obtain a clear view of the simulated fun-
dus; another penalizes operators unless the fundus is viewed
through simulation of an undilated pupil.
During the same hour, the traditional training group ex-
amined fellow students with their personal direct ophthal-
moscopes under the guidance of 2 faculty instructors (S.F.,
G.N.H.). Pupillary dilation of examined students was op-
tional. After training, all students were encouraged to prac-
tice ophthalmoscopy independently, using only their as-
signed training method (practice on the simulator vs prac-
tice on family members, friends, or fellow students).
All students were assessed at a single session, 4 days af-
ter the final training day. Students were placed in ran-
dom order to begin assessments. Patient volunteers were
placed in 5 examination lanes. Study eyes were dilated with
tropicamide 1% and phenylephrine 2.5%. Before examina-
tions, students were oriented to an answer sheet on which
they were to record (1) abnormal findings in the posterior
JUNE 2022
FIGURE 1. Clinical images illustrating pertinent findings in the fundi of 4 patient volunteers who were examined by first-year
medical students randomized to simulator-based vs traditional training of direct ophthalmoscopy. A. Patient 1: enlarged optic disc
cup, left eye. B. Patient 2: toxoplasmic retinochoroidal scar inferotemporal to the fovea, right eye. C. Patient 4: red-free image of a
large macular disciform scar, left eye. D. Patient 5: age-related macular degeneration with central pigmentary changes and drusen,
left eye. An additional patient volunteer (patient 3) had no retinal or optic disc lesions and was considered to have a normal fundus.

pole and (2) location of those findings on fundus diagrams
printed on the answer sheet. Students were asked to de-
scribe lesion characteristics (size, color, and shape) in their
own words. Diagnoses were not required. Two additional
options were “normal fundus,” if they found no abnormali-
ties, and “could not visualize fundus.”
Students examined all 5 patient volunteers in the same
order; after the first student examined the first patient and
moved to the second patient, the second student in line ex-
amined the first patient, and so forth until each patient had
been examined by every student. A single masked ophthal-
mologist observer was assigned to each examination lane
for the entire assessment period. On entry into the exami-
nation lane, students were told which eye was to be exam-
ined, but no further instructions were given. Students were
told not to reveal their training technique to observers or
patients, and patients had been told not to reveal their eye
problems to students. Both observers and patients were told
not to provide students with feedback during the examina-
tion. Students had 1 minute to perform each examination,
during which observers assigned them 3 scores (technique,
efficiency, and global, as described below). After each ex-
amination, students exited the examination lane and had
1 minute to complete answer sheets before moving to the
next patient. During the same minute, patients assigned a
single global score for student performance, as described be-
low. Students were asked not to discuss their answers with
other students until all examinations were completed and

VOL. 238 DIRECT OPHTHALMOSCOPY SIMULATION 189

answer sheets submitted. At the end of the assessment ses-
sion, observers confirmed that no students had revealed
their training techniques.
After examining the fifth patient and submitting the an-
swer sheet, each student completed a standard postassess-
ment survey. The following information was collected: in-
dependent practice time (none, <30 minutes, 30-59 min-
utes, 1-3 hours, >3 hours), perceived advantages and dis-
advantages of the training method to which the student
was assigned (open-field question), and confidence in di-
rect ophthalmoscopy (5 categories ranging from “not at all
comfortable” to “very comfortable”).
The graders (S.A.-H., P.A.Q.) reviewed all examination
answer sheets and independently assigned 1 of 3 grades (0,
1, or 2 points) for answers to the 2 questions for each study
eye (location, description of findings); grading criteria are
described below. After grading, they met to adjudicate dis-
crepant results and achieve a single, final grade for each
question. The graders had had no contact with students dur-
ing the course of the study, and students’ training assign-
ments were not listed on the answer sheets.
• CONVENTIONS AND DEFINITIONS: Rubrics were cre-
ated prospectively to guide evaluators (ophthalmologist ob-
servers, patient volunteers) in assigning scores using 5-point
Likert scales (Table 1). The global score assigned by an oph-
thalmologist observer was based on a gestalt of the student’s
overall performance. The single score assigned by patients
was based on their perceptions of a student’s performance in
3 areas: professionalism, technique, and confidence. Oph-
thalmologist graders were also provided with guidelines for
assigning grades to answer sheets (Table 1).
• DATA ANALYSIS AND STATISTICAL TECHNIQUES: Pri-
mary outcome measures were examination grades and
ophthalmologist-observer scores. Secondary outcomes in-
cluded patient-reported scores, student-reported practice
time, and perceived advantages/disadvantages of each
training method. Pretraining survey results were compared
between training groups, to identify any differences de-
spite randomization. Each component of the examination
grades (location, clinical findings) and ophthalmologist-
observer scores (technique, efficiency, global) was com-
pared between groups for each patient individually. The
sum of each student’s 3 observer scores for all 5 patients
and the sum of each student’s 2 examination grades for all
5 patients were compared between groups. Postassessment
survey results were compared between groups.
Statistical analysis was performed using SAS software
version 9.3 (SAS Inc, Cary, North Carolina, USA). The
Fisher exact test was used to calculate differences in pre-
training and postassessment survey results, and to com-
pare proportions of specific grades assigned to individual
patients. Summary grades and scores were compared using
the Kruskal-Wallis test. Group differences after adjusting
for time practiced were calculated using multivariable lin-
190 AMERICAN JOURNAL OF OPHTHALMOLOGY
ear regression models. P values of <.05 were considered
statistically significant. We evaluated agreement between
grades assigned by the ophthalmologist graders using a sim-
ple Kappa statistic. Values of <0.4 were considered to be no
or mild agreement, 0.41-0.6 to be moderate agreement, and
>0.6 to be good or excellent agreement.
RESULTS
The study adhered to guidelines of the International Net-
work for Simulation-based Pediatric Innovation, Research,
and Education.17 A total of 39 first-year medical students
volunteered to participate and were randomized to sim-
ulator (n = 20) or traditional (n = 19) training groups
(Supplemental Figure). Before training and announce-
ment of randomization assignments, 4 students (1 in the
simulator group; 3 in the traditional group) decided not
to participate. An additional 2 students in the simulator
group did not attend the assessment session because of
conflicts with other student activities. Table 2 describes
characteristics of the 33 students (17 in the simulator
group; 16 in the traditional group) who completed train-
ing, assessment, and surveys. No significant differences
were found between the groups. Only 1 student noted
limited prior exposure to direct ophthalmoscopy. Among
17 students in the simulator group, 16 (94.1%) accessed
the third module with simulated pathologic fundus lesions
while practicing; 7 completed the entire module.
Table 3 summarizes grades assigned to students in each
group. Grades for both groups were low overall, but the sim-
ulator group had higher grades than the traditional group for
all measures, although differences were small and not statis-
tically significant. In addition, differences were not signifi-
cantly different when adjusted for practice time. The high-
est total grade was 11 of 20 (55%) points, assigned to a stu-
dent in the simulator group.
When grades were considered on an individual patient
basis, the simulator group had significantly higher location
and clinical finding grades for patient 4, who had a large dis-
ciform scar (Table 4; Figure, C). Differences remained sig-
nificant when adjusted for practice time. A mistake among
both groups was to label the scar as a large pale optic disc;
however, simulator-trained students were less likely to mis-
take the location of the lesion (odds ratio: 0.28, 95% con-
fidence interval: 0.056-1.35, P = .013).
When all 330 grades assigned to students (33 stu-
dents × 5 patients per student × 2 questions about each
patient) were compared, there was good agreement between
the 2 graders (270 [81.8%] of the 330 assigned grades were
identical between graders; simple Kappa estimate 0.613
[95% confidence interval: 0.532-0.694]). Agreement was
similar when comparisons were made between grades for lo-
cation only and for findings only (data not shown).
JUNE 2022
 TABLE 1. Prospective Rubrics to Guide Evaluators in Assigning Scores and Grades to Medical Student Participants During a
Randomized Controlled Trial Comparing Simulator-based vs Traditional Training Techniques for Direct Ophthalmoscopy.

Score or Grade Factors Related to Student Performance That Were Appropriate for Assigning Specific Scores and Grades

For ophthalmologist observersa , b

Technique
Score 1 (poor) Fumbles with the direct ophthalmoscope
Positions incorrectly (too far from the eye, wrong angle to the patient, uses the left eye to examine the patient’s
right eye or vice versa)
Has difficulty keeping light directed into the eye
Constantly shifts position
Unable to find the optic disc
Score 5 (excellent) Holds direct ophthalmoscope correctly
Properly distances from the patient
Initially positions 15 degrees to the patient
Remains stable (ophthalmoscope against the brow, hand on shoulder or forehead, and does not lose alignment)
Scans the macula correctly
Efficiency
Score 1 (poor) Is hesitant during the entire examination
Needs to “start over” many times
Does not finish the examination
Score 5 (excellent) Displays confidence
Proceeds methodically throughout the examination
Finishes the examination in the allotted time
For patient volunteersa , c
Score 1 (poor) Professionalism characterized by roughness and heavy-handling, poor communication, and use of an
uncomfortable level of light
Technique characterized by fumbling with the ophthalmoscope, inability to keep the light directed into the eye,
and constant shifting of position
Lack of confidence characterized by hesitancy and appearance of being unsure of examination procedures
Failure to complete the examination
Score 5 (excellent) Professionalism characterized by a gentle touch, demonstrated consideration of the patient’s comfort, and
awareness of the patient’s tolerance of light intensity
Technique characterized by being methodical, steady, and efficient
Confidence characterized by appearance of knowing correct examination procedures
Finished without being told to stop
For ophthalmologist gradersd
Location
Grade 0 No response or incorrect fundus region
Grade 1 A generally correct region of the fundus, but no detail regarding the relationship of lesions to fundus landmarks
Grade 2 Provides detailed information regarding correct lesion location
Clinical findings
Grade 0 Student could not visualize the fundus; or
Noted a “normal” fundus despite the presence of abnormalities; or
Listed incorrect findings
Grade 1 Answers described a lesion in general terms, but with no specific details (eg, “white spot” or “weird disc”)
Grade 2e A lesion was described accurately with greater detail than for grade 1 (eg, “white scar with surrounding dark
pigment” or “increased cup-to-disc ratio”); or
A nonspecific description was provided with correct diagnosis (eg, “lesion consistent with ocular
toxoplasmosis”)
a
Ophthalmologist observers and patient volunteers assigned scores using Likert scales with values from 1 through 5; for all scales, scores
were defined as follows, for the factor being evaluated: 1 = poor performance; 2 = below average performance; 3 = average performance
(neither good nor bad); 4 = above average performance; and 5 = excellent performance.
b
Guidelines were distributed to ophthalmologist observers only for scores 1 and 5. Scores could be assigned for any of the factors listed, a
combination of those factors, or similar factors that, in the opinion of the evaluator, warranted a given score.
c
A single score was based on the patient volunteer’s perception of a student’s performance in 3 areas: professionalism, technique, and
confidence.
d
Ophthalmologist graders assigned separate grades of 0, 1, or 2 to students’ written description of lesion location and to students’ written
description of the characteristics of the fundus lesions that they observed, guided by the listed factors.
e
A grade of 2 points was also assigned to students who correctly identified the normal fundus of patient volunteer 3.

VOL. 238 DIRECT OPHTHALMOSCOPY SIMULATION 191

 TABLE 2. Characteristics of 33 First-Year Medical Student Participants in a Randomized Controlled Trial Comparing Simulator-based
vs Traditional Training Techniques for Direct Ophthalmoscopy.

Characteristic Simulator Group (n = 17) Traditional Group (n = 16) P Valuea

Prior ophthalmoscopy experience, n (%) .485

No 17 (100) 15 (93.75)
Yes 0 1 (6.25)
Prior study of eye disease, n (%) .656
No 15 (88.24) 13 (81.25)
Yes 2 (11.76) 3 (18.75)
Pre-existing eye condition, n (%) .728
None 8 (47.06) 6 (37.50)
Routine use of glassesb 5 (29.41) 3 (18.75)
Routine use of contactsb 4 (23.53) 5 (31.25)
Otherc 0 2 (12.50)
a
Fisher exact test.
b
Students were asked to identify the presence of refractive errors and the correction they would wear during direct ophthalmoscopy assess-
ment.
c
One participant with strabismus since childhood, with best corrected visual acuity less than 20/20, both eyes; another participant alternated
between glasses and contact lenses.

TABLE 3. Summary Grades Assigned by Masked Ophthalmologist Graders to 33 First-Year Medical Student Participants in a
Randomized Controlled Trial of Training Techniques for Learning Direct Ophthalmoscopy, Based on Examination of 5 Patient
Volunteers.

Grade Simulator Group (n = 17) Traditional Group (n = 16) P Valuea Adjusted P Valueb

Locationc .768 .692

Mean ± SD 2.5 ± 1.7 2.2 ± 1.2
Median (range) 2.0 (0.0-6.0) 2.0 (0.0-4.0)
IQR 1.0, 3.0 2.0, 3.0
Clinical findingsc .386 .407
Mean ± SD 2.9 ± 1.7 2.4 ± 1.3
Median (range) 3.0 (0.0-5.0) 2.0 (0.0-5.0)
IQR 2.0, 4.0 2.0, 3.0
Totald .513 .524
Mean ± SD 5.4 ± 3.3 4.6 ± 2.4
Median (range) 5.0 (0.0-11.0) 4.0 (0.0-9.0)
IQR 3.0, 7.0 4.0, 6.0

IQR = interquartile range, SD = standard deviation.

a
Kruskal-Wallis test.
b
Multivariable linear regression models, adjusting for practice time (≥1 hour vs <1 hour).
c
A grade of 0, 1, or 2 was assigned to each student for each patient volunteer examined (maximum grade = 10 points).
d
A combination of Location and Clinical Findings grades (maximum grade = 20 points).

Table 5 shows ophthalmologist-observer and patient-
volunteer scores assigned to students. Observers scores
assigned to simulator-trained students were consistently
higher than scores assigned to traditionally trained students
for technique (P = .011), efficiency (P = .007), and global
performance (P = .004). Technique (P = .034) and global
performance (P = .025) scores remained statistically signif-
icant after adjustment for practice time. Patients assigned
higher scores to students in the simulator group than to
those in the traditional group, although the difference was
not statistically significant (P = .178). Scores did not im-
prove in either group as students progressed from patient to
patient (data not included).
Table 6 summarizes student responses in the postassess-
ment survey. The simulator group reported longer indepen-
dent practice time than the traditional group (P = .002).
With respect to students’ perceptions of their assigned
training method vs their understanding of the alternative
technique, the number of students within each group who
felt that their method was strictly advantageous was simi-

192 AMERICAN JOURNAL OF OPHTHALMOLOGY JUNE 2022

 TABLE 4. Proportions of 33 First-Year Medical Student Participants Assigned Each Grade Level During Examination of a Patient With
a Macular Disciform Scar in a Randomized Controlled Trial of Training Techniques for Learning Direct Ophthalmoscopy, With
Comparison Between Training Groups.

Gradea Simulator Group (n = 17) Traditional Group (n = 16) P Valueb Adjusted P Valuec

Location, n (%) .013 .027

0 5 (29.41) 13 (81.25)
1 7 (41.18) 2 (12.50)
2 5 (29.41) 1 (6.25)
Clinical findings, n (%) .018 .044
0 5 (29.41) 12 (75.00)
1 4 (23.53) 3 (18.75)
2 8 (47.06) 1 (6.25)
a
A grade of 0, 1, or 2 was assigned to each student for location and for clinical findings.
b
Fisher exact test.
c
Multivariable logistic regression models, adjusting for practice time (≥1 hour vs <1 hour).

TABLE 5. Summary Scores Assigned by Masked Ophthalmologist Observers to 33 First-Year Medical Student Participants, Based
on Examination of 5 Patient Volunteers, and a Summary Score Assigned to the Students by Those Patient Volunteers in a
Randomized Controlled Trial Comparing Simulator-based vs Traditional Training of Direct Ophthalmoscopy.

Scoresa Simulator Group (n = 16)b Traditional Group (n = 16) P Valuec Adjusted P Valued

Ophthalmologist-observer scores
Technique .011 .034
Mean ± SD 19.1 ± 2.0 16.4 ± 2.8
Median (range) 19.0 (16.0-23.0) 17.0 (11.0-21.0)
IQR 17.5, 21.0 14.0, 18.0
Efficiency .007 .066
Mean ± SD 19.8 ± 2.1 17.1 ± 3.0
Median (range) 20.0 (16.0-22.0) 17.5 (12.5-23.0)
IQR 18.5, 22.0 15.0, 18.5
Global performance .004 .025
Mean ± SD 19.8 ± 1.8 17.2 ± 2.4
Median (range) 20.0 (17.0-23.0) 18.0 (12.5-22.0)
IQR 18.3, 21.0 15.8, 18.8
Patient-volunteer score
Global performance .178 .207
Mean ± SD 21.7 ± 1.3 20.9 ± 1.8
Median (range) 22.0 (19.0-24.0) 21.0 (18.0-24.0)
IQR 21.0, 23.0 19.0, 22.0

IQR = interquartile range, SD = standard deviation.

a
Each score based on a 5-point Likert scale. Maximum possible points assigned to each student for each score was 25 (5 points for each of
5 patient volunteers examined).
b
One participant did not receive a score from an ophthalmologist observer and was not included in analysis.
c
Kruskal-Wallis test.
d
Multivariable linear regression models, adjusting for practice time (≥1 hour vs <1 hour).

lar to the number who felt it was strictly disadvantageous;
however, substantially more students in the simulator group
specifically felt that there were both advantages and dis-
advantages associated with simulator training (9 students
[52.94%] vs 2 students [12.5%] in the traditional group,
P = .026, Fisher exact test).
On the basis of open-field responses, major themes
among stated advantages of simulator training by students
in the simulator group were the ability to visualize simu-
lated fundus lesions (10 [76.9%] of 13 respondents) and
ability to practice without bothering a patient or classmate
(4 respondents [30.8%]). Students in the traditional group

VOL. 238 DIRECT OPHTHALMOSCOPY SIMULATION 193

 TABLE 6. Results of a Postassessment Survey of 33 First-Year Medical Student Participants in a Randomized Controlled Trial
Comparing Simulator-based vs Traditional Training of Direct Ophthalmoscopy.

Survey Responses Simulator Group (n = 17) Traditional Group (n = 16) P Valuea

Self-reported estimate of independent practice timeb , n (%) .002

None 1 (5.88) 1 (6.25)
<30 min 0 3 (18.75)
30-59 min 2 (11.76) 8 (50.00)
1-3 h 14 (82.35) 4 (25.00)
Perception of assigned training techniquec , n (%) .128
Advantageous 4 (23.53) 6 (37.50)
Disadvantageous 4 (23.53) 7 (43.75)
Neither an advantage nor a disadvantage 0 1 (6.25)
Both advantages and disadvantages 9 (52.94) 2 (12.50)
Confidence in ability to perform direct ophthalmoscopy, n (%) .516
Not at all comfortable 1 (5.88) 2 (12.50)
Somewhat uncomfortable 5 (29.41) 8 (50.00)
Uncertain 5 (29.41) 2 (12.50)
Somewhat comfortable 6 (35.29) 4 (25.00)
Very comfortable 0 0
a
Fisher exact test.
b
No student reported practice time more than 3 hours.
c
The survey included an open-field option to provide further explanation of perceptions.

viewed lack of these factors as disadvantages while practic-
ing on students with normal fundi or patients who might be
bothered by extended examinations. Major themes among
stated disadvantages of simulator training by students in the
simulator group were the unrealistic training environment
(5 [38.4%] of 13 respondents), lack of experience in posi-
tioning relative to, and interacting with, a live person (3
respondents [23.1%]), and unfamiliarity with actual direct
ophthalmoscope controls before examination of patients
(8 respondents [61.5%]). Conversely, students in the tradi-
tional group considered working with live individuals, using
a real ophthalmoscope, to be an advantage. Lack of real-
time feedback was viewed as a disadvantage by 1 student in
the traditional group.
With regard to comfort in performing direct ophthal-
moscopy during assessments, no student in either train-
ing group felt “very comfortable.” When training groups
were compared, more students in the simulator group felt
“somewhat comfortable,” whereas more students in the tra-
ditional group felt “somewhat uncomfortable” or “not at all
comfortable,” although the differences were not statistically
significant.
DISCUSSION
Direct ophthalmoscopy is a difficult skill for many medical
students to learn. This study sought to determine the effect
of a direct ophthalmoscope simulator on the early stages
of learning ophthalmoscopy. We identified better perfor-
mance among students in the simulator-trained group on
3 measures: independent practice time, technique (ability
to handle a direct ophthalmoscope and conduct an exami-
nation), and, based on the results of 1 patient examination,
the ability to locate fundus lesions accurately.
Longer practice time among students in the simulator
group is likely attributable to the novelty of the device and
the fact that students did not have to depend on others to
practice. The exercises are engaging and the simulator pro-
vides immediate feedback. One can hypothesize that longer
practice time will ultimately result in better skills, but prac-
tice time alone did not explain better technique scores or
ability to localize lesions by the simulator group.
With regard to technique, handling of the direct oph-
thalmoscope was taught to both groups in the didactic ses-
sion, but proper technique is reinforced by the simulator; for
example, correct positioning of the hand piece is required
to view the simulated fundus. By mapping that portion of
the fundus viewed by the operator, the simulator reinforced
didactic instructions about how to scan the posterior pole
efficiently, which might explain why the simulator group
was better able to identify the location of the lesion in 1
patient’s eye.
On a computerized literature search using PubMed, we
found only 1 previous study that investigated the effect
of training on the Eyesi Direct Ophthalmoscope simula-
tor. Boden and associates12 randomized 34 German medical
students to “classic” and “simulator” training groups dur-
ing an ophthalmology rotation. All students received the

194 AMERICAN JOURNAL OF OPHTHALMOLOGY JUNE 2022

same 5-minute introduction, after which the classic group
practiced on dilated students and looked at pictures of fun-
dus lesions, whereas the simulator group looked at repre-
sentations of fundus lesions in the simulator. After the 45-
minute training session, a course instructor assessed each
student using a 23-point scale based on a combination of
examination milestones (ability to see various fundus land-
marks) and examination procedures not specifically related
to the ophthalmoscope itself. During the assessment, stu-
dents used either a direct ophthalmoscope or the simulator
according to each student’s randomization group. Examina-
tion milestones were self-reported by students in the clas-
sic group and reported objectively by the simulator for stu-
dents who used it. On the basis of the assessment scale, the
reported “learning success” for the simulator group (91%)
was higher than for the classic group (78%, statistical com-
parison not reported). A student survey about the learning
experience yielded favorable responses, with no statistical
differences between the 2 groups. Students in the simulator
group also responded positively to 3 additional questions
about the simulator.
In our study, the overall ability of students to perform di-
rect ophthalmoscopy on patients was poor in both groups,
with the majority in both groups reporting low confidence
in ophthalmoscopy skills. Lack of confidence during oph-
thalmoscopy has been described previously at all levels of
medical training.2-4 Our purpose was not to achieve high
levels of proficiency in direct ophthalmoscopy among stu-
dents, but to identify any influence of simulator training
during the early phase of learning. Students in the tradi-
tional group most likely practiced on people whose pupils
were not dilated, whereas students in the simulator group
had the option of looking through simulations of either di-
lated or undilated pupils. Although looking through undi-
lated pupils requires examiners to position carefully, we be-
lieve that, during early training, it is easier for students to
develop a proper direct ophthalmoscope technique and pro-
cedures (such as scanning of the macula) while examining
eyes with dilated pupils. This distinction would favor prac-
tice on the simulator.
There are many advantages of using simulation in med-
ical training in general.18 Simulation allows the learner to
develop proper motor skills before performing those skills
on patients. Simulation facilitates learning through repeti-
tion with real-time feedback and objective tracking of per-
formance. Independent practice, spaced out over time, re-
sults in more permanent memory encoding of motor skills;
however, operators receive no feedback from human exam-
inees, and simulation does not fully duplicate the position-
ing of clinicians in relation to patients’ bodies. With regard
to student perceptions, those in the simulator group recog-
nized both advantages and disadvantages to simulator train-
VOL. 238 DIRECT OPHTHALMOSCOPY SIMULATION
ing. The most commonly reported disadvantage was lack of
exposure to a real ophthalmoscope before use on a patient.
Thus, simulation is likely most appropriate as a complement
to traditional training, not as a replacement. We propose
that learning is best achieved by a combination of didac-
tic instruction, initial supervised practice, both on people
and a simulator, followed by independent practice and re-
finement of skills with the simulator. Initial supervision can
reinforce techniques and clinician-patient interactions that
are not specifically addressed by the simulator.
Strengths of our study include the use of masked assess-
ments and the fact that all students underwent the same as-
sessment. Evaluation of performance on real patients helps
to determine the transferability of skills from the simulator
to people. Limitations of our study include use of a nonvali-
dated scoring system and lack of a reference standard against
which medical student skills could be compared (eg, perfor-
mance of ophthalmology residents on the same assessment).
The use of open-field questions required grader interpreta-
tion of student responses, which might limit the accuracy
with which skills were assessed; we addressed this limitation
by comparing grades assigned by 2 graders and adjudicat-
ing discrepancies. Despite this potential problem, we chose
not to use multiple-choice answer sheets to reduce the risk
of guessing by students. The power of the study to detect
differences was limited by the relatively small number of
student and patient volunteers for the trial. It is unlikely,
however, that patients would have tolerated a substantially
larger number of examinations. We did not perform a sta-
tistical correction for numerous comparisons during anal-
yses, as we considered our study to be exploratory, look-
ing for various ways that simulation influences the learn-
ing of ophthalmoscopy. Although we may have identified
false relationships because of the many questions asked, a
correction for numerous comparisons might also have pre-
vented us from identifying factors that should be investi-
gated in larger, future studies. We sensed that many students
were overwhelmed by the assessment experience, which
may have led to poor performance. The long-term effect
simulation-based training on skills is unknown, but will be
the subject of a future study.
In summary, simulation appears to enhance early teach-
ing of direct ophthalmoscopy. Students who practiced on
the simulator demonstrated a better technique using a di-
rect ophthalmoscope during patient examinations. Simula-
tion aims to train operators to scan the fundus completely
and locate lesions accurately. The simulator also appeared
to motivate students to practice longer, but duration of
practice did not account for other training benefits in the
timeframe of this study. We propose that simulation will
likely be most useful as a complement to traditional, super-
vised training methods.
195
Acknowledgments: Marshall Dial, North American representative of VRmagic GmbH, helped to instruct medical students who had been randomized to
the simulator group about how to operate the simulator; he had no further role in conduction of the randomized controlled trial or analysis and reporting of
data. The following ophthalmologists from the UCLA Department of Ophthalmology served as observers during student examinations of patients: JoAnn
A. Giaconi, MD, Lynn K. Gordon, MD, PhD, Ralph D. Levinson, MD, Tania Onclinx, MD, and Sandip Suresh, MD.
All authors have completed and submitted the ICMJE form for disclosure of potential conflicts of interest.
Funding/Support: David Geffen School of Medicine at UCLA Office of the Vice Dean for Education (C.H.B.); unrestricted funds from the UCLA Stein
Eye Institute (B.R.S.); the Skirball Foundation (New York, NY) (G.N.H.); and Research to Prevent Blindness, Inc (New York, NY) through an unrestricted
grant to the UCLA Stein Eye Institute for research. Funding organizations had no role in the design or conduction of the study.
Financial Disclosures: C.A.M. is an unpaid consultant to VRmagic GmbH (Mannheim, Germany), manufacturer of the Eyesi Direct Ophthalmoscope
Simulator; VRmagic GmbH had no role in the design or conduct of this research. The other authors have no conflict of interest to disclose. All authors
attest that they meet the current ICMJE criteria for authorship.

REFERENCES 10. Ricci LH, Ferraz CA. Ophthalmoscopy simulation: advances

1. Graubart EB, Waxman EL, Forster SH, et al. Ophthal-
mology objectives for medical students: revisiting what ev-
ery graduating medical student should know. Ophthalmology.
2018;125:1842–1843.
2. Gupta RR, Lam WC. Medical students’ self-confidence in
performing direct ophthalmoscopy in clinical training. Can J
Ophthalmol. 2006;41:169–174.
3. Mottow-Lippa L. Ophthalmology in the medical school cur-
riculum: reestablishing our value and effecting change. Oph-
thalmology. 2009;116:1235–1236 1236.e1.
4. Wu EH, Fagan MJ, Reinert SE, Diaz JA. Self-confidence in
and perceived utility of the physical examination: a compari-
son of medical students, residents, and faculty internists. J Gen
Intern Med. 2007;22:1725–1730.
5. Shuttleworth GN, Marsh GW. How effective is undergradu-
ate and postgraduate teaching in ophthalmology? Eye (Lond).
1997;11(Pt 5):744–750.
6. Stern GA. Teaching ophthalmology to primary care
physicians. The Association of University Professors of
Ophthalmology Education Committee. Arch Ophthalmol.
1995;113:722–724.
7. Mackay DD, Garza PS, Bruce BB, Newman NJ, Biousse V.
The demise of direct ophthalmoscopy: a modern clinical chal-
lenge. Neurol Clin Pract. 2015;5:150–157.
8. Kelly LP, Garza PS, Bruce BB, Graubart EB, Newman NJ,
Biousse V. Teaching ophthalmoscopy to medical students
(the TOTeMS study). Am J Ophthalmol. 2013;156:1056–1061
e1010.
9. Akaishi Y, Otaki J, Takahashi O, et al. Validity of direct oph-
thalmoscopy skill evaluation with ocular fundus examination
simulators. Can J Ophthalmol. 2014;49:377–381.
in training and practice for medical students and young oph-
thalmologists. Adv Med Educ Pract. 2017;8:435–439.
11. de Souza PHL, Nunes GMN, de Alencar Bastos JMG, Fon-
seca TM, Cortizo V. A new model for teaching opthalmoscopy
to medical students. MedEdPublish. 2017;28. doi:10.15694/
mep.2017.000197.
12. Boden KT, Rickmann A, Fries FN, et al. [Evaluation of a
virtual reality simulator for learning direct ophthalmoscopy
in student teaching]. Ophthalmologe. 2020;117:44–49 [in Ger-
man].
13. Ferris JD, Donachie PH, Johnston RL, Barnes B, Olaitan M,
Sparrow JM. Royal College of Ophthalmologists’ National
Ophthalmology Database study of cataract surgery: report 6.
The impact of EyeSi virtual reality training on complications
rates of cataract surgery performed by first and second year
trainees. Br J Ophthalmol. 2020;104:324–329.
14. Jacobsen MF, Konge L, Bach-Holm D, et al. Correlation of
virtual reality performance with real-life cataract surgery per-
formance. J Cataract Refract Surg. 2019;45:1246–1251.
15. McCannel CA, Reed DC, Goldman DR. Ophthalmic
surgery simulator training improves resident performance
of capsulorhexis in the operating room. Ophthalmology.
2013;120:2456–2461.
16. Borgersen NJ, Skou Thomsen AS, Konge L, Sørensen TL,
Subhi Y. Virtual reality-based proficiency test in direct oph-
thalmoscopy. Acta Ophthalmol. 2018;96:e259–e261.
17. Cheng A, Kessler D, Mackinnon R, et al. Reporting guidelines
for health care simulation research: extensions to the consort
and strobe statements. Simul Healthc. 2016;11:238–248.
18. Issenberg SB, Mcgaghie WC, Petrusa ER, Gordon DL,
Scalese RJ. Features and uses of high-fidelity medical simu-
lations that lead to effective learning: a BEME systematic re-
view. Medical Teacher. 2005;27:10–28.

196 AMERICAN JOURNAL OF OPHTHALMOLOGY JUNE 2022