ORIGINAL INVESTIGATION

A Novel Automated Visual Acuity Test Using a Portable

Head-mounted Display
Sze Chuan Ong, MBBS,1 Li Cheng (Ivy) Pek, BSc,2 Tsuey Ling (Carol) Chiang, BSc,2 Hock Wei Soon, MEng,3 Kuang Chua Chua, PhD,3
Chanakarn Sassmann, BME,3 Muhammad Azri Bin Razali, BEE,3 and Teck Chang (Victor) Koh, MRCSEd1,2*

SIGNIFICANCE: We developed a head-mounted display (HMD) as an automated way of testing visual acuity (VA) to
increase workplace efficiency. This study raises its potential utility and advantages, analyzes reasons for its current
limitations, and discusses areas of improvement in the development of this device.
PURPOSE: Manual VA testing is important but labor-intensive in ophthalmology and optometry clinics. The pur-
pose of this exploratory study is to assess the performance and identify potential limitations of an automated
Downloaded from http://journals.lww.com/optvissci by BhDMf5ePHKbH4TTImqenVOm0oSGVjfT8jCxH6smnuq6G3+01QLgP+F0KFmSDeUmP on 09/08/2020

HMD for VA testing.

METHODS: Sixty patients from National University Hospital, Singapore, were enrolled in a prospective observa-
tional study. The HMD was constructed based on the Snellen chart, with single optotypes displayed at a time. Each
subject underwent VA testing of both eyes with the manual Snellen chart tested at 6 m from the subject and the
HMD.
RESULTS: Fifty-three subjects were included in the final analysis, with an incompletion rate of 11.7% (n = 7). The
mean difference in estimated acuity between the HMD and Snellen chart was 0.05 logMAR. However, 95% limits Author Affiliations:
of agreement were large at ±0.33 logMAR. The HMD overestimated vision in patients with poorer visual acuities. In 1
Yong Loo Lin School of Medicine,
detecting VA worse than 0.30 logMAR (6/12), sensitivity was 63.6% (95% confidence interval, 0.31 to 0.89%), National University of Singapore,
and specificity was 81.0% (95% confidence interval, 0.66 to 0.91%). No significant correlation existed between Singapore, Singapore
2
mean difference and age (r = −0.15, P = .27) or education level (r = 0.04, P = .76). Department of Ophthalmology,
National University Hospital, Singapore,
CONCLUSIONS: Advantages of our novel HMD technology include its fully automated nature and its portability. Singapore
However, the device in its current form is not ready for widespread clinical use primarily because of its low accu- 3
School of Engineering, Ngee Ann
racy, which is limited by both technical and user factors. Future studies are needed to improve its accuracy and Polytechnic, Singapore, Singapore
completion rate and to evaluate for test-retest reliability in a larger population. *victor_koh@nuhs.edu.sg

Optom Vis Sci 2020;97:591–597. doi:10.1097/OPX.0000000000001551

Copyright © 2020 American Academy of Optometry

Visual acuity testing is an essential clinical investigation for the
screening and management of visual impairment.1 Distance visual
acuity assessment is commonly performed with the Snellen visual
acuity chart, a standardized test at the outpatient ophthalmology
clinics. It requires an individual to identify what is displayed on a
screen or printed chart from the biggest to the smallest optotype
size. The smallest optotype size read correctly is defined as one's
smallest perceivable visual angle. However, owing to the time and
manpower requirement to manually test the visual acuity in
high-volume ophthalmology clinics, there is often a “bottleneck”
of patients at the visual acuity test. This results in longer patient
waiting times and a strain on manpower resource.
With recent advances in technology, there are increased opportuni-
ties for the development of an automated method to assess visual acu-
ity, which might increase workplace efficiency. Several studies have
demonstrated the possibility of automated techniques as reliable al-
ternative measurement methods when applied to patients with a
specific age group or eye condition. This includes evaluation of com-
puterized logMAR measurement system on patients with AMD or a
self-developed computerized test on the pediatric population.2–4
We developed a novel automated visual acuity testing system,
with a software algorithm that uses optotypes of varying sizes
projected into a head-mounted display. We aimed to assess its
performance as an alternative method to test visual acuity and to
identify limitations of the current prototype.
METHODS
This prospective observational case study was reviewed and ap-
proved by the National Health Group Domain Specific Review
Board (reference number 2017/00736). It was conducted in ac-
cordance with the ethical principles that have their origin in the
Declaration of Helsinki and that are consistent with the Singapore
Good Clinical Practice and the applicable regulatory requirements.
Informed consent was obtained from every subject before his/her
participation in the study.
Automated Visual Test System
The automated visual acuity test consists of a pair of augmented
reality smart glasses with an eye shield as the head-mounted dis-
play, a computerized visual acuity measurement program, and a
tablet. The headset used in the study was the Epson Moverio
BT350 smart glasses (Seiko Epson Corporation, Suwa, Nagano,
Japan; Fig. 1), which has a 1280  720-pixel resolution per eye,
a 0.43-inch-wide panel, a virtual screen size of 40 inches, and a

www.optvissci.com Optom Vis Sci 2020; Vol 97(8) 591

Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.

Automated VA Testing Using Head-mounted Display — Ong et al.
FIGURE 1. A visualization of an optotype displayed on the Epson
Moverio BT350 smart glasses when testing for the left eye, with an
eye shield occluding the right eye.
screen distance of 2.5 m. The screen size corresponded to a visual
angle of 23.0°. The screen resolution corresponded to 31.6 cycles
per degree. It can be worn over a pair of existing refractive glasses.
Fig. 1 shows the projection of an optotype within the Epson smart
glasses. The optotype is displayed on a white opaque background,
which completely blocks the environment that is directly behind it.
However, users will still able to see the environment surrounding this
background (as it does not occupy the entire field of view of the
headset). The tablet used in the study for subjects to key in their re-
sponse was the Lenovo Tab 2 A7-10 (Lenovo Group Ltd, Beijing,
China). The display resolution was 1024  600 pixels over a 7.0-inch
diagonal screen. Fig. 2 shows the response interface of the tablet
for subjects to input their answers after viewing the optotypes.
We developed a computerized distance visual acuity measure-
ment program via the use of Android Studio (School of Engineering,
Ngee Ann Polytechnic, Singapore). The program runs in a similar
way as does the Snellen chart. Single optotypes in the form of let-
ters were displayed sequentially, and the intervals of visual acuities
tested were based on that tested on every line of the Snellen chart
(starting from 6/120 to 6/60, 6/48, 6/38, 6/30, 6/24, 6/18, 6/15,
6/12, 6/9, 6/7.5, and finally 6/6). Letters with similar relative leg-
ibility were selected as optotypes, as part of the British Standards
Institution guidelines for visual acuity chart design (BSI 4274-1
2003).5–7
Snellen designed the optotype on a 5  5 grid on which each
box is 1 unit and the letter width and height are 5 units each. As
such, a “normal vision” is defined as the ability of one to recognize
an optotype that subtends 5 arc minute and discriminate a single
stroke of the optotype of 1 arc minute. A Snellen distant chart
should be read while standing 6 m away from it. The visual acuity
is presented as fraction, with the standardized distance between
the chart and the observer as the numerator (if this is 6 m, then the
numerator is “6”). The denominator is the distance at which the
smallest optotype still subtends an angle of 5 arc minute for a normal
person. If the smallest optotype size a person can read is twice as large
as the reference standard of 6/6, the visual acuity is denoted as 6/12.
This means that, for someone with 6/6 vision on the Snellen chart, the
6/6 optotype would subtend 5arc minute over 6m. To this same
person, the 6/60 optotype can be read at 60 m because the same
optotype would also subtend 5 arc minute. To create an optotype
testing for 6/6 visual acuity at 2.5 m (our virtual screen distance),
the following formula was used: tangent (5/60) = height of optotype
(in meters)/distance from optotype to eye (in meters). A 6/6 Snellen
www.optvissci.com Optom Vis Sci 2020; Vol 97(8)
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
optotype will thus be 3.636 mm in height at 2.5 m from the observer.
The subsequent optotypes were determined using a linear scale. For
example, the height of a 6/12 optotype (7.272 mm) is twice the
height of a 6/6 optotype (3.636 mm). This was then converted into
pixels and displayed on the screen used for the current study.
The algorithm comprises three phases. Phase 1 was created
with the intention of reducing the time needed to complete the
test. The program begins by displaying the biggest optotype testing
for a visual acuity of 6/120. With every correct input by the subject,
the program sequentially displays a smaller optotype correspond-
ing to the next best visual acuity being tested. Once it narrows down
the range of the subject's visual acuity, phase 2 is prompted.
In phase 2, the program displays five sequential optotypes of
the same sizes corresponding to the visual acuity it is testing for.
If all inputs by the subject for each optotype are correct, the pro-
gram continues on to test the next best visual acuity. If the subject
fails to achieve three or more correct responses out of five (<50%
correct response), the program will end with the corresponding vi-
sual acuity score. If only one or two inputs are correct, the program
advances to phase 3.
Phase 3 was designed to increase the accuracy of the program,
preventing false negatives in the event of accidentally pressing the
wrong response tab. This phase double-checks the score by testing
the next best visual acuity. If the subject subsequently responds
correctly for all the optotypes displayed, it likely means that the
wrong inputs in the previous best visual acuity tested in phase 2
were accidental responses (false negatives), and the program
returns and continues on with phase 2. Otherwise, the program cal-
culates the subject's best visual acuity score based on the com-
bined results of phases 2 and 3.
In total, 60 adult subjects were recruited from the Eye Surgery
Centre, National University Hospital, Singapore. Subjects were in-
cluded if they were 21 years or older, had a visual acuity of at least
6/60 or better in both eyes determined on a standard Snellen chart,
and were able to provide written informed consent. Each subject
underwent distance visual acuity testing using the Snellen visual
acuity chart administered by a trained medical professional in a
standardized manner at a distance of 6 m. Subsequently, the auto-
mated visual testing was conducted by a nonmedical professional
FIGURE 2. Patient interface on the Lenovo Tab 2 A7-10 for subjects to
key in their responses to the optotype displayed on the HMD. Nine
large tabs were available as response options for the subject, eight of
which contained a single letter and one of which was a skip button if the
subject was unable to read the optotype. HMD = head-mounted display.
592
 Automated VA Testing Using Head-mounted Display — Ong et al.

(HWS, KCC, or CS). The role of the nonmedical professional was TABLE 1. Demographics of included participants in the study
to start the device or to troubleshoot any technical difficulties.
Once the program started, the test was fully automated and Descriptor Total sample (n = 53) Proportion of sample (%)
self-administered by the subject, with instructions provided on Age (y)
the headset display or tablet for the subject at each step of the test. 31–40 2 3.7
Participants wore the head-mounted display over their eyes or re-
fractive glasses (if any), with the eye shield used to occlude the fel- 41–50 8 15.1
low eye during monocular automated visual acuity testing. To 51–60 18 34.0
account for presbyopia, the subject was instructed to hold the tab- 61–70 18 34.0
let at a distance where the response tabs displayed appeared clear
71–80 7 13.2
to them. Both tests were administered for both eyes, first on the
right eye and then the left. Only visual acuities without pinhole Race
for the right eye were included in the analysis. The time taken to Chinese 38 71.7
measure visual acuities in both eyes without pinhole occlusion Malay 5 9.4
was measured in minutes for each method. At the end of the test,
Indian 7 13.2
each subject completed a written questionnaire. It consists of their
age, highest education level obtained, and an open-ended section Others 3 5.7
for feedback regarding the ease of use and comfort of the device. Sex
Male 22 41.5
Statistical Analysis
All visual acuities were converted to logMAR for analysis. The Female 31 58.5
converted analysis ranged from 0.00 (equivalent to 6/6 on the Highest education level
Snellen chart) to 1.00 logMAR (equivalent to 6/60), and t test anal- Primary education 14 26.3
ysis was used to evaluate for the presence of systematic bias.
Secondary education 18 34.0
Agreement was determined by 95% limits of agreement repre-
sented on a Bland-Altman plot. The limits of agreement were con- Tertiary education 8 15.1
sidered as a pair when calculating its confidence intervals.8 All Bachelor's degree 2 3.8
statistical analyses were conducted with SPSS for Windows (ver- Master’s degree 8 15.1
sion 25.0; SPSS, Inc., Chicago, IL).
PhD degree 1 1.9
Others 2 3.8

RESULTS
Subjects
Of the 60 subjects recruited, 53 subjects from the age of 31 to
79 years were included in the final statistical analysis. The remain-
ing seven subjects were excluded for the following reasons that
made their visual acuity results unreliable or incomplete. One sub-
ject did not look at the tablet when responding; three subjects did
not wear the smart glasses properly; one subject admitted to inten-
tionally keying in wrong responses as an attempt to test the validity
of the system; one other subject reported to have severe dry eyes,
which affected his ability to focus on the optotypes; and lastly,
one subject claimed to be distracted by the background.
The mean age of the included subjects was 59.4 ± 20.0 years.
The remaining demographics of the subjects are shown in Table 1.
Visual Acuity Testing Comparison
The mean distance visual acuity without pinhole occlusion mea-
sured for the right eye with a Snellen chart was 0.18 ± 0.42
logMAR. The mean distance visual acuity of the right eyes mea-
sured by the head-mounted display was 0.22 ± 0.36 logMAR. Table 2
shows the agreement between manual testing using the Snellen chart
and automated testing using the head-mounted display. The mean
difference was 0.05 logMAR, with the head-mounted display
underestimating the visual acuity measured by the Snellen chart.
Ninety-five percent limits of agreement were ±0.33 logMAR; the
respective values (with their confidence intervals) were −0.28
(−0.23 to −0.37) and 0.38 (0.33 to 0.49). The results are repre-
sented in the Bland-Altman plot in Fig. 3. Pearson correlations
between the user's age or education level and mean differences
were −0.15 (P = .27) and 0.04 (P = .76).
In detecting visual acuities of 6/12 Snellen (0.30 logMAR) or
worse, the head-mounted display had a specificity of 81.0%
(95% confidence interval, 0.66 to 0.91%) and a sensitivity of
63.6% (95% confidence interval, 0.31 to 0.89%), with a positive
predictive value of 46.7% (95% confidence interval, 0.29 to
0.65%) and a negative predictive value of 90.0% (95% confi-
dence interval, 0.79 to 0.95%). The area under receiver operating
characteristic curve was 0.87 (95% confidence interval, 0.77 to
0.98%; Fig. 4).
The mean time taken for the manual and automated test
methods was 1.64 ± 1.29 (95% confidence interval, 1.46 to
1.82 minutes) and 3.18 ± 3.44 minutes (95% confidence interval,
2.70 to 3.67 minutes), respectively.
There was no verbal or written feedback provided by the partic-
ipants regarding difficulties of wearing the headset over existing re-
fractive glasses.
DISCUSSION
Visual acuity measurement is a common and important test
conducted at outpatient ophthalmology clinics, primary health
care clinics, health screenings, and optometry centers. Poor visual
acuities could reflect an intraocular pathology, which have substan-
tial health burden on patients.1,9 It also helps to guide management

www.optvissci.com Optom Vis Sci 2020; Vol 97(8) 593

Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.

Automated VA Testing Using Head-mounted Display — Ong et al.

TABLE 2. Comparison of agreement between the HMD and the Snellen chart
Standard deviation 95% Limits of Maximum Maximum
Mean difference for differences agreement overestimation* underestimation†
HMD compared with Snellen chart +0.05 0.17 −0.28 to 0.38 0.62 0.35
(logMAR)
*0.62 = maximum overestimation of visual acuity by HMD when compared with Snellen chart. †0.32 = maximum underestimation of visual acuity by
HMD when compared with Snellen chart. HMD = head-mounted display.

for various diseases and is often a surrogate measure for the success
of a treatment.10–12
There are several studies on similar automated visual acuity test
systems, most commonly the Freiburg Visual Acuity test, comput-
erized logMAR visual acuity measurement system, tablet applica-
tions, and smartphone applications.2,13–16
The Freiburg Visual Acuity test is a widely validated automated
visual acuity test using a computer monitor for optotype display. It
has a high level of agreement when compared with conventional vi-
sual acuity charts (to within 1%).13 Although it is highly accurate,
its use on a standard computer monitor is limited by a minimum
screen viewing distance of 3 m, consuming a large operating
space. The need for a screen monitor limits its portability as a visual
acuity test.
The computerized logMAR test is a computerized visual acuity
measurement system that requires a laptop PC capable of running
Microsoft Windows XP, a secondary monitor for the display of
optotypes, and a software program running within the Microsoft
dotnet framework. The computerized logMAR test algorithm com-
prises two phases, both of which require an examiner's judgment
of whether the responses keyed in by the patient are correct or in-
correct.14 This predisposes to interrater variations and bias be-
tween observers. The requirement of an examiner to control the
progression of the test also makes this process of visual acuity as-
sessment semiautomated, requiring trained manpower.
Smartphone and tablet applications are widely available com-
mercially as an alternative method of measuring visual acuity, vi-
sual fields, and color vision. Some of these applications display
the optotype and response keys simultaneously on the device
screen, allowing for a fully automated assessment by the patient.
However, the accuracy of these electronic devices is largely limited
by visual glare. Black et al.15 showed that the presence of glare
could lead to a variation in visual acuities measured by the “Visual
Acuity XL” iOS application by 0.18 logMAR (nine letters). For our
system, glare was not an issue because the headset did not involve
a screen, but rather, subjects looked through a clear augmented reality
glass panel to visualize the optotype. No subjects provided verbal or
written feedback regarding the presence of glare as well. Pathipati
et al.,16 when evaluating the use of “Paxos Checkup” application, also
noted that the presence of presbyopia in the user would affect the accu-
racy and variability of the visual acuity results. Some of the smartphone
and tablet applications also require the presence of an external exam-
iner, as well as a substantial distance between the user and the iPad
during the test procedure. Gounder et al.17 demonstrated that the
“EyeSnellen” iOS application necessitated a distance of 6 m between
the user and the iPad and an external examiner to validate the results.
Although we did consider the use of virtual reality technology,
there were significant drawbacks associated with it. A previous pre-
liminary study we conducted using virtual reality and a smartphone
to test for visual acuity revealed technical challenges. These

FIGURE 3. Bland-Altman plot comparing the visual acuities measured by the HMD and the Snellen chart. In patients with poorer visual acuities, the
HMD overestimated their visual acuities compared with the Snellen chart. HMD = head-mounted display.

www.optvissci.com Optom Vis Sci 2020; Vol 97(8) 594

Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.


FIGURE 4. Receiver operating characteristics curve for the HMD in de-
tecting visual acuity worse than 6/12 Snellen.
limitations were mainly the presence of a screen-door effect (a vi-
sual artifact where subpixels became visible in displayed images),
low resolutions of the smartphone, and loss of real-world percep-
tion. The aforementioned limitations directly affect the ability of
the user to read the smaller optotypes displayed, hence reducing
the program's accuracy of estimating the user's visual acuity. Such
limitations were not observed with the augmented reality smart
glasses we chose as the head-mounted display. Although we ac-
knowledge that professional virtual reality headsets such as “Vive”
(HTC Corporation, New Taipei, Taiwan and Valve Corporation, Belle-
vue, WA) do have comparable resolutions to the head-mounted dis-
play, most of such headsets ran on an internal operating system,
which did not allow us to develop and run our algorithm and applica-
tion. Also, such virtual reality headsets are fully immersive, preventing
visualization of the external environment. Subjects are unable to input
their responses into the tablet without removing these headsets. Con-
versely, the head-mounted display headset allows for visualization of
the external environment as it uses augmented reality.
This is an exploratory study, aiming to assess the current accu-
racy and performance of the head-mounted display, to identify lim-
itations in the current prototype, and to analyze potential reasons
for these limitations.
The mean difference demonstrated a systematic bias of 0.05
logMAR. This is clinically insignificant, suggesting an acceptable
optotype size calibration for our current test. The underestimation
of 0.05 logMAR by the head-mounted display was not due to hard-
ware limitations affecting the clarity or resolution of the optotype,
as the smallest possible clear optotype that could be generated
for this device corresponded to a visual angle of 0.00 logMAR. It
could reflect a clinically insignificant calibration error, which needs
to be verified by further studies with a larger sample size. In terms
of vision screening, our test subjects provided verbal or written
feedback that the instructions on the head-mounted display are
easy to follow and can be self-administered.
The head-mounted display chart was created based on the prin-
ciples of the Snellen chart rather than the logMAR chart. The initial
reason for choosing the former was it still being the most common
visual acuity measurement method in clinical practice.18 However,
www.optvissci.com Optom Vis Sci 2020; Vol 97(8)
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Automated VA Testing Using Head-mounted Display — Ong et al.
the limitations of the Snellen design have been well studied (such
as its nongeometric progression in letter sizing).19–21 We aim to
conduct further studies by designing a head-mounted display
chart based on the logMAR chart principles and comparing their
level of agreement.
One of the key limitations of the current head-mounted display
prototype is its poor precision; the large 95% limits of agreement
(±0.33 logMAR) suggested that the head-mounted display
mis-estimated visual acuity in a considerable number of individ-
uals. This could be multifactorial, contributed by both technical
and user factors.
In terms of technical factors, one cause could be the reduced
effective resolution of our device. In theory, its calculated resolu-
tion of 31.6 cycles per degree allows us to test a 6/6 optotype accu-
rately. However, its effective resolution might be reduced to
accommodate for beat frequency and experience degradation,
which is insufficient for some subjects to accurately view the 6/6
optotype. Because of the unavailability of certain technical specifi-
cations, we were unable to calculate the effective resolution of our
device. We attempted to determine it in a separate study; we tested
the ability of volunteers with 6/6 vision to discern the gap between
two black lines separated by a white pixel, thereby testing the max-
imum contrast of the screen at the spatial frequency of 31.6 cycles
per degree. For a 6/6 optotype to be effectively displayed on our de-
vice screen, the subject must be able to distinguish features of the
smallest representation a screen can produce (which is a 2-pixel
cycle that corresponds to 31.6 cycles per degree). The study
showed that 13.6% of participants with 6/6 vision failed to do so,
suggesting that the effective resolution for our device may be insuf-
ficient to test the reported stimulus range for some subjects. We in-
tend to overcome this limitation in future studies by using a device
with a higher resolution for the same virtual screen distance.
Our analysis suggested a range bias in the head-mounted dis-
play. Compared with measurements by the Snellen chart, it tended
to give an overestimation in patients with poorer visual acuities
(Fig. 3). This could be due to better contrast provided by the smart
glasses compared with the room's lighting for the Snellen chart, as
feedbacked by subjects with poorer visual acuities. The eight-tab
patient response interface on the tablet (Fig. 2) also increases the
possibilities of a false-positive response (where patients correctly
guessed an optotype by chance) as compared with manual testing
using the Snellen chart (in which subjects choose from a range of
26 letters). Currently, our algorithm only checks for false negatives.
There is thus a need to increase the number of response tabs available
and incorporate a check for false-positive responses in our algorithm.
Lastly, we did not use any crowding bars around the single
optotypes to simulate the crowding effect of a standard Snellen chart,
which can affect the ability of the patient to determine optotypes cor-
rectly, resulting in measurement bias. In subsequent studies, we aim
to use a bounding box surrounding the optotypes, which has been clin-
ically validated to simulate the crowding effect.19
In terms of user factors, there could be a certain degree of learn-
ing curve and uncertainty for the users because it is their first expo-
sure to such a device. It is a fully unguided test, and variations
might be more compared with manual testing with the more famil-
iar Snellen chart, which is guided by a medically trained examiner.
In manual testing, interaction between the examiner and the sub-
ject might lead to visual or verbal cues for test subjects, which
may be a potential source of bias. We thus aim to repeat the auto-
mated test in subjects to observe for any improvement in accuracy
and consistency after they are repeatedly exposed to the device.
595
Automated VA Testing Using Head-mounted Display — Ong et al.

There was no significant correlation between the mean differ-
ence and the user's age (r = −0.15) or education level (r = 0.04),
suggesting that the accuracy of our current technology was not lim-
ited by these two user factors.
Choosing the functional value of 6/12 (the minimum visual acu-
ity required for driving based on local regulations) as the cutoff, the
head-mounted display had a specificity of 81.0% and a sensitivity
of 63.6% in detecting visual acuities worse than 6/12 Snellen acu-
ity (0.30 logMAR). The negative predictive value of 90.0% sug-
gests that there is a 10.0% risk for patients with visual acuity
worse than 6/12 that went undetected by our head-mounted dis-
play. The positive predictive value of 46.7% suggests that our
head-mounted display false positively identifies an individual with
a visual acuity worse than 6/12 in 53.3% of the time. Such perfor-
mance would make the device a poor screening tool for reduced vi-
sual acuity and would lead to excessive false-positive referrals,
requiring manual confirmatory testing. The low positive predictive
value could be due to the mean difference of 0.05 logMAR, where
the head-mounted display underestimated visual acuity deter-
mined by the Snellen chart. This is considered a considerable lim-
itation of the head-mounted display in the current form and would
be something that the investigators will address in future iterations
of the device.
In our study, seven subjects (11.7%) were excluded because
they could not provide any usable data. This is in contrast to man-
ual testing with the Snellen chart, where all subjects screened had
usable visual acuity data. Although there remains a small minority
of untestable subjects in any automated test, our completion rate
remains suboptimal for a test that most people traditionally find
easy to perform. There is a need to address the reported reasons
for incompletion in future studies, by simplifying the instructions
for the test (for subjects who did not look at the tablet when
responding) and improving the comfort and fit of the headset (for
subjects who did not wear the head-mounted display properly).
The main advantage of our novel algorithm and head-mounted
display lies in its fully automated nature, allowing multiple patients
to be screened simultaneously with only one operator's help. How-
ever, the head-mounted display test took a longer time to administer
than the manual Snellen test on average (3.18 vs. 1.64 minutes).
In future studies, there is a need to evaluate its true effect on pro-
ductivity by having multiple users being tested simultaneously with
the help of one single technician.
Other potential uses of our portable head-mounted display in-
clude visual acuity testing of bed-bound or immobile patients. Ul-
timately, our aim is to develop it into a platform technology, with
its use expanding to other related ophthalmology examinations,
such as color vision and contrast sensitivity. These examinations
require similar environment and methods of testing as visual acuity
and have potential to be incorporated owing to the device's high
technical specifications (a contrast ratio of 100,000:1 and a
24-bit color reproduction).
Limitations of our study methodology included a small sample
size that consisted of adults only. It did not include the pediatric
population where screening is equally as important to detect and
manage conditions early before progression, such as that for refrac-
tive errors or amblyopia.22 Best-corrected visual acuity with pinhole
occlusion was not compared between the two tests, and we did not
evaluate the test-retest reliability.
In conclusion, the potential of the head-mounted display
and software algorithm lies in its fully automated nature to in-
crease workplace efficiency. Its portability extends its utility
to easy visual acuity screenings for immobile patients. However,
this exploratory study highlights the current limitations of our
preliminary prototype. It has an unacceptable completion rate
and large limits of agreement, contributed by both technical and
user factors.
Moving forward, there is a need to improve the accuracy of the
head-mounted display through various methods (such as the incor-
poration of a crowding box and enhancing the speed and specificity
of our software algorithm). The incorporation of a validated psycho-
physical algorithm such as the Psi method or QUEST+ may also
help.23,24 Future studies are needed evaluate the test-retest reli-
ability through repeated studies on a larger sample size involving
a pediatric population and to incorporate simultaneous testing of
multiple subjects to determine the true effect of the head-mounted
display on increasing labor productivity. The final aim is to develop
it as a platform-portable technology for multiple types of visual
testing, such as color vision and contrast sensitivity.

ARTICLE INFORMATION
Submitted: February 20, 2019
Accepted: June 6, 2020
Funding/Support: National University Health System (NUHS)
Summit Programmes in INnovation (SPIN; to TCVK).
Conflict of Interest Disclosure: None of the authors have
reported a financial conflict of interest. This study received
funding from the National University Health System (NUHS)
Summit Programmes in INnovation (SPIN). The sponsor
provided financial support but had no role in the study
design, conduct, analysis, and interpretation or writing
of the report. The sponsor had no role in the decision to
submit the manuscript for peer-review.
Study Registration Information: Registration Database:
Domain Specific Review Board, Singapore (registration/
protocol number: 2017/00736).
Author Contributions and Acknowledgments: Conceptualization:
TCVK; Data Curation: SCO, LCIP, TLCC, HWS, KCC, CS;
Formal Analysis: SCO, MABR; Funding Acquisition:
TCVK; Investigation: SCO, LCIP, TLCC, HWS, KCC,
CS; Methodology: SCO, LCIP, TLCC, HWS, KCC, CS,
MABR, TCVK; Project Administration: TCVK; Resources:
TCVK; Software: HWS, KCC, CS; Supervision: MABR,
TCVK; Validation: TCVK; Writing – Original Draft: SCO;
Writing – Review & Editing: LCIP, TLCC, HWS, KCC,
CS, MABR, TCVK.
The authors wish to thank the following people for their
help in this study: Lyana Shaffiee from the Eye Surgery
Centre, National University Hospital, who provided her
insight, expertise, and help in participant recruitment
for this research project; colleagues from the School of
Engineering, Ngee Ann Polytechnic, who provided their
expertise in programming the software for automated
visual acuity testing; Chan Yiong Huak from the National
University of Singapore Medicine Biostatistics Unit, who
provided his expertise in statistical analysis; and Danial
Bin Bohan from the Eye Surgery Centre, National
University Hospital, who provided his optometry insight
and expertise.
REFERENCES
1. Wang X, Lamoureux E, Zheng Y, et al. Health Burden
Associated with Visual Impairment in Singapore. Oph-
thalmology 2014;121:1837–42.
2. Bokinni Y, Shah N, Maguire O, et al. Performance of a
Computerised Visual Acuity Measurement Device in
Subjects with Age-related Macular Degeneration: Com-
parison with Gold Standard ETDRS Chart Measure-
ments. Eye 2015;29:1085–91.
3. Aslam TM, Tahir HJ, Parry NRA, et al. Automated
Measurement of Visual Acuity in Pediatric Ophthalmic
Patients Using Principles of Game Design and Tablet
Computers. Am J Ophthalmol 2016;170:223–7.
4. Jones PR, Kalwarowsky S, Atkinson J, et al. Auto-
mated Measurement of Resolution Acuity in Infants
Using Remote Eye-tracking. Invest Ophthalmol Vis Sci
2014;55:8102–10.
5. Keirl A, Christie C. Clinical Optics and Refraction:
A Guide for Optometrists, Contact Lens Opticians and

www.optvissci.com Optom Vis Sci 2020; Vol 97(8) 596

Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.


Dispensing Opticians. Oxford: Elsevier Butterworth-
Heineman; 2007.
6. British Standards Institution. Visual Acuity Test
Types—Part 1: Test Charts for Clinical Determination
of Distance Visual Acuity Specification: BS 4274-1:
2003. London: British Standards Institution; 2003.
7. Jackson A, Bailey I. Visual Acuity. Optom Pract 2004;
5:53–68.
8. Carkeet A. Exact Parametric Confidence Intervals for
Bland-Altman Limits of Agreement. Optom Vis Sci
2015;92:e71–80.
9. Chou R, Dana T, Bougatsos C, et al. Screening for Im-
paired Visual Acuity in Older Adults: Updated Evidence
Report and Systematic Review for the US Preventive
Services Task Force. JAMA 2016;315:915–33.
10. McClintic SM, Prajna NV, Srinivasan M, et al. Visual
Outcomes in Treated Bacterial Keratitis: Four Years of
Prospective Follow-up. Invest Ophthalmol Vis Sci
2014;55:2935–40.
11. Scott IU, Moraczewski AL, Smiddy WE, et al. Long-
term Anatomic and Visual Acuity Outcomes After Initial
Anatomic Success with Macular Hole Surgery. Am J
Ophthalmol 2003;135:633–40.
www.optvissci.com
Copyright © American Academy of Optometry. Unauthorized reproduction of this article is prohibited.
Automated VA Testing Using Head-mounted Display — Ong et al.
12. Naidu G, Correia M, Nirmalan P, et al. Functional
and Visual Acuity Outcomes of Cataract Surgery in
Timor-Leste (East Timor). Ophthalmic Epidemiol
2014;21:397–405.
13. Bach M. The Freiburg Visual Acuity Test—Automatic
Measurement of Visual Acuity. Optom Vis Sci 1996;73:
49–53.
14. Laidlaw DA, Tailor V, Shah N, et al. Validation of a
Computerised Logmar Visual Acuity Measurement Sys-
tem (COMPLOG): Comparison with ETDRS and the
Electronic ETDRS Testing Algorithm in Adults and Am-
blyopic Children. Br J Ophthalmol 2008;92:241–4.
15. Black JM, Jacobs RJ, Phillips G, et al. An Assess-
ment of the iPad as a Testing Platform for Distance Visual
Acuity in Adults. BMJ Open 2013;3:e002730.
16. Pathipati AS, Wood EH, Lam CK, et al. Visual Acuity
Measured with a Smartphone App Is More Accurate Than
Snellen Testing by Emergency Department Providers.
Graefes Arch Clin Exp Ophthalmol 2016;254:1175–80.
17. A Gounder P, Cole E, Colley S, et al. Validation of a
Portable Electronic Visual Acuity System. J Mob Technol
Med 2014;3:35–9.
18. Kaiser PK. Prospective Evaluation of Visual Acuity
Assessment: A Comparison of Snellen versus ETDRS
Optom Vis Sci 2020; Vol 97(8)
Charts in Clinical Practice (an AOS Thesis). Trans Am
Ophthalmol Soc 2009;107:311–24.
19. Bastawrous A, Rono HK, Livingstone IAT, et al. De-
velopment and Validation of a Smartphone-based Visual
Acuity Test (Peek Acuity) for Clinical Practice and
Community-based Fieldwork. JAMA Ophthalmol 2015;
133:930–7.
20. Elliott DB, Sheridan M. The Use of Accurate Visual
Acuity Measurements in Clinical Anti-cataract Formula-
tion Trials. Ophthalmic Physiol Opt 1988;8:397–401.
21. Ferris FL, 3rd, Kassoff A, Bresnick GH, et al. New Vi-
sual Acuity Charts for Clinical Research. Am J
Ophthalmol 1982;94:91–6.
22. Tarczy-Hornoch K, Varma R, Cotter SA, et al. Risk
Factors for Decreased Visual Acuity in Preschool Chil-
dren: The Multi-ethnic Pediatric Eye Disease and
Baltimore Pediatric Eye Disease Studies. Ophthalmol-
ogy 2011;118:2262–73.
23. Kontsevich LL, Tyler CW. Bayesian Adaptive Estima-
tion of Psychometric Slope and Threshold. Vision Res
1999;39:2729–37.
24. Watson AB. Quest+: A General Multidimensional
Bayesian Adaptive Psychometric Method. J Vis 2017;
17:1–10.
597