Challenges in Using Signs and Symptoms to

Evaluate New Biomarkers of Dry Eye Disease

BENJAMIN SULLIVAN, PHD
INTRODUCTION
T
he lack of association between signs and symptoms
in dry eye disease (DED)
1-4
presents a challenge in
designing clinical trials to objectively evaluate the
performance of new biomarkers. When making a diagnosis
based on multiple signs, one must balance the sensitivity of
therapeutic response, temporal variability, the a priori prob-
ability distribution of each sign, and the patients history of
therapy against the practical considerations of patient
recruitment. Proper classication will almost entirely deter-
mine the success or failure of a clinical trial, yet there is little
agreement between the constituent signs in DED.
Tear osmolarity, a test that has been proposed as the
gold standard of DED and has recently become widely
available, represents an example of such a conundrum that
clinicians and researchers must learn how to resolve.
Many decades of basic research have conrmed the role of
tear hyperosmolarity as a causative mechanism and clinical
endpoint in the pathogenesis of DED.
5-10
Current literature
reveals that tear osmolarity demonstrated the highest corre-
lation with disease severity and was found to be the single
best metric to diagnose and classify DED.
11,12
Osmolarity
has been found to be superior in overall accuracy to any
other single test for dry eye diagnosis, even when the other
test measures were applied to a diagnosis within the sample
groups from which they were derived,
9
and recent data sug-
gest that patients with severe keratoconjunctivitis sicca have
a signicantly higher tear lm osmolarity than healthy
controls.
Testing tear lm osmolarity can be a very effective objec-
tive diagnostic tool in the diagnosis of DED.
13
Further, with
tear osmolarity >305 mOsm/L selected as the cut-off value
for dry eye, the likelihood ratio was 10.99, higher than that
used for the other tests. The positive predictive value was
also very high (98.4). Osmolarity performed well in dry
eye diagnosis, better than the other tests considered, mainly
in severe dry eye. Tear osmolarity values should be inter-
preted as an indicator of the DED evolutionary progression
to severity.
14
Basic studies have revealed that tear osmolarity
may be more relevant to the clinician in the diagnosis of
ocular surface diseases with an increased level of inamma-
tory mediators.
15
Finally, more recent data demonstrated
that tear osmolarity was the least variable of all the common
signs across a clinically relevant timeframe, and was the only
sign to reduce its variation upon application of effective
therapy.
16
Despite the demonstrated success of tear osmolarity as a
marker of dry eye, critics suggest that it provides little clin-
ical value. Messmer et al
17
and Szalai et al
18
reported that
osmolarity measurements were not able to distinguish
between healthy volunteers and patients with dry eye.
However, these early papers failed to 1) take into account
patient therapeutic status, 2) use the maximum of two eyes
indiagnosis, 3) report validated quality controls, and 4) recog-
nize the heteroscedasticity of the marker as an indicator of
tear lm instability. Probably the most important aspect
of these studies is that they uniformly relied upon a series of
threshold-limited, uncorrelated variables (corneal staining,
tear lm breakup time [TFBUT], etc.) to determine which
patients were and were not classied as dry eye subjects.
Such inconsistent results highlight the dilemma of using
combinations of signs to inform patient diagnosis or assess
disease severity. This editorial explores the various implica-
tions and approaches of using multiple uncorrelated signs
for diagnosis and severity assessment in DED.
STATISTICAL INDEPENDENCE AND SEVERITY
ASSESSMENT
Hypothetically, if tear osmolarity were perfectly corre-
lated with staining, Schirmer test results, or symptom ques-
tionnaires, there would be no reason to measure it e the
information would already be available. Correlation with in-
dividual signs is therefore a poor choice for evaluating the
clinical utility of new biomarkers, especially in a disease
such as dry eye in which no consensus gold standard mea-
surement exists. If a new marker gives new information, one
must presuppose that clinical markers will routinely disagree
Accepted for publication October 2013.
From TearLab Corporation, San Diego, CA.
The author is Chief Scientic Ofcer of TearLab Corp., which manufactures
and distributes the TearLab Osmolarity System.
Corresponding author: Benjamin Sullivan, PhD, Chief Scientic Ofcer,
TearLab Corp., 9980 Huennekens St., Suite 100, San Diego, CA 92121.
Tel: 760-224-4595. E-mail address: bdsulliv@TearLab.com
2014 Elsevier Inc. All rights reserved.
Guest Editorial
2 THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com
and that conicting information is the norm rather than the
exception (Table 1). For instance, the absence of staining
gives the clinician no information as to whether that same
patient will have normal TFBUT, yet we nd few papers
that are critical of staining due to a lack of correlation to
TFBUT. Correlation across small subsets of patients is
certainly possible and has been observed,
19,20
but in large
cohorts where there are diverse underlying etiologies, signs
tend to diverge.
21
The implications of statistical independence have a pro-
found impact on how combinations of signs of dry eye inu-
ence diagnosis and severity assessment. A particularly good
example can be found in a paper by Chotikavanich et al, de-
tailing the production and activity of MMP-9 on the ocular
surface.
22
Patients were included based on a threshold of
Ocular Surface Disease Index (OSDI) symptoms >20, and
one or more of the following signs: TFBUT 7, punctate
corneal uorescein staining, or Schirmer I score <10 mm.
Computerized videokeratoscopy provided a measure of
ocular surface regularity. Subjects were then classied into
dysfunctional tear syndrome severity levels (DTS1eDTS4)
based on increases in corneal staining, while the other signs
were unconstrained and allowed to vary. Predictably, there
was no discernable pattern to the symptoms, surface regu-
larity index, TFBUT, or conjunctival staining for the 35 sub-
jects classied as DTS1eDTS3. Despite the attempt to sort
patients into increasing disease severity categories, symp-
toms decreased from DTS1 to DTS3, the ocular surface reg-
ularity actually improved from DTS1 to DTS3, and the
TFBUT and conjunctival staining were less severe in DTS2
than in DTS3. While all of the signs were much worse in
DTS4, that cohort was populated with patients with graft
vs host disease, Stevens-Johnson syndrome, or Sjgren syn-
drome (SS), whereas DTS1eDTS3 were representative of
common dry eye patients, featuring meibomian gland dis-
ease, blepharitis, and history of LASIK.
So what happened? Chotikavanich and co-authors are
world-class experts at identifying dry eye, so these outcomes
cannot be an indictment of the clinical technique. Rather,
the distributions are a direct result of the statistical indepen-
dence of dry eye signs. When uncorrelated signs are sorted
based on a small subset of markers, the signs that were not
used in the stratication will become randomly distributed.
This phenomenon is also observed in the critiques of tear
osmolarity listed above; patients stratied based on
thresholds of uncorrelated variables produced random
distributions in tear osmolarity. Chotikavanich et al do
not comment on the value of symptoms, TFBUT, or ocular
surface regularity in DED any more than Szalai
18
and
Messmer
17
provide information regarding the clinical utility
of tear osmolarity. Instead, the random distributions are
entirely a consequence of the choices made when stratifying
patients due to the underlying independence of dry eye
signs.
A similar investigation by Huang et al, also performed
by highly qualied and experienced researchers, measured
biomarkers in 102 subjects.
23
Dry eye severity was deter-
mined by symptoms and stratied by increasing corneal
staining. As expected, DE1eDE3 groups exhibited linear in-
creases in staining with low intragroup variation, while
Schirmer tests and tear osmolarity exhibited noisy, random
behavior with large spreads within each group. If severity
classications based on staining and symptoms were accu-
rate, we would expect Schirmer scores to be lower in pa-
tients with severe dry eye compared to those with more
moderate disease, yet DE3 (11.39.3 mm) showed higher
wetting than DE2 (8.86.7 mm). We also expect that tear
osmolarity should rise when progressing from the moder-
ately affected DE2 patients to the more severely affected
DE3 patients, yet this is not what is observed when staining
and symptoms alone estimate disease severity
(DE2306.811.6 mOsm/L to DE3302.610.2 mOsm/L).
Clearly, the methods used to formally stratify disease
severity in a clinical trial setting can have a variety of unin-
tended consequences that disagree with routine clinical
experience.
SENSITIVITY OF THERAPEUTIC RESPONSE
One of the most signicant confounding variables in
classifying patients in a trial is the varying sensitivity each
marker has to concomitant therapy. For instance, several
articial tear formulations have been shown to signicantly
reduce tear osmolarity prior to the response from other signs
or symptoms, which tend to be less sensitive to therapeutic
intervention than osmolarity.
16,24,25
If the relative sensitiv-
ities and dynamics are not taken into consideration in form-
ing inclusion criteria, a cross-sectional study of SS subjects
who are heavily treated would be expected to have a lower
incidence of hyperosmolarity but moderate-to-high symp-
toms. Indeed, that is what has been reported in the litera-
ture.
18,20
Compare this to a case report of tear osmolarity
readings on untreated SS patients presenting to a general
ophthalmologist
26
:
I really didnt know how [tear osmolarity] would impact my
practice. I gured I was pretty good at telling if someone had
dry eye, and I wasnt sure what a tear osmolarity number would
do for my treatment algorithm. Then, a patient well call Tammy
walked in to my ofce. She had bounced fromophthalmologist to
ophthalmologist and had even been recommended to see a psy-
chiatrist. She had severe ocular surface discomfort, dryness, pain,
and intermittent blurred vision. Her story was classic for dry eye.
The only problem was that she had no corneal staining whatso-
ever. She seemed to have an adequate tear lake and lacked the
OUTLINE
Introduction
Statistical Independence and Severity Assessment
Sensitivity of Therapeutic Response
Temporal Variation
The Problem with Symptoms
Possible Solutions and Strategies for Combinations of
Markers
Conclusion
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com 3
Table 1. Typical presentation of dry eye signs (worse of both eyes reported) and symptoms. Data extracted from the multi-center TearLab core validation study.
2,11,12
Conicts between individual dry eye sign diagnoses and the composite disease severity score are shaded. Note that no one sign consistently agrees with the
consensus and that conicting information is the norm, not the exception. Diagnostic cutoffs used: osmolarity >308 mOsm/L; TFBUT< 7 s; Schirmers <7 mm;
corneal staining >2/16; conjunctival staining >2/12; meibomian grading score >5, OSDI >15
Gender Age Osmolarity TFBUT Schirmer Corneal Staining Conjunctival Staining Meibomian Grading OSDI Composite Severity
Normal Patients
Female 63 300 10.0 8 2 0 1 2.5 0.15
Female 27 298 9.7 25 0 0 8 0 0.16
Female 36 303 6.3 25 2 4 2 4.2 0.18
Female 27 310 12.3 25 1 1 7 2.1 0.19
Female 32 291 5.3 19 0 1 6 27.1 0.19
Male 69 301 7.0 35 0 0 5 27.5 0.20
Mild/Moderate Dry Eye
Female 54 311 9.0 15 0 2 0 22.5 0.22
Female 24 317 5.3 10 3 1 0 6.3 0.23
Female 45 318 3.0 35 2 3 0 52.3 0.30
Female 62 319 2.3 5 5 7 0 7.5 0.31
Female 55 328 1.7 10 4 5 2 22.2 0.33
Severe Dry Eye
Female 46 341 2 1 6 7 11 18.8 0.43
Female 66 356 4 0 1 6 10 65.9 0.48
Female 59 368 2 5 1 8 12 25 0.51
Female 54 391 6 12 4 8 15 52.1 0.62
E
V
A
L
U
A
T
I
N
G
B
I
O
M
A
R
K
E
R
S
O
F
D
R
Y
E
Y
E
/
S
u
l
l
i
v
a
n
4
T
H
E
O
C
U
L
A
R
S
U
R
F
A
C
E
/
J
A
N
U
A
R
Y
2
0
1
4
,
V
O
L
.
1
2
N
O
.
1
/
w
w
w
.
t
h
e
o
c
u
l
a
r
s
u
r
f
a
c
e
.
c
o
m
other typical signs of dry eye that we are used to seeing. Every
doctor before had told her she was crazy and that she didnt
have dry eye. I thought, Lets get the tear osmolarity and see
if the quality of her tears could be the real problem. We did a
quick test, and found that her osmolarity was off the charts
and consistent with SS. Upon further questioning, she did reveal
some mild oral dryness and some arthritis she had recently been
dealing with. We promptly set her up to see a rheumatologist
who conrmed the diagnosis with further testing. She is now
on systemic steroid sparing immune modulating therapy and
her symptoms are under much better control. Every time she
comes in, she thanks me for nally giving her a diagnosis and
explanation of her symptoms, and for getting her in the hands
of a rheumatologist for treatment.
I learned a great deal about the widely variable presentation of
SS from this patient. Over the past year, I have diagnosed about
12 other patients with SS based on their tear osmolarity and a
convincing history, many times in the absence of other stigmata
of dry eye.
26
Similarly, a 2013 study examined the frequency of DED
in early rheumatoid arthritis (RA) where none of the pa-
tients enrolled in the study had received antirheumatic
drugs or steroids, or used any other topical or systemic med-
ications.
27
Even though the investigators tested only one eye,
thereby substantially reducing the statistical power of the
tear osmolarity test,
16
there was clear agreement between
overall RA disease severity and tear osmolarity (r0.710
vs DAS28, p <0.001). Subjects with severe dry eye demon-
strated signicantly higher osmolarity than those in the
mild and moderate groups. These results are similar to those
of other studies,
11,19,28
where osmolarity correlated with
overall disease severity, with little or no correlation with
the other individual tests.
2
By contrast, Szalai et al noted that 100% of their SS
group and 43% of their dry eye group were treated with a
series of compounds known to normalize tear osmolarity,
and no attempt was made to wash out or control for thera-
peutic intervention
18
; despite these structural shortcomings,
about half of the subjects had elevated tear osmolarity in the
SS group.
Another study examining a highly treated SS population
found that a majority of subjects (67%) had elevated levels of
tear osmolarity,
21
even though subjects were qualied using
the American-European Consensus Criteria, which allowed
the use of articial tears >3 times per day as a parallel inclu-
sion criterion, and the authors chose to report the mean of
the two eyes rather than the maximum, which noticeably de-
grades the statistical performance of tear osmolarity.
16
In
that cohort, a trend toward lowered osmolarity was observed
in the systemic anti-inammatory and hydroxychloroquine
groups, although the difference was not statistically signi-
cant. The impacts of sh oil, topical cyclosporine, predni-
sone, methotrexate, or axseed oil on the population were
not separately analyzed, although other studies suggest an
attenuation of tear osmolarity with these compounds.
16,29
Finally, Utine et al studied an SS subset, 100% of whom
were on systemic hydroxychloroquine, articial tears, and a
variety of steroids and anti-inammatory compounds. They
observed that the correlation between tear osmolarity and a
series of other signs improved after the patients on pilocar-
pine and cyclosporine A were removed from the regression
analysis.
20
Again, this study found that osmolarity carried
the highest correlation with the overall DEWS classication,
generally poor agreement with the subset signs, and evi-
dence that therapeutic intervention differentially affects clin-
ical signs.
TEMPORAL VARIATION
The discontinuity between measures of DED is likely
physiological. Signs such as Schirmer score, poor lipid
composition, and meibomian gland grade tend to appear
only in subset etiologies of the disease, while tear lm insta-
bility and osmolarity are considered more global markers.
30
Moreover, tear instability may cause time-varying expres-
sion,
16,31,32
which superimposes biological noise atop analyt-
ical inaccuracy. The impact temporal variability can have on
calculations of disease severity was illustrated in a study by
Sullivan et al.
16
It was shown that without a change in ther-
apy over a 3-month timeframe, corneal staining (out of 16)
can spontaneously change from grade 7 to 3 to 8 in one eye,
grade 5 to 2 to 7 in the other eye, all while OSDI symptoms
steadily increased from a score of 25 to 38 to 45 out of 100.
In another patient, Schirmer score decreased from 20 to 15
to 8 mm in one eye, increased from 10 to 18 to 20 mm in the
other eye, while OSDI symptoms decreased from a score of
61 to 48 to 44. Finally, a third patient reported oscillating
symptom scores of 53 to 17 to 58, while staining spontane-
ously improved in both eyes, from a grade of 5 to 3 to 2 in
one eye and a grade of 5 to 4 to 2 in the other eye.
16
For a
cross-sectional study, depending on the day these patients
were screened, temporal variability alone would create false
positives and false negatives for the signs not used in the
stratication.
Unlike most DED signs, the variability of tear osmolarity
has received much attention.
33-35
Older studies went so far
as to suggest that the testing methodology itself was variable.
34
However, recent reports of real-world performance from over
2,200 tests across 113 individual ophthalmology and optom-
etry clinics in the US monitored under the Clinical Laboratory
Improvement Act (CLIA) regulations demonstrated consis-
tent analytical precision of 4e5 mOsm/L using a point-
of-care tear osmometer.
36
It should also be recognized that
after several years of broad utility, the temporal changes in
tear osmolarity have been placed into appropriate context.
Tear osmolarity exhibits very little change over time or
between eyes in the healthy tear lm, but as the body loses
control in disease, tear lm instability is reected in the
steadily increasing eye-to-eye changes.
12,33
This is a statisti-
cal characteristic called heteroscedasticity, or increasing vari-
ation with increasing value. Keech et al found that after
several thousand serial tests of symptomatic and asymptom-
atic subjects, the standard deviation of tear osmolarity could
reliably and linearly indicate the loss of homeostasis
(r
2
0.87 against tear instability) and that variation in
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com 5
osmolarity is, indeed, a hallmark of DED.
33
Along with the
data that demonstrate an increasing eye-to-eye deviation
with increasing disease severity, variability is something cli-
nicians should be looking for to help identify patients with
DED.
12
In essence, eye-to-eye differences beyond the
threshold of 8 mOsm/L should be considered a conrming,
not confounding, indication of the loss of tear lm homeo-
stasis, ie, DED.
Remarkably, Keech et al also showed that in healthy in-
dividuals, the ocular surface is so good at maintaining ho-
meostasis that day-to-day and test-to-test variation is often
indistinguishable from the serial measurement of quality
control solutions traceable to national reference standards.
33
In longitudinal studies on a clinically relevant timeline, the
variation of tear osmolarity was statistically identical or bet-
ter than the other dry eye signs.
16
Formally, repeat testing
on 14 people demonstrated a reference change value for
tear osmolarity between 7 mOsm/L @ 80% to 14 mOsm/L
at 95% condence,
37
which is in line with current data.
THE PROBLEM WITH SYMPTOMS
DED is typically considered a symptomatic disease, yet
the DEWS report stated: For its assignment, the Diagnostic
Methodology Subcommittee regarded dry eye as a chronic,
symptomatic ocular surface disease, which may, however,
occasionally be asymptomatic. Asymptomatic dry eye im-
plies that in the absence of symptoms, some objective
criteria of dry eye may still be satised, such as tear hyper-
osmolarity, the presence of interpalpebral ocular surface
staining, reduced tear production, or tear instability. The
presence of symptoms may not always be clear-cut, particu-
larly when they develop insidiously.
10
Like any other measure of disease, symptoms are subject
to signicant variation over time, even within the same
day.
3,4,16,38,39
Additional complicating factors include the
well-known corneal hyperalgesia associated with neuropa-
thy,
40
aberrant spontaneous and stimulus-evoked nerve im-
pulse ring, ie, phantom corneas,
41
and the overlap in
symptoms with distinct diseases, such as conjunctivochala-
sis, asthenopia conditions, and recurrent corneal erosions,
among others.
42
Additionally, there are age- and gender-
related, cultural, and ethnic inuences on symptoms
43-45
that may not be fully appreciated but contribute to the
lack of association between patient reporting of symptoms
and disease severity.
As such, these and other factors pose a signicant risk of
misclassication if one relies on symptoms.
46
A pertinent
example is the study used to validate the minimal clinically
important difference for the OSDI.
38
In that report, the
baseline OSDI score of 310 subjects was used to separate
subjects into categories of normal (OSDI average7.83.1),
mild (17.22.9), moderate (26.52.5), and severe
(52.616.4). Upon follow-up, the previously classied
normal subjects reported higher symptoms (22.619.4)
than either the mild (16.113.6) or moderate (19.413.5)
cohorts, with some previously normal subjects stretching
into the severe category based on the authors own criteria.
Substantial mean regression from the severe subjects (down
to 33.623.2) was also observed.
38
Other studies have
shown that different symptom questionnaires tend to
show only moderate agreement, with r
2
ranging between
0.05 to 0.59 for the OSDI compared to McMonnies, the
NEIVFQ-25, and the SF-12,
47
or even newer instruments
such as the DEQ-5 (r
2
0.53).
48
Accordingly, symptoms should not be given favored sta-
tus in disease severity assessment, as they behave identically
to the other statistically independent signs. Studies that
choose to rely upon symptoms or self-reported assessments
to gauge disease presence or severity should fully expect that
the distributions of the signs measured in parallel will be
largely random.
POSSIBLE SOLUTIONS AND STRATEGIES FOR
COMBINATIONS OF MARKERS
The landmark document, the 2007 Tear Film and Ocular
Surface Society Report of the Dry Eye WorkShop (DEWS),
is silent with regard to how best to objectively combine
markers for diagnosis and severity assessment.
10
The
DEWS dry eye severity grading scheme gives broad cate-
gories and a few numerical breakpoints on the path to
severity, but fails to address conicts among signs. What
happens if a subject has a TFBUT of 4 seconds but a 35
mm Schirmer test? The original Delphi panel, which was
cited as the basis for the DEWS severity scheme, found a
consensus that severity of disease should be the primary
determinant for the therapeutic strategy chosen.
49
More-
over, the panel reached consensus that the level of severity
should be based primarily on symptoms and clinical signs.
The panel members agreed that diagnostic tests are second-
ary considerations in determining disease severity. The value
of diagnostic tests was considered to be in conrming clin-
ical assessment. Again, many of the available tests were
deemed not useful for diagnosis, staging, or evaluating
response to therapy in dysfunctional tear syndrome. In
addition, observation of the patient response to initial ther-
apy was deemed as an important indicator of disease
severity and further treatment selection.
49
It is important to note that the expert panel was sur-
veyed several years prior to the widespread availability of
tear osmolarity, a sign repeatedly shown to track overall dis-
ease severity and offer a sensitive means to track therapeutic
response. Based on the low opinion of the available tests at
the time, it seems preferable that tear osmolarity not corre-
late to those signs, but that it reect what the panel suggests
is most useful: severity and therapeutic response.
At the time, the Delphi study preferred that doctors
consider a myriad of qualitative clinical signs, such as lid tel-
angiectasia, hyperemia, inspissation, tear lm debris, the
luster of the conjunctiva, symblepharon, erosions, scarring
and blurry uctuating vision, in addition to symptoms, in
order to gauge severity.
49
Unfortunately, no quantitative
(or evidence-based) methods were suggested as to how to
actually combine signs cited in the manuscript. The nal
grading scheme relied almost entirely upon increasing
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
6 THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com
symptoms and increasing staining to dene severity, while
largely omitting the integration of the other clinical signs.
Nevertheless, a very important insight can be gleaned
from the Delphi report: when evaluating many bits of noisy,
loosely related information, doctors prefer to conrm a
clinical assessment by measuring ocular surface parameters.
In a formal sense, this equates to the use of a logical OR
from a long list of possible inputs to support what the physi-
cian already suspects (OR as in A or B, where if any signs
are positive, conrm the diagnosis of DED). While this is a
perfectly acceptable protocol for diagnosis in certain situa-
tions, eg, if the perceived risk of inaction mandates that
intervention take precedence over diagnostic accuracy
(which is most certainly true presurgically or for acute cases
of ocular surface disease), this approach leaves no room for
tests that disagree with the assumed etiology. A common
refrain from established physicians is that they dont need
a diagnostic test to identify when someone has DED. But,
what about when they dont have DED? Clinicians now
report, with increasing frequency, patients complaining of
DED-like symptoms who display osmolarity in the normal
range.
12
Many of these patients have been treated, somewhat
unsuccessfully, for years. After taking a closer look at that
subset of patients, physicians generally discover ocular sur-
face complications that are not related to the dysfunction
of the lacrimal or meibomian gland. In the long-term man-
agement of a chronic disease, the logical OR approach will
tend to over-include subjects if diagnoses are based on clin-
ical signs, as observations do not give much information
about the molecular cause of the disease, and there is a
good chance that at least a few signs will be positive.
Therefore, if a logical OR is used, it is necessary to accu-
mulate data from signs that are highly specic and selective
for dry eye. Tear osmolarity is one, if not the most specic,
sign of DED.
12
In addition to its agreement with overall dis-
ease severity, the sensitivity to therapeutic intervention and
the high specicity of tear osmolarity gives the marker much
of its clinical value e normal subjects do not have elevated
tear lm osmolarity. That many of the preferred clinical
signs lack specicity for dry eye and can appear in other
forms of ocular surface disease was not addressed by the
Delphi panel.
An example of a well-executed logical OR approach was
recently published by Versura et al.
50
The authors examined
combinations of highly specic but individually less sensi-
tive protein markers (albumin, total protein level,
lipocalin-1, lactotransferrin) to achieve likelihood ratios up-
ward of 20 times higher than combinations of TFBUT and
Schirmer values, and 4 times higher than combinations of
TFBUT and corneal staining values, when measured against
the DEWS severity grade. Note that, like tear osmolarity, in-
dividual protein expression demonstrated no correlation
with traditional clinical signs, ranging from r
2
0.00 to
0.17. The lack of overlap in protein expression patterns be-
tween patients led to a low sensitivity for each of the individ-
ual markers; yet because the information was directly linked
to underlying molecular mechanisms of disease,
51
the
combination of signs in a logical OR context provided
very high diagnostic performance against the DEWS com-
posite without increasing the rate of false-positives.
50
There is a growing literature regarding composite
indices of dry eye. These techniques typically involve map-
ping a series of signs onto a common basis through normal-
ization or ordinal ranking, followed by a weighted
summation of the constituent signs.
11,52-55
The methods to
construct composites can be as straightforward as ranking
each sign on a 1-3 scale and then adding up each mea-
sure,
52,53
to advanced statistical approaches such as indepen-
dent components analysis, factor analysis, or latent class
analysis.
53-55
One of the main benets of any composite
approach is that random temporal variability is dampened
across multiple signs. Composites also solve the problems
of stratifying data based on small subsets of signs, where
lack of correlation results in substantial distortions of the
nonstratied variables. Instead, composite approaches pro-
vide a general trend of increasing severity of signs with
increasing disease, across all of the constituent signs. In
many ways, composite methods attempt to mimic the role
physicians play in the clinic, by aggregating disparate data
into a reliable decision. Finally, composite rankings can be
plotted as a continuous variable that allows probability den-
sity functions to be estimated, which may be the most useful
aspect of any composite. As long as there are a sufcient
number of signs included in the new basis to remove bias,
many of the non-linearities and non-Gaussian behaviors
of the constituents can be elucidated through this
approach.
11
The downside is that, in addition to being quite difcult
to implement de novo, comparing the outputs of composite
methods all suffer from a fundamental problem: which
truth, ie, gold standard, was used to validate the compos-
ite? Do you judge a composite based on the ability to
recreate clinical opinion? If so, it would be far simpler to
just use clinical opinion as the arbiter of disease for a clinical
trial. Given that clinical opinion can be misled by a paucity
of information regarding the molecular causality of a dis-
ease, how then do we certify truth? No amount of statistical
machination or complexity can substitute for quality infor-
mation. One might judge composites based on which con-
stituent signs end up ranked highest in their weighting
schema. Would you trust a composite based primarily
upon symptoms and osmolarity or a composite that claims
Schirmer and TFBUT values are the most informative signs?
My guess is that if Schirmer and TFBUT results were the
answer, we probably would have had more than one drug
approved by the FDA for DED in the past few decades.
Essentially, the question still comes down to: Which signs
do you trust to establish truth? I suppose we should gauge
composites the same way one nds utility in signs. Does it
give you an estimate of disease severity that allows someone
to make a prescribing decision? Does the measure improve
with therapy, and does it lag or lead symptoms? Do other
diseases affect the composite, and does it provide informa-
tion regarding the underlying etiology of the disease?
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com 7
CONCLUSION
With the recent technical advances in the ability to mea-
sure biomarkers in nanoliter volumes of tear lm, we are on
the precipice of a paradigm shift in the traditional methods
of diagnosing and managing ocular surface disease. The new
era will most likely be characterized by an abundance of mo-
lecular information. This will challenge our notions of how
to treat, monitor, and classify disease. It is very likely that we
will get closer to the causes of many disorders, rather than
simply observing the nonspecic effects. It is my expectation
that broad etiological categories, such as DED, will give way
to combinations of causal phenomena. If we are lucky, we
may soon be discussing patients presenting with hyperos-
molarity, high IL-1R, elevated HLA-DR, low IgE, and high
glycated albumin. Finding a way to evaluate multidimen-
sional data will be paramount for this upcoming era.
REFERENCES
1. Nichols KK, Nichols JJ, Mitchell GL. The lack of association between
signs and symptoms in patients with dry eye disease. Cornea 2004;23:
762-70
2. Sullivan BD, Crews LA, Messmer EM, et al. Correlations between
commonly used objective signs and symptoms for the diagnosis of
dry eye disease: clinical implications. Acta Ophthalmol 2012 Dec 28.
http://dx.doi.org/10.1111/aos.12012 [Epub ahead of print]
3. Cardona G, Marcelln C, Fornieles A, et al. Temporal stability in the
perception of dry eye ocular discomfort symptoms. Optom Vis Sci
2010;87:1023-9. http://dx.doi.org/10.1097/OPX.0b013e3181ff99ab
4. Begley CG, Chalmers RL, Abetz L, et al. The relationship between
habitual patient-reported symptoms and clinical signs among patients
with dry eye of varying severity. Invest Ophthalmol Vis Sci 2003;44:
4753-61
5. Krogh A, Lund CG, Pedersen-Bjergaard K. The osmotic concentration
of human lacrymal uid. Acta Physiol Scand 1945;10:88
6. Schaeffer AJ. Osmotic pressure of the extraocular and intraocular uids.
Arch Ophthalmol 1950;43:1026-35
7. Mastman GJ, Baldes EJ, Henderson JW. The total osmotic pressure of
tears in normal and various pathologic conditions. Arch Ophthalmol
1961;65:509-13
8. Farris RL. Tear osmolarityea new gold standard? Adv Exp Med Biol
1994;350:495-503
9. Tomlinson A, Khanal S, Ramaesh K, et al. Tear lm osmolarity: deter-
mination of a referent for dry eye diagnosis. Invest Ophthalmol Vis Sci
2006;47:4309-15
10. [No authors listed]. The denition and classication of dry eye disease:
report of the Denition and Classication Subcommittee of the Inter-
national Dry Eye WorkShop (2007). Ocul Surf 2007;5:75-92
11. Sullivan BD, Whitmer D, Nichols KK, et al. An objective approach to
dry eye disease severity. Invest Ophthalmol Vis Sci 2010;51:6125-30.
http://dx.doi.org/10.1167/iovs.10-5390. Epub 2010 Jul 14
12. Lemp MA, Bron AJ, Baudouin C, et al. Tear osmolarity in the diagnosis
and management of dry eye disease. Am J Ophthalmol 2011;151:792-798.
e1. http://dx.doi.org/10.1016/j.ajo.2010.10.032. Epub 2011 Feb 18
13. Jacobi C, Jacobi A, Kruse FE, Cursiefen C. Tear lm osmolarity
measurements in dry eye disease using electrical impedance technol-
ogy. Cornea 2011;30:1289-92. http://dx.doi.org/10.1097/ICO.
0b013e31821de383
14. Versura P, Profazio V, Campos EC. Performance of tear osmolarity
compared to previous diagnostic tests for dry eye diseases. Curr Eye
Res 2010;35:553-64. http://dx.doi.org/10.3109/02713683.2010.484557
15. VanDerMeid KR, Su SP, Ward KW, Zhang JZ. Correlation of tear in-
ammatory cytokines and matrix metalloproteinases with four dry
eye diagnostic tests. Invest Ophthalmol Vis Sci 2012;53:1512-8
16. Sullivan BD, Crews LA, Snmez B, et al. Clinical utility of objective tests
for dry eye disease: variability over time and implications for clinical tri-
als and disease management. Cornea 2012;31:1000-8. http://dx.doi.org/
10.1097/ICO.0b013e318242fd60
17. Messmer EM, Bulgen M, Kampik A. Hyperosmolarity of the tear lm in
dry eye syndrome. Dev Ophthalmol 2010;45:129-38. http://dx.doi.org/
10.1159/000315026. Epub 2010 May 18
18. Szalai E, Berta A, Szekanecz Z, et al. Evaluation of tear osmolarity in
non-Sjgren and Sjgren syndrome dry eye patients with the TearLab
system. Cornea 2012;31:867-71. http://dx.doi.org/10.1097/ICO.
0b013e3182532047
19. Suzuki M, Massingale ML, Ye F, et al. Tear osmolarity as a biomarker
for dry eye disease severity. Invest Ophthalmol Vis Sci 2010;51:4557-61.
http://dx.doi.org/10.1167/iovs.09-4596. Epub 2010 Apr 14
20. Utine CA, Bakgil M, Yavuz S, ifti F. Tear osmolarity measure-
ments in dry eye related to primary Sjgrens syndrome. Curr Eye
Res 2011;36:683-90
21. Bunya VY, Langelier N, Chen S, et al. Tear osmolarity in Sjgren syn-
drome. Cornea 2013;32:922-7. http://dx.doi.org/10.1097/ICO.
0b013e31827e2a5e
22. Chotikavanich S, de Paiva CS, Li de Q, et al. Production and activity of
matrix metalloproteinase-9 on the ocular surface increase in dysfunc-
tional tear syndrome. Invest Ophthalmol Vis Sci 2009;50:3203-9
23. Huang JF, Zhang Y, Rittenhouse KD, et al. Evaluations of tear protein
markers in dry eye disease: repeatability of measurement and correla-
tion with disease. Invest Ophthalmol Vis Sci 2012;53:4556-64
24. Montani G. Intrasubject tear osmolarity changes with two different
types of eyedrops. Optom Vis Sci 2013;90:372-7. http://dx.doi.org/10.
1097/OPX.0b013e318288bdbe
25. Benelli U, Nardi M, Posarelli C, Albert TG. Tear osmolarity measure-
ment using the TearLab Osmolarity System in the assessment of dry
eye treatment effectiveness. Cont Lens Anterior Eye 2010;33:61-7.
http://dx.doi.org/10.1016/j.clae.2010.01.003. Epub 2010 Feb 12
26. Wortz G. Sjgrens signs, seldom simple and straightforward; the role of
multiple tear tests in diagnosis. The Dry Eye Review, http://www.
thedryeyereview.com/2013/08/sjogrens-signs-seldom-simple-the-role-
of-multiple-tear-tests-in-diagnosis/
27. Trkylmaz K, Kkali Trkylmaz A, Kurt A, et al. Investigation of
tear osmolarity in early rheumatoid arthritis: relation to disease activity.
Can J Ophthalmol 2013;48:235-9
28. Liu H, Begley C, Chen M, et al. A link between tear instability and
hyperosmolarity in dry eye. Invest Ophthalmol Vis Sci 2009;50:3671-9.
http://dx.doi.org/10.1167/iovs.08-2689. Epub 2009 Mar 25
29. Larmo PS, Jrvinen RL, Setl NL, et al. Oral sea buckthorn oil atten-
uates tear lm osmolarity and symptoms in individuals with dry eye.
J Nutr 2010;140:1462-8
30. Bron AJ. Diagnosis of dry eye. Surv Ophthalmol 2001;45(Suppl 2):S221-6
31. Cuevas M, Gonzlez-Garca MJ, Castellanos E, et al. Correlations
among symptoms, signs, and clinical tests in evaporative-type dry eye
disease caused by Meibomian gland dysfunction (MGD). Curr Eye
Res 2012;37:855-63
32. Nichols KK, Mitchell GL, Zadnik K. The repeatability of clinical mea-
surements of dry eye. Cornea 2004;23:272-85
33. Keech A, Senchyna M, Jones L. Impact of time between collection and
collection method on human tear uid osmolarity. Curr Eye Res 2013;38:
428-36. http://dx.doi.org/10.3109/02713683.2013.763987. Epub2013Feb12
34. Khanal S, Millar TJ. Barriers to clinical uptake of tear osmolarity mea-
surements. Br J Ophthalmol 2012;96:341-4. http://dx.doi.org/10.1136/
bjo.2011.202754. Epub 2011 May 23
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
8 THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com
35. Eperjesi F, Aujla M, Bartlett H. Reproducibility and repeatability of the
OcuSense TearLab osmometer. Graefes Arch Clin Exp Ophthalmol
2012;250:1201-5. http://dx.doi.org/10.1007/s00417-012-1961-4. Epub
2012 Feb 21
36. Sullivan BD, Zmina SG, Zmina MG, et al. Real world analytical perfor-
mance of the TearLab osmolarity system with an enhanced temperature
sensor. Invest Ophthalmol Vis Sci 2013;54. ARVO E-Abstract 4336
37. Fortes MB, Diment BC, Di Felice U, et al. Tear uid osmolarity as a po-
tential marker of hydration status. Med Sci Sports Exerc 2011;43:1590-7.
http://dx.doi.org/10.1249/MSS.0b013e31820e7cb6
38. Miller KL, Walt JG, Mink DR, et al. Minimal clinically important dif-
ference for the ocular surface disease index. Arch Ophthalmol
2010;128:94-101. http://dx.doi.org/10.1001/archophthalmol.2009.356
39. Nichols KK, Nichols JJ, Mitchell GL. The reliability and validity of
McMonnies Dry Eye Index. Cornea 2004;23:365-71
40. Rosenthal P, Borsook D. The corneal pain system. Part I: the missing
piece of the dry eye puzzle. Ocul Surf 2012;10:2-14
41. Belmonte C. Eye dryness sensations after refractive surgery: impaired
tear secretion or phantom cornea? J Refract Surg 2007;23:598-602
42. Lonsberry B, Karpecki PM. Can you spot these dry eye imposters? Rev
Optom, http://www.revoptom.com/continuing_education/tabviewtest/
lessonid/108536/
43. Tran N, Graham AD, Lin MC. Ethnic differences in dry eye symptoms:
Effects of corneal staining and length of contact lens wear. Cont Lens
Anterior Eye 2013 Jul 11. pii: S1367e0484(13)00082-9. http://dx.doi.
org/10.1016/j.clae.2013.06.001. [Epub ahead of print]
44. Schaumberg DA, Sullivan DA, Buring JE, Dana MR. Prevalence of dry
eye syndrome among US women. Am J Ophthalmol 2003;136:318-26
45. Schaumberg DA, Dana R, Buring JE, Sullivan DA. Prevalence of dry eye
disease among US men: estimates from the Physicians Health Studies.
Arch Ophthalmol 2009;127:763-8. http://dx.doi.org/10.1001/archoph-
thalmol.2009.103
46. Nichols JJ, Mitchell GL, Nichols KK. An assessment of self-reported
disease classication in epidemiological studies of dry eye. Invest Oph-
thalmol Vis Sci 2004;45:3453-7
47. Schiffman RM, Christianson MD, Jacobsen G, et al. Reliability and val-
idity of the Ocular Surface Disease Index. Arch Ophthalmol 2000;118:
615-21
48. Caffery B, et al. Performance of the dry eye questionnaire 5 (DEQ-5)
compared to Ocular Surface Disease Index (OSDI) in Sjogrens and
non-Sjogrens dry eye subjects. Invest Ophthalmol Vis Sci 2011;52.
ARVO E-Abstract 3850
49. Behrens A, Doyle JJ, Stern L, et al. Dysfunctional tear syndrome: a Del-
phi approach to treatment recommendations. Cornea 2006;25:900-7
50. Versura P, Bavelloni A, Grillini M, et al. Diagnostic performance of a
tear protein panel in early dry eye. Mol Vis 2013;19:1247-57
51. Srinivasan S, Thangavelu M, Zhang L, et al. iTRAQ quantitative prote-
omics in the analysis of tears in dry eye patients. Invest Ophthalmol Vis
Sci 2012;53:5052-9
52. Schiffman RM, Christianson MD, Jacobsen G, et al. Reliability and val-
idity of the Ocular Surface Disease Index. Arch Ophthalmol 2000;118:
615-21
53. Baum JL. Reliability and validity of a proposed dry eye evaluation
scheme. Arch Ophthalmol 2001;119:456
54. See CW, Bilonick RA, Feuer WJ, Galor A. Comparison of two methods
for composite score generation in dry eye syndrome. Invest Ophthalmol
Vis Sci 2013;54:6280-6. http://dx.doi.org/10.1167/iovs.13-12150
55. Tomlinson A, Hair M, McFadyen A. Statistical approaches to assessing
single and multiple outcome measures in dry eye therapy and diagnosis.
Ocul Surf 2013;11:267-84
EVALUATING BIOMARKERS OF DRY EYE / Sullivan
THE OCULAR SURFACE / JANUARY 2014, VOL. 12 NO. 1 / www.theocularsurface.com 9