ORIGINAL STUDY
Confirmation of High Specificity of an Automated
Enzyme Immunoassay Test for Serological Diagnosis
of Syphilis: Retrospective Evaluation Versus Results
After Implementation
Laura van Dommelen, MD, PhD,* Christian J.P.A. Hoebe, MD, PhD,*† Frank H. van Tiel, MD, PhD,*
Carel Thijs, MD, PhD,‡ Valère J. Goossens, MD, PhD,* Cathrien A. Bruggeman, PhD,*
and Inge H.M. van Loo, MD, PhD*
Background: The optimal algorithm for serological syphilis screening
is still a matter of debate. We have previously evaluated the perfor-
mance of the Bioelisa Syphilis 3.0, using a selection of archived sera,
and in this study compare these results with the Bioelisa results after
clinical implementation.
Methods: All Bioelisa Syphilis 3.0 results obtained since clinical imple-
mentation were analyzed. Bioelisa-positive or borderline samples were
retested using Treponema pallidum particle agglutination, rapid plasma
reagin test, fluorescent treponemal antibody-absorption test, and/or im-
munoblot. On sera sent in together with cerebrospinal fluid, occasionally
both the T. pallidum particle agglutination and Bioelisa were performed.
Results: The Bioelisa was performed on 14,622 sera. Bioelisa-positive
samples, which were not retested by the previously described assays,
were withdrawn from the database (n = 36). In 1.3% of the samples
(187/14,586), the Bioelisa was positive or borderline and, ultimately,
115 sera were considered true positive (prevalence 0.8%). The specificity
of the Bioelisa was 99.5%.
Conclusions: Based on the results of all performed diagnostic assays,
the specificity of the Bioelisa of 99.5% is very consistent with that found
in the initial study (100%; 95% confidence interval was 98.0%–100%).
Interpreting (positive) test results is difficult in the absence of a gold stan-
dard, especially when the disease prevalence is low. Results should be
viewed in the light of the patients' characteristics.
I n the absence of a gold standard, the optimal serological syphilis
screening algorithm is still a matter of debate. Generally, there
are 2 screening methods: (1) using a treponemal assay (e.g.,
enzyme immunoassay [EIA]), which is called the “reverse algo-
rithm” in the literature, or (2) a nontreponemal assay (rapid
plasma reagin test [RPR] or Venereal Disease Research Labora-
tory). Also, there is variation in the confirmatory assays used.
From the *Department of Medical Microbiology, CAPHRI School of Pub-
lic Health and Primary Care, Maastricht University Medical Centre,
AZ, Maastricht, the Netherlands; †Department of Sexual Health, Infec-
tious Diseases and Environmental Health, Public Health Service South
Limburg, HA, Geleen, the Netherlands; and ‡Department of Epidemi-
ology, CAPHRI School of Public Health and Primary Care, Maastricht
University Medical Centre, AZ, Maastricht, the Netherlands
Conflict of interest: None declared.
Correspondence: Laura van Dommelen, MD, PhD, Department of Medical
Microbiology, CAPHRI School of Public Health and Primary Care,
Maastricht University Medical Centre, PO Box 5800, 6202 AZ, Maastricht,
the Netherlands. E-mail: lauravandommelen@yahoo.com.
Financial support: None declared.
This study is approved by the Medical Ethics Committee of the Maastricht
University Medical Centre (no. 12-4-118)
Received for publication July 30, 2014, and accepted December 12, 2014.
DOI: 10.1097/OLQ.0000000000000240
Copyright © 2015 American Sexually Transmitted Diseases Association
All rights reserved.
120
Copyright © 2015 by the American Sexually Transmitted Diseases Association. Unauthorized reproduction of this article is prohibited.
We have previously evaluated the performance of the
Bioelisa Syphilis 3.0 EIA as a primary screening assay, in com-
parison with the performance of the Treponema pallidum particle
agglutination (TPPA) in a selected collection of serum samples.1
This EIA detects IgM and IgG antibodies against T. pallidum
antibodies. The sample collection included 145 sera from syphilis
patients with active or latent disease, 41 sera from healthy con-
trols, and 144 sera from patients with underlying conditions which
may influence T. pallidum antibody testing. The sensitivity and
specificity were both 100% (95% confidence interval [95% CI]
sensitivity 97.5%–100%, specificity 98.0%–100%). Because the
Bioelisa Syphilis 3.0 can be used on an automated EIA reading
system, this EIA was implemented in our laboratory to replace
the TPPA as a screening method.
Because this high specificity was based on the results of
tests performed on a selected sample collection, with an artifi-
cially high syphilis prevalence (44%), it was not surprising that
we noticed a relative high number of false-positive results in the
first year after implementation of the Bioelisa as a screening-assay
for syphilis. A false-positive Bioelisa was defined as a positive re-
sult in the Bioelisa with all confirmatory testing with TPPA, immuno-
fluorescense, rapid plasma regain, and/or additional testing with an
immunoblot at a reference laboratory, being negative. Therefore,
we decided to perform a retrospective analysis of the Bioelisa Syphilis
3.0, to assess whether the high specificity found in the initial evalua-
tion would stand up in clinical practice.
MATERIALS AND METHODS
The Bioelisa Syphilis 3.0 (Biokit SA, Barcelona, Spain)
was introduced in our laboratory on October 1, 2007. All Bioelisa
results (n = 14,622) obtained between October 2007 and February
2010 were analyzed. The Bioelisa detects IgM and IgG antibodies.
If the Bioelisa was positive, the TPPA (MHA-TP; Fujirebio,
Tokyo, Japan), RPR (Syfacard-R*; Abbott Murex, Dartford,
UK), and fluorescent treponemal antibody-absorption test (FTA-
Abs; Trepo Spot IF, BioMerieux SA, Marcy l'Etoile, France) were
additionally performed on this sample for confirmation. The TPPA
and FTA-Abs both detect IgM and IgG antibodies against T.
pallidum. The RPR uses tissue lipid cardiolipin (antigen) to detect
“reagin,” that is, antibodies directed against tissue components,
which appear in blood of a patient with syphilis as a reaction to
tissue damage. If the previously mentioned serological tests yielded
an inconclusive serological profile (e.g., a discrepancy between the
Bioelisa and TPPA and/or FTA Abs results), the serum was sent
to the national reference laboratory (National Institute of Public
Health and the Environment, Bilthoven, the Netherlands) for addi-
tional testing with an immunoblot. The TPPA was occasionally
performed on Bioelisa-negative sera, which were sent to our labo-
ratory together with cerebrospinal fluid, to exclude neurosyphilis. All
Sexually Transmitted Diseases • Volume 42, Number 3, March 2015
 Automated EIA Test for Diagnosis of Syphilis

TABLE 1. Bioelisa Results After Clinical Implementation2,3

Final Conclusion
Positive Negative Total 95% CI
Total Bioelisa Positive 115 72 187 Se 100% 96.8%–100%
Negative 0 14399 14399 Sp 99.5% 94.6%–99.6%
Total 115 14471 14586 PPV 61.5% 54.1%–68.5%
NPV 100% 100%–100%
Syphilis prevalence 0.79%

No clinical data Bioelisa Positive 67 26 93 Se 100% 94.6%–100%

Negative 0 4684 4684 Sp 99.5% 99.2%–99.6%
Total 67 4710 4777 PPV 72.0% 61.8%–80.9%
NPV 100% 100%–100%
Syphilis prevalence 1.40%

STD screening Bioelisa Positive 38 24 62 Se 61.3% 90.1%–73.4%

Negative 0 4468 4468 Sp 99.5% 99.4%–99.7%
Total 38 4492 4530 PPV 61.3% 48.1%–73.4%
NPV 100% 99.9%–100%
Syphilis prevalence 0.55%

Pregnancy Bioelisa Positive 3 11 14 Se 100% 29.2%–50.8%

Negative 0 3154 3154 Sp 99.7% 99.4%–99.8%
Total 3 3165 3168 PPV 21.4% 4.6%–50.8%
NPV 100% 99.9%–100%
Syphilis prevalence 0.09%

Se indicates sensitivity; Sp, specificity; NPV, negative predictive value; STD, sexual transmitted diseases; 95% CI, exact binomial 95% CI, confidence
interval; PPV, positive predictive value.

diagnostic assays were performed according to the manufacturers'
instructions, as previously described.1 Data were unlinked from pos-
sible patient identifiers, and each patient was included only once in
the database. Results were analyzed using the 2-way contingency
table analysis (ctab2x2) and exact binominal CIs calculation
(confint) programs from www.statpages.org.2,3 A test result
was considered as true positive if the Bioelisa was positive and
was confirmed by a positive TPPA and FTA-Abs, or by a positive
immunoblot result. Exceptions are displayed in Table 2. All
Bioelisa-negative samples were considered true negative results,
based on the results of our previous study.
RESULTS
All Bioelisa-positive samples that were not tested according
to the algorithm described in the previous section were withdrawn
from the database (n = 36). In 1.3% of samples (187/14,586), the
Bioelisa was positive or borderline. Because only 6 of these 187
samples were tested borderline and confirmatory testing was similar
to the Bioelisa-positive samples, these samples were considered
Bioelisa positive in further analysis. Sixty-two Bioelisa-positive sam-
ples (33%) showed discrepant results in the confirmatory assays and
were sent to the national reference laboratory. Immunoblot testing re-
sulted in an additional 6 true positive samples. All Bioelisa border-
line samples were TPPA negative and considered as true negatives.
Because TPPA is routinely performed on serum and cerebrospinal
fluid to exclude neurosyphilis, confirmatory assays, including TPPA,
were also performed on 41 such serum samples that had been
tested Bioelisa negative. One of these samples showed reaction
in the TPPA (1:80) and FTA-Abs, whereas RPR result was nega-
tive and reference laboratory conclusion stated “borderline” in
the immunoblot. In total, 115 (0.8%) of 14,586 samples were con-
sidered true positives (Tables 1 and 2).

TABLE 2. Results of Confirmatory Tests in Bioelisa-positive samples*

TPPA RPR FTA Reference Laboratory Final Conclusion Positive (n) Final Conclusion Negative (n) Total (n)
Positive Positive Positive NA 56 0 56
Positive Negative Positive NA 51 0 51
Positive Negative Positive Positive 4 0 4
Positive Negative Positive Negative 0 1 1
Negative Positive Negative Negative 0 1 1
Negative Negative Positive Positive 2 1† 3
Negative Negative Positive Negative 0 18 18
Negative Negative Positive NA 2 1 3
Negative Negative Positive Negative 0 5 5
Negative Negative Negative NA 0 15 15
Negative Negative Negative Positive 0 3† 3
Negative Negative Negative Negative 0 27 27
115 72 187

*Postive and bordeline positive samples are grouped in this table to preserve readability.
†
Result from reference laboratory bordeline positive.
NA indicates not applicable.

Sexually Transmitted Diseases • Volume 42, Number 3, March 2015 121

Copyright © 2015 by the American Sexually Transmitted Diseases Association. Unauthorized reproduction of this article is prohibited.
van Dommelen et al.
The main reasons for syphilis screening were sexually trans-
mitted disease screening (n = 4530; 31% of all samples), screening
during pregnancy (n = 3168; 22%), and workup for infertility
(n = 1167; 8%). In another 6% (n = 944) of the cases, syphilis
screening was requested for other reasons (e.g., suspicion of
neurosyphilis), leaving 33% (n = 4777) of the patients for whom
no clinical data were available. Considering all true positive samples,
clinical data were unfortunately only available for 1 patient: an HIV-
positive man with lymphadenopathy, malaise, and weight loss (EIA
positive, TPPA negative, RPR negative, FTA-Abs strongly positive).
Based on the results of all performed diagnostic assays, the
specificity of the Bioelisa was 99.5% (95% CI, 99.4%–99.6%) and
the positive predictive value (PPV) was 62% (95% CI, 54.1%–
68.5%; Table 1). The overall specificity of 99.5% is very similar
to that found in the initial evaluation (100% with the 95% CI being
98.0%–100% in the initial study), with an even narrower CI of spec-
ificity in this study (95% CI, 99.4%–99.6%). When analyzing the
results of the largest patient categories according to available clinical
data, the values of specificity were very similar (Table 1).
DISCUSSION
Performing a prospective evaluation is time consuming and
costly. It took us several years to gather 115 serologically confirmed
syphilis cases. In our previous evaluation study, we have therefore
used a selection of archived sera. In the present study, we have com-
pared the results of the previous study with the performance after
implementation of this assay in our laboratory. In the initial evaluation,
all syphilis serologically positive samples comprehended 44% of the
total number of sera. The prevalence of syphilis in the Netherlands,
however, is only 0.2% among pregnant women and 2.3% in men hav-
ing sex with men.4 Our current analysis shows that the high specificity
found in the initial study stands up after implementation in a popula-
tion with a low syphilis prevalence (0.8%). The specificity was sim-
ilar in all subgroups of patients, and hence, differences in PPV are
solely due to differences in the prevalence of syphilis, as can be seen
in Table 1, with PPVs ranging from 21.4% (in the low-risk group of
pregnancy screening) to 72.0% (in the group without clinical data).
Even with a very high specificity, the high number of nondiseased
can result in a substantial number of false-positive results. Under-
standing this phenomenon is important when interpreting a test re-
sult.5 Confirmatory assays are therefore essential in the serological
diagnosis of syphilis, especially in low-prevalence patient groups.
A comparison between both our evaluations is hampered
by the fact that, in our initial study, the TPPA was performed on
all samples, whereas in our current analysis—performed after
implementation—the TPPA was almost strictly limited to Bioelisa-
positive samples. This causes a verification bias compared with
the initial study: not all samples were subjected to the reference
test.6 False-negative Bioelisa results could have been missed.
As was seen in our evaluation, one Bioelisa-negative sample was
reactive in the TPPA and FTA-Abs and borderline reactive in the
immunoblot. In our previous evaluation, however, the Bioelisa
did not miss any positive sample. The first version of the Bioelisa
Syphilis was evaluated by Ebel et al.7 in 1998 using 824 sera.
The T. pallidum hemagglutination assay and FTA-Abs were both
positive in 53% of these samples. The sensitivity and specificity
found in their study were 99.5% and 99.4%, respectively. The
current version of the assay (3.0) was recently evaluated by
Donkers et al.8 in 1251 samples (syphilis prevalence 8.0%), and
they found an agreement of 100% with the other 2 EIAs used
in their study. Castro et al.9 analyzed the analytical level of agree-
ment between several EIAs in comparison with the TPPA and
found that the Bioelisa (version not stated) was relatively sen-
sitive compared with the other EIAs used. Taking together all the
122 Sexually Transmitted Diseases
Copyright © 2015 by the American Sexually Transmitted Diseases Association. Unauthorized reproduction of this article is prohibited.
above-mentioned results, it is unlikely that a significant number
of patients were missed using the Bioelisa. An overview of the
performance of several other syphilis EIAs can be found in our pre-
vious article.1 In general, EIAs are considered equally sensitive or
more sensitive for the detection of T. pallidum antibodies compared
with TPPA or T. pallidum hemaglutinine agglutination.9–13
Another interesting point in the study by Castro et al.,9
mentioned previously, is that they question the use of the FTA-Abs
as a confirmatory assay based on their results because it was the least
sensitive. Also, the TPPA was found less sensitive compared with the
Bioelisa in their study. Several underlying conditions, such as preg-
nancy or autoimmune disease, can interfere with all syphilis assays,
resulting in false-positive results.1 In case of discrepancy between as-
says and/or weakly positive results, it is therefore difficult to assess if
it concerns false-positive results or (weak) true positive results. Un-
fortunately, the sensitivity of syphilis nucleic acid amplification tests
in the absence of skin lesions is poor.13–15 As shown in Table 2, even
with the same test results, it is not always easy to come to a final con-
clusion. In the absence of a gold standard, results should always be
viewed in light of the patients' medical history, symptoms, and sex-
ual history before treatment decisions are made.
REFERENCES
1. van Dommelen L, Smismans A, Goossens VJ, et al. Evaluation of a
rapid one-step immunochromatographic test and two
immunoenzymatic assays for the detection of anti–Treponema
pallidum antibodies. Sex Transm Infect 2008;84:292–296.
2. Clopper CS, Pearson ES. The use of confidence or fiducial limits
illustrated in the case of the binomial. Biometrika 1934;26:404–413
(http://statpages.org). Accessed April 13, 2014.
3. Rosner B. Fundamentals of Biostatistics, 2006. Available at:
http://statpages.org). Accessed April 13, 2014.
4. Vriend HJ, Koedijk FDH, van den Broek IVF, et al. Sexually
transmitted infections, including HIV, in the Netherlands in 2010.
Bilthoven, the Netherlands: National Institute for Public Health and
the Environment, 2011.
5. Billings PR, Bernstein MS. Physicians poor at prevalence and positive
predictive value. JAMA 1985;254:1173–1174.
6. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of
design-related bias in studies of diagnostic tests. JAMA 1999;282:
1061–1066.
7. Ebel A, Bachelart L, Alonso JM. Evaluation of a new competitive
immunoassay (BioElisa Syphilis) for screening for Treponema pallidum
antibodies at various stages of syphilis. J Clin Microbiol 1998;36:358–361.
8. Donkers A, Levy HR, Letens-van Vliet A. Syphilis detection using
the Siemens ADVIA Centaur Syphilistreponemal assay.
Clin Chim Acta 2014;433C:84–87.
9. Castro A, Jost H, Cox D, et al. A comparison of the analytical level
of agreement of nine treponemal assays for syphilis and possible
implications for screening algorithms. BMJ Open 2013;3:e003347.
10. Binnicker MJ, Jespersen JD, Rollins LO. Treponema-specific tests
for serodiagnosis of syphilis: comparative evaluation of seven assays.
J Clin Microbiol 2011;49:1313–1317.
11. Maple PA, Ratcliffe D, Smit E. Characterization of Treponema
pallidum particle agglutination assay–negative sera following
screening by treponemal total antibody enzyme immunoassays.
Clin Vaccine Immunol 2010;17:1718–1722.
12. Sena AC, White BL, Sparling PF. Novel Treponema pallidum
serologic tests: A paradigm shift in syphilis screening for the 21st
century. Clin Infect Dis 2010;51:700–708.
13. Vulcano F, Milazzo L, Volpi S, et al. Italian national survey of blood
donors: external quality assessment (EQA) of syphilis testing.
J Clin Microbiol 2010;48:753–757.
14. Grange PA, Gressier L, Dion PL, et al. Evaluation of a PCR test for
detection of treponema pallidum in swabs and blood. J Clin Microbiol
2012;50:546–552.
15. Heymans RJJ, van der Helm JJ, de Vries HJ, et al. Clinical value of
Treponema pallidum real-time PCR for diagnosis of syphilis.
J Clin Microbiol 2010;48:497–502.
• Volume 42, Number 3, March 2015