Journal Pre-proof

Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification

testing in a low prevalence setting

Jordan P. Skittrall, Michael Wilson, Anna A. Smielewska, Surendra Parmar, Mary D.

Fortune, Dominic Sparkes, Martin D. Curran, Hongyi Zhang, Hamid Jalal

PII: S1198-743X(20)30614-5
DOI: https://doi.org/10.1016/j.cmi.2020.10.003
Reference: CMI 2280

To appear in: Clinical Microbiology and Infection

Received Date: 25 August 2020

Revised Date: 1 October 2020
Accepted Date: 3 October 2020

Please cite this article as: Skittrall JP, Wilson M, Smielewska AA, Parmar S, Fortune MD, Sparkes
D, Curran MD, Zhang H, Jalal H, Specificity and positive predictive value of SARS-CoV-2 nucleic
acid amplification testing in a low prevalence setting, Clinical Microbiology and Infection, https://
doi.org/10.1016/j.cmi.2020.10.003.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2020 European Society of Clinical Microbiology and Infectious Diseases. Published by Elsevier Ltd. All
rights reserved.
 First positive result

Same Cross
+

f
platform platform

roo
e-p
Confirmation

Pr
al
u rn
Jo
2 or 3 tests positive
��confirmed positive

1 test positive
��false positive Black/grey range: most pessimistic to most optimistic point estimates; pink range: all 95% confidence intervals
 1 Original article

2 Specificity and positive predictive value of

3 SARS-CoV-2 nucleic acid amplification testing
4 in a low prevalence setting
5 Jordan P. Skittrall1,2,3, Michael Wilson2,3, Anna A. Smielewska3,4, Surendra Parmar3, Mary D. Fortune5,

6 Dominic Sparkes2,3, Martin D. Curran3, Hongyi Zhang3, Hamid Jalal3

7
1
8 Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge,

of
9 United Kingdom

ro
2
10 Department of Infection, Cambridge University Hospitals NHS Foundation Trust, Cambridge, United

11 Kingdom
-p
re
3
12 Cambridge Clinical Microbiology and Public Health Laboratory, Public Health England, Cambridge,
lP

13 United Kingdom
4
14 Department of Pathology, University of Cambridge, Cambridge, United Kingdom
na

5
15 Department of Public Health and Primary Care, University of Cambridge, Cambridge, United
ur

16 Kingdom
Jo

17

18 Addresses for correspondence: Jordan P. Skittrall, Department of Applied Mathematics and

19 Theoretical Physics, Centre for Mathematical Sciences, Wilberforce Road, Cambridge, CB3 0WA,

20 United Kingdom. jps55@cam.ac.uk. Telephone: +44 1223 339896. Fax: +44 1223 765900

21 Hamid Jalal, Clinical Microbiology and Public Health Laboratory, Level 6, Box 236, Addenbrooke’s

22 Hospital, Hills Road, Cambridge, CB2 0QW, United Kingdom. Hamid.Jalal@addenbrookes.nhs.uk.

23 Telephone: +44 1223 216816. Fax: +44 1223 24277

1
24 Abstract

25 Objectives

26 When SARS-CoV-2 prevalence is low, many positive test results are false positives. Confirmatory

27 testing reduces overdiagnosis and nosocomial infection and enables real-world estimates of test

28 specificity and positive predictive value. This study estimates these parameters to evaluate the

29 impact of confirmatory testing, and to improve clinical diagnosis, epidemiological estimation and

30 interpretation of vaccine trials.

of
31 Methods

ro
32 Over one month, we took all respiratory samples from our laboratory with a patient’s first detection

33
-p
of SARS-CoV-2 RNA (Hologic Aptima SARS-CoV-2 assay or in-house RT-PCR platform), and repeated
re
34 testing using two platforms. Samples were categorised by source, and by whether clinical details
lP

35 suggested COVID-19 or corroborative testing from another laboratory. We estimated specificity and

36 positive predictive value using maximum likelihood-based approaches.

na

37 Results
ur

38 Of 19,597 samples, SARS-CoV-2 RNA was detected in 107. 52 corresponded to first-time detection
Jo

39 (0.27% of tests on samples without previous detection); further testing detected SARS-CoV-2 RNA

40 ≥1 time (“confirmed”) in 29 (56%), and failed to detect SARS-CoV-2 RNA (“not confirmed”) in 23

41 (44%). Depending upon assumed parameters, point estimates for specificity and positive predictive

42 value were 99.91%–99.98% and 61.8%–89.8% respectively using the Hologic Aptima SARS-CoV-2

43 assay, and 97.4%–99.1% and 20.1%–73.8% respectively using an in-house assay.

44 Conclusions

45 Nucleic acid amplification testing for SARS-CoV-2 is highly specific. Nevertheless, when prevalence is

46 low a significant proportion of initially positive results fail to confirm and confirmatory testing

2
47 substantially reduces false positive detections. Omitting additional testing in samples with higher

48 prior detection probabilities focuses testing where clinically impactful and minimises delay.

of
ro
-p
re
lP
na
ur
Jo

3
49 Introduction

50 The COVID-19 pandemic [1,2] continues to cause morbidity and mortality [3–6]. In some regions,

51 following seasonal changes and infection control measures, incidence and prevalence of SARS-CoV-2

52 infection has decreased [7], whilst testing capacity has increased [8–10]. As much testing capacity

53 remains operational, in such settings many being tested will be infection-free.

54

55 The commonest clinical laboratory testing employed to detect SARS-CoV-2 is nucleic acid

of
56 amplification testing (NAAT). NAAT techniques detect SARS-CoV-2 RNA using amplification steps

57 that bind specific complementary primers to nucleic acid; probes assay amplification progress. With

ro
58 well-designed primers and probes, NAAT is highly sensitive and specific. However, in practice, issues

59
-p
including inadequate clinical sampling, sample degradation, and reaction inhibition affect sensitivity,
re
60 and issues including non-specific probe breakdown, amplicon contamination and sample error affect
lP

61 specificity, even in well-run laboratories. Initial technical validations of new diagnostic assays often
na

62 only partially reflect clinical workflows. They often test limited numbers of samples, and so are

63 underpowered to detect small proportions of false positives or false negatives. In low prevalence,
ur

64 high throughput settings, false positive results will occur regularly, despite high specificity, causing
Jo

65 unnecessary community isolation and contact tracing, and nosocomial infection if inpatients with

66 false positive tests are cohorted with infectious patients. However, the low positive result incidence

67 makes confirmatory testing to reduce false positive rates feasible. This additionally enables us to

68 understand predictors that should prompt re-evaluation of results, allowing laboratories to prepare

69 for when an increased positive result incidence makes confirmatory testing on all initially positive

70 samples impracticable. Beyond these clinical benefits, this work allows improved epidemiological

71 estimation of infection rates, which will guide policy, and allows more accurate statistical analysis of

72 infection rates in study settings, e.g. vaccine trials.

73

4
74 The Cambridge Clinical Microbiology and Public Health Laboratory is a regional clinical laboratory in

75 the East of England. Since early 2020 it has undertaken SARS-CoV-2 NAAT on many inpatient and

76 community samples. On 22nd June 2020, following decreasing local positive test result incidences,

77 the laboratory began same- and cross-platform confirmatory testing for patients with positive

78 results but without previous positive tests. By combining highly specific tests in a way that

79 minimises repetition of errors, the specificity of reported results could be increased further,

80 reducing inconvenience and harm to the few uninfected people with initially positive results. We

81 report the results from testing undertaken during the first month after policy implementation.

of
82

ro
83 Methods

84 Patient sampling,
sampling, initial preparation and testing
-p
re
85 Samples were received within the routine diagnostic service. Upper airway samples were collected
lP

86 into viral transport medium following Public Health England guidelines [11]. Lower airway samples
na

87 (sputum, tracheal aspirate or broncho-alveolar lavage washings) were transported in universal

ur

88 containers and underwent mucolysis with Mucolyse Sputum Digestant (Pro-Lab Diagnostics,
Jo

89 Richmond Hill, Canada). Initial testing used one of two platforms: reverse transcription polymerase

90 chain reaction (RT-PCR) against one or two targets using an in-house assay, or transcription-

91 mediated amplification against two targets using the Aptima SARS-CoV-2 assay on the Panther

92 System (Hologic, Marlborough, United States). Assays are detailed in the web-only Supplementary

93 Methods.

94

95 Confirmation of first positive samples

96 All samples testing positive for SARS-CoV-2 were identified. Samples from patients with a preceding

97 positive SARS-CoV-2 NAAT were reported positive without confirmation, reflecting the higher prior

98 probability of true positive results, and the lower probability of significant adverse impact from false

5
 99 positives. Only preceding positive results from samples tested by our laboratory, or our local

100 hospital’s point-of-care laboratory [12], were considered here. Although data on previous tests in

101 other laboratories were sought, we conducted confirmatory testing without awaiting this

102 information.

103

104 Samples from patients without a preceding positive SARS-CoV-2 NAAT underwent repeat testing on

105 both platforms (same- and cross-platform), starting from the original sample or aliquot. Aliquots

106 were stored at −20°C for at most 24 hours, with at most one freeze-thaw cycle, before confirmatory

of
107 testing. Samples with at least two positive results were reported confirmed positive; samples with

ro
108 one of three positive results (i.e. that twice failed to confirm) were reported negative, i.e. false

109
-p
positive. Where sample was insufficient for two confirmatory tests, we aimed to prioritise cross-
re
110 platform confirmation; we reported one of two positive results as indeterminate and requested a
lP

111 repeat sample.

112
na

113 For all positive tests, we recorded whether clinical details indicated symptoms or a previous positive
ur

114 SARS-CoV-2 NAAT, using data submitted with testing requests, or electronic patient records where
Jo

115 available. We initially advised requestors of positive results awaiting confirmation, but following

116 consultation with stakeholders during the study period, switched to reporting only after

117 confirmation, as we judged risks from reporting false positive results exceeded risks from short

118 delays in reporting true positives in our population at this stage.

119

120 Effect of storage on confirmation

121 We investigated the effect of freezing/thawing on positive result repeatability: see the web-only

122 Supplementary Appendix.

123

6
124 Data analysis

125 We prospectively recorded and analysed all samples with first test results issued between 22nd June

126 and 21st July 2020. Descriptive statistics (patient sex and age, sample type) were collected and

127 analysed by testing platform and initial result. Statistical analysis was undertaken in R, version 3.2.3

128 [13]. The Supplementary Materials contain code to reproduce analyses. Binomial conditional

129 probabilities for RNA detection were calculated using the GenBinomApps package (version 1.1). The

130 odds ratio of RNA detection by initial test given a previous positive testing sample from the

131 individual versus no previous positive was calculated using the exact2x2 package (version 1.4.1).

of
132 Additional calculations used the packages stats4 (base package), numDeriv (version 2016.8.1.1).

ro
133

134
-p
For samples from patients without a previous positive result, specificity, prevalence of samples
re
135 containing virus, and positive predictive value (PPV) were estimated for both Aptima SARS-CoV-2
lP

136 and in-house PCR assays using a maximum likelihood approach, described in the Supplementary
na

137 Appendix. As these estimates depend upon test sensitivity, which is unknown but not expected to

138 impact estimates heavily in a low-prevalence setting for plausible sensitivity values [14], estimates
ur

139 were generated for a range of sensitivities (50%–100%), to ensure results were not excessively
Jo

140 sensitive to assumed sensitivity values. To obtain a full range of possible estimates, specificity,

141 prevalence and PPV were estimated under extreme cases of all indeterminate results confirming,

142 and no indeterminate result confirming, the initial positive result. There were few positive in-house

143 assay results, so these were grouped together regardless of extraction platform or number of

144 targets.

145

146 The impact of clinical information on predicting positive result confirmation was evaluated using

147 Freeman-Halton tests. The impact of quantitative assay output (relative light units (RLU) for the

148 Panther system; threshold cycle (Ct) for RNA-dependent RNA polymerase detection by the in-house

149 assay) on predicting positive result confirmation was evaluated using Mann-Whitney U tests

7
150 (conducting a sensitivity analysis for indeterminate results by removing them from analysis, and

151 grouping them with positive and with negative results).

152

153 Ethical approval

154 All testing was undertaken within routine care; analysis was undertaken within routine service

155 evaluation. Ethical approval for this study was therefore not required. Our laboratory evaluates

156 assays for validation and improves them as part of its role [15]. Testing and analysis were

157 undertaken in accordance with the UK Human Tissue Act 2004, as amended [16].

of
158

ro
159 Results
-p
re
160 Sample and test outcome characteristics
lP

161 19,597 samples had valid test results during the one-month period (Figure 1). 19,261 were initially

162 tested with the Aptima SARS-CoV-2 assay, and 336 by in-house RT-PCR (Table 1 gives patient
na

163 demographics and sample characteristics). 484 samples were from people with a previous positive
ur

164 test, of which 55 (11.3%, 95% confidence interval (CI) (8.7%,14.5%)) tested positive. 19,113 samples
Jo

165 were from people without a previous positive test, of which 52 (0.27%, 95% CI (0.20%,0.36%)) had

166 RNA detected in the first test. The odds ratio of RNA detection by initial test given a previous

167 positive testing sample from the individual versus no previous positive was 46.9 (95% CI (31.4,70.5),

168 p<0.001). There was substantial heterogeneity in specimen type and patient sex and age between

169 samples tested on different platforms and between samples with different testing outcomes

170 (Table 1). Notably, first positive initial tests from the Aptima SARS-CoV-2 assay predominantly came

171 from upper airway samples, whereas first positive initial tests from the in-house RT-PCR

172 predominantly came from lower airway samples.

173

8
174 Effect of freeze-
freeze-thaw cycle

175 Clinical sample storage did not significantly affect positive results: see the Supplementary Appendix.

176

177 Specificity, prevalence and positive predictive value

178 Depending upon the assumed sensitivity and status of indeterminate results, testing with the Aptima

179 SARS-CoV-2 assay was found to have a specificity of 99.91%–99.98%, with positive sample

180 prevalence 0.14%–0.41% and PPV 61.8%–89.8% (unions of 95% CIs (99.85%,100.00%), (0.10%,0.59%)

181 and (46.3%,98.6%), respectively). Testing with the in-house RT-PCR had a specificity of 97.4%–

of
182 99.1%, with positive sample prevalence 0.62%–4.6% and PPV 20.1%–73.8% (unions of 95% CIs

ro
183 (94.6%,99.9%), (0.13%,10.7%) and (4.0%,96.5%), respectively). Figure 2 and web-only

184
-p
Supplementary Figure S1 summarize these results. Web-only Supplementary Tables S1–S4 contain
re
185 output data.
lP

186
na

187 Demographic and analytical

analytical characteristics
characteristics of confirming versus non-
non-confirming results
ur

188 Few specimens were submitted with sufficient clinical information to determine whether a patient

189 was symptomatic or previously had a positive test from another laboratory. Of the 43 specimens
Jo

190 with initial first positive results from the Hologic Aptima SARS-CoV-2 assay, 4 had information

191 indicating symptoms or a previous positive test (2 both, 1 symptoms only, 1 previous positive only),

192 of which one specimen with both symptoms and a previous positive test was ultimately reported

193 indeterminate, and the remaining three positive (Freeman-Halton p=0.071). Of the 9 specimens

194 with initial first positive results from the in-house assay, 5 had information indicating symptoms (and

195 none a previous positive test), of which two were ultimately reported positive, two indeterminate,

196 and one negative (Freeman-Halton p=0.38). 8 of these 9 specimens, including all 5 with information

197 of symptoms, were lower airway specimens.

198

9
199 Figure 3 shows RLU output of the Panther analysis of Aptima SARS-CoV-2 assay results initially

200 determined to be first positives, and web-only Supplementary Figure S2 shows output by total

201 number of positive assays. There is a statistically significant difference between RLU output of initial

202 positive results ultimately reported negative versus those ultimately reported positive (p<0.001 in all

203 cases, difference in RLU and 95% CIs 389 (271,456), 397 (311,461) and 378 (199,444) as

204 indeterminate results are removed, considered negative, and considered positive, respectively).

205 However, there is overlap in RLU outputs for samples ultimately reported positive and samples

206 ultimately reported negative.

of
207

ro
208 Figure 4 shows Ct output of in-house assay results initially determined to be first positives, and web-

209
-p
only Supplementary Figure S3 shows output by total number of positive assays. There is no
re
210 statistically significant difference between Ct values of initial positive results ultimately reported
lP

211 negative versus those ultimately reported positive (p=0.53, 0.38 and 1.00 as indeterminate results

212 are removed, considered negative, and considered positive, respectively).

na

213
ur

214 Discussion
Jo

215 Our results show that, regardless of testing sensitivity, both platforms have high specificity, and

216 single positive results from either are reliable for diagnosis in high-risk populations. However,

217 despite a small absolute difference in false positive rates, the relative difference between testing

218 platforms is marked. This requires consideration when screening low-risk populations, to avoid

219 adverse effects to individuals from overdiagnosis and to avoid heterogeneity in estimates of

220 infection rates attributable to different testing platforms. We have shown that when prevalence is

221 low, a clinically significant proportion of first positive results are unrepeatable on the same testing

222 platform, meaning confirmatory testing has utility for smaller laboratories using single platforms.

223 Conducting confirmatory tests on all platforms, including the first used, ensures the most sensitive

224 platform is used for confirmation, even with unknown sensitivities. We have shown that clinically

10
225 significant positive results are repeatable, so confirmatory testing helps discriminate true from false

226 positive results.

227

228 We have further demonstrated that, in a large clinical laboratory, appropriately stratifying patients

229 by prior test probability is key to achieving accurate and timely results whilst minimising workload.

230 The most important subpopulation to stratify is those with prior evidence of SARS-CoV-2 shedding,

231 where both the risk and the impact of a false positive test are much lower. Because the biochemical

232 properties of assaying SARS-CoV-2 in samples from those with late acute immune responses might

of
233 differ from assaying in other samples, it is especially important to analyse this population separately

ro
234 when inferring prevalences and test properties.

235
-p
re
236 In clinical laboratory settings, much focus is on minimising and detecting false positive results from
lP

237 contamination or nonspecific probe breakdown, as opposed to other causes. The quantitative

238 difference between confirming and non-confirming results for one of our assays supports the idea
na

239 that many false positive results could be attributed to nonspecific reactions, or low-level sample
ur

240 contamination. (Another explanation is a low-level true positive, but non-repeatable, sample, which
Jo

241 would seldom be clinically significant.) When confirming all initial first positive results is impossible,

242 this observation allows stratification to decide which tests to repeat. However, the substantial

243 overlap in quantitative results between those confirming and those not confirming highlights the

244 presence of other causes of false positive results. Restricting confirmatory testing to an

245 intermediate category will incorrectly categorise samples giving apparently strong positive, but non-

246 repeatable, results. Considering sources of false positive results also yields this study’s largest

247 limitation: errors before the confirmatory testing process branches from the initial process will not

248 be detected, and confirmatory testing may falsely reassure.

249

11
250 When analytical test sensitivity is low, failure to confirm may represent a false negative confirmatory

251 test. However, our data on repeatability, coupled with validation literature indicating SARS-CoV-2

252 tests have high analytical sensitivity [17,18], suggest lower reported overall clinical sensitivity

253 predominantly arises during sampling: confirmation tests of an initially positive sample would not

254 reflect such insensitivity. Any clinical sensitivity issues must be considered when making prevalence

255 estimates in a population of people, rather than the population of samples they have produced.

256

257 We observed substantial heterogeneity between samples following different testing pathways

of
258 through our laboratory. We highlight the different extraction methods used for our in-house assay

ro
259 and the additional primer/probe set in some tests, meaning the reported assay specificity represents

260
-p
a composite: for real-world clinical and epidemiological applications, this remains useful. Another
re
261 issue is overrepresentation of more complex, lower respiratory, samples in the in-house assay
lP

262 positive results. Processing these samples involves additional steps, yielding a potential confounder

263 when comparing our assays’ performances. Nonetheless, it is perhaps unsurprising the commercial
na

264 assay, with fewer manual steps, shows a lower false positive rate. Multiple possible factors may
ur

265 explain other aspects of observed heterogeneity, e.g. whether there are differences in age and sex
Jo

266 representation in screened residential facilities, in symptomatic individuals, or in those shedding

267 viral RNA over a prolonged period.

268

269 Prevalence and PPV estimates reported refer to the populations tested, rather than general

270 populations from which they are drawn. These estimates are appropriate when considering clinical

271 interpretation of assay results, but differences between these populations and the underlying

272 general populations must be accounted for when making epidemiological estimates in underlying

273 populations.

274

12
275 This study gives quantitative estimates of SARS-CoV-2 NAAT specificity, useful to guide clinical

276 interpretation of test results, in planning clinical services, and for power calculations for vaccine and

277 therapeutic trials. The methods and code can calculate testing parameters in other laboratories.

278 We have shown the utility and limitations of confirmatory testing, reminding us to focus on

279 managing patients and not treat test results in isolation.

280

281 Acknowledgements

of
282 We are extremely grateful to the staff of the Cambridge Clinical Microbiology and Public Health

283 Laboratory, and particularly to the laboratory managers, senior virology biomedical scientists, staff

ro
284 involved in SARS-CoV-2 testing, volunteers from other Public Health England (PHE) sites, and data

285
-p
analysts, whose tireless work has been crucial in serving our patient population. We are particularly
re
286 grateful to our clinical colleagues Mir Mubariz Hussain, Chloe Myers, and Kathryn Rolfe, for their
lP

287 involvement in clinical case management and discussion as routine procedures were developed. The
na

288 discussions of diagnostic assays and extraction platforms in this report should not be taken as a

289 recommendation by PHE or any of the other institutions involved.

ur

290
Jo

291 Transparency
Transparency declaration

292 No author has any conflict of interests to disclose. J.P.S. is funded by a fellowship from the Mason

293 Medical Research Foundation. No other author has received external funding for this study.

294

295 Author contributions

296 J.P.S: Conceptualisation, methodology, software, formal analysis, investigation, data curation,

297 writing – original draft, visualisation, project administration. M.W: Investigation, writing – review &

298 editing. A.A.S: Investigation, writing – review & editing. S.P: Methodology, validation, resources,

299 writing – review & editing. M.D.F: Conceptualisation, methodology, formal analysis, writing – review

13
300 & editing. D.S: Investigation, writing – review & editing. M.D.C: Methodology, resources, writing –

301 review & editing. H.Z: Conceptualisation, writing – review & editing. H.J: Conceptualisation,

302 methodology, writing – review & editing, supervision.

303

304 References

305 1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with

306 pneumonia in China, 2019. N Engl J Med 2020;382:727–33.

of
307 2. Wu F, Zhao S, Yu B, Chen YM, Wang W, Song ZG, et al. A new coronavirus associated with

308 human respiratory disease in China. Nature 2020;579:265–9.

ro
309 3. Sinnathamby MA, Whitaker H, Coughlan L, Lopez Bernal J, Ramsay M, Andrews N. All-cause

310
-p
excess mortality observed by age group and regions in the first wave of the COVID-19
re
311 pandemic in England. Eurosurveillance 2020;25:pii=2001239.
lP

312 4. Michelozzi P, De’Donato F, Scortichini M, de Sario M, Noccioli F, Rossi P, et al. Mortality

na

313 impacts of the coronavirus disease (COVID-19) outbreak by sex and age: rapid mortality

314 surveillance system, Italy, 1 February to 18 April 2020. Eurosurveillance 2020;25:pii=2000620.

ur

315 5. Nogueirai PJ, De Araújo Nobre M, Nicola PJ, Furtado C, Vaz Carneiro A. Excess mortality
Jo

316 estimation during the COVID-19 pandemic: preliminary data from Portugal. Acta Med Port

317 2020;33:376–83.

318 6. Menachemi N, Yiannoutsos CT, Dixon BE, Duszynski TJ, Fadel WF, Wools-Kaloustian KK, et al.

319 Population Point Prevalence of SARS-CoV-2 Infection Based on a Statewide Random Sample-

320 Indiana, April 25-29, 2020. Morb Mortal Wkly Rep 2020;69:960–4.

321 7. Pouwels KB, House T, Robotham J V, Birrell PJ, Gelman A, Bowers N, et al. Community

322 prevalence of SARS-CoV-2 in England: results from the ONS coronavirus infection survey pilot.

323 medRxiv 2020;10.1101/2020.07.06.20147348.

324 8. Reusken CBEM, Broberg EK, Haagmans B, Meijer A, Corman VM, Papa A, et al. Laboratory

325 readiness and response for novel coronavirus (2019-nCoV) in expert laboratories in 30

14
326 EU/EEA countries, January 2020. Eurosurveillance 2020;25:pii=2000082.

327 9. Matheeussen V, Loens K, Lammens C, Vilken T, Koopmans M, Goossens H, et al. Preparedness

328 of European diagnostic microbiology labs for detection of SARS-CoV-2, March 2020. J Clin

329 Virol 2020;128:104432.

330 10. Matheeussen V, Corman VM, Donoso Mantke O, McCulloch E, Lammens C, Goossens H, et al.

331 International external quality assessment for SARS-CoV-2 molecular detection and survey on

332 clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020.

333 Eurosurveillance 2020;25:pii=2001223.

of
334 11. Public Health England. COVID-19: laboratory investigations and sample requirements for

ro
335 diagnosis [Internet]. Available from: https://www.gov.uk/government/publications/wuhan-

336
-p
novel-coronavirus-guidance-for-clinical-diagnostic-laboratories/laboratory-investigations-
re
337 and-sample-requirements-for-diagnosing-and-monitoring-wn-cov-infection
lP

338 12. Collier D, Assennato S, Sithole N, Sharrocks K, Ritchie A, Ravji P, et al. Rapid point of care

339 nucleic acid testing for SARS-CoV-2 in hospitalised patients: a clinical trial and
na

340 implementation study. Cell Reports Med 2020;100062.

ur

341 13. R Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna,
Jo

342 Austria; 2015. Available from: https://www.r-project.org/

343 14. Woloshin S, Patel N, Kesselheim AS. False negative tests for SARS-CoV-2 infection —

344 challenges and implications. N Engl J Med 2020;383:e38.

345 15. Public Health England. UK Standards for Microbiology Investigations: Q1. Evaluation,

346 validation and verifications of diagnostic tests. UK Standards for Microbiology Investigations

347 2017. Report No Q1.

348 16. The National Archives. Human Tissue Act 2004.

349 17. Gorzalski AJ, Tian H, Laverdure C, Morzunov S, Verma SC, VanHooser S, et al. High-

350 Throughput Transcription-mediated amplification on the Hologic Panther is a highly sensitive

351 method of detection for SARS-CoV-2. J Clin Virol 2020;129:104501.

15
352 18. Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DKW, et al. Detection of 2019

353 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance 2020;25:pii=2000045.

of
ro
-p
re
lP
na
ur
Jo

16
 All Hologic Aptima SARS-CoV-2 TMA In-house PCR

samples (1) All People with previous positive People without previous (1) All People with previous positive People without previous

samples test positive test samples test positive test

(2) All (3) (3) Initial (2) All (4) (4) Initial (5) All (6) (6) Initial (5) All (7) (7) Initial

Negative positive Negative positive Negative positive Negative positive

f
test test test test test test test test

o
ro
No. of 19,597 19,261 439 392 47 18,822 18,779 43 336 45 37 8 291 282 9

-p
samples

re
Female 11,992 11,808 249 231 18 11,559 11,523 36 184 12 12 0 172 167 5
1 2 3 3 2 4 4 1 5 5

lP
sex – no. (61.2) (61.3) (56.7) (58.9) (38.3) (61.4) (61.3) (83.7) (54.8) (26.7) (32.4) (0) (59.1) (59.2) (55.6)

(%)

na
Median 58 58 74 74 74 58 58 71 64 69 63 74 63 63 61

age – yr (38–77) (38–77)

6
(57– (56–87) (65–83)
ur (38– (38–77)
8
(56-88)
8
(47–77)
6
(52– (49–78)
9
(74–74)
9
(45– (46–77) (37–77)
Jo
7 7
(IQR) 86) 77) 75) 77)

URT 19,359 19,148 431 384 47 18,717 18,677 40 211 33 25 8 178 177 1
10 11 11 12 12 10 13 13
specimens (98.8) (99.4) (98.2) (97.8) (100) (99.4) (99.5) (93.0) (62.8) (73.3) (67.6) (100) (61.2) (62.8) (11.1)

– no. (%)

354 Table 1: Characteristics of patients and sample types

355 Ages are rounded to the nearest year. TMA=transcription-mediated amplification. PCR=polymerase chain reaction. URT=upper respiratory tract.

356 Heterogeneity comparisons are made within rows between the columns whose headers are marked with identical numbers in parentheses. Proportion

17
357 female and proportion URT specimen comparisons are made using Fisher’s exact test. Age comparisons are made using the Mann-Whitney U test. Values

358 yielding p-values less than 0.05 are marked and reported, together with odds ratios (OR) reported to two significant figures or medians of difference

359 between samples reported to one decimal place, as appropriate, followed by 95% confidence intervals. Given 21 separate independent tests, a p-value less

360 than 0.05 would be expected to occur by chance on average 1.05 times. Note that the table’s construction means that the tests are not independent of

361 each other.

o f
ro
1
362 p=0.015, OR=1.3 (1.1,1.6); 2p=0.047, OR=0.82 (0.68,1.00); 3p=0.0081, OR=2.3 (1.2,4.5); 4p=0.0025, OR=0.31 (0.13,0.68); 5p<0.001, OR=0.25 (0.12,0.52);

-p
6
363 p=0.0043, median of difference=-3.7 yr (-6.3,-1.2); 7p<0.001, median of difference=12.6 yr (10.3,14.9); 8p=0.0023, median of difference=-11.6 yr (-19.5,-

re
364 4.2); 9p=0.045, median of difference=-12.3 yr (-24.9,0.0); 10p<0.001, OR=100 (75,130); 11p=0.004, OR=0.30 (0.15,0.68); 12p=0.0018, OR=14 (3.5,43);

lP
13
365 p=0.0027, OR=13 (2.0, 300).

na
ur
Jo

18
 All samples
(19597)

Initial test Hologic Initial test in-house

Aptima SARS-CoV-2 RT-PCR
(19261) (336)

RNA not detected on initial RNA detected on initial RNA not detected on initial RNA detected on initial
assay - report negative assay assay - report negative assay
(19171) (90) (319) (17)

Previous detection; Included in Previous detection; Included in Previous detection; Included in Previous detection; Included in
excluded from main main excluded from main main excluded from main main excluded from main main
analysis analysis analysis analysis analysis analysis analysis analysis
(392) (18779) (47) (43) (37) (282) (8) (9)

Sufficient sample for one Sufficient sample for two Sufficient sample for one Sufficient sample for two
confirmatory test confirmatory tests confirmatory test confirmatory tests
(20) (23) (4) (5)

RNA detected on 1/2 or RNA detected on 0/2 RNA detected on 1/2 or RNA detected on 0/2
2/2 confirmatory tests - confirmatory tests - 2/2 confirmatory tests - confirmatory tests -
report positive report negative report positive report negative
(9) (14) (1) (4)

Confirmatory test using Confirmatory test using Confirmatory test using Confirmatory test using
Hologic Aptima SARS-CoV-2 in-house RT-PCR Hologic Aptima SARS-CoV-2 in-house RT-PCR
(2) (18) (2) (2)

RNA detected on 1/1 RNA detected on 0/1 RNA detected on 1/1 RNA detected on 0/1 RNA detected on 1/1 RNA detected on 0/1 RNA detected on 1/1 RNA detected on 0/1

of
confirmatory test - confirmatory test - confirmatory test - confirmatory test - confirmatory test - confirmatory test - confirmatory test - confirmatory test -
report positive report indeterminate report positive report indeterminate report positive report indeterminate report positive report indeterminate
(2) (0) (16) (2) (0) (2) (1) (1)

ro
-p
re
lP
na
ur
Jo
Jo
ur
na
lP
re
-p
ro
of
Jo
ur
na
lP
re
-p
ro
of
Jo
ur
na
lP
re
-p
ro
of